fix: harden dynamic rectification protocol
This commit is contained in:
@@ -0,0 +1,75 @@
|
|||||||
|
# Task 1 — Dynamic Choice Contracts and Stop Policy
|
||||||
|
|
||||||
|
## Implementation
|
||||||
|
|
||||||
|
- Added browser-safe dynamic choice and time-range Zod schemas. Public question parsing is strict and rejects hidden partition fields.
|
||||||
|
- Added internal-only dynamic choice contracts, persisted/private question schemas, candidate-difference packet schemas, and an explicit public projection helper.
|
||||||
|
- Added pure deterministic stop policy with the specified precedence and a material-change calculation for candidate range, representative time, and two-point margin changes.
|
||||||
|
- Added separate `DynamicNextAction` and `DynamicJourneyProgress` schemas, preserving the legacy guided-v1 `NextAction` and `JourneyProgress` parser path.
|
||||||
|
- Kept the internal contract module dependency-free as resolved by the user. A source-contract test scans components, hooks, client transports, and response schemas to prohibit imports of the private module.
|
||||||
|
- Dynamic IDs are opaque nonempty server-issued strings, rather than being overconstrained to UUIDs.
|
||||||
|
|
||||||
|
## Files changed
|
||||||
|
|
||||||
|
- `frontend/src/lib/birth-time-dynamic-choice.ts`
|
||||||
|
- `frontend/src/lib/birth-time-dynamic-choice-internal.ts`
|
||||||
|
- `frontend/src/lib/birth-time-dynamic-stop-policy.ts`
|
||||||
|
- `frontend/src/lib/birth-time-journey-turn-protocol.ts`
|
||||||
|
- `frontend/src/lib/birth-time-journey-turn.ts`
|
||||||
|
- `frontend/tests/birth-time-dynamic-choice.test.ts`
|
||||||
|
- `frontend/tests/birth-time-dynamic-stop-policy.test.ts`
|
||||||
|
|
||||||
|
## RED
|
||||||
|
|
||||||
|
1. `/Users/jesse/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node --test tests/birth-time-dynamic-choice.test.ts`
|
||||||
|
- Failed as expected before the public contract existed: `ERR_MODULE_NOT_FOUND` for `birth-time-dynamic-choice.ts`.
|
||||||
|
2. `/Users/jesse/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node --test tests/birth-time-dynamic-stop-policy.test.ts`
|
||||||
|
- Failed as expected before the policy existed: `ERR_MODULE_NOT_FOUND` for `birth-time-dynamic-stop-policy.ts`.
|
||||||
|
3. After the boundary resolution, the dynamic choice test failed as expected while the obsolete `server-only` marker remained: `ERR_MODULE_NOT_FOUND: Cannot find package 'server-only'`.
|
||||||
|
4. The opaque-ID regression initially failed because the first implementation required UUIDs.
|
||||||
|
|
||||||
|
## GREEN
|
||||||
|
|
||||||
|
1. `/Users/jesse/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node --test tests/birth-time-dynamic-choice.test.ts tests/birth-time-dynamic-stop-policy.test.ts tests/birth-time-journey-turn.test.ts`
|
||||||
|
- `14` passed, `0` failed.
|
||||||
|
2. `/Users/jesse/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node --test tests/birth-time-*.test.ts`
|
||||||
|
- `194` passed, `0` failed, duration `1449ms`.
|
||||||
|
3. `git diff --check`
|
||||||
|
- Passed with no whitespace errors.
|
||||||
|
|
||||||
|
## Self-review
|
||||||
|
|
||||||
|
- Public choices are strict, require 2–4 primary options plus exactly one unknown and one unmatched option, reject duplicate IDs, cap labels at 80 characters, and reject private fields.
|
||||||
|
- Persisted primary choices require nonempty partitions and finite score maps. Unknown/unmatched choices require both private fields to be `null`.
|
||||||
|
- The public projection parses through the public schema, so partition IDs and candidate scores cannot cross the browser boundary.
|
||||||
|
- Stop ordering is high confidence, effective-answer safety cap, plateau, no information gain, repeated partition, then continue. Non-effective answers retain the prior plateau count.
|
||||||
|
- Legacy schemas and turn behavior remain unchanged; v2 schemas use distinct dynamic names and are re-exported from the turn module.
|
||||||
|
- All created/modified source files are within the 250 pure-LOC threshold (largest: `birth-time-journey-turn.ts`, 229 lines; new internal contract, 208 lines).
|
||||||
|
|
||||||
|
## Concerns
|
||||||
|
|
||||||
|
- Full `tsc --noEmit --incremental false` remains blocked by an unrelated existing error in `frontend/tests/profile-persistence.test.ts:7`: the project targets ES2017 while that test uses an ES2018 regular-expression flag. None of the Task 1 files produced a TypeScript error.
|
||||||
|
- The supplied no-excuse checker could not run because it is outside the frontend dependency tree and cannot resolve its own `typescript` package. The focused runtime suite, full birth-time suite, diff check, and manual forbidden-pattern scan completed successfully.
|
||||||
|
|
||||||
|
## Review fixes
|
||||||
|
|
||||||
|
- `DynamicStopInput.result` is now nullable, so a dynamic flow can finish before its first score. It also carries the explicit `forcedReason` union: `user_finished`, `generation_unavailable`, or `null`.
|
||||||
|
- Forced terminal reasons now win over every score-derived condition. A null result preserves the current plateau count instead of attempting score comparison.
|
||||||
|
- Added and re-exported `dynamicJourneyTurnStateSchema` / `DynamicJourneyTurnState`. The schema is strict and explicitly requires `journeyProtocol: "dynamic-choice-v2"`, a nonnegative turn version, a dynamic action, dynamic progress, and the existing permissions shape. The legacy `journeyTurnStateSchema` is unchanged.
|
||||||
|
- Added regressions for both forced terminal reasons, their high-confidence precedence, the dynamic discriminator, and rejection of a valid legacy action under the v2 schema.
|
||||||
|
|
||||||
|
### Review RED
|
||||||
|
|
||||||
|
`/Users/jesse/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node --test tests/birth-time-dynamic-choice.test.ts tests/birth-time-dynamic-stop-policy.test.ts`
|
||||||
|
|
||||||
|
- Failed before implementation because `dynamicJourneyTurnStateSchema` was not exported.
|
||||||
|
- Existing stop policy threw on `result: null` and returned `high_confidence` instead of the forced `user_finished` reason.
|
||||||
|
|
||||||
|
### Review GREEN
|
||||||
|
|
||||||
|
1. `/Users/jesse/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node --test tests/birth-time-dynamic-choice.test.ts tests/birth-time-dynamic-stop-policy.test.ts tests/birth-time-journey-turn.test.ts`
|
||||||
|
- `16` passed, `0` failed.
|
||||||
|
2. `/Users/jesse/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node --test tests/birth-time-*.test.ts`
|
||||||
|
- `196` passed, `0` failed, duration `1472ms`.
|
||||||
|
3. `/Users/jesse/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node ./node_modules/typescript/bin/tsc --noEmit --incremental false`
|
||||||
|
- Still reports only the existing `tests/profile-persistence.test.ts:7` ES2018-regexp/ES2017-target incompatibility; no Task 1 diagnostic was emitted.
|
||||||
@@ -1,19 +1,20 @@
|
|||||||
import type { CandidateResult } from "./birth-time-evidence.ts";
|
import type { CandidateResult } from "./birth-time-evidence.ts";
|
||||||
|
|
||||||
export type DynamicStopInput = {
|
export type DynamicStopInput = {
|
||||||
readonly result: CandidateResult;
|
readonly result: CandidateResult | null;
|
||||||
readonly effectiveAnswer: boolean;
|
readonly effectiveAnswer: boolean;
|
||||||
readonly previousResult: CandidateResult | null;
|
readonly previousResult: CandidateResult | null;
|
||||||
readonly priorPlateauCount: number;
|
readonly priorPlateauCount: number;
|
||||||
readonly usefulOpportunityCount: number;
|
readonly usefulOpportunityCount: number;
|
||||||
readonly repeatedOnly: boolean;
|
readonly repeatedOnly: boolean;
|
||||||
readonly effectiveAnswerCount: number;
|
readonly effectiveAnswerCount: number;
|
||||||
|
readonly forcedReason: "user_finished" | "generation_unavailable" | null;
|
||||||
};
|
};
|
||||||
|
|
||||||
export type DynamicStopDecision =
|
export type DynamicStopDecision =
|
||||||
| {
|
| {
|
||||||
readonly kind: "finish";
|
readonly kind: "finish";
|
||||||
readonly reason: "high_confidence" | "safety_cap" | "plateau" | "no_information_gain" | "repeated_partition";
|
readonly reason: "user_finished" | "generation_unavailable" | "high_confidence" | "safety_cap" | "plateau" | "no_information_gain" | "repeated_partition";
|
||||||
readonly plateauCount: number;
|
readonly plateauCount: number;
|
||||||
}
|
}
|
||||||
| { readonly kind: "continue"; readonly plateauCount: number };
|
| { readonly kind: "continue"; readonly plateauCount: number };
|
||||||
@@ -34,10 +35,11 @@ export function materiallyChanged(
|
|||||||
}
|
}
|
||||||
|
|
||||||
export function decideDynamicStop(input: DynamicStopInput): DynamicStopDecision {
|
export function decideDynamicStop(input: DynamicStopInput): DynamicStopDecision {
|
||||||
const plateauCount = input.effectiveAnswer
|
const plateauCount = input.effectiveAnswer && input.result !== null
|
||||||
? materiallyChanged(input.previousResult, input.result) ? 0 : input.priorPlateauCount + 1
|
? materiallyChanged(input.previousResult, input.result) ? 0 : input.priorPlateauCount + 1
|
||||||
: input.priorPlateauCount;
|
: input.priorPlateauCount;
|
||||||
if (input.result.confidence === "high") return { kind: "finish", reason: "high_confidence", plateauCount };
|
if (input.forcedReason !== null) return { kind: "finish", reason: input.forcedReason, plateauCount };
|
||||||
|
if (input.result?.confidence === "high") return { kind: "finish", reason: "high_confidence", plateauCount };
|
||||||
if (input.effectiveAnswerCount >= 10) return { kind: "finish", reason: "safety_cap", plateauCount };
|
if (input.effectiveAnswerCount >= 10) return { kind: "finish", reason: "safety_cap", plateauCount };
|
||||||
if (plateauCount >= 2) return { kind: "finish", reason: "plateau", plateauCount };
|
if (plateauCount >= 2) return { kind: "finish", reason: "plateau", plateauCount };
|
||||||
if (input.usefulOpportunityCount === 0) return { kind: "finish", reason: "no_information_gain", plateauCount };
|
if (input.usefulOpportunityCount === 0) return { kind: "finish", reason: "no_information_gain", plateauCount };
|
||||||
|
|||||||
@@ -81,6 +81,16 @@ export type DynamicJourneyProgress = z.infer<typeof dynamicJourneyProgressSchema
|
|||||||
|
|
||||||
export type { PublicDynamicChoiceQuestion, TimeRange };
|
export type { PublicDynamicChoiceQuestion, TimeRange };
|
||||||
|
|
||||||
|
export const dynamicJourneyTurnStateSchema = z.object({
|
||||||
|
journeyProtocol: z.literal("dynamic-choice-v2"),
|
||||||
|
turnVersion: z.number().int().nonnegative(),
|
||||||
|
nextAction: dynamicNextActionSchema,
|
||||||
|
progress: dynamicJourneyProgressSchema,
|
||||||
|
permissions: journeyPermissionsSchema,
|
||||||
|
}).strict().readonly();
|
||||||
|
|
||||||
|
export type DynamicJourneyTurnState = z.infer<typeof dynamicJourneyTurnStateSchema>;
|
||||||
|
|
||||||
export const journeyTurnStateSchema = z.object({
|
export const journeyTurnStateSchema = z.object({
|
||||||
turnVersion: z.number().int().nonnegative(),
|
turnVersion: z.number().int().nonnegative(),
|
||||||
nextAction: nextActionSchema,
|
nextAction: nextActionSchema,
|
||||||
|
|||||||
@@ -18,10 +18,12 @@ export {
|
|||||||
questionSpecSchema,
|
questionSpecSchema,
|
||||||
dynamicJourneyProgressSchema,
|
dynamicJourneyProgressSchema,
|
||||||
dynamicNextActionSchema,
|
dynamicNextActionSchema,
|
||||||
|
dynamicJourneyTurnStateSchema,
|
||||||
} from "./birth-time-journey-turn-protocol.ts";
|
} from "./birth-time-journey-turn-protocol.ts";
|
||||||
export type {
|
export type {
|
||||||
DynamicJourneyProgress,
|
DynamicJourneyProgress,
|
||||||
DynamicNextAction,
|
DynamicNextAction,
|
||||||
|
DynamicJourneyTurnState,
|
||||||
EvidenceDraft,
|
EvidenceDraft,
|
||||||
JourneyPermissions,
|
JourneyPermissions,
|
||||||
JourneyProgress,
|
JourneyProgress,
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ import { join } from "node:path";
|
|||||||
import test from "node:test";
|
import test from "node:test";
|
||||||
import { publicDynamicChoiceQuestionSchema } from "../src/lib/birth-time-dynamic-choice.ts";
|
import { publicDynamicChoiceQuestionSchema } from "../src/lib/birth-time-dynamic-choice.ts";
|
||||||
import { persistedDynamicChoiceQuestionSchema } from "../src/lib/birth-time-dynamic-choice-internal.ts";
|
import { persistedDynamicChoiceQuestionSchema } from "../src/lib/birth-time-dynamic-choice-internal.ts";
|
||||||
|
import { dynamicJourneyTurnStateSchema, journeyTurnStateSchema } from "../src/lib/birth-time-journey-turn-protocol.ts";
|
||||||
|
|
||||||
const internalQuestion = {
|
const internalQuestion = {
|
||||||
questionId: "11111111-1111-4111-8111-111111111111",
|
questionId: "11111111-1111-4111-8111-111111111111",
|
||||||
@@ -133,3 +134,42 @@ test("public code never imports the internal dynamic choice contract", () => {
|
|||||||
assert.equal(readFileSync(path, "utf8").includes("birth-time-dynamic-choice-internal"), false, path);
|
assert.equal(readFileSync(path, "utf8").includes("birth-time-dynamic-choice-internal"), false, path);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test("dynamic turn state is explicitly discriminated from the legacy protocol", () => {
|
||||||
|
const dynamicTurn = {
|
||||||
|
journeyProtocol: "dynamic-choice-v2",
|
||||||
|
turnVersion: 0,
|
||||||
|
nextAction: { kind: "generate_dynamic_question" },
|
||||||
|
progress: {
|
||||||
|
phase: "question",
|
||||||
|
answeredCount: 0,
|
||||||
|
effectiveAnswerCount: 0,
|
||||||
|
currentRange: { startTime: "09:00", endTime: "10:00" },
|
||||||
|
previousRange: null,
|
||||||
|
plateauCount: 0,
|
||||||
|
},
|
||||||
|
permissions: { canConfirmCandidate: false },
|
||||||
|
};
|
||||||
|
|
||||||
|
assert.equal(dynamicJourneyTurnStateSchema.safeParse(dynamicTurn).success, true);
|
||||||
|
assert.equal(journeyTurnStateSchema.safeParse(dynamicTurn).success, false);
|
||||||
|
assert.equal(dynamicJourneyTurnStateSchema.safeParse({
|
||||||
|
...dynamicTurn,
|
||||||
|
journeyProtocol: "legacy-guided-v1",
|
||||||
|
}).success, false);
|
||||||
|
assert.equal(dynamicJourneyTurnStateSchema.safeParse({
|
||||||
|
...dynamicTurn,
|
||||||
|
nextAction: {
|
||||||
|
kind: "ask_baseline_evidence",
|
||||||
|
question: {
|
||||||
|
questionId: "legacy",
|
||||||
|
phase: "baseline",
|
||||||
|
domain: "career",
|
||||||
|
requestedPrecision: ["year"],
|
||||||
|
allowUnknown: true,
|
||||||
|
purposeCode: "candidate_difference_career",
|
||||||
|
plannerVersion: "candidate-difference-v1",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}).success, false);
|
||||||
|
});
|
||||||
|
|||||||
@@ -34,6 +34,7 @@ function decisionFor(overrides: Partial<Parameters<typeof decideDynamicStop>[0]>
|
|||||||
usefulOpportunityCount: 1,
|
usefulOpportunityCount: 1,
|
||||||
repeatedOnly: false,
|
repeatedOnly: false,
|
||||||
effectiveAnswerCount: 1,
|
effectiveAnswerCount: 1,
|
||||||
|
forcedReason: null,
|
||||||
...overrides,
|
...overrides,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
@@ -53,6 +54,7 @@ test("two effective unchanged scores stop without starting another question", ()
|
|||||||
usefulOpportunityCount: 3,
|
usefulOpportunityCount: 3,
|
||||||
repeatedOnly: false,
|
repeatedOnly: false,
|
||||||
effectiveAnswerCount: 6,
|
effectiveAnswerCount: 6,
|
||||||
|
forcedReason: null,
|
||||||
});
|
});
|
||||||
|
|
||||||
assert.deepEqual(decision, { kind: "finish", reason: "plateau", plateauCount: 2 });
|
assert.deepEqual(decision, { kind: "finish", reason: "plateau", plateauCount: 2 });
|
||||||
@@ -67,12 +69,15 @@ test("unknown answers do not advance plateau or the effective safety count", ()
|
|||||||
usefulOpportunityCount: 2,
|
usefulOpportunityCount: 2,
|
||||||
repeatedOnly: false,
|
repeatedOnly: false,
|
||||||
effectiveAnswerCount: 4,
|
effectiveAnswerCount: 4,
|
||||||
|
forcedReason: null,
|
||||||
});
|
});
|
||||||
|
|
||||||
assert.deepEqual(decision, { kind: "continue", plateauCount: 1 });
|
assert.deepEqual(decision, { kind: "continue", plateauCount: 1 });
|
||||||
});
|
});
|
||||||
|
|
||||||
test("terminal conditions are deterministic", () => {
|
test("terminal conditions are deterministic", () => {
|
||||||
|
assert.equal(finishReason({ result: null, forcedReason: "user_finished" }), "user_finished");
|
||||||
|
assert.equal(finishReason({ result: null, forcedReason: "generation_unavailable" }), "generation_unavailable");
|
||||||
assert.equal(finishReason({ result: { ...lowCandidate, confidence: "high", canApply: true, winningSegment: {
|
assert.equal(finishReason({ result: { ...lowCandidate, confidence: "high", canApply: true, winningSegment: {
|
||||||
startTime: "09:00", endTime: "09:05", representativeTime: "09:03", widthMinutes: 5,
|
startTime: "09:00", endTime: "09:05", representativeTime: "09:03", widthMinutes: 5,
|
||||||
}, eventCount: 4, domainCount: 3, marginPercent: 20 } }), "high_confidence");
|
}, eventCount: 4, domainCount: 3, marginPercent: 20 } }), "high_confidence");
|
||||||
@@ -81,6 +86,15 @@ test("terminal conditions are deterministic", () => {
|
|||||||
assert.equal(finishReason({ effectiveAnswerCount: 10 }), "safety_cap");
|
assert.equal(finishReason({ effectiveAnswerCount: 10 }), "safety_cap");
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test("forced terminal reasons win over a high-confidence score", () => {
|
||||||
|
assert.equal(finishReason({
|
||||||
|
result: { ...lowCandidate, confidence: "high", canApply: true, winningSegment: {
|
||||||
|
startTime: "09:00", endTime: "09:05", representativeTime: "09:03", widthMinutes: 5,
|
||||||
|
}, eventCount: 4, domainCount: 3, marginPercent: 20 },
|
||||||
|
forcedReason: "user_finished",
|
||||||
|
}), "user_finished");
|
||||||
|
});
|
||||||
|
|
||||||
test("a two point margin change resets the plateau", () => {
|
test("a two point margin change resets the plateau", () => {
|
||||||
const decision = decisionFor({
|
const decision = decisionFor({
|
||||||
result: { ...mediumCandidate, marginPercent: 17 },
|
result: { ...mediumCandidate, marginPercent: 17 },
|
||||||
|
|||||||
Reference in New Issue
Block a user