fix: harden dynamic rectification protocol
This commit is contained in:
@@ -0,0 +1,75 @@
|
||||
# Task 1 — Dynamic Choice Contracts and Stop Policy
|
||||
|
||||
## Implementation
|
||||
|
||||
- Added browser-safe dynamic choice and time-range Zod schemas. Public question parsing is strict and rejects hidden partition fields.
|
||||
- Added internal-only dynamic choice contracts, persisted/private question schemas, candidate-difference packet schemas, and an explicit public projection helper.
|
||||
- Added pure deterministic stop policy with the specified precedence and a material-change calculation for candidate range, representative time, and two-point margin changes.
|
||||
- Added separate `DynamicNextAction` and `DynamicJourneyProgress` schemas, preserving the legacy guided-v1 `NextAction` and `JourneyProgress` parser path.
|
||||
- Kept the internal contract module dependency-free as resolved by the user. A source-contract test scans components, hooks, client transports, and response schemas to prohibit imports of the private module.
|
||||
- Dynamic IDs are opaque nonempty server-issued strings, rather than being overconstrained to UUIDs.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `frontend/src/lib/birth-time-dynamic-choice.ts`
|
||||
- `frontend/src/lib/birth-time-dynamic-choice-internal.ts`
|
||||
- `frontend/src/lib/birth-time-dynamic-stop-policy.ts`
|
||||
- `frontend/src/lib/birth-time-journey-turn-protocol.ts`
|
||||
- `frontend/src/lib/birth-time-journey-turn.ts`
|
||||
- `frontend/tests/birth-time-dynamic-choice.test.ts`
|
||||
- `frontend/tests/birth-time-dynamic-stop-policy.test.ts`
|
||||
|
||||
## RED
|
||||
|
||||
1. `/Users/jesse/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node --test tests/birth-time-dynamic-choice.test.ts`
|
||||
- Failed as expected before the public contract existed: `ERR_MODULE_NOT_FOUND` for `birth-time-dynamic-choice.ts`.
|
||||
2. `/Users/jesse/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node --test tests/birth-time-dynamic-stop-policy.test.ts`
|
||||
- Failed as expected before the policy existed: `ERR_MODULE_NOT_FOUND` for `birth-time-dynamic-stop-policy.ts`.
|
||||
3. After the boundary resolution, the dynamic choice test failed as expected while the obsolete `server-only` marker remained: `ERR_MODULE_NOT_FOUND: Cannot find package 'server-only'`.
|
||||
4. The opaque-ID regression initially failed because the first implementation required UUIDs.
|
||||
|
||||
## GREEN
|
||||
|
||||
1. `/Users/jesse/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node --test tests/birth-time-dynamic-choice.test.ts tests/birth-time-dynamic-stop-policy.test.ts tests/birth-time-journey-turn.test.ts`
|
||||
- `14` passed, `0` failed.
|
||||
2. `/Users/jesse/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node --test tests/birth-time-*.test.ts`
|
||||
- `194` passed, `0` failed, duration `1449ms`.
|
||||
3. `git diff --check`
|
||||
- Passed with no whitespace errors.
|
||||
|
||||
## Self-review
|
||||
|
||||
- Public choices are strict, require 2–4 primary options plus exactly one unknown and one unmatched option, reject duplicate IDs, cap labels at 80 characters, and reject private fields.
|
||||
- Persisted primary choices require nonempty partitions and finite score maps. Unknown/unmatched choices require both private fields to be `null`.
|
||||
- The public projection parses through the public schema, so partition IDs and candidate scores cannot cross the browser boundary.
|
||||
- Stop ordering is high confidence, effective-answer safety cap, plateau, no information gain, repeated partition, then continue. Non-effective answers retain the prior plateau count.
|
||||
- Legacy schemas and turn behavior remain unchanged; v2 schemas use distinct dynamic names and are re-exported from the turn module.
|
||||
- All created/modified source files are within the 250 pure-LOC threshold (largest: `birth-time-journey-turn.ts`, 229 lines; new internal contract, 208 lines).
|
||||
|
||||
## Concerns
|
||||
|
||||
- Full `tsc --noEmit --incremental false` remains blocked by an unrelated existing error in `frontend/tests/profile-persistence.test.ts:7`: the project targets ES2017 while that test uses an ES2018 regular-expression flag. None of the Task 1 files produced a TypeScript error.
|
||||
- The supplied no-excuse checker could not run because it is outside the frontend dependency tree and cannot resolve its own `typescript` package. The focused runtime suite, full birth-time suite, diff check, and manual forbidden-pattern scan completed successfully.
|
||||
|
||||
## Review fixes
|
||||
|
||||
- `DynamicStopInput.result` is now nullable, so a dynamic flow can finish before its first score. It also carries the explicit `forcedReason` union: `user_finished`, `generation_unavailable`, or `null`.
|
||||
- Forced terminal reasons now win over every score-derived condition. A null result preserves the current plateau count instead of attempting score comparison.
|
||||
- Added and re-exported `dynamicJourneyTurnStateSchema` / `DynamicJourneyTurnState`. The schema is strict and explicitly requires `journeyProtocol: "dynamic-choice-v2"`, a nonnegative turn version, a dynamic action, dynamic progress, and the existing permissions shape. The legacy `journeyTurnStateSchema` is unchanged.
|
||||
- Added regressions for both forced terminal reasons, their high-confidence precedence, the dynamic discriminator, and rejection of a valid legacy action under the v2 schema.
|
||||
|
||||
### Review RED
|
||||
|
||||
`/Users/jesse/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node --test tests/birth-time-dynamic-choice.test.ts tests/birth-time-dynamic-stop-policy.test.ts`
|
||||
|
||||
- Failed before implementation because `dynamicJourneyTurnStateSchema` was not exported.
|
||||
- Existing stop policy threw on `result: null` and returned `high_confidence` instead of the forced `user_finished` reason.
|
||||
|
||||
### Review GREEN
|
||||
|
||||
1. `/Users/jesse/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node --test tests/birth-time-dynamic-choice.test.ts tests/birth-time-dynamic-stop-policy.test.ts tests/birth-time-journey-turn.test.ts`
|
||||
- `16` passed, `0` failed.
|
||||
2. `/Users/jesse/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node --test tests/birth-time-*.test.ts`
|
||||
- `196` passed, `0` failed, duration `1472ms`.
|
||||
3. `/Users/jesse/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node ./node_modules/typescript/bin/tsc --noEmit --incremental false`
|
||||
- Still reports only the existing `tests/profile-persistence.test.ts:7` ES2018-regexp/ES2017-target incompatibility; no Task 1 diagnostic was emitted.
|
||||
@@ -1,19 +1,20 @@
|
||||
import type { CandidateResult } from "./birth-time-evidence.ts";
|
||||
|
||||
export type DynamicStopInput = {
|
||||
readonly result: CandidateResult;
|
||||
readonly result: CandidateResult | null;
|
||||
readonly effectiveAnswer: boolean;
|
||||
readonly previousResult: CandidateResult | null;
|
||||
readonly priorPlateauCount: number;
|
||||
readonly usefulOpportunityCount: number;
|
||||
readonly repeatedOnly: boolean;
|
||||
readonly effectiveAnswerCount: number;
|
||||
readonly forcedReason: "user_finished" | "generation_unavailable" | null;
|
||||
};
|
||||
|
||||
export type DynamicStopDecision =
|
||||
| {
|
||||
readonly kind: "finish";
|
||||
readonly reason: "high_confidence" | "safety_cap" | "plateau" | "no_information_gain" | "repeated_partition";
|
||||
readonly reason: "user_finished" | "generation_unavailable" | "high_confidence" | "safety_cap" | "plateau" | "no_information_gain" | "repeated_partition";
|
||||
readonly plateauCount: number;
|
||||
}
|
||||
| { readonly kind: "continue"; readonly plateauCount: number };
|
||||
@@ -34,10 +35,11 @@ export function materiallyChanged(
|
||||
}
|
||||
|
||||
export function decideDynamicStop(input: DynamicStopInput): DynamicStopDecision {
|
||||
const plateauCount = input.effectiveAnswer
|
||||
const plateauCount = input.effectiveAnswer && input.result !== null
|
||||
? materiallyChanged(input.previousResult, input.result) ? 0 : input.priorPlateauCount + 1
|
||||
: input.priorPlateauCount;
|
||||
if (input.result.confidence === "high") return { kind: "finish", reason: "high_confidence", plateauCount };
|
||||
if (input.forcedReason !== null) return { kind: "finish", reason: input.forcedReason, plateauCount };
|
||||
if (input.result?.confidence === "high") return { kind: "finish", reason: "high_confidence", plateauCount };
|
||||
if (input.effectiveAnswerCount >= 10) return { kind: "finish", reason: "safety_cap", plateauCount };
|
||||
if (plateauCount >= 2) return { kind: "finish", reason: "plateau", plateauCount };
|
||||
if (input.usefulOpportunityCount === 0) return { kind: "finish", reason: "no_information_gain", plateauCount };
|
||||
|
||||
@@ -81,6 +81,16 @@ export type DynamicJourneyProgress = z.infer<typeof dynamicJourneyProgressSchema
|
||||
|
||||
export type { PublicDynamicChoiceQuestion, TimeRange };
|
||||
|
||||
export const dynamicJourneyTurnStateSchema = z.object({
|
||||
journeyProtocol: z.literal("dynamic-choice-v2"),
|
||||
turnVersion: z.number().int().nonnegative(),
|
||||
nextAction: dynamicNextActionSchema,
|
||||
progress: dynamicJourneyProgressSchema,
|
||||
permissions: journeyPermissionsSchema,
|
||||
}).strict().readonly();
|
||||
|
||||
export type DynamicJourneyTurnState = z.infer<typeof dynamicJourneyTurnStateSchema>;
|
||||
|
||||
export const journeyTurnStateSchema = z.object({
|
||||
turnVersion: z.number().int().nonnegative(),
|
||||
nextAction: nextActionSchema,
|
||||
|
||||
@@ -18,10 +18,12 @@ export {
|
||||
questionSpecSchema,
|
||||
dynamicJourneyProgressSchema,
|
||||
dynamicNextActionSchema,
|
||||
dynamicJourneyTurnStateSchema,
|
||||
} from "./birth-time-journey-turn-protocol.ts";
|
||||
export type {
|
||||
DynamicJourneyProgress,
|
||||
DynamicNextAction,
|
||||
DynamicJourneyTurnState,
|
||||
EvidenceDraft,
|
||||
JourneyPermissions,
|
||||
JourneyProgress,
|
||||
|
||||
@@ -4,6 +4,7 @@ import { join } from "node:path";
|
||||
import test from "node:test";
|
||||
import { publicDynamicChoiceQuestionSchema } from "../src/lib/birth-time-dynamic-choice.ts";
|
||||
import { persistedDynamicChoiceQuestionSchema } from "../src/lib/birth-time-dynamic-choice-internal.ts";
|
||||
import { dynamicJourneyTurnStateSchema, journeyTurnStateSchema } from "../src/lib/birth-time-journey-turn-protocol.ts";
|
||||
|
||||
const internalQuestion = {
|
||||
questionId: "11111111-1111-4111-8111-111111111111",
|
||||
@@ -133,3 +134,42 @@ test("public code never imports the internal dynamic choice contract", () => {
|
||||
assert.equal(readFileSync(path, "utf8").includes("birth-time-dynamic-choice-internal"), false, path);
|
||||
}
|
||||
});
|
||||
|
||||
test("dynamic turn state is explicitly discriminated from the legacy protocol", () => {
|
||||
const dynamicTurn = {
|
||||
journeyProtocol: "dynamic-choice-v2",
|
||||
turnVersion: 0,
|
||||
nextAction: { kind: "generate_dynamic_question" },
|
||||
progress: {
|
||||
phase: "question",
|
||||
answeredCount: 0,
|
||||
effectiveAnswerCount: 0,
|
||||
currentRange: { startTime: "09:00", endTime: "10:00" },
|
||||
previousRange: null,
|
||||
plateauCount: 0,
|
||||
},
|
||||
permissions: { canConfirmCandidate: false },
|
||||
};
|
||||
|
||||
assert.equal(dynamicJourneyTurnStateSchema.safeParse(dynamicTurn).success, true);
|
||||
assert.equal(journeyTurnStateSchema.safeParse(dynamicTurn).success, false);
|
||||
assert.equal(dynamicJourneyTurnStateSchema.safeParse({
|
||||
...dynamicTurn,
|
||||
journeyProtocol: "legacy-guided-v1",
|
||||
}).success, false);
|
||||
assert.equal(dynamicJourneyTurnStateSchema.safeParse({
|
||||
...dynamicTurn,
|
||||
nextAction: {
|
||||
kind: "ask_baseline_evidence",
|
||||
question: {
|
||||
questionId: "legacy",
|
||||
phase: "baseline",
|
||||
domain: "career",
|
||||
requestedPrecision: ["year"],
|
||||
allowUnknown: true,
|
||||
purposeCode: "candidate_difference_career",
|
||||
plannerVersion: "candidate-difference-v1",
|
||||
},
|
||||
},
|
||||
}).success, false);
|
||||
});
|
||||
|
||||
@@ -34,6 +34,7 @@ function decisionFor(overrides: Partial<Parameters<typeof decideDynamicStop>[0]>
|
||||
usefulOpportunityCount: 1,
|
||||
repeatedOnly: false,
|
||||
effectiveAnswerCount: 1,
|
||||
forcedReason: null,
|
||||
...overrides,
|
||||
});
|
||||
}
|
||||
@@ -53,6 +54,7 @@ test("two effective unchanged scores stop without starting another question", ()
|
||||
usefulOpportunityCount: 3,
|
||||
repeatedOnly: false,
|
||||
effectiveAnswerCount: 6,
|
||||
forcedReason: null,
|
||||
});
|
||||
|
||||
assert.deepEqual(decision, { kind: "finish", reason: "plateau", plateauCount: 2 });
|
||||
@@ -67,12 +69,15 @@ test("unknown answers do not advance plateau or the effective safety count", ()
|
||||
usefulOpportunityCount: 2,
|
||||
repeatedOnly: false,
|
||||
effectiveAnswerCount: 4,
|
||||
forcedReason: null,
|
||||
});
|
||||
|
||||
assert.deepEqual(decision, { kind: "continue", plateauCount: 1 });
|
||||
});
|
||||
|
||||
test("terminal conditions are deterministic", () => {
|
||||
assert.equal(finishReason({ result: null, forcedReason: "user_finished" }), "user_finished");
|
||||
assert.equal(finishReason({ result: null, forcedReason: "generation_unavailable" }), "generation_unavailable");
|
||||
assert.equal(finishReason({ result: { ...lowCandidate, confidence: "high", canApply: true, winningSegment: {
|
||||
startTime: "09:00", endTime: "09:05", representativeTime: "09:03", widthMinutes: 5,
|
||||
}, eventCount: 4, domainCount: 3, marginPercent: 20 } }), "high_confidence");
|
||||
@@ -81,6 +86,15 @@ test("terminal conditions are deterministic", () => {
|
||||
assert.equal(finishReason({ effectiveAnswerCount: 10 }), "safety_cap");
|
||||
});
|
||||
|
||||
test("forced terminal reasons win over a high-confidence score", () => {
|
||||
assert.equal(finishReason({
|
||||
result: { ...lowCandidate, confidence: "high", canApply: true, winningSegment: {
|
||||
startTime: "09:00", endTime: "09:05", representativeTime: "09:03", widthMinutes: 5,
|
||||
}, eventCount: 4, domainCount: 3, marginPercent: 20 },
|
||||
forcedReason: "user_finished",
|
||||
}), "user_finished");
|
||||
});
|
||||
|
||||
test("a two point margin change resets the plateau", () => {
|
||||
const decision = decisionFor({
|
||||
result: { ...mediumCandidate, marginPercent: 17 },
|
||||
|
||||
Reference in New Issue
Block a user