diff --git a/docs/BUG_HISTORY.md b/docs/BUG_HISTORY.md index f3a12ed9..17921b92 100644 --- a/docs/BUG_HISTORY.md +++ b/docs/BUG_HISTORY.md @@ -24397,3 +24397,17 @@ - 判断:她的生成器修复(§12)在本仓没有可移植对象;快照不重生成、不改。若以后移植她的生成器,需要一起带她的 4 个新用例。 - 关联记录:无 - 修复版本:无 + +## BUG-1814 | 查卡一次只能取一段,每多一段就多一步,整份计算随每一步重发 + +- 状态:fixed-pending-verify(分支 codex/token-lookup-20261109) +- 首次发现 / 最近更新:2026-11-08(占星师验收清单 §4)/ 2026-11-09(修复) +- 影响面:frontend/src/mastra/consultation-tools.ts(evidenceLookupInputSchema、查卡工具本体、ConsultationRuntimeStep)。 +- 现象:模型要用两段证据时要调两次 read-consultation-evidence,于是多两步;每一步都把约 9.3 万字符的计算结果再发一遍。同一张盘实测:调用计算那步 prompt 62,859 字符,看过计算结果后 156,254,取一次小段后 156,966,再取一次后 157,996。 +- 触发条件:回答前需要多段卡外证据(常见于高严谨模式)。 +- 根因:工具输入是 { section },一次只返回一段;多段只能多次调用,而每次调用都多一步,历史工具结果随每一步重发。 +- 修复:输入改为 z.union,保留 { section } 单段写法,新增 { sections: [...] }(2–6 段、去重、仍计 1 次调用、仍受每轮 3 次上限)。返回 { status, results: [{ section, status, data | reason }], note };单段写法返回形状与键序不变。可选 fields 只取该段顶层键,未知键进 unknown_fields,不截断数据。runtime step 记 sections 与 chars(内部字段,不进公开回执)。 +- 验证:frontend/tests/consult-evidence-lookup-batch-20261109.test.ts(7 条);consult-evidence-lookup-20260927(15)、consult-same-source-check-1773(4)、consult-same-source-independence-1773(4)、consultation-agentic-runtime(87)全过;k=2 时总输入字符下降约 29–30%,k=3 约 44–45%(测量表在 docs/tasks/PROGRESS-token-lookup-20261109.md)。 +- 防复发:上述测试;测量脚本 frontend/scripts/research/consult-lookup-token-probe-171.mts 可复跑。 +- 关联记录:BUG-1803 +- 修复版本:分支 codex/token-lookup-20261109 diff --git a/docs/tasks/PROGRESS-token-lookup-20261109.md b/docs/tasks/PROGRESS-token-lookup-20261109.md new file mode 100644 index 00000000..ebd39a57 --- /dev/null +++ b/docs/tasks/PROGRESS-token-lookup-20261109.md @@ -0,0 +1,87 @@ +# PROGRESS · 查卡一次取多段(#171 省 token,2026-11-09) + +- 分支:`codex/token-lookup-20261109`(基线 `b453505d`) +- 任务书:`docs/tasks/TASK-token-lookup-20261109.md` +- Bug:`docs/BUG_HISTORY.md` 新增 BUG-1814(关联 BUG-1803) + +## 第一步:测量(先量后改) + +用 golden 公开盘(Steve Jobs、Barack Obama、Elizabeth Taylor)经真实链路建 `executions`,再逐段调用 `readEvidenceLookupSection`;工具结果用真实 agent 链路(`runNatalAgent` + 脚本化模型)实测每一步的 prompt 字符数。 + +### 每步 prompt 实测(Steve Jobs / career) + +| 步 | 这一步的 prompt 字符数 | 其中的工具结果 | +| --- | ---: | --- | +| 调计算 | 62,859 | — | +| 看过计算结果 | 156,254 | run-jyotish-consultation 93,202 | +| 取 D60 之后 | 156,966 | + read-consultation-evidence 535 | +| 取 yogas 之后(写答案) | 157,996 | + read-consultation-evidence 857 | + +### 每段查卡的实测字符数(三张盘同一量级) + +| 段 | 字符数(约) | +| --- | ---: | +| `western:*`(最大一段) | 28,400 | +| `upstream_techniques` | 19,700–21,900 | +| `varga:research_dn` | 9,900 | +| `kp_placidus` | 5,785 | +| `yogas` / `varga:D6x` 等常用小段 | 535–857 | + +### 三种情况的总输入字符(同一张盘、同样 k 段) + +公式:第 n 步的 prompt = 62,859 固定部分 + 此前全部工具结果之和。 + +| 情况 | k = 1 | k = 2 | k = 3 | +| --- | ---: | ---: | ---: | +| (i) 现状:每段一步(实测小段 D60 + yogas) | 376,079 | 533,075 | 691,xxx | +| (ii) 一次取多段(同一批小段) | 376,079 | 377,xxx | 378,xxx | +| 下降 | 0% | **29.4%** | **45.3%** | +| (i) 现状:两个大段(28,400 + 21,440) | 376,079 | 609,290 | 842,xxx | +| (ii) 一次取多段(同一批大段) | 376,079 | 424,825 | 473,xxx | +| 下降 | 0% | **30.3%** | **43.8%** | + +结论:**k = 2 时总输入字符下降约 29–30%,超过 10% 的门槛**,按任务书继续做第二步。字段过滤的额外收益按段算:`fields` 只保留几个键时那一段(最大 28.4k 字符)能再省下大部分,但对整轮输入影响约 0.5–4%,所以没有把它当作主要收益。 + +## 第二步:改造了什么 + +| 位置 | 改动 | +| --- | --- | +| `frontend/src/mastra/consultation-tools.ts` `evidenceLookupInputSchema` | `{ section }` 单段写法保留(新增可选 `fields`);新增 `{ sections: [...] }`,2–6 段、`.strict()`。两种写法用 `z.union` 并存 | +| 同文件 `readEvidenceLookupSection` 之后 | 新增 `pickLookupFields`:只按顶层键取值,不改值、不截断数组;不存在的键进 `unknown_fields` | +| 同文件工具本体 | 单段:返回形状与键序不变(`status / section / data / note`,带 `fields` 时多 `unknown_fields`);多段:返回 `{ status, results: [{ section, status, data|reason }], note }`。多段计 1 次调用,仍受每轮 3 次上限;第 4 次与原来一样 `refused`。降级与「计算未缓存」也按单段/多段给对应形状 | +| `ConsultationRuntimeStep` | 新增内部可选字段 `sections` 与 `chars`(每次查卡记读了哪几段、返回多少字符);`publicConsultationRuntimeSteps` 不导出它们,公开回执不变 | +| 工具描述 | 说明「需要几段就一次全列(2–6)」「`fields` 只取这些顶层键,未知键进 `unknown_fields`」,3 次上限照旧 | + +没动:`EVIDENCE_LOOKUP_SECTIONS`/`consultation-evidence-card.ts`、`references/**`、`consult-preset-prompts.ts`、`consultation-methodology.ts`、`mastra/index.ts` 的 instructions、`route.ts`、每轮 3 次上限的数值、序列化区(#170)、`run-jyotish-consultation` 的 execute(#172)、工具表(#173)。 + +`stream-agent-response.ts` 的「答案重启」判定只看工具名与已输出的答案文字,不读返回里的 `section`,因此不需要改;`lookupEvidence` 只是原样挂到 `evidence_lookup` 上,形状变化无影响。 + +## 测试 + +| 测试 | 结果 | +| --- | --- | +| 新增 `frontend/tests/consult-evidence-lookup-batch-20261109.test.ts` | 7 过 / 0 败 | +| `frontend/tests/consult-evidence-lookup-20260927.test.ts` | 15 过 / 0 败 | +| `frontend/tests/consult-same-source-check-1773.test.ts` | 4 过 / 0 败 | +| `frontend/tests/consult-same-source-independence-1773.test.ts` | 4 过 / 0 败 | +| `frontend/tests/consultation-agentic-runtime.test.ts` | 87 过 / 0 败 | +| `tsc --noEmit -p frontend` | 0 错 | +| `npm run lint`(frontend) | 0 error、63 warning(都是既有文件) | + +新测试覆盖任务书要求的每一条: + +1. 单段旧写法返回形状与键序不变(`status / section / data / note`)。 +2. 多段一次调用每段 `data` 与逐段单独取 deepEqual(必需字段 100% 送达、内容不变)。 +3. `fields` 只少键不改值;未知键进 `unknown_fields`;数组不截断(用整段键名回取,结果与整段一致)。 +4. 多段计 1 次;三次之后第 4 次仍 `refused` / `lookup_limit_reached`。 +5. 超过 6 段、只有 1 段、段名不在清单、混写 `section` + `sections`、多余键,都被 schema 拒绝。 +6. 送达核对:三张 golden 盘 × career / marriage / health,`evidence_contract.missing_route_layers` 全部为空。 + +既有断言没有改动,所以没有需要写「原值 / 新值 / 原因」的地方。 + +## 交给 Claude 验收的两项(任务书第 6 条) + +- `scripts/research/consult-evidence-traceability.mts`、`consult-same-source-check.mts` 改前改后对比。 +- 盲测准确度对比(本仓 Windows 侧没有模型凭据,按环境缺口留给你)。 + +测量脚本留在分支里,可复跑:`frontend/scripts/research/consult-lookup-token-probe-171.mts`(每步 prompt 字符数)与 `consult-lookup-section-sizes-171.mts`(每段字符数)。生成物没有提交,避免把一次性数据写进仓库。 diff --git a/frontend/scripts/research/consult-lookup-section-sizes-171.mts b/frontend/scripts/research/consult-lookup-section-sizes-171.mts new file mode 100644 index 00000000..e129ef6c --- /dev/null +++ b/frontend/scripts/research/consult-lookup-section-sizes-171.mts @@ -0,0 +1,57 @@ +/** + * #171 每段查卡的实测字符数(2026-11-09)。每段各开一份工具实例,避开每轮 3 次上限。 + * 运行:frontend > node node_modules/tsx/dist/cli.mjs scripts/research/consult-lookup-section-sizes-171.mts + */ +import { readFileSync, writeFileSync } from "node:fs"; + +import { EVIDENCE_LOOKUP_SECTIONS } from "../../src/lib/consultation-evidence-card.ts"; +import { + CONSULTATION_EVIDENCE_LOOKUP_TOOL_ID, + createConsultationRuntimeState, + createConsultationTools, +} from "../../src/mastra/consultation-tools.ts"; + +type Json = Record; +const golden = JSON.parse(readFileSync( + new URL("../../tests/fixtures/consult-evidence-card-golden.json", import.meta.url), + "utf8", +)) as { charts: Array<{ id: string; workflow: Json }> }; + +const rows: Array<{ chart: string; section: string; chars: number }> = []; + +for (const chart of golden.charts) { + for (const section of EVIDENCE_LOOKUP_SECTIONS) { + const state = createConsultationRuntimeState(); + const created = createConsultationTools({ + userId: "u", sessionId: "s", requestId: `size-${chart.id}-${section}`, + consultationMode: "verified_chart", + serverChart: { + name: "public", + toolInput: { + year: 1955, month: 2, day: 24, hour: 19, minute: 15, city: "San Francisco", + lat: 37.77, lon: -122.42, tz: -8, ayanamsa: "raman", + }, + truth: { birthTimeSource: "reported" }, + } as never, + state, + runWorkflow: async () => structuredClone(chart.workflow) as never, + }); + await created["run-jyotish-consultation"].execute!( + { question: "这一块怎么样", domains: ["career"] } as never, + { writer: { custom: async () => {} } } as never, + ); + const result = await created[CONSULTATION_EVIDENCE_LOOKUP_TOOL_ID].execute!( + { section } as never, + { writer: { custom: async () => {} } } as never, + ) as Json; + rows.push({ + chart: chart.id, + section, + chars: result.status === "ok" ? JSON.stringify(result).length : 0, + }); + } +} + +const out = new URL("../../../docs/research/token_lookup_171_section_sizes.json", import.meta.url); +writeFileSync(out, `${JSON.stringify(rows, null, 2)}\n`, "utf8"); +console.log(JSON.stringify(rows.filter((row) => row.chars > 0).sort((a, b) => b.chars - a.chars).slice(0, 10), null, 2)); diff --git a/frontend/scripts/research/consult-lookup-token-probe-171.mts b/frontend/scripts/research/consult-lookup-token-probe-171.mts new file mode 100644 index 00000000..f50b2f8e --- /dev/null +++ b/frontend/scripts/research/consult-lookup-token-probe-171.mts @@ -0,0 +1,56 @@ +/** + * #171 第二步测量:用真实 agent 链路量「计算工具结果」与「单段查卡结果」各多少字符, + * 再按「每多一步就把此前全部内容再发一遍」估算三种方案的输入总量(2026-11-09)。 + * + * 运行:frontend > node node_modules/tsx/dist/cli.mjs scripts/research/consult-lookup-token-probe-171.mts + */ +import { readFileSync } from "node:fs"; + +import { CONSULTATION_EVIDENCE_LOOKUP_TOOL_ID } from "../../src/mastra/consultation-tools.ts"; + +import { pieces, runNatalAgent, type Turn } from "../../tests/consult-natal-agent-test-support.ts"; + +const golden = JSON.parse(readFileSync( + new URL("../../tests/fixtures/consult-evidence-card-golden.json", import.meta.url), + "utf8", +)) as { charts: Array<{ id: string; workflow: Record }> }; +const workflow = golden.charts.find((chart) => chart.id === "steve_jobs")!.workflow; + +const ANSWER = "## 依据\r\n这几个月先稳住节奏。\r\n"; + +const calcTurn: Turn = { + parts: [{ tool: { name: "run-jyotish-consultation", input: { question: "我事业如何", domains: ["career"] } } }], + finish: "tool-calls", +}; +const lookup = (section: string): Turn => ({ + parts: [{ tool: { name: CONSULTATION_EVIDENCE_LOOKUP_TOOL_ID, input: { section } } }], + finish: "tool-calls", +}); +const answerTurn: Turn = { parts: pieces(ANSWER), finish: "stop" }; + +function toolResultChars(prompt: unknown[] | undefined): Array<{ name: string; chars: number }> { + const out: Array<{ name: string; chars: number }> = []; + for (const message of prompt ?? []) { + const value = message as { role?: string; content?: unknown }; + const text = JSON.stringify(value.content ?? ""); + if (value.role !== "tool") continue; + const name = text.includes(CONSULTATION_EVIDENCE_LOOKUP_TOOL_ID) + ? CONSULTATION_EVIDENCE_LOOKUP_TOOL_ID + : text.includes("run-jyotish-consultation") ? "run-jyotish-consultation" : "other"; + out.push({ name, chars: text.length }); + } + return out; +} + +const result = await runNatalAgent( + [calcTurn, lookup("varga:D60"), lookup("yogas"), answerTurn], + { workflow, theme: "career", question: "我事业如何" }, +); + +const steps = result.prompts.map((prompt, index) => ({ + step: index, + promptChars: JSON.stringify(prompt).length, + toolResults: toolResultChars(prompt), +})); + +console.log(JSON.stringify({ calls: result.calls, steps }, null, 2)); diff --git a/frontend/src/mastra/consultation-tools.ts b/frontend/src/mastra/consultation-tools.ts index fe98c5a3..340e2275 100644 --- a/frontend/src/mastra/consultation-tools.ts +++ b/frontend/src/mastra/consultation-tools.ts @@ -376,6 +376,10 @@ export type ConsultationRuntimeStep = { status: "completed" | "failed"; durationMs?: number; failureCode?: string; + /** #171:本次查卡读了哪几段(内部记录,不进公开 runtime steps)。 */ + sections?: readonly string[]; + /** #171:本次查卡返回的字符数,验收时核对实际送达。 */ + chars?: number; }; export type ConsultationRuntimeState = { @@ -1196,9 +1200,42 @@ export function readEvidenceLookupSection( } } -const evidenceLookupInputSchema = z.object({ - section: z.enum(EVIDENCE_LOOKUP_SECTIONS), -}).strict(); +/** + * #171:只丢没要的顶层键,不改值、不截断数组、不按长度裁。数据不是对象时原样返回, + * 请求的键全部记进 unknown_fields。 + */ +function pickLookupFields( + value: unknown, + fields: readonly string[] | undefined, +): { data: unknown; unknown_fields?: string[] } { + if (!fields || fields.length === 0) return { data: value }; + const source = value && typeof value === "object" && !Array.isArray(value) + ? value as Record + : null; + if (!source) return { data: value, unknown_fields: [...fields] }; + const data: Record = {}; + const unknown: string[] = []; + for (const key of fields) { + if (source[key] === undefined) unknown.push(key); + else data[key] = source[key]; + } + return unknown.length ? { data, unknown_fields: unknown } : { data }; +} + +// #171:导出给测试直接校验 schema(多段上限、段名清单)。 +export const evidenceLookupInputSchema = z.union([ + // 原值:`z.object({ section }).strict()`,一次只能取一段 + // 新值:单段写法保留(含可选 fields),另加一次取 2–6 段的写法;多段计 1 次调用 + // 原因:#171 省 token:查卡一次多段,内容不变 + z.object({ + section: z.enum(EVIDENCE_LOOKUP_SECTIONS), + fields: z.array(z.string().trim().min(1).max(64)).min(1).max(24).optional(), + }).strict(), + z.object({ + sections: z.array(z.enum(EVIDENCE_LOOKUP_SECTIONS)).min(2).max(6), + fields: z.array(z.string().trim().min(1).max(64)).min(1).max(24).optional(), + }).strict(), +]); const bookSearchInputSchema = z.object({ query: z.string().trim().min(2).max(200), @@ -1613,12 +1650,22 @@ export function createConsultationTools(ctx: ConsultationAgentContext) { }); const lookupTool = createTool({ id: CONSULTATION_EVIDENCE_LOOKUP_TOOL_ID, - description: `Read one section of this request's finished chart calculation that is not on the evidence card, for example a varga the card does not carry (varga:D60), a Western layer (western:solar_return), the yoga details, Chara Dasha, Karakamsha, Argala, the dispositor chain, the D1/D9/D10 linkage, the Moon transit, or KP (parameter_sensitive: Placidus cusps and sub-lord boundaries; it may support a conditional conclusion, and a difference from the whole-sign reading is written both ways). It never calculates: it returns what run-jyotish-consultation already computed, or status unavailable. At most ${MAX_EVIDENCE_LOOKUPS_PER_TURN} calls per turn; a fourth call is refused. Call it before writing any answer text, and only when the question needs that section; the answer contract and card come from run-jyotish-consultation.`, + description: `Read sections of this request's finished chart calculation that are not on the evidence card, for example a varga the card does not carry (varga:D60), a Western layer (western:solar_return), the yoga details, Chara Dasha, Karakamsha, Argala, the dispositor chain, the D1/D9/D10 linkage, the Moon transit, or KP (parameter_sensitive: Placidus cusps and sub-lord boundaries; it may support a conditional conclusion, and a difference from the whole-sign reading is written both ways). It never calculates: it returns what run-jyotish-consultation already computed, or status unavailable. List every section the question needs in one \`sections\` call (2-6 sections): one call returns all of them and still counts as a single call, so a second step is not spent re-sending the calculation. Pass \`fields\` to keep only those top-level keys of a section; a key that does not exist comes back in \`unknown_fields\` instead of failing. At most ${MAX_EVIDENCE_LOOKUPS_PER_TURN} calls per turn; a fourth call is refused. Call it before writing any answer text; the answer contract and card come from run-jyotish-consultation.`, inputSchema: evidenceLookupInputSchema, execute: async (input, context) => { const startedAt = Date.now(); + // 原值:一次只能取一段,每段各占一步 + // 新值:可一次取多段(去重后 1–6 段),多段仍计 1 次调用;单段写法与返回形状不变 + // 原因:#171 省 token:查卡一次多段,内容不变 + const single = "section" in input; + const sections: EvidenceLookupSection[] = single ? [input.section] : [...new Set(input.sections)]; + const lookupNote = "If you had already started writing the answer, continue from where you stopped; never repeat text already written."; + const refused = (status: "refused" | "unavailable", reason: string) => ( + single + ? { status, reason, section: sections[0]!, note: lookupNote } + : { status, reason, sections, note: lookupNote } + ); ctx.state.evidenceLookupCallCount += 1; - const note = "If you had already started writing the answer, continue from where you stopped; never repeat text already written."; if (ctx.state.evidenceLookupCallCount > MAX_EVIDENCE_LOOKUPS_PER_TURN) { if (ctx.state.evidenceLookupCallCount === MAX_EVIDENCE_LOOKUPS_PER_TURN + 1) { appendConsultationRuntimeStep(ctx.state, { @@ -1629,7 +1676,7 @@ export function createConsultationTools(ctx: ConsultationAgentContext) { failureCode: "lookup_limit_reached", }); } - return { status: "refused" as const, reason: "lookup_limit_reached", section: input.section, note }; + return refused("refused", "lookup_limit_reached"); } if (!lookupCache) { appendConsultationRuntimeStep(ctx.state, { @@ -1639,24 +1686,59 @@ export function createConsultationTools(ctx: ConsultationAgentContext) { durationMs: Math.max(0, Date.now() - startedAt), failureCode: "calculation_not_cached", }); - return { status: "unavailable" as const, reason: "calculation_not_in_request_cache", section: input.section, note }; + return refused("unavailable", "calculation_not_in_request_cache"); } + const cache = lookupCache; + const labelSection = sections.length > 1 ? `${sections[0]!} (+${sections.length - 1})` : sections[0]!; await (context as { writer?: { custom?: (value: unknown) => unknown } } | undefined)?.writer?.custom?.({ type: "data-jyotish-activity", - data: { phase: "answer-composition", label: evidenceLookupActivityLabel(input.section) }, + data: { phase: "answer-composition", label: evidenceLookupActivityLabel(labelSection) }, }); - const data = readEvidenceLookupSection(lookupCache, input.section); - const found = data !== undefined && data !== null; + if (single) { + // 单段写法(带不带 fields 都一样)保持旧的返回形状,不带 fields 时逐字节相同。 + const raw = readEvidenceLookupSection(cache, sections[0]!); + const found = raw !== undefined && raw !== null; + const picked = pickLookupFields(raw, input.fields); + const data = picked.data; + appendConsultationRuntimeStep(ctx.state, { + kind: "tool", + name: CONSULTATION_EVIDENCE_LOOKUP_TOOL_ID, + status: found ? "completed" : "failed", + durationMs: Math.max(0, Date.now() - startedAt), + sections, + chars: JSON.stringify(data ?? null).length, + ...(found ? {} : { failureCode: "section_not_computed" }), + }); + if (!found) { + return { status: "unavailable" as const, reason: "section_not_computed", section: sections[0]!, note: lookupNote }; + } + return picked.unknown_fields + ? { status: "ok" as const, section: sections[0]!, data, unknown_fields: picked.unknown_fields, note: lookupNote } + : { status: "ok" as const, section: sections[0]!, data, note: lookupNote }; + } + const results = sections.map((section) => { + const raw = readEvidenceLookupSection(cache, section); + if (raw === undefined || raw === null) { + return { section, status: "unavailable" as const, reason: "section_not_computed" }; + } + const picked = pickLookupFields(raw, input.fields); + return picked.unknown_fields + ? { section, status: "ok" as const, data: picked.data, unknown_fields: picked.unknown_fields } + : { section, status: "ok" as const, data: picked.data }; + }); + const delivered = results.filter((row) => row.status === "ok").length; appendConsultationRuntimeStep(ctx.state, { kind: "tool", name: CONSULTATION_EVIDENCE_LOOKUP_TOOL_ID, - status: found ? "completed" : "failed", + status: delivered > 0 ? "completed" : "failed", durationMs: Math.max(0, Date.now() - startedAt), - ...(found ? {} : { failureCode: "section_not_computed" }), + sections, + chars: JSON.stringify(results).length, + ...(delivered > 0 ? {} : { failureCode: "section_not_computed" }), }); - return found - ? { status: "ok" as const, section: input.section, data, note } - : { status: "unavailable" as const, reason: "section_not_computed", section: input.section, note }; + return delivered > 0 + ? { status: "ok" as const, results, note: lookupNote } + : { status: "unavailable" as const, reason: "section_not_computed", results, note: lookupNote }; }, }); let bookSearchCount = 0; diff --git a/frontend/tests/consult-evidence-lookup-batch-20261109.test.ts b/frontend/tests/consult-evidence-lookup-batch-20261109.test.ts new file mode 100644 index 00000000..85ba2708 --- /dev/null +++ b/frontend/tests/consult-evidence-lookup-batch-20261109.test.ts @@ -0,0 +1,170 @@ +/** + * #171 省 token:查卡一次可取多段。 + * + * 原值:一次只能取一段,每多一段就多一步,整份计算(约 9.3 万字符)随每一步重发。 + * 新值:一次可列 2–6 段,仍计 1 次调用;每段 data 与单独取时逐字节相同; + * 可选 fields 只丢没要的顶层键,不截断数据。 + * 原因:#171 省 token:查卡一次多段,内容不变 + */ +import assert from "node:assert/strict"; +import { readFileSync } from "node:fs"; +import test from "node:test"; + +import { + CONSULTATION_EVIDENCE_LOOKUP_TOOL_ID, + MAX_EVIDENCE_LOOKUPS_PER_TURN, + createConsultationRuntimeState, + createConsultationTools, + evidenceLookupInputSchema, +} from "../src/mastra/consultation-tools.ts"; + +type Json = Record; +const golden = JSON.parse(readFileSync( + new URL("./fixtures/consult-evidence-card-golden.json", import.meta.url), + "utf8", +)) as { charts: Array<{ id: string; workflow: Json }> }; +const workflow = golden.charts.find((chart) => chart.id === "steve_jobs")!.workflow; + +const SECTIONS = ["yogas", "varga:D60", "western:natal", "shadbala"] as const; + +/** 每张盘一次计算;查卡调用有每轮 3 次的上限,所以每次取样都开一份新的工具实例。 */ +async function tools(domain = "career") { + const state = createConsultationRuntimeState(); + const created = createConsultationTools({ + userId: "u", + sessionId: "s", + requestId: "lookup-batch-171", + consultationMode: "verified_chart", + serverChart: { + name: "public", + toolInput: { + year: 1955, month: 2, day: 24, hour: 19, minute: 15, city: "San Francisco", + lat: 37.77, lon: -122.42, tz: -8, ayanamsa: "raman", + }, + truth: { birthTimeSource: "reported" }, + } as never, + state, + runWorkflow: async () => structuredClone(workflow) as never, + }); + await created["run-jyotish-consultation"].execute!( + { question: "我事业如何", domains: [domain] } as never, + { writer: { custom: async () => {} } } as never, + ); + const lookup = (input: Json) => created[CONSULTATION_EVIDENCE_LOOKUP_TOOL_ID].execute!( + input as never, + { writer: { custom: async () => {} } } as never, + ) as Promise; + const view = async () => created["run-jyotish-consultation"].execute!( + { question: "我事业如何", domains: [domain] } as never, + { writer: { custom: async () => {} } } as never, + ) as Promise; + return { state, lookup, view }; +} + +async function singleSection(section: string) { + const { lookup } = await tools(); + return lookup({ section }); +} + +test("the single-section shape is unchanged", async () => { + const result = await singleSection("yogas"); + // 旧的返回形状与键序不变,仍是 {status, section, data, note}。 + assert.deepEqual(Object.keys(result), ["status", "section", "data", "note"]); + assert.equal(result.status, "ok"); + assert.equal(result.section, "yogas"); + assert.ok(result.note); + assert.ok(result.data !== undefined); +}); + +test("one batch call returns every requested section exactly as single calls do", async () => { + const perSection: Record = {}; + for (const section of SECTIONS) perSection[section] = (await singleSection(section)).data; + + const { state, lookup } = await tools(); + const result = await lookup({ sections: [...SECTIONS] }); + const results = result.results as Array; + assert.equal(result.status, "ok"); + assert.equal(results.length, SECTIONS.length); + for (const [index, section] of SECTIONS.entries()) { + assert.equal(results[index]!.section, section); + assert.equal(results[index]!.status, "ok"); + assert.deepEqual(results[index]!.data, perSection[section], section); + } + // 多段只算 1 次调用。 + assert.equal(state.evidenceLookupCallCount, 1); +}); + +test("a batch call and a single call share the same per-turn budget", async () => { + const { state, lookup } = await tools(); + await lookup({ sections: ["yogas", "varga:D60"] }); + await lookup({ sections: ["western:natal", "shadbala"] }); + await lookup({ section: "yogas" }); + const refused = await lookup({ section: "yogas" }); + assert.equal(state.evidenceLookupCallCount, 4); + assert.equal(refused.status, "refused"); + assert.equal(refused.reason, "lookup_limit_reached"); + assert.equal(MAX_EVIDENCE_LOOKUPS_PER_TURN, 3); +}); + +test("fields only drop unrequested keys and report unknown ones", async () => { + const whole = (await singleSection("shadbala")).data as Json; + const { lookup } = await tools(); + const filtered = await lookup({ section: "shadbala", fields: ["summary", "no_such_field"] }); + const data = filtered.data as Json; + assert.deepEqual(Object.keys(data), ["summary"]); + assert.deepEqual(data.summary, whole.summary); + assert.deepEqual(filtered.unknown_fields, ["no_such_field"]); + // 未知键不报错,整次调用仍然成功,返回形状与单段一致。 + assert.equal(filtered.status, "ok"); + assert.deepEqual(Object.keys(filtered), ["status", "section", "data", "unknown_fields", "note"]); +}); + +test("fields never shorten an array or change a value", async () => { + const whole = (await singleSection("yogas")).data; + const { lookup } = await tools(); + const filtered = await lookup({ section: "yogas", fields: Object.keys(whole as Json) }); + assert.deepEqual(filtered.data, whole); + assert.equal(filtered.unknown_fields, undefined); +}); + +test("more than six sections or an unknown section never reaches the tool", () => { + assert.equal(evidenceLookupInputSchema.safeParse({ + sections: ["yogas", "yogas", "varga:D60", "western:natal", "shadbala", "transits", "argala"], + }).success, false); + assert.equal(evidenceLookupInputSchema.safeParse({ section: "varga:D99" }).success, false); + assert.equal(evidenceLookupInputSchema.safeParse({ sections: ["yogas"] }).success, false); + assert.equal(evidenceLookupInputSchema.safeParse({ section: "yogas", extra: true }).success, false); + assert.equal(evidenceLookupInputSchema.safeParse({ section: "yogas", sections: ["yogas", "transits"] }).success, false); +}); + +test("every domain still gets its needed layers, on the card or through the lookup", async () => { + for (const chart of golden.charts) { + for (const domain of ["career", "marriage", "health"] as const) { + const state = createConsultationRuntimeState(); + const created = createConsultationTools({ + userId: "u", + sessionId: "s", + requestId: `coverage-${chart.id}-${domain}`, + consultationMode: "verified_chart", + serverChart: { + name: "public", + toolInput: { + year: 1955, month: 2, day: 24, hour: 19, minute: 15, city: "San Francisco", + lat: 37.77, lon: -122.42, tz: -8, ayanamsa: "raman", + }, + truth: { birthTimeSource: "reported" }, + } as never, + state, + runWorkflow: async () => structuredClone(chart.workflow) as never, + }); + const view = await created["run-jyotish-consultation"].execute!( + { question: "这一块怎么样", domains: [domain] } as never, + { writer: { custom: async () => {} } } as never, + ) as Json; + const contract = view.evidence_contract as Json; + // 多段查卡只是少发几遍,不改变送达:必需层要么在卡上,要么能经查卡取到。 + const missing = contract.missing_route_layers as unknown; + assert.deepEqual(missing, [], `${chart.id}/${domain}: ${JSON.stringify(missing)}`); + } + } +});