fix(consult): state the domain vocabulary in the schema, and stop delivering the model's narration as the answer
Independent Staging Quality Gate / validate (push) Has been cancelled
Independent Staging Quality Gate / publish (push) Has been cancelled

A staging run answered "which year?" with nothing but the model explaining that
its own tool calls had failed. Two causes, one upstream of the other.

The domains parameter accepted any string, so it stated no vocabulary at all
while the skill's methodology names strict-workflow checklists the tool has
never accepted. The model followed the skill, the schema took it, and the call
died in the registry two steps later. Enumerating the accepted values puts the
vocabulary where the model reads it. Aliases stay in the enum: they are a
promise the instructions make and a test pins.

Mastra reports an input-schema rejection by resolving with a validation
envelope rather than throwing, so enumerating alone would have turned those
rejections into tool.completed for calls that never ran. The stream now reads
that envelope for what it is.

Text written before the runtime contract is ready was held rather than dropped,
so a later successful call released the model's narration of its own failures as
the entire visible answer. Dropping it means a run that cannot answer says so.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Jesse_Chen
2026-08-18 17:43:22 +08:00
parent 188cff99ec
commit 1bdff42cbc
6 changed files with 245 additions and 26 deletions
+37 -1
View File
@@ -4026,9 +4026,10 @@
- 验证:`frontend/tests/consultation-agentic-runtime.test.ts` 40 项通过(新增 3 项)。新增:(1) 只喂 `tool-call` + `tool-error` 两个 chunk(模拟 `execute` 从未运行),断言 state 里出现带 `tool_call_rejected` 的失败 tool 步、`stepBudget.used` 变为 2、公开回执的 steps 状态序列为 `["completed","failed"]`,且序列化后不含该内部码;(2) 工具已记录过的失败不得被流重复记一遍,且原有的 `workflow_queue_full` 不被覆盖;(3) `consultationToolFailureCode()` 对工作流错误、计划错误、普通 Error 与非 Error 值逐个断言有码。已验证第 (1) 条在修复前失败(`fail 1`),修复后 40 全通过。非数据库前端套件 1675/1675 通过,`tsc --noEmit` 与改动文件 `eslint` 清洁。未做的验证:**没有在 staging 上复现一次 schema 拒绝来复看回执**——本次修复后线上是否真的出现带失败步的回执尚未观测,且触发它需要模型再犯一次同样的参数错误,无法主动构造。
- 待跟进:模型当时到底传了什么参数仍然不知道——正因为没被记下来。修复后若再次出现,日志会给出 `tool_call_rejected`,但具体是哪个字段不合法仍不可见;若该情况反复出现,下一步是在服务端日志里补一个「被拒字段名」的封闭枚举(只记字段名,不记值,避免把模型输出当日志内容)。另外 `consultationToolCallCount` 在这条路径上仍然不增(工具体没跑),本轮刻意未动:改它会破掉「never starts a calculation」与「invalid model input does not poison a later valid contract retry」两条现存的刻意断言,而失败已经能从 `steps` 看见,计数本身不再是唯一线索。
- 防复发:把「某件事发生过」的记录放在能观测到它的那一层。工具无法记录一次它从未收到的调用,因此这类记录必须由流(或同等的外层观察点)兜底;判断「哪一层能看见」的可靠证据是活动事件的有无,而不是错误码。失败记录的原因字段不得可选省略:`...(code ? {code} : {})` 这种写法会让分类器覆盖不到的错误静默变成无原因记录,而那恰好是最需要原因的一类。公开与内部两个通道要分别满足:客户端需要知道「失败过」,运维需要知道「为什么」,把后者塞进前者会泄漏后端内情,把前者省掉会让回执与客户端亲眼所见互相矛盾。
- 相关记录:BUG-268(同为「诊断量算出来就丢」,本条是「诊断量压根没被创建」)、BUG-258(同为失败时回执信息不足)、BUG-255(同为模型参数被拒白扔步数,当时的修法是把互斥字段从 schema 里删掉)
- 相关记录:BUG-268(同为「诊断量算出来就丢」,本条是「诊断量压根没被创建」)、BUG-258(同为失败时回执信息不足)、BUG-255(同为模型参数被拒白扔步数,当时的修法是把互斥字段从 schema 里删掉)、BUG-278(补正本条对触发路径的归因,并接手枚举化之后新的拒绝路径)
- 复发自:无
- 修复版本:`b5bcbaed`staging
- 生产验证(2026-08-18 补记):staging run `a5f4409e``2605f34a`)出现连续两次 `tool.failed`,回执 `steps` 里如实带上两条 `status: "failed"`39ms / 56ms)、`stepBudget.used: 4`。这正是本条修复前会丢掉的那两条记录,「无法主动构造」的验证点由手测撞上。同时补正本条根因里的一处归因:实际观测到的触发路径不是 `inputSchema` 拒绝,而是 `execute` 已进入、但 `canonicalDomainPlan()` 在任何状态写入之前就抛(领域不在注册表里),因此工具体同样没能记录任何东西。「失败发生在工具记录任何东西之前」这个机制是对的,落到 `inputSchema` 这一层的具体归因是错的——真正的 `inputSchema` 拒绝路径见 BUG-278Mastra 对它 resolve 而不抛,压根不会走到 `tool-error` 分支。
## BUG-272 | 首页 hero 第三行重复问候被删除,真实性边界随之迁移;onboarding 契约去掉无渲染点的欢迎语并扩展到全部十个主题
@@ -4113,3 +4114,38 @@
- 相关记录:BUG-249(草稿持久化清空已发送问题,本条保留该行为)、咨询流恢复迁移 `20260808030000_consultation_stream_recovery.sql`(断线后续跑只覆盖已占位的 reserved 请求)
- 复发自:无
- 修复版本:已推入 staging,待部署
## BUG-277 | 模型写给自己的过程叙述被当成回答交付给用户:契约未绿的文本没有丢弃,而是攒起来晚点一次性放出
- 状态:resolved(本地修复,待提交与发布)
- 首次发现:2026-08-18
- 最近更新:2026-08-18
- 影响面:`frontend/src/lib/stream-agent-response.ts``outputText`。所有 `/api/consult` 运行都经过它,但只有「契约变绿之前模型输出过文本」的运行会显形。
- 用户现象:staging 手测 run `a5f4409e``2605f34a`)最终回答里没有任何占星结论,整段是模型在向用户解释自己的工具调用出错以及打算怎么改参数(大意:域名单有误、我改用另一组域、两次都不被接受、我改为不指定域),然后就结束了。用户问的问题(「大概哪一年」)没有得到任何回答。三句自述装在**同一个** `answer.delta` 里——这就是一次性 flush 的指纹。回执显示计算其实成功了:4 步、两次失败 tool 之后一次成功 tool。
- 触发条件:模型在契约未绿(skill 未加载完或工具尚无一次成功)之前输出过文本,且此后契约变绿。失败重试的运行几乎必然满足。
- 根因:守卫写成了「先累加再判断」。`held += text` 无条件执行,紧随其后的 `if (!held || !contractReady(options)) return` 只是「暂不发送」。设计意图(原测试名写着 holds answer text until the contract completes)是「计算未验证前不要把文本流给用户」,但实现选的是**攒着晚点发**而不是**丢掉**。契约一变绿,下一次 `outputText` 就把攒下的内容整段 flush`emitted` 随之置真,`ensureFinalResponseText()` 的空回答兜底因此也不会触发,用户既拿不到回答也拿不到「本次没能生成回答」的提示。要命的地方在于这段文本的内容与运行是否顺利强相关:顺利的运行里它是无害的开场语,失败重试的运行里它正好是模型在复述后端错误——而失败重试恰恰是它唯一会变长的场合。换句话说,这个缓冲在最需要它闭嘴的时候话最多。
- 修复:契约未绿时直接丢弃,不进 `held``attemptOutput` 仍然无条件累加,所以「模型说过话但始终没给出回答」(`runtime_contract_incomplete`)与「模型全程沉默」(`empty_answer`)的诊断区分不受影响。随之 `first.held` 恒为空串,已连同 `consumeAttempt()` 的该返回字段一起删除:留一个恒假的判断在最终结算处,会让后来人以为 `held` 仍然跨越契约边界携带意义。
- 验证:`frontend/tests/consultation-agentic-runtime.test.ts` 43 项通过。原有的 holds-answer-text 断言按新意图改写并改名(契约前文本不得以任何形式到达客户端、契约后文本原样送出,并断言序列化后的事件流里搜不到那段自述);新增一条按 run `a5f4409e` 形状构造的回归:模型只有自述、契约变绿后再无文本,断言用户拿到的是空回答兜底文案而**不是**那段自述。已确认改写后的断言在修复前失败。`tsc --noEmit` 与改动文件 `eslint` 清洁;与 consult/流相关的 8 个套件 65 项全通过(`tests/rectification-v9-database.test.ts` 在本机因缺少数据库预先就失败,与本改动无关,已用 stash 对照确认)。
- 待跟进:丢弃是一刀切的,合法的开场寒暄也会被丢。目前判断可接受(寒暄没有信息量,回答本体一定在工具结果之后产生),但如果将来要有意展示模型的推理过程,必须走**独立事件类型**而不是放宽这里——本条正是「把模型的中间输出混进 answer 通道」的后果。未做的验证:没有在 staging 上复现一次失败重试来复看回答。
- 防复发:不要用「暂存」实现「不该展示」。缓冲意味着数据仍在管道里,只是等一个条件;一旦那个条件与内容的性质无关(这里是「计算是否成功」与「这段文本是不是回答」无关),它迟早会在错误的时刻把错误的内容放出来。真正的判据是**内容属于哪个通道**,而不是**时机到没到**。同一个通道不得同时承载「给用户的回答」和「模型的过程自述」:一旦混流,任何「先攒着」的实现都会在失败路径上把两者对调。
- 相关记录:BUG-278(同一次 staging 运行暴露,且是本条的上游:域词汇不对导致失败重试,失败重试才让自述变长)、BUG-214(同为契约门与可见输出之间的耦合)
- 复发自:无
- 修复版本:待提交
## BUG-278 | 领域词汇在模型能看到的地方从未被声明:schema 收任意字符串,skill 又教了一套不存在的名字,模型照 skill 传参白扔工具调用
- 状态:resolved(本地修复,待提交与发布)
- 首次发现:2026-08-18
- 最近更新:2026-08-18
- 影响面:`frontend/src/mastra/consultation-tools.ts``consultationToolInputSchema.domains``frontend/src/lib/consultation-domain-registry.ts`、Agent 指令,以及 skill 侧的 `SKILL.md` 第 223 行与 `references/strict-workflow-router.md`
- 用户现象:staging 手测 run `a5f4409e``2605f34a`)连续两次 `tool.failed`,模型在回答里自述 `"event-timing-strict" 不在服务端支持列表`。同一批另一次成功运行的回答开头,模型自称「按 wealth-timing-strict + career 两域来算」——把 skill 的检查单标签当成服务端域名在向用户复述。两次失败各白吃一步预算,并间接造成 BUG-277(自述变长后被当成回答交付)。
- 触发条件:模型需要自己挑领域,且它遵循 skill 的方法文档来命名。
- 根因:合同有两份,没有一份权威——与 BUG-267、BUG-270 同一片土壤,只是这次两份分别是「不说话的」和「说错话的」。其一,`domains` 声明为 `z.string().trim().min(1).max(64)`,于是模型收到的 JSON schema 里 `items` 只是一个 `type: "string"`,**完全不陈述合法词汇**;真正的白名单藏在 `execute` 背后的 `validateConsultationDomainPlan()` 里,只有调用失败之后模型才能知道它的存在。其二,模型手上的 SKILL.md 第 223 行明确要求「先判断问题类型,再自动选择 `career-timing-strict` / `relationship-timing-strict` / `wealth-timing-strict` / `event-timing-strict` / `event-verification-strict`」,被它引用的 `references/strict-workflow-router.md` 里这套名字出现 10 次,而 `skill.referenceReads` 证明模型确实在读这些文件。这些名字在方法学里是**技法检查单**的标签,不是任何工具参数,但没有任何一处告诉模型这个区别。模型于是老实照方法文档选名,schema 照收不误,一路走到注册表查表才被拒。
- 修复:把 `domains` 的元素改为枚举。枚举取「规范 id + 全部既有别名」共 37 个值(新增 `consultationDomainPlanValues` / `consultationDomainPlanValueSchema`),而不是只取 10 个规范 id:模型可以传别名、由服务端归一,这是 Agent 指令里的明文承诺,也有既有测试钉住(`["career","finance","home"]` → career/wealth/migration),枚举化不应顺手收紧它。已实测该枚举确实进入模型收到的 JSON schema`items.enum` 37 项),否则整个改动等于没做。同时在 Agent 指令里点明检查单标签不是域 id:指令层不受 skill 包哈希约束,且按本项目设计其优先级高于 skill 内容。**刻意未改 SKILL.md**:根 `SKILL.md``skills/jyotish-vedic-astrology/versions/6.9.14/SKILL.md` 之间有逐字节相等校验(`frontend/src/mastra/index.ts`),包目录整树的 sha256 钉在 `skills/skill-package-registry.json`,改内容的正确做法是升版本;但 `6.9.14` 同时是 `pyproject.toml``CHANGELOG.md``README.md` 与 SKILL.md 自身版本横幅的发布身份,为一行澄清升版本会造出「skill 6.9.15 / Python 包 6.9.14」这类新的不一致,而就地改钉住的版本会让同一个版本号在历史上指两份内容。枚举本身已经是权威约束,文档澄清留待下一次真实的 skill 发布搭车。
- 附带发现(本轮实测,此前无记录):Mastra 的 `createTool` **会**校验 `inputSchema`,但校验失败时**不抛异常,而是正常 resolve 一个 `{ error: true, message, validationErrors }` 信封**(信封的 message 里会列出枚举值,这也是模型自我纠正的唯一线索)。这直接改变了枚举化的风险面:被拒调用从「进入 `execute` 后抛错」变成「resolve 一个值」,流层原本会照常把它当 `tool-result` 发出 `tool.completed`——把一次从未执行的计算报告成完成,比枚举化之前更误导。因此在 `tool-result` 分支按**信封的结构**`error === true` 且带 `validationErrors` 对象)识别拒绝,而不是匹配上游的英文文案,改发 `tool.failed` 并沿用 BUG-271 的补记逻辑记一条 `tool_call_rejected` 步。同时更正 BUG-271 对触发路径的归因,详见该条的生产验证补记。
- 验证:`frontend/tests/consultation-agentic-runtime.test.ts` 43 项通过(本条新增 1 项、改写 3 项)。新增「schema 陈述它接受的词汇」:断言 `event-timing-strict` / `wealth-timing-strict` 被拒、规范 id 与别名(`finance``感情`)被收,并断言枚举值可从 schema 结构上读出(`shape.domains.unwrap().element.options`)——这一条专门防「换成 `z.preprocess` 之类的包装后,校验仍然通过但 JSON schema 里的枚举没了」这种静默失效。改写:原先钉住「注册表抛错」的两条断言改为钉住信封(机制变了,意图不变:不得启动计算、不得污染请求级缓存),并额外断言信封 message 里列出了合法 id。另新增「被 schema 拒绝的调用不得报告为 completed」,断言事件流里没有 `tool.completed`、有 `tool.failed`、state 里落下 `tool_call_rejected` 步,且内部码不泄漏到客户端。`tsc --noEmit`、改动文件 `eslint``tests/test_consultation_consumer_context.py` 17 项均清洁。未做的验证:没有在 staging 上观测模型见到枚举后是否不再传检查单标签。
- 待跟进:SKILL.md 第 223 行与 `references/strict-workflow-router.md` 里的 10 处仍在教这套词汇,须随下一次 skill 版本发布一并澄清。另外 37 项枚举里含中文别名,若日后确认模型只用规范 id,可考虑收窄到 10 项以减少 schema 噪声——但那是收紧合同,需要单独一轮并同步 Agent 指令里的别名承诺。
- 防复发:模型必须遵守的词汇,要写在模型读得到的地方,也就是 schema,而不是只写在校验它的代码里。`z.string()` 这类「结构上合法、语义上无约束」的字段等于把合同留在服务端自言自语:模型只能靠试错发现规则,而每次试错都是一次真实的调用预算。给模型的工具 schema 里出现自由字符串时,先问一句「合法值是有限集吗」——如果是,就必须枚举出来。枚举化之后要**实测生成的 JSON schema**,不能假设 zod 的包装层会把枚举透传。还要顺带检查「被拒」这条路径在框架里是抛还是返回:抛与返回会落到流层完全不同的分支上,误判会把失败报告成成功。
- 相关记录:BUG-271(本条更正了它对触发路径的归因,并接手枚举化之后新出现的拒绝路径)、BUG-277(本条是它的上游)、BUG-267 与 BUG-270(同为「同一合同两份声明、没有一份权威」)、BUG-255(同为模型参数被 schema 拒白扔步数)
- 复发自:无
- 修复版本:待提交
@@ -49,6 +49,22 @@ for (const definition of consultationDomainRegistry) {
for (const alias of definition.aliases) aliasToDomain.set(alias, definition.id);
}
/**
* Every value a domain plan accepts, canonical ids first.
*
* This exists so the model-facing tool schema can enumerate the vocabulary
* instead of accepting any string. As a free-form string the schema stated no
* vocabulary at all, so a name the skill's methodology happened to use passed
* validation and only failed deep inside the call. Enumerating trades the old
* tolerance for surrounding whitespace and casing for a stated contract: a
* mismatch is now refused with the accepted values named.
*/
export const consultationDomainPlanValues = [
...new Set<string>([...consultationDomainIds, ...aliasToDomain.keys()]),
] as [string, ...string[]];
export const consultationDomainPlanValueSchema = z.enum(consultationDomainPlanValues);
export function normalizeConsultationDomain(value: unknown): ConsultationDomain | null {
if (typeof value !== "string") return null;
return aliasToDomain.get(value.trim().toLowerCase()) ?? null;
+39 -4
View File
@@ -85,6 +85,24 @@ function safeToolError(error: unknown) {
return "calculation_failed" as const;
}
/**
* Whether a tool result is really an input rejection Mastra resolved with.
*
* Mastra validates arguments against the tool's inputSchema before `execute`
* and reports a mismatch by *resolving* with an error envelope rather than
* throwing. Passing that through as `tool.completed` would tell the client a
* calculation finished while the tool body never ran. Matched on the envelope's
* shape, not its English message, so an upstream wording change cannot silently
* turn a rejection back into a success.
*/
function isToolInputRejection(result: unknown) {
if (!result || typeof result !== "object") return false;
const envelope = result as { error?: unknown; validationErrors?: unknown };
return envelope.error === true
&& typeof envelope.validationErrors === "object"
&& envelope.validationErrors !== null;
}
/**
* Record a tool failure the tool itself could not record.
*
@@ -147,9 +165,15 @@ function mapChunk(
return [{ type: "skill.completed", name: "jyotish-vedic-astrology" }];
}
if (toolName === "run-jyotish-consultation") {
const durationMs = Math.max(0, Date.now() - (startedAt.get(callId) ?? Date.now()));
if (isToolInputRejection(payload.result)) {
toolErrors.seen += 1;
if (options.state) recordUnrecordedToolFailure(options.state, toolErrors.seen, durationMs);
return [{ type: "tool.failed", callId, tool: "run-jyotish-consultation", code: "calculation_failed" }];
}
return [{
type: "tool.completed", callId, tool: "run-jyotish-consultation", status: options.toolStatus(),
durationMs: Math.max(0, Date.now() - (startedAt.get(callId) ?? Date.now())),
durationMs,
}];
}
}
@@ -243,8 +267,16 @@ export function streamAgentResponse(options: StreamAgentResponseOptions) {
let composingSent = false;
const outputText = async (text: string) => {
attemptOutput += text;
// Text the model writes before the contract is ready is not the answer: it
// is the model narrating its own in-progress or failed tool calls. Holding
// it meant a later successful call released that narration as the entire
// visible answer, so a run where the model recovered read as a run where it
// explained itself instead of answering. Drop it. attemptOutput still
// records that the model spoke, which is what separates an incomplete
// contract from silence below.
if (!contractReady(options)) return;
held += text;
if (!held || !contractReady(options)) return;
if (!held) return;
if (!composingSent) {
composingSent = true;
send(controller, { type: "activity", phase: "answer-composition", label: "正在组织回答" });
@@ -274,7 +306,7 @@ export function streamAgentResponse(options: StreamAgentResponseOptions) {
}
}
await outputText(visible.finish(""));
return { held, attemptOutput };
return { attemptOutput };
}
const body = new ReadableStream<Uint8Array>({
@@ -301,7 +333,10 @@ export function streamAgentResponse(options: StreamAgentResponseOptions) {
emitted = true;
}
if (!/\S/.test(fullOutput)) {
if (/\S/.test(first.held) || /\S/.test(first.attemptOutput)) throw new Error("runtime_contract_incomplete");
// attemptOutput records everything the model wrote, including the
// discarded pre-contract narration, so a model that spoke but never
// produced an answer is still distinguished from one that stayed silent.
if (/\S/.test(first.attemptOutput)) throw new Error("runtime_contract_incomplete");
throw new Error("empty_answer");
}
settling = true;
+10 -2
View File
@@ -2,6 +2,7 @@ import { isDeepStrictEqual } from "node:util";
import { createTool } from "@mastra/core/tools";
import { z } from "zod";
import {
consultationDomainPlanValueSchema,
validateConsultationDomainPlan,
type ConsultationDomain,
} from "../lib/consultation-domain-registry.ts";
@@ -51,7 +52,14 @@ export const MAX_CONSULTATION_DOMAINS = Math.max(
// canonicalizes instead of failing outright. The executable cap is enforced
// after canonicalization, where it can degrade and disclose rather than throw.
const MAX_CONSULTATION_DOMAIN_PLAN_VALUES = 6;
const domainPlanValueSchema = z.string().trim().min(1).max(64);
// The legal values have to be stated in the schema the model is handed, not only
// enforced in the registry behind execute(). As a free-form string this accepted
// any identifier the skill's methodology happened to name—the strict-workflow
// checklist labels are not domains—so an invented value passed validation and
// died inside execute, spending a step and a tool.failed to learn a vocabulary
// the schema could have listed. Aliases stay accepted, so this enumerates them
// alongside the canonical ids rather than narrowing what a call may say.
const domainPlanValueSchema = consultationDomainPlanValueSchema;
// The model may only express a domain plan one way. A second, mutually
// exclusive field was representable here but rejected at execution, so every
@@ -466,7 +474,7 @@ export function createConsultationTools(ctx: ConsultationAgentContext) {
let calculation: Promise<ReturnType<typeof toModelDomainPlanContext>> | null = null;
const consultationTool = createTool({
id: "run-jyotish-consultation",
description: `Run one server-validated plan of at most ${MAX_CONSULTATION_DOMAINS} allowlisted personal Jyotish consultation domains. Send only question and domains. The single ordered domains array is the only way to select domains: list them in priority order, or omit it entirely to accept the domain the server already selected for this consultation. Domains execute one after another and each costs about ${Math.round(CONSULTATION_DOMAIN_DURATION_MS / 1000)}s of the run's wall clock, so a shorter plan leaves more time to write the answer; if the clock runs short the server executes the domains that fit and returns the rest in omitted_domains. Birth data is server-bound and must never be supplied. The result always carries one top-level answer contract—status, evidence_contract, claim_cards, rectification—which for several domains is the most restrictive merge of the executed ones, with per-domain detail in consultations. One calculation is executed per request and reused, so repeating the call with different parameters cannot change the result.`,
description: `Run one server-validated plan of at most ${MAX_CONSULTATION_DOMAINS} allowlisted personal Jyotish consultation domains. Send only question and domains. The single ordered domains array is the only way to select domains: list them in priority order, or omit it entirely to accept the domain the server already selected for this consultation. Use only the ids enumerated in the schema; workflow or checklist names from the skill's methodology are not domain ids. Domains execute one after another and each costs about ${Math.round(CONSULTATION_DOMAIN_DURATION_MS / 1000)}s of the run's wall clock, so a shorter plan leaves more time to write the answer; if the clock runs short the server executes the domains that fit and returns the rest in omitted_domains. Birth data is server-bound and must never be supplied. The result always carries one top-level answer contract—status, evidence_contract, claim_cards, rectification—which for several domains is the most restrictive merge of the executed ones, with per-domain detail in consultations. One calculation is executed per request and reused, so repeating the call with different parameters cannot change the result.`,
inputSchema: consultationToolInputSchema,
execute: async (input, context) => {
const requestedDomains = canonicalDomainPlan(input, ctx);
+1 -1
View File
@@ -35,7 +35,7 @@ const jyotishInstructions = `You are the guide for a conversational Vedic astrol
Write in concise Simplified Chinese as a natural conversation, not a report or fixed template. Use Markdown only when it improves scanning; tables are allowed only for genuinely comparative information.
For Vedic astrology questions, load the jyotish-vedic-astrology skill before deciding which calculation tool or workflow to use. Follow the skill's method and truth boundaries, but use run-jyotish-consultation for actual chart calculations instead of inventing results.
For questions that require a new chart claim, call run-jyotish-consultation before answering. Simple conversational follow-ups may use the existing context.
Select consultation domains only through the single ordered domains array of run-jyotish-consultation, whether the question covers one domain or several; omit it to accept the domain the server already selected. At most ${MAX_CONSULTATION_DOMAINS} domains may be requested in one run, because they are calculated one after another inside a fixed time budget: list them in priority order and prefer the smallest plan that answers the question, since every extra domain takes time away from writing the answer. The server canonicalizes aliases, rejects unsupported/product domains, executes each accepted domain, and returns the actual domains in the tool context and receipt. A rejected domain plan is final for this run: correct the domains once, and never re-send the same call with extra parameters.
Select consultation domains only through the single ordered domains array of run-jyotish-consultation, whether the question covers one domain or several; omit it to accept the domain the server already selected. At most ${MAX_CONSULTATION_DOMAINS} domains may be requested in one run, because they are calculated one after another inside a fixed time budget: list them in priority order and prefer the smallest plan that answers the question, since every extra domain takes time away from writing the answer. The server canonicalizes aliases, rejects unsupported/product domains, executes each accepted domain, and returns the actual domains in the tool context and receipt. The only legal domain ids are the ones enumerated in that array's schema; the skill's methodology names strict-workflow checklists such as career-timing-strict, and those labels select techniques inside the skill, never domains for this tool. A rejected domain plan is final for this run: correct the domains once, and never re-send the same call with extra parameters.
The tool result always carries one top-level answer contract—status, evidence_contract, claim_cards, rectification—even when several domains ran. For a multi-domain plan that top level is the most restrictive merge of the executed domains, so obey it exactly as written and read consultations only for per-domain detail. Never treat an absent top-level field as permission to answer without a contract.
When omitted_domains is non-empty, the server did not calculate those domains in this run. Name the domains you did cover, say plainly that the remaining ones were not calculated, and never present the answer as covering the whole plan.
Activity, progress, tool status, and execution receipts are server-owned. Never imitate data-jyotish-activity, activity events, tool-started/tool-completed messages, or receipts in the answer text.
@@ -22,10 +22,16 @@ import {
consultationWorkflowFailureCode,
} from "../src/mastra/consultation-workflow.ts";
import { agentExecutionReceiptSchema } from "../src/lib/consultation-agent-events.ts";
import { consultationDomainIds, consultationDomainPlanValues } from "../src/lib/consultation-domain-registry.ts";
import { getJyotishAgent } from "../src/mastra/index.ts";
import { consultationAgentPublicEventSchema, createNdjsonParser } from "../src/lib/consultation-agent-events.ts";
import { createConsultationPlan } from "../src/lib/consultation-plan.ts";
import { collectAgentPublicEvents, ensureFinalResponseText, streamAgentResponse } from "../src/lib/stream-agent-response.ts";
import {
collectAgentPublicEvents,
ENSURE_FINAL_RESPONSE_FALLBACK,
ensureFinalResponseText,
streamAgentResponse,
} from "../src/lib/stream-agent-response.ts";
const serverChart = {
name: "测试",
@@ -452,18 +458,47 @@ test("domain plan rejects unknown and product domains before any workflow runs",
serverChart, state,
runWorkflow: async () => { calls += 1; return workflow(); },
})["run-jyotish-consultation"];
await assert.rejects(
tool.execute!(
modelInput({ question: "测试", domains: ["career", domain] }),
{ observe: { span: async (_n: string, fn: () => Promise<unknown>) => fn(), log() {} } } as never,
),
/unsupported_consultation_domain/,
);
// Refused by the enumerated schema before execute, so Mastra resolves with a
// validation envelope instead of the tool throwing from the registry check.
const rejected = await tool.execute!(
modelInput({ question: "测试", domains: ["career", domain] }),
{ observe: { span: async (_n: string, fn: () => Promise<unknown>) => fn(), log() {} } } as never,
) as { error?: unknown };
assert.equal(rejected.error, true, domain);
assert.equal(calls, 0);
assert.equal(state.consultationToolCompleted, false);
}
});
test("the model-facing schema names the domain vocabulary it accepts", async () => {
const state = createConsultationRuntimeState();
const tool = createConsultationTools({
userId: "u", sessionId: "s", requestId: "r-vocabulary", consultationMode: "verified_chart",
serverChart, state,
runWorkflow: async () => workflow(),
})["run-jyotish-consultation"];
const inputSchema = tool.inputSchema as unknown as {
safeParse: (value: unknown) => { success: boolean };
shape: { domains: { unwrap: () => { element: { options?: readonly string[] } } } };
};
// The skill's methodology names strict-workflow checklists, and while this was
// a free-form string those labels passed validation and died inside the call.
// Enumerating the values is what puts the vocabulary in front of the model.
assert.equal(inputSchema.safeParse({ question: "测试", domains: ["event-timing-strict"] }).success, false);
assert.equal(inputSchema.safeParse({ question: "测试", domains: ["wealth-timing-strict"] }).success, false);
assert.equal(inputSchema.safeParse({ question: "测试", domains: ["career"] }).success, true);
// Aliases stay accepted: enumerating states the vocabulary, it does not narrow it.
assert.equal(inputSchema.safeParse({ question: "测试", domains: ["finance"] }).success, true);
assert.equal(inputSchema.safeParse({ question: "测试", domains: ["感情"] }).success, true);
// Enumerable, so the JSON schema handed to the model carries the values rather
// than an opaque string. A wrapper that hid them would pass the checks above.
const options = inputSchema.shape.domains.unwrap().element.options;
assert.deepEqual(consultationDomainPlanValues, options);
for (const id of consultationDomainIds) assert.ok(options?.includes(id), id);
});
test("domain plan enforces the raw plan upper bound and one input mode", async () => {
let calls = 0;
const state = createConsultationRuntimeState();
@@ -538,12 +573,21 @@ test("invalid model input does not poison a later valid contract retry", async (
})["run-jyotish-consultation"];
const context = { observe: { span: async (_n: string, fn: () => Promise<unknown>) => fn(), log() {} } } as never;
// BUG-205: an input the schema accepts but the domain registry rejects must
// still be refused before the request-scoped calculation cache is written.
await assert.rejects(
tool.execute!(modelInput({ question: "先给出错误参数", domains: ["career", "unknown"] }), context),
/unsupported_consultation_domain/,
);
// BUG-205: a bad domain must be refused before the request-scoped calculation
// cache is written. The refusal now happens at the schema, one layer earlier
// than the registry check it used to reach, because the domain ids are
// enumerated in the schema the model is handed. Mastra reports that refusal by
// resolving with a validation envelope instead of throwing, so this asserts the
// envelope rather than a rejection.
const rejected = await tool.execute!(
modelInput({ question: "先给出错误参数", domains: ["career", "unknown"] }),
context,
) as { error?: unknown; message?: unknown };
assert.equal(rejected.error, true);
// The envelope has to name the legal ids: it is the only correction the model
// gets, and an unnamed vocabulary is what produced the invalid call.
assert.match(String(rejected.message), /'career'/);
assert.match(String(rejected.message), /'timing'/);
assert.equal(calls, 0);
assert.equal(state.consultationToolCallCount, 0);
assert.equal(state.consultationToolSuccessCount, 0);
@@ -862,11 +906,15 @@ function receipt(state: ReturnType<typeof createConsultationRuntimeState>) {
};
}
test("holds answer text until the Skill and server tool contract completes", async () => {
test("text written before the contract completes is dropped, not released later", async () => {
const state = createConsultationRuntimeState();
let completed = 0;
async function* chunks() {
yield { type: "text-delta", payload: { text: "只在合同完成后显示。" } };
// Production shape (run a5f4409e): between rejected calls the model narrates
// its own tool errors. Holding that text meant the eventual success released
// it as the visible answer, so a recovered run read as the model explaining
// itself and never answering the question.
yield { type: "text-delta", payload: { text: "域名单有误,我改为不指定域。" } };
yield { type: "tool-call", payload: { toolCallId: "skill-1", toolName: "skill", args: { name: "jyotish-vedic-astrology" } } };
state.jyotishSkillLoaded = true;
yield { type: "tool-result", payload: { toolCallId: "skill-1", toolName: "skill", result: {} } };
@@ -876,6 +924,7 @@ test("holds answer text until the Skill and server tool contract completes", asy
state.consultationToolCompleted = true;
state.workflowReceipt = { route: "career", status: "ready", preciseTiming: "blocked", missingLayers: [] };
yield { type: "tool-result", payload: { toolCallId: "tool-1", toolName: "run-jyotish-consultation", result: {} } };
yield { type: "text-delta", payload: { text: "这是真正的回答。" } };
}
const response = streamAgentResponse({
runId: "run", requestId: "req", state, stream: chunks(), requireTool: true,
@@ -887,8 +936,83 @@ test("holds answer text until the Skill and server tool contract completes", asy
parser.finish(await response.text());
assert.equal(completed, 1);
assert.equal(events.filter((event) => (event as { type?: string }).type === "run.completed").length, 1);
assert.equal(events.filter((event) => (event as { type?: string }).type === "answer.delta").length, 1);
assert.equal((events.find((event) => (event as { type?: string }).type === "answer.delta") as { text?: string }).text, "只在合同完成后显示。");
const answer = events
.filter((event): event is { type: string; text: string } => (event as { type?: string }).type === "answer.delta")
.map((event) => event.text)
.join("");
assert.equal(answer, "这是真正的回答。");
assert.doesNotMatch(JSON.stringify(events), /域名单有误/);
});
test("a run that only narrated its failures answers with the fallback, not the narration", async () => {
const state = createConsultationRuntimeState();
async function* chunks() {
yield { type: "tool-call", payload: { toolCallId: "skill-1", toolName: "skill", args: { name: "jyotish-vedic-astrology" } } };
state.jyotishSkillLoaded = true;
yield { type: "tool-result", payload: { toolCallId: "skill-1", toolName: "skill", result: {} } };
yield { type: "text-delta", payload: { text: "两次域名单都不被服务端接受,我改为不指定域。" } };
yield { type: "tool-call", payload: { toolCallId: "tool-1", toolName: "run-jyotish-consultation", args: {} } };
state.consultationToolCallCount = 1;
state.consultationToolSuccessCount = 1;
state.consultationToolCompleted = true;
state.workflowReceipt = { route: "general", status: "ready", preciseTiming: "allowed", missingLayers: [] };
yield { type: "tool-result", payload: { toolCallId: "tool-1", toolName: "run-jyotish-consultation", result: {} } };
}
const response = streamAgentResponse({
runId: "run", requestId: "req", state, stream: chunks(), requireTool: true,
toolStatus: () => "ready", receipt: () => receipt(state),
});
const events: unknown[] = [];
const parser = createNdjsonParser((event) => events.push(event));
parser.finish(await response.text());
const answer = events
.filter((event): event is { type: string; text: string } => (event as { type?: string }).type === "answer.delta")
.map((event) => event.text)
.join("");
// Saying nothing usable is honest; presenting the narration as the reading is not.
assert.equal(answer, ENSURE_FINAL_RESPONSE_FALLBACK);
assert.doesNotMatch(JSON.stringify(events), /不被服务端接受/);
});
test("a call Mastra rejected against the input schema is not reported as completed", async () => {
const state = createConsultationRuntimeState({ plannedSteps: 8 });
async function* chunks() {
yield { type: "tool-call", payload: { toolCallId: "skill-1", toolName: "skill", args: { name: "jyotish-vedic-astrology" } } };
state.jyotishSkillLoaded = true;
appendConsultationRuntimeStep(state, { kind: "skill", name: "jyotish-vedic-astrology", status: "completed", durationMs: 300 });
yield { type: "tool-result", payload: { toolCallId: "skill-1", toolName: "skill", result: {} } };
yield { type: "tool-call", payload: { toolCallId: "tool-1", toolName: "run-jyotish-consultation", args: { domains: ["event-timing-strict"] } } };
// Mastra resolves rather than throws when arguments fail inputSchema, so the
// tool body never runs and cannot record anything. Reported as completed this
// would claim a calculation that never happened.
yield {
type: "tool-result",
payload: {
toolCallId: "tool-1",
toolName: "run-jyotish-consultation",
result: { error: true, message: "Tool input validation failed", validationErrors: { errors: [], fields: {} } },
},
};
}
const response = streamAgentResponse({
runId: "run", requestId: "req", state, stream: chunks(), requireTool: true,
toolStatus: () => "ready", receipt: () => ({ ...receipt(state), steps: publicConsultationRuntimeSteps(state) }),
onError: () => {},
});
const events: unknown[] = [];
const parser = createNdjsonParser((event) => events.push(event));
parser.finish(await response.text());
assert.equal(events.some((event) => (event as { type?: string }).type === "tool.completed"), false);
const failed = events.find((event) => (event as { type?: string }).type === "tool.failed") as { code: string };
assert.equal(failed.code, "calculation_failed");
assert.deepEqual(
state.steps.map((step) => `${step.kind}:${step.status}`),
["skill:completed", "tool:failed"],
);
assert.equal(state.steps[1].failureCode, "tool_call_rejected");
// The rejection reason is a server-side diagnostic; the client sees only that a step failed.
assert.doesNotMatch(JSON.stringify(events), /tool_call_rejected|validationErrors/);
});
test("a calculation that succeeds only after failed attempts still satisfies the contract", async () => {