fix(consult): state the domain vocabulary in the schema, and stop delivering the model's narration as the answer
A staging run answered "which year?" with nothing but the model explaining that its own tool calls had failed. Two causes, one upstream of the other. The domains parameter accepted any string, so it stated no vocabulary at all while the skill's methodology names strict-workflow checklists the tool has never accepted. The model followed the skill, the schema took it, and the call died in the registry two steps later. Enumerating the accepted values puts the vocabulary where the model reads it. Aliases stay in the enum: they are a promise the instructions make and a test pins. Mastra reports an input-schema rejection by resolving with a validation envelope rather than throwing, so enumerating alone would have turned those rejections into tool.completed for calls that never ran. The stream now reads that envelope for what it is. Text written before the runtime contract is ready was held rather than dropped, so a later successful call released the model's narration of its own failures as the entire visible answer. Dropping it means a run that cannot answer says so. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -49,6 +49,22 @@ for (const definition of consultationDomainRegistry) {
|
||||
for (const alias of definition.aliases) aliasToDomain.set(alias, definition.id);
|
||||
}
|
||||
|
||||
/**
|
||||
* Every value a domain plan accepts, canonical ids first.
|
||||
*
|
||||
* This exists so the model-facing tool schema can enumerate the vocabulary
|
||||
* instead of accepting any string. As a free-form string the schema stated no
|
||||
* vocabulary at all, so a name the skill's methodology happened to use passed
|
||||
* validation and only failed deep inside the call. Enumerating trades the old
|
||||
* tolerance for surrounding whitespace and casing for a stated contract: a
|
||||
* mismatch is now refused with the accepted values named.
|
||||
*/
|
||||
export const consultationDomainPlanValues = [
|
||||
...new Set<string>([...consultationDomainIds, ...aliasToDomain.keys()]),
|
||||
] as [string, ...string[]];
|
||||
|
||||
export const consultationDomainPlanValueSchema = z.enum(consultationDomainPlanValues);
|
||||
|
||||
export function normalizeConsultationDomain(value: unknown): ConsultationDomain | null {
|
||||
if (typeof value !== "string") return null;
|
||||
return aliasToDomain.get(value.trim().toLowerCase()) ?? null;
|
||||
|
||||
@@ -85,6 +85,24 @@ function safeToolError(error: unknown) {
|
||||
return "calculation_failed" as const;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a tool result is really an input rejection Mastra resolved with.
|
||||
*
|
||||
* Mastra validates arguments against the tool's inputSchema before `execute`
|
||||
* and reports a mismatch by *resolving* with an error envelope rather than
|
||||
* throwing. Passing that through as `tool.completed` would tell the client a
|
||||
* calculation finished while the tool body never ran. Matched on the envelope's
|
||||
* shape, not its English message, so an upstream wording change cannot silently
|
||||
* turn a rejection back into a success.
|
||||
*/
|
||||
function isToolInputRejection(result: unknown) {
|
||||
if (!result || typeof result !== "object") return false;
|
||||
const envelope = result as { error?: unknown; validationErrors?: unknown };
|
||||
return envelope.error === true
|
||||
&& typeof envelope.validationErrors === "object"
|
||||
&& envelope.validationErrors !== null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Record a tool failure the tool itself could not record.
|
||||
*
|
||||
@@ -147,9 +165,15 @@ function mapChunk(
|
||||
return [{ type: "skill.completed", name: "jyotish-vedic-astrology" }];
|
||||
}
|
||||
if (toolName === "run-jyotish-consultation") {
|
||||
const durationMs = Math.max(0, Date.now() - (startedAt.get(callId) ?? Date.now()));
|
||||
if (isToolInputRejection(payload.result)) {
|
||||
toolErrors.seen += 1;
|
||||
if (options.state) recordUnrecordedToolFailure(options.state, toolErrors.seen, durationMs);
|
||||
return [{ type: "tool.failed", callId, tool: "run-jyotish-consultation", code: "calculation_failed" }];
|
||||
}
|
||||
return [{
|
||||
type: "tool.completed", callId, tool: "run-jyotish-consultation", status: options.toolStatus(),
|
||||
durationMs: Math.max(0, Date.now() - (startedAt.get(callId) ?? Date.now())),
|
||||
durationMs,
|
||||
}];
|
||||
}
|
||||
}
|
||||
@@ -243,8 +267,16 @@ export function streamAgentResponse(options: StreamAgentResponseOptions) {
|
||||
let composingSent = false;
|
||||
const outputText = async (text: string) => {
|
||||
attemptOutput += text;
|
||||
// Text the model writes before the contract is ready is not the answer: it
|
||||
// is the model narrating its own in-progress or failed tool calls. Holding
|
||||
// it meant a later successful call released that narration as the entire
|
||||
// visible answer, so a run where the model recovered read as a run where it
|
||||
// explained itself instead of answering. Drop it. attemptOutput still
|
||||
// records that the model spoke, which is what separates an incomplete
|
||||
// contract from silence below.
|
||||
if (!contractReady(options)) return;
|
||||
held += text;
|
||||
if (!held || !contractReady(options)) return;
|
||||
if (!held) return;
|
||||
if (!composingSent) {
|
||||
composingSent = true;
|
||||
send(controller, { type: "activity", phase: "answer-composition", label: "正在组织回答" });
|
||||
@@ -274,7 +306,7 @@ export function streamAgentResponse(options: StreamAgentResponseOptions) {
|
||||
}
|
||||
}
|
||||
await outputText(visible.finish(""));
|
||||
return { held, attemptOutput };
|
||||
return { attemptOutput };
|
||||
}
|
||||
|
||||
const body = new ReadableStream<Uint8Array>({
|
||||
@@ -301,7 +333,10 @@ export function streamAgentResponse(options: StreamAgentResponseOptions) {
|
||||
emitted = true;
|
||||
}
|
||||
if (!/\S/.test(fullOutput)) {
|
||||
if (/\S/.test(first.held) || /\S/.test(first.attemptOutput)) throw new Error("runtime_contract_incomplete");
|
||||
// attemptOutput records everything the model wrote, including the
|
||||
// discarded pre-contract narration, so a model that spoke but never
|
||||
// produced an answer is still distinguished from one that stayed silent.
|
||||
if (/\S/.test(first.attemptOutput)) throw new Error("runtime_contract_incomplete");
|
||||
throw new Error("empty_answer");
|
||||
}
|
||||
settling = true;
|
||||
|
||||
@@ -2,6 +2,7 @@ import { isDeepStrictEqual } from "node:util";
|
||||
import { createTool } from "@mastra/core/tools";
|
||||
import { z } from "zod";
|
||||
import {
|
||||
consultationDomainPlanValueSchema,
|
||||
validateConsultationDomainPlan,
|
||||
type ConsultationDomain,
|
||||
} from "../lib/consultation-domain-registry.ts";
|
||||
@@ -51,7 +52,14 @@ export const MAX_CONSULTATION_DOMAINS = Math.max(
|
||||
// canonicalizes instead of failing outright. The executable cap is enforced
|
||||
// after canonicalization, where it can degrade and disclose rather than throw.
|
||||
const MAX_CONSULTATION_DOMAIN_PLAN_VALUES = 6;
|
||||
const domainPlanValueSchema = z.string().trim().min(1).max(64);
|
||||
// The legal values have to be stated in the schema the model is handed, not only
|
||||
// enforced in the registry behind execute(). As a free-form string this accepted
|
||||
// any identifier the skill's methodology happened to name—the strict-workflow
|
||||
// checklist labels are not domains—so an invented value passed validation and
|
||||
// died inside execute, spending a step and a tool.failed to learn a vocabulary
|
||||
// the schema could have listed. Aliases stay accepted, so this enumerates them
|
||||
// alongside the canonical ids rather than narrowing what a call may say.
|
||||
const domainPlanValueSchema = consultationDomainPlanValueSchema;
|
||||
|
||||
// The model may only express a domain plan one way. A second, mutually
|
||||
// exclusive field was representable here but rejected at execution, so every
|
||||
@@ -466,7 +474,7 @@ export function createConsultationTools(ctx: ConsultationAgentContext) {
|
||||
let calculation: Promise<ReturnType<typeof toModelDomainPlanContext>> | null = null;
|
||||
const consultationTool = createTool({
|
||||
id: "run-jyotish-consultation",
|
||||
description: `Run one server-validated plan of at most ${MAX_CONSULTATION_DOMAINS} allowlisted personal Jyotish consultation domains. Send only question and domains. The single ordered domains array is the only way to select domains: list them in priority order, or omit it entirely to accept the domain the server already selected for this consultation. Domains execute one after another and each costs about ${Math.round(CONSULTATION_DOMAIN_DURATION_MS / 1000)}s of the run's wall clock, so a shorter plan leaves more time to write the answer; if the clock runs short the server executes the domains that fit and returns the rest in omitted_domains. Birth data is server-bound and must never be supplied. The result always carries one top-level answer contract—status, evidence_contract, claim_cards, rectification—which for several domains is the most restrictive merge of the executed ones, with per-domain detail in consultations. One calculation is executed per request and reused, so repeating the call with different parameters cannot change the result.`,
|
||||
description: `Run one server-validated plan of at most ${MAX_CONSULTATION_DOMAINS} allowlisted personal Jyotish consultation domains. Send only question and domains. The single ordered domains array is the only way to select domains: list them in priority order, or omit it entirely to accept the domain the server already selected for this consultation. Use only the ids enumerated in the schema; workflow or checklist names from the skill's methodology are not domain ids. Domains execute one after another and each costs about ${Math.round(CONSULTATION_DOMAIN_DURATION_MS / 1000)}s of the run's wall clock, so a shorter plan leaves more time to write the answer; if the clock runs short the server executes the domains that fit and returns the rest in omitted_domains. Birth data is server-bound and must never be supplied. The result always carries one top-level answer contract—status, evidence_contract, claim_cards, rectification—which for several domains is the most restrictive merge of the executed ones, with per-domain detail in consultations. One calculation is executed per request and reused, so repeating the call with different parameters cannot change the result.`,
|
||||
inputSchema: consultationToolInputSchema,
|
||||
execute: async (input, context) => {
|
||||
const requestedDomains = canonicalDomainPlan(input, ctx);
|
||||
|
||||
@@ -35,7 +35,7 @@ const jyotishInstructions = `You are the guide for a conversational Vedic astrol
|
||||
Write in concise Simplified Chinese as a natural conversation, not a report or fixed template. Use Markdown only when it improves scanning; tables are allowed only for genuinely comparative information.
|
||||
For Vedic astrology questions, load the jyotish-vedic-astrology skill before deciding which calculation tool or workflow to use. Follow the skill's method and truth boundaries, but use run-jyotish-consultation for actual chart calculations instead of inventing results.
|
||||
For questions that require a new chart claim, call run-jyotish-consultation before answering. Simple conversational follow-ups may use the existing context.
|
||||
Select consultation domains only through the single ordered domains array of run-jyotish-consultation, whether the question covers one domain or several; omit it to accept the domain the server already selected. At most ${MAX_CONSULTATION_DOMAINS} domains may be requested in one run, because they are calculated one after another inside a fixed time budget: list them in priority order and prefer the smallest plan that answers the question, since every extra domain takes time away from writing the answer. The server canonicalizes aliases, rejects unsupported/product domains, executes each accepted domain, and returns the actual domains in the tool context and receipt. A rejected domain plan is final for this run: correct the domains once, and never re-send the same call with extra parameters.
|
||||
Select consultation domains only through the single ordered domains array of run-jyotish-consultation, whether the question covers one domain or several; omit it to accept the domain the server already selected. At most ${MAX_CONSULTATION_DOMAINS} domains may be requested in one run, because they are calculated one after another inside a fixed time budget: list them in priority order and prefer the smallest plan that answers the question, since every extra domain takes time away from writing the answer. The server canonicalizes aliases, rejects unsupported/product domains, executes each accepted domain, and returns the actual domains in the tool context and receipt. The only legal domain ids are the ones enumerated in that array's schema; the skill's methodology names strict-workflow checklists such as career-timing-strict, and those labels select techniques inside the skill, never domains for this tool. A rejected domain plan is final for this run: correct the domains once, and never re-send the same call with extra parameters.
|
||||
The tool result always carries one top-level answer contract—status, evidence_contract, claim_cards, rectification—even when several domains ran. For a multi-domain plan that top level is the most restrictive merge of the executed domains, so obey it exactly as written and read consultations only for per-domain detail. Never treat an absent top-level field as permission to answer without a contract.
|
||||
When omitted_domains is non-empty, the server did not calculate those domains in this run. Name the domains you did cover, say plainly that the remaining ones were not calculated, and never present the answer as covering the whole plan.
|
||||
Activity, progress, tool status, and execution receipts are server-owned. Never imitate data-jyotish-activity, activity events, tool-started/tool-completed messages, or receipts in the answer text.
|
||||
|
||||
Reference in New Issue
Block a user