fix(report): do not fail a written report when progress projection throws
Independent Staging Quality Gate / validate (push) Has been cancelled
Independent Staging Quality Gate / publish (push) Has been cancelled

Staging wrote four ready chapters then marked the job schema-invalid 34ms
later with no summary telemetry. Keep chapter persistence and assemble
going if onProgress or a non-lease heartbeat error fails.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Jesse_Chen
2026-09-03 09:54:14 +08:00
parent f127ada413
commit fbd6e48036
5 changed files with 101 additions and 18 deletions
+47 -2
View File
@@ -146,9 +146,54 @@ Job 在写作过程中 `lease_expired` 两次(默认租约 120s,单章可超
无每章 `inputTokens`/`outputTokens`。Run A 墙钟约 10.4 分钟(含两次租约回收与 marriage/timing 的 section repair)。数据摆出,不裁决。
### Run C · requestId `89c04491-81ca-457e-b09d-dc81c3af36b3`
`GET /api/health` `.deployment.gitCommit` = `5c0bec0c9419e12e2c1fcab8e75fd35d535cb50e`。Web 容器 `2026-09-02T18:24:50Z`。未提升 main。
standard `personal_full`,主题 career / marriage / wealth / timing。started 01:31:16 UTC,终态 01:45:53 UTC(约 14.6 分钟)。reservation `released`
| 章 | status | attempts | narrative 字数 | last_error_code |
|---|---|---:|---:|---|
| theme-career | ready | 1 | 170 | (空) |
| theme-marriage | ready | 2 | 1338 | (空) |
| theme-timing | ready | 1 | 926 | (空) |
| theme-wealth | ready | 1 | 745 | (空) |
Telemetry`[personal-report-agent]`,本轮 docker logs 可读):
| 调用 | outcome | elapsedMs | inputTokens | outputTokens | finishReason | repair |
|---|---|---:|---:|---:|---|---|
| career | resolved | 5721 | 5215 | 560 | stop | false |
| marriage | failed | 9430 | null | null | tripwire | false |
| marriage | failed | 9447 | null | null | tripwire | true |
| marriage | resolved | 13618 | 5076 | 1240 | stop | false |
| timing | resolved | 8052 | 4474 | 816 | stop | false |
| wealth | resolved | 7446 | 5228 | 759 | stop | false |
| summary | (无日志) | | | | | |
`finishReason=length`。四章 `stop` 合计 input 19993 / output 3375。无 summary telemetry。
Job attempt 1 在 marriage 写作中心跳停更(heartbeat 与 lease 同步老化,最长观测 584s),`lease_expired` 后 attempt 2。四章随后都写成 ready。wealth 落库 `01:45:53.549`,整份失败 `01:45:53.583`(约 34ms)。进度停在 **57%**marriage 完成后的 2/4),timing/wealth 的 onProgress 未落库。无 `[personal-report] generation_failed` 行 → 不是 `failSchema` 的摘要/装配返回,而是 `onProgress`/续租抛错被 worker 映射成 `report_schema_invalid`
### Run C 回溯抽查(只记是否对得上 ref)
| # | 章 | 叙事层(布尔) | 对回的 evidenceRef |
|---|---|---|---|
| 1 | 四章皆含 yoga 词 | true | wealth 另有 `ev-tech-yoga` |
| 2 | marriage / timing / wealth | 大运/主运/副运 truecareer false | 四章皆有 `ev-tech-vimshottari` |
| 3 | marriage | yoga + 大运 | `ev-tech-d9` + `ev-tech-dk` |
| 4 | career | yoga;无大运用词 | `ev-tech-amk` + `ev-tech-vimshottari` |
| 5 | wealth | yoga + 大运 | `ev-tech-d2` + `ev-tech-d11` |
未扫到强弱/shadbala 用词。
### 相对 2 倍线
四章 writer 墙钟合计约 35s 模型时间;整份 14.6 分钟含一次租约回收与 marriage tripwire。inputTokens 已测(上表)。数据摆出,不裁决。
### 任务 4 结论
Writer 修正在章节层成立:四章有正文、career 一次通过、无截断码。整份 ready 仍被 120s 租约和最后一步 `report_schema_invalid` 挡住。租约、心跳续租与摘要 cap 已推 staging;公开 health 在跟进部署前仍指向 `eda37c15`。整份 ready 要等新 SHA 部署后再验一次
Writer 修正在章节层成立。整份 ready 被两件事挡住:Next.js instrumentation 里章节写作期间心跳定时器仍不续租;最后一章 `complete` 之后 `onProgress` 抛错被标成 schema 失败,摘要从未开始。跟进补丁:进度投影失败不得中止已落库章节的摘要/装配
---
@@ -164,4 +209,4 @@ Writer 修正在章节层成立:四章有正文、career 一次通过、无截
未跑全量 `tests/*.test.ts`,不声称全套绿。未改 `.gitea/workflows/**`,未提升 main。
任务 4 补丁(租约 600s、摘要 cap 3072、心跳保持 ref + 进度续租):`tsc --noEmit` 通过;plan / generation-v2 / worker 聚焦测试通过。
任务 4 补丁(租约 600s、摘要 cap 3072、心跳保持 ref + 进度续租 + onProgress 不中止装配):`tsc --noEmit` 通过;generation-v2 / worker / writer-failure 聚焦测试通过。
+1 -1
View File
@@ -7643,7 +7643,7 @@
- 触发条件:章节 writer 输出未通过 `assertWriterOutput`(常见为 `evidenceRefs` 与 plan 集合不等),或输出在 token 预算内被截断;随后整份报告失败。
- 根因:三层叠加。旧错误分类把任何含 `evidence` 的消息打成 `section_evidence_insufficient``report_writer_evidence_refs_mismatch` 被误标;章节输出预算用英文「字符 ÷ 2」,standard 上限 1200 字中文只分到 1024 token;一章 blocked 或未捕获抛错可以中止整份,job 停在约 43%。失败详情只有聚合码,页面无法区分截断、引用不齐或未开始。staging 事故窗口的 writer telemetry 在容器 recreate 后丢失,本事故不能写成 `finishReason=length`
- 修复:中文口径重算每章/摘要 `maxOutputTokens`,去掉 ÷2;length 修复要求压到字数下限且预算不低于首次。Prompt 要求逐字复制 `plan.evidenceRefs`repair 只加失败类别。refs/identity 先于泛化 `evidence` 分类。单章失败后继续其余主题;`all_sections_blocked` 走统一失败日志。详情与列表聚合已有 section 行,展示可读摘要与错误码,不改表、不放宽 writer schema。
- 验证:预算表锁定 concise 1218 / standard 2128 / deep 3072 / research 3072 / summary 3072。refs mismatch 分类为 `section_refs_mismatch` 且其余主题仍交付。repair 提示含类别词、不含内容。失败详情从 section 行汇总。`tsc --noEmit`、改动文件 ESLint、个人报告聚焦测试。staging `eda37c15` 上真实 standard 四章均 readycareer 1 次成功,无截断码),整份仍因 120s 租约与最后一步 `report_schema_invalid` 未落 `report_document`;租约改为 600s,心跳保持 ref,章节进度先续租
- 验证:预算表锁定 concise 1218 / standard 2128 / deep 3072 / research 3072 / summary 3072。refs mismatch 分类为 `section_refs_mismatch` 且其余主题仍交付。repair 提示含类别词、不含内容。失败详情从 section 行汇总。`tsc --noEmit`、改动文件 ESLint、个人报告聚焦测试。staging `5c0bec0c` 上真实 standard 四章均 ready`finishReason=length`),整份仍因 `onProgress` 抛错被标成 `report_schema_invalid`,摘要未开始;进度投影失败改为不中止已落库章节
- 防复发:输出预算不得再用字符 ÷ 2。`assertWriterOutput` 保持 id/theme 全等与 refs 集合相等。section 错误码不得把 refs mismatch 归进 evidence insufficient。一章失败不得中止其余 write 主题。用户可见失败必须有错误码级摘要,不得只展示 `report_schema_invalid`。日志与 PROGRESS 不得写入 prompt、bundle、模型原文或用户资料。
- 相关记录:BUG-352、BUG-451、BUG-486、BUG-489
- 复发自:BUG-451(分章后仍把写作失败压成整份 schema 码,截断预算与失败分类未按中文口径收口)
+13 -6
View File
@@ -3155,6 +3155,13 @@ async function generateSectionedPersonalReport(
return failSchema("sectioned_dependencies_missing");
}
const writePlans = plan.sections.filter((entry) => entry.kind === "thematic" && entry.disposition === "write");
const emitProgress = async (phase: string, completed: number): Promise<void> => {
try {
await deps.onProgress?.({ phase, completed, total: writePlans.length });
} catch (error) {
rethrowIfAborted(error, deps.signal);
}
};
for (const entry of writePlans) {
await sectionService.ensure({
userId: deps.userId, requestId: deps.requestId, sectionId: entry.id, maxAttempts: 2,
@@ -3177,7 +3184,7 @@ async function generateSectionedPersonalReport(
});
if (!blockedRow) throw new Error("section_block_failed");
blocked.push(blockedRow);
await deps.onProgress?.({ phase: `section:${entry.id}`, completed: ready.length + blocked.length, total: writePlans.length });
await emitProgress(`section:${entry.id}`, ready.length + blocked.length);
continue;
}
const outputBudget = sectionOutputTokenBudget(entry.targetCharacters.max);
@@ -3204,7 +3211,7 @@ async function generateSectionedPersonalReport(
const completed = await sectionService.complete({ userId: deps.userId, requestId: deps.requestId, sectionId: entry.id, payload: output });
if (!completed) throw new Error("section_complete_failed");
ready.push(completed);
await deps.onProgress?.({ phase: `section:${entry.id}`, completed: ready.length + blocked.length, total: writePlans.length });
await emitProgress(`section:${entry.id}`, ready.length + blocked.length);
break;
} catch (error) {
rethrowIfAborted(error, deps.signal);
@@ -3214,7 +3221,7 @@ async function generateSectionedPersonalReport(
});
if (!blockedRow) throw new Error("section_block_failed");
blocked.push(blockedRow);
await deps.onProgress?.({ phase: `section:${entry.id}`, completed: ready.length + blocked.length, total: writePlans.length });
await emitProgress(`section:${entry.id}`, ready.length + blocked.length);
break;
}
}
@@ -3232,13 +3239,13 @@ async function generateSectionedPersonalReport(
});
if (!blockedRow) throw new Error("section_block_failed");
blocked.push(blockedRow);
await deps.onProgress?.({ phase: `section:${entry.id}`, completed: ready.length + blocked.length, total: writePlans.length });
await emitProgress(`section:${entry.id}`, ready.length + blocked.length);
}
}
if (writePlans.length > 0 && ready.length === 0) {
return failSchema("all_sections_blocked");
}
await deps.onProgress?.({ phase: "summary", completed: writePlans.length, total: writePlans.length });
await emitProgress("summary", writePlans.length);
let summary;
try {
summary = await deps.agent.generateSummary(ready.map((item) => ({
@@ -3260,7 +3267,7 @@ async function generateSectionedPersonalReport(
missingEvidence: [sectionFailureReason(item.lastErrorCode)], conflictNotes: [], evidenceRefs: refs,
};
});
await deps.onProgress?.({ phase: "assemble", completed: writePlans.length, total: writePlans.length });
await emitProgress("assemble", writePlans.length);
try {
const candidate = assembleReportDocumentV2({
reportId: deps.reportId, generatedAt: (deps.now ?? (() => new Date()))().toISOString(),
@@ -338,14 +338,22 @@ export function createPersonalReportWorker(deps: PersonalReportWorkerDeps) {
signal: controller.signal,
sectionService: deps.sectionService,
onProgress: async (progress) => {
await refreshLease();
if (heartbeatError !== null) throw heartbeatError;
const percent = progress.total > 0
? Math.min(89, 30 + Math.floor((progress.completed / progress.total) * 55))
: 30;
await deps.jobs.updateProgress({
jobId: job.id, leaseToken: job.leaseToken!, phase: progress.phase, percent,
});
try {
await refreshLease();
if (heartbeatError !== null) throw heartbeatError;
const percent = progress.total > 0
? Math.min(89, 30 + Math.floor((progress.completed / progress.total) * 55))
: 30;
await deps.jobs.updateProgress({
jobId: job.id, leaseToken: job.leaseToken!, phase: progress.phase, percent,
});
} catch (error) {
if (isLeaseLoss(error) || heartbeatError !== null) {
heartbeatError = heartbeatError ?? error;
controller.abort(error);
throw error;
}
}
},
});
await heartbeatChain;
@@ -426,8 +426,14 @@ test("production writer and worker keep the same signal on initial and repair mo
new URL("../src/lib/personal-report-worker-core.ts", import.meta.url),
"utf8",
);
assert.match(workerCore, /onProgress: async \(progress\) => \{\s*await refreshLease\(\)/);
assert.match(workerCore, /onProgress: async \(progress\) => \{\s*try \{\s*await refreshLease\(\)/);
assert.doesNotMatch(workerCore, /timerUnref\(heartbeatTimer\)/);
const generationSource = readFileSync(
new URL("../src/lib/personal-report-generation.ts", import.meta.url),
"utf8",
);
assert.match(generationSource, /const emitProgress = async \(phase: string, completed: number\)/);
assert.match(generationSource, /rethrowIfAborted\(error, deps\.signal\);/);
});
test("an all-blocked plan still produces a valid v2 document with honest disclosures and no invented thematic section", async () => {
@@ -828,6 +834,23 @@ test("sectioned generation makes one filtered call per write theme, then summary
assert.match(document.executiveSummary.summary, /事业与方向/);
});
test("sectioned generation still assembles when onProgress throws after a persisted section", async () => {
const bundle = makeBundle({
themes: fullThemes.slice(0, 3),
charts: [chart("D1"), chart("D2", 1), chart("D9", 2), chart("D10", 3), chart("D11", 4)],
});
const result = await runSectioned(
bundle,
sectionedAgent({}),
inMemorySectionService(),
async () => {
throw new Error("progress_projection_failed");
},
);
const document = readyV2(result);
assert.deepEqual(document.thematicNarrative.map((section) => section.theme), ["career", "marriage", "wealth"]);
});
test("sectioned resume skips ready sections after an interruption", async () => {
const bundle = makeBundle({ themes: fullThemes, charts: [chart("D1"), chart("D2", 1), chart("D9", 2), chart("D10", 3), chart("D11", 4), chart("D24", 5)] });
const service = inMemorySectionService();