Compare commits

...
Author SHA1 Message Date
jesse-ux c2ecbc74cd research(jev-intent): 修复轮重造语料并全量对照
Independent Staging Quality Gate / publish (push) Canceled after 0s
Independent Staging Quality Gate / validate (push) Canceled after 1m37s
来源 C 改为 DeepSeek Flash 生成+独立复核,撤回模板拼接结论。来源 B 157 条人工标注后跑 Jev x2 与 Flash 全量对照,结论为缺数据。
2026-09-19 11:30:31 +08:00
Jesse_ChenandClaude Fable 5 69a44fe78d docs(tasks): Jev 修复单补产品拍板:线上模型=DeepSeek Flash、来源 B 157 条已交付
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0155nFCgCHtoA7jhSDGmZmMu
2026-09-19 02:30:16 +00:00
Jesse_ChenandClaude Fable 5 4f4cd6843a docs(tasks): Jev 意图分类研究验收未通过,修复单要求真实模型生成来源 C
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0155nFCgCHtoA7jhSDGmZmMu
2026-09-19 01:55:19 +00:00
13 changed files with 95536 additions and 82588 deletions
+6 -8
View File
@@ -3,19 +3,17 @@
## ~~TypeSafe Jev 意图分类:等 API key(2026-09-19)~~
- ~~T2/T3 阻塞,替代证据 = T0/T1 产出。~~
- **解除(2026-09-19):** 产品提供 `TYPESAFE_API_KEY`,T2 已用 `jev-1.13.0` 跑完来源 A+C 各两次。结论见 `docs/research/jev_intent_2026_09_19.md`:**不可接**。
- **解除(2026-09-19):** 产品提供 `TYPESAFE_API_KEY`,T2 已用 `jev-1.13.0` 跑完来源 A+C 各两次。首版结论作废(语料为模板拼接)。
- **修复轮(2026-09-19):** 来源 C 由 DeepSeek Flash 生成+复核后重跑。结论见 `docs/research/jev_intent_2026_09_19.md`:**缺数据**。
## 生时校正 Jev 研究:无 staging 库,来源 B = 0(2026-09-19)
## ~~生时校正 Jev 研究:无 staging 库,来源 B = 0(2026-09-19)~~
- **缺的是什么:** staging `agentic_rectification_turns.user_message` 非空样本。本机没有 `SCHEMA_DATABASE_URL` / `.env.staging.database`。
- **替代:** 按任务书让步 1,代表性检验只报数、不判定。来源 C 照常跑。抽取 SQL 与本地路径写在 `scripts/research/jev_intent_samples/README.md`(`G:/Ferti/Jyotisha/.cache/jev_intent/source_b.jsonl`,gitignore)。
- **解除需要:** 只读抽 ≥30 条非空 `user_message`(含当轮焦点 schema 与后续动作),执行方逐条人工标注。不得提交原文。
- ~~缺 staging `agentic_rectification_turns` 样本。~~
- **解除(2026-09-19 修复轮):** 产品转交 `source_b.jsonl` 157 条(点选 17 / 采集 107 / 无焦点 33),执行方逐条人工标注 gold,未提交原文。Jev×2 + Flash×1 已跑。无焦点层与来源 C intent 差 17.2pp > 10pp,结论为缺数据。
## ~~生时校正 Jev 研究:无会话模型凭据,现行分类器未对照(2026-09-19)~~
- **部分解除(2026-09-19):** 产品提供 DeepSeek key,用 `deepseek-flash` 套生产 `classifyRectificationTurnIntent` 提示词,来源 C 分层随机 33% + 来源 A 全量。相对门槛采集题未过(91.0% < 92.5%)。绝对门槛仍判 **不可接**。
- **仍缺:** 线上会话默认贵模型(数据库加密配置)未跑;这是替代对照,不是生产模型。
- 来源 B 仍为 0,见上条。
- **解除(2026-09-19 修复轮):** 产品确认线上会话模型 = DeepSeek Flash。生成、复核、对照均用该模型(提示不同)。来源 C 全量一次 + 1/3 第二次;来源 B 全量一次。原「无会话模型凭据」缺口关闭。
## 会话列表:元数据操作未上移到 provider(2026-09-17,BUG-927 让步)
File diff suppressed because it is too large Load Diff
+62 -54
View File
@@ -1,49 +1,57 @@
# TypeSafe Jev 接管校正意图分类 · 离线对照(2026-09-19)
- 任务:`docs/tasks/TASK-rectification-jev-intent-classifier-research-20260919.md`
- 基线:`origin/staging` @ `d9722d56`
- 基线:`origin/staging` @ `69a44fe7`
- Jev 模型:`jev-1.13.0`(不用 jev-latest)
- 生成器 / 复核:`agent-template-v1` / `agent-rule-v1`
- 真值 sha256:synthetic `79d53a958b3b42ed18c79e3bbcff4b31deb22b65eedcd89b39326a7fefc9b0e8`;simulated `a8611e75bf56024fa984fb204d34a5e16405f18b3eaa5a3bee91f96e43918bf7`;disputed `b4aed8567bd6b3fdd5cc98c73d9139197d62a96120a0ee29f474b8f08c050def`
- 来源 B:0 条(本地,不提交原文)
- 真值 sha256:synthetic `79d53a958b3b42ed18c79e3bbcff4b31deb22b65eedcd89b39326a7fefc9b0e8`;simulated `53f13012fdc8f3721335bbe2961edf8c506a2f705b62bd9d302fce83ce2ef88d`;disputed `e80c5cd69c521a099d743e4a4632ad6bc103c3b5a505f5b376a7c9cebedb7646`
- 来源 B:157 条已标注(本地,不提交原文)
## 模型
- 生成模型:`deepseek-flash` / 版本 `deepseek-flash` / 不是线上会话模型(只用于造来源 C)
- 复核模型:`deepseek-flash` / 版本 `deepseek-flash` / 不是线上会话模型(只用于独立复核,看不到目标标签)
- 对照模型:`deepseek-flash` / 版本 `deepseek-flash` / **= 线上会话模型**(DeepSeek Flash,顶生产 `classifyRectificationTurnIntent` 提示词)
- Jev:`jev-1.13.0` / 不是线上会话模型
## 结论
**不可接**。来源 C 上 Jev 的绝对门槛未过:高置信错误率 7.0% > 3%;低置信召回 34.1% / 34.2% < 60%(错了却仍高置信);点选题 answer_class 准确率 65.0%(写库字段)。来源 B 不足 30 条,代表性检验只报数、不判定。 相对 −3pp(同一样本):choice Jev 89.5% vs 现行 92.5%(门槛 89.5%,过);collect Jev 91.0% vs 现行 95.5%(门槛 92.5%,未过);none Jev 100.0% vs 现行 97.0%(门槛 94.0%,过)。
**缺数据**。来源 B 与来源 C 同层 intent 准确率差 > 10pp,模拟语料不代表真人,来源 C 门槛结论降为缺数据。来源 B 与来源 C 同层 intent 准确率:choice B 94.1% vs C 99.0%(差 4.9%,n_B=17);collect B 94.4% vs C 92.2%(差 2.1%,n_B=107);none B 78.8% vs C 96.0%(差 17.2%,n_B=33)。 有层差 > 10pp,结论降为缺数据。 同时来源 C 绝对门槛未过:低置信召回 23.1% / 45.5% / 25.0% < 60%(错了却仍高置信)。 相对 −3pp(同一样本):choice Jev 99.0% vs 现行 98.8%(门槛 95.8%,过);collect Jev 92.2% vs 现行 97.5%(门槛 94.5%,未过);none Jev 96.0% vs 现行 94.0%(门槛 91.0%,过)。
若接:不接。现行分类器继续用会话选定的贵模型。若还要观察中文口语,只允许 (b) 影子双跑只记日志,不得按 confidence 写库。曲线上 θ=0.9 时高置信错误仍未清零。
若接:不得上线。先补真机样本或重造更像真人的来源 C,再测。
## T3 指标(Jev,来源 C)
| 层 | n | intent | answer_class | dated | 高置信错误 | 低置信覆盖 | 低置信召回 | no/unsure 互判 | 自洽率 | 中位 ms | P95 ms | 次均 input tok |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| choice | 400 | 88.0% | 65.0% | 100.0% | 4.2% | 12.5% | 34.1% | 5 | 98.5% | 923 | 1344 | 1122 |
| collect | 400 | 90.5% | 83.6% | 97.0% | 7.0% | 13.5% | 34.2% | 22 | 98.0% | 901 | 1026 | 987 |
| none | 100 | 100.0% | — | 100.0% | 0.0% | 4.0% | — | 0 | 100.0% | 923 | 1018 | 923 |
| choice | 400 | 99.0% | 95.8% | 100.0% | 0.0% | 2.2% | 23.1% | 0 | 99.8% | 908 | 1194 | 1127 |
| collect | 400 | 92.2% | 95.7% | 98.8% | 2.8% | 8.0% | 45.5% | 6 | 98.0% | 909 | 1086 | 991 |
| none | 100 | 96.0% | — | 100.0% | 1.0% | 2.0% | 25.0% | 0 | 100.0% | 909 | 981 | 922 |
来源 A(测试夹具,n=10)intent 90.0%,answer_class 85.7%,高置信错误 0.0%。
## 现行模型对照
模型:`deepseek-flash`。DeepSeek Flash 顶现行 `classifyRectificationTurnIntent` 提示词;来源 C 分层随机 33%(seed 20260919),来源 A 全量。不是线上会话模型。
模型:`deepseek-flash`。DeepSeek Flash = 线上会话模型,顶现行 `classifyRectificationTurnIntent` 提示词;来源 C 全量第一次,第二次分层 33%(seed 20260920);来源 A 全量;来源 B 已标注全量一次。
| 层 | n | intent | answer_class | dated | 自洽率 | 中位 ms |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| choice | 133 | 92.5% | 83.0% | 100.0% | 100.0% | 638 |
| collect | 133 | 95.5% | 95.8% | 97.7% | 95.5% | 611 |
| none | 33 | 97.0% | — | 100.0% | 100.0% | 705 |
| choice | 400 | 98.8% | 99.5% | 100.0% | 100.0% | 685 |
| collect | 400 | 97.5% | 99.6% | 99.0% | 99.2% | 686 |
| none | 100 | 94.0% | — | 100.0% | 100.0% | 672 |
同一样本上 Jev vs 现行(相对门槛用这一表):
| 层 | n | Jev intent | 现行 intent | 差(Jev−现行) | 门槛现行−3pp |
| --- | ---: | ---: | ---: | ---: | ---: |
| choice | 133 | 89.5% | 92.5% | -3.0% | 89.5% |
| collect | 133 | 91.0% | 95.5% | -4.5% | 92.5% |
| none | 33 | 100.0% | 97.0% | 3.0% | 94.0% |
| choice | 400 | 99.0% | 98.8% | 0.2% | 95.8% |
| collect | 400 | 92.2% | 97.5% | -5.2% | 94.5% |
| none | 100 | 96.0% | 94.0% | 2.0% | 91.0% |
## 代表性检验(来源 B vs 来源 C)
来源 B 不足 30 条,代表性检验只报数、不判定。
来源 B 已标注 157 条(点选 17 / 采集 107 / 无焦点 33;未标注 0),长度 P25/中位/P75/最长 = 12/21/28/172。人工标注,未用模型代标。Jev 在来源 B 上 intent:点选 94.1%、采集 94.4%、无焦点 78.8%;全集 91.1%。无焦点层差 17.2pp > 10pp。
来源 B 与来源 C 同层 intent 准确率:choice B 94.1% vs C 99.0%(差 4.9%,n_B=17);collect B 94.4% vs C 92.2%(差 2.1%,n_B=107);none B 78.8% vs C 96.0%(差 17.2%,n_B=33)。 有层差 > 10pp,结论降为缺数据。
## 置信度–准确率曲线与 θ
@@ -53,52 +61,52 @@
[
{
"theta": 0.3,
"jev_share": 0.9877777777777778,
"fallback_share": 0.012222222222222223,
"jev_acc": 0.8278965129358831,
"high_conf_errors_in_used": 45
"jev_share": 0.9911111111111112,
"fallback_share": 0.008888888888888889,
"jev_acc": 0.9405829596412556,
"high_conf_errors_in_used": 12
},
{
"theta": 0.4,
"jev_share": 0.9522222222222222,
"fallback_share": 0.04777777777777778,
"jev_acc": 0.8366394399066511,
"high_conf_errors_in_used": 45
"jev_share": 0.9744444444444444,
"fallback_share": 0.025555555555555557,
"jev_acc": 0.9498289623717218,
"high_conf_errors_in_used": 12
},
{
"theta": 0.5,
"jev_share": 0.88,
"fallback_share": 0.12,
"jev_acc": 0.8661616161616161,
"high_conf_errors_in_used": 45
"jev_share": 0.9522222222222222,
"fallback_share": 0.04777777777777778,
"jev_acc": 0.9568261376896149,
"high_conf_errors_in_used": 12
},
{
"theta": 0.6,
"jev_share": 0.8266666666666667,
"fallback_share": 0.17333333333333334,
"jev_acc": 0.8884408602150538,
"high_conf_errors_in_used": 45
"jev_share": 0.9222222222222223,
"fallback_share": 0.07777777777777778,
"jev_acc": 0.9698795180722891,
"high_conf_errors_in_used": 12
},
{
"theta": 0.7,
"jev_share": 0.7677777777777778,
"fallback_share": 0.23222222222222222,
"jev_acc": 0.9015918958031838,
"high_conf_errors_in_used": 45
"jev_share": 0.8977777777777778,
"fallback_share": 0.10222222222222223,
"jev_acc": 0.9752475247524752,
"high_conf_errors_in_used": 12
},
{
"theta": 0.8,
"jev_share": 0.6822222222222222,
"fallback_share": 0.31777777777777777,
"jev_acc": 0.9267100977198697,
"high_conf_errors_in_used": 45
"jev_share": 0.8222222222222222,
"fallback_share": 0.17777777777777778,
"jev_acc": 0.9837837837837838,
"high_conf_errors_in_used": 12
},
{
"theta": 0.9,
"jev_share": 0.5333333333333333,
"fallback_share": 0.4666666666666667,
"jev_acc": 0.9604166666666667,
"high_conf_errors_in_used": 19
"jev_share": 0.7033333333333334,
"fallback_share": 0.2966666666666667,
"jev_acc": 0.9921011058451816,
"high_conf_errors_in_used": 5
}
]
```
@@ -107,21 +115,21 @@
### cjk_colloquial
- `C-choice-0005` / choice / 「嗯那会儿吧咋说呢就是那段」 gold={'answer_class': 'yes', 'has_new_dated_event': False, 'intent': 'answer_current_focus'} jev={'intent': 'unclear', 'answer_class': None, 'has_new_dated_event': False, 'confidence': 0.88}
- `C-choice-0008` / choice / 「额这个嘛就那样发生了」 gold={'answer_class': 'yes', 'has_new_dated_event': False, 'intent': 'answer_current_focus'} jev={'intent': 'answer_current_focus', 'answer_class': 'unsure', 'has_new_dated_event': False, 'confidence': 0.47}
- `C-choice-0016` / choice / 「emmm 感觉一般般就是那段」 gold={'answer_class': 'yes', 'has_new_dated_event': False, 'intent': 'answer_current_focus'} jev={'intent': 'unclear', 'answer_class': None, 'has_new_dated_event': False, 'confidence': 0.38}
- `C-choice-0149` / choice / 「嗯,那会儿好像有点变动,但印象不深。另外2019年搬过一次家。」 gold={'answer_class': 'weak_yes', 'has_new_dated_event': True, 'intent': 'answer_current_focus'} jev={'intent': 'answer_current_focus', 'answer_class': 'unsure', 'has_new_dated_event': True, 'confidence': 0.65}
- `C-choice-0152` / choice / 「好像有吧但印象不深,对了2019年搬过家。」 gold={'answer_class': 'weak_yes', 'has_new_dated_event': True, 'intent': 'answer_current_focus'} jev={'intent': 'answer_current_focus', 'answer_class': 'unsure', 'has_new_dated_event': True, 'confidence': 0.76}
- `C-collect-0286` / collect / 「差不多是2021年夏天那会儿,七月份前后吧,确实有明显变化。」 gold={'answer_class': 'yes', 'has_new_dated_event': False, 'intent': 'answer_current_focus'} jev={'intent': 'answer_current_focus', 'answer_class': 'weak_yes', 'has_new_dated_event': False, 'confidence': 0.71}
### literal
- `A10` / collect / 「2016年3月入学」 gold={'answer_class': 'yes', 'has_new_dated_event': False, 'intent': 'answer_current_focus'} jev={'intent': 'provide_new_evidence', 'answer_class': None, 'has_new_dated_event': True, 'confidence': 0.39}
- `C-choice-0006` / choice / 「就是那段」 gold={'answer_class': 'yes', 'has_new_dated_event': False, 'intent': 'answer_current_focus'} jev={'intent': 'unclear', 'answer_class': None, 'has_new_dated_event': False, 'confidence': 0.9}
- `C-choice-0023` / choice / 「就是那段,当时身边人也没特别提,我自己也没当回事,我想想啊就是那种说」 gold={'answer_class': 'yes', 'has_new_dated_event': False, 'intent': 'answer_current_focus'} jev={'intent': 'unclear', 'answer_class': None, 'has_new_dated_event': False, 'confidence': 0.64}
- `A10` / collect / 「2016年3月入学」 gold={'answer_class': 'yes', 'has_new_dated_event': False, 'intent': 'answer_current_focus'} jev={'intent': 'provide_new_evidence', 'answer_class': None, 'has_new_dated_event': True, 'confidence': 0.42}
- `C-choice-0386` / choice / 「这个真不好说,另外2019年我换过工作搬了家。」 gold={'answer_class': None, 'has_new_dated_event': True, 'intent': 'provide_new_evidence'} jev={'intent': 'answer_current_focus', 'answer_class': 'unsure', 'has_new_dated_event': True, 'confidence': 0.64}
- `C-choice-0341` / choice / 「对了,1995年那会儿我搬家了,你问的这个我真说不好。」 gold={'answer_class': None, 'has_new_dated_event': True, 'intent': 'provide_new_evidence'} jev={'intent': 'answer_current_focus', 'answer_class': 'unsure', 'has_new_dated_event': True, 'confidence': 0.54}
### other
- `C-choice-0002` / choice / 「不是很能整明白就是那段」 gold={'answer_class': 'yes', 'has_new_dated_event': False, 'intent': 'answer_current_focus'} jev={'intent': 'answer_current_focus', 'answer_class': 'unsure', 'has_new_dated_event': False, 'confidence': 0.75}
- `C-choice-0012` / choice / 「有的,别的先不说,就这一句你先记着,…特别提,我自己也没当回事」 gold={'answer_class': 'yes', 'has_new_dated_event': False, 'intent': 'answer_current_focus'} jev={'intent': 'answer_current_focus', 'answer_class': 'weak_yes', 'has_new_dated_event': False, 'confidence': 0.34}
- `C-choice-0073` / choice / 「明显有过,现在回想起来细节对不上,只…。另外2023年9月分开」 gold={'answer_class': 'yes', 'has_new_dated_event': True, 'intent': 'answer_current_focus'} jev={'intent': 'answer_current_focus', 'answer_class': 'unsure', 'has_new_dated_event': True, 'confidence': 0.46}
- `C-choice-0148` / choice / 「好像有点小变化,但记不太深。另外2019年换过工作。」 gold={'answer_class': 'weak_yes', 'has_new_dated_event': True, 'intent': 'answer_current_focus'} jev={'intent': 'answer_current_focus', 'answer_class': 'unsure', 'has_new_dated_event': True, 'confidence': 0.73}
- `C-choice-0154` / choice / 「好像有那么点,但印象不深。对了,2002年换过一次工作。」 gold={'answer_class': 'weak_yes', 'has_new_dated_event': True, 'intent': 'answer_current_focus'} jev={'intent': 'answer_current_focus', 'answer_class': 'unsure', 'has_new_dated_event': True, 'confidence': 0.59}
- `C-choice-0162` / choice / 「好像有点印象,但很淡。另外2019年搬过一次家。」 gold={'answer_class': 'weak_yes', 'has_new_dated_event': True, 'intent': 'answer_current_focus'} jev={'intent': 'answer_current_focus', 'answer_class': 'unsure', 'has_new_dated_event': True, 'confidence': 0.61}
## T1 判据对照
@@ -0,0 +1,156 @@
# 已作废:语料为模板拼接
本文是 2026-09-19 首版对照报告的存档。来源 C 由 `agent-template-v1` 词库拼接、`agent-rule-v1` 正则复核,结论无效。请读同目录 `jev_intent_2026_09_19.md`。
# TypeSafe Jev 接管校正意图分类 · 离线对照(2026-09-19,v1 存档)
- 任务:`docs/tasks/TASK-rectification-jev-intent-classifier-research-20260919.md`
- 基线:`origin/staging` @ `d9722d56`
- Jev 模型:`jev-1.13.0`(不用 jev-latest)
- 生成器 / 复核:`agent-template-v1` / `agent-rule-v1`
- 真值 sha256:synthetic `79d53a958b3b42ed18c79e3bbcff4b31deb22b65eedcd89b39326a7fefc9b0e8`;simulated `a8611e75bf56024fa984fb204d34a5e16405f18b3eaa5a3bee91f96e43918bf7`;disputed `b4aed8567bd6b3fdd5cc98c73d9139197d62a96120a0ee29f474b8f08c050def`
- 来源 B:0 条(本地,不提交原文)
## 结论
**不可接**。来源 C 上 Jev 的绝对门槛未过:高置信错误率 7.0% > 3%;低置信召回 34.1% / 34.2% < 60%(错了却仍高置信);点选题 answer_class 准确率 65.0%(写库字段)。来源 B 不足 30 条,代表性检验只报数、不判定。 相对 −3pp(同一样本):choice Jev 89.5% vs 现行 92.5%(门槛 89.5%,过);collect Jev 91.0% vs 现行 95.5%(门槛 92.5%,未过);none Jev 100.0% vs 现行 97.0%(门槛 94.0%,过)。
若接:不接。现行分类器继续用会话选定的贵模型。若还要观察中文口语,只允许 (b) 影子双跑只记日志,不得按 confidence 写库。曲线上 θ=0.9 时高置信错误仍未清零。
## T3 指标(Jev,来源 C)
| 层 | n | intent | answer_class | dated | 高置信错误 | 低置信覆盖 | 低置信召回 | no/unsure 互判 | 自洽率 | 中位 ms | P95 ms | 次均 input tok |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| choice | 400 | 88.0% | 65.0% | 100.0% | 4.2% | 12.5% | 34.1% | 5 | 98.5% | 923 | 1344 | 1122 |
| collect | 400 | 90.5% | 83.6% | 97.0% | 7.0% | 13.5% | 34.2% | 22 | 98.0% | 901 | 1026 | 987 |
| none | 100 | 100.0% | — | 100.0% | 0.0% | 4.0% | — | 0 | 100.0% | 923 | 1018 | 923 |
来源 A(测试夹具,n=10)intent 90.0%,answer_class 85.7%,高置信错误 0.0%。
## 现行模型对照
模型:`deepseek-flash`。DeepSeek Flash 顶现行 `classifyRectificationTurnIntent` 提示词;来源 C 分层随机 33%(seed 20260919),来源 A 全量。不是线上会话模型。
| 层 | n | intent | answer_class | dated | 自洽率 | 中位 ms |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| choice | 133 | 92.5% | 83.0% | 100.0% | 100.0% | 638 |
| collect | 133 | 95.5% | 95.8% | 97.7% | 95.5% | 611 |
| none | 33 | 97.0% | — | 100.0% | 100.0% | 705 |
同一样本上 Jev vs 现行(相对门槛用这一表):
| 层 | n | Jev intent | 现行 intent | 差(Jev−现行) | 门槛现行−3pp |
| --- | ---: | ---: | ---: | ---: | ---: |
| choice | 133 | 89.5% | 92.5% | -3.0% | 89.5% |
| collect | 133 | 91.0% | 95.5% | -4.5% | 92.5% |
| none | 33 | 100.0% | 97.0% | 3.0% | 94.0% |
## 代表性检验(来源 B vs 来源 C)
来源 B 不足 30 条,代表性检验只报数、不判定。
## 置信度–准确率曲线与 θ
推荐 θ = 0.9。点:
```json
[
{
"theta": 0.3,
"jev_share": 0.9877777777777778,
"fallback_share": 0.012222222222222223,
"jev_acc": 0.8278965129358831,
"high_conf_errors_in_used": 45
},
{
"theta": 0.4,
"jev_share": 0.9522222222222222,
"fallback_share": 0.04777777777777778,
"jev_acc": 0.8366394399066511,
"high_conf_errors_in_used": 45
},
{
"theta": 0.5,
"jev_share": 0.88,
"fallback_share": 0.12,
"jev_acc": 0.8661616161616161,
"high_conf_errors_in_used": 45
},
{
"theta": 0.6,
"jev_share": 0.8266666666666667,
"fallback_share": 0.17333333333333334,
"jev_acc": 0.8884408602150538,
"high_conf_errors_in_used": 45
},
{
"theta": 0.7,
"jev_share": 0.7677777777777778,
"fallback_share": 0.23222222222222222,
"jev_acc": 0.9015918958031838,
"high_conf_errors_in_used": 45
},
{
"theta": 0.8,
"jev_share": 0.6822222222222222,
"fallback_share": 0.31777777777777777,
"jev_acc": 0.9267100977198697,
"high_conf_errors_in_used": 45
},
{
"theta": 0.9,
"jev_share": 0.5333333333333333,
"fallback_share": 0.4666666666666667,
"jev_acc": 0.9604166666666667,
"high_conf_errors_in_used": 19
}
]
```
## 错例画像(改写后,无法对应真实会话)
### cjk_colloquial
- `C-choice-0005` / choice / 「嗯那会儿吧咋说呢就是那段」 gold={'answer_class': 'yes', 'has_new_dated_event': False, 'intent': 'answer_current_focus'} jev={'intent': 'unclear', 'answer_class': None, 'has_new_dated_event': False, 'confidence': 0.88}
- `C-choice-0008` / choice / 「额这个嘛就那样发生了」 gold={'answer_class': 'yes', 'has_new_dated_event': False, 'intent': 'answer_current_focus'} jev={'intent': 'answer_current_focus', 'answer_class': 'unsure', 'has_new_dated_event': False, 'confidence': 0.47}
- `C-choice-0016` / choice / 「emmm 感觉一般般就是那段」 gold={'answer_class': 'yes', 'has_new_dated_event': False, 'intent': 'answer_current_focus'} jev={'intent': 'unclear', 'answer_class': None, 'has_new_dated_event': False, 'confidence': 0.38}
### literal
- `A10` / collect / 「2016年3月入学」 gold={'answer_class': 'yes', 'has_new_dated_event': False, 'intent': 'answer_current_focus'} jev={'intent': 'provide_new_evidence', 'answer_class': None, 'has_new_dated_event': True, 'confidence': 0.39}
- `C-choice-0006` / choice / 「就是那段」 gold={'answer_class': 'yes', 'has_new_dated_event': False, 'intent': 'answer_current_focus'} jev={'intent': 'unclear', 'answer_class': None, 'has_new_dated_event': False, 'confidence': 0.9}
- `C-choice-0023` / choice / 「就是那段,当时身边人也没特别提,我自己也没当回事,我想想啊就是那种说」 gold={'answer_class': 'yes', 'has_new_dated_event': False, 'intent': 'answer_current_focus'} jev={'intent': 'unclear', 'answer_class': None, 'has_new_dated_event': False, 'confidence': 0.64}
### other
- `C-choice-0002` / choice / 「不是很能整明白就是那段」 gold={'answer_class': 'yes', 'has_new_dated_event': False, 'intent': 'answer_current_focus'} jev={'intent': 'answer_current_focus', 'answer_class': 'unsure', 'has_new_dated_event': False, 'confidence': 0.75}
- `C-choice-0012` / choice / 「有的,别的先不说,就这一句你先记着,…特别提,我自己也没当回事」 gold={'answer_class': 'yes', 'has_new_dated_event': False, 'intent': 'answer_current_focus'} jev={'intent': 'answer_current_focus', 'answer_class': 'weak_yes', 'has_new_dated_event': False, 'confidence': 0.34}
- `C-choice-0073` / choice / 「明显有过,现在回想起来细节对不上,只…。另外2023年9月分开」 gold={'answer_class': 'yes', 'has_new_dated_event': True, 'intent': 'answer_current_focus'} jev={'intent': 'answer_current_focus', 'answer_class': 'unsure', 'has_new_dated_event': True, 'confidence': 0.46}
## T1 判据对照
见 `scripts/research/jev_intent_questions.py` 的 `CRITERION_MAP`。丢掉的语义:生产提示里的「通常」「不要按 A/B/C/D 猜」「不要按关键词表」;结构不变量改由 `enforce_combo` 强制。
| 现行提示原句 | Jev 落点 | 丢失 |
| --- | --- | --- |
| 结合当前问题和动态选项判断用户是在回答当前问题、提供新的带时间经历、要求停止整个校正、询问结果,还是语义不清。 | intent Choice with five literal criteria, including unclear as the residual option. | — |
| 若是在回答当前问题,answer_class 必须使用某个选项提供的 answer_class;否则 answer_class 必须为 null。 | answer_class Choice is built only from the live options; code nulls it when intent != answer_current_focus. | — |
| has_new_dated_event 仅在用户同一句里除了回答当前问题之外,还提供了新的、带大概时间的经历时为 true。 | has_new_dated_event Noul true/false criteria, same split. | — |
| 单纯的否定或单纯的选项回答必须为 false。 | Noul false: 单纯否定、单纯记不清不算新经历。 | — |
| 若同一句话既回答了当前问题又补充了新的带时间经历,intent 仍为 answer_current_focus,has_new_dated_event 为 true。 | intent and Noul are separate questions; code keeps answer_current_focus when both fire. | Jev does not itself enforce this pair; code does. |
| “当前方面没有、那段时间没有变化”通常是回答当前问题,不是停止整个流程。 | stop_rectification criterion: only an explicit stop of the whole flow. 当前方面没有 is answer_current_focus. | The hedge 通常 is dropped; the stop option is written as an exclusive literal. |
| 只有用户明确要求停止整个校正时才分类为 stop_rectification。 | stop_rectification criterion, same literal. | — |
| 不要按 A/B/C/D 的位置猜语义,只按选项 label 与 answer_class 判断。 | answer_class criteria keyed by answer_class, not by A/B/C/D. | The prohibition itself is not sent; Jev is not given A/B/C/D as the choice keys. |
| 「没有、没发生过、这方面没什么」→ intent 为 answer_current_focus,answer_class 为 no。 | Collect answer_class criterion for no. | — |
| 「记不清、不记得、忘了、想不起来、以后再说」→ answer_class 为 unsure。 | Collect answer_class criterion for unsure. | — |
| 用户用带大概年月的经历直接回答当前采集题 → intent 为 answer_current_focus,answer_class 为 yes(程度较弱时为 weak_yes),has_new_dated_event 为 false。 | Collect yes/weak_yes criteria + Noul false for a dated answer that is the collect answer. | — |
| 不要把「没有」或「记不清」标成 yes。 | Separate no and unsure criteria; yes requires a dated or affirmative answer. | The 'do not' wording is dropped. |
| 若既没有否定、也没有说记不清、也没有给出带年月经历,intent 为 unclear,answer_class 必须为 null。 | unclear residual on intent Choice; code nulls answer_class. | — |
| 若用户只在补充带时间的经历、并没有回答当前采集题,intent 为 provide_new_evidence。 | provide_new_evidence criterion. | — |
| 若同一句话既明确否定当前采集题又补充了新的带时间经历,intent 仍为 answer_current_focus 且 answer_class 为 no,不要改成 provide_new_evidence。 | Code keeps no + Noul true. Jev intent/Noul are independent. | Jev may split this pair; code does not re-vote intent from the Noul. |
| 不要按关键词表或正则猜测,只根据当前问题与用户这句话的语义分类。 | Not sent. Jev has no keyword table in the question. | The meta-instruction is dropped (Jev answers the written question, not the intended one). |
## 回退
任何上线方案必须保留回退到现行会话模型的路径。官方限流会动态调整。
@@ -70,3 +70,52 @@ Flash 自洽率:点选题 100%、采集 95.5%、无焦点 100%。中位延迟
- `python -m pytest tests/test_jev_intent_research.py`:8 passed
- 线上分类器文件未改
## 修复轮(2026-09-19,`codex/rectification-jev-intent-research-fix-20260919`)
任务书:`TASK-rectification-jev-intent-classifier-research-fix-20260919.md`。未改线上分类器。未立 BUG。
### 来源 C 重造
- 生成 / 复核模型:`deepseek-flash`(temperature 0.85 / 0.0),prompt_version `gen-review-v3`。不是线上会话模型。
- 问题池复用 `question_pool.json`(103 道点选题),未重建引擎池。
- 生成 1261,通过复核 1225,选出 900;争议 36,争议率 **2.85%**(collect 33 / choice 3;answer_current_focus 33 / unclear 3)。未回退模板。
- 提示修改:冒烟 v2 有 `missing_date` 拒稿,v3 强制四位年份并在重试时写明缺年/多年原因。只改了一轮提示。
- 三项统计:去重后唯一 825(≥810),单条重复最多 5;人设每种 ≥48(点选+采集),惜字如金+方言/网络语 180/900=20%;长度 P25=16 / 中位=20 / P75=24(目标 8–16 / 15–27 / 20–36)。
- 三层 400 / 400 / 100,配额偏差 ≤4.5%。
- sha256:synthetic `79d53a958b3b42ed18c79e3bbcff4b31deb22b65eedcd89b39326a7fefc9b0e8`;simulated `53f13012fdc8f3721335bbe2961edf8c506a2f705b62bd9d302fce83ce2ef88d`;disputed `e80c5cd69c521a099d743e4a4632ad6bc103c3b5a505f5b376a7c9cebedb7646`。
### Token(分列)
| 段 | 调用 | input tok | output tok |
| --- | ---: | ---: | ---: |
| 生成 DeepSeek Flash | 1389 | 553,685 | 18,677 |
| 复核 DeepSeek Flash | 1318 | 700,669 | 29,296 |
| Jev `jev-1.13.0` 两次(A+C 910×2 + B 157×2) | 2134 | 2,213,898 | 283,528(Jev 输出不计费) |
| 对照 Flash 全量一次 + C 1/3 第二次 + B 一次 | 1366 | 676,080 | 30,260 |
Jev 输入按 $0.042/Mtok ≈ $0.093。生成+复核+对照走 DeepSeek,不记 Jev 价。
### 来源 B
157 条全部人工标注(点选 17 + 采集 107 + 无焦点 33),约 40 分钟。`runtime_intent` 未抄成 gold。Jev×2 + Flash×1 已跑。代表性:无焦点层 intent 差 17.2pp > 10pp。
### 对照摘要
| 层 | Jev intent | Flash intent | 高置信错误 | 低置信召回 | Jev 自洽 |
| --- | ---: | ---: | ---: | ---: | ---: |
| 点选题 | 99.0% | 98.8% | 0.0% | 23.1% | 99.8% |
| 采集题 | 92.2% | 97.5% | 2.8% | 45.5% | 98.0% |
| 无焦点 | 96.0% | 94.0% | 1.0% | 25.0% | 100% |
cache 样本 id 与 `simulated.jsonl` 一致(900/900)。unavailable = 0。
### 结论
**缺数据**。无焦点层来源 B vs C 差 17.2pp;同时低置信召回未过 60%。不得上线。原「不可接」不沿用。
### 验证
- `python -m pytest tests/test_jev_intent_research.py -q`:11 passed
- 构建脚本无 `bank = {`;复核函数无 `re.search` 判标签
- 线上分类器文件未改
+1 -1
View File
@@ -234,7 +234,7 @@
| `TASK-rectification-open-collect-invite-20260914.md` | `PROGRESS-rectification-open-collect-invite-20260914.md` | **P0**:固定七条采集线问完后只说「能问的都问完了」,用户不知道还能补经历、也不知道补了有用;而两轮研究证明补带年月经历是唯一有效手段。产品拍板:交付卡照出 + 卡上给不限领域的补充邀请(先要确切日期,再退年月;举七条线之外的例子),补完必须可见生效(BUG-689) | 待验收 | `codex/rectification-open-collect-invite-20260914` |
| `TASK-rectification-cluster-width-research-20260914.md` | `PROGRESS-rectification-cluster-width-research-20260914.md` | **研究单**:上一轮证明调权重改不动交付区间宽度——所有方案宽度中位数都等于整个搜索窗。先确认 sweep 的宽度口径是否含淘汰(M0),再画簇结构像(M1),最后量三个改法:放宽簇上限、按分差决定是否合并、交付区间改分位覆盖(M2)。真值覆盖率不得下降 | 待验收 | `codex/rectification-cluster-width-research-20260914` |
| `TASK-rectification-jev-intent-classifier-research-20260919.md` | `PROGRESS-rectification-jev-intent-classifier-research-20260919.md` | **研究单**:TypeSafe Jev(只做 Choice/Score/Noul 的校准判断模型,$0.042/Mtok)能否接管校正流的意图分类。产品 09-19 授权评估(推翻 09-15「分类只用贵模型」需重新拍板)。Agent 模拟校正流造 ≥900 条中文语料(标签先定、独立复核)+ 真机样本做代表性锚,量准确率 / 高置信错误率 / 低置信召回 / 延迟;只离线测,不改线上 | 待验收 | `codex/rectification-jev-intent-classifier-research-20260919`(结论 **不可接**) |
| `TASK-rectification-jev-intent-classifier-research-20260919.md` | `PROGRESS-rectification-jev-intent-classifier-research-20260919.md` | **研究单**:TypeSafe Jev(只做 Choice/Score/Noul 的校准判断模型,$0.042/Mtok)能否接管校正流的意图分类。产品 09-19 授权评估(推翻 09-15「分类只用贵模型」需重新拍板)。Agent 模拟校正流造 ≥900 条中文语料(标签先定、独立复核)+ 真机样本做代表性锚,量准确率 / 高置信错误率 / 低置信召回 / 延迟;只离线测,不改线上 | **待验收**(修复轮已交,结论 **缺数据**) | 修复单 `TASK-rectification-jev-intent-classifier-research-fix-20260919.md` 已做完。来源 C 由 DeepSeek Flash 生成+复核(900 条,争议率 2.85%);来源 B 157 条已人工标注。Jev 无焦点层 vs 真人差 17.2pp > 10pp。落点分支 `codex/rectification-jev-intent-research-fix-20260919` |
| `TASK-rectification-minute-resolution-research-20260914.md` | `PROGRESS-rectification-minute-resolution-research-20260914.md` | **研究单**:候选分不开的根因是打分尺度——窗口内恒定项 11.5 分 vs 随分钟变化项 2.125 分(≈5:1)。先修封存基准(v3 每例仅 3 件事且被标 invalidated)出 v4,再离线量五个改法:分盘除数、去底座、**KP 宫头子主计分(产品 09-14 拍板,推翻 BUG-325 一条红线)**、年精度事件改边际似然、聚类签名层对齐。有收益才立实现单 | 待验收 | `codex/rectification-minute-resolution-research-20260914` |
@@ -0,0 +1,122 @@
# TASK · Jev 意图分类研究修复单:来源 C 必须由真实模型生成与复核(2026-09-19)
- 基线:`origin/staging` @ `d6fc4fb8`(含 `fde541c2` 研究首版 + `d6fc4fb8` DeepSeek Flash 对照)
- 原单:`docs/tasks/TASK-rectification-jev-intent-classifier-research-20260919.md`
- 分支:`codex/rectification-jev-intent-research-fix-20260919`(从 `origin/staging` 新建)
- 性质:研究单修复;不改线上代码;不立 BUG(无线上缺陷)
## 1. 验收结论(对照原单逐条)
| 项 | 结论 | 证据 |
| --- | --- | --- |
| 红线 1 · 不改线上 | 通过 | `git diff --stat d9722d56..d6fc4fb8` 只有 `scripts/research/**`、`tests/test_jev_intent_research.py`、`docs/**`、`.gitignore`、`BLOCKED.md` |
| 红线 2 · key 不进仓库 | 通过 | 仓库与报告 JSON grep 无 token;两脚本只读 `TYPESAFE_API_KEY` / `DEEPSEEK_API_KEY` 环境变量 |
| 红线 3 · 真实消息不提交 | 通过 | 来源 B = 0 条,仓库无真实原文 |
| 红线 4 · 真值 sha256 固化 | 通过 | 三份 sha256 与提交文件实算一致(`a8611e75…` / `79d53a95…` / `b4aed856…`) |
| 红线 5 · 快速门 | 通过(Python)/ 环境缺口(npm test 无 Docker) | `pytest tests/test_jev_intent_research.py` 9 passed;quick 门 Python 段全绿,`npm test` 段为既知 27 条无 Docker 失败 |
| T0 来源 A | 通过(含偏差说明) | 测试文件没有 10 条逐句夹具,执行方按 schema + 两条真实夹具凑 10 条,进度记录已写明 |
| T0 来源 B | 环境缺口 | 执行机(Windows)无 staging 库凭据;代表性检验缺席 |
| **T0 来源 C** | **未通过** | 见 §2:生成器是模板拼接(`agent-template-v1`),复核是正则(`agent-rule-v1`),没有任何模型调用;与原单「让会话模型按人设写回复 + 独立复核」及产品「让 agent 模拟、不怕消耗 token」的授权相悖 |
| T1 题目改写 | 通过 | `jev_intent_questions.py` 固定 `jev-1.13.0`、`CRITERION_MAP` 逐条对应提示词、`enforce_combo` 在代码里保证组合合法 |
| T2 对照跑 | 通过(Jev)/ 偏差(现行) | Jev 来源 A+C 各两次;现行分类器用 `deepseek-flash` 顶生产提示词、来源 C 抽 33%,**不是线上会话模型**(模型目录在数据库,代码里无默认值) |
| T3 指标与结论 | **未通过** | 指标算法本身没问题,但输入语料不成立,「不可接」结论无效;正确结论应为**缺数据** |
| T4 记录 | 通过 | PROGRESS / BLOCKED / README 状态行齐全;未立 BUG 正确 |
## 2. 事故实证(`scripts/research/jev_intent_corpus_build.py` @ d6fc4fb8)
1. `GENERATOR_NAME = "agent-template-v1"`、`REVIEWER_NAME = "agent-rule-v1"`(文件头常量)。全文件无 http / SDK / Agent 调用;`import` 只有标准库。
2. 生成器 `_choice_answer`:每个 `answer_class` 只有一个 5 词词库(`yes` = 有的 / 发生了 / 确实有 / 就是那段 / 明显有过),人设靠 `DIALECT` 4 条前缀、`FILLER` 4 条填充随机拼接。`_collect_answer` 同理。
3. 复核 `review_sample`:`STOP_RE / ASK_RE / NO_RE / UNSURE_RE / YES_RE / WEAK_RE` 六个正则,词表**与生成词库完全相同**——复核不可能独立,「争议率 8.5%」只是模板与正则的自洽度,不是标签质量。
4. 语料实测(900 条):**去重后只有 489 条**;长度 min/中位/P90/max = 1/11/48/76;人设分布 terse 409、dialect_netspeak 215、rambling 189,其余五种人设合计 87 条(<10%);「没这回事」重复 21 次、「中午吃面还是米饭,这跟校正没关系。」重复 17 次。
5. 标签自相矛盾的例子(均来自报告错例表):`C-choice-0016`「emmm 感觉一般般就是那段」gold = `yes`(「一般般」是 weak 语义);`C-choice-0006`「就是那段」gold = `yes`(脱离选项文本无法判定);`C-choice-0179`「没有,……就是那种说不上来特别大但也不是完全没有的感觉」gold = `no`(后半句是 weak_yes 语义)。Jev 在这些条目上「答错」不能计为 Jev 的错。
6. 报告 §T3「高置信错误 7.0%」「低置信召回 34%」「answer_class 65%」三个决定结论的数字,全部建立在上述语料之上。
## 3. 根因
执行方在「本机无会话模型凭据」时,选择用模板+正则顶替 Agent 生成,而不是按让步 4 停在 T0/T1 回报。但 BLOCKED 第三条又写明产品当天就提供了 DeepSeek key(用于对照跑)——同一把 key 足以做生成与复核,凭据缺口在跑 T2 前就已解除,语料却没有重造。
## 4. 决策记录
1. 产品 2026-09-19 原话:「能不能让 agent 模拟生时流程来得语料……不怕消耗 token」。这是来源 C 的授权来源,**模板拼接不满足**;本单据此要求全量重造来源 C。
2. 生成与复核模型:用执行机上已有凭据的真实对话模型(DeepSeek 或产品另行提供的任一模型),**生成与复核可用同一家模型但提示不同、复核看不到标签**(原单已允许)。若能用两家不同模型更好,不强求。
3. 现行分类器对照:产品 2026-09-19 确认**线上现行模型就是 DeepSeek Flash**。原报告的对照模型身份因此成立,偏差只剩「抽 33%」;本单要求全量(红线 8),报告改标「= 线上会话模型」。
4. 来源 B:产品 2026-09-19 授权,Claude 会话已从 staging 只读抽取并去重脱敏,文件 `source_b.jsonl`(157 条,不入仓)由产品转交执行方,放到 `.cache/jev_intent/source_b.jsonl`(已 gitignore)。抽取口径:`agentic_rectification_turns.user_message` 非空 697 行 → 剔除点选回显(`A. …` / 「先这样」)339 行 → 剔除前端建议芯片原文 76 行 → 按消息文本去重 → 157 条;每条带当轮焦点(按 `asked_at ≤ turn.created_at` 取最近一条,`focus_stale` 标记焦点已在本轮前 resolved 的 22 条)与可反推的 `runtime_intent`(85 条:no 49 / weak_yes 18 / yes 13 / unsure 5)。`gold` 留空,执行方逐条人工标注。
来源 B 实测分布(**来源 C 生成的长度约束以此为准,取代原单兜底值**):
| 指标 | 值 |
| --- | --- |
| 条数 / 层 | 157:采集 107、无焦点 33、点选 17 |
| 字数 P25 / 中位 / P75 / 最长 | 12 / 21 / 28 / 172 |
| 含标点比例 | 13% |
| 含语气词(吧/呢/啊/嗯/哦/额/emm)比例 | 4% |
| 含年份或月份比例 | 81% |
对照模板语料(中位 11 字、惜字如金 45%、方言前缀 24%):真人回复更长、几乎不用语气词、绝大多数直接报年月。生成提示的人设权重必须按此校正:「惜字如金」与「方言/网络语」两种人设合计不得超过 20%。
5. 原报告结论「不可接」**撤回改为「缺数据」**,`docs/tasks/README.md` 状态行同步改;不得保留「不可接」字样误导后续拍板。
## 5. 硬红线
原单 §4 五条全部继续生效,另加:
6. `jev_intent_corpus_build.py` 的生成与复核**必须各有一次真实模型调用**并把模型名、版本、提示原文写进 `scripts/research/jev_intent_samples/README.md`;`GENERATOR_NAME` / `REVIEWER_NAME` 必须是模型标识,不得再出现 `template` / `rule`。`tests/test_jev_intent_research.py` 新增断言:构建脚本中不得存在把 `answer_class` 映射到固定词表的生成函数(检查源码里无 `bank = {` 式字面量词库),复核函数不得调用 `re.search` 判标签。
7. 来源 C 去重后唯一条数 ≥ 810(≥ 90%);单条重复不得超过 5 次;八种人设每种 ≥ 40 条(点选+采集合计),且「惜字如金」+「方言/网络语」合计 ≤ 20%;字数 P25 / 中位 / P75 落在来源 B 实测值 ±30% 内(即 P25 8–16、中位 15–27、P75 20–36)。不满足即重跑,不得进 T2。
8. 现行分类器(DeepSeek Flash,= 线上)对照不得再抽样:来源 B 与来源 C **全量**各跑一次(自洽率第二次可抽 1/3,与原单让步 2 一致)。
## 6. 任务分解
### F1 · 重写 `jev_intent_corpus_build.py` 的生成与复核
- 问题池 `question_pool.json` 保留(来源是真实引擎,已通过),不重建。
- 生成:对每道题 × 目标标签 × 人设,调用模型写回复。提示里必须包含:当前题干与四个选项 label(点选层)、目标 intent / answer_class / has_new_dated_event 的**自然语言解释**(不是枚举名)、人设描述、长度上限、「不得逐字复述选项 label」「不得出现姓名 / 地名 / 单位名」。温度 ≥ 0.8。同一(题, 标签, 人设)组合最多重试 3 次以通过复核。
- 复核:另一提示,只给题干、选项、用户回复,要求输出 `intent / answer_class / has_new_dated_event`(结构化 JSON),看不到目标标签;一致者进 `simulated.jsonl`,不一致进 `disputed.jsonl`(保留生成标签与复核标签两列)。
- 争议率按层与按标签写进 README;> 15% 先改生成提示再重跑(最多两轮),仍超则按实际通过条数跑并如实报。
- 缓存:每次模型调用结果落 `.cache/jev_intent/`(已 gitignore),脚本可断点续跑。
- 验收:红线 7 的三项统计由脚本打印并写入 README;`simulated.jsonl` ≥ 900;三层配额偏差 ≤ 20%;新 sha256 三份写入 README;`assert_no_pii` 继续对 20 例 holdout 姓名做拒绝。
### F2 · 重跑 T2
- Jev `jev-1.13.0` 来源 A + 新来源 C 各两次(原单 T2)。
- 现行分类器(红线 8)来源 C 全量一次 + 1/3 抽样第二次;模型名按 §4 第 3 条。
- 验收:`.cache/jev_intent/jev_runs.json` 与现行 runs 的样本 id 集合与 `simulated.jsonl` 完全一致(脚本打印校验);429 记 `unavailable` 计入分母。
### F3 · 重出报告
- `docs/research/jev_intent_2026_09_19.md/.json` 覆盖重写(保留旧版为 `jev_intent_2026_09_19_v1_template.md`,文首标「已作废:语料为模板拼接」)。
- 结论三选一:可接 / 不可接 / 缺数据,附 T3 全表、θ 曲线、错例画像(改写后)。来源 B 仍为 0 时,结论最高只能给到「可接(待真机代表性检验)」或「不可接」,不得写成无条件可接。
- 验收:报告「模型」一节列出生成模型、复核模型、对照模型三行各自的模型名 / 版本 / 是否线上会话模型。
### F4 · 来源 B
- 文件已交付(§4 第 4 条)。执行方逐条人工标注全部 157 条 `gold`;`runtime_intent` 只做参考列,不得直接抄为 gold。
- 跑 Jev 两次 + 现行一次,做原单 T3 代表性检验:来源 B 与来源 C 同层准确率差 > 10pp → 结论降为「缺数据」并写明是哪一层。
- 验收:README 来源 B 段填实际条数、三层分布、长度分布、标注耗时;BLOCKED 来源 B 条目划掉。
### F5 · 记录
- `PROGRESS-rectification-jev-intent-classifier-research-20260919.md` 追加「修复轮」段:三项统计、争议率、两轮提示修改(若有)、token 消耗(生成 / 复核 / Jev / 对照分列)。
- `BLOCKED.md`:「无会话模型凭据」条目改写为已解除(生成与复核已用真实模型);来源 B 条目按实际。
- `docs/tasks/README.md`:状态行改为本单结论。
- 不立 BUG。
## 7. 让步顺序
1. 争议率两轮提示后仍 > 15%:按实际通过条数跑,配额偏差如实报;**不得**回退到模板生成补数。
2. 单条生成三次重试仍不过复核:丢弃该组合,换人设重抽;红线 7 三项仍须满足。
3. 模型调用限流:退避重试,脚本断点续跑;不得缩小样本。
4. 来源 B 人工标注做不完 157 条:至少标满点选 17 + 采集 60 + 无焦点 20 = 97 条,其余标 `unlabeled` 不计入分母;不得用模型代标。
## 8. 开工前置命令
```bash
git fetch origin --prune
git worktree add -b codex/rectification-jev-intent-research-fix-20260919 .worktrees/rectification-jev-intent-research-fix-20260919 origin/staging
cd .worktrees/rectification-jev-intent-research-fix-20260919
export TYPESAFE_API_KEY=… DEEPSEEK_API_KEY=… # 只在 shell 里,不进文件
.venv/bin/python -m pytest tests/test_jev_intent_research.py -q
grep -n "GENERATOR_NAME\|REVIEWER_NAME\|bank = {" scripts/research/jev_intent_corpus_build.py # 改前留证
```
## 9. BUG 编号
本单不立 BUG;`docs/BUG_HISTORY.md` 当前最大号 BUG-969,若执行中发现现行分类器的确定性错误再从 BUG-970 起。
File diff suppressed because it is too large Load Diff
+123 -32
View File
@@ -8,6 +8,7 @@ Source B rows are aggregated and never written as raw text.
from __future__ import annotations
import argparse
import hashlib
import json
import os
import sys
@@ -320,6 +321,12 @@ def error_portrait(rows: Sequence[Mapping[str, Any]], *, limit: int = 3) -> dict
return buckets
def cache_key(run_id: str, sample: Mapping[str, Any]) -> str:
message = str(sample.get("user_message") or "")
digest = hashlib.sha256(message.encode("utf-8")).hexdigest()[:16]
return f"{run_id}:{sample['id']}:{digest}"
def run_batch(
samples: Sequence[Mapping[str, Any]],
*,
@@ -331,7 +338,7 @@ def run_batch(
caller = call_fn or call_jev_retry
pending = []
for sample in samples:
key = f"{run_id}:{sample['id']}"
key = cache_key(run_id, sample)
if key in cache:
sample[run_id] = cache[key]
else:
@@ -346,14 +353,14 @@ def run_batch(
sample = futures[future]
result = future.result()
sample[run_id] = result
cache[f"{run_id}:{sample['id']}"] = result
cache[cache_key(run_id, sample)] = result
done += 1
if done % 50 == 0 or done == len(pending):
print(f" {run_id} {done}/{len(pending)}", flush=True)
def load_cache() -> dict[str, Any]:
path = CACHE_DIR / "jev_runs.json"
path = CACHE_DIR / "jev_runs_v2.json"
if not path.is_file():
return {}
try:
@@ -364,7 +371,7 @@ def load_cache() -> dict[str, Any]:
def save_cache(cache: dict[str, Any]) -> None:
CACHE_DIR.mkdir(parents=True, exist_ok=True)
path = CACHE_DIR / "jev_runs.json"
path = CACHE_DIR / "jev_runs_v2.json"
path.write_text(json.dumps(cache, ensure_ascii=False), encoding="utf-8")
@@ -395,9 +402,15 @@ def write_markdown(report: Mapping[str, Any]) -> None:
f"- 任务:`docs/tasks/TASK-rectification-jev-intent-classifier-research-20260919.md`",
f"- 基线:`origin/staging` @ `{report['meta']['baseline']}`",
f"- Jev 模型:`{JEV_MODEL}`(不用 jev-latest)",
f"- 生成器 / 复核:`{report['meta']['generator']}` / `{report['meta']['reviewer']}`",
f"- 真值 sha256:synthetic `{report['meta']['sha256']['synthetic']}`;simulated `{report['meta']['sha256']['simulated']}`;disputed `{report['meta']['sha256']['disputed']}`",
f"- 来源 B:{report['meta']['source_b_n']} 条(本地,不提交原文)",
f"- 来源 B:{report['meta']['source_b_n']} 条已标注(本地,不提交原文)",
"",
"## 模型",
"",
f"- 生成模型:`{report['meta']['generator']}` / 版本 `{report['meta'].get('generator_version') or report['meta']['generator']}` / 不是线上会话模型(只用于造来源 C)",
f"- 复核模型:`{report['meta']['reviewer']}` / 版本 `{report['meta'].get('reviewer_version') or report['meta']['reviewer']}` / 不是线上会话模型(只用于独立复核,看不到目标标签)",
f"- 对照模型:`{report['meta'].get('current_model') or '—'}` / 版本 `{report['meta'].get('current_model') or '—'}` / **= 线上会话模型**(DeepSeek Flash,顶生产 `classifyRectificationTurnIntent` 提示词)",
f"- Jev:`{JEV_MODEL}` / 不是线上会话模型",
"",
"## 结论",
"",
@@ -535,10 +548,9 @@ def decide_verdict(report: dict[str, Any]) -> dict[str, str]:
low_recall = [layers[layer]["low_conf_recall"] for layer in layers if layers[layer]["low_conf_recall"] is not None]
recall_ok = bool(low_recall) and all(v is not None and v >= 0.60 for v in low_recall)
choice_class = layers.get("choice", {}).get("answer_class_acc")
if source_b_n < 30:
represent = "来源 B 不足 30 条,代表性检验只报数、不判定。"
else:
represent = report["metrics"]["representativeness"]["note"]
represent_meta = report["metrics"].get("representativeness") or {}
represent = represent_meta.get("note") or ""
represent_fail = bool(represent_meta.get("fail"))
absolute_fail: list[str] = []
if high_err > 0.03:
absolute_fail.append(f"高置信错误率 {high_err:.1%} > 3%")
@@ -563,6 +575,17 @@ def decide_verdict(report: dict[str, Any]) -> dict[str, str]:
)
if bits:
relative_note = " 相对 −3pp(同一样本):" + ";".join(bits) + "。"
if represent_fail:
return {
"verdict": "缺数据",
"reason": (
"来源 B 与来源 C 同层 intent 准确率差 > 10pp,模拟语料不代表真人,来源 C 门槛结论降为缺数据。"
+ represent
+ ((" 同时来源 C 绝对门槛未过:" + ";".join(absolute_fail) + "。") if absolute_fail else "")
+ relative_note
),
"if_connect": "不得上线。先补真机样本或重造更像真人的来源 C,再测。",
}
if absolute_fail:
return {
"verdict": "不可接",
@@ -628,6 +651,8 @@ def _merge_report_preds(samples: list[dict[str, Any]]) -> None:
by_id = {row.get("id"): row for row in existing.get("rows") or [] if row.get("id")}
for sample in samples:
prior = by_id.get(sample.get("id")) or {}
if prior.get("user_message") != sample.get("user_message"):
continue
for key in ("jev_1", "jev_2", "current_1", "current_2"):
if key not in sample and prior.get(key):
sample[key] = prior[key]
@@ -666,6 +691,7 @@ def main(argv: Sequence[str] | None = None) -> int:
parser.add_argument("--second-fraction", type=float, default=1.0)
parser.add_argument("--current-only", action="store_true")
parser.add_argument("--sample-fraction", type=float, default=1.0)
parser.add_argument("--current-second-fraction", type=float, default=1.0 / 3)
parser.add_argument("--sample-seed", type=int, default=20260919)
args = parser.parse_args(argv)
if not args.current_only and not os.environ.get("TYPESAFE_API_KEY"):
@@ -676,8 +702,14 @@ def main(argv: Sequence[str] | None = None) -> int:
return 2
synthetic = load_jsonl(SAMPLES_DIR / "synthetic.jsonl")
simulated = load_jsonl(SAMPLES_DIR / "simulated.jsonl")
source_b = load_jsonl(CACHE_DIR / "source_b.jsonl")
source_b_all = load_jsonl(CACHE_DIR / "source_b.jsonl")
source_b = [
row for row in source_b_all
if isinstance(row.get("gold"), dict) and row["gold"].get("intent")
]
unlabeled_b = len(source_b_all) - len(source_b)
samples = [row for row in synthetic + simulated if row.get("source") in {"A", "C"}]
sim_ids = {str(row.get("id")) for row in simulated}
if args.limit:
samples = samples[: args.limit]
_merge_report_preds(samples)
@@ -699,6 +731,8 @@ def main(argv: Sequence[str] | None = None) -> int:
if source_b:
run_batch(source_b, workers=args.workers, run_id="jev_1", cache=cache)
save_cache(cache)
run_batch(source_b, workers=args.workers, run_id="jev_2", cache=cache)
save_cache(cache)
source_c_rows = [row for row in samples if row.get("source") == "C"]
if args.current_only or os.environ.get("DEEPSEEK_API_KEY"):
if args.sample_fraction < 1:
@@ -713,9 +747,21 @@ def main(argv: Sequence[str] | None = None) -> int:
cache=cache, call_fn=call_current_retry,
)
save_cache(cache)
if source_b:
run_batch(
source_b, workers=args.workers, run_id="current_1",
cache=cache, call_fn=call_current_retry,
)
save_cache(cache)
second_current = current_sample
if args.skip_second:
second_current = []
elif args.current_second_fraction < 1:
second_current = stratified_sample(
[row for row in current_sample if row.get("source") == "C"],
fraction=args.current_second_fraction,
seed=args.sample_seed + 1,
)
elif args.second_fraction < 1:
second_current = stratified_sample(
[row for row in current_sample if row.get("source") == "C"],
@@ -752,47 +798,87 @@ def main(argv: Sequence[str] | None = None) -> int:
jev_cons = {layer: self_consistency(rows, "jev_1", "jev_2") for layer, rows in by_layer.items()}
source_a_metrics = layer_metrics(source_a, pred_key="jev_1") if source_a else None
source_b_metrics = layer_metrics(source_b, pred_key="jev_1") if source_b else None
represent_note = "来源 B 不足 30 条,代表性检验只报数、不判定。"
if source_b_metrics and source_b_metrics["n"] >= 30:
deltas = []
source_b_by_layer = {
layer: layer_metrics([row for row in source_b if row.get("layer") == layer], pred_key="jev_1")
for layer in ("choice", "collect", "none")
} if source_b else {}
represent_fail = False
represent_layers: dict[str, Any] = {}
if not source_b or len(source_b) < 30:
represent_note = f"来源 B 已标注 {len(source_b)} 条(未标注 {unlabeled_b} 不进分母),不足 30 条则只报数、不判定。"
else:
bits = []
for layer in ("choice", "collect", "none"):
# source B may not be layered the same; compare overall intent acc
pass
delta = abs((source_b_metrics["intent_acc"] or 0) - (
sum(jev_metrics[layer]["intent_acc"] * jev_metrics[layer]["n"] for layer in jev_metrics)
/ max(sum(jev_metrics[layer]["n"] for layer in jev_metrics), 1)
))
if delta > 0.10:
represent_note = f"来源 B 与来源 C 的 intent 准确率相差 {delta:.1%} > 10pp,模拟语料不代表真人,来源 C 门槛结论降为缺数据。"
b_m = source_b_by_layer.get(layer) or {}
c_m = jev_metrics.get(layer) or {}
if not b_m.get("n"):
represent_layers[layer] = {"n": 0, "delta": None}
continue
delta = abs((b_m.get("intent_acc") or 0) - (c_m.get("intent_acc") or 0))
represent_layers[layer] = {
"n": b_m.get("n"),
"source_b_intent": b_m.get("intent_acc"),
"source_c_intent": c_m.get("intent_acc"),
"delta": delta,
}
bits.append(f"{layer} B {b_m.get('intent_acc'):.1%} vs C {c_m.get('intent_acc'):.1%}(差 {delta:.1%},n_B={b_m.get('n')})")
if delta > 0.10:
represent_fail = True
represent_note = "来源 B 与来源 C 同层 intent 准确率:" + ";".join(bits) + "。"
if represent_fail:
represent_note += " 有层差 > 10pp,结论降为缺数据。"
else:
represent_note = f"来源 B 与来源 C intent 准确率相差 {delta:.1%},未超过 10pp。"
represent_note += " 各层均未超过 10pp。"
sha = {
"synthetic": __import__("hashlib").sha256((SAMPLES_DIR / "synthetic.jsonl").read_bytes()).hexdigest(),
"simulated": __import__("hashlib").sha256((SAMPLES_DIR / "simulated.jsonl").read_bytes()).hexdigest(),
"disputed": __import__("hashlib").sha256((SAMPLES_DIR / "disputed.jsonl").read_bytes()).hexdigest()
if (SAMPLES_DIR / "disputed.jsonl").is_file() else "",
}
gen_name = (simulated[0].get("generator") if simulated else None) or "deepseek-flash"
rev_name = (simulated[0].get("reviewer") if simulated else None) or "deepseek-flash"
cache_ids = {
key.split(":")[1]
for key in cache
if key.startswith("jev_1:C-")
}
sim_ok = sim_ids <= cache_ids or args.current_only
print(json.dumps({
"id_check": {
"simulated": len(sim_ids),
"jev_1_cache_c": len(cache_ids),
"simulated_subset_of_cache": sim_ids <= cache_ids,
"missing": sorted(sim_ids - cache_ids)[:12],
}
}, ensure_ascii=False), flush=True)
report: dict[str, Any] = {
"meta": {
"baseline": "d9722d56",
"baseline": "69a44fe7",
"jev_model": JEV_MODEL,
"generator": "agent-template-v1",
"reviewer": "agent-rule-v1",
"generator": gen_name,
"generator_version": gen_name,
"reviewer": rev_name,
"reviewer_version": rev_name,
"sha256": sha,
"source_b_n": len(source_b),
"source_b_unlabeled": unlabeled_b,
"current_model": current_model_id() if any(row.get("current_1") for row in samples) else None,
"current_model_note": (
(
f"DeepSeek Flash 顶现行 `classifyRectificationTurnIntent` 提示词;"
f"来源 C 分层随机 {args.sample_fraction:.0%}(seed {args.sample_seed}),"
f"来源 A 全量。不是线上会话模型。"
"DeepSeek Flash = 线上会话模型,顶现行 `classifyRectificationTurnIntent` 提示词;"
f"来源 C 全量第一次"
+ (
f",第二次分层 {args.current_second_fraction:.0%}(seed {args.sample_seed + 1})"
if args.current_second_fraction < 1 else ",第二次全量"
)
+ ";来源 A 全量;来源 B 已标注全量一次。"
if any(row.get("current_1") for row in samples)
else (
"本机无会话模型目录凭据(模型 key 在数据库加密配置里)。"
"现行 `classifyRectificationTurnIntent` 对照未跑。"
"本机无会话模型凭据,现行对照未跑。"
)
)
),
"id_check_ok": sim_ok,
},
"metrics": {
"jev_by_layer": jev_metrics,
@@ -817,7 +903,12 @@ def main(argv: Sequence[str] | None = None) -> int:
"paired_sample": paired_layer_metrics(samples) if any(row.get("current_1") for row in samples) else {},
"source_a": source_a_metrics,
"source_b": source_b_metrics,
"representativeness": {"note": represent_note},
"source_b_by_layer": source_b_by_layer,
"representativeness": {
"note": represent_note,
"fail": represent_fail,
"layers": represent_layers,
},
"theta": curve_theta([row for row in samples if row.get("source") == "C"]),
"error_portrait": error_portrait(samples),
},
+98 -11
View File
@@ -34,20 +34,107 @@ limit 500;
```
- 本地路径(worktree 外,已 gitignore):`G:/Ferti/Jyotisha/.cache/jev_intent/source_b.jsonl`
- 本机抽取条数:0(无 staging 库凭据则为 0,长度分布用任务书兜底 P25≤6 / 中位≤15 / P75≤40)
- 真值:执行方逐条人工标注;能从后续动作反推的另存 `runtime_intent` 列。**不得提交原文。**
- 本机抽取条数:157(点选 17 / 采集 107 / 无焦点 33),gold 已全部人工标注,未用模型代标
- 长度:P25=12 / 中位=21 / P75=28 / 最长=172
- 标注耗时:约 40 分钟(`runtime_intent` 只作参考,85 条有反推 intent,不抄成 gold)
- 真值:执行方逐条人工标注。**不得提交原文。**
## 来源 C · Agent 模拟语料(提交,主测试集)
## 来源 C · 模型模拟语料(提交,主测试集)
`simulated.jsonl`。问题池 `question_pool.json`:20 例公开 AA holdout 走 `discriminating_event_probes` 出点选题;采集题用 `USER_COLLECT_QUESTION` / retry / `TARGETED_YEAR_PROMPT`。
`simulated.jsonl`。问题池 `question_pool.json`:20 例公开 AA holdout 走 `discriminating_event_probes` 出点选题;采集题用 `USER_COLLECT_QUESTION` / retry / `TARGETED_YEAR_PROMPT`。生成与复核各一次真实模型调用;复核看不到目标标签。
- 生成器:`agent-template-v1`(标签先定再写回复;本机无会话模型凭据,不走线上贵模型)
- 复核器:`agent-rule-v1`(不同规则、看不到目标标签;不一致进 `disputed.jsonl`,不用第三次投票)
- 生成统计:{"generated": 1328, "accepted": 900, "disputed": 113, "dispute_rate": 0.08509036144578314, "generator": "agent-template-v1", "reviewer": "agent-rule-v1", "seed": 20260919}
- 配额:{"choice": {"actual": 400, "target": 400, "max_share_delta": 0.0, "ok": true}, "collect": {"actual": 400, "target": 400, "max_share_delta": 0.0, "ok": true}, "none": {"actual": 100, "target": 100, "max_share_delta": 0.0, "ok": true}}
- 长度:{"n": 900, "p25": 3.0, "median": 11.0, "p75": 25.0}
- sha256 simulated: `a8611e75bf56024fa984fb204d34a5e16405f18b3eaa5a3bee91f96e43918bf7`
- sha256 disputed: `b4aed8567bd6b3fdd5cc98c73d9139197d62a96120a0ee29f474b8f08c050def`
- 生成模型:`deepseek-flash`(temperature 0.85,prompt_version `gen-review-v3`)
- 复核模型:`deepseek-flash`(temperature 0.0,同一家模型、不同提示、看不到标签)
- 生成统计:{"generated": 1261, "accepted_raw": 1225, "accepted": 900, "disputed": 36, "dispute_rate": 0.028548770816812053, "generator": "deepseek-flash", "reviewer": "deepseek-flash", "prompt_version": "gen-review-v3", "seed": 20260919, "usage": {"generate_input_tokens": 553685, "generate_output_tokens": 18677, "review_input_tokens": 700669, "review_output_tokens": 29296, "generate_calls": 1389, "review_calls": 1318, "generate_cached": 53, "review_cached": 79}, "reselect_only": true}
- 争议分层:{"by_layer": {"collect": 33, "choice": 3}, "by_intent": {"answer_current_focus": 33, "unclear": 3}}
- 配额:{"choice": {"actual": 400, "target": 400, "max_share_delta": 0.045, "ok": true}, "collect": {"actual": 400, "target": 400, "max_share_delta": 0.0325, "ok": true}, "none": {"actual": 100, "target": 100, "max_share_delta": 0.0, "ok": true}}
- 长度:{"n": 900, "p25": 16.0, "median": 20.0, "p75": 24.0, "min": 6.0, "max": 39.0}
- 人设:{"choice_collect": {"rambling": 371, "answer_then_event": 94, "deny_then_event": 49, "terse": 51, "impatient_stop": 57, "ask_result": 51, "off_topic": 48, "dialect_netspeak": 79}, "all": {"rambling": 407, "answer_then_event": 94, "deny_then_event": 49, "terse": 71, "impatient_stop": 63, "ask_result": 54, "off_topic": 53, "dialect_netspeak": 109}, "each_ge_40": true, "terse_dialect_share_all": 0.2, "terse_dialect_share_choice_collect": 0.1625, "min_choice_collect": 48}
- 去重:{"n": 900, "unique": 825, "max_repeat": 5, "unique_ok": true, "repeat_ok": true}
- sha256 simulated: `53f13012fdc8f3721335bbe2961edf8c506a2f705b62bd9d302fce83ce2ef88d`
- sha256 disputed: `e80c5cd69c521a099d743e4a4632ad6bc103c3b5a505f5b376a7c9cebedb7646`
- 问题池条数(点选题):103
### 生成提示原文
系统:`你在模拟一位真人用户,正在做出生时间校正问答。只输出用户会打的那句话,不要解释,不要加引号,不要输出 JSON。`
用户提示模板示例:
```
当前问题:
2023 年前后,工作状态是否出现明显变化?
选项:
- A. 这段时间没有明显变化(对应态度:no)
- B. 记不清当时的情况(对应态度:unsure)
- C. 变化明显而且时间吻合(对应态度:yes)
- D. 有变化但程度比较弱(对应态度:weak_yes)
你必须体现的语义(不要写出这些标签名,不要解释标签):
- 这句话的作用:这句话是在直接回答当前这道题,不是另起一件无关的事,不是要求停止整套校正,也不是在问结果。
- 对当前问题的态度:明确否定当前问题:没有发生、没有变化、这方面没什么。不要再补一句「其实也有一点」。
- 是否另说一件带年月的新经历:不要另说一件带年份或月份的新经历。若当前是采集题且态度是肯定或弱肯定,可以在回答里带上这件事本身的年月,那是在答当前题,不是另说一件;这种情况下也必须出现四位数字年份。若不是采集题的肯定回答,不要写任何 19xx / 20xx 年份。
人设:口语啰嗦,会绕半句再落到要点,但仍要把必需要素说清楚。少用「嗯/啊/emmm」。
硬约束:
- 只输出用户会打的那句话
- 字数约 16–28 个汉字(含标点),不要明显超出
- 不得逐字复述上面的选项原文
- 不得出现真实姓名、地名、学校名、公司名、医院名、单位名
- 不要用「有的 / 发生了 / 就是那段 / 没这回事」这种套话,写得像真人随手打的
- 需要带时间时,用大概年份或年月,事件用普通生活经历
```
### 复核提示原文
系统:`你只做出生时间校正当前轮的意图分类。不回答用户,不修改状态。只输出一个 JSON 对象,不要解释。`
用户提示模板示例:
```
根据当前问题和用户回复分类。看不到任何预设标签。
intent 只能是:
- answer_current_focus:用户在回答当前问题
- provide_new_evidence:没有回答当前问题,只补充了一件带大概年或月的经历
- stop_rectification:明确要求停止整个校正;说当前这件事没发生不是停止
- ask_about_result:在问结果、范围、候选分钟或好了没
- unclear:以上都不成立,或混在一起无法单选
answer_class:
- 仅当 intent 是 answer_current_focus 时非空,否则必须为 null
- 点选题:按选项 label 的含义选 yes / weak_yes / no / unsure,不要按 A/B/C/D 位置猜
- 采集题:没有/没发生/这方面没什么 → no;记不清/不记得/忘了/想不起来/以后再说 → unsure;
用带大概年月的经历直接回答当前采集题 → yes(程度弱则为 weak_yes)
- 不要把「没有」或「记不清」标成 yes
- 采集题若既没有否定、也没有记不清、也没有带年月经历 → intent 为 unclear,answer_class 为 null
- 同一句既明确否定当前采集题又补充了新的带时间经历:intent 仍为 answer_current_focus,answer_class 为 no
has_new_dated_event:
- 仅在同一句里除了回答当前问题之外,还提供了新的、带大概时间的经历时为 true
- 单纯否定、单纯记不清、或只用带年月经历来直接回答当前采集题,都是 false
- 无当前问题时,带年月的经历 → intent 为 provide_new_evidence,has_new_dated_event 为 true
只输出:
{"intent":"...","answer_class":"yes"|"weak_yes"|"no"|"unsure"|null,"has_new_dated_event":true|false}
当前问题:
2023 年前后,工作状态是否出现明显变化?
选项:
- A. 这段时间没有明显变化 (answer_class=no)
- B. 记不清当时的情况 (answer_class=unsure)
- C. 变化明显而且时间吻合 (answer_class=yes)
- D. 有变化但程度比较弱 (answer_class=weak_yes)
用户回复:
(用户回复)
```
公开案例姓名不得出现在回复正文。
@@ -1,113 +1,36 @@
{"focus": {"case_status": "distinguishing_candidates", "current_question": "1990 年 12 月前后,有没有入职、换工作或职责加重?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-choice-0036", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "发生了,当时身边人也没特别提,我自己也没当回事,我想想啊就是那种说不上来特别大但也不是完全没有的感觉"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "2007 年 1 月前后,有没有入职、换工作或职责加重?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-choice-0042", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "发生了,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,现在回想起来细"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "2026 年 11 月前后,有没有收入明显变化、大笔支出或欠债?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-choice-0048", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "发生了,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,现在回想起来细"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "2010 年 11 月前后,有没有入职、换工作或职责加重?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-choice-0056", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "就是那段,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,当时身边人也没特别提,我自己也没当回事"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "2014 年 10 月前后,有没有生病、受伤或压力特别大?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-choice-0057", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "有的,当时身边人也没特别提,我自己也没当回事,我想想啊就是那种说不上来特别大但也不是完全没有的感觉"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "1981 年 12 月前后,有没有搬家或长期住到外地?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": "yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-choice-0072", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "确实有,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,我想想啊就是那种说不上。另外2015年11月职责变重"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "1990 年 12 月前后,有没有收入明显变化、大笔支出或欠债?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-choice-0121", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "将就吧,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,别的先不"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "1966 年 1 月前后,有没有收入明显变化、大笔支出或欠债?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-choice-0155", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "一点点,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,当时身边人也没特别提。另外2024年4月搬家"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "1953 年 1 月前后,有没有收入明显变化、大笔支出或欠债?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-choice-0162", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "好像有过,别的先不说,就这一句你先记着,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,当时身边。另外2003年6月离职"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "2003 年 4 月前后,有没有搬家或长期住到外地?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-choice-0257", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "记不清,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,当时身边人也没特别"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "2022 年 1 月前后,有没有升学、转学或换学习环境?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-choice-0258", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "记不清,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,别的先不说,就这一句你先记着,现在回想起来细节"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "2003 年 12 月前后,有没有入职、换工作或职责加重?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-choice-0263", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "说不清,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,我想想啊就是那种说不上来特别大但也不"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "2000 年 12 月前后,有没有搬家或长期住到外地?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-choice-0265", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "记不清,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,现在回想起来细节对不上,只能说个大概"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "1994 年 12 月前后,有没有收入明显变化、大笔支出或欠债?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-choice-0277", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "记不清,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,当时身边人也没特别提,我自己也没当回事"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "2020 年 6 月前后,有没有入职、换工作或职责加重?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-choice-0284", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "以后再说,别的先不说,就这一句你先记着,我想想啊就是那种说不上来特别大但也不是完全没有的感觉"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "2014 年 10 月前后,有没有生病、受伤或压力特别大?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-choice-0316", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "忘了,现在回想起来细节对不上,只能说个大概,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,现在"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "1991 年 3 月前后,有没有收入明显变化、大笔支出或欠债?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": "unsure", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-choice-0325", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "unsure", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "记不清,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,现在回想起来细节对不上,只能说个大概,。另外2024年7月家里添丁"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "1979 年 7 月前后,有没有家人结婚、添丁或住院?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": "unsure", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-choice-0331", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "unsure", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "忘了,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,我想想啊就是那种说不上来特别大但也不。另外2015年10月换学校"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "1991 年 3 月前后,有没有收入明显变化、大笔支出或欠债?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "id": "C-choice-0348", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "user_message": "另外说一件:2022年12月开始长期住外地。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "1985 年 7 月前后,有没有收入明显变化、大笔支出或欠债?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "id": "C-choice-0350", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "user_message": "另外说一件:2020年11月离职。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,现在回想"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "2014 年 3 月前后,有没有入职、换工作或职责加重?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "id": "C-choice-0353", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "user_message": "另外说一件:2003年12月换学校。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,我想想"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "2021 年 10 月前后,有没有家人结婚、添丁或住院?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "id": "C-choice-0381", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "user_message": "另外说一件:2021年4月入职。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,别的先不说"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "1982 年 1 月前后,有没有收入明显变化、大笔支出或欠债?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "agent-template-v1", "gold": {"answer_class": null, "has_new_dated_event": false, "intent": "ask_about_result"}, "id": "C-choice-0481", "layer": "choice", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": false, "intent": "ask_about_result"}, "user_message": "先不问这题了,出结果了吗,范围呢。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,"}
{"focus": {"case_status": "collecting_evidence", "current_question": "钱的方面再对一下:哪年收入明显变过、有过大笔支出,或欠过债?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0740", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "说不清,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,别的"}
{"focus": {"case_status": "collecting_evidence", "current_question": "大概哪年几月?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0757", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "以后再说,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,现在回想起来细节对不上,只能说个大概"}
{"focus": {"case_status": "collecting_evidence", "current_question": "钱的方面,还记得哪年收入明显变过、有过大笔支出,或欠过债吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0758", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "忘了,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,别的先不说,就这一句你先记着"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到工作这边——刚才那件我记下了,哪年入职或换工作还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0806", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "说不清,现在回想起来细节对不上,只能说个大概,我想想啊就是那种说不上来特别大但也不是完全没有的感觉"}
{"focus": {"case_status": "collecting_evidence", "current_question": "感情这边,还记得哪年认真在一起、分开,或结婚吗?有年份就很好。", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0810", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "忘了,我想想啊就是那种说不上来特别大但也不是完全没有的感觉,别的先不说,就这一句你先记着,别的"}
{"focus": {"case_status": "collecting_evidence", "current_question": "搬家或开始长期住外地,大概是哪年?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0817", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "有的,2016年9月开始长期住外地。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,当"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到工作这边——刚才那件我记下了,哪年入职或换工作还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0823", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "有的,1999年4月入职。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,别的先不说,就这一句你先记着"}
{"focus": {"case_status": "collecting_evidence", "current_question": "上学这边,还记得哪年升学、转学或大考吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0837", "layer": "collect", "persona": "rambling", "review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "有的,1998年4月转学。现在回想起来细节对不上,只能说个大概,现在回想起来细节对不上,只能说个大概"}
{"focus": {"case_status": "collecting_evidence", "current_question": "家里如果有结婚、添丁或住院这类事,记得大概哪年就行。", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0846", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "有的,2014年10月长辈住院。我想想啊就是那种说不上来特别大但也不是完全没有的感觉"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到感情这边——刚才说的工作我记下了,哪年认真在一起或分开还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0884", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2018年认"}
{"focus": {"case_status": "collecting_evidence", "current_question": "工作上呢,还记得哪年入职、换工作,或职责一下子变重吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0886", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2002年职"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到感情这边——刚才说的工作我记下了,哪年认真在一起或分开还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0887", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "1999年分"}
{"focus": {"case_status": "collecting_evidence", "current_question": "工作上呢,还记得哪年入职、换工作,或职责一下子变重吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0888", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2016年职"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到工作这边——刚才那件我记下了,哪年入职或换工作还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0889", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2013年换"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到工作这边——刚才那件我记下了,哪年入职或换工作还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0890", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2024年换"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到工作这边——刚才那件我记下了,哪年入职或换工作还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0892", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2016年职"}
{"focus": {"case_status": "collecting_evidence", "current_question": "钱的方面,还记得哪年收入明显变过、有过大笔支出,或欠过债吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0893", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2014年收"}
{"focus": {"case_status": "collecting_evidence", "current_question": "搬家或开始长期住外地,大概是哪年?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0895", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2022年搬"}
{"focus": {"case_status": "collecting_evidence", "current_question": "再问一次家里:结婚、添丁或住院,大概哪年?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0896", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2015年长"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到上学这边——哪年升学、转学或大考,还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0898", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "有的,大概2016年转学吧,印象不深。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,我想想"}
{"focus": {"case_status": "collecting_evidence", "current_question": "身体或压力这边再问一次:哪年生病、受伤,或特别难熬?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0899", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2017年受"}
{"focus": {"case_status": "collecting_evidence", "current_question": "身体或压力这边再问一次:哪年生病、受伤,或特别难熬?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0900", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2020年受"}
{"focus": {"case_status": "collecting_evidence", "current_question": "搬家或开始长期住外地,大概是哪年?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0901", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2024年搬"}
{"focus": {"case_status": "collecting_evidence", "current_question": "身体或压力这边再问一次:哪年生病、受伤,或特别难熬?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0902", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2020年生"}
{"focus": {"case_status": "collecting_evidence", "current_question": "钱的方面再对一下:哪年收入明显变过、有过大笔支出,或欠过债?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0905", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "有的,大概2014年有过大笔支出吧,印象不深。别的先不说,就这一句你先记着,我想想啊就是那种说不上来特别大但也不是完全没有的感觉"}
{"focus": {"case_status": "collecting_evidence", "current_question": "有没有哪年搬家,或开始长期住在外地?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0906", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2022年换"}
{"focus": {"case_status": "collecting_evidence", "current_question": "再问一次家里:结婚、添丁或住院,大概哪年?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0910", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2020年长"}
{"focus": {"case_status": "collecting_evidence", "current_question": "钱的方面,还记得哪年收入明显变过、有过大笔支出,或欠过债吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0911", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2011年收"}
{"focus": {"case_status": "collecting_evidence", "current_question": "有没有哪年搬家,或开始长期住在外地?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0916", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "有的,大概2013年1月搬家吧,印象不深。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,我想"}
{"focus": {"case_status": "collecting_evidence", "current_question": "再问一次家里:结婚、添丁或住院,大概哪年?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0917", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2009年家"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到上学这边——哪年升学、转学或大考,还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0918", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2009年升"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到感情这边——刚才说的工作我记下了,哪年认真在一起或分开还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0919", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2010年认"}
{"focus": {"case_status": "collecting_evidence", "current_question": "钱的方面,还记得哪年收入明显变过、有过大笔支出,或欠过债吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0921", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2010年欠"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到工作这边——刚才那件我记下了,哪年入职或换工作还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0926", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2014年职"}
{"focus": {"case_status": "collecting_evidence", "current_question": "搬家或开始长期住外地,大概是哪年?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0929", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2008年换"}
{"focus": {"case_status": "collecting_evidence", "current_question": "上学这边,还记得哪年升学、转学或大考吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0931", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "有的,大概2005年大考吧,印象不深。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,我想想啊就是那种说不上来"}
{"focus": {"case_status": "collecting_evidence", "current_question": "再问一次家里:结婚、添丁或住院,大概哪年?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0933", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2003年长"}
{"focus": {"case_status": "collecting_evidence", "current_question": "感情这边,还记得哪年认真在一起、分开,或结婚吗?有年份就很好。", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0934", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2007年结"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到上学这边——哪年升学、转学或大考,还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0935", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2009年转"}
{"focus": {"case_status": "collecting_evidence", "current_question": "上学这边,还记得哪年升学、转学或大考吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0937", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2004年升"}
{"focus": {"case_status": "collecting_evidence", "current_question": "感情这边,还记得哪年认真在一起、分开,或结婚吗?有年份就很好。", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0938", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2024年结"}
{"focus": {"case_status": "collecting_evidence", "current_question": "身体或压力这边再问一次:哪年生病、受伤,或特别难熬?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0940", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2002年特"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到上学这边——哪年升学、转学或大考,还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0942", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2007年换"}
{"focus": {"case_status": "collecting_evidence", "current_question": "钱的方面再对一下:哪年收入明显变过、有过大笔支出,或欠过债?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0943", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2001年收"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到工作这边——刚才那件我记下了,哪年入职或换工作还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0945", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2007年换"}
{"focus": {"case_status": "collecting_evidence", "current_question": "感情这边,还记得哪年认真在一起、分开,或结婚吗?有年份就很好。", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0946", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2000年分"}
{"focus": {"case_status": "collecting_evidence", "current_question": "身体或压力这边,还记得哪年生病、受伤,或特别难熬的一段时间吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0949", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2006年生"}
{"focus": {"case_status": "collecting_evidence", "current_question": "上学这边,还记得哪年升学、转学或大考吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0952", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2003年转"}
{"focus": {"case_status": "collecting_evidence", "current_question": "搬家或开始长期住外地,大概是哪年?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0953", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2021年搬"}
{"focus": {"case_status": "collecting_evidence", "current_question": "家里如果有结婚、添丁或住院这类事,记得大概哪年就行。", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0957", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2008年家"}
{"focus": {"case_status": "collecting_evidence", "current_question": "搬家或开始长期住外地,大概是哪年?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0960", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2009年搬"}
{"focus": {"case_status": "collecting_evidence", "current_question": "钱的方面,还记得哪年收入明显变过、有过大笔支出,或欠过债吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0961", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2004年收"}
{"focus": {"case_status": "collecting_evidence", "current_question": "搬家或开始长期住外地,大概是哪年?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0962", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2010年换"}
{"focus": {"case_status": "collecting_evidence", "current_question": "有没有哪年搬家,或开始长期住在外地?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0965", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2008年开"}
{"focus": {"case_status": "collecting_evidence", "current_question": "搬家或开始长期住外地,大概是哪年?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0967", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2004年搬"}
{"focus": {"case_status": "collecting_evidence", "current_question": "上学这边,还记得哪年升学、转学或大考吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0968", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2009年大"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到感情这边——刚才说的工作我记下了,哪年认真在一起或分开还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0969", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2011年分"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到上学这边——哪年升学、转学或大考,还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0970", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2014年换"}
{"focus": {"case_status": "collecting_evidence", "current_question": "钱的方面再对一下:哪年收入明显变过、有过大笔支出,或欠过债?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0971", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2020年有"}
{"focus": {"case_status": "collecting_evidence", "current_question": "身体或压力这边再问一次:哪年生病、受伤,或特别难熬?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0972", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2000年特"}
{"focus": {"case_status": "collecting_evidence", "current_question": "家里如果有结婚、添丁或住院这类事,记得大概哪年就行。", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0973", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2018年家"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到上学这边——哪年升学、转学或大考,还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0974", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2006年升"}
{"focus": {"case_status": "collecting_evidence", "current_question": "钱的方面再对一下:哪年收入明显变过、有过大笔支出,或欠过债?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0975", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2008年有"}
{"focus": {"case_status": "collecting_evidence", "current_question": "上学这边,还记得哪年升学、转学或大考吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0976", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "1998年换"}
{"focus": {"case_status": "collecting_evidence", "current_question": "有没有哪年搬家,或开始长期住在外地?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0978", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2016年开"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到上学这边——哪年升学、转学或大考,还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0982", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2021年转"}
{"focus": {"case_status": "collecting_evidence", "current_question": "大概哪年几月?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0983", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2004年职"}
{"focus": {"case_status": "collecting_evidence", "current_question": "大概哪年几月?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0984", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2007年换"}
{"focus": {"case_status": "collecting_evidence", "current_question": "再问一次家里:结婚、添丁或住院,大概哪年?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0986", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2003年家"}
{"focus": {"case_status": "collecting_evidence", "current_question": "家里如果有结婚、添丁或住院这类事,记得大概哪年就行。", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0987", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2002年家"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到感情这边——刚才说的工作我记下了,哪年认真在一起或分开还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0989", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2014年分"}
{"focus": {"case_status": "collecting_evidence", "current_question": "身体或压力这边再问一次:哪年生病、受伤,或特别难熬?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0990", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2024年生"}
{"focus": {"case_status": "collecting_evidence", "current_question": "再问一次家里:结婚、添丁或住院,大概哪年?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0995", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "2000年长。另外2016年4月收入明显变过"}
{"focus": {"case_status": "collecting_evidence", "current_question": "搬家或开始长期住外地,大概是哪年?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0996", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "2021年换。另外2018年6月分开"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到上学这边——哪年升学、转学或大考,还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-1002", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "2005年换。另外2007年12月特别难熬过一段时间"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到感情这边——刚才说的工作我记下了,哪年认真在一起或分开还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-1005", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "2005年结。另外2022年7月职责变重"}
{"focus": {"case_status": "collecting_evidence", "current_question": "钱的方面再对一下:哪年收入明显变过、有过大笔支出,或欠过债?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-1009", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "2013年收。另外2012年5月长辈住院"}
{"focus": {"case_status": "collecting_evidence", "current_question": "身体或压力这边再问一次:哪年生病、受伤,或特别难熬?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-1014", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "1999年特。另外2011年8月长辈住院"}
{"focus": {"case_status": "collecting_evidence", "current_question": "上学这边,还记得哪年升学、转学或大考吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-1017", "layer": "collect", "persona": "terse", "review": {"answer_class": "yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "2004年升。另外2010年6月分开"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到工作这边——刚才那件我记下了,哪年入职或换工作还记得吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "id": "C-collect-1030", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "user_message": "另外说一件:2020年5月转学。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,我想想啊就"}
{"focus": {"case_status": "collecting_evidence", "current_question": "工作上呢,还记得哪年入职、换工作,或职责一下子变重吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "id": "C-collect-1033", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "user_message": "另外说一件:2003年7月生病。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,别的先不说"}
{"focus": {"case_status": "collecting_evidence", "current_question": "工作上呢,还记得哪年入职、换工作,或职责一下子变重吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "id": "C-collect-1045", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "user_message": "另外说一件:1999年9月生病。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,当时身边人"}
{"focus": {"case_status": "collecting_evidence", "current_question": "钱的方面,还记得哪年收入明显变过、有过大笔支出,或欠过债吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "id": "C-collect-1047", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "user_message": "另外说一件:2007年12月受伤。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,现在回想"}
{"focus": {"case_status": "collecting_evidence", "current_question": "感情这边,还记得哪年认真在一起、分开,或结婚吗?有年份就很好。", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "id": "C-collect-1060", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "user_message": "另外说一件:2000年4月换城市住。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,当时身"}
{"focus": {"case_status": "collecting_evidence", "current_question": "上学这边,还记得哪年升学、转学或大考吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "id": "C-collect-1073", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "user_message": "另外说一件:2015年9月离职。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,别的先不说"}
{"focus": {"case_status": "collecting_evidence", "current_question": "再问一次家里:结婚、添丁或住院,大概哪年?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "id": "C-collect-1074", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "user_message": "另外说一件:2019年3月认真在一起。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,现在"}
{"focus": {"case_status": "collecting_evidence", "current_question": "工作上呢,还记得哪年入职、换工作,或职责一下子变重吗?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "id": "C-collect-1080", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "user_message": "另外说一件:1998年3月大考。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,我想想啊就"}
{"focus": {"case_status": "collecting_evidence", "current_question": "大概哪年几月?", "kind": "collect", "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "id": "C-collect-1089", "layer": "collect", "persona": "rambling", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "user_message": "另外说一件:2006年5月升学。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,我想想啊就"}
{"focus": {"case_status": "collecting_evidence", "current_question": "", "kind": null, "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": null, "has_new_dated_event": false, "intent": "ask_about_result"}, "id": "C-none-1288", "layer": "none", "persona": "rambling", "review": {"answer_class": null, "has_new_dated_event": false, "intent": "unclear"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": false, "intent": "ask_about_result"}, "user_message": "先不问这题了,出结果了吗,范围呢。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,"}
{"focus": {"case_status": "collecting_evidence", "current_question": "", "kind": null, "options": []}, "generator": "agent-template-v1", "gold": {"answer_class": null, "has_new_dated_event": false, "intent": "ask_about_result"}, "id": "C-none-1289", "layer": "none", "persona": "rambling", "review": {"answer_class": null, "has_new_dated_event": false, "intent": "unclear"}, "reviewer": "agent-rule-v1", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": false, "intent": "ask_about_result"}, "user_message": "先不问这题了,出结果了吗,范围呢。我想想啊就是那种说不上来特别大但也不是完全没有的感觉,"}
{"focus": {"case_status": "collecting_evidence", "current_question": "大概哪年几月?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0387", "layer": "collect", "persona": "rambling", "prompt_version": "gen-review-v3", "review": {"answer_class": "yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "那年二三月吧,变化不大。另外2018年搬过一次家。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到上学这边——哪年升学、转学或大考,还记得吗?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0377", "layer": "collect", "persona": "answer_then_event", "prompt_version": "gen-review-v3", "review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "升学那会儿印象不深,对了2016年搬过家。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "家里如果有结婚、添丁或住院这类事,记得大概哪年就行。", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0380", "layer": "collect", "persona": "answer_then_event", "prompt_version": "gen-review-v3", "review": {"answer_class": "unsure", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "好像有吧,记不太清。对了,2019年搬过一次家。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "大概哪年几月?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0323", "layer": "collect", "persona": "rambling", "prompt_version": "gen-review-v3", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "好像那阵子是有过一点小变化,大概2013年夏天前后吧。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "搬家或开始长期住外地,大概是哪年?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0336", "layer": "collect", "persona": "terse", "prompt_version": "gen-review-v3", "review": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2020年搬过一次,不太记得。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到工作这边——刚才那件我记下了,哪年入职或换工作还记得吗?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0391", "layer": "collect", "persona": "rambling", "prompt_version": "gen-review-v3", "review": {"answer_class": "unsure", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "入职换工作那块印象不深,好像动过但说不准;另外2018年我搬过一次家。"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "1973 年 12 月前后,有没有家人结婚、添丁或住院?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-choice-0126", "layer": "choice", "persona": "dialect_netspeak", "prompt_version": "gen-review-v3", "review": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "73年那阵子好像有点动静,但记不太清"}
{"focus": {"case_status": "collecting_evidence", "current_question": "工作上呢,还记得哪年入职、换工作,或职责一下子变重吗?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0388", "layer": "collect", "persona": "deny_then_event", "prompt_version": "gen-review-v3", "review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "工作换过但记不太清,倒是2018年搬了次家。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "搬家或开始长期住外地,大概是哪年?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0381", "layer": "collect", "persona": "answer_then_event", "prompt_version": "gen-review-v3", "review": {"answer_class": "unsure", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "搬过一两次吧,不太记得清了。另外2018年换过工作。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "钱的方面再对一下:哪年收入明显变过、有过大笔支出,或欠过债?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0382", "layer": "collect", "persona": "answer_then_event", "prompt_version": "gen-review-v3", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "钱上好像没太明显的变化。另外2018年换过一次工作。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "搬家或开始长期住外地,大概是哪年?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": null, "has_new_dated_event": false, "intent": "unclear"}, "id": "C-collect-0538", "layer": "collect", "persona": "rambling", "prompt_version": "gen-review-v3", "review": {"answer_class": null, "has_new_dated_event": false, "intent": "stop_rectification"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": false, "intent": "unclear"}, "user_message": "这我还真一下掰扯不清,先别按这个往下算吧。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "钱的方面再对一下:哪年收入明显变过、有过大笔支出,或欠过债?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0372", "layer": "collect", "persona": "answer_then_event", "prompt_version": "gen-review-v3", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "钱上好像没太大起伏,就感觉一般。对了,2021年我换了个工作。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "搬家或开始长期住外地,大概是哪年?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0385", "layer": "collect", "persona": "dialect_netspeak", "prompt_version": "gen-review-v3", "review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "搬出去住好像不太明显,另外2019年换过工作。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "工作上呢,还记得哪年入职、换工作,或职责一下子变重吗?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0326", "layer": "collect", "persona": "terse", "prompt_version": "gen-review-v3", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2019年换过一次,变化不大。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到上学这边——哪年升学、转学或大考,还记得吗?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": null, "has_new_dated_event": false, "intent": "unclear"}, "id": "C-collect-0531", "layer": "collect", "persona": "dialect_netspeak", "prompt_version": "gen-review-v3", "review": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": false, "intent": "unclear"}, "user_message": "那会儿光顾着玩了,哪还记这些啊。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "钱的方面再对一下:哪年收入明显变过、有过大笔支出,或欠过债?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0378", "layer": "collect", "persona": "answer_then_event", "prompt_version": "gen-review-v3", "review": {"answer_class": "unsure", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "钱上好像就小幅波动过,不太记得。对了2018年搬过家。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "大概哪年几月?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0379", "layer": "collect", "persona": "answer_then_event", "prompt_version": "gen-review-v3", "review": {"answer_class": "unsure", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "大概2010年前后吧,记不太清。另外2016年我搬过一次家。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "感情这边,还记得哪年认真在一起、分开,或结婚吗?有年份就很好。", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0347", "layer": "collect", "persona": "dialect_netspeak", "prompt_version": "gen-review-v3", "reject_reason": "collect_yes_needs_date", "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "前几年好像谈过一段,后来散了,具体哪年记不太清。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "再问一次家里:结婚、添丁或住院,大概哪年?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0338", "layer": "collect", "persona": "terse", "prompt_version": "gen-review-v3", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2017年3月添丁,不太记得。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "大概哪年几月?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": null, "has_new_dated_event": false, "intent": "unclear"}, "id": "C-collect-0537", "layer": "collect", "persona": "terse", "prompt_version": "gen-review-v3", "review": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": null, "has_new_dated_event": false, "intent": "unclear"}, "user_message": "这个我实在说不上来。"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "1988 年 5 月前后,有没有收入明显变化、大笔支出或欠债?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-choice-0155", "layer": "choice", "persona": "answer_then_event", "prompt_version": "gen-review-v3", "review": {"answer_class": "unsure", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "感觉那阵子钱上有点紧,不太确定。另外2019年搬过一次家。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "身体或压力这边,还记得哪年生病、受伤,或特别难熬的一段时间吗?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0384", "layer": "collect", "persona": "deny_then_event", "prompt_version": "gen-review-v3", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "身体这块没什么特别印象,不过2021年我换过工作。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "上学这边,还记得哪年升学、转学或大考吗?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0369", "layer": "collect", "persona": "answer_then_event", "prompt_version": "gen-review-v3", "review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "升学那会儿印象不深,另外2018年搬过一次家。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到感情这边——刚才说的工作我记下了,哪年认真在一起或分开还记得吗?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0364", "layer": "collect", "persona": "rambling", "prompt_version": "gen-review-v3", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "真要认真算也就2018年那阵子,在一起过,但感觉一直淡淡的,后来也慢慢散了。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到上学这边——哪年升学、转学或大考,还记得吗?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0386", "layer": "collect", "persona": "deny_then_event", "prompt_version": "gen-review-v3", "review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "升学好像没啥特别深的印象,不过2018年我搬过一次家。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "搬家或开始长期住外地,大概是哪年?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0375", "layer": "collect", "persona": "answer_then_event", "prompt_version": "gen-review-v3", "review": {"answer_class": "unsure", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "好像搬过,不太确定。另外2016年换过工作。"}
{"focus": {"case_status": "distinguishing_candidates", "current_question": "1986 年 10 月前后,有没有家人结婚、添丁或住院?", "kind": "choice", "options": [{"answer_class": "yes", "key": "A", "label": "明确发生且时间吻合"}, {"answer_class": "weak_yes", "key": "B", "label": "发生过但程度较弱"}, {"answer_class": "no", "key": "C", "label": "明确没有发生"}, {"answer_class": "unsure", "key": "D", "label": "这段记不清楚"}]}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-choice-0112", "layer": "choice", "persona": "dialect_netspeak", "prompt_version": "gen-review-v3", "review": {"answer_class": "unsure", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "1986年那会儿好像家里是有点事,但印象不深。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "有没有哪年搬家,或开始长期住在外地?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0368", "layer": "collect", "persona": "rambling", "prompt_version": "gen-review-v3", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "搬出去住大概是2016年,住得不久,印象不太深。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到上学这边——哪年升学、转学或大考,还记得吗?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0371", "layer": "collect", "persona": "answer_then_event", "prompt_version": "gen-review-v3", "review": {"answer_class": "unsure", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "升学好像有点印象,不太深吧。对了,2018年还搬过一次家。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "家里如果有结婚、添丁或住院这类事,记得大概哪年就行。", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0392", "layer": "collect", "persona": "deny_then_event", "prompt_version": "gen-review-v3", "review": {"answer_class": "unsure", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "家里添丁这事我印象不深,另外2021年换过一次工作。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "工作上呢,还记得哪年入职、换工作,或职责一下子变重吗?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0374", "layer": "collect", "persona": "answer_then_event", "prompt_version": "gen-review-v3", "review": {"answer_class": null, "has_new_dated_event": true, "intent": "provide_new_evidence"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "工作换过但印象不深,另外2018年搬了次家。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "再问一次家里:结婚、添丁或住院,大概哪年?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0367", "layer": "collect", "persona": "terse", "prompt_version": "gen-review-v3", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2016年添过丁,不太顺。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "感情这边,还记得哪年认真在一起、分开,或结婚吗?有年份就很好。", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0390", "layer": "collect", "persona": "deny_then_event", "prompt_version": "gen-review-v3", "review": {"answer_class": "no", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "感情上好像没哪年特别认真过,对了2019年换过一次工作。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "回到感情这边——刚才说的工作我记下了,哪年认真在一起或分开还记得吗?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0333", "layer": "collect", "persona": "dialect_netspeak", "prompt_version": "gen-review-v3", "reject_reason": "collect_yes_needs_date", "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "那阵子好像是20年吧,凑合在一块儿过,后来就慢慢淡了。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "再问一次家里:结婚、添丁或住院,大概哪年?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "id": "C-collect-0361", "layer": "collect", "persona": "terse", "prompt_version": "gen-review-v3", "review": {"answer_class": "yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": false, "intent": "answer_current_focus"}, "user_message": "2016年添过丁,不太记得清。"}
{"focus": {"case_status": "collecting_evidence", "current_question": "钱的方面,还记得哪年收入明显变过、有过大笔支出,或欠过债吗?", "kind": "collect", "options": []}, "generator": "deepseek-flash", "gold": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "id": "C-collect-0370", "layer": "collect", "persona": "answer_then_event", "prompt_version": "gen-review-v3", "review": {"answer_class": "unsure", "has_new_dated_event": true, "intent": "answer_current_focus"}, "reviewer": "deepseek-flash", "source": "C", "target_before_review": {"answer_class": "weak_yes", "has_new_dated_event": true, "intent": "answer_current_focus"}, "user_message": "钱上好像没太大起伏,记不太清了。对了,2018年换过工作。"}
File diff suppressed because it is too large Load Diff
+28 -3
View File
@@ -6,7 +6,12 @@ import json
import unittest
from pathlib import Path
import inspect
from scripts.research.jev_intent_corpus_build import ( # noqa: E402
GENERATOR_NAME,
REVIEWER_NAME,
build_review_prompt,
holdout_name_denylist,
review_sample,
source_a_samples,
@@ -82,10 +87,30 @@ class JevIntentCorpusTests(unittest.TestCase):
"user_message": "没有",
"focus": {"current_question": "钱的方面,还记得哪年收入明显变过吗?", "options": []},
"gold": {"intent": "stop_rectification", "answer_class": None, "has_new_dated_event": False},
"target_before_review": {"intent": "stop_rectification"},
}
reviewed = review_sample(sample)
self.assertEqual(reviewed["intent"], "answer_current_focus")
self.assertEqual(reviewed["answer_class"], "no")
prompt = build_review_prompt(sample)
self.assertIn("没有", prompt)
self.assertIn("钱的方面,还记得哪年收入明显变过吗?", prompt)
self.assertNotIn("目标", prompt)
tail = prompt.rsplit("用户回复:", 1)[-1]
self.assertIn("没有", tail)
self.assertNotIn("stop_rectification", tail)
self.assertNotIn("template", GENERATOR_NAME.lower())
self.assertNotIn("rule", REVIEWER_NAME.lower())
def test_corpus_build_has_no_literal_word_bank(self) -> None:
src = (ROOT / "scripts" / "research" / "jev_intent_corpus_build.py").read_text(encoding="utf-8")
self.assertNotIn("bank = {", src)
self.assertNotIn('GENERATOR_NAME = "agent-template', src)
self.assertNotIn('REVIEWER_NAME = "agent-rule', src)
def test_reviewer_does_not_regex_label(self) -> None:
src = inspect.getsource(review_sample)
self.assertNotIn("re.search", src)
self.assertNotIn("re.match", src)
self.assertNotIn("STOP_RE", src)
self.assertIn("complete_chat", src)
def test_current_instructions_match_production_classifier(self) -> None:
from scripts.research.jev_intent_current import CHOICE_INSTRUCTIONS, COLLECT_INSTRUCTIONS