research(rectification): add reported-offset evaluation and frozen rerun integrity
Preserve closed confirmation gates and previously-exposed dataset boundaries. Add auditable 900-trial sensitivity results, current scorer freshness checks, and the v5 collection protocol. Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
jesse-ux
@@ -0,0 +1,193 @@
|
||||
{
|
||||
"baseline_commit": "5049e7847",
|
||||
"scope": "frontend_default_parallel_npm_test",
|
||||
"counts": {
|
||||
"baseline": {
|
||||
"tests": 3470,
|
||||
"pass": 3387,
|
||||
"fail": 83,
|
||||
"skipped": 0
|
||||
},
|
||||
"current": {
|
||||
"tests": 3486,
|
||||
"pass": 3409,
|
||||
"fail": 77,
|
||||
"skipped": 0
|
||||
}
|
||||
},
|
||||
"failure_titles": {
|
||||
"baseline": [
|
||||
"Better Auth supports shared user OTP/password sessions for admins",
|
||||
"Owner recovery grants only one currently loginable synced identity admin",
|
||||
"PR-4 candidate decisions use server UUIDs, receipt-derived gates and separate acceptance/confirmation",
|
||||
"a multi-domain plan carries every executed route's checklist once",
|
||||
"a route the skill declares no checklist for is reported, not filled in with another route's",
|
||||
"admin code functions reject immutable codes, revoked redemption, and roll back on audit failure",
|
||||
"admin customer reset clears only rebuildable application state",
|
||||
"append_consultation_question ignores thinking fields and enforces the physical JSON cap",
|
||||
"billing order adjustments and redemption reasons are atomic and audited",
|
||||
"billing, subscriptions, usage authorization, RBAC, and model publication remain transactional",
|
||||
"block_scan RPCs are service_role-only and advance a declared period",
|
||||
"candidate scores stay out of the specified client ownership boundary",
|
||||
"changed staging workflows are syntactically valid YAML",
|
||||
"checked-in registry verifies hashed product packages and leaves consult on the live skill",
|
||||
"creates every absent backup path component privately despite a permissive caller umask",
|
||||
"database drops secret refs, generates immutable codes, and invalidates evidence after key rotation",
|
||||
"database env validator accepts a private valid file without printing values",
|
||||
"database env validator accepts punctuated literal required secrets",
|
||||
"database env validator enforces an explicit staging owner uid without printing values",
|
||||
"database env validator rejects symlinks and unsafe modes",
|
||||
"document v2 and durable job migrations are exact db/supabase mirrors",
|
||||
"document v2 migration preserves v1 reads and adds explicit product depth",
|
||||
"durable job migration atomically enqueues reports and exposes lease RPCs only to service_role",
|
||||
"durable job migration defines the complete state, lease, retry, and identity contract",
|
||||
"every color token used as a Tailwind utility is exposed through @theme",
|
||||
"find enumeration failures preserve existing backups and do not report completion",
|
||||
"first immutable deployment rolls back to validated local image IDs",
|
||||
"further reading offers the references the skill names, and only ones that exist",
|
||||
"general agent runtime has no Jyotish skill package and no personal chart tool",
|
||||
"health uses the skill's health-timing-strict checklist",
|
||||
"immutable Skill migration enforces RPC-only identity, legacy adoption, and cascade-safe receipts",
|
||||
"ingest P0: education kinds, batch confirm, opening focus reuse, precision lock",
|
||||
"is-docs-only-range.sh decides from local history and refuses non-ancestor ranges",
|
||||
"job migration mirror is exact and preserves personal_reports as the document projection",
|
||||
"live consult skill reads a hand-updated tree without a registry hash",
|
||||
"live staging sync preserves env, state, incoming files, and encrypted backups",
|
||||
"live staging sync repairs nested deploy-tree drift without preserving foreign ownership",
|
||||
"local PostgreSQL applies the reviewed business schema and serves authenticated business calls",
|
||||
"longform appendices are owner-read, service-written, and never change report status",
|
||||
"no plan can spend the answer's context on method",
|
||||
"operation-level admin email reauthentication is removed from routes and UI",
|
||||
"path traversal and symlink escape fail closed",
|
||||
"personal Agent exposes the Jyotish Skill and named server tool",
|
||||
"personal report job migration is atomic, lease-bound, recoverable, and owner read-only",
|
||||
"personal report sections enforce owner-read RLS and service-owned durable transitions",
|
||||
"personal_reports.card_summary is nullable, owner-read, service-written, and length-capped",
|
||||
"production env validators accept only self-hosted production selectors and role URLs",
|
||||
"public code never imports the internal dynamic choice contract",
|
||||
"read_report_candidate_range is service_role-only, returns only the window, and stays empty without rows",
|
||||
"rectification agent maps setup failures without calling the rest of the handler",
|
||||
"redeem security: case-sensitive hashing, rate limiting, idempotency and order ownership",
|
||||
"refuses full disks and removes a failed-pipeline partial file",
|
||||
"rejects a direct canonical sticky shared backup directory before chmod",
|
||||
"rejects destructive backup directory aliases and symlink components before mutation",
|
||||
"rejects unsafe writable backup parents before creating the target",
|
||||
"same-second backups publish once without overwriting the completed archive",
|
||||
"saving a published product forks a draft and delete retires or removes it",
|
||||
"self-hosted identity migration creates Better Auth tables with least privilege",
|
||||
"service and restricted admin database identities stay separated",
|
||||
"staging backups are encrypted, atomic, private, and retain the newest three",
|
||||
"staging env validator rejects selector drift, duplicates, and unsafe permissions",
|
||||
"staging model provider env preparation removes legacy settings and keeps one stable key",
|
||||
"symbolic links are rejected even when their target stays inside the project root",
|
||||
"tests\\\\birth-time-guide-agent.test.ts",
|
||||
"tests\\\\consultation-workflow-request.test.ts",
|
||||
"tests\\\\rectification-confirmation-gate.test.ts",
|
||||
"tests\\\\rectification-settled-render-split.test.ts",
|
||||
"tests\\\\rectification-skipped-health-deadend-20260909.test.ts",
|
||||
"tests\\\\rectification-split-holdout.test.ts",
|
||||
"tests\\\\rectification-spoken-prompt.test.ts",
|
||||
"tests\\\\rectification-v9-agent.test.ts",
|
||||
"tests\\\\skill-binding.test.ts",
|
||||
"the delivered method is quoted from the live skill tree",
|
||||
"the route's own strict checklist reaches the answer, not just the package listing",
|
||||
"the widest legal plan still fits the budget",
|
||||
"v9 agent api migration applies, seeds the runtime flag and guards consent",
|
||||
"v9 enforces profile gating, ownership and terminal read-only",
|
||||
"v9 evidence lifecycle: quote grounding, idempotency, confirm and revision lineage",
|
||||
"v9 ignores a historical active minute and allows reselection within the live result",
|
||||
"v9 legacy backfill maps statuses, keeps one resumable per user and is idempotent",
|
||||
"v9 migration applies on a fresh database and re-applies idempotently",
|
||||
"v9 open is atomic, idempotent and allows separate homepage cases",
|
||||
"wealth uses finance-timing-strict as the live checklist, with the wealth alias in the heading"
|
||||
],
|
||||
"current": [
|
||||
"Better Auth supports shared user OTP/password sessions for admins",
|
||||
"Owner recovery grants only one currently loginable synced identity admin",
|
||||
"PR-4 candidate decisions use server UUIDs, receipt-derived gates and separate acceptance/confirmation",
|
||||
"a multi-domain plan carries every executed route's checklist once",
|
||||
"a route the skill declares no checklist for is reported, not filled in with another route's",
|
||||
"admin code functions reject immutable codes, revoked redemption, and roll back on audit failure",
|
||||
"admin customer reset clears only rebuildable application state",
|
||||
"append_consultation_question ignores thinking fields and enforces the physical JSON cap",
|
||||
"billing order adjustments and redemption reasons are atomic and audited",
|
||||
"billing, subscriptions, usage authorization, RBAC, and model publication remain transactional",
|
||||
"block_scan RPCs are service_role-only and advance a declared period",
|
||||
"candidate scores stay out of the specified client ownership boundary",
|
||||
"changed staging workflows are syntactically valid YAML",
|
||||
"checked-in registry verifies hashed product packages and leaves consult on the live skill",
|
||||
"creates every absent backup path component privately despite a permissive caller umask",
|
||||
"database env validator accepts a private valid file without printing values",
|
||||
"database env validator accepts punctuated literal required secrets",
|
||||
"database env validator enforces an explicit staging owner uid without printing values",
|
||||
"database env validator rejects symlinks and unsafe modes",
|
||||
"document v2 and durable job migrations are exact db/supabase mirrors",
|
||||
"document v2 migration preserves v1 reads and adds explicit product depth",
|
||||
"durable job migration atomically enqueues reports and exposes lease RPCs only to service_role",
|
||||
"durable job migration defines the complete state, lease, retry, and identity contract",
|
||||
"every color token used as a Tailwind utility is exposed through @theme",
|
||||
"find enumeration failures preserve existing backups and do not report completion",
|
||||
"first immutable deployment rolls back to validated local image IDs",
|
||||
"further reading offers the references the skill names, and only ones that exist",
|
||||
"general agent runtime has no Jyotish skill package and no personal chart tool",
|
||||
"health uses the skill's health-timing-strict checklist",
|
||||
"immutable Skill migration enforces RPC-only identity, legacy adoption, and cascade-safe receipts",
|
||||
"ingest P0: education kinds, batch confirm, opening focus reuse, precision lock",
|
||||
"is-docs-only-range.sh decides from local history and refuses non-ancestor ranges",
|
||||
"job migration mirror is exact and preserves personal_reports as the document projection",
|
||||
"live consult skill reads a hand-updated tree without a registry hash",
|
||||
"live staging sync preserves env, state, incoming files, and encrypted backups",
|
||||
"live staging sync repairs nested deploy-tree drift without preserving foreign ownership",
|
||||
"local PostgreSQL applies the reviewed business schema and serves authenticated business calls",
|
||||
"longform appendices are owner-read, service-written, and never change report status",
|
||||
"no plan can spend the answer's context on method",
|
||||
"operation-level admin email reauthentication is removed from routes and UI",
|
||||
"path traversal and symlink escape fail closed",
|
||||
"personal Agent exposes the Jyotish Skill and named server tool",
|
||||
"personal report job migration is atomic, lease-bound, recoverable, and owner read-only",
|
||||
"personal report sections enforce owner-read RLS and service-owned durable transitions",
|
||||
"personal_reports.card_summary is nullable, owner-read, service-written, and length-capped",
|
||||
"production env validators accept only self-hosted production selectors and role URLs",
|
||||
"public code never imports the internal dynamic choice contract",
|
||||
"read_report_candidate_range is service_role-only, returns only the window, and stays empty without rows",
|
||||
"rectification agent maps setup failures without calling the rest of the handler",
|
||||
"redeem security: case-sensitive hashing, rate limiting, idempotency and order ownership",
|
||||
"refuses full disks and removes a failed-pipeline partial file",
|
||||
"rejects a direct canonical sticky shared backup directory before chmod",
|
||||
"rejects destructive backup directory aliases and symlink components before mutation",
|
||||
"rejects unsafe writable backup parents before creating the target",
|
||||
"same-second backups publish once without overwriting the completed archive",
|
||||
"saving a published product forks a draft and delete retires or removes it",
|
||||
"self-hosted identity migration creates Better Auth tables with least privilege",
|
||||
"service and restricted admin database identities stay separated",
|
||||
"staging backups are encrypted, atomic, private, and retain the newest three",
|
||||
"staging env validator rejects selector drift, duplicates, and unsafe permissions",
|
||||
"staging model provider env preparation removes legacy settings and keeps one stable key",
|
||||
"symbolic links are rejected even when their target stays inside the project root",
|
||||
"tests\\\\birth-time-guide-agent.test.ts",
|
||||
"tests\\\\consultation-workflow-request.test.ts",
|
||||
"tests\\\\rectification-v9-agent.test.ts",
|
||||
"tests\\\\skill-binding.test.ts",
|
||||
"the delivered method is quoted from the live skill tree",
|
||||
"the route's own strict checklist reaches the answer, not just the package listing",
|
||||
"the widest legal plan still fits the budget",
|
||||
"v9 agent api migration applies, seeds the runtime flag and guards consent",
|
||||
"v9 enforces profile gating, ownership and terminal read-only",
|
||||
"v9 evidence lifecycle: quote grounding, idempotency, confirm and revision lineage",
|
||||
"v9 ignores a historical active minute and allows reselection within the live result",
|
||||
"v9 legacy backfill maps statuses, keeps one resumable per user and is idempotent",
|
||||
"v9 migration applies on a fresh database and re-applies idempotently",
|
||||
"v9 open is atomic, idempotent and allows separate homepage cases",
|
||||
"wealth uses finance-timing-strict as the live checklist, with the wealth alias in the heading"
|
||||
]
|
||||
},
|
||||
"new_failures": [],
|
||||
"not_claiming_full_pass": true,
|
||||
"serial_followup": "stopped_without_final_summary_while_database_fixture_waited",
|
||||
"build_both": "Turbopack rejects external node_modules junction",
|
||||
"tsc_both_exit_code": 0,
|
||||
"lint_both": {
|
||||
"errors": 0,
|
||||
"warnings": 120
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
# 生时校正验证补缺 · 补验清单（2026-09-20）
|
||||
|
||||
本轮无产品/界面/数据库行为改动。不要让用户提供私人出生资料以补本轮验收；偏移评测使用已登记公开数据。
|
||||
|
||||
## 在完整依赖的 Linux 环境补验
|
||||
|
||||
1. 用本轮提交创建干净工作树，安装仓库锁定依赖，不借用其他账号/凭据。
|
||||
2. 运行 `python scripts/pre_work_check.py --remote-timeout 8 --command-timeout 45`。本机已知基线碎片镜像断言失败，不能以远端 verified 代替预检通过。
|
||||
3. 运行 `python -m pytest tests/test_reported_offset_research.py tests/test_sealed_holdout_contract_freshness.py tests/test_rectification_validation_integrity_gate.py tests/test_rectification_confirmation_and.py tests/test_minute_rectification_holdout_validator.py tests/test_minute_rectification_fact_blind_eval_v4.py -q`。旧 v2 封存哈希测试在基线已失败，不改历史值来消除失败。
|
||||
4. 运行 `python scripts/run_quality_gate.py --profile quick`；Windows 本机在 source inventory 检查因缺 mcp 阻塞。完整输出与同基线逐项比对，不把“新增回归通过”写成“整个快速门通过”。
|
||||
5. 前端运行 `tsc --noEmit`、`npm run lint`、`npm test`、`npm run build`；单独确认 `rectification-confirmation-gate.test.ts` 全绿。构建必须得到完整 `/` Static 与首屏 gzip 产物再比较，不能以未修改业务代码替代。
|
||||
6. 核对冻结的 12 个打分文件与执行基线字节相同，`status=not_ready`、`confirmation_coverage_rate=0`、官方独立盲测次数仍为零。
|
||||
|
||||
## 交付后才执行
|
||||
|
||||
- 获准推 staging 后，检查 Gitea 的对应 SHA 门禁和部署；不参考只读 GitHub 镜像。
|
||||
- 核对 `/api/health` 的 `deployment.gitCommit` 为最近含门禁路径改动的 staging SHA；若之后仅文档提交，另跑 docs-only-range 验证。
|
||||
- 未部署不得写“线上验证通过”。本轮不提升 main。
|
||||
|
||||
## 研究遗留（不是软件环境补齐就能解决）
|
||||
|
||||
- 新独立盲测必须等待未曝光 v5 的独立标注、人审、封存与预注册；v3 重跑不能替代。
|
||||
- 真实申报偏差分布需要受控用户样本、独立出生记录与合规授权。本轮敏感性表不代表真实用户总体表现。
|
||||
- v5 备选人物只是候选筛选，不是资料已经采集完毕或人审已通过。
|
||||
Reference in New Issue
Block a user