fix(web): recover the home screen when the bundle never runs

This commit is contained in:
jesse-ux
2026-09-22 12:13:17 +08:00
parent 6ac61be07d
commit 337d820076
18 changed files with 882 additions and 22 deletions
+4
View File
@@ -4,6 +4,10 @@
普通聊天不再在切换人物后仍用登录用户本人的资料。会话锁定的是本人或当前用户名下的另一份资料;对方资料缺失、被删或不完整时会拦住发送,不会退回本人。已有消息的会话不能改绑。报告、每日星语和生时校正仍只用本人资料。Skill 版本不变。
## 2026-09-22 — 首页脚本没跑起来时可以重新加载
首屏如果一直停在载入账户,大约 13 秒后换成一句说明和「重新加载」,不再只转圈。不会自动刷新。旧浏览器上本仓断句不再使用正则后行断言。Skill 版本不变。
## 2026-09-21 — 侧栏当前会话是一整条高亮
选中或悬停会话时,标题和右边的「⋯」共用一块底,不再拆成两个块。Skill 版本不变。
+24 -8
View File
@@ -12232,9 +12232,9 @@
## BUG-936 | 首页永远停在「正在载入账户」,没有超时也没有报错
- 状态:investigating
- 状态:兜底已补、根因待分流
- 首次发现:2026-09-17
- 最近更新:2026-09-17
- 最近更新:2026-09-22
- 影响面:`/` 首屏揭幕(`app/page.tsx` 的 `!hydrated || (!account && !accountError)` 门)、`AppLoadingIndicator`、`app/layout.tsx`
- 用户现象:产品负责人转述,一位已有账号的用户在 iPhone Safari 打开 `https://staging.jyotisha.chat/`,永远停在「正在载入账户 / 同步个人资料与对话记录」,进不去页面。转圈动画还在动。
- 触发条件:尚未定位到具体条件。已排除接口侧。
@@ -12246,13 +12246,13 @@
5. 线上 bundle 的语法下限是 **Safari 16.4**:`1stw4tc266s7a.js`(Next 自己的 app-router 运行时)含类静态块 `class y extends Component{static{this.contextType=...}}`;`089-80cjt8-1t.js`(本仓中文断句 `split(/(?<=[。!?;;])\s*/)`)与 `0dmsli_y64717.js`(链接识别依赖)含正则后行断言。两者在 Safari < 16.4 都是**解析期** SyntaxError,core-js 这类运行时 polyfill 救不了。
6. 该下限不是新引入:`next: 16.3.1` 从仓库首个提交 `4aa0105f`(2026-07-15)就在。所以如果这台设备以前能用,(5) 不是本次原因。
- 待定位(需要用户侧一条信息即可分流):① 设备 iOS 版本 < 16.4 → 命中 (5);② 某个 chunk 在弱网下没下全或被内容拦截器挡掉 → 无痕窗口 / 清除网站数据后恢复;③ Safari 缓存里留着坏掉的 chunk(chunk 带 `cache-control: public, max-age=31536000, immutable`)。
- 根因:未确认,不得提前写。
- 修复:未做。
- 验证:未做。
- 防复发(已可确定的一条,独立于根因):首屏不得把「出错了」的唯一出口放在可能失败的模块 bundle 里。根 layout 需要一段与 bundle 无关的内联经典脚本,在 hydrate 超时或语法不支持时把这一屏换成可读的提示与「重新加载」。见 `docs/tasks/TASK-first-paint-dead-screen-fallback-20260917.md`。
- 相关记录:BUG-716(`/chart` 白屏 45 秒,另一条链路)
- 根因:未确认,不得提前写。兜底补上不等于这台 iPhone Safari 已经修好。
- 修复:根 layout 增加一段与 bundle 无关的内联经典脚本。13 秒后,若仍有 `.app-loading` 且 `<html data-hydrated>` 不是 `"1"`,把加载内容换成「这个页面没能加载完」和只在点击时刷新的「重新加载」。揭幕成功后由首页装配 hook 写 `data-hydrated="1"`,正常路径不闪这一屏。匿名标记只留类别、构建标识和耗时,不留异常原文或请求体。接口超时仍走 bundle 里原来的「连接云端服务超时」,不并进这一屏。本仓断句的后行断言另见 BUG-998,那一项不能把 Next 运行时的语法下限降到 Safari 16.4 以下。
- 验证:`frontend/tests/first-paint-fallback-contract.test.ts` 锁定 ES5、超时大于 8000+4000、未揭幕才替换、按钮才刷新,以及 hydrate 超时 / chunk 404 / 解析失败 / 旧缓存 / 弱网 / 接口错误六类不互相冒充。未在 iPhone Safari 或断 chunk 的浏览器里走查。
- 防复发:首屏不得把「出错了」的唯一出口放在可能失败的模块 bundle 里。不得自动刷新。见 `docs/tasks/TASK-home-bootstrap-reliability-20260922.md` 与 `docs/testing/home-bootstrap-reliability-20260922.md`。
- 相关记录:BUG-716(`/chart` 白屏 45 秒,另一条链路)、BUG-998
- 复发自:无
- 修复版本:未发布
- 修复版本:未发布(本分支未推送,不得声称 staging 已部署)
## BUG-937 | 咨询第 0 步 `toolChoice: required` 让全部咨询整轮失败
@@ -13215,3 +13215,19 @@
- 相关记录:BUG-001、BUG-010、BUG-011、BUG-018、BUG-073、BUG-076、BUG-188
- 复发自:无
- 修复版本:本次分支提交(未推送,未部署)
## BUG-998 | 本仓中文断句使用正则后行断言,Safari 16.4 以下在解析期失败
- 状态:resolved
- 首次发现:2026-09-17(写在 BUG-936 的已确认事实里,当时未单独编号;BUG-937 / BUG-938 已被咨询故障占用)
- 最近更新:2026-09-22
- 影响面:`frontend/src/lib/sentence-split.ts`;原先后行断言在个人报告断句、校正开场句、引擎含义展示、采用旁白、采集提示、轮次旁白
- 用户现象:无单独的用户报告。若设备 Safari 低于 16.4,含后行断言的 chunk 会在解析期抛出 SyntaxError,React 不会 hydrate。这不能解释「以前能用、这次不能用」的设备,也不能单独修好 BUG-936。
- 触发条件:浏览器解析本仓断句 chunk,且不支持正则后行断言。
- 根因:八处 `split` 使用后行断言来「在句末标点后切开并保留标点」。这是语法,不是运行时 polyfill 能补的。
- 修复:共用 `splitAfterSentencePunctuation`,用手写扫描在标点后切开并保留标点。`frontend/src` 不再出现后行断言。Next 运行时里的类静态块改不了,语法下限仍是 Safari 16.4。
- 验证:`frontend/tests/sentence-split.test.ts` 用表驱动用例和旧正则对照,确认切开结果逐字一致;源码合同扫描 `frontend/src` 无后行断言。校正与报告的既有断言未改。
- 防复发:`frontend/src` 不得再写后行断言。测试文件可以保留旧正则作对照。
- 相关记录:BUG-936
- 复发自:无
- 修复版本:未发布(本分支未推送)
@@ -0,0 +1,38 @@
# PROGRESS · 首页加载失败可恢复性(2026-09-22)
工作树:`.worktrees/home-bootstrap-reliability-20260922`
分支:`codex/home-bootstrap-reliability-20260922`
开工时 `origin/staging` = `10baeb2fa865743c428806e115ac92ca9d92fb71`。未 push。未改 `page.tsx`。
既有 `TASK-first-paint-dead-screen-fallback-20260917.md` 尚未合入。本分支按它的未完成项做了唯一一套内联兜底,并完成断句后行断言清理。不另写一份 first-paint 进度。
## T1 兜底
- 根 layout 在主题脚本旁增加 `firstPaintFallbackBootScript()`。生成结果是经典 `<script>`:无 `type="module"`,无 `=>` / `const ` / `let ` / `class ` / 模板串 / `?.` / `??`。
- 触发:`setTimeout` **13000ms**(大于 `page.tsx` 的 8000 与准备揭幕的 4000)。回调里仍有 `.app-loading` 且 `documentElement` 的 `data-hydrated` 不是 `"1"` 时,替换 `.app-loading-content`。
- 文案:标题「这个页面没能加载完」,正文说明网络中断或浏览器过旧,并提示清除 Safari 本站数据或升级到 iOS 16.4 以上。唯一按钮「重新加载」,只在点击时 `location.reload()`。
- 揭幕标记写在 `useHomeShellRegistration`:`hydrated` 为真时设 `document.documentElement.dataset.hydrated = "1"`。`Home()` 未加 `useState` / `useRef`,`page.tsx` 未改。
- 复用 `.app-loading-error` 卡片,没有第二套转圈或骨架。
## T2 分流
六类不并成「接口慢」:`hydrate-timeout`、`chunk-404`、`chunk-parse`、`stale-cache`、`weak-network`、`api-error`。前五类可以由内联脚本在 bundle 没跑起来时标记;`api-error` 仍是 bundle 里的「连接云端服务超时」那一屏。匿名对象只有 `category`、`buildId`、`durationMs`。
真人清单:`docs/testing/home-bootstrap-reliability-20260922.md`。BUG-936 改为「兜底已补、根因待分流」,未标 resolved。低版本 Safari 与真实 chunk 失败未在浏览器验收。
## T3 后行断言
`frontend/src` 已无 `(?<=`。共用 `splitAfterSentencePunctuation`。表驱动用例覆盖中文句号、问号、叹号、分号、英文标点、换行、连续标点、结尾无标点,并与旧正则逐字对照。既有校正与报告断言未改。另立 BUG-998(不占用 996 / 997 / 999)。
## 验证
定向测试(未跑全量 `npm test`,未跑 `next build`):
| 命令 | 结果 |
| --- | --- |
| `npx tsx --test tests/sentence-split.test.ts tests/first-paint-fallback-contract.test.ts` | 6 pass / 0 fail |
| `npx tsx --test tests/home-shell-growth-contract.test.ts tests/agent-voice-copy-contract.test.ts tests/rectification-collect-prompt.test.ts tests/rectification-adopt-narration-20260904.test.ts` | 48 pass / 0 fail |
| `.\node_modules\.bin\tsc --noEmit` | 0 错 |
| `npm run lint` | 0 error / 119 warning(既有 warning,未顺手改) |
`home-shell-growth-contract` 通过,因此 `Home()` 的 `useState` / `useRef` 没有超过现有上限,行数也没有超过粗护栏。
@@ -0,0 +1,23 @@
# 真人 / 模拟清单 · 首页加载失败可恢复(2026-09-22)
BUG-936 的根因还没定。下面能做的是看兜底是否出现、点了是否只刷新一次。低版本 Safari 和故意弄坏 chunk 的浏览器检查是环境缺口,不是通过。
兜底补上不等于每个 Safari 用户都已修好。要完成分流,仍需要当事人的 iOS 版本,以及无痕窗口或清除本站数据之后是否恢复。
## 本机可以模拟
1. 打开首页 HTML。在开发者工具里把后续脚本请求阻断,或在元素面板里留下 `.app-loading` 且不要给 `<html>` 加上 `data-hydrated="1"`。等超过 13 秒。
2. 应看到标题「这个页面没能加载完」,正文说明网络中断或浏览器版本过旧,并提示 Safari 清除本站数据或升级到 iOS 16.4 以上。只有一枚「重新加载」。页面不会自己刷新。
3. 点「重新加载」才整页刷新。再停在同一屏时,仍然不会自动连刷。
4. 正常打开现代浏览器的首页:揭幕完成前不要闪出上面这句。揭幕后轨道环不再出现。
5. 把网络掐断后,如果客户端 bundle 已经跑起来并在 8 秒内失败,仍应是原来的「暂时无法进入 Jyotisha / 连接云端服务超时」,不要换成死屏这句。这两种失败不是同一件事。
## 环境缺口(本轮未做,不能写成通过)
- 没有当事人的 iPhone。不知道 iOS / Safari 版本,也不知道清除网站数据或无痕窗口之后是否恢复。BUG-936 根因因此仍是「待分流」。
- 没有一台 Safari 16.4 以下的设备。本仓断句已经去掉后行断言,但 Next 运行时仍有类静态块,低版本 Safari 仍可能在解析期失败。本清单不能证明那一类设备已经能进首页。
- 没有在已构建页面上故意把某个 `/_next/static/chunks/*.js` 改成 404 或截断后再用真实浏览器等 13 秒。自动化只在假 DOM 里跑过这段内联脚本。
## 不该看到的东西
兜底文案、按钮和 `data-bootstrap-failure` 都不写异常原文、出生资料、token 或请求体。观察标记如果出现,只有类别、构建标识和耗时。
+16
View File
@@ -650,6 +650,22 @@ Agent 的 live 标记只有 `InlineSpinner` 一种。曾经并存的 canvas 小
首页只揭幕一次。揭幕前的加载屏分两阶段:先取账户、模型目录与会话列表,再并行取今日星语与校正入口摘要,并预热校正分包;全部就绪或 4 秒预算到期(`BOOTSTRAP_PREPARE_TIMEOUT_MS`)才揭幕。揭幕后不得再出现任何阻塞等待或组件级 spinner:今日星语未到显示静态文案「今天的星语还没写出来。」(不带 `aria-busy`),校正卡用无摘要文案,内容到达后静默替换。登录后的空首页只剩问候语、今日星语卡、生时校正卡和输入框,没有主题卡。切换到消息尚未缓存的会话时消息区留白并只给 `sr-only` 文案,不转圈;揭幕后按侧栏顺序后台预取最近 5 条会话(`SESSION_PREFETCH_COUNT`)让常见切换零等待。轨道环消失后不得再换一套动效继续等。
### 首屏死屏兜底
首页揭幕前的轨道环是纯 CSS。客户端 bundle 没执行、关键 chunk 没加载或没解析成功时,8 秒的云端超时和 4 秒揭幕都不会发生,轨道会一直转。
根 layout 里有一段与 bundle 无关的内联经典脚本。它只在同时满足这两条时换掉 `.app-loading-content`:页面上仍有 `.app-loading`,且 `<html data-hydrated>` 不是 `"1"`。时限是 13 秒,大于载入账户的 8 秒加上准备阶段的 4 秒,正常揭幕不会闪出这一屏。应用揭幕(`hydrated` 为真)后,现有首页装配 hook 写下 `data-hydrated="1"`。
出现时复用 `.app-loading-error` 的卡片,不新增第二套转圈、骨架或自动刷新。
| 位置 | 文案 |
| --- | --- |
| 标题 | 这个页面没能加载完 |
| 正文 | 网络中断或浏览器版本过旧都会这样。可以重新加载试一次;如果反复出现,请在 Safari 设置里清除本站数据,或把系统升级到 iOS 16.4 以上。 |
| 唯一动作 | 「重新加载」,只在点击时 `location.reload()` |
接口超时仍是原来的「暂时无法进入 Jyotisha / 连接云端服务超时」,不要把脚本没跑起来写成云端慢。
### 报告生成等待态
个人报告要写几分钟,属于"等到有实质进度可报"的一类,因此在上表三类之外单独规定:**只有准备阶段用 `InlineSpinner`,进入写作阶段后换成分章进度,不再转圈。**
+12
View File
@@ -117,6 +117,18 @@ Jyotisha 的可见文案是产品的一部分。正确性红线(真实性、
余额不足时输入框占位写「余额不足,可在账户与点数中补充」,不要叫用户去会员页。
## 首屏没加载完
脚本没跑起来、关键文件没下到,或浏览器版本过旧时,转圈停在「正在载入账户」不会自己结束。超时后只留这一屏,不自动刷新:
| 位置 | 文案 |
|---|---|
| 标题 | 这个页面没能加载完 |
| 正文 | 网络中断或浏览器版本过旧都会这样。可以重新加载试一次;如果反复出现,请在 Safari 设置里清除本站数据,或把系统升级到 iOS 16.4 以上。 |
| 动作 | 重新加载 |
不写异常原文、接口路径、状态码或「正在加载」。接口超时仍用原来的「连接云端服务超时」那一屏,不要把脚本没跑起来说成云端慢。
- 不得虚构星盘事实或唯一出生分钟。
- 问哪道题、年份、选项语义仍由服务端 focus 唯一所有;Agent 用 `spokenPrompt` 写题干,不得改年份、不得改选项含义。正文不得出题、复述或改写题干——题干作为同一条消息里正文之后的独立段落出现。
- 正文不得断言界面当前状态(「界面上有/出现了下一问」);过渡用「接下来我们继续」这类中性句。
+3
View File
@@ -5,6 +5,7 @@ import { Toaster } from "@/components/ui/sonner";
import { StaleBuildGuard } from "@/components/stale-build-guard";
import { StaleClientRecovery } from "@/components/stale-client-recovery";
import { ViewportScrollLock } from "@/components/viewport-scroll-lock";
import { firstPaintFallbackBootScript } from "@/lib/first-paint-fallback";
import { themePreferenceBootScript } from "@/lib/theme-preference";
// Vendored Inter latin variable (OFL). Do not switch back to next/font/google;
@@ -54,6 +55,8 @@ export default function RootLayout({ children }: Readonly<{ children: React.Reac
{/* Synchronous on purpose: a pinned theme must be on <html> before the
first paint, or the page flashes the other theme on every load. */}
<script dangerouslySetInnerHTML={{ __html: themePreferenceBootScript }} />
{/* Classic script, not a module: it has to run when the client bundle never does. */}
<script dangerouslySetInnerHTML={{ __html: firstPaintFallbackBootScript() }} />
{enableReactDevTools && (
<>
<Script
@@ -80,6 +80,12 @@ export function useHomeShellRegistration(params: HomeShellRegistrationParams) {
visibleSessions,
} = params;
useEffect(() => {
// The inline dead-screen script stands down only after the app has revealed.
if (!hydrated || typeof document === "undefined") return;
document.documentElement.dataset.hydrated = "1";
}, [hydrated]);
useEffect(() => {
if (!hydrated || !account) {
registerShellControls(null);
+129
View File
@@ -0,0 +1,129 @@
/**
* Classic inline script for the home loading screen.
*
* It has to run when the client bundle never runs, so the generated string is
* ES5: no modules, no React, no reload until the button is clicked.
* The timer is longer than the 8000ms account budget plus the 4000ms prepare
* reveal, so a normal hydrate does not flash this copy.
*/
import {
HOME_BOOTSTRAP_FAILURE,
sanitizeBootstrapBuildId,
} from "./home-bootstrap-failure.ts";
export const FIRST_PAINT_FALLBACK_TIMEOUT_MS = 13_000;
export const FIRST_PAINT_FALLBACK_TITLE = "这个页面没能加载完";
export const FIRST_PAINT_FALLBACK_BODY =
"网络中断或浏览器版本过旧都会这样。可以重新加载试一次;如果反复出现,请在 Safari 设置里清除本站数据,或把系统升级到 iOS 16.4 以上。";
export const FIRST_PAINT_FALLBACK_ACTION = "重新加载";
function jsDoubleQuoted(value: string): string {
let escaped = "";
for (let index = 0; index < value.length; index += 1) {
const char = value.charAt(index);
const code = value.charCodeAt(index);
if (char === "\\") escaped += "\\\\";
else if (char === "\"") escaped += "\\\"";
else if (char === "\n") escaped += "\\n";
else if (char === "\r") escaped += "\\r";
else if (char === "<") escaped += "\\u003c";
else if (code === 0x2028 || code === 0x2029) escaped += `\\u${code.toString(16)}`;
else escaped += char;
}
return `"${escaped}"`;
}
export function firstPaintFallbackBootScript(buildId?: string): string {
const safeBuild = sanitizeBootstrapBuildId(buildId ?? process.env.NEXT_PUBLIC_GIT_COMMIT);
const timeout = String(FIRST_PAINT_FALLBACK_TIMEOUT_MS);
const hydrateTimeout = jsDoubleQuoted(HOME_BOOTSTRAP_FAILURE.hydrateTimeout);
const chunk404 = jsDoubleQuoted(HOME_BOOTSTRAP_FAILURE.chunk404);
const chunkParse = jsDoubleQuoted(HOME_BOOTSTRAP_FAILURE.chunkParse);
const staleCache = jsDoubleQuoted(HOME_BOOTSTRAP_FAILURE.staleCache);
const weakNetwork = jsDoubleQuoted(HOME_BOOTSTRAP_FAILURE.weakNetwork);
return `(function () {
var started = Date.now();
var buildId = ${jsDoubleQuoted(safeBuild)};
var category = ${hydrateTimeout};
function fromCache(source) {
try {
if (!source || !window.performance || !window.performance.getEntriesByName) return false;
var entries = window.performance.getEntriesByName(source);
var i;
for (i = 0; i < entries.length; i++) {
var entry = entries[i];
if (entry && entry.decodedBodySize > 0 && entry.transferSize === 0) return true;
}
} catch (ignore) {}
return false;
}
function classify(message, source) {
var text = String(message || "");
var from = String(source || "");
var combined = text + " " + from;
if (/SyntaxError|Unexpected token|Invalid regular expression/i.test(text)) {
return fromCache(from) ? ${staleCache} : ${chunkParse};
}
if (/ChunkLoadError|Loading chunk|Failed to fetch dynamically imported module|Failed to load/i.test(combined) || /\\/_next\\/static\\//.test(from)) {
if (fromCache(from)) return ${staleCache};
if (navigator && navigator.onLine === false) return ${weakNetwork};
return ${chunk404};
}
return "";
}
window.addEventListener("error", function (event) {
var message = "";
var source = "";
if (event) {
message = event.message || "";
source = event.filename || "";
if (!source && event.target && event.target.src) source = event.target.src;
}
var next = classify(message, source);
if (next) category = next;
}, true);
window.setTimeout(function () {
var root = document.documentElement;
if (!root || root.getAttribute("data-hydrated") === "1") return;
var loading = document.querySelector(".app-loading");
if (!loading) return;
if ((category === ${hydrateTimeout} || category === ${chunk404}) && navigator && navigator.onLine === false) {
category = ${weakNetwork};
}
var slot = loading.querySelector(".app-loading-content");
if (!slot) return;
if (loading.getAttribute("data-bootstrap-fallback") === "1") return;
loading.setAttribute("data-bootstrap-fallback", "1");
loading.setAttribute("aria-busy", "false");
if (loading.className.indexOf("app-loading-error") === -1) {
loading.className += " app-loading-error";
}
slot.setAttribute("role", "alert");
slot.innerHTML = "";
var title = document.createElement("strong");
title.appendChild(document.createTextNode(${jsDoubleQuoted(FIRST_PAINT_FALLBACK_TITLE)}));
var body = document.createElement("span");
body.appendChild(document.createTextNode(${jsDoubleQuoted(FIRST_PAINT_FALLBACK_BODY)}));
var actions = document.createElement("div");
actions.className = "app-loading-actions";
var button = document.createElement("button");
button.className = "button-primary";
button.type = "button";
button.appendChild(document.createTextNode(${jsDoubleQuoted(FIRST_PAINT_FALLBACK_ACTION)}));
button.onclick = function () { window.location.reload(); };
actions.appendChild(button);
slot.appendChild(title);
slot.appendChild(body);
slot.appendChild(actions);
var duration = Date.now() - started;
root.setAttribute("data-bootstrap-failure", category);
root.setAttribute("data-bootstrap-build", buildId);
root.setAttribute("data-bootstrap-duration", String(duration));
window.__jyotishaBootstrapObservation = { category: category, buildId: buildId, durationMs: duration };
}, ${timeout});
})();`;
}
@@ -0,0 +1,87 @@
/**
* Home bootstrap failures are not one "the API is slow" bucket.
*
* `api-error` is the in-bundle account/session path (HTTP status or the 8s
* cloud timeout). The other categories are what the inline dead-screen script
* can see before that bundle runs. Observation records only category, build
* id, and duration — never a payload, exception string, or birth data.
*/
export const HOME_BOOTSTRAP_FAILURE = {
hydrateTimeout: "hydrate-timeout",
chunk404: "chunk-404",
chunkParse: "chunk-parse",
staleCache: "stale-cache",
weakNetwork: "weak-network",
apiError: "api-error",
} as const;
export type HomeBootstrapFailureCategory =
(typeof HOME_BOOTSTRAP_FAILURE)[keyof typeof HOME_BOOTSTRAP_FAILURE];
export type HomeBootstrapFailureInput = {
/** `false` when the browser reports offline. `null` when the signal is absent. */
online: boolean | null;
scriptMessage?: string;
scriptSource?: string;
/** A failed script whose resource timing says it was served from cache. */
resourceFromCache?: boolean;
/** HTTP status from an account/session/model request the bundle actually made. */
apiStatus?: number | null;
/** Bundle-side cloud timeout (page.tsx 8000ms), not the dead-screen timer. */
apiTimedOut?: boolean;
};
const PARSE_FAILURE = /SyntaxError|Unexpected token|Invalid regular expression/i;
const CHUNK_LOAD_FAILURE = /ChunkLoadError|Loading chunk|Failed to fetch dynamically imported module|Failed to load/i;
export function sanitizeBootstrapBuildId(value: string | null | undefined): string {
if (typeof value !== "string") return "unknown";
const trimmed = value.trim();
return /^[0-9a-fA-F]{7,64}$/.test(trimmed) ? trimmed : "unknown";
}
export function classifyHomeBootstrapFailure(
input: HomeBootstrapFailureInput,
): HomeBootstrapFailureCategory {
if (typeof input.apiStatus === "number" && input.apiStatus >= 400) {
return HOME_BOOTSTRAP_FAILURE.apiError;
}
const message = input.scriptMessage ?? "";
const source = input.scriptSource ?? "";
const parseFailed = PARSE_FAILURE.test(message);
const chunkFailed = CHUNK_LOAD_FAILURE.test(`${message} ${source}`) || /\/_next\/static\//.test(source);
if (parseFailed) {
return input.resourceFromCache
? HOME_BOOTSTRAP_FAILURE.staleCache
: HOME_BOOTSTRAP_FAILURE.chunkParse;
}
if (chunkFailed) {
if (input.resourceFromCache) return HOME_BOOTSTRAP_FAILURE.staleCache;
if (input.online === false) return HOME_BOOTSTRAP_FAILURE.weakNetwork;
return HOME_BOOTSTRAP_FAILURE.chunk404;
}
if (input.online === false) return HOME_BOOTSTRAP_FAILURE.weakNetwork;
if (input.apiTimedOut) return HOME_BOOTSTRAP_FAILURE.apiError;
return HOME_BOOTSTRAP_FAILURE.hydrateTimeout;
}
export type BootstrapObservation = {
category: HomeBootstrapFailureCategory;
buildId: string;
durationMs: number;
};
export function buildBootstrapObservation(
category: HomeBootstrapFailureCategory,
buildId: string,
durationMs: number,
): BootstrapObservation {
const safeDuration = Number.isFinite(durationMs) ? Math.max(0, Math.round(durationMs)) : 0;
return {
category,
buildId: sanitizeBootstrapBuildId(buildId),
durationMs: safeDuration,
};
}
@@ -3,6 +3,7 @@ import {
computeEvidenceHash,
safeParseServerReportDocument,
} from "./personal-report-contract.server-core.ts";
import { splitAfterSentencePunctuation } from "./sentence-split.ts";
import {
BLOCKED_DETERMINISTIC_PHRASES,
CHART_IDS,
@@ -2892,8 +2893,10 @@ export function findForbiddenDeterministicClaims(textValue: string): ForbiddenCl
}
export function splitSentences(textValue: string): string[] {
return textValue
.split(/(?<=[。!?!?;;])\s*|\n+/u)
return splitAfterSentencePunctuation(textValue, "。!?!?;;", {
consumeFollowingWhitespace: true,
splitOnNewlines: true,
})
.map((part) => part.trim())
.filter((part) => part.length > 0);
}
@@ -6,6 +6,7 @@
* no unique-minute claim.
*/
import { splitAfterSentencePunctuation } from "../sentence-split.ts";
import { PROBE_EXPLAIN_COPY } from "./v9/probe-explain.ts";
import { STEP_STATE_COPY } from "./v9/step-state.ts";
import {
@@ -66,12 +67,11 @@ export function openingSpokenBody(range: readonly [string, string] | null): stri
}
const OPENING_YEAR = /(?:19|20)\d{2}/;
const OPENING_SENTENCE = /(?<=[。!?])/;
export function isAcceptableOpeningBody(body: string): boolean {
const spoken = body.trim();
if (!spoken || OPENING_YEAR.test(spoken)) return false;
const sentences = spoken.split(OPENING_SENTENCE).map((part) => part.trim()).filter(Boolean);
const sentences = splitAfterSentencePunctuation(spoken, "。!?").map((part) => part.trim()).filter(Boolean);
if (sentences.length === 0 || sentences.length > 4) return false;
const hits = OPENING_COLLECT_DOMAINS.filter((domain) => spoken.includes(domain)).length;
return hits >= 5;
@@ -280,8 +280,7 @@ export function withCompareFailedRetryNotice(body: string): string {
export function stripRangeProgressClaims(body: string): string {
const spoken = body.trim();
if (!spoken) return "";
const kept = spoken
.split(/(?<=[。!?])/)
const kept = splitAfterSentencePunctuation(spoken, "。!?")
.map((part) => part.trim())
.filter((part) => part && !/收窄|范围从/.test(part));
return kept.join("");
@@ -511,8 +510,7 @@ export function engineMeaningToDisplayCopy(meaning: string | null | undefined):
(text, [pattern, next]) => text.replace(pattern, next),
meaning,
);
return replaced
.split(/(?<=[。!?;;])\s*/)
return splitAfterSentencePunctuation(replaced, "。!?;;", { consumeFollowingWhitespace: true })
.filter((sentence) => sentence.trim() && !INSTRUCTION_TONE_SENTENCE.test(sentence))
.join("")
.replace(/\s+/g, " ")
@@ -6,6 +6,7 @@
* checked after adopt. It never changes sessionOutcome or focus.
*/
import { splitAfterSentencePunctuation } from "../../sentence-split.ts";
import type { RectificationDecision } from "../core/rectification-decision.ts";
import { openingRangeFromCandidateRange } from "../user-copy.ts";
import { RECTIFICATION_USER_COPY } from "../user-copy.ts";
@@ -272,7 +273,9 @@ export function validateAdoptNarration(
const trimmed = text.trim().replace(/\s+/g, " ");
if (!trimmed) return { ok: false, reason: "empty" };
if (trimmed.length > ADOPT_NARRATION_MAX_CHARS) return { ok: false, reason: "length" };
const sentences = trimmed.split(/(?<=[。!?!?.])/).map((item) => item.trim()).filter(Boolean);
const sentences = splitAfterSentencePunctuation(trimmed, "。!?!?.")
.map((item) => item.trim())
.filter(Boolean);
if (sentences.some((item) => /[??]$/.test(item))) {
return { ok: false, reason: "question" };
}
@@ -4,9 +4,14 @@
* “did this body already ask”.
*/
const SENTENCE_SPLIT = /(?<=[。!??\n])/;
import { splitAfterSentencePunctuation } from "../../sentence-split.ts";
const NARRATIVE_SENTENCE = /范围|记下|对照|\d{1,2}:\d{2}/;
function splitCollectBody(body: string): string[] {
return splitAfterSentencePunctuation(body, "。!??\n");
}
export const EVIDENCE_VALUE_JUDGMENT_PHRASES = [
"很有帮助",
"很有价值",
@@ -15,7 +20,7 @@ export const EVIDENCE_VALUE_JUDGMENT_PHRASES = [
] as const;
function spokenSentences(body: string): string[] {
return body.split(SENTENCE_SPLIT).map((part) => part.trim()).filter(Boolean);
return splitCollectBody(body).map((part) => part.trim()).filter(Boolean);
}
export function trimEvidenceTurnBody(body: string, options?: { maxSentences?: number }): string {
@@ -68,7 +73,7 @@ export function stripQuestionSentences(body: string, stem: string): string {
if (!spoken) return "";
const kept: string[] = [];
let dropContinuation = false;
for (const part of spoken.split(SENTENCE_SPLIT)) {
for (const part of splitCollectBody(spoken)) {
const text = part.trim();
if (!text) continue;
if (isQuestionSentence(text, prompt)) {
@@ -1,3 +1,4 @@
import { splitAfterSentencePunctuation } from "../../sentence-split.ts";
import { parseAgentChoiceCopy } from "./choice-card";
import type { V9CaseDossier } from "./tool-service";
import { RECTIFICATION_USER_COPY } from "../user-copy.ts";
@@ -98,7 +99,7 @@ const MARKER_GROUPS: readonly (readonly string[])[] = [
export function extractSpokenQuestion(spoken: string): string | null {
const parts = spoken
.split(/\n{2,}/)
.flatMap((block) => block.split(/(?<=[。!??])\s*/u))
.flatMap((block) => splitAfterSentencePunctuation(block, "。!??", { consumeFollowingWhitespace: true }))
.map((part) => part.trim())
.filter((part) => part.length > 0 && /[??]/.test(part));
const last = parts.at(-1);
@@ -152,7 +153,7 @@ export function bindSpokenToOpenQuestion(spoken: string, nextQuestion: string |
if (!lock) return spoken.trim();
const ack = spoken
.split(/\n{2,}/)
.flatMap((block) => block.split(/(?<=[。!])\s*/u))
.flatMap((block) => splitAfterSentencePunctuation(block, "。!", { consumeFollowingWhitespace: true }))
.map((part) => part.trim())
.filter((part) => (
part.length > 0
+63
View File
@@ -0,0 +1,63 @@
/**
* Split after sentence punctuation and keep the punctuation on the left piece.
*
* Replaces lookbehind splits. Those are syntax errors on Safari before 16.4,
* so this file must not grow one. A newline inside `punctuation`
* is a kept boundary. `splitOnNewlines` instead discards `\n+` runs. Do not
* combine that option with a newline in `punctuation`.
*/
export type SentenceSplitOptions = {
/** Drop whitespace after the kept punctuation. Same as a trailing `\s*`. */
consumeFollowingWhitespace?: boolean;
/** Discard `\n+` even when the newline is not after punctuation. */
splitOnNewlines?: boolean;
};
function isJsWhitespace(char: string): boolean {
return char.length === 1 && /\s/u.test(char);
}
export function splitAfterSentencePunctuation(
text: string,
punctuation: string,
options?: SentenceSplitOptions,
): string[] {
const consumeFollowingWhitespace = options?.consumeFollowingWhitespace === true;
const splitOnNewlines = options?.splitOnNewlines === true;
const marks = new Set(punctuation);
const parts: string[] = [];
let start = 0;
let index = 0;
while (index < text.length) {
const char = text.charAt(index);
if (marks.has(char)) {
const boundaryEnd = index + 1;
let delimiterEnd = boundaryEnd;
if (consumeFollowingWhitespace) {
while (delimiterEnd < text.length && isJsWhitespace(text.charAt(delimiterEnd))) {
delimiterEnd += 1;
}
}
// A zero-width match at the end of the string does not add a trailing empty piece.
if (delimiterEnd === text.length && delimiterEnd === boundaryEnd) break;
parts.push(text.slice(start, boundaryEnd));
start = delimiterEnd;
index = delimiterEnd;
continue;
}
if (splitOnNewlines && char === "\n") {
let newlineEnd = index + 1;
while (newlineEnd < text.length && text.charAt(newlineEnd) === "\n") newlineEnd += 1;
parts.push(text.slice(start, index));
start = newlineEnd;
index = newlineEnd;
continue;
}
index += 1;
}
parts.push(text.slice(start));
return parts;
}
@@ -0,0 +1,304 @@
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import test from "node:test";
import vm from "node:vm";
import {
FIRST_PAINT_FALLBACK_ACTION,
FIRST_PAINT_FALLBACK_BODY,
FIRST_PAINT_FALLBACK_TIMEOUT_MS,
FIRST_PAINT_FALLBACK_TITLE,
firstPaintFallbackBootScript,
} from "../src/lib/first-paint-fallback.ts";
import {
buildBootstrapObservation,
classifyHomeBootstrapFailure,
HOME_BOOTSTRAP_FAILURE,
} from "../src/lib/home-bootstrap-failure.ts";
const projectRoot = new URL("../", import.meta.url);
const read = (path: string) => readFileSync(new URL(path, projectRoot), "utf8");
class FakeNode {
tag: string;
className = "";
type = "";
src = "";
onclick: (() => void) | null = null;
children: FakeNode[] = [];
text: string;
private readonly attributes = new Map<string, string>();
constructor(tag: string, text = "") {
this.tag = tag;
this.text = text;
}
setAttribute(name: string, value: string) {
this.attributes.set(name, value);
}
getAttribute(name: string) {
return this.attributes.get(name) ?? null;
}
appendChild(child: FakeNode) {
this.children.push(child);
return child;
}
set innerHTML(value: string) {
if (value === "") this.children = [];
}
querySelector(selector: string): FakeNode | null {
for (const child of this.children) {
if (hasClass(child, selector)) return child;
const nested = child.querySelector(selector);
if (nested) return nested;
}
return null;
}
}
function hasClass(node: FakeNode, selector: string) {
if (!selector.startsWith(".")) return false;
return node.className.split(/\s+/).includes(selector.slice(1));
}
function textOf(node: FakeNode): string {
if (node.tag === "#text") return node.text;
return node.children.map(textOf).join("");
}
function findTag(node: FakeNode, tag: string): FakeNode | null {
if (node.tag === tag) return node;
for (const child of node.children) {
const found = findTag(child, tag);
if (found) return found;
}
return null;
}
type BootOptions = {
hydrated?: boolean;
withLoading?: boolean;
online?: boolean;
entries?: (name: string) => Array<{ decodedBodySize: number; transferSize: number }>;
buildId?: string;
};
function boot(options: BootOptions = {}) {
const html = new FakeNode("html");
const body = new FakeNode("body");
const loading = new FakeNode("main");
loading.className = "app-loading";
loading.setAttribute("aria-busy", "true");
const slot = new FakeNode("div");
slot.className = "app-loading-content";
const oldTitle = new FakeNode("strong");
oldTitle.appendChild(new FakeNode("#text", "正在载入账户"));
slot.appendChild(oldTitle);
loading.appendChild(slot);
body.appendChild(loading);
html.appendChild(body);
if (options.hydrated) html.setAttribute("data-hydrated", "1");
if (options.withLoading === false) body.children = [];
let reloaded = false;
let delay = 0;
let fire: () => void = () => {
throw new Error("fallback timer was not installed");
};
let onError: ((event: Record<string, unknown>) => void) | null = null;
let capture: boolean | undefined;
const performance = {
getEntriesByName(name: string) {
return options.entries?.(name) ?? [];
},
};
const window = {
performance,
location: {
reload() {
reloaded = true;
},
},
setTimeout(fn: () => void, ms: number) {
delay = ms;
fire = fn;
return 1;
},
addEventListener(type: string, listener: (event: Record<string, unknown>) => void, useCapture?: boolean) {
assert.equal(type, "error");
onError = listener;
capture = useCapture;
},
};
const document = {
documentElement: html,
createElement(tag: string) {
return new FakeNode(tag);
},
createTextNode(text: string) {
return new FakeNode("#text", text);
},
querySelector(selector: string) {
return html.querySelector(selector);
},
};
const navigator = { onLine: options.online !== false };
vm.runInNewContext(firstPaintFallbackBootScript(options.buildId ?? "abcdef0"), {
window,
document,
navigator,
performance,
});
return {
html,
loading,
slot,
delay,
capture,
fire,
emitError(event: Record<string, unknown>) {
onError?.(event);
},
reloaded: () => reloaded,
observation: () => (window as { __jyotishaBootstrapObservation?: { category: string; buildId: string; durationMs: number } }).__jyotishaBootstrapObservation,
};
}
test("fallback script is classic ES5 and waits past the bootstrap budgets", () => {
const page = read("src/app/(app)/page.tsx");
const bootstrap = read("src/lib/home-bootstrap.ts");
const layout = read("src/app/layout.tsx");
const hook = read("src/hooks/use-home-shell-registration.ts");
assert.match(page, /\}, 8000\);/);
assert.match(bootstrap, /BOOTSTRAP_PREPARE_TIMEOUT_MS = 4000/);
assert.ok(FIRST_PAINT_FALLBACK_TIMEOUT_MS > 8000 + 4000);
const script = firstPaintFallbackBootScript("abcdef0");
assert.match(script, new RegExp(String(FIRST_PAINT_FALLBACK_TIMEOUT_MS)));
for (const forbidden of ["=>", "const ", "let ", "class ", "`", "${", "?.", "??"]) {
assert.equal(script.includes(forbidden), false, forbidden);
}
assert.doesNotMatch(script, /type\s*=\s*["']module["']/);
assert.equal(script.includes("连接云端服务超时"), false);
assert.equal(script.includes(HOME_BOOTSTRAP_FAILURE.apiError), false);
assert.equal(script.includes(FIRST_PAINT_FALLBACK_TITLE), true);
assert.equal(script.includes(FIRST_PAINT_FALLBACK_BODY), true);
assert.equal(script.includes(FIRST_PAINT_FALLBACK_ACTION), true);
assert.match(layout, /firstPaintFallbackBootScript\(\)/);
assert.doesNotMatch(layout, /firstPaintFallbackBootScript\(\)[\s\S]{0,80}type\s*=\s*["']module["']/);
assert.match(hook, /document\.documentElement\.dataset\.hydrated = "1"/);
assert.equal(page.includes("dataset.hydrated"), false);
assert.match(page, /连接云端服务超时/);
});
test("fallback replaces the loading screen only while it is still unhydrated", () => {
const shown = boot();
assert.equal(shown.delay, FIRST_PAINT_FALLBACK_TIMEOUT_MS);
assert.equal(shown.capture, true);
shown.fire();
assert.equal(shown.reloaded(), false);
assert.match(textOf(shown.slot), new RegExp(FIRST_PAINT_FALLBACK_TITLE));
assert.match(textOf(shown.slot), new RegExp(FIRST_PAINT_FALLBACK_BODY));
assert.doesNotMatch(textOf(shown.slot), /正在载入账户|SyntaxError|token|出生/);
assert.match(shown.loading.className, /app-loading-error/);
assert.equal(shown.loading.getAttribute("aria-busy"), "false");
assert.equal(shown.slot.getAttribute("role"), "alert");
const button = findTag(shown.slot, "button");
assert.ok(button);
assert.equal(button.type, "button");
assert.equal(textOf(button), FIRST_PAINT_FALLBACK_ACTION);
button.onclick?.();
assert.equal(shown.reloaded(), true);
const observation = shown.observation();
assert.deepEqual(observation && Object.keys(observation).sort(), ["buildId", "category", "durationMs"]);
assert.equal(observation?.category, HOME_BOOTSTRAP_FAILURE.hydrateTimeout);
assert.equal(observation?.buildId, "abcdef0");
assert.equal(typeof observation?.durationMs, "number");
assert.equal(shown.html.getAttribute("data-bootstrap-failure"), HOME_BOOTSTRAP_FAILURE.hydrateTimeout);
const revealed = boot({ hydrated: true });
revealed.fire();
assert.match(textOf(revealed.slot), /正在载入账户/);
assert.equal(revealed.reloaded(), false);
assert.equal(revealed.observation(), undefined);
const otherPage = boot({ withLoading: false });
otherPage.fire();
assert.equal(otherPage.observation(), undefined);
assert.equal(otherPage.reloaded(), false);
});
test("fallback distinguishes script failures without treating them as a slow API", () => {
const cases: Array<{ name: string; online?: boolean; event?: Record<string, unknown>; cached?: boolean; category: string }> = [
{
name: "parse",
event: { message: "Uncaught SyntaxError: Unexpected token '{'", filename: "https://example/_next/static/chunks/app.js" },
category: HOME_BOOTSTRAP_FAILURE.chunkParse,
},
{
name: "missing",
event: { message: "Loading chunk 12 failed.", filename: "https://example/_next/static/chunks/12.js" },
category: HOME_BOOTSTRAP_FAILURE.chunk404,
},
{
name: "cached",
cached: true,
event: { message: "Uncaught SyntaxError: Unexpected token", filename: "https://example/_next/static/chunks/stale.js" },
category: HOME_BOOTSTRAP_FAILURE.staleCache,
},
{ name: "offline", online: false, category: HOME_BOOTSTRAP_FAILURE.weakNetwork },
];
for (const sample of cases) {
const page = boot({
online: sample.online,
buildId: "abc1234",
entries: () => (sample.cached ? [{ decodedBodySize: 20, transferSize: 0 }] : []),
});
if (sample.event) page.emitError(sample.event);
page.fire();
assert.equal(page.observation()?.category, sample.category, sample.name);
assert.equal(page.html.getAttribute("data-bootstrap-build"), "abc1234");
assert.doesNotMatch(textOf(page.slot), /SyntaxError|Loading chunk|Unexpected token/);
}
const noise = boot();
noise.emitError({ message: "ResizeObserver loop limit exceeded", filename: "" });
noise.fire();
assert.equal(noise.observation()?.category, HOME_BOOTSTRAP_FAILURE.hydrateTimeout);
assert.equal(classifyHomeBootstrapFailure({ online: true, apiStatus: 500 }), HOME_BOOTSTRAP_FAILURE.apiError);
assert.equal(classifyHomeBootstrapFailure({ online: true, apiTimedOut: true }), HOME_BOOTSTRAP_FAILURE.apiError);
assert.equal(
classifyHomeBootstrapFailure({ online: false, scriptMessage: "Loading chunk 3 failed.", scriptSource: "/_next/static/chunks/3.js" }),
HOME_BOOTSTRAP_FAILURE.weakNetwork,
);
assert.equal(
classifyHomeBootstrapFailure({ online: true, scriptMessage: "SyntaxError: Unexpected token", resourceFromCache: true }),
HOME_BOOTSTRAP_FAILURE.staleCache,
);
assert.equal(classifyHomeBootstrapFailure({ online: true }), HOME_BOOTSTRAP_FAILURE.hydrateTimeout);
const distinct = new Set([
classifyHomeBootstrapFailure({ online: true }),
classifyHomeBootstrapFailure({ online: true, scriptMessage: "Loading chunk 1 failed.", scriptSource: "/_next/static/chunks/1.js" }),
classifyHomeBootstrapFailure({ online: true, scriptMessage: "SyntaxError: Unexpected token" }),
classifyHomeBootstrapFailure({ online: true, scriptMessage: "SyntaxError: Unexpected token", resourceFromCache: true }),
classifyHomeBootstrapFailure({ online: false }),
classifyHomeBootstrapFailure({ online: true, apiStatus: 503 }),
]);
assert.equal(distinct.size, 6);
assert.equal(distinct.has(HOME_BOOTSTRAP_FAILURE.apiError), true);
const observation = buildBootstrapObservation(HOME_BOOTSTRAP_FAILURE.apiError, "not a sha <script>", -4);
assert.deepEqual(observation, {
category: HOME_BOOTSTRAP_FAILURE.apiError,
buildId: "unknown",
durationMs: 0,
});
assert.deepEqual(Object.keys(observation).sort(), ["buildId", "category", "durationMs"]);
});
+149
View File
@@ -0,0 +1,149 @@
import assert from "node:assert/strict";
import { readdirSync, readFileSync } from "node:fs";
import { join } from "node:path";
import test from "node:test";
import { fileURLToPath } from "node:url";
import {
splitAfterSentencePunctuation,
type SentenceSplitOptions,
} from "../src/lib/sentence-split.ts";
const sourceRoot = fileURLToPath(new URL("../src/", import.meta.url));
function walk(dir: string): string[] {
const files: string[] = [];
for (const entry of readdirSync(dir, { withFileTypes: true })) {
const path = join(dir, entry.name);
if (entry.isDirectory()) files.push(...walk(path));
else files.push(path);
}
return files;
}
test("frontend/src has no lookbehind assertions", () => {
const textExtensions = new Set([".ts", ".tsx", ".js", ".jsx", ".mjs", ".cjs", ".css", ".json", ".md"]);
const hits: string[] = [];
for (const file of walk(sourceRoot)) {
const dot = file.lastIndexOf(".");
if (!textExtensions.has(file.slice(dot))) continue;
const source = readFileSync(file, "utf8");
if (source.includes("(?<=")) hits.push(file);
}
assert.deepEqual(hits, []);
});
type HabitCase = {
habit: string;
input: string;
punctuation: string;
options?: SentenceSplitOptions;
expected: string[];
};
const habits: HabitCase[] = [
{ habit: "chinese-period", input: "你好。世界", punctuation: "。!?", expected: ["你好。", "世界"] },
{ habit: "chinese-period", input: "甲。乙。丙", punctuation: "。!?", expected: ["甲。", "乙。", "丙"] },
{ habit: "question", input: "你好?世界", punctuation: "。!?", expected: ["你好?", "世界"] },
{ habit: "question", input: "真的吗?是的。", punctuation: "。!?", expected: ["真的吗?", "是的。"] },
{ habit: "exclamation", input: "你好!世界", punctuation: "。!?", expected: ["你好!", "世界"] },
{ habit: "exclamation", input: "停!别走。", punctuation: "。!?", expected: ["停!", "别走。"] },
{
habit: "semicolon",
input: "你好;世界",
punctuation: "。!?;;",
options: { consumeFollowingWhitespace: true },
expected: ["你好;", "世界"],
},
{
habit: "semicolon",
input: "甲;乙;丙",
punctuation: "。!?;;",
options: { consumeFollowingWhitespace: true },
expected: ["甲;", "乙;", "丙"],
},
{
habit: "english-punctuation",
input: "Hello! World",
punctuation: "。!?!?.",
expected: ["Hello!", " World"],
},
{
habit: "english-punctuation",
input: "Hello. World",
punctuation: "。!?!?.",
expected: ["Hello.", " World"],
},
{
habit: "newline",
input: "你好\n世界",
punctuation: "。!?!?;;",
options: { consumeFollowingWhitespace: true, splitOnNewlines: true },
expected: ["你好", "世界"],
},
{
habit: "newline",
input: "甲。\n乙",
punctuation: "。!?!?;;",
options: { consumeFollowingWhitespace: true, splitOnNewlines: true },
expected: ["甲。", "乙"],
},
{ habit: "repeated-punctuation", input: "你好。。世界", punctuation: "。!?", expected: ["你好。", "。", "世界"] },
{ habit: "repeated-punctuation", input: "啊!!停", punctuation: "。!?!?", expected: ["啊!", "!", "停"] },
{ habit: "no-trailing-punctuation", input: "没有标点的一段", punctuation: "。!?", expected: ["没有标点的一段"] },
{ habit: "no-trailing-punctuation", input: "结尾无句号就这样", punctuation: "。!?", expected: ["结尾无句号就这样"] },
];
test("sentence splits keep punctuation and match each call-site habit", () => {
const seen = new Map<string, number>();
for (const sample of habits) {
seen.set(sample.habit, (seen.get(sample.habit) ?? 0) + 1);
assert.deepEqual(
splitAfterSentencePunctuation(sample.input, sample.punctuation, sample.options),
sample.expected,
sample.habit,
);
}
for (const [habit, count] of seen) {
assert.ok(count >= 2, `${habit} needs two cases, has ${count}`);
}
});
test("sentence splits stay character-identical to the removed lookbehind patterns", () => {
const presets: Array<{
punctuation: string;
options?: SentenceSplitOptions;
legacy: RegExp;
}> = [
{ punctuation: "。!?", legacy: /(?<=[。!?])/ },
{ punctuation: "。!?;;", options: { consumeFollowingWhitespace: true }, legacy: /(?<=[。!?;;])\s*/ },
{
punctuation: "。!?!?;;",
options: { consumeFollowingWhitespace: true, splitOnNewlines: true },
legacy: /(?<=[。!?!?;;])\s*|\n+/u,
},
{ punctuation: "。!?!?.", legacy: /(?<=[。!?!?.])/ },
{ punctuation: "。!??\n", legacy: /(?<=[。!??\n])/ },
{ punctuation: "。!??", options: { consumeFollowingWhitespace: true }, legacy: /(?<=[。!??])\s*/u },
{ punctuation: "。!", options: { consumeFollowingWhitespace: true }, legacy: /(?<=[。!])\s*/u },
];
const atoms = ["", "甲", "Hello", "1", "。", "!", "?", ";", ";", "!", "?", ".", "\n", "\n\n", " ", " ", "\t", "\r\n", "\u3000", "没有标点"];
const samples = new Set<string>(["", "你好。", "你好。 ", "你好。 "]);
for (const left of atoms) {
for (const right of atoms) samples.add(left + right);
}
for (const left of atoms) {
for (const mark of ["。", "!", "\n", " "]) {
for (const right of ["乙", "", "\n", "."]) samples.add(left + mark + right);
}
}
for (const preset of presets) {
for (const sample of samples) {
assert.deepEqual(
splitAfterSentencePunctuation(sample, preset.punctuation, preset.options),
sample.split(preset.legacy),
`${preset.punctuation} :: ${JSON.stringify(sample)}`,
);
}
}
});