fix(test): wait for a PostgreSQL fixture slot while the queue moves (BUG-1254)
Independent Staging Quality Gate / validate (push) Successful in 14m10s
Independent Staging Quality Gate / publish (push) Successful in 4m3s

The slot wait was a fixed 5 minutes from the moment a file started queueing.
With 46 fixture files sharing 2 slots the queue outlasted that, so whichever
file came last failed the gate at random (runs 1742 and 1746). The deadline
now restarts whenever a slot changes hands; only 10 minutes with no slot
changing hands (a hung holder) fails. The 2-slot cap from BUG-281 stays.

Full suite 4967 / fail 24, identical to 171a3f25; tsc 0; lint 0 errors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017eEAG8HD3mm8gsKXgk8uU8
This commit is contained in:
Jesse_Chen
2026-10-06 23:21:48 +08:00
co-authored by Claude Opus 5.5
parent fe9d03a9b7
commit 52d14e8fef
4 changed files with 71 additions and 7 deletions
+4 -4
View File
@@ -16781,16 +16781,16 @@
## BUG-1254 | staging 门禁偶发红:数据库测试排队等 PostgreSQL 槽位超时
- 状态:investigating(已定位原因,未修)
- 状态:fixed-pending-verify(分支 `codex/postgres-fixture-queue-20261006`;需在门禁上连续通过才标 resolved)
- 首次发现 / 最近更新:2026-10-06 / 2026-10-06
- 来源:门禁 run 1746(`171a3f25`)「Frontend and database tests」红;同日 run 1742(`da226db1`)同一步红,随后同代码的 run 1743(`e62d0a32`)通过。
- 影响面:Gitea `backend-quality-gate.yml` 的 `npm test --prefix frontend`;`frontend/tests/helpers/postgres-fixture.ts`。与被测提交的改动无关时也会红,阻断部署。
- 现象:`not ok - corrective migration clears untrusted clocks and serializes concurrent fourth/fifth people`,原文 `timed out waiting for a PostgreSQL fixture slot (2 concurrent compose networks). Docker address pools cannot host one network per parallel test file.`,`ERR_TEST_FAILURE`。
- 触发条件:全量并行 `npm test` 时,使用 `startPostgresFixture()` 的测试文件(2026-10-06 共 46 个)同时排队抢 2 个槽位,每个文件最多等 `FIXTURE_SLOT_WAIT_MS` = 5 分钟;排在后面的文件等满 5 分钟即失败,谁失败取决于调度,所以同一代码时红时绿。
- 根因:BUG-281 为防 Docker 地址池耗尽把并发限到 2 个槽位,但等待上限是固定 5 分钟;数据库测试文件从当时约 20 个增长到 46 个,排队总时长已超过 5 分钟。属于 BUG-281 修复的容量假设失效,不是新代码引入。
- 修复:未做。候选(待决定):①等待上限改为「只要持槽进程还活着就继续等」或按排队文件数放大;②`run-tests.mjs` 把 `database-*` 等用 fixture 的文件拆到单独一轮串行跑(与 `test:db` 同口径),其余仍并行。不改 `.gitea/workflows/**`。
- 验证:本机无可用 Docker,未复现;依据为门禁日志原文与 `postgres-fixture.ts` 的槽位 / 等待常量。
- 防复发:待修复时补:fixture 数量增长不应让门禁变成随机红。
- 修复:产品要求根治(2026-10-06)。`postgres-fixture.ts` 的等待从「从开始排队起固定 5 分钟」改为「按进展续期」:每次轮询记录各槽位的持有者(`slot:pid`),持有者有变化(有人释放或接手)就把截止时间推到现在 + `FIXTURE_SLOT_STALL_MS`(10 分钟);只有连续 10 分钟没有任何槽位易手(持有者疑似卡死)才失败,报错写明这一点。槽位上限 2 不变(BUG-281 的地址池保护不动);不改 `.gitea/workflows/**`。曾考虑把数据库测试拆成单独串行一轮,未采用:改动面大、门禁更慢,且排队本身已由槽位串行化。
- 验证:本机无可用 Docker,未复现;依据为门禁日志原文与 `postgres-fixture.ts` 的槽位 / 等待常量。`postgres-fixture-contract.test.ts` 新增 1 条(续期纯函数三种情形 + 取槽循环每次轮询重算截止时间),4/4 通过;全量 4967 项 fail 24,与 `171a3f25` 名单逐条一致;tsc 0、lint 0 error。门禁上的实际效果待推送后看。
- 防复发:等待不再与 fixture 文件数量挂钩;契约测试禁止再出现固定等待常量 `FIXTURE_SLOT_WAIT_MS`。若门禁总时长逼近 validate 的 45 分钟上限,再考虑提高槽位数(先查地址池余量)。
- 相关记录:BUG-281(槽位上限的来源)、BUG-266(runner 资源回收)。
- 复发自:BUG-281(同一机制的容量问题以另一种形式出现)。
- 修复版本:—
+1
View File
@@ -427,3 +427,4 @@
| `TASK-consult-conversational-answer-20261006.md` | `PROGRESS-consult-conversational-answer-20261006.md` | **普通对话首轮去汇报骨架,改成聊天**(10-06 产品反馈「像机器在汇报」):首轮取消全部 `##`(三个固定节 + 按对象标题,多人改分段落点名)、行动不强制(盘上有具体指向才顺口一句,问「怎么办」再展开)、首轮三到六段;示范删可抄的行动句(实测被逐字照抄);思考栏去「这周可以做什么」。推翻 10-01 D2 与 D8 首轮部分(BUG-1244~1245) | 已验收 | 28113fa0 验收未过 → 修复单 `TASK-consult-conversational-answer-fix-20261006.md` 由 Claude 直接执行(含去掉助手头像),复验通过后推 staging;真机清单 10 步待产品 |
| `TASK-rectification-in-chat-step1-20261006.md` | `PROGRESS-rectification-in-chat-step1-20261006.md` | **生时校正并入聊天 · 第一步**:删首页「生时校正」按钮与文字口令;普通对话里用户说不准时由 agent 分清读法 / 时间问题,时间问题出提议卡(本人、开关、价格服务端校验,同会话最多主动提一次);点卡带来源打开校正;采用 / 未采用 / 手动返回都回到原对话,分隔线 + 旧回答标「按校正前的时间」+ 点了才发的「按新时间重新看」;模型历史标注旧时间。须等 dup-adopt 单合入后开工;第二步(卡片嵌进同一对话)另立单(BUG-1247~1251) | 待领取 | — |
| —(直接执行,无任务书) | `PROGRESS-consult-readable-20261006.md` | **回答铺满一行、输出不抖、不念分析清单**(10-06 真机截图):答案正文去 `text-wrap: pretty`(WebKit 整段等长 + 逐字重排,BUG-1252);清单开头写明只用于分析、婚恋三层改生活说法、字段白话对照、括号最多两条(BUG-1253) | 已验收 | Claude 直接执行;真机清单 `docs/testing/consult-readable-20261006.md` 待产品 |
| —(直接执行,无任务书) | —(记录在 BUG-1254) | **门禁偶发红:数据库测试排队等槽位超时**:46 个 fixture 文件抢 2 个槽位、固定等 5 分钟,排在后面的随机失败;改为槽位易手即续期、连续 10 分钟无进展才失败(BUG-1254) | 已验收 | Claude 直接执行;门禁实际效果待看 |
+42 -3
View File
@@ -79,7 +79,15 @@ function releasePort(reservation: PortReservation): void {
}
const DEFAULT_MAX_CONCURRENT_POSTGRES_FIXTURES = 2;
const FIXTURE_SLOT_WAIT_MS = 5 * 60 * 1000;
/**
* How long the slot queue may stand still before a waiter gives up (BUG-1254).
* It used to be a fixed 5-minute wait from the moment a file started queueing;
* with 46 fixture files sharing 2 slots the queue itself outlasted that, and
* whichever file happened to be last failed the gate at random. The window now
* restarts whenever a slot changes hands, so a moving queue is waited out and
* only a queue where no slot changes for this long (a hung holder) fails.
*/
export const FIXTURE_SLOT_STALL_MS = 10 * 60 * 1000;
type FixtureSlot = { id: number; release(): void };
@@ -135,9 +143,37 @@ function tryClaimSlot(id: number): boolean {
return true;
}
/** Who holds each slot right now: `id:pid` per held slot, empty when free. */
function slotOwnersSnapshot(limit: number): string {
const owners: string[] = [];
for (let id = 0; id < limit; id += 1) {
try {
owners.push(`${id}:${readFileSync(join(fixtureSlotDirectory(id), "pid"), "utf8")}`);
} catch {
owners.push(`${id}:`);
}
}
return owners.join("|");
}
/**
* The waiter's deadline after one poll: pushed out by a full stall window when
* the slot owners changed since the last poll, unchanged otherwise.
*/
export function nextFixtureSlotDeadline(
previousOwners: string,
currentOwners: string,
deadline: number,
now: number,
stallMs: number = FIXTURE_SLOT_STALL_MS,
): number {
return currentOwners === previousOwners ? deadline : now + stallMs;
}
function acquireFixtureSlot(): FixtureSlot {
const limit = maxConcurrentPostgresFixtures();
const deadline = Date.now() + FIXTURE_SLOT_WAIT_MS;
let owners = slotOwnersSnapshot(limit);
let deadline = Date.now() + FIXTURE_SLOT_STALL_MS;
while (Date.now() < deadline) {
for (let id = 0; id < limit; id += 1) {
if (tryClaimSlot(id)) {
@@ -150,9 +186,12 @@ function acquireFixtureSlot(): FixtureSlot {
}
}
Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 50);
const current = slotOwnersSnapshot(limit);
deadline = nextFixtureSlotDeadline(owners, current, deadline, Date.now());
owners = current;
}
throw new Error(
`timed out waiting for a PostgreSQL fixture slot (${limit} concurrent compose networks). Docker address pools cannot host one network per parallel test file.`,
`timed out waiting for a PostgreSQL fixture slot (${limit} concurrent compose networks): no slot changed hands for ${FIXTURE_SLOT_STALL_MS / 60000} minutes, so a holder is likely hung. Docker address pools cannot host one network per parallel test file.`,
);
}
@@ -3,6 +3,8 @@ import { readFileSync } from "node:fs";
import test from "node:test";
import { fileURLToPath } from "node:url";
import { FIXTURE_SLOT_STALL_MS, nextFixtureSlotDeadline } from "./helpers/postgres-fixture.ts";
const fixtureSource = readFileSync(
fileURLToPath(new URL("./helpers/postgres-fixture.ts", import.meta.url)),
"utf8",
@@ -42,3 +44,25 @@ test("large PostgreSQL scripts use stdin without changing shared -c helpers", ()
assert.match(fixtureSource, /psql\(sql\)[\s\S]*"-Atc",\s*sql/);
assert.match(fixtureSource, /psqlAs\(role, password, sql\)[\s\S]*"-Atc",\s*sql/);
});
test("a waiter keeps waiting while the slot queue moves and gives up only when it stalls (BUG-1254)", () => {
// Fixed 5-minute waits failed the gate at random once 46 fixture files
// queued for 2 slots: the queue itself outlasted the wait.
assert.doesNotMatch(fixtureSource, /FIXTURE_SLOT_WAIT_MS/);
assert.equal(FIXTURE_SLOT_STALL_MS, 10 * 60 * 1000);
const start = 1_000_000;
const deadline = start + FIXTURE_SLOT_STALL_MS;
// Nobody released or claimed a slot: the deadline does not move.
assert.equal(nextFixtureSlotDeadline("0:11|1:12", "0:11|1:12", deadline, start + 60_000), deadline);
// A slot changed hands: a full stall window from now.
const later = start + 9 * 60_000;
assert.equal(nextFixtureSlotDeadline("0:11|1:12", "0:13|1:12", deadline, later), later + FIXTURE_SLOT_STALL_MS);
// A slot was released and is momentarily free: that is progress too.
assert.equal(nextFixtureSlotDeadline("0:11|1:12", "0:|1:12", deadline, later), later + FIXTURE_SLOT_STALL_MS);
// The acquire loop recomputes the deadline from the owners after every poll.
assert.match(
fixtureSource,
/function acquireFixtureSlot\(\)[\s\S]*slotOwnersSnapshot\(limit\)[\s\S]*deadline = nextFixtureSlotDeadline\(owners, current, deadline, Date\.now\(\)\)/,
);
assert.match(fixtureSource, /no slot changed hands for/);
});