Harden staging SSH secret and env ownership #15

2026-08-06T15:34:46+08:00

jesse commented

2026-08-06 15:34:46 +08:00

Incident

A failed staging-only deploy exposed the multiline staging SSH key in Gitea logs and then stopped because root-run deployment validation conflicted with deploy-owned mode-0600 env files. Public staging never switched from the previous SHA; production was not involved.

Immediate containment completed

rotated the staging ED25519 key in Gitea and GitHub
verified the new key before revoking the old authorized key
proved the old key no longer authenticates and deleted the old local key
deleted 28 potentially affected Gitea deploy/migration runs
retained the successful quality gate and immutable image manifest, which never received the SSH secret

Code changes

store staging SSH secret as one unwrapped base64 line
decode only into a mode-0600 temporary key and validate with ssh-keygen
reject direct multiline secret injection in contract tests
make deploy/migration validate env ownership against the deployment tree owner
preserve owner/gid when rollout atomically replaces .env.staging
record BUG-128 and ERR-094; document the base64 secret contract

Verification

staging workflow contracts: 31/31
personal report focused: 142/142
explicit owner validator regression: pass
YAML and shell syntax: pass
TypeScript and targeted ESLint: pass
mandatory pre-work: pass
strict SSH with rotated key: pass
remote env owner/mode: deploy:deploy 0600
complete Docker fixtures: pending PR quality-gate runner (local Docker CLI unavailable)

No production workflow, key, migration, or deployment was changed or triggered.

## Incident A failed staging-only deploy exposed the multiline staging SSH key in Gitea logs and then stopped because root-run deployment validation conflicted with deploy-owned mode-0600 env files. Public staging never switched from the previous SHA; production was not involved. ## Immediate containment completed - rotated the staging ED25519 key in Gitea and GitHub - verified the new key before revoking the old authorized key - proved the old key no longer authenticates and deleted the old local key - deleted 28 potentially affected Gitea deploy/migration runs - retained the successful quality gate and immutable image manifest, which never received the SSH secret ## Code changes - store staging SSH secret as one unwrapped base64 line - decode only into a mode-0600 temporary key and validate with `ssh-keygen` - reject direct multiline secret injection in contract tests - make deploy/migration validate env ownership against the deployment tree owner - preserve owner/gid when rollout atomically replaces `.env.staging` - record BUG-128 and ERR-094; document the base64 secret contract ## Verification - staging workflow contracts: 31/31 - personal report focused: 142/142 - explicit owner validator regression: pass - YAML and shell syntax: pass - TypeScript and targeted ESLint: pass - mandatory pre-work: pass - strict SSH with rotated key: pass - remote env owner/mode: deploy:deploy 0600 - complete Docker fixtures: pending PR quality-gate runner (local Docker CLI unavailable) No production workflow, key, migration, or deployment was changed or triggered.

jesse added 1 commit 2026-08-06 15:34:47 +08:00

fix(staging): rotate SSH secret handling and env ownership

Staging Backend Quality Gate / validate (pull_request) Successful in 16m32s

Details

Staging Backend Quality Gate / publish (pull_request) Has been skipped

Details

cfc7af6b46

jesse added 1 commit 2026-08-06 15:52:48 +08:00

docs(bugs): record staging credential incident resolved

Staging Backend Quality Gate / validate (pull_request) Successful in 14m56s

Details

Staging Backend Quality Gate / publish (pull_request) Has been skipped

Details

0ddbe11eda

jesse merged commit f7a615a5bf into main

2026-08-06 15:54:06 +08:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: root/Jyotisha#15