Jyotisha/debug-staging-image-build-failure.md

Debug Session: staging-image-build-failure

• Status: OPEN
• Started: 2026-07-28
• Symptom: Gitea staging workflow fails resolving the API base image with insufficient_scope.
• Expected: staging push builds and publishes API/Web images, then deploys them to jyotisha-staging successfully.

Hypotheses

1. The Aliyun library/python path is not a public mirrored repository.
2. The authenticated private ACR context is incorrectly reused for the public image endpoint.
3. The Node base image uses the same invalid mirror pattern.
4. A stable reachable base-image source or preloaded image exists on xiaoxin.
5. Further SSH, Compose, or environment failures remain after the image issue.

Evidence

• Pre-fix Gitea log: registry.cn-hangzhou.aliyuncs.com/library/python:3.12-slim returned pull access denied and insufficient_scope.

Iterations

• Confirmed both Aliyun library/python and library/node paths reject pulls.
• Confirmed direct Docker Hub access times out from xiaoxin.
• Confirmed complete pulls of m.daocloud.io/docker.io/library/python:3.12-slim and swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/library/node:22-alpine.
• Minimal fix: replace only the two Dockerfile base-image sources, preserving Python 3.12 slim and Node 22 alpine variants.
• Process-tree evidence corrected the initial attribution: the run was blocked in apt-get update; the later pip command had not started.
• Runtime APT benchmark: Aliyun ~0.11s/~1.29 MB/s; Tsinghua ~0.30s/~461 KB/s; Debian official ~1.22s/~115 KB/s.
• Runtime PyPI benchmark: Aliyun ~0.21s/~350 KB/s; Tsinghua ~0.81s/~90 KB/s; official ~1.20s/~88 KB/s.
• Added Aliyun Debian and PyPI sources, APT retries and 30-second network timeouts, plus a 60-second pip timeout.
• npm official and npmmirror throughput were comparable, so no npm source change was made.