Jyotisha/deploy/README.md

Production deployment and maintenance

This file is the operational source of truth for Jyotisha deployment. The production migration is governed by docs/operations/production-server-migration-2026-08.md; do not change DNS or retire the old environment outside that runbook.

Production migration state

Item	Value
Public domain	https://jyotisha.chat
DNS	Spaceship nameservers (launch1.spaceship.net, launch2.spaceship.net)
Current public host	Old VPS; keep as a rollback asset until reconciliation completes
Target host	118.194.235.34, Ubuntu x86_64
Target SSH	dedicated deploy user, confirmed variable port, public-key authentication only
Target capacity	2 vCPU / 4 GB RAM; no application builds on host
Target app directory	/opt/jyotisha-production
Target environment files	.env.production and .env.production.database (0600)
Primary source repository	https://git.copse.top/root/Jyotisha.git
GitHub upstream/mirror	https://github.com/jesse-ux/Jyotisha.git
Migration source	Supabase project vtvnfqmonbfuxmqkqdlc + Supabase Auth
Migration target	private PostgreSQL 17 + Better Auth

This migration changes both infrastructure and persistence. It is a controlled Supabase-to-private-PostgreSQL ETL, not a database-volume copy or full-dump restore. Keep the old VPS for 7–14 days and Supabase for 14–30 days after cutover.

Architecture

Spaceship DNS -> Caddy :80/:443 -> web:3000 -> api:5200
                                  |           -> local astrology engines
                                  -> private PostgreSQL 17 + Better Auth
                                  -> external model and mail providers

Only Caddy publishes host ports. Ports 3000 and 5200 must remain private.

DNS and Supabase Auth

Final Spaceship resource records (apply only during the approved cutover window):

A      @      118.194.235.34
A      admin  118.194.235.34
CNAME  www    jyotisha.chat

After cutover, verify both authoritative nameservers:

dig +short @launch1.spaceship.net A jyotisha.chat
dig +short @launch2.spaceship.net A admin.jyotisha.chat

Both user and admin hosts must return 118.194.235.34. Caddy provisions and renews HTTPS automatically after DNS resolves.

Production environment

.env.production contains runtime-only application settings. Database bootstrap, migration, and backup credentials belong only in .env.production.database.

SITE_ADDRESS=https://jyotisha.chat
APP_ENV_FILE=../.env.production
CADDYFILE_PATH=./Caddyfile.production.selfhosted
JYOTISH_API_BASE=http://api:5200
GEOAPIFY_API_KEY=<server-side Geoapify geocoding key>

AUTH_PROVIDER=self-hosted
SELF_HOSTED_IDENTITY_ENABLED=true
AUTH_USER_ORIGIN=https://jyotisha.chat
ADMIN_USER_ORIGIN=https://admin.jyotisha.chat
IDENTITY_DATABASE_URL=postgresql://identity_runtime:<encoded-secret>@postgres:5432/jyotisha
APP_DATABASE_URL=postgresql://app_runtime:<encoded-secret>@postgres:5432/jyotisha
SERVICE_DATABASE_URL=postgresql://service_runtime:<encoded-secret>@postgres:5432/jyotisha
ADMIN_DATABASE_URL=postgresql://admin_runtime:<encoded-secret>@postgres:5432/jyotisha
BETTER_AUTH_USER_SECRET=<production-only-secret>
RESEND_API_KEY=<production-only-secret>
RESEND_FROM_EMAIL=<verified-sender>
ADMIN_EMAILS=...

# Required to save/read database-backed 易支付 settings. Base64 decoding must
# produce exactly 32 random bytes. Generate independently; never reuse auth keys.
EPAY_CONFIG_ENCRYPTION_KEY=<independent-base64-encoded-32-byte-key>
# Legacy EPAY_GATEWAY_URL / EPAY_PID / EPAY_KEY / EPAY_NOTIFY_URL /
# EPAY_RETURN_URL / EPAY_SITE_NAME remain fallback-only when no database row exists.
# Online packages stay hidden by default; only explicit true enables the fallback.
EPAY_CHAT_ENABLED=false

# Fixed birth-time rectification fee shown by the account UI.
RECTIFICATION_PRICE_CREDITS=3

# Required to encrypt/decrypt model-provider API keys stored in the admin database.
# Base64 decoding must produce exactly 32 random bytes; do not reuse other keys.
MODEL_PROVIDER_CONFIG_ENCRYPTION_KEY=<independent-base64-encoded-32-byte-key>

# The admin database model catalog is the only runtime model/provider source.
# Provider API keys are entered in the admin UI and are never read from provider env vars.

# Required VedAstro server-side upstream for chart creation and rectification:
VEDASTRO_GATEWAY_MODE=official_first
VEDASTRO_API_ENDPOINT=https://api.vedastro.org/api
VEDASTRO_ENABLE_NETWORK=1
# Compatibility flag only; rectification range scans follow VEDASTRO_ENABLE_NETWORK.
VEDASTRO_RANGE_SCAN_NETWORK_ENABLED=1
VEDASTRO_TIMEOUT_SECONDS=20
VEDASTRO_API_KEY=<server-secret>

Never commit either production env file, model/payment keys, user tokens, SSH private keys, database URLs, or passwords. Do not retain Supabase runtime selectors in the target env; Supabase is migration source/rollback storage only.

After changing runtime variables, use the exact-SHA Gitea workflow to recreate services; do not rebuild or rsync a local tree on the host.

Connect and inspect

ssh -p <confirmed-port> deploy@118.194.235.34
cd /opt/jyotisha-production
COMPOSE='docker compose -p jyotisha-production --env-file .env.production -f deploy/docker-compose.server.yml -f deploy/docker-compose.postgres.yml -f deploy/docker-compose.production.yml'
$COMPOSE ps
$COMPOSE logs --tail=100 postgres api web caddy
free -h
docker stats --no-stream

UFW permits only the confirmed SSH port, HTTP, and HTTPS. PostgreSQL, Web, API, and the Docker API remain private.

Manual production deployment with Gitea Actions

GitHub production deployment is retired. Production changes are released only by manually dispatching .gitea/workflows/deploy-production.yml. The workflow requires an exact SHA shared by main and staging, an exact-SHA staging push gate and image manifest, the manual release gate, and matching public staging health. It deploys immutable registry digests and never builds application images on the production host.

The internal verification mode is for pre-DNS checks on the new host. The public mode is for the same SHA after authoritative DNS and Caddy TLS converge. Neither mode imports data, applies pending migrations, or changes DNS.

Required Gitea Actions secret:

PRODUCTION_SSH_PRIVATE_KEY = one-line base64 of the dedicated deploy private-key file

The independently verified host key is stored in PRODUCTION_KNOWN_HOSTS. See the production migration runbook for all variables, data gates, and rollback boundaries.

Staging deployment

Staging is isolated from production:

Item	Value
URL	https://staging.jyotisha.chat
Admin URL	https://admin.staging.jyotisha.chat
Host	118.26.111.127
Path	/opt/jyotisha-staging
Runtime app env	/opt/jyotisha-staging/.env.staging (0600)
Runtime database env	/opt/jyotisha-staging/.env.staging.database (0600)
PostgreSQL	private Compose network; no published host port
Business database	local private PostgreSQL (jyotisha-staging Compose project)
Identity	Better Auth + Resend OTP on the same private PostgreSQL cluster
Actions control plane	Gitea 1.26.2 (git.copse.top)

Gitea is the primary source repository and Actions control plane. Gitea automatically injects the per-job ${{ secrets.GITEA_TOKEN }} token; its access is limited by each workflow's permissions block and it must not be configured as a repository secret. Configure repository Actions secrets REGISTRY_USERNAME, REGISTRY_PASSWORD, and STAGING_SSH_PRIVATE_KEY, plus variables STAGING_HOST, STAGING_PORT, STAGING_USER, STAGING_PATH, STAGING_URL, and STAGING_KNOWN_HOSTS. STAGING_SSH_PRIVATE_KEY must be the private-key file encoded as one unwrapped base64 line (for example, base64 < key | tr -d '\n'), not a multiline PEM/OpenSSH value; staging workflows decode it only into a mode-0600 temporary file and validate it with ssh-keygen. The workflow_run controller is loaded from the default main branch while separately requiring the successfully tested upstream branch to be staging. The controller checks out only main with full history, requires the requested staging SHA to be an ancestor of that reviewed history, and uploads only the allowlisted deploy/ control files. It never executes deployment validators or remote orchestration scripts from the target/rollback revision. The staging key, database, Resend key, and model-provider keys must not be shared with production. Staging image publishing has no Supabase build variables. GitHub workflows are upstream/mirror fallback only, not the normal staging release path.

Staging Backend Quality Gate runs for relevant pull_request paths, pushes to staging, and workflow_dispatch. It validates the Python/database/frontend contract; only a successful push to staging publishes the API/web images and a run-bound manifest containing their sha256 digests. .gitea/workflows/deploy-staging.yml consumes that exact successful run, validates its manifest against the full 40-character commit, and deploys digest references rather than trusting the discoverability tags.

The staging env file must include these non-secret selectors so Compose cannot fall back to production paths:

APP_ENV_FILE=../.env.staging
CADDYFILE_PATH=./Caddyfile.staging
SITE_ADDRESS=https://staging.jyotisha.chat

模型供应商的 base_url 不再依赖域名白名单，任意公网 HTTPS origin 均可由管理员配置；部署环境不需要 MODEL_PROVIDER_BASE_URL_ALLOWLIST。服务端仍强制 HTTPS、禁止凭据、localhost/内网/保留地址，并在 DNS 解析、请求地址 pinning 和重定向处理上执行 SSRF 防护。

Staging is fully self-hosted: set AUTH_PROVIDER=self-hosted and SELF_HOSTED_IDENTITY_ENABLED=true. Add the four role-specific server-only database URLs, the exact AUTH_USER_ORIGIN=https://staging.jyotisha.chat and ADMIN_USER_ORIGIN=https://admin.staging.jyotisha.chat, the single BETTER_AUTH_USER_SECRET, and staging-only Resend settings listed in deploy/.env.staging.identity.example. Both hosts run the same application and Better Auth service, but cookies remain host-only; the admin host / redirects to /admin, and unauthenticated admin requests continue to /login on that host. Better Auth trusts only those two origins, while unknown identity hosts fail closed. Persisted identity.users.role=admin is the only self-hosted backend role, while viewer and ordinary users are denied. Browser code uses same-origin APIs; it receives neither database credentials nor Supabase keys. Production uses the same architecture only after the reviewed migration and cutover. See docs/operations/self-hosted-identity.md for identity validation.

After source sync and before up, the workflow validates .env.staging mode/selectors, explicitly pins the three staging selectors against ambient shell overrides, and runs docker compose --env-file .env.staging -f deploy/docker-compose.server.yml config --quiet. For later manual inspections, run the same checks only after the tracked deployment files exist on the server. Do not use a manual gate run from main as the first publishing path: publishing requires a successful push to staging, while manual Deploy staging requires a successful gate run for the exact SHA.

First-deploy sequence

1. Complete the server and Gitea bootstrap: create both mode-0600 env files, preload the reviewed postgres:17-alpine image, and configure the listed Actions variables/secrets. No repository-level Supabase variables are required. Deployment and migration workflows use --pull never for PostgreSQL, so database image upgrades remain an explicit operator-controlled maintenance action rather than an application-deploy side effect.
2. Open a PR and merge the reviewed change to main, then fast-forward/push that same exact SHA to staging; do not create a staging-only target or rely on a main workflow dispatch to publish images.
3. The Staging Backend Quality Gate runs for that push and, when successful, publishes API/web images plus an artifact binding the exact SHA to both immutable image digests.
4. The automatic Deploy staging workflow downloads that gate-run artifact, syncs only the trusted default-main controller's allowlisted deploy/ files under the shared staging host lock, and validates both .env.staging and .env.staging.database before any app change. The target application's code is carried only by the digest-pinned images.
5. If environment validation fails, fix the server-side env files without committing or copying secrets, then manually rerun Deploy staging from main with the same successful SHA in deploy_sha; the workflow rechecks a successful staging gate for that exact SHA.
6. If the read-only checker reports a pending migration, stop app deployment and run Migrate Staging Database manually with the same full SHA. Migration success does not dispatch deployment.
7. After migration succeeds, the operator must manually start Deploy staging from main with that same exact SHA, then confirm https://staging.jyotisha.chat/api/health reports it and private API health.

Application rollback uses the same workflow: manually dispatch Deploy staging from the main controller with a previous known-good full SHA that has a successful Staging Backend Quality Gate run, and explicitly set allow_rollback=true. Normal and migration-triggered deployments reject stale, divergent, or backward revisions. Rollback still consumes the selected gate run's digest manifest and is supported only during that artifact's 30-day retention window; after expiry, stop and prepare a separately reviewed republish/recovery change rather than substituting a mutable tag or assuming the old run can still be rerun. Database migrations are separate and are not rolled back by an application deployment. Restore a staging database backup before running any destructive migration rehearsal.

Inspect staging without printing secrets:

ssh -i ~/.ssh/jyotisha-staging deploy@118.26.111.127
cd /opt/jyotisha-staging
docker compose --env-file .env.staging -f deploy/docker-compose.server.yml ps
docker compose --env-file .env.staging -f deploy/docker-compose.server.yml logs --tail=100 api web caddy
curl -fsS https://staging.jyotisha.chat/api/health

The normal application deployment workflow never runs database migrations. Apply migrations to the private staging PostgreSQL cluster first, verify them, and only then deploy application code that depends on them.

Staging PostgreSQL operations

This section is the server-side runbook for the disposable staging PostgreSQL volume. It does not replace the production instructions above.

Bootstrap and environment-file boundary

SSH to the staging host as the deployment user and create both environment files with a restrictive umask. The application file and the database file are separate, both are mode 0600, the database file is owned by the deployment user, and neither is committed or copied through rsync:

cd /opt/jyotisha-staging
umask 077
touch .env.staging
chmod 600 .env.staging
touch .env.staging.database
chmod 600 .env.staging.database

.env.staging contains application selectors and server-only application credentials. SCHEMA_DATABASE_URL must not appear in .env.staging; neither may any database bootstrap password, STAGING_BACKUP_ENCRYPTION_KEY, or migration-runner credential. In particular, there is no SCHEMA_DATABASE_URL or SERVICE_RUNTIME_PASSWORD in .env.staging; the schema URL and raw role password exist only in .env.staging.database, which is read by PostgreSQL and the opt-in migrator. The web/API runtime receives only the percent-encoded SERVICE_DATABASE_URL from .env.staging.

Generate every <generated> value from independently generated 32 random bytes (for example, run openssl rand -base64 32 separately for each value and place it directly into the mode-0600 file or an approved secret store). Do not reuse a password between roles, paste values into chat, commit either file, or print them in workflow logs. The schema-owner password in SCHEMA_DATABASE_URL is the same secret as SCHEMA_OWNER_PASSWORD; use a percent-encoded URL password component only, and do not encode the scheme, host, port, or database name.

The exact database keys are:

POSTGRES_DB=jyotisha
POSTGRES_USER=postgres
POSTGRES_PASSWORD=<generated>
SCHEMA_OWNER_PASSWORD=<generated>
IDENTITY_RUNTIME_PASSWORD=<generated>
APP_RUNTIME_PASSWORD=<generated>
SERVICE_RUNTIME_PASSWORD=<generated>
ADMIN_RUNTIME_PASSWORD=<generated>
MIGRATION_RUNNER_PASSWORD=<generated>
BACKUP_READER_PASSWORD=<generated>
STAGING_BACKUP_ENCRYPTION_KEY=<generated>
SCHEMA_DATABASE_URL=postgresql://schema_owner:<percent-encoded-password>@postgres:5432/jyotisha

Use the same independently generated service-role secret in exactly two host-managed locations: the raw SERVICE_RUNTIME_PASSWORD in .env.staging.database, and its percent-encoded password component in SERVICE_DATABASE_URL=postgresql://service_runtime:<percent-encoded-password>@postgres:5432/jyotisha inside .env.staging. Keep both files owned by the staging deploy user with mode 0600. The deploy controller runs both validators with shell tracing disabled before Compose changes; Gitea stores neither value and the workflow must never echo, interpolate, or pass either password as a workflow environment variable.

PostgreSQL is private: deploy/docker-compose.postgres.yml has no ports mapping, so the staging database is reachable only on the Docker app network. The CI overlay is the only host binding and is loopback-only (127.0.0.1:${POSTGRES_HOST_PORT:-55432}:5432); do not add a public database port, firewall exception, or browser-facing SQL tool. Normal web/API containers never receive SCHEMA_DATABASE_URL.

Exact deployment and migration order

Use this order for every staging revision:

1. Open a PR and merge the reviewed revision to main, then fast-forward/push that same exact SHA to staging.
2. Wait for Staging Backend Quality Gate to pass and publish that exact full SHA's API/web digest manifest.
3. The automatic Deploy staging workflow checks the exact SHA in read-only migration-check mode before changing API, web, or Caddy. If it reports pending or drifted migrations, stop; do not retry the application deployment as if it were a migration.
4. Open Migrate Staging Database -> Run workflow, select Use workflow from: main, and enter the reported full lowercase 40-character SHA in deploy_sha. The controller validates that exact SHA against a successful staging gate and reviewed main history, starts only PostgreSQL, and runs the digest-pinned migrator without executing scripts from the target revision.
5. A successful migration rechecks that staging still points at the same exact SHA and prints the ordered migration ledger, but does not dispatch deployment. The operator must then open Deploy staging -> Run workflow, select Use workflow from: main, and enter the same exact SHA in deploy_sha with allow_rollback=false. If staging advanced, stop rather than substituting a branch name, short SHA, or newer commit.
6. Confirm https://staging.jyotisha.chat/api/health and verify that its deployment SHA is the SHA from step 2.
7. After health verification, create the local encrypted backup described below.

The deploy and migration workflows share the staging-mutation Actions concurrency group, and their live-tree sync plus Compose work runs under /opt/jyotisha-staging/.state/mutation.lock. The synchronized tree explicitly preserves /backups/, .env*, .state, and .incoming. The read-only checker exits before app changes when a migration is pending. Its message includes the exact SHA and the Migrate Staging Database workflow name. A failed migration does not re-dispatch deployment. Application rollback restores the previously recorded digest references and SHA, falling back to validated local image IDs only when transitioning from the pre-foundation local-image deployment; it does not roll back database state.

Local encrypted staging backups (three-copy limit)

After the health check, run the repository backup helper from the synchronized staging checkout:

cd /opt/jyotisha-staging
./deploy/backup-staging-postgres.sh \
  .env.staging.database \
  /opt/jyotisha-staging/backups/staging-db

The helper invokes pg_dump --format=custom --no-owner in the PostgreSQL container and encrypts the stream with openssl enc -aes-256-cbc -salt -pbkdf2 -pass env:STAGING_BACKUP_ENCRYPTION_KEY. It creates mode-0600 .dump.enc files in a mode-0700 directory, refuses disk usage at or above 70%, publishes atomically, and retains only the newest three encrypted local backups. The encryption passphrase is supplied through the environment, never as a command-line argument or printed value. Keep the archive directory on this staging VPS only; there is no off-site staging recovery and no off-site staging backup. These three local encrypted copies are rehearsal/rollback aids, not disaster-recovery backups.

Restore drill into a disposable database

Run a restore drill only against the disposable jyotisha_restore_check database. Choose one archive and use a temporary decrypted custom-format dump; the commands below match the backup helper's AES-256-CBC/PBKDF2 and pg_dump --format=custom interfaces:

set -euo pipefail
cd /opt/jyotisha-staging
export DATABASE_ENV_FILE=../.env.staging.database
BACKUP_DIR=/opt/jyotisha-staging/backups/staging-db
BACKUP_FILE="$(find "$BACKUP_DIR" -maxdepth 1 -type f -name 'jyotisha-staging-*.dump.enc' -print | LC_ALL=C sort | tail -n 1)"
test -n "$BACKUP_FILE"
test -f "$BACKUP_FILE"
test ! -L "$BACKUP_FILE"
test -s "$BACKUP_FILE"
RESTORE_DUMP="$(mktemp /tmp/jyotisha-staging-restore.XXXXXX.dump)"
chmod 600 "$RESTORE_DUMP"
trap 'rm -f -- "$RESTORE_DUMP"' EXIT
read -r -s -p 'Backup passphrase: ' STAGING_BACKUP_ENCRYPTION_KEY
printf '\n' >&2
export STAGING_BACKUP_ENCRYPTION_KEY

openssl enc -d -aes-256-cbc -pbkdf2 \
  -pass env:STAGING_BACKUP_ENCRYPTION_KEY \
  -in "$BACKUP_FILE" -out "$RESTORE_DUMP"

docker compose -p jyotisha-staging -f deploy/docker-compose.postgres.yml \
  exec -T postgres createdb -U postgres jyotisha_restore_check
docker compose -p jyotisha-staging -f deploy/docker-compose.postgres.yml \
  exec -T postgres pg_restore -U postgres --no-owner --exit-on-error \
  --dbname=jyotisha_restore_check < "$RESTORE_DUMP"

# Inspect the restored disposable database, then remove only the drill target.
docker compose -p jyotisha-staging -f deploy/docker-compose.postgres.yml \
  exec -T postgres dropdb -U postgres --if-exists jyotisha_restore_check
rm -f -- "$RESTORE_DUMP"
unset STAGING_BACKUP_ENCRYPTION_KEY

The passphrase is read silently into an environment variable; do not put it in argv, shell history, logs, or support messages. The cleanup scope is deliberately narrow: delete only jyotisha_restore_check and the temporary decrypted dump. Do not run docker compose down, down -v, dropdb jyotisha, volume deletion, or archive deletion as part of this drill. If restore fails, preserve the encrypted archive and PostgreSQL volume for inspection, remove only the temporary dump, and investigate before retrying.

Staging/production boundary

This disposable staging procedure does not authorize a production migration, production backup policy, production database replacement, domain switch, Supabase deletion, or production cutover. Production deployment and migration remain manual-only and require a separate reviewed approval, off-site encrypted backups, and a successful production restore drill. Keep the production .env.production and all production credentials on the production host; never copy them into staging.

Manual deployment fallback

There is no unreviewed rsync/build fallback for the new production. If Gitea Actions or the immutable artifact is unavailable, stop the release and restore the control plane; do not substitute a mutable image tag or copy a local working tree to production.

Verification

curl -fsS https://jyotisha.chat/login >/dev/null
curl -fsS -o /dev/null -w '%{http_code}\n' https://jyotisha.chat/api/account

The second command should return 401 while logged out. Verify the private Python API from inside the web container:

ssh -p <confirmed-port> deploy@118.194.235.34 \
  'cd /opt/jyotisha-production && docker compose -p jyotisha-production --env-file .env.production -f deploy/docker-compose.server.yml -f deploy/docker-compose.postgres.yml -f deploy/docker-compose.production.yml exec -T web node -e "fetch(\"http://api:5200/api/health\").then(async r=>{console.log(r.status); console.log(await r.text())})"'

Expected: HTTP 200, "status": "ok", and "swisseph_available": true. Public access to 118.194.235.34:5200 and :5432 must fail. The deployment workflow also executes SELECT 1 through all four runtime database roles.

Do not treat a green app deployment as proof of database migration or data reconciliation. The production deployment refuses pending target migrations but does not apply them. Follow the schema-first ETL and verification gates in the production migration runbook. Supabase remains the read-only rollback source until the retention and reconciliation window closes.

Agentic birth-time rectification

The maintained web flow uses POST /api/rectification/agent. It has no standalone rectification worker and no V3/V4 rollout selector. Apply all pending forward migrations before deploying the matching web image; never delete or reverse the historical migrations or their billing, receipt, and audit rows.

After an exact-SHA deployment, verify /api/health, then use a synthetic account to confirm the browser calls /api/rectification/agent. Confirm retired unfinished cases do not block the Agentic entry, charging remains idempotent, and a previously accepted or confirmed profile time is not changed without an explicit current acceptance. Do not record birth data, narrative, account identifiers, tokens, or model prompts in smoke evidence.

Common operations

# Restart without rebuilding
docker compose -p jyotisha-production --env-file .env.production \
  -f deploy/docker-compose.server.yml -f deploy/docker-compose.postgres.yml \
  -f deploy/docker-compose.production.yml up -d

# Pull/deploy application images only through the exact-SHA Gitea workflow.

# Follow logs
docker compose -p jyotisha-production --env-file .env.production \
  -f deploy/docker-compose.server.yml -f deploy/docker-compose.postgres.yml \
  -f deploy/docker-compose.production.yml logs -f --tail=100 postgres api web caddy

Optional Railway deployment

Railway is not the current production target. If needed, create web and api services from the same repository using deploy/railway-web.Dockerfile and deploy/railway-api.Dockerfile; keep api private and set the web service's JYOTISH_API_BASE to Railway's private API hostname.