grenade/moments
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(ci): share one nginx-vhost renderer; drop unused deploy.sh #5

Merged
grenade merged 1 commits from fix/render-drift-remove-deploy-sh into main 2026-07-26 12:54:48 +00:00
Conversation 0 Commits 1 Files Changed 8 +105 -602
grenade commented 2026-07-26 12:52:48 +00:00
Owner
Copy Link

Why

cichlid.internal (and ~a dozen other internal vhosts on oolon) have been serving certs that expired days ago, even though fresh certs sit renewed on disk. Root cause is a CI drift bug in this repo, not in the consuming services.

deploy.yml and refresh.yml each rendered asset/nginx/site.conf.tmpl with their own inline Python substitution. Commit bb2f5b1 templated the listen line as {{WEB_LISTEN}} (moving the vhost behind oolon's stream SNI router) and taught the template + deploy.yml + deploy.sh about it — but not refresh.yml. So the nightly refresh (cron 17 4 * * *) rsynced a literal listen {{WEB_LISTEN}}; into /etc/nginx/conf.d/rob.tn.conf.

Because the rendered file is written into the live conf.d before nginx -t runs, that one unrendered placeholder failed the config test for the whole edge and left the wreckage in place — freezing every vhost reload on oolon, including the step@*.timer cert renewals. Certs lapsed on the wire while their renewals piled up unused on disk.

A live fix to rob.tn.conf would be undone by the next 04:17 UTC refresh, so the fix has to be in the pipeline.

What

  • One shared renderer — script/render-site-conf.py, called by both workflows, so they can no longer drift on what they substitute.
  • Guard rails — the renderer fails the build if any {{PLACEHOLDER}} lacks an env value or survives substitution. A forgotten/misnamed variable is now a red run on the runner instead of a broken vhost on the edge (this exact incident would have failed loudly before shipping).
  • Close the immediate drift — add the missing WEB_LISTEN to refresh.yml's env.
  • Template — rename {{DOCROOT}} → {{WEB_ROOT}} so every placeholder maps to the env var of the same name.
  • Remove script/deploy.sh — the third, unused renderer of the same template (superseded by the Actions workflows) and a standing source of drift.
  • Docs — readme.md / CLAUDE.md updated to the Actions-only deploy path.

Validation

  • render-site-conf.py renders the correct vhost from the full env (listen 127.0.0.1:14443 ssl proxy_protocol, no surviving {{ }}).
  • With WEB_LISTEN unset (the old refresh.yml state) it exits non-zero and writes no file.
  • Both workflows still parse as YAML; no {{DOCROOT}} or deploy.sh references remain.

Merging this heals oolon

A merge to main triggers deploy.yml, which re-renders the correct rob.tn.conf; nginx -t then passes (rob.tn.conf is the only broken file) and the reload lands, so cichlid.internal and the other frozen vhosts immediately pick up their current on-disk certs. The next nightly refresh stays correct too.

Known follow-up (out of scope — needs a sudoers change + infra-setup.sh re-run on oolon)

The rendered vhost is still rsynced straight into the live conf.d and only then nginx -t'd, so a valid-but-wrong config (e.g. a directive nginx -t can't catch across http{}/stream{}) could still wedge nginx. A stage → validate → swap with rollback would close that, but the gitea_ci scoped sudoers only permits rsync … rob.tn.conf (no cp/mv/install for the vhost), so it needs a provisioning change and is left for a separate PR.

🤖 Generated with Claude Code

## Why `cichlid.internal` (and ~a dozen other internal vhosts on oolon) have been serving certs that expired days ago, even though fresh certs sit renewed on disk. Root cause is a **CI drift bug in this repo**, not in the consuming services. `deploy.yml` and `refresh.yml` each rendered `asset/nginx/site.conf.tmpl` with their **own inline Python substitution**. Commit `bb2f5b1` templated the listen line as `{{WEB_LISTEN}}` (moving the vhost behind oolon's stream SNI router) and taught the template + `deploy.yml` + `deploy.sh` about it — **but not `refresh.yml`**. So the nightly refresh (cron `17 4 * * *`) rsynced a literal `listen {{WEB_LISTEN}};` into `/etc/nginx/conf.d/rob.tn.conf`. Because the rendered file is written into the **live** `conf.d` *before* `nginx -t` runs, that one unrendered placeholder failed the config test for the whole edge and left the wreckage in place — freezing **every** vhost reload on oolon, including the `step@*.timer` cert renewals. Certs lapsed on the wire while their renewals piled up unused on disk. A live fix to `rob.tn.conf` would be undone by the next 04:17 UTC refresh, so the fix has to be in the pipeline. ## What - **One shared renderer** — `script/render-site-conf.py`, called by both workflows, so they can no longer drift on what they substitute. - **Guard rails** — the renderer fails the build if any `{{PLACEHOLDER}}` lacks an env value or survives substitution. A forgotten/misnamed variable is now a red run on the runner instead of a broken vhost on the edge (this exact incident would have failed loudly before shipping). - **Close the immediate drift** — add the missing `WEB_LISTEN` to `refresh.yml`'s `env`. - **Template** — rename `{{DOCROOT}}` → `{{WEB_ROOT}}` so every placeholder maps to the env var of the same name. - **Remove `script/deploy.sh`** — the third, unused renderer of the same template (superseded by the Actions workflows) and a standing source of drift. - **Docs** — `readme.md` / `CLAUDE.md` updated to the Actions-only deploy path. ## Validation - `render-site-conf.py` renders the correct vhost from the full env (`listen 127.0.0.1:14443 ssl proxy_protocol`, no surviving `{{ }}`). - With `WEB_LISTEN` unset (the old `refresh.yml` state) it exits non-zero and writes **no** file. - Both workflows still parse as YAML; no `{{DOCROOT}}` or `deploy.sh` references remain. ## Merging this heals oolon A merge to `main` triggers `deploy.yml`, which re-renders the correct `rob.tn.conf`; `nginx -t` then passes (rob.tn.conf is the only broken file) and the reload lands, so cichlid.internal and the other frozen vhosts immediately pick up their current on-disk certs. The next nightly refresh stays correct too. ## Known follow-up (out of scope — needs a sudoers change + `infra-setup.sh` re-run on oolon) The rendered vhost is still rsynced straight into the live `conf.d` and only then `nginx -t`'d, so a *valid-but-wrong* config (e.g. a directive `nginx -t` can't catch across `http{}`/`stream{}`) could still wedge nginx. A stage → validate → swap with rollback would close that, but the `gitea_ci` scoped sudoers only permits `rsync … rob.tn.conf` (no `cp`/`mv`/`install` for the vhost), so it needs a provisioning change and is left for a separate PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
grenade added 1 commit 2026-07-26 12:52:49 +00:00
fix(ci): share one nginx-vhost renderer; drop unused deploy.sh 3260bfb35b 
The nightly refresh.yml and deploy.yml each substituted asset/nginx/
site.conf.tmpl with their own inline python. When bb2f5b1 templated the
listen line as {{WEB_LISTEN}} (moving the vhost behind oolon's stream SNI
router), it added the substitution to the template and deploy.yml but not
to refresh.yml. The daily refresh then rsynced a literal
`listen {{WEB_LISTEN}};` into /etc/nginx/conf.d/rob.tn.conf, `nginx -t`
failed for the whole edge, and — because the file is written into the live
conf.d before it is tested — every vhost's reload (including the step@
cert renewals) stayed frozen. Internal vhosts, cichlid.internal among
them, served certs that had expired days earlier while the renewed certs
sat unused on disk.

- Replace both inline renderers with script/render-site-conf.py, shared by
  deploy.yml and refresh.yml so they cannot drift on what they substitute.
- Guard rails: the renderer fails if any {{PLACEHOLDER}} lacks an env value
  or survives substitution, so a forgotten/misnamed variable is a red build
  on the runner instead of a broken vhost on the edge.
- Add the missing WEB_LISTEN to refresh.yml's env (the immediate drift).
- Rename the template's {{DOCROOT}} to {{WEB_ROOT}} so every placeholder
  maps to the env var of the same name.
- Remove script/deploy.sh: the third, unused renderer of the same template
  (superseded by the Actions workflows) and a standing source of drift.
- Docs (readme, CLAUDE.md) updated to the Actions-only deploy path.

Known follow-up (needs a sudoers change + infra-setup re-run on oolon, so
out of scope here): the rendered vhost is still rsynced straight into the
live conf.d and only then `nginx -t`'d, so a valid-but-wrong config could
still wedge nginx. Stage-validate-swap with rollback would close that.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QsH1rcWQYtRVhvaftiKm22
grenade merged commit 9905cef1d2 into main 2026-07-26 12:54:48 +00:00
grenade deleted branch fix/render-drift-remove-deploy-sh 2026-07-26 12:54:48 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
grenade
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: grenade/moments#5
Delete Branch "fix/render-drift-remove-deploy-sh"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?