grenade/moments
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(deploy): split ingress to oolon, expose api on nikola interface

Browse Source 
The per-site nginx ingress for rob.tn lives on oolon (the host the
external router forwards 443 traffic to), not on nikola. Adjust the
topology so:

- web (static ui + nginx) → oolon.hanzalova.internal
- api binds 0.0.0.0:42424 on nikola.kosherinata.internal so oolon
  can reverse-proxy across the WG mesh
- new firewalld service moments-api opens 42424 in the default zone
  on nikola
- oolon labels port 42424 http_port_t so httpd_t may name_connect
  outbound to it (httpd_can_network_connect was already set)
- nginx ssl_certificate switched to oolon's host cert; upstream
  rewritten to nikola.kosherinata.internal:42424

Plaintext between oolon and nikola for now — the WG mesh provides
the encryption layer and the data is already public. Documented
the deferral so a future move to per-hop mTLS is obvious.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
rob thijssen
2026-05-03 20:20:07 +03:00
parent 110b523fd0
commit 52b7d0be9b
6 changed files with 52 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
asset/config/api.env.tmpl
View File

@@ -4,6 +4,6 @@
JOURNAL_STREAM=1 JOURNAL_STREAM=1
RUST_LOG=info,sqlx=warn,tower_http=info RUST_LOG=info,sqlx=warn,tower_http=info
BIND_ADDR=127.0.0.1:42424 BIND_ADDR=0.0.0.0:42424
DATABASE_URL=postgres://moments_ro@magrathea.kosherinata.internal:5432/moments?sslmode=verify-full&sslrootcert=/etc/pki/ca-trust/source/anchors/root-internal.pem&sslcert=/etc/pki/tls/misc/{{HOSTNAME}}.pem&sslkey=/etc/pki/tls/private/{{HOSTNAME}}.pem DATABASE_URL=postgres://moments_ro@magrathea.kosherinata.internal:5432/moments?sslmode=verify-full&sslrootcert=/etc/pki/ca-trust/source/anchors/root-internal.pem&sslcert=/etc/pki/tls/misc/{{HOSTNAME}}.pem&sslkey=/etc/pki/tls/private/{{HOSTNAME}}.pem

6
asset/firewalld/moments-api.xml Normal file
View File

@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<service>
<short>moments-api</short>
<description>moments read-only HTTP API. Reverse-proxied by nginx on oolon (the per-site rob.tn ingress) across the WG mesh; the data is the public timeline already exposed at rob.tn, so no source-IP restriction is currently applied. Add a &lt;source/&gt; element here if defence-in-depth scoping to oolon's WG IP becomes desirable.</description>
<port protocol="tcp" port="42424"/>
</service>

9
asset/manifest.yml
View File

@@ -5,7 +5,10 @@ environments:
api: api:
hosts: [nikola.kosherinata.internal] hosts: [nikola.kosherinata.internal]
config: config:
bind: 127.0.0.1:42424 # Reachable across the WG mesh from oolon (the per-site nginx
# ingress for rob.tn). Firewalld restricts ingress; see
# asset/firewalld/moments-api.xml.
bind: 0.0.0.0:42424
db_role: moments_ro db_role: moments_ro
db_host: magrathea.kosherinata.internal db_host: magrathea.kosherinata.internal
db_port: 5432 db_port: 5432
@@ -29,8 +32,8 @@ environments:
GITHUB_TOKEN: github.com/grenade/admin-token GITHUB_TOKEN: github.com/grenade/admin-token
# GITEA_TOKEN, BUGZILLA_API_KEY: optional, omit unless required. # GITEA_TOKEN, BUGZILLA_API_KEY: optional, omit unless required.
web: web:
hosts: [nikola.kosherinata.internal] hosts: [oolon.hanzalova.internal]
config: config:
server_name: rob.tn server_name: rob.tn
root: /var/www/moments root: /var/www/moments
api_upstream: http://127.0.0.1:42424 api_upstream: http://nikola.kosherinata.internal:42424

16
asset/nginx/rob.tn.conf
View File

@@ -1,12 +1,14 @@
# /etc/nginx/conf.d/rob.tn.conf — rob.tn site config for moments. # /etc/nginx/conf.d/rob.tn.conf — rob.tn site config for moments.
# #
# Static frontend out of /var/www/moments; /api/* reverse-proxied to the # Lives on oolon (the per-site nginx ingress that terminates rob.tn 443
# moments-api binary on loopback. The UI fetches /api/v1/... so the strip # traffic). Static frontend out of /var/www/moments; /api/* reverse-
# matches what Vite's dev proxy does (drop the /api prefix before sending # proxied across the WG mesh to the moments-api binary on nikola. The
# to axum, whose routes are mounted at /v1/*). # UI fetches /api/v1/... so the strip matches what Vite's dev proxy
# does (drop the /api prefix before sending to axum, whose routes are
# mounted at /v1/*).
upstream moments_api { upstream moments_api {
server 127.0.0.1:42424 max_fails=3 fail_timeout=30s; server nikola.kosherinata.internal:42424 max_fails=3 fail_timeout=30s;
keepalive 8; keepalive 8;
} }
@@ -15,8 +17,8 @@ server {
listen [::]:443 ssl http2; listen [::]:443 ssl http2;
server_name rob.tn; server_name rob.tn;
ssl_certificate /etc/pki/tls/misc/nikola.kosherinata.internal.pem; ssl_certificate /etc/pki/tls/misc/oolon.hanzalova.internal.pem;
ssl_certificate_key /etc/pki/tls/private/nikola.kosherinata.internal.pem; ssl_certificate_key /etc/pki/tls/private/oolon.hanzalova.internal.pem;
# Public forge — visitors are not on the internal mTLS mesh, so no # Public forge — visitors are not on the internal mTLS mesh, so no
# client-cert verification here. The X25519MLKEM768 default falls # client-cert verification here. The X25519MLKEM768 default falls

14
readme.md
View File

@@ -47,15 +47,17 @@ Migrations live in `crates/moments-data/migrations/` and run automatically on wo
Topology: Topology:
| Component | Host | Notes | | Component | Host | Notes |
| --------- | --------------------------------- | --------------------------------------------- | | --------- | --------------------------------- | ------------------------------------------------------------------ |
| api | `nikola.kosherinata.internal` | binds `127.0.0.1:42424`, fronted by local nginx | | api | `nikola.kosherinata.internal` | binds `0.0.0.0:42424`; firewalld service `moments-api` |
| worker | `frootmig.kosherinata.internal` | no listening port; pollers only | | worker | `frootmig.kosherinata.internal` | no listening port; pollers only |
| web | `nikola.kosherinata.internal` | static `ui/dist/` under `/var/www/moments` | | web | `oolon.hanzalova.internal` | per-site nginx ingress for rob.tn; `/api/*` → nikola across the WG |
| db | `magrathea.kosherinata.internal` | postgres mTLS, passwordless | | db | `magrathea.kosherinata.internal` | postgres mTLS, passwordless |
Postgres roles `moments_rw` and `moments_ro` must exist on the primary, with `pg_ident.conf` mappings in place for `nikola.kosherinata.internal` → `moments_ro` and `frootmig.kosherinata.internal` → `moments_rw`. See `asset/sql/bootstrap-moments.sql` and `asset/postgres/ident.conf.tmpl`. Postgres roles `moments_rw` and `moments_ro` must exist on the primary, with `pg_ident.conf` mappings in place for `nikola.kosherinata.internal` → `moments_ro` and `frootmig.kosherinata.internal` → `moments_rw`. See `asset/sql/bootstrap-moments.sql` and `asset/postgres/ident.conf.tmpl`.
Inter-host traffic over the WG mesh: oolon's nginx connects to `http://nikola.kosherinata.internal:42424` in plaintext. The mesh provides the encryption layer; per-hop TLS for an internal HTTP read-only API on already-public data is deferred. If that changes, swap the api binary to rustls + the host cert pair, and update the nginx upstream to `https://`.
Secrets resolved by `deploy.sh` via `pass`: Secrets resolved by `deploy.sh` via `pass`:
- `github.com/grenade/admin-token` — GitHub PAT for events + search APIs (worker only). - `github.com/grenade/admin-token` — GitHub PAT for events + search APIs (worker only).

27
script/deploy.sh
View File

@@ -121,7 +121,7 @@ deploy_api() {
stage="$(mktemp -d)" stage="$(mktemp -d)"
trap "rm -rf '$stage'" RETURN trap "rm -rf '$stage'" RETURN
install -d "$stage/etc/moments" "$stage/etc/systemd/system" "$stage/etc/sysusers.d" "$stage/usr/local/bin" install -d "$stage/etc/moments" "$stage/etc/systemd/system" "$stage/etc/sysusers.d" "$stage/etc/firewalld/services" "$stage/usr/local/bin"
# Render env file with hostname substitution. # Render env file with hostname substitution.
sed "s|{{HOSTNAME}}|${fqdn}|g" "${repo_root}/asset/config/api.env.tmpl" \ sed "s|{{HOSTNAME}}|${fqdn}|g" "${repo_root}/asset/config/api.env.tmpl" \
@@ -133,6 +133,7 @@ deploy_api() {
install -m 0644 "${repo_root}/asset/systemd/moments-api.service" "$stage/etc/systemd/system/" install -m 0644 "${repo_root}/asset/systemd/moments-api.service" "$stage/etc/systemd/system/"
install -m 0644 "${repo_root}/asset/systemd/moments-api-cert-reload.service" "$stage/etc/systemd/system/" install -m 0644 "${repo_root}/asset/systemd/moments-api-cert-reload.service" "$stage/etc/systemd/system/"
install -m 0644 "${repo_root}/asset/systemd/moments.sysusers.conf" "$stage/etc/sysusers.d/moments.conf" install -m 0644 "${repo_root}/asset/systemd/moments.sysusers.conf" "$stage/etc/sysusers.d/moments.conf"
install -m 0644 "${repo_root}/asset/firewalld/moments-api.xml" "$stage/etc/firewalld/services/moments-api.xml"
install -m 0755 "${repo_root}/target/release/moments-api" "$stage/usr/local/bin/moments-api" install -m 0755 "${repo_root}/target/release/moments-api" "$stage/usr/local/bin/moments-api"
# Permissions on the rendered env: root-owned, moments group readable. # Permissions on the rendered env: root-owned, moments group readable.
@@ -159,13 +160,21 @@ chmod 0640 /etc/moments/api.env
# the postgres mTLS connection. # the postgres mTLS connection.
setfacl -m u:moments:r "/etc/pki/tls/private/${fqdn}.pem" || true setfacl -m u:moments:r "/etc/pki/tls/private/${fqdn}.pem" || true
# Label loopback API port. Idempotent — the -m flag turns "already labelled" # Label the API port. Idempotent — the -m fallback turns "already labelled"
# into a no-op. # into a no-op.
if ! semanage port -l | awk '{print $1, $3}' | grep -qE "^http_port_t .*42424"; then if ! semanage port -l | awk '{print $1, $3}' | grep -qE "^http_port_t .*42424"; then
semanage port -a -t http_port_t -p tcp 42424 || \ semanage port -a -t http_port_t -p tcp 42424 || \
semanage port -m -t http_port_t -p tcp 42424 semanage port -m -t http_port_t -p tcp 42424
fi fi
# Firewalld: install the named service and enable it in the default zone.
firewall-cmd --reload
zone="$(firewall-cmd --get-default-zone)"
if ! firewall-cmd --zone="$zone" --query-service=moments-api >/dev/null 2>&1; then
firewall-cmd --permanent --zone="$zone" --add-service=moments-api
firewall-cmd --zone="$zone" --add-service=moments-api
fi
restorecon -Rv /usr/local/bin/moments-api /etc/moments /var/lib/moments restorecon -Rv /usr/local/bin/moments-api /etc/moments /var/lib/moments
systemctl daemon-reload systemctl daemon-reload
@@ -173,9 +182,10 @@ systemctl enable --now moments-api-cert.path
systemctl enable --now moments-api.service systemctl enable --now moments-api.service
systemctl restart moments-api.service systemctl restart moments-api.service
# Health probe. # Health probe — hit the bound interface, not loopback, so we exercise the
# same path nginx will use from oolon.
for i in 1 2 3 4 5 6 7 8 9 10; do for i in 1 2 3 4 5 6 7 8 9 10; do
if curl -fsS http://127.0.0.1:42424/v1/healthz >/dev/null; then if curl -fsS "http://${fqdn}:42424/v1/healthz" >/dev/null; then
echo "moments-api healthy" echo "moments-api healthy"
exit 0 exit 0
fi fi
@@ -291,9 +301,16 @@ deploy_web() {
ssh_run "$host" "sudo bash -s" <<'REMOTE_EOF' ssh_run "$host" "sudo bash -s" <<'REMOTE_EOF'
set -euo pipefail set -euo pipefail
# Allow nginx to talk upstream to the loopback API socket. # Allow nginx to make outbound connections to the moments-api upstream
# across the WG mesh.
setsebool -P httpd_can_network_connect on setsebool -P httpd_can_network_connect on
# Label the upstream port so httpd_t may name_connect to it.
if ! semanage port -l | awk '{print $1, $3}' | grep -qE "^http_port_t .*42424"; then
semanage port -a -t http_port_t -p tcp 42424 || \
semanage port -m -t http_port_t -p tcp 42424
fi
restorecon -Rv /var/www/moments /etc/nginx/conf.d/rob.tn.conf restorecon -Rv /var/www/moments /etc/nginx/conf.d/rob.tn.conf
if ! nginx -t; then if ! nginx -t; then