rob thijssen
8867ff5df3
deploy.sh:
- never rsync into /; stage to /tmp on the remote and install at final
paths via sudo bash heredoc, closing the parent-dir attribute leak
that broke three hosts in the earlier rsync incident
- shell-quote heredoc args via ${var@Q}
- drop -A -X on the remaining (web) rsyncs
- generic worker.secrets loop reads (env-var → pass path) from manifest;
GITEA_TOKEN now flows through automatically
- in-memory bash substitution for templates (secrets never on argv)
- simplify semanage port labelling: --add 2>/dev/null || --modify (the
old grep pre-check matched only the first listed port)
- restorecon back to short flags (Fedora policycoreutils has no long
forms; --recursive errored at deploy time)
- quieter health probe loop: curl diagnostics only on final failure
manifest as source of truth:
- api.config.bind drives BIND_ADDR, firewalld port, semanage label,
health-probe URL
- web.config.{server_name,root,api_upstream} drives nginx render,
rsync targets, restorecon scope
- nginx config renamed to site.conf.tmpl; firewalld svc to
moments-api.xml.tmpl; both rendered at deploy time
- topology flip: api → nikola, worker → frootmig (anjie freed)
new scripts:
- script/teardown.sh: idempotent component teardown, never rsyncs,
shared-state cleanup gated on absence of remaining env files,
--remove-docroot guard against shallow / system paths
- script/db-perms.sh: rewritten — fixes grep/append role mismatch that
appended duplicates on re-run, adds postgres reload, hits primary +
standby in a single invocation
readme: genericized; deployment topology no longer carries real host
or site names.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
70 lines
3.8 KiB
Markdown
70 lines
3.8 KiB
Markdown
# moments
|
|
|
|
Personal activity timeline. Polls public sources (GitHub, Gitea, Mercurial, Bugzilla), stores raw payloads in Postgres, and serves a reshaped timeline to a static React frontend.
|
|
|
|
Successor to the now-defunct [grenade-events-react](https://github.com/grenade/grenade-events-react), which depended on MongoDB Stitch (retired by MongoDB in September 2022).
|
|
|
|
## Layout
|
|
|
|
```
|
|
crates/
|
|
moments-entities/ # types and DTOs
|
|
moments-core/ # ingestion + reshape logic
|
|
moments-data/ # postgres adapter + migrations
|
|
moments-api/ # axum read-only HTTP API (binary)
|
|
moments-worker/ # ingestion daemon (binary)
|
|
ui/ # vite + react + swc + ts frontend
|
|
asset/ # systemd, nginx, firewalld, manifest.yml
|
|
script/deploy.sh
|
|
```
|
|
|
|
Architectural conventions follow [grenade/architecture/generic.md](https://git.lair.cafe/grenade/architecture/src/branch/main/generic.md).
|
|
|
|
## Local development
|
|
|
|
```sh
|
|
cargo build --workspace
|
|
cargo run -p moments-api # serves on 127.0.0.1:8080
|
|
cargo run -p moments-worker # one-shot ingest tick (until --interval is wired up)
|
|
```
|
|
|
|
The API expects a Postgres reachable at `DATABASE_URL`. In production this is an mTLS connection using the host cert. For local dev against a throwaway database:
|
|
|
|
```sh
|
|
DATABASE_URL=postgres://localhost/moments cargo run -p moments-api
|
|
```
|
|
|
|
Migrations live in `crates/moments-data/migrations/` and run automatically on worker startup. The API connects as `moments_ro` and never runs migrations — the worker (as `moments_rw`) is the schema owner.
|
|
|
|
## Deployment
|
|
|
|
```sh
|
|
./script/deploy.sh <env> all # api + worker + web
|
|
./script/deploy.sh <env> api worker # subset
|
|
./script/deploy.sh <env> default # api + web only (worker untouched)
|
|
./script/deploy.sh <env> all --dry-run
|
|
```
|
|
|
|
Concrete hosts, ports, and the site's `server_name` live in `asset/manifest.yml`. The shape of the deployment:
|
|
|
|
| Component | Notes |
|
|
| --------- | ---------------------------------------------------------------------- |
|
|
| api | binds the port from `api.config.bind`; firewalld service `moments-api` |
|
|
| worker | no listening port; pollers only |
|
|
| web | per-site nginx ingress; `/api/*` reverse-proxies to the api host |
|
|
| db | postgres mTLS, passwordless |
|
|
|
|
Postgres roles `moments_rw` and `moments_ro` must exist on the primary, with `pg_ident.conf.d/<host>.conf` mapping the api host's FQDN → `moments_ro` and the worker host's FQDN → `moments_rw`. See `asset/sql/bootstrap-moments.sql`, `asset/postgres/ident.conf.tmpl`, and `script/db-perms.sh` (idempotently adds the cert_cn lines on the postgres primary + standby and reloads postgres).
|
|
|
|
Inter-host traffic over the WG mesh: web's nginx connects to the api host in plaintext. The mesh provides the encryption layer; per-hop TLS for an internal HTTP read-only API on already-public data is deferred. If that changes, swap the api binary to rustls + the host cert pair, and update the nginx upstream to `https://`.
|
|
|
|
Secrets are resolved at deploy time via `pass`. The mapping of env-var name → pass-store path lives under `worker.secrets` in `manifest.yml`; `deploy.sh` iterates the map, fetches each secret, and substitutes the matching `{{NAME}}` placeholder in `worker.env.tmpl`. To add a secret: add a `worker.secrets` entry, add `NAME={{NAME}}` to `worker.env.tmpl`, and ensure `pass show <path>` returns the value on the deploying machine.
|
|
|
|
### First-time setup
|
|
|
|
After the first successful prod deploy:
|
|
|
|
1. Point public DNS for the site at the web host's public IP (unproxied).
|
|
2. Confirm `curl --fail --silent --show-error https://<site>/api/v1/healthz` returns `ok`.
|
|
3. If migrating from a predecessor, archive the old repo with a pointer to this one.
|