docs: add port-allocations.md — choosing conflict-unlikely service ports
Guidance for picking a listening port that won't collide on a shared-host fleet: use the registered range (1024-49151), avoid the crowded alt-HTTP cluster (3000/5000/8000/8080/8081/8443/9000...) and never the ephemeral range (49152+, kernel source ports), derive from the service name into a sparse band, and — the actual anti-collision mechanism — record every allocation in a fleet registry. Seeds the registry with the services documented across this repo (pg, neuron, bench, gongfoo controller/agent, cortex) plus newsfeed-api. Motivated by newsfeed-api having been put on 8081 (the alt-HTTP port). Linked from generic.md §9. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP
This commit is contained in:
@@ -390,6 +390,8 @@ If the service must run as root or with extra capabilities, document the reason
|
||||
|
||||
All hosts run `firewalld`. Every service that listens on a port must ship a **named firewalld service definition** rather than opening bare ports in a zone. The service name matches the systemd unit name (minus `.service`).
|
||||
|
||||
**Choosing the port itself** — an uncommon number in the registered range, never a crowded alt-HTTP port (`8080`/`8081`/…) or the ephemeral range, recorded in a fleet registry — is covered in **`port-allocations.md`**. The number there must match this XML, the service's bind config, and the SELinux `semanage port` label (§10).
|
||||
|
||||
### Why named services
|
||||
Named services are self-documenting (`firewall-cmd --list-services` tells you what's actually running), removable atomically on app decommission, and survive zone reassignment without reconfiguration.
|
||||
|
||||
|
||||
105
port-allocations.md
Normal file
105
port-allocations.md
Normal file
@@ -0,0 +1,105 @@
|
||||
# Port allocations for internal services
|
||||
|
||||
Extends `generic.md` §9 (firewalld) and §3 (Binaries and Runtime). Every service that
|
||||
listens on a TCP port needs a port *number*, and on a fleet where many services share
|
||||
hosts — and where a runner, a proxy, and an app can land on the same box — a careless
|
||||
choice collides. This doc is how we pick numbers that don't, and the **registry** of what's
|
||||
already assigned.
|
||||
|
||||
Decision rule (the whole thing in one line):
|
||||
|
||||
> **Pick an uncommon port in the IANA *registered* range (1024–49151), never a crowded
|
||||
> alt-HTTP port and never the ephemeral range, and record it in the registry below.**
|
||||
|
||||
The registry — not cleverness in the number — is what actually prevents collisions. Check
|
||||
it before you assign; add your service when you do.
|
||||
|
||||
---
|
||||
|
||||
## 1. Why the default ports bite
|
||||
|
||||
The ports frameworks and tools reach for by default form a dense cluster that half the
|
||||
software world also uses:
|
||||
|
||||
`3000` `3001` `4000` `5000` `5173` `8000` `8008` `8080` **`8081`** `8443` `9000` `9090` …
|
||||
|
||||
On a dedicated single-service host these are fine. On our fleet they are not: hosts run
|
||||
gongfoo runners, nginx, sometimes more than one app, plus whatever a container or a dev
|
||||
shell brought up. Two things pick `8080`/`8081` and one fails to bind (`EADDRINUSE`) — or,
|
||||
worse, *doesn't* fail: it binds a different interface, the health probe hits the **other**
|
||||
service on that port, and you debug a green check that lies. Avoid the whole cluster.
|
||||
|
||||
## 2. The three ranges
|
||||
|
||||
| Range | IANA class | Use for a listener? |
|
||||
| --- | --- | --- |
|
||||
| `0–1023` | well-known / system | **No.** Requires privilege; reserved for standard services. TLS terminates at nginx (`:443`), so apps don't need these. |
|
||||
| `1024–49151` | **registered / user** | **Yes.** This is where our services live. |
|
||||
| `49152–65535` | dynamic / ephemeral | **No.** The kernel hands these out as *source* ports for outbound connections. A fixed listener here can collide with an ephemeral allocation and fail intermittently — the worst kind of bug. |
|
||||
|
||||
So: **registered range, and specifically a sparse part of it.** The crowded cluster (§1)
|
||||
sits at the bottom of the registered range; climb out of it. A good default band is
|
||||
**`20000–29999`** — clear of the alt-HTTP mess below and the ephemeral range above, and
|
||||
sparsely used. `30000–39999` is equally fine (cortex already lives there).
|
||||
|
||||
## 3. Picking the number
|
||||
|
||||
1. **Derive, don't guess.** Human-chosen ports cluster (everyone likes round numbers and
|
||||
`x080`). Map the service name deterministically into the band instead:
|
||||
|
||||
```sh
|
||||
name=newsfeed
|
||||
python3 -c "import hashlib;print(20000 + int(hashlib.sha256('$name'.encode()).hexdigest(),16) % 10000)"
|
||||
# newsfeed -> 22672
|
||||
```
|
||||
|
||||
This is reproducible (anyone can re-derive it) and spreads services across the band.
|
||||
2. **Check the registry (§5).** If the derived number is taken, bump by 1 until free, or
|
||||
pick a memorable nearby number — then it's a deliberate, recorded choice.
|
||||
3. **Record it** in the registry with the same change that introduces the service.
|
||||
4. A memorable hand-picked number in the band is equally valid (`neuron` = `13131`,
|
||||
`cortex` = `31313`). The rule is *uncommon + registered + recorded*, not *hashed*.
|
||||
|
||||
## 4. Consequences elsewhere
|
||||
|
||||
- **firewalld** (`generic.md` §9): ship the port in the app's named service XML. The
|
||||
registry and the XML must agree.
|
||||
- **SELinux** (`generic.md` §10): a non-standard port the daemon binds must be labelled, or
|
||||
the bind is denied. `semanage port -a -t http_port_t -p tcp <port>` (guard with
|
||||
`semanage port -l | grep` so re-runs are no-ops). Do this in the host-prep step
|
||||
(`infra-setup.sh`), before the first service start.
|
||||
- **Binding scope**: bind loopback (`127.0.0.1`) when nginx is on the same host, or the
|
||||
mesh-routable address when the proxy is on a different host (e.g. an API on one host
|
||||
fronted by the site's edge proxy on another). firewalld's named service bounds who may
|
||||
reach it; the registry bounds what else may bind it.
|
||||
- **Reverse proxy** (`reverse-proxies.md`): the nginx `upstream` points at
|
||||
`<host>:<port>` — the same number as the registry and the service's bind config.
|
||||
|
||||
## 5. The registry
|
||||
|
||||
Canonical list of internal-service port assignments. **Append here when you allocate a
|
||||
port**; treat a duplicate as a bug. (Seeded from the services documented across this repo;
|
||||
extend as you encounter others — this table is authoritative going forward.)
|
||||
|
||||
| Port | Service | Host(s) | Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| `5432` | PostgreSQL | `magrathea`, `frankie` (standby) | standard well-known assignment; mTLS |
|
||||
| `13131` | helexa **neuron** | each GPU host | OpenAI-compatible inference API |
|
||||
| `13132` | helexa-bench read API | `bob` | benchmark telemetry |
|
||||
| `18443` | gongfoo **controller** | `golgafrinchans` | agent-facing RPC (mTLS) |
|
||||
| `22672` | **newsfeed-api** | `slartibartfast` | REST/JSON; fronted by oolon |
|
||||
| `28443` | gongfoo **agent** | each runner host | controller-facing RPC (mTLS) |
|
||||
| `31313` | helexa **cortex** API | `hanzalova` | unified OpenAI/Anthropic gateway |
|
||||
| `31314` | helexa **cortex** metrics | `hanzalova` | Prometheus |
|
||||
|
||||
`443` (nginx edge, every proxy host) and the ephemeral range are intentionally omitted —
|
||||
they aren't allocations.
|
||||
|
||||
## 6. Checklist for a new listening service
|
||||
|
||||
1. Derive a candidate port in `20000–29999` from the service name (§3).
|
||||
2. Check it's free in the registry (§5); bump if taken.
|
||||
3. Record it in the registry in the same change.
|
||||
4. Put the number in exactly these places, all agreeing: the service's bind config, its
|
||||
firewalld service XML (§9), its SELinux `semanage port` label (§10, in host prep), and
|
||||
any nginx `upstream` (`reverse-proxies.md`).
|
||||
@@ -13,6 +13,7 @@ The goal is boring consistency: the same crate layout, the same deploy flow, the
|
||||
- **`generic.md`** — the baseline. Applies to every project unless that project explicitly overrides a section. Covers workspace layout, separation of concerns, configuration, secrets, deployment, service accounts, firewalld, SELinux, and code quality.
|
||||
- **`deployment-gitea-actions.md`** — CI-driven deployment via a Gitea Actions workflow, as an alternative to the `deploy.sh` + `manifest.yml` flow in `generic.md` §7. The workflow is the source of infra truth; the runner deploys as a scoped `gitea_ci` user.
|
||||
- **`gitea-runners.md`** — the catalogue of `gongfoo`-managed CI runner images (what `runs-on:` label gives you which toolchain), how to pick the right one, and how to add or extend an image instead of `dnf`-installing at run time. Makes `deployment-gitea-actions.md` §5 concrete.
|
||||
- **`port-allocations.md`** — how to choose a conflict-unlikely TCP port for an internal service (uncommon, registered range, not the alt-HTTP cluster or the ephemeral range) and the fleet registry of what's already assigned. Extends `generic.md` §9/§3.
|
||||
- **`internal-tls.md`** — provisioning and renewing per-service internal TLS certs (`<service>.internal`) for mesh-only nginx vhosts, extending the PKI conventions in `generic.md` §11.
|
||||
- **`external-tls.md`** — publicly-trusted certs for WAN-facing vhosts via Let's Encrypt (certbot, Cloudflare DNS-01, ECDSA). The external counterpart to `internal-tls.md`.
|
||||
- **`reverse-proxies.md`** — the per-site nginx edge proxies (`oolon` for kosherinata, `hanzalova.internal` for the office), what sits behind each, the public-vs-mesh access paths, and the per-vhost cert choice. Names the topology behind `generic.md` §11 Ingress.
|
||||
|
||||
Reference in New Issue
Block a user