Files
architecture/port-allocations.md
rob thijssen d76083f1ea docs: add port-allocations.md — choosing conflict-unlikely service ports
Guidance for picking a listening port that won't collide on a shared-host
fleet: use the registered range (1024-49151), avoid the crowded alt-HTTP
cluster (3000/5000/8000/8080/8081/8443/9000...) and never the ephemeral range
(49152+, kernel source ports), derive from the service name into a sparse band,
and — the actual anti-collision mechanism — record every allocation in a fleet
registry. Seeds the registry with the services documented across this repo
(pg, neuron, bench, gongfoo controller/agent, cortex) plus newsfeed-api.

Motivated by newsfeed-api having been put on 8081 (the alt-HTTP port). Linked
from generic.md §9.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP
2026-07-08 12:24:16 +03:00

106 lines
5.6 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Port allocations for internal services
Extends `generic.md` §9 (firewalld) and §3 (Binaries and Runtime). Every service that
listens on a TCP port needs a port *number*, and on a fleet where many services share
hosts — and where a runner, a proxy, and an app can land on the same box — a careless
choice collides. This doc is how we pick numbers that don't, and the **registry** of what's
already assigned.
Decision rule (the whole thing in one line):
> **Pick an uncommon port in the IANA *registered* range (102449151), never a crowded
> alt-HTTP port and never the ephemeral range, and record it in the registry below.**
The registry — not cleverness in the number — is what actually prevents collisions. Check
it before you assign; add your service when you do.
---
## 1. Why the default ports bite
The ports frameworks and tools reach for by default form a dense cluster that half the
software world also uses:
`3000` `3001` `4000` `5000` `5173` `8000` `8008` `8080` **`8081`** `8443` `9000` `9090`
On a dedicated single-service host these are fine. On our fleet they are not: hosts run
gongfoo runners, nginx, sometimes more than one app, plus whatever a container or a dev
shell brought up. Two things pick `8080`/`8081` and one fails to bind (`EADDRINUSE`) — or,
worse, *doesn't* fail: it binds a different interface, the health probe hits the **other**
service on that port, and you debug a green check that lies. Avoid the whole cluster.
## 2. The three ranges
| Range | IANA class | Use for a listener? |
| --- | --- | --- |
| `01023` | well-known / system | **No.** Requires privilege; reserved for standard services. TLS terminates at nginx (`:443`), so apps don't need these. |
| `102449151` | **registered / user** | **Yes.** This is where our services live. |
| `4915265535` | dynamic / ephemeral | **No.** The kernel hands these out as *source* ports for outbound connections. A fixed listener here can collide with an ephemeral allocation and fail intermittently — the worst kind of bug. |
So: **registered range, and specifically a sparse part of it.** The crowded cluster (§1)
sits at the bottom of the registered range; climb out of it. A good default band is
**`2000029999`** — clear of the alt-HTTP mess below and the ephemeral range above, and
sparsely used. `3000039999` is equally fine (cortex already lives there).
## 3. Picking the number
1. **Derive, don't guess.** Human-chosen ports cluster (everyone likes round numbers and
`x080`). Map the service name deterministically into the band instead:
```sh
name=newsfeed
python3 -c "import hashlib;print(20000 + int(hashlib.sha256('$name'.encode()).hexdigest(),16) % 10000)"
# newsfeed -> 22672
```
This is reproducible (anyone can re-derive it) and spreads services across the band.
2. **Check the registry (§5).** If the derived number is taken, bump by 1 until free, or
pick a memorable nearby number — then it's a deliberate, recorded choice.
3. **Record it** in the registry with the same change that introduces the service.
4. A memorable hand-picked number in the band is equally valid (`neuron` = `13131`,
`cortex` = `31313`). The rule is *uncommon + registered + recorded*, not *hashed*.
## 4. Consequences elsewhere
- **firewalld** (`generic.md` §9): ship the port in the app's named service XML. The
registry and the XML must agree.
- **SELinux** (`generic.md` §10): a non-standard port the daemon binds must be labelled, or
the bind is denied. `semanage port -a -t http_port_t -p tcp <port>` (guard with
`semanage port -l | grep` so re-runs are no-ops). Do this in the host-prep step
(`infra-setup.sh`), before the first service start.
- **Binding scope**: bind loopback (`127.0.0.1`) when nginx is on the same host, or the
mesh-routable address when the proxy is on a different host (e.g. an API on one host
fronted by the site's edge proxy on another). firewalld's named service bounds who may
reach it; the registry bounds what else may bind it.
- **Reverse proxy** (`reverse-proxies.md`): the nginx `upstream` points at
`<host>:<port>` — the same number as the registry and the service's bind config.
## 5. The registry
Canonical list of internal-service port assignments. **Append here when you allocate a
port**; treat a duplicate as a bug. (Seeded from the services documented across this repo;
extend as you encounter others — this table is authoritative going forward.)
| Port | Service | Host(s) | Notes |
| --- | --- | --- | --- |
| `5432` | PostgreSQL | `magrathea`, `frankie` (standby) | standard well-known assignment; mTLS |
| `13131` | helexa **neuron** | each GPU host | OpenAI-compatible inference API |
| `13132` | helexa-bench read API | `bob` | benchmark telemetry |
| `18443` | gongfoo **controller** | `golgafrinchans` | agent-facing RPC (mTLS) |
| `22672` | **newsfeed-api** | `slartibartfast` | REST/JSON; fronted by oolon |
| `28443` | gongfoo **agent** | each runner host | controller-facing RPC (mTLS) |
| `31313` | helexa **cortex** API | `hanzalova` | unified OpenAI/Anthropic gateway |
| `31314` | helexa **cortex** metrics | `hanzalova` | Prometheus |
`443` (nginx edge, every proxy host) and the ephemeral range are intentionally omitted —
they aren't allocations.
## 6. Checklist for a new listening service
1. Derive a candidate port in `2000029999` from the service name (§3).
2. Check it's free in the registry (§5); bump if taken.
3. Record it in the registry in the same change.
4. Put the number in exactly these places, all agreeing: the service's bind config, its
firewalld service XML (§9), its SELinux `semanage port` label (§10, in host prep), and
any nginx `upstream` (`reverse-proxies.md`).