feat(deploy): stake BUH_NODE_PORT on 31415 + testnet peering doc
Some checks failed
deploy / Build buh-api + buh-cli (static musl) (push) Waiting to run
deploy / Deploy buh node to slartibartfast (push) Has been cancelled

Settle buh's standardised ingress port on 31415/tcp — unassigned in IANA,
clear of the crowded 8443/9443/6443 appliance space, and below the Linux
ephemeral range so it's stable to forward from the edge. Updated the code
default (config.rs node_bind), the CI workflow env, firewalld service,
operator-model deploy.sh/manifest.yml, and the asset readme.

Also drop the SELinux `semanage port` step (and its two sudoers rules):
buh-api runs as unconfined_service_t and may bind any port, so BUH_NODE_PORT
needs no http_port_t label. The firewalld rules are keyed on the service
NAME (buh-node), not the port — so a port change is now purely repo-side and
needs no infra-setup re-run on the host.

Add doc/testnet.md: the per-node-CA trust model, the 31415 convention, a
node roster (incl. the slarti dogfood node + its CA fingerprint), and the
mutual trust / peer ping workflow for joining the testnet.

Gate: cargo fmt + clippy -D warnings clean; config renders node_bind
0.0.0.0:31415; sudoers passes visudo -c; deploy.yml valid YAML.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EB3LjarCdXxqrJ4tFLn8LB
This commit is contained in:
2026-06-30 18:05:45 +03:00
parent 0a6d05c5da
commit 9d5919b119
8 changed files with 147 additions and 18 deletions

View File

@@ -41,7 +41,7 @@ concurrency:
env:
# --- infra truth ---
NODE_HOST: slartibartfast.kosherinata.internal
NODE_PORT: "8443" # BUH_NODE_PORT — the only edge port (PQ-mTLS ingress)
NODE_PORT: "31415" # BUH_NODE_PORT — buh's staked port; the only edge port (PQ-mTLS ingress)
ADMIN_PORT: "8081" # loopback operator admin API (health probe target)
NODE_SANS: '["slartibartfast.kosherinata.internal"]' # TOML array for the leaf SANs
MUSL_TARGET: x86_64-unknown-linux-musl
@@ -169,22 +169,22 @@ jobs:
asset/firewalld/buh-node.xml \
gitea_ci@"${NODE_HOST}":/etc/firewalld/services/buh-node.xml
- name: Apply SELinux + firewalld (idempotent)
- name: Apply SELinux contexts + firewalld (idempotent)
run: |
ssh gitea_ci@"${NODE_HOST}" '
set -euo pipefail
sudo /usr/sbin/restorecon -R /usr/local/bin/buh-api /usr/local/bin/buh-cli /etc/buh /var/lib/buh
# Label the PQ-mTLS ingress port (guarded so re-runs are no-ops; 8443
# is usually already http_port_t).
if ! sudo /usr/sbin/semanage port -l | grep -E "^http_port_t" | grep -qw '"${NODE_PORT}"'; then
sudo /usr/sbin/semanage port -a -t http_port_t -p tcp '"${NODE_PORT}"'
fi
# firewalld only learns a freshly-shipped custom service after reload.
# No SELinux port labeling: buh-api runs as unconfined_service_t (a plain
# systemd unit with no policy module), which may bind any port — so
# BUH_NODE_PORT needs no http_port_t entry. Keeping it out means a port
# change is purely repo-side: the firewalld rules below are keyed on the
# service NAME (buh-node), and the freshly-shipped buh-node.xml carries the
# new port, so a reload picks it up with no sudoers/infra-setup change.
sudo /usr/bin/firewall-cmd --reload
if ! sudo /usr/bin/firewall-cmd --query-service=buh-node; then
sudo /usr/bin/firewall-cmd --add-service=buh-node --permanent
sudo /usr/bin/firewall-cmd --reload
fi'
fi
sudo /usr/bin/firewall-cmd --reload'
- name: Restart buh-node
run: |

View File

@@ -50,7 +50,7 @@ here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
: "${PKI_ENABLED:=true}"
: "${PKI_DIR:=${STATE_DIR}/pki}"
: "${NODE_BIND:=0.0.0.0:8443}"
: "${NODE_BIND:=0.0.0.0:31415}"
: "${PKI_SANS:=[\"localhost\"]}"
: "${LEAF_TTL_HOURS:=48}"
: "${ROTATE_EVERY_HOURS:=24}"

View File

@@ -5,5 +5,5 @@
the relay/blob API served over X25519MLKEM768 mutual TLS, with peers pinned per-CA. The plain
loopback health port (127.0.0.1:8080) is never opened. Forward this port from the edge
(OPNsense/router) to the node host.</description>
<port protocol="tcp" port="8443"/>
<port protocol="tcp" port="31415"/>
</service>

View File

@@ -33,7 +33,7 @@ environments:
pki:
enabled: true
dir: /var/lib/buh/pki # CA key + cert; the node generates these on first start
node_bind: 0.0.0.0:8443 # BUH_NODE_PORT — the only port exposed at the edge
node_bind: 0.0.0.0:31415 # BUH_NODE_PORT — the only port exposed at the edge
sans: [node1.example.com] # hostnames/IPs the leaf answers to
leaf_ttl_hours: 48
rotate_every_hours: 24 # in-process leaf rotation (no step-ca, no .path units)

View File

@@ -23,7 +23,7 @@ plane, no shared database, and no central PKI**:
process* (`[relay].sweep_interval_seconds`), because Turso locks the datastore exclusively — a
separate `buh-cli sweep` cannot open the DB while the daemon holds it.
- `systemd/buh.sysusers.conf` — the unprivileged `buh` service account.
- `firewalld/buh-node.xml` — opens **`BUH_NODE_PORT` (8443)**, the single PQ-mTLS ingress port.
- `firewalld/buh-node.xml` — opens **`BUH_NODE_PORT` (31415)**, the single PQ-mTLS ingress port.
The plain loopback health port is never exposed.
- `deploy.sh` — installs the above on the local host, renders the config, has the node generate
its CA, and prints the CA fingerprint to share with peers.

View File

@@ -20,10 +20,12 @@ gitea_ci ALL=(root) NOPASSWD: /usr/bin/systemd-sysusers
gitea_ci ALL=(root) NOPASSWD: /usr/bin/install -d -o root -g buh -m 0750 /etc/buh
gitea_ci ALL=(root) NOPASSWD: /usr/bin/install -d -o buh -g buh -m 0750 /var/lib/buh
# --- SELinux + firewalld ---
# --- SELinux contexts + firewalld ---
# No `semanage port` rules: buh-api runs as unconfined_service_t and may bind any
# port, so BUH_NODE_PORT needs no http_port_t label. The firewalld rules below are
# keyed on the service NAME (buh-node), not the port number, so changing
# BUH_NODE_PORT is purely repo-side and needs no edit here / no infra-setup re-run.
gitea_ci ALL=(root) NOPASSWD: /usr/sbin/restorecon -R /usr/local/bin/buh-api /usr/local/bin/buh-cli /etc/buh /var/lib/buh
gitea_ci ALL=(root) NOPASSWD: /usr/sbin/semanage port -l
gitea_ci ALL=(root) NOPASSWD: /usr/sbin/semanage port -a -t http_port_t -p tcp 8443
gitea_ci ALL=(root) NOPASSWD: /usr/bin/firewall-cmd --reload
gitea_ci ALL=(root) NOPASSWD: /usr/bin/firewall-cmd --query-service\=buh-node
gitea_ci ALL=(root) NOPASSWD: /usr/bin/firewall-cmd --add-service\=buh-node --permanent

View File

@@ -131,7 +131,7 @@ impl Default for AppConfig {
pki: PkiConfig {
enabled: false,
dir: "/var/lib/buh/pki".to_string(),
node_bind: "0.0.0.0:8443".to_string(),
node_bind: "0.0.0.0:31415".to_string(),
sans: vec!["localhost".to_string()],
leaf_ttl_hours: 48,
rotate_every_hours: 24,

127
doc/testnet.md Normal file
View File

@@ -0,0 +1,127 @@
# buh testnet — peering guide
How to stand up a buh node, join it to the testnet, and establish a peering with
another node. buh is an **anti-hub**: there is no central server, no shared root
CA, and no registry. A "testnet" is just a set of nodes that have chosen to trust
each other's certificate authorities.
## Trust model in one paragraph
Every node is **its own CA**. On first start it generates a CA (`ca.cert.der` +
`ca.key.pem` under its pki dir) and rotates a short-lived TLS leaf from it in
process. Its identity is the **CA fingerprint** — the lowercase hex SHA-256 of the
CA certificate DER (`:` separators are accepted on input but not required). Two
nodes talk over **mutual** PQ-mTLS (X25519MLKEM768): a handshake succeeds only
when **each side has pinned the other's CA fingerprint**. Trust is therefore
explicit, pairwise, and symmetric — there is nothing to "join", only peers to
exchange fingerprints with. Removing trust (`peer distrust`) takes effect on the
peer's **next handshake**, with no restart.
## The staked port: 31415
buh standardises on **`BUH_NODE_PORT = 31415/tcp`** — the single PQ-mTLS ingress
port a node exposes (chosen to avoid the crowded `8443`/`9443`/`6443` space; it is
unassigned in IANA and below the Linux ephemeral range, so it is safe to forward).
Everything else stays local:
| Port | Bind | Purpose | Exposed? |
|------|------|---------|----------|
| `31415` | `0.0.0.0:31415` | PQ-mTLS relay/blob API (peers + clients) | **yes — forward from the edge** |
| `8081` | `127.0.0.1:8081` | operator admin API (peer-trust mgmt) | no — loopback only, no auth |
| `8080` | `127.0.0.1:8080` | plain health/debug (only when pki is off) | no |
Forward `31415/tcp` from your edge (OPNsense/router) to the node host. Peers reach
you at `your-edge-hostname:31415`.
## Testnet roster
Record each node's reachable address and CA fingerprint here as it joins.
| Node | Edge address (`host:31415`) | CA fingerprint | Role |
|------|------------------------------|----------------|------|
| `slartibartfast` (dogfood) | _\<edge-fqdn\>_:31415 — internal mesh `slartibartfast.kosherinata.internal:31415` | `3c8f125861f3c39f849a469cb32ef599000c71896ea1ccc8a5baaad7419ef808` | relay + blob (fs) |
> Fill in slarti's `<edge-fqdn>` once edge port-forwarding to `31415` is in place.
> Re-keying a node (`buh-cli ca rotate --force`) changes its fingerprint — update
> this table and every peer must re-pin.
## Join the testnet
### 1. Stand up your node
Either model from `asset/readme.md`:
- **CI** — push to a buh checkout whose `.gitea/workflows/deploy.yml` targets your
host (the dogfood node uses this).
- **Manual** — `sudo PKI_SANS='["your-node.example"]' ./asset/deploy.sh` on the host.
Confirm it is healthy (on the host, via the loopback admin API):
```sh
curl -fsS http://127.0.0.1:8081/admin/info
# {"ca_fingerprint":"…","trusted_peers":N}
```
### 2. Learn your CA fingerprint
```sh
buh-cli ca show # prints this node's CA fingerprint
```
This is the value you hand to peers. Exchange fingerprints **out of band** (signal,
in person, an existing secure channel) — buh deliberately has no fingerprint
directory to lie to you.
### 3. Trust your peer, and have them trust you
Peering is symmetric — **both** sides run a `trust`:
```sh
# on YOUR node — pin the peer's CA
buh-cli peer trust 3c8f125861f3c39f849a469cb32ef599000c71896ea1ccc8a5baaad7419ef808 \
--note "slartibartfast (dogfood)"
# on the PEER node — they pin YOUR CA
buh-cli peer trust <your-ca-fingerprint> --note "my-node"
```
`peer` commands talk to the running node's loopback admin API, so trust changes are
live — they apply on the next handshake without a restart. (With the daemon
stopped, the CLI falls back to opening the datastore directly.) Review with:
```sh
buh-cli peer list
```
### 4. Verify the peering
```sh
buh-cli peer ping <peer-edge-host>:31415
```
`peer ping` performs a real mutual PQ-mTLS handshake and reports the peer's
advertised CA fingerprint + health. It **succeeds only when both directions of
trust are in place** — if it fails, the usual cause is that one side hasn't pinned
the other yet (or the fingerprints don't match what was exchanged).
## Operations
- **Revoke a peer:** `buh-cli peer distrust <ca-fp>` — refused on their next
handshake, no restart.
- **Re-key this node:** `buh-cli ca rotate --force` — destructive; generates a new
CA (old one backed up to `*.bak`), so **every** peer must re-pin the new
fingerprint and the roster must be updated.
- **Keep the admin API loopback:** it has no auth; the daemon refuses a
non-loopback `[admin].bind`, and it must never be opened in firewalld.
- **Changing the port:** `BUH_NODE_PORT` is repo-side only — edit
`NODE_PORT`/`node_bind` and `asset/firewalld/buh-node.xml`; the firewalld rules
are keyed on the service name, and the node runs unconfined (no SELinux port
label needed), so no host re-provisioning is required. Re-point edge forwarding
to the new port.
## Reference
- CA fingerprint = lowercase hex SHA-256 of the CA cert DER (`buh-cli ca show`).
- State: `/var/lib/buh` (`relay.db`, `pki/`, `blobs/`); config: `/etc/buh/config.toml`.
- CLI talks to a running node via `--admin-url` (default `http://127.0.0.1:8081`,
env `BUH_ADMIN_URL`).