Run 8's validate reported FAIL quantus-arena bob:25033 unknown FAIL quantus-miner quadbrat:9900 unknown ok quantus-miner benjy:9900 up ok quantus-node bob:9615 up which is not a failure: a target Prometheus has only just learned about reports 'unknown' until its first scrape completes, and this step runs seconds after the config reload with a 15s scrape interval. Both were up moments later. The old retry loop broke as soon as ANY quantus target existed, which was immediately true because the node target predated this run. It now waits for the unknowns to resolve, bounded at 120s. A genuinely dead target reports 'down' rather than 'unknown', so this does not paper over real failures -- the same distinction the readiness wait on the node makes. Third race of this shape: deploy finishes before the thing it deployed is observable.
10 KiB
CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
What this repo is
Deployment automation only — systemd units, firewalld definitions, config
templates, one Gitea Actions workflow and one provisioning script, for a Quantus
Planck node and its external GPU miner. There is no application code, no build,
and no test suite: the deployed artifacts are upstream release binaries fetched
by the workflow. Because there is nothing to compile, generic.md §1–§2 (Rust
workspace layout, crate naming) do not apply here.
No secrets live in this repo. QUANTUS_INNER_HASH is a Gitea Actions secret;
the miner's auth token and TLS pin are generated on the node host and copied
host-to-host by the deploy.
Read readme.md first — it carries the topology, the measured hashrate numbers
the design rests on, the port rationale, and the list of deliberate deviations
from house convention.
The one thing that will surprise you
Setting --miner-listen-port on the node disables its built-in CPU mining.
A node with that flag and no external miner attached is a node that authors
nothing, while systemctl is-active reports it perfectly healthy. The same trap
exists on the miner side: a quantus-miner that enumerated no GPU adapter also
looks active. This is why the workflow's validate mode checks
miner_hashes_total is advancing and that the GPU is drawing power, rather
than trusting unit state. Do not "simplify" those checks away.
Facts established by measurement, not assumption
These came from benchmarking the real fleet; don't re-derive or contradict them without re-measuring:
- GPU beats CPU by ~200×. One RTX 5090 at its 400 W floor out-hashes every CPU in the fleet combined by ~24×. CPU mining is disabled everywhere on purpose.
- RTX 5090: 368 MH/s @ 600 W; 281 MH/s @ 400 W (18% better MH/s per watt). RTX 4090: 183 MH/s @ 450 W.
- The PoW is Poseidon2 over Goldilocks — register-resident, no memory hardness. VRAM size and memory bandwidth are irrelevant to hashrate.
quantus-nodehas no environment-variable form for any flag (its--helpcarries no[env:]annotations), so--rewards-inner-hashmust be a CLI argument and is visible inpson the node host. TheEnvironmentFilekeeps it out of the world-readable unit; it does not hide it from a local user.- The node will not start without a pre-generated node key — it exits with
NetworkKeyNotFound(".../network/secret_dilithium")rather than generating one.infra-setup.shcreates it; a deploy must never rotate it, because it is the node's stable network identity. - Node Prometheus is on 9615, not 9616. Upstream's
MINING.mdparameter table is wrong about this; the running binary is the authority.
gitea_ci needs a real shell
Create it with /bin/bash, never /usr/sbin/nologin. The deploy runs
ssh gitea_ci@host <command>; a nologin shell authenticates the key and then
refuses the command with "This account is currently not available". Every
pre-existing gitea_ci on the fleet has bash — infra-setup.sh repairs one that
does not, because a host provisioned with nologin can never be deployed to and
the failure looks like an auth problem rather than a shell one.
Hardening knobs that are load-bearing
RestrictAddressFamilies on the node must include AF_NETLINK. libp2p calls
getifaddrs(), which opens a NETLINK_ROUTE socket; without it the node starts,
reports active, and never binds its P2P port — an undialable peer that looks
healthy. Verified live. The same class of failure is why the validate step
checks listeners rather than trusting systemctl is-active.
If the miner ever fails to reach the node, netlink is the first thing to suspect there too — it is deliberately left out of the miner unit (least privilege, and a QUIC client to a known address should not need interface enumeration), but that is an assumption, not a verified fact.
Deploys must be no-ops when nothing changed
push() uses rsync -ic — checksum, not size+mtime. The fetch job downloads
fresh binaries every run, so mtimes always differ and the default quick check
reports a spurious change on every deploy. Restart is gated on RESTART, set
only when rsync itemizes an actual content difference. Do not restore an
unconditional systemctl restart: this workflow doubles as the health check for
an untouched deployment, and bouncing a syncing node drops every peer.
--node-addr takes an IP, not a name
quantus-miner --node-addr parses as a Rust SocketAddr: no DNS resolution, so
a hostname fails at argument parsing with "invalid socket address syntax". The
deploy resolves the node host on the MINER host at deploy time and renders the
literal into miner.env — which also keeps 10.x out of the repo. Do not
"simplify" it back to ${{ matrix.node }}:$PORT.
Prometheus targets report "unknown" before their first scrape
Any run that ADDS a scrape target will see it as health unknown, not up —
Prometheus has not scraped it yet, and validate runs seconds after the config
reload. validate metrics waits for unknowns to resolve before judging. A
genuinely dead target reports down, so the wait does not hide real failures.
Validate runs seconds after a restart
systemctl restart returns as soon as the process is exec'd, not when it is
serving. The node opens RocksDB and initialises litep2p before binding its P2P
port, which takes appreciably longer than the deploy step's own teardown — so an
immediate assertion reads a starting node as a broken one. Validate waits for
readiness on a bounded poll. Do not replace that with a fixed sleep, and do not
remove it because "it passed locally".
Inline Python in workflow steps
Use a quoted heredoc (python3 - <<'PY') reading a file or an env var. Do NOT
use python3 -c '...' with escaped quotes: \" inside an f-string expression is
read by Python as a line continuation and fails with "unexpected character after
line continuation character". A quoted heredoc needs no shell escaping at all.
Grafana checks cannot use its API
/api/search and most of Grafana's API require auth (401 unauthenticated), and
this workflow deliberately holds no Grafana credentials. Assert provisioning from
journalctl -u grafana.service instead — gitea_ci is in systemd-journal, and
a dashboard that fails to parse logs logger=provisioning.dashboard level=error
and is then silently absent from the UI. /api/health is unauthenticated and is
fine for liveness.
Pipes in validate steps
The workflow shell is bash -e -o pipefail. Any pipe whose reader exits early
— awk '...; exit', grep -q, grep -m1, head — SIGPIPEs the writer, and
pipefail turns that into exit 141. On a large body (a metrics scrape) it fires
reliably; on a small one it hides, so it passes review and fails in CI. Capture
into a variable and parse with a here-string instead:
m=$(run "curl -fsS $url")
v=$(awk '/^metric/{print $2; exit}' <<<"$m") # correct
v=$(run "curl -fsS $url" | awk '/^metric/{print $2; exit}') # 141
ssh argument quoting
run() in the deploy is ssh ... "$@". ssh joins its argument vector with
spaces and the REMOTE shell re-splits it, so any argument containing a space
must be passed as one pre-quoted string:
run "sudo firewall-cmd --zone=$zone --add-rich-rule='$rich'" # correct
run sudo firewall-cmd --zone="$zone" --add-rich-rule="$rich" # word-splits remotely
This bit the firewalld rich rules on run 1 and nothing else, because no other argument in the deploy contains a space. Check this before adding any command with a quoted multi-word argument.
Working on the deploy
- The workflow is the source of infra truth (
deployment-gitea-actions.md). Hosts, ports, versions and paths live in.gitea/workflows/deploy.yaml'senv:block — never introduce a separate manifest. - Anything one-time, or anything involving key material, belongs in
script/infra-setup.sh, not the workflow. infra-setup.shmust stay idempotent and must skip unreachable hosts rather than aborting the run.- Never suppress errors (
generic.md§7): no2>/dev/null, no|| true. If a command may legitimately fail, handle it with a visible message. - Never print a secret (
agent-credentials.md§6).--mint-reward-keyemits only a "value withheld" acknowledgement. Keep it that way; do not add a debug echo. - The
inner_hashis not a secret and must not be turned back into one. The node embeds it verbatim in thePreRuntimedigest of every block it authors, so it is public on-chain from the first block. Nothing here needs a seed phrase either: the node wants only the preimage, and the miner has no knowledge of the payout address at all. - One
inner_hashper node host, in thedeploy-nodematrix. Never collapse them to a shared value "for simplicity" — the value is static, so sharing one publicly ties those nodes to a single operator.doc/wormhole-rewards.md§5. - Version pins are deliberate. Do not add auto-update to a validator.
Further reading
doc/wormhole-rewards.md — what the inner_hash is cryptographically, why the
node takes a preimage rather than an address, what publishing it does and does
not expose, and why the seed-phrase derivation step is a design flaw rather than
an operator error. Read it before changing anything about reward configuration.
Before enabling a miner on any host
A miner host may already be serving helexa neuron from the same GPU. The miner
will saturate it. readme.md documents the options; the deploy deliberately
does not arbitrate. This is an open decision, not an oversight.
Keep the fleet's shape in the workflow
Host assignments, hardware inventories, site or geographic names, ISPs and
addressing belong in the deploy-node / deploy-miner matrices in
.gitea/workflows/deploy.yaml — the single source of infra truth — and nowhere
else. Do not restate them in readme.md, CLAUDE.md, doc/, or unit-file
comments, even as helpful context. Prose should describe roles and measured
hardware classes, never which box is where.