observer/CLAUDE.md

CLAUDE.md

Guidance for Claude Code working in this repository.

Read readme.md first — it carries the mechanism (how authorship is decoded from block headers), the reasoning behind the window semantics, and the list of deliberate deviations from house convention. This file is the things that will bite you.

Conventions: ~/git/architecture — generic.md for the workspace shape, deployment-gitea-actions.md for the deploy, port-allocations.md for the port, reverse-proxies.md and external-tls.md/internal-tls.md for the vhosts.

The things that cost an afternoon

substrate-telemetry sends its JSON in binary WebSocket frames, not text. A client that handles only Message::Text connects, subscribes, reports itself healthy, and receives absolutely nothing — no error, no decode failure, an empty feed and telemetry_nodes: null. Worse, a throwaway probe written in Python works, because json.loads accepts bytes, so the two disagree for no visible reason. Both frame types are accepted in telemetry.rs and rpc.rs. Do not "simplify" either match arm.

Difficulty is a little-endian U512. state_call returns a SCALE-encoded U512, byte-reversed relative to how a hash reads. Decoding it big-endian yields a plausible-looking number that is wrong by ~10^150, and every hashrate derived from it is wrong without ever looking broken. digest::u512_le has the test.

A measured block interval is only valid at the tip. A syncing node imports history at disk speed; dividing difficulty by that gives a hashrate wrong by orders of magnitude that still renders as a number. RollingWindow::push takes an explicit at_tip flag and backfilled blocks contribute no timing sample. The Interval enum exists so a caller cannot forget to say which kind it has.

Windows are block counts, never durations. See readme.md. Anything that turns Window into a time range is wrong.

The difficulty of block h lives in the state of block h-1. pallet_qpow::on_finalize retargets at the end of every block, so QPoWApi_get_difficulty at the latest state returns what the next block must meet, not what the tip met. Recording that against the block just seen is off by one, which on a gradual retarget is a number that still looks entirely right. RpcClient::difficulty_for_child_of asks at header.parent_hash, which is the same call the miner made when it built the block. It needs state the node may have pruned; an error there is None, never a fallback to the current value — substituting it is how a gap fill stamps today's difficulty across a stretch of old heights and draws a flat plateau in the history chart.

Attribution confidence is measured over votes cast, not over the window. votes holds one slot per block checked and a None for every block whose first reporter the feed could not separate from its second. Those are abstentions. Dividing agreement by the window length counts them as votes against, and on a chain where clean leads are the minority that makes MIN_CONFIDENCE unreachable: mainnet resolves a lead on about a quarter of blocks, which capped every author at 0.25 against a 0.6 bar and named nobody, however unanimous the cast votes were. MIN_VOTES is the companion guard, so one lucky vote in a silent window is not 100% agreement.

Two things that look like this bug and are not: the feed sends a 190-node roster before any live block traffic — ImportedBlock does not start until about 40 s after subscribe:, so any probe shorter than that sees zero and concludes the feed is dead. And the roster's AddedNode entries carry a placeholder block of height 0 with a zero hash; counting those as import reports inflates the apparent number of nodes claiming to be first.

observed_at is only a measurement when at_tip is true. The flag is now a column (0002). A gap fill writes a whole batch of observed_at within the same second, so any query treating it as timing data must filter on it. The chain series sidesteps this entirely by using authored_at, the chain's own clock.

chain_subscribeNewHeads skips blocks. It reports the best head, so when several import at once the intermediates never arrive. It is a liveness signal, not a ledger — ingest fills gaps against chain_getBlockHash. Removing that would quietly under-count exactly the miners who won blocks during a burst.

Blocks are keyed on (chain, height), not hash. This chain reorgs; the upsert is what makes a replacement overwrite rather than accumulate. A schema keyed on the hash would inflate the losing fork's author forever.

A reorg copies the old row into block_displacement first (0003), in the same statement as the upsert. That is not optional bookkeeping: the node cannot be asked for the block later. blocks-pruning defaults to archive-canonical, so non-canonical bodies go once finality passes them — about a hundred blocks behind the tip on Planck — and difficulty is a state read against a 256-block default. Whatever record_blocks captures is all there will ever be. The two halves are one statement because every data-modifying CTE sees the same snapshot, so the copy reads the pre-update row; splitting them would need an explicit transaction to be equivalent.

Facts established by measurement

Taken from the live Planck chain, 2026-09-04. Don't re-derive or contradict without re-measuring.

Planck is no longer the tracked chain. Mainnet launched 2026-09-09 and the node on bob was switched to it; the [[chains]] entry followed the node, so quantus is now what that endpoint serves and Planck is telemetry-listed only. The decoding facts below still hold — same runtime family, same digest shape — but every timing figure was measured on Planck against a 6 s target and says nothing about mainnet, which targets 12 s.

• Header digest shape is exactly 0x06 + 706f775f + compact 0x80 (32) + 32-byte preimage. Genesis is the only header without one; 12/12 recent headers decoded.
• Telemetry propagation: first reporter stamped 0, others 50–620 ms. The ATTRIBUTION_LEAD_MS = 20 threshold only has to exclude a tie — identity over many blocks does the discriminating.
• Planck's observed block interval is ~13–15 s against a 6 s target, with consecutive gaps of 1.6 s, 4.4 s, 13.5 s, 26 s, 27 s. This is why MIN_TIP_SAMPLES is 20: at five samples the headline hashrate swings by a factor of three between refreshes.
• system_properties: PLK, 12 decimals, ss58 prefix 189. Genesis 0x4901bf5c…e65e72.
• Mainnet, read from bob 2026-09-09: system_chain "Quantus", token QTC, 12 decimals, ss58 189, genesis 0xfb5487c0…626fba. Its target block time is 12 s, not Planck's 6 — TARGET_BLOCK_TIME_MS = 12_000 at commit b017e642, which is the 0.11.1-b017e6420aa the node reports. Check the constant at the commit the node actually runs, not at the tip of the chain repo: this figure is the denominator for every hashrate shown before twenty tip samples exist, so getting it wrong publishes a headline that is wrong by exactly that factor and labelled "nominal" rather than broken.
• The telemetry feed URL is wss://feed-telemetry.quantus.cat/feed — found in /tmp/env-config.js on the telemetry site, not the /feed path on the main host, which 404s.

The frontend's types are generated

web/src/api/generated/ is written by ts-rs from blackbeard-entities when cargo test -p blackbeard-entities runs. Never edit those files. CI fails the build if the committed output is stale.

u64 maps to bigint by default, which is wrong — serde_json puts a u64 on the wire as a JSON number, so the type and the runtime value disagree. Every u64 DTO field carries #[ts(type = "number")]. Add it to any new one.

Separation of concerns is load-bearing here

• blackbeard-core has no I/O, no clock, no sockets. That is what lets the decoding and the statistics — the two parts that are easy to get subtly wrong and hard to notice — be exercised by unit tests instead of only against a live chain. Keep it that way; if something there needs the time, pass it in.
• blackbeard-data owns retries, reconnects and schema.
• blackbeard-api wires them together and owns nothing else.

Locking

ChainRuntime::inner is a std::sync::RwLock, never held across an .await. Everything under it is CPU work on in-memory collections. Do not switch it to tokio's — that variant makes it easy to accidentally hold a lock across an await and stall a worker, for no benefit here.

Database work

Queries use sqlx::query! (compile-time checked) with the offline cache in .sqlx/ committed, so CI builds with SQLX_OFFLINE=true and no database.

After changing any query, regenerate the cache or CI fails on a stale one:

podman run -d --rm --name bb-pg -e POSTGRES_PASSWORD=dev \
  -e POSTGRES_DB=blackbeard -p 55432:5432 docker.io/library/postgres:18-alpine
export DATABASE_URL='postgres://postgres:dev@127.0.0.1:55432/blackbeard'
cargo sqlx prepare --workspace -- --all-targets

Migrations are sequentially versioned and immutable once committed. Correct a mistake with a new file, never by editing one that has landed — the runner's checksum diverges and it refuses to start.

Charts

Any chart added here must be read against the dataviz skill first. The constraint already in force: bronze #bd8829 and crimson #d8453a are adjacent hues and fail CVD separation as a categorical pair. Every chart on the site is therefore single-series — magnitude, one hue — with identity carried by labels and row treatment. A second series in the accent colour is the one change that would break the palette, and it would not look broken.

This is why the stat-tile sparklines do not follow the usual "current period in the accent" convention: the newest point is emphasised with --data-bright, a lighter step of the same hue. They also do not start at zero — a sparkline is read for shape — which is why the hover readout and the aria-label both carry real numbers, and why the readout borrows the tile's note line instead of floating a tooltip: five overlays in a row of 180px tiles is not a hover layer, it is a pile.

web/src/index.css documents the validated values. Re-run the validator after touching them:

node <skill>/scripts/validate_palette.js "#bd8829" --mode dark --surface "#14110d"

Deployment gotchas learned the hard way

A sudoers grant matches the whole argument vector. The restorecon grant names three paths; running it with one path is a different command and asks for a password. Run the command exactly as infra-setup.sh grants it, which is what the workflow does.

The config check runs as blackbeard, not root — runas (blackbeard) in sudoers. A root-run check passes on a config the service account cannot read, which is the failure it exists to catch.

CorsLayer::allow_origin replaces, it does not append. Folding over a list of origins leaves only the last one allowed. It does not break the site — the frontend is same-origin and never consults CORS — so it is invisible until something else tries to call the API. Use AllowOrigin::list.

The API binds the host's mesh address, never loopback and never 0.0.0.0. The fleet has one firewalld default zone, so a wildcard bind plus the named service would publish it on every address the host carries; and the edge proxy is at another site, so loopback-only would not serve the site at all. Anything probing 127.0.0.1:25864 gets connection-refused against a perfectly healthy daemon — which is exactly how the first green build still failed its health check. Use $(hostname -f), which resolves to the mesh address.

The edge proxy is at a different site from the API. oolon (kosherinata) fronts the name; the API is on bob (hanzalova) because that is where the Planck node lives. A health probe on the API host cannot see that hop, so there is a separate check that curls the API from oolon. A firewalld service scoped to bob's own /16 would leave a live site with a dead /v1 and nothing would fail.

The edge proxy cannot reach its own public name. From inside the mesh blackbeard.observer resolves to the site's WAN address and dead-ends on the OPNsense LAN interface (reverse-proxies.md §2). Verify the vhost with --resolve blackbeard.observer:443:127.0.0.1, which still exercises the :443 stream router, SNI, the vhost, the cross-site hop and the API.

Runner labels. fedora-* images have no cargo — Rust builds go on rust, which is based on runner-fedora-44 and so carries node + pnpm too. Never corepack enable; pnpm is already on PATH (gitea-runners.md §4).

WebSocket upgrade tests need --http1.1. The vhost serves HTTP/2, and curl negotiates h2, where the Connection: Upgrade handshake is not how WebSockets work — you get a 400 from axum that looks like a proxy misconfiguration.

Verifying a change

systemctl is-active is not evidence this works. A daemon with a node it cannot decode, or a telemetry feed it silently ignores, is perfectly "active" while serving an empty leaderboard. Check the numbers:

cargo run -p blackbeard-cli -- probe --rpc-url http://bob.hanzalova.internal:9944
cargo run -p blackbeard-cli -- standings --chain quantus
curl -s localhost:25864/v1/chains/quantus/summary | python3 -m json.tool

The block route degrades rather than failing, so it is worth checking both ends of that: a recent height should carry difficulty, extrinsics and seconds_since_parent, while one a few days old keeps the body and loses difficulty to state pruning unless this observer recorded it at the time.

curl -s localhost:25864/v1/chains/quantus/blocks/1000 | python3 -m json.tool

A working deployment shows a non-null telemetry_nodes, distinct_miners above one, and named rows in the standings.