rob thijssen
7e5c2de936
Adds `GET /v1/chains/{chain}/series` and a sparkline under network hashrate,
difficulty, block time and miners. Height gets none — it only goes up, and a
straight diagonal reports nothing.
The chart is bucketed by height, not by time: every point is the same number of
blocks, so every point carries equal statistical weight and a stretch nobody
recorded stays the same width on the axis as one that was. A bucket is the
window over fifty, which makes a full window exactly fifty buckets — that is
what lets the rolling miner count at the last point be the same number printed
in the tile above it, rather than a per-bucket count that would sit well below
it. Buckets are anchored to absolute height for the same reason `miner_series`
bins from the epoch: anchored to the tip instead, every boundary slides by one
on every block and the chart shifts under a reader watching it.
Interval comes from `authored_at`, never `observed_at`. A gap fill writes a
whole batch of observation times within the same second, so deltas taken from
it read as a chain producing hundreds of blocks a second — the trap
`RollingWindow`'s `at_tip` flag exists to avoid, which until now lived only in
memory. It is a column as of `0002`, so the distinction survives a restart and
`observed_at` is interpretable at all.
Each point's interval is a three-bucket moving average. The mean of one
bucket's gaps still carries real Poisson error — 72 blocks of a 15 s interval
lands at ±1.8 s — and across fifty points that draws a chain that appears to
change size every few minutes and does not.
Fixes a wrong number found on the way in: `pallet_qpow` retargets in
`on_finalize`, so `QPoWApi_get_difficulty` at the latest state is the
difficulty for the *next* block, not the one just seen. Every stored difficulty
was off by one block — invisible, because a gradual retarget still looks
plausible. It is now asked at `header.parent_hash`, the same call the miner
made when it built the block. A node that has pruned that state gets `None`
rather than the current value: substituting it is exactly how a gap fill would
stamp today's difficulty across a stretch of old heights. The CLI backfill had
the same bug at greater scale, copying one reading across an entire range.
Unknowns are drawn as breaks in the line, never interpolated: an outage, a
difficulty a pruned node cannot supply, a point without a full window of
authors behind it. A chart that guesses across what was never recorded is the
same class of lie as a hashrate divided by a sync speed, and just as hard to
catch afterwards.
Read against the dataviz skill first, per CLAUDE.md. Single series, one hue;
emphasis on the newest point is `--data-bright`, a lighter step of the same
bronze, because the usual "current period in the accent" convention is the one
change that would break this palette. The hover readout borrows each tile's
note line — five floating tooltips across a row of 180px tiles is a pile, not a
hover layer — and the same readout is on keyboard focus and in the aria-label,
since the line does not start at zero and its height is not a quantity anyone
should be estimating.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jp6a8EDar9ueEhAxzep4V5
blackbeard.observer
Live miner leaderboard and network hashrate for the Quantus
blockchain and its Planck testnet, at https://blackbeard.observer
(blackbeard.internal from the mesh).
The point of it: a miner can find their own row, see what share of the network they are actually winning, and watch it move — against everyone else, in real time. Mining is a competition that normally gives its participants no scoreboard. This is the scoreboard.
How it knows who mined what
Every block header carries its author's 32-byte wormhole reward preimage in a
PreRuntime digest stamped with the engine id pow_. The node writes it there so
the mining-rewards pallet can read it back and derive the payout address
on-chain — and as a side effect, authorship for every miner on the network is
derivable from headers alone.
That is the whole mechanism, and its consequences are worth stating plainly:
- No registration, no opt-in, no way to be left out. Every miner appears the moment they win a block. Nothing here depends on anyone's cooperation.
- The preimage is public, not secret. It is published from a miner's first
authored block onward. Spending the rewards needs a plonky2 proof of knowledge
of the underlying secret, which the preimage does not reveal — but it does make
mining income permanently attributable. The privacy in this design is at the
exit, not at receipt.
lair/quantus'sdoc/wormhole-rewards.mdhas the full analysis. - A leaderboard row is a preimage, not a person. One operator may run several; the count of distinct preimages is a lower bound on miner count.
Difficulty comes from the QPoWApi_get_difficulty runtime call. Difficulty is
the expected number of hashes to win a block, so network hashrate is difficulty
divided by the block interval — and a miner's implied hashrate is their share of
recent blocks times that.
The mechanics were first worked out in lair/quantus's Prometheus arena
exporter (asset/arena/quantus-arena-exporter.py), which remains the reference
for anyone checking this implementation against a second one.
Names
The chain gives us preimages; a board of 64-character hex strings is honest and unreadable. substrate-telemetry closes the gap indirectly.
The feed reports block imports with a propagation time, stamping the first
reporter of a hash with 0. A node imports its own block before announcing it,
so the author — if it is on telemetry at all — is nearly always first. One block
proves nothing (measured on Planck, an author's lead over the second reporter is
50–620 ms, which a well-peered bystander can produce), so the observer votes
over a rolling window and only shows a name once enough blocks agree.
An attributed name is therefore an inference, never a claim the miner made and never an identity check. The UI marks it as such, carries the confidence in a tooltip, and falls back to the abbreviated preimage rather than guessing. Where two miners resolve to the same node name — one operator with several preimages, or two people who never renamed their node — every member of the colliding group gets a preimage suffix, so no row is silently privileged as "the real one".
Layout
Cargo workspace plus a Vite frontend, per ~/git/architecture/generic.md §1.
crates/
blackbeard-entities/ domain types + the browser wire protocol; ts-rs exports
the TypeScript the frontend consumes. No I/O.
blackbeard-core/ header/SCALE decoding, hashrate maths, the rolling
window, telemetry attribution voting. Pure — no I/O,
no clock, no sockets, exercised entirely by unit tests.
blackbeard-data/ chain JSON-RPC, the telemetry feed, Postgres.
blackbeard-api/ the daemon: ingest tasks, REST, WebSocket fanout.
blackbeard-cli/ operator tools: probe, backfill, standings.
web/ Vite + React + SWC + TS. Static build, served by nginx.
asset/ systemd, firewalld, nginx, config template, bootstrap SQL.
script/infra-setup.sh one-time host provisioning, operator-run.
The frontend's types are generated from the Rust. cargo test -p blackbeard-entities writes web/src/api/generated/; CI fails the build if the
committed output is stale. Edit the Rust DTO, never the TypeScript — otherwise
the two compile cleanly and disagree at runtime.
Real time, without observables
One WebSocket per browser tab. On subscribe the client gets a full snapshot;
after that only deltas — a block on every block, a summary when the headline
numbers move, a leaderboard when the standings change. The page never polls and
never refetches.
The head stream is itself a push: chain_subscribeNewHeads over the node's
WebSocket, so the observer learns of a block the moment the node imports it. The
only polling anywhere is difficulty and sync state, on a four-second timer.
RxJS was considered and not used. This needs one stream, one reducer and one
subscriber list; useSyncExternalStore is React's own contract for exactly that,
gets concurrent rendering right, and costs no dependency and no idiom in every
component. The store is web/src/api/socket.ts — about 200 lines including the
reconnect and resubscribe logic.
Server-side, messages are serialised once per broadcast and every socket
writes the same Arc<str>; leaderboards are recomputed only for windows that
actually have a subscriber.
Windows are block counts, not durations
Every window on the site — 1h, 6h, 24h, 7d — is a block count (600, 3 600, 14 400, 100 800). The labels are the approximate duration at the measured interval, and the UI says "~".
This is not pedantry. A duration is meaningless while a node is catching up: it imports historical blocks at disk speed, so "the last hour" can contain half a million blocks. The same reasoning is why a measured block interval is only used when the node is at the tip; while syncing, the configured target is used and every hashrate on the page is labelled nominal.
The sparklines under the headline tiles hold the same line. They are bucketed by height, not by time: every point is the same number of blocks, so every point carries the same statistical weight and a stretch where nothing was recorded stays the same width on the axis as one where everything was. A bucket is the window divided by fifty, which is why every window is a multiple of it — a full window is then exactly fifty buckets, and the rolling miner count at the last point is the same number printed in the tile above it.
Their block interval comes from the author's own timestamp inherent rather than from when this observer saw the block, for the same reason the measured interval only counts tip observations: a batch of blocks fetched from history all carry very nearly the same observation time, and dividing by that reads as a chain producing hundreds of blocks a second. Each point is a moving average over three buckets, because the mean of one bucket's gaps still carries enough Poisson noise to draw a chain that looks like it is changing size every few minutes.
Data
Postgres on magrathea.kosherinata.internal, mTLS and passwordless — the host's
own certificate is the credential and pg_ident.conf maps its CN to the role.
There is no password anywhere in this repo and nowhere in the config to put one.
Blocks are keyed on (chain, height), not on the block hash, deliberately: this
is a proof-of-work chain and it reorgs. An upsert on that key means a block
replacing another at the same height overwrites it, so the standings reflect the
canonical chain rather than the union of every fork the observer witnessed.
The database is what lets the site keep promises a rolling window cannot: "how did I do last week", a hashrate line older than the process, and a leaderboard that survives a deploy. On start the daemon replays the window and the held attribution names out of it, so a restart does not serve an empty site.
Build and run
# A throwaway Postgres for the dev loop and for the sqlx query cache.
podman run -d --rm --name bb-pg -e POSTGRES_PASSWORD=dev \
-e POSTGRES_DB=blackbeard -p 55432:5432 docker.io/library/postgres:18-alpine
export DATABASE_URL='postgres://postgres:dev@127.0.0.1:55432/blackbeard'
export TEST_DATABASE_URL="$DATABASE_URL"
cargo fmt --all && cargo clippy --all-targets -- -D warnings && cargo test --all
# The daemon, against a real node. BLACKBEARD_DEV_DATABASE_URL is the
# development-only path that skips mTLS; the systemd unit never sets it.
BLACKBEARD_DEV_DATABASE_URL="$DATABASE_URL" \
cargo run -p blackbeard-api -- --config dev-config.toml
cd web && pnpm install && pnpm dev # proxies /v1 to 127.0.0.1:25864
cargo test skips the database integration tests when TEST_DATABASE_URL is
unset, so it stays useful on a machine with no Postgres.
Before adding a chain, probe it. Every failure the probe reports is one the
deployed daemon would hit silently — an RPC without the QPoW runtime API gives a
site with no hashrate, and headers without a pow_ digest give an empty
leaderboard, both of which look like a working deployment:
cargo run -p blackbeard-cli -- probe --rpc-url http://bob.hanzalova.internal:9944
Deploy
CI-driven (architecture/deployment-gitea-actions.md): push to main, or run
the workflow from the Actions UI. Hosts, ports and paths live in
.gitea/workflows/deploy.yaml and nowhere else.
One-time host provisioning is operator-run, from a workstation with full sudo:
./script/infra-setup.sh --pubkey ~/.ssh/id_gitea_ci.pub
Re-run it whenever the deploy gains a new file to ship — each deploy job preflights the target's sudoers against the grants in that script and fails up front naming what is missing, rather than dying partway through an rsync.
Two things it deliberately does not automate, both because they touch shared
infrastructure: the public Let's Encrypt certificate (external-tls.md) and the
split-horizon blackbeard.internal record, which must be added to both site
routers' Unbound or it NXDOMAINs everywhere but one site.
| Port | What | Where |
|---|---|---|
25864 |
blackbeard-api REST + WebSocket |
bob (hanzalova), mesh address only, plain HTTP behind nginx |
The edge proxy is oolon (kosherinata) and the API is on bob
(hanzalova), so /v1 is a cross-site hop over the mesh. That is deliberate —
the API has to sit beside the node whose loopback RPC it reads — and it is the
one hop a loopback health probe cannot see, so the deploy checks it explicitly.
Registered in architecture/port-allocations.md §5; derived from the service
name per §3.
Mainnet
Quantus mainnet is not yet live. It is a commented-out [[chains]] block in
asset/config/config.toml.tmpl, waiting on a published chain spec and an
endpoint.
A chain may be configured before it launches: the observer reports it as
awaiting, the UI says so plainly, and it comes alive on its own the moment the
node starts answering. No redeploy, no restart. Adding it is a config change and
a probe run.
Deviations from house convention
runs-on: infrafor the deploy jobs, notfedora-43. The targets are mesh-only.internalnames and the fedora runners have no route to the mesh. Same reasoning aslair/quantusandlair/mail.- Dark theme only.
generic.mdsays nothing about themes, but the dataviz conventions expect a selected dark mode alongside light. This site commits to one look: it is a scoreboard for a proof-of-work chain, and a light variant would be a second palette to validate for a context this content does not have. BLACKBEARD_DEV_DATABASE_URLbypasses the mandated mTLS Postgres connection. It exists socargo runworks on a workstation with nopg_identmapping on the fleet cluster. It is an environment variable rather than a config key precisely so it cannot be reached by editing a deployed config, and taking it logs a warning loud enough to spot in the journal.
What this cannot tell you
- Hashrate figures are estimates, not measurements. A miner's implied hashrate is their share of blocks won over a finite window times the network estimate. Winning blocks is a Poisson process: a miner holding 1% of the network will quite ordinarily show anywhere from 2 to 11 blocks in a 600-block window. Short windows flatter the lucky and libel the unlucky.
- Distinct miners is a lower bound. One operator, several preimages.
- Orphan and stale-work loss is not measured. Answering "do I win the share
of blocks my hashrate predicts?" honestly needs a hashrate the observer cannot
see.
lair/quantusissue #1 tracks the same gap from the exporter side. - A telemetry name is an inference. See Names above.
- History has holes, and shows them. Difficulty is a state read, so a pruning node cannot say what it was at a height fetched long after the fact, and a stretch the observer was not running for was never recorded at all. Both are gaps in the sparkline rather than a line drawn across them.
Related
lair/quantus— the node and miner deployment, the arena exporter these mechanics come from, and the measured hashrate numbers behind the fleet.~/git/architecture— the house conventions this project follows.