blackbeard.observer

Live miner leaderboard and network hashrate for the Quantus blockchain and its Planck testnet, at https://blackbeard.observer (blackbeard.internal from the mesh).

The point of it: a miner can find their own row, see what share of the network they are actually winning, and watch it move — against everyone else, in real time. Mining is a competition that normally gives its participants no scoreboard. This is the scoreboard.

How it knows who mined what

Every block header carries its author's 32-byte wormhole reward preimage in a PreRuntime digest stamped with the engine id pow_. The node writes it there so the mining-rewards pallet can read it back and derive the payout address on-chain — and as a side effect, authorship for every miner on the network is derivable from headers alone.

That is the whole mechanism, and its consequences are worth stating plainly:

• No registration, no opt-in, no way to be left out. Every miner appears the moment they win a block. Nothing here depends on anyone's cooperation.
• The preimage is public, not secret. It is published from a miner's first authored block onward. Spending the rewards needs a plonky2 proof of knowledge of the underlying secret, which the preimage does not reveal — but it does make mining income permanently attributable. The privacy in this design is at the exit, not at receipt. lair/quantus's doc/wormhole-rewards.md has the full analysis.
• A leaderboard row is a preimage, not a person. One operator may run several; the count of distinct preimages is a lower bound on miner count.

Difficulty comes from the QPoWApi_get_difficulty runtime call. Difficulty is the expected number of hashes to win a block, so network hashrate is difficulty divided by the block interval — and a miner's implied hashrate is their share of recent blocks times that.

The mechanics were first worked out in lair/quantus's Prometheus arena exporter (asset/arena/quantus-arena-exporter.py), which remains the reference for anyone checking this implementation against a second one.

Names

The chain gives us preimages; a board of 64-character hex strings is honest and unreadable. substrate-telemetry closes the gap indirectly.

The feed reports block imports with a propagation time, stamping the first reporter of a hash with 0. A node imports its own block before announcing it, so the author — if it is on telemetry at all — is nearly always first. One block proves nothing (measured on Planck, an author's lead over the second reporter is 50–620 ms, which a well-peered bystander can produce), so the observer votes over a rolling window and only shows a name once enough blocks agree.

An attributed name is therefore an inference, never a claim the miner made and never an identity check. The UI marks it as such, carries the confidence in a tooltip, and falls back to the abbreviated preimage rather than guessing. Where two miners resolve to the same node name — one operator with several preimages, or two people who never renamed their node — every member of the colliding group gets a preimage suffix, so no row is silently privileged as "the real one".

Layout

Cargo workspace plus a Vite frontend, per ~/git/architecture/generic.md §1.

crates/
  blackbeard-entities/   domain types + the browser wire protocol; ts-rs exports
                         the TypeScript the frontend consumes. No I/O.
  blackbeard-core/       header/SCALE decoding, hashrate maths, the rolling
                         window, telemetry attribution voting. Pure — no I/O,
                         no clock, no sockets, exercised entirely by unit tests.
  blackbeard-data/       chain JSON-RPC, the telemetry feed, Postgres.
  blackbeard-api/        the daemon: ingest tasks, REST, WebSocket fanout.
  blackbeard-cli/        operator tools: probe, backfill, standings.
web/                     Vite + React + SWC + TS. Static build, served by nginx.
asset/                   systemd, firewalld, nginx, config template, bootstrap SQL.
script/infra-setup.sh    one-time host provisioning, operator-run.

The frontend's types are generated from the Rust. cargo test -p blackbeard-entities writes web/src/api/generated/; CI fails the build if the committed output is stale. Edit the Rust DTO, never the TypeScript — otherwise the two compile cleanly and disagree at runtime.

Real time, without observables

One WebSocket per browser tab. On subscribe the client gets a full snapshot; after that only deltas — a block on every block, a summary when the headline numbers move, a leaderboard when the standings change. The page never polls and never refetches.

The head stream is itself a push: chain_subscribeNewHeads over the node's WebSocket, so the observer learns of a block the moment the node imports it. The only polling anywhere is difficulty and sync state, on a four-second timer.

RxJS was considered and not used. This needs one stream, one reducer and one subscriber list; useSyncExternalStore is React's own contract for exactly that, gets concurrent rendering right, and costs no dependency and no idiom in every component. The store is web/src/api/socket.ts — about 200 lines including the reconnect and resubscribe logic.

Server-side, messages are serialised once per broadcast and every socket writes the same Arc<str>; leaderboards are recomputed only for windows that actually have a subscriber.

Windows are block counts, not durations

Every window on the site — 1h, 6h, 24h, 7d — is a block count (600, 3 600, 14 400, 100 800). The labels are the approximate duration at the measured interval, and the UI says "~".

This is not pedantry. A duration is meaningless while a node is catching up: it imports historical blocks at disk speed, so "the last hour" can contain half a million blocks. The same reasoning is why a measured block interval is only used when the node is at the tip; while syncing, the configured target is used and every hashrate on the page is labelled nominal.

The sparklines under the headline tiles hold the same line. They are bucketed by height, not by time: every point is the same number of blocks, so every point carries the same statistical weight and a stretch where nothing was recorded stays the same width on the axis as one where everything was. A bucket is the window divided by fifty, which is why every window is a multiple of it — a full window is then exactly fifty buckets, and the rolling miner count at the last point is the same number printed in the tile above it.

Their block interval comes from the author's own timestamp inherent rather than from when this observer saw the block, for the same reason the measured interval only counts tip observations: a batch of blocks fetched from history all carry very nearly the same observation time, and dividing by that reads as a chain producing hundreds of blocks a second. Each point is a moving average over three buckets, because the mean of one bucket's gaps still carries enough Poisson noise to draw a chain that looks like it is changing size every few minutes.

A block has two addresses, and only one of them is stable

/quantus/block/1069799 is a position. Because block is keyed on (chain, height) and a reorg overwrites, a link to a height quietly comes to mean a different block the moment the chain forks there. So opening one resolves it and rewrites the address to /quantus/block/0x…, which names one block for good — including after it has lost, which is the only way an orphan is linkable at all.

The page is assembled from both sources because neither is sufficient. The node holds the block and forgets it: blocks-pruning defaults to archive-canonical, so a losing fork's body goes once finality passes it, and difficulty is a state read behind a 256-block window. The observer holds what the node never had — when the block was seen, and whether that sighting was at the tip — and what it has since forgotten, the difficulty recorded while the state still existed. Anything neither can answer says so in words rather than going blank, because a gap on a block page reads as a zero.

A runtime describes itself

/quantus/runtime/152 is every pallet, call, event, error, storage entry and constant the runtime declares — with the constants' values, decoded against their own declared types. Nothing on that page is transcribed from a source tree, which is the distinction worth having: a source tree says what the chain should be running, and metadata says what it ran, at a height, possibly months after it was upgraded past.

The set of runtimes is found rather than waited for. Decoding caches a runtime the moment it needs one, so left alone the cache is "whatever the backfill has walked past" — days, on a chain a million blocks deep. spec_version only increases, so the upgrade boundaries are a sorted sequence and bisection finds each in log₂(height) probes. On Heisenberg that turned up six runtimes and the block each took over at: v126 from 1, v128 from 132, v131 from 342,813, v136 from 669,129, v144 from 812,055, v148 from 977,079.

It runs as its own task, not in the poll loop, and the reason is worth recording: state_getRuntimeVersion at an old block makes the node load and instantiate the runtime WASM out of that block's state, which measured four seconds against Heisenberg's public endpoint against a quarter of a second at the tip. A hundred of those inside a four-second poll loop would stall difficulty, the summary broadcast and the telemetry attach for minutes on every start.

Reading the two ends of Heisenberg's history side by side is the case for the page existing. Between v126 and v148 the chain dropped Referenda, ConvictionVoting, Recovery, Assets and AssetsHolder, added Vesting and Origins, gained a WeightReclaim signed extension and moved ChargeTransactionPayment after the two Quantus-specific ones. Every one of those is a change that breaks a decoder written against the other version, and none of them is discoverable from a block.

Two renderings are not what the registry literally says, both for the same reason and both identified by registry path rather than by shape: AccountId32 becomes 0x hex rather than thirty-two numbers, and U256/U512 become one decimal rather than four or eight little-endian limbs. Difficulty as [1189189, 0, 0, 0, 0, 0, 0, 0] is a correct description of the bytes and tells a reader nothing.

One account, from our own index

/quantus/account/qzp2AxZw… answers what the official explorer does not: what an address has actually been paid, every reward, back as far as the index has read. The address is converted to a raw account id at the edge, so a typo is rejected as malformed rather than answered with an empty history — which reads exactly like a real account that has never been paid — and a Polkadot address is rejected too: same thirty-two bytes, different network, and answering for it would show someone activity that is not theirs.

The activity table has no case per event type. Every chain_event row carries the accounts it mentions in a column the decoder filled — it knows which values are AccountId32 by their registry path, which is also how it avoids rendering a block hash as an address — so accounts @> array[…] finds everything, including from pallets that do not exist yet. The frontend labels and formats what it recognises and shows anything else as its own key and value, so a new event type appears as itself rather than not at all.

The preimage and the address are the two ends of one identity, and each page links to the other: the miner page counts blocks, the account page counts tokens, and the derivation joins them. Going the other way — address to preimage — is a lookup among preimages already seen, not a calculation, because the Poseidon2 derivation runs one way only.

Every view has an address

Paths, not a hash: /quantus/day, /quantus/block/13160, /quantus/miner/0x…. A fragment is never sent to a server, so a hash URL cannot be given a title, cannot be crawled, and cannot be resolved by anything but the page itself — which was a fine trade while the whole surface was a chain and a window, and stopped being one once a block and a miner each became a page worth sending someone. Both vhosts already answer any path with index.html, so nothing outside the client changed. lib/routes holds the grammar, and fromLegacyHash rewrites the old form on load, so every #/… link already published still lands where it meant to.

Route segments are named — /quantus/block/13160 rather than /quantus/13160 — rather than told apart by shape. Shape works right up until two kinds of thing can look alike, and an SS58 address is arbitrary base58 with no rule keeping it clear of a window name forever.

Everything navigable is an <a>, including the chain chips and the window control. That is what "real URLs" has to mean in practice: middle-click opens the 24-hour view of another chain in a tab, right-click copies a link to a miner, and the back button walks the panels the way it walks pages. The one deliberate exception is a height resolving to its hash, which replaces rather than pushes — it is the same block under a better name, not a second place the reader has been.

Data

Postgres on magrathea.kosherinata.internal, mTLS and passwordless — the host's own certificate is the credential and pg_ident.conf maps its CN to the role. There is no password anywhere in this repo and nowhere in the config to put one.

Blocks are keyed on (chain, height), not on the block hash, deliberately: this is a proof-of-work chain and it reorgs. An upsert on that key means a block replacing another at the same height overwrites it, so the standings reflect the canonical chain rather than the union of every fork the observer witnessed.

What that overwrites is copied into block_displacement on the way past, so a contested height leaves a trace of who lost it. The node will not answer for the losing block for long — it keeps only canonical bodies once finality passes them, and difficulty is a state read behind a 256-block window — so the row written at the moment of the reorg is the only record there will be. It is not an orphan store: the block itself is gone, and only what the observer had already seen of it survives.

The database is what lets the site keep promises a rolling window cannot: "how did I do last week", a hashrate line older than the process, and a leaderboard that survives a deploy. On start the daemon replays the window and the held attribution names out of it, so a restart does not serve an empty site.

The runtime is the oracle

A header tells you who mined a block. It does not tell you what they were paid: the amount is remaining supply over an emission divisor plus collected fees, quantized with the dust carried into the next block, so it is only knowable from the MiningRewards::MinerRewarded event the runtime emits. Reading that means decoding SCALE against the type registry of the runtime that produced the block — and this chain's encodings differ from vanilla Substrate, because the post-quantum signature scheme changes the shapes.

Rather than transcribe those shapes into Rust and re-transcribe them after every upgrade, the observer asks the chain to describe itself. state_getMetadata at a block hash makes the node execute Metadata_metadata against the runtime WASM in that block's state, which returns metadata v14: every pallet, every event, every field, as a portable scale-info registry. blackbeard-core::runtime decodes against that registry and knows nothing about mining, rewards or transfers — INDEXED_EVENTS in ingest.rs is the only place a name appears, and adding transfers or multisig or whatever the chain grows next is one line there and a query, not a decoder.

Metadata is cached per spec_version in runtime_metadata, because a node that starts pruning would otherwise make every historical block undecodable. Blocks are decoded against the tip's runtime first — which costs no extra RPC call and is right for all but a handful of blocks — and decode_events refuses a partial read, so a block from before an upgrade fails rather than decoding into plausible nonsense. That failure is what triggers asking which runtime actually produced it. It is the property the whole approach rests on.

Events land in one table, chain_event, in the runtime's own vocabulary: pallet, variant, and the fields as jsonb. Account ids are normalised to 0x hex on the way in, and integers past 2^53 to strings — balances here are u128 and a JSON number would be silently rounded by the browser. A jsonb_path_ops GIN index over the fields is what makes "everything involving this account" a query rather than a table per event type.

ingest::backfill walks the chain outward from the tip: to the tip first, then toward genesis, one contiguous interval recorded in event_scan. Live indexing alone is not enough for an account page — a restart, a brief RPC failure or a gap fill that ran before the metadata was cached each leave a hole that nothing would return to. Progress is a cursor rather than max(height) from chain_event, because a block with no indexed event and a block never looked at are otherwise indistinguishable, and the first is the common case.

A miner's pow_ preimage and their reward event agree without a lookup table: wormhole::reward_address runs the same Poseidon2 derivation the chain does, and its output is exactly the miner field of MinerRewarded. Verified on mainnet block 13160 — preimage 0x02dacf59…, address qzowWAgb…, account 0xc6801725…, which is the event's own field.

Build and run

# A throwaway Postgres for the dev loop and for the sqlx query cache.
podman run -d --rm --name bb-pg -e POSTGRES_PASSWORD=dev \
  -e POSTGRES_DB=blackbeard -p 55432:5432 docker.io/library/postgres:18-alpine

export DATABASE_URL='postgres://postgres:dev@127.0.0.1:55432/blackbeard'
export TEST_DATABASE_URL="$DATABASE_URL"

cargo fmt --all && cargo clippy --all-targets -- -D warnings && cargo test --all

# The daemon, against a real node. BLACKBEARD_DEV_DATABASE_URL is the
# development-only path that skips mTLS; the systemd unit never sets it.
BLACKBEARD_DEV_DATABASE_URL="$DATABASE_URL" \
  cargo run -p blackbeard-api -- --config dev-config.toml

cd web && pnpm install && pnpm dev     # proxies /v1 to 127.0.0.1:25864

cargo test skips the database integration tests when TEST_DATABASE_URL is unset, so it stays useful on a machine with no Postgres.

Before adding a chain, probe it. Every failure the probe reports is one the deployed daemon would hit silently — an RPC without the QPoW runtime API gives a site with no hashrate, and headers without a pow_ digest give an empty leaderboard, both of which look like a working deployment:

cargo run -p blackbeard-cli -- probe --rpc-url http://bob.hanzalova.internal:9944

Deploy

CI-driven (architecture/deployment-gitea-actions.md): push to main, or run the workflow from the Actions UI. Hosts, ports and paths live in .gitea/workflows/deploy.yaml and nowhere else.

One-time host provisioning is operator-run, from a workstation with full sudo:

./script/infra-setup.sh --pubkey ~/.ssh/id_gitea_ci.pub

Re-run it whenever the deploy gains a new file to ship — each deploy job preflights the target's sudoers against the grants in that script and fails up front naming what is missing, rather than dying partway through an rsync.

Two things it deliberately does not automate, both because they touch shared infrastructure: the public Let's Encrypt certificate (external-tls.md) and the split-horizon blackbeard.internal record, which must be added to both site routers' Unbound or it NXDOMAINs everywhere but one site.

Port	What	Where
25864	blackbeard-api REST + WebSocket	bob (hanzalova), mesh address only, plain HTTP behind nginx

The edge proxy is oolon (kosherinata) and the API is on bob (hanzalova), so /v1 is a cross-site hop over the mesh. That is deliberate — the API has to sit beside the node whose loopback RPC it reads — and it is the one hop a loopback health probe cannot see, so the deploy checks it explicitly.

Registered in architecture/port-allocations.md §5; derived from the service name per §3.

Mainnet

Quantus mainnet is not yet live. It is a commented-out [[chains]] block in asset/config/config.toml.tmpl, waiting on a published chain spec and an endpoint.

A chain may be configured before it launches: the observer reports it as awaiting, the UI says so plainly, and it comes alive on its own the moment the node starts answering. No redeploy, no restart. Adding it is a config change and a probe run.

Deviations from house convention

• runs-on: infra for the deploy jobs, not fedora-43. The targets are mesh-only .internal names and the fedora runners have no route to the mesh. Same reasoning as lair/quantus and lair/mail.
• Dark theme only. generic.md says nothing about themes, but the dataviz conventions expect a selected dark mode alongside light. This site commits to one look: it is a scoreboard for a proof-of-work chain, and a light variant would be a second palette to validate for a context this content does not have.
• BLACKBEARD_DEV_DATABASE_URL bypasses the mandated mTLS Postgres connection. It exists so cargo run works on a workstation with no pg_ident mapping on the fleet cluster. It is an environment variable rather than a config key precisely so it cannot be reached by editing a deployed config, and taking it logs a warning loud enough to spot in the journal.

What this cannot tell you

• Hashrate figures are estimates, not measurements. A miner's implied hashrate is their share of blocks won over a finite window times the network estimate. Winning blocks is a Poisson process: a miner holding 1% of the network will quite ordinarily show anywhere from 2 to 11 blocks in a 600-block window. Short windows flatter the lucky and libel the unlucky.
• Distinct miners is a lower bound. One operator, several preimages.
• Orphan and stale-work loss is not measured. Answering "do I win the share of blocks my hashrate predicts?" honestly needs a hashrate the observer cannot see. lair/quantus issue #1 tracks the same gap from the exporter side.
• A telemetry name is an inference. See Names above.
• History has holes, and shows them. Difficulty is a state read, so a pruning node cannot say what it was at a height fetched long after the fact, and a stretch the observer was not running for was never recorded at all. Both are gaps in the sparkline rather than a line drawn across them.

Related

• lair/quantus — the node and miner deployment, the arena exporter these mechanics come from, and the measured hashrate numbers behind the fleet.
• ~/git/architecture — the house conventions this project follows.