main
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
0b3c1b2e1b
|
feat(telemetry): decode the whole of NodeDetails, and locate nodes by country
We were reading two fields of eleven — the name and the peer id — and dropping the rest, including the hardware every node already announces. `NodeMetadata` now carries the slow-moving half: implementation, version, the target triple, CPU, cores, memory, kernel, distro, whether it is virtualised, and the process start time. The volatile half stays out on purpose. Peer count, transaction queue, bandwidth and state cache change every few seconds, are already a rolling series in the feed, and would be a lot of rows for something nobody would page back through. `startup_time` is kept rather than derived, so node uptime becomes a fact about the node rather than about how long this observer has been watching it. **Location is resolved to a country and no finer.** The feed sends `[latitude, longitude, city]` and no country, so `blackbeard_core::geo` derives one offline from a 508 KiB boundary set — no network call, and no third party told where these nodes are. The city is read and discarded: those coordinates come from an IP geolocation database and at city resolution are wrong often enough that publishing one would assert something we do not know. Nothing downstream can render a city because nothing downstream is given one. The lookup returns subdivisions before countries, and `GB-ENG` is exactly the extra precision this is avoiding, so only the two-letter code survives. `LocatedNode` is a separate frame because the telemetry server geolocates asynchronously — `AddedNode` almost always carries a null location and the country follows a moment later. Which means a reconnect, which re-sends every node with a null location again, must not erase a country already resolved. There is a test for that, because it would present as countries silently vanishing hours after a restart. The mapping is pinned by tests built from frames captured off the live feed rather than from a spec: the format is positional and differs between telemetry releases, so bytes the feed actually sent are the only honest fixture. If a future release renumbers a field, those tests are what says so. Refs #15 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jp6a8EDar9ueEhAxzep4V5 |
|||
|
4fb856827c
|
fix: a bogus length prefix aborted the daemon, and the gap fill never resumed
Two faults, one visible symptom: both testnet leaderboards frozen for a day at the height their gap opened at, while mainnet was fine. The daemon had aborted 316 times in ten hours, always identically — `memory allocation of 82014765760 bytes failed`, 76.4 GiB. That is 1,025,184,572 x 80, and 80 is `size_of::<scale_value::Value<u32>>()`: scale-value sizes a sequence's `Vec` from the compact length in the blob before decoding a single item, so a blob that disagrees with the registry asks for an allocation of any size at all and Rust aborts on the failed one. The step that would have caught the mismatch is the one that never runs, which is why `decode_events(&blob).ok()` could not help — there was no `Err`, only SIGABRT. Every decode now goes through `Runtime::decode_checked`, which walks the bytes first with scale-decode's `IgnoreVisitor`. That crate has no `with_capacity` anywhere, so an impossible length runs out of input on the first item and comes back as an error. The regression test is the exact four-byte blob, reproduced against the mainnet fixture before the guard went in — the abort message matched production byte for byte. `fill_gap` then turned a short process lifetime into no progress whatsoever: it accumulated the whole gap and wrote once after the loop, so every interruption discarded everything read. Each block is four RPC round trips, one a historical state read; 1,696 blocks against a remote endpoint does not fit in the two minutes the aborts allowed. And this is not only about the aborts — `blackbeard-api-cert.path` restarts the service several times a day, so any gap wider than that interval was already unfillable, logging `filling a gap in the head stream` on every start with the same `from` and looking like progress. It flushes every 256 blocks now, and says how many it recorded. Both traps are in CLAUDE.md, because neither looks broken from the outside: one presents as a service that is merely restarting, the other as a catch-up that is merely slow. Closes #10 Closes #11 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jp6a8EDar9ueEhAxzep4V5 |
|||
|
6b0e02e059
|
feat: read chain state, decoded against the runtime
The observer could read headers, events, extrinsics, metadata and difficulty, and not a single pallet storage item. Answering "which address is this chain's treasury" meant going outside it and hand-deriving twox128 prefixes, which is a gap in the tool rather than an answer. The treasury is the case that shows why. `set_treasury_account` reads *never* on the call index and `TreasuryAccountUpdated` reads *never* on the event index — both true, because the address was set at genesis, so no extrinsic ever carried it and no event ever announced it. It exists only in state. It is qzjsuLN7Nhu4bjvmUbjSTr2ZTeZ7oRxXpQP9fdv6PcHUCRrVR, and it has never been funded. `Runtime::storage_key` builds a key entirely from the runtime's own description — the pallet's storage prefix, the item name, and each key's declared hasher — because a wrong hasher yields a key that reads as *absent* rather than as an error, and nothing would catch it. Its test pins two keys against ones read off the live chain by hand. Keys are SCALE-encoded against the type the entry declares, so a map on an account, a u32 or a tuple all work without this knowing which it is. Absent is not zero, and only the modifier knows which: an `optional` entry holding nothing means nothing, a `default` entry means the runtime's default, and the state page marks the latter rather than passing it off as something the chain wrote. `read_balance` deliberately does not apply the default — `AccountInfo` zeroes, Substrate reaps empty accounts, and falling back would turn "does not exist" into "holds nothing", which is the treasury's exact case. /:chain/state lists all 40 keyless entries decoded, and an account page now carries a real balance beside the flows it already summed. Those are different numbers: rewards say what an account was paid, a balance says what it has, and they differ by every transfer out. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jp6a8EDar9ueEhAxzep4V5 |
|||
|
edce856263
|
feat: decode extrinsics, and show them
An event is the chain's account of what happened; an extrinsic is the request that caused it, and carries what no event does — who signed, what they called, the nonce and tip, and whether it worked. A history built from events alone omits every failed attempt and never names a submitter. The same oracle does both. The metadata's extrinsic type carries four parameters and the registry names all of them, including `qp_dilithium_crypto::types::DilithiumSignatureScheme` — the part that looked like it would need hand-written post-quantum knowledge and does not. The chain describes its own signature scheme, so `decode_extrinsic` reads a Quantus transaction without a line of code that knows Quantus exists. Two facts about this chain's extrinsics, both now in CLAUDE.md. The first byte is not the version: the top two bits are a type tag, and mainnet carries `0x84` (signed, v4) and `0x05` (bare, v5) in the same block while the metadata declares version 4 — check the byte against the metadata and you reject every timestamp inherent on the chain. And a signature is 5.3 KiB, two orders of magnitude larger than the call it authorises; it is decoded to find where the call begins and then discarded, keeping only the scheme's name and the byte count. Both halves of a block index in one pass, off calls already being made. Whether a dispatch succeeded comes from the `ExtrinsicSuccess`/`ExtrinsicFailed` events in that same decoded list — read, used, not stored. The block page grows a table below the facts: call, signer, arguments, and the signature's size. The account page merges extrinsics into the history it already had, ordered by block with the request above the results it caused. `source` is 1 for an extrinsic and 0 for an event so that every part of the sort key descends, which lets the paging cursor be a plain row comparison rather than a mixed-direction one Postgres cannot express. Nested calls are why this had to be generic. A `Utility::batch_all` carries whole calls in its arguments, so a transfer's recipient can be three levels down — and because the decoder collects account ids wherever they appear, that batched transfer shows on the recipient's page tagged *received* even though they signed nothing. Verified on mainnet 12,616: the batch renders both its transfers with destinations and values, above the two `Balances::Transfer` events they produced. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jp6a8EDar9ueEhAxzep4V5 |
|||
|
eb26a7bb3c
|
feat: decode events against the runtime's own metadata
A header says who mined a block, not what they were paid: the amount is remaining supply over an emission divisor plus fees, quantized with dust carried forward, so only the `MinerRewarded` event knows it. Reading it means decoding SCALE against the registry of the runtime that produced the block, and this chain's shapes differ from vanilla Substrate because of the post-quantum signatures. So the chain describes itself. `state_getMetadata` at a block hash executes `Metadata_metadata` against the runtime WASM in that block's state; the v14 registry that comes back drives a decoder that knows nothing about mining, rewards or transfers. `INDEXED_EVENTS` is the only place a name appears — adding transfers or anything the chain grows next is one line and a query, not a migration, a struct and a decoder. Metadata is cached per spec_version, because a node that starts pruning would otherwise make history undecodable. Blocks are tried against the tip's runtime first, which costs no extra call; `decode_events` refuses a partial read, so a block from before an upgrade fails loudly rather than decoding into plausible nonsense, and that failure is what asks which runtime actually produced it. `backfill` walks outward from the tip — to the tip first, then toward genesis — because live indexing alone leaves holes an account page would show: a restart, an RPC blip, a gap fill that ran before the metadata was cached. Progress is a cursor rather than `max(height)`, since a block with no indexed event and a block never read are otherwise the same answer. Verified on mainnet: block 13160's preimage derives, through the same Poseidon2 the chain runs, to exactly the `miner` field of its own reward event — so the header author and the payee join with no lookup table. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jp6a8EDar9ueEhAxzep4V5 |
|||
|
a3eb670d11
|
feat: identify miners by the address they are paid at, not the preimage
A reward preimage is exact and unrecognisable — no miner knows theirs by sight. The address is the string they configured and the one their balance shows against, and the two are related by a derivation the chain performs anyway: `pallets/mining-rewards` pays `qp_wormhole::derive_wormhole_address`, which is `qp_poseidon_core::rehash_to_bytes`. Running the same function reproduces the address exactly, with no RPC and no lookup table. The derivation takes the preimage and nothing else — no genesis, no chain parameters — so one preimage is one account on every Quantus chain. The unit test pins it against a real pair (this observer's own node and the address it was paid at), so a dependency upgrade that changed the derivation fails the build rather than quietly relabelling every miner on the site. Unnamed miners now render as their address. The miner panel shows all three identifiers, because they answer different questions: the node name is an inference and may be absent, the address is what the miner recognises, and the preimage is what the header actually carries and the only one the chain asserts. SS58 is written out rather than pulled from `sp-core`, which would bring a substrate runtime's worth of dependencies to format 36 bytes — the same trade the readme records for declining subxt. Base58 is hand-rolled because it is long division and nothing else; blake2 is not, because checksums are not worth hand-rolling. Both are covered by the address test vector. `RecentBlock` gains the attribution source: the ticker cannot tell a node name from an address by looking, since `quanpool-payout-mainnet` and `quantus-radar` both begin with the letter Quantus addresses do. Two tests that asserted the old display policy now assert the new one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jp6a8EDar9ueEhAxzep4V5 |
|||
|
110fbc3631
|
feat: blackbeard.observer — live Quantus mining leaderboard
Cargo workspace plus a Vite frontend, following ~/git/architecture/generic.md. Every block header carries its author's wormhole reward preimage in a `pow_` PreRuntime digest, so authorship for the whole network is derivable from headers alone — no indexer, no registration, no way for a miner to be left out. That decoding, the hashrate maths and the telemetry name attribution live in blackbeard-core with no I/O at all, so the parts that are easy to get subtly wrong are exercised by unit tests rather than only against a live chain. The browser holds one WebSocket: snapshot on subscribe, deltas thereafter. The head stream is itself a push (chain_subscribeNewHeads), so a block reaches the page the moment the node imports it. Messages are serialised once per broadcast, and leaderboards are recomputed only for windows a socket is actually watching. No RxJS — useSyncExternalStore is React's own contract for this. Verified against the live Planck testnet: 12/12 headers decoded, telemetry names attributed (quanpool-planck, baba-gorchitsa, …), warm start restoring 84 blocks and 5 held names across a restart. Three findings worth recording, all in CLAUDE.md: - substrate-telemetry sends its JSON in *binary* frames. A text-only client connects, subscribes, reports healthy and receives nothing at all — and a Python probe hides it, because json.loads accepts bytes. - Difficulty is a little-endian U512; decoding it big-endian gives a number wrong by ~10^150 that still renders fine. - Planck's real block interval is ~13-15s against a 6s target with enormous variance, so a measured interval needs 20 tip samples before it is publishable. Deploy assets, the Gitea Actions workflow and script/infra-setup.sh are included; port 25864 is registered in architecture/port-allocations.md. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MSDYiibCtELsrjQq6KXnoi |
