Cache the node metadata the telemetry feed already sends, and show it on the miner route #15

New Issue

grenade · 2026-09-13T13:58:55Z

grenade commented

2026-09-13 13:58:55 +00:00

The feed carries far more about a node than we read. We decode two fields of eleven from NodeDetails — the name and the peer id — and drop the rest on the floor, including the hardware the miner is actually running on.

The shapes, mapped from the live feed

Captured from wss://feed-telemetry.quantus.cat/feed subscribed to mainnet, 75 seconds, 287 AddedNode frames. Recording it here because the substrate-telemetry wire format is positional, undocumented in any version-specific way, and differs between releases — this is what this feed sends today.

Action 3, AddedNode — arity 8:

[0] node_id       0
[1] details       (see below)
[2] stats         [peer_count, transactions_in_queue]
[3] io            [[state_cache_size samples…]]
[4] hardware      [[upload…], [download…], [timestamps…]]
[5] block         [height, hash, block_time_ms, timestamp_ms, propagation_ms]
[6] location      null | [latitude, longitude, city]
[7] startup_time  1789225142469   (ms since epoch)

details — arity 11, and the interesting one:

[0]  name             "Wallace-Vast-ODB-G"
[1]  implementation   "Quantus Node"
[2]  version          "1.0.1-f1176cea6a6"
[3]  validator        null
[4]  network_id       "QmcjjPmnyM6KYXp1U4Rdy6v2stskYQnTpgk4FH1cMfhMpB"   <- the only other field we read
[5]  target_os        "linux"
[6]  target_arch      "x86_64"
[7]  target_env       "gnu"
[8]  (null in every sample)
[9]  sysinfo          object, below
[10] (null in every sample)

sysinfo is the one object in the whole format — keys stable across every sample:

{
  "cpu": "AMD Ryzen 5 5600 6-Core Processor",
  "memory": 65192095744,
  "core_count": 6,
  "linux_kernel": "7.0.0-31-generic",
  "linux_distro": "Ubuntu 24.04.4 LTS",
  "is_virtual_machine": false
}

The update messages, which carry the volatile half:

5   LocatedNode      [node_id, lat, long, city]
8   NodeStatsUpdate  [node_id, [peer_count, transactions_in_queue]]
9   NodeHardware     [node_id, [[upload…], [download…], [timestamps…]]]
21  NodeIOUpdate     [node_id, [[state_cache_size…]]]
23  (feed metadata)  {"git_hash": ""}

Frame counts over those 75 seconds: 3×287, 6×3832, 7×4138, 8×750, 9×310, 21×310. Volume is dominated by block imports, which we already handle; the metadata frames are cheap.

What to cache

The slow-moving half is what belongs in Postgres, keyed by network_id (the peer id), overwritten whenever a newer frame arrives: name, implementation, version, target os/arch/env, the whole of sysinfo, location, and startup_time.

The volatile half — peer count, transactions in queue, bandwidth, state cache — should stay in memory and be read live. Persisting a bandwidth series per node would be a lot of rows for something nobody would page back through, and it is already a rolling series in the feed.

startup_time is worth storing rather than deriving: it makes "node uptime" a fact about the node rather than about how long we have been watching, and it survives our restarts.

The part that needs care

A miner is joined to a node by inference, and the hardware inherits that. readme.md is emphatic that an attributed name "is an inference, never a claim the miner made and never an identity check" — the join runs miner preimage → attribution → NodeKey::Peer → this metadata, so every CPU model shown on a miner page rests on the same telemetry voting that gave it its name.

That has to be visible in the UI, not buried. A miner whose attribution is a single lucky vote must not display a CPU as though the chain asserted it. The existing confidence and AttributionSource already travel with the row; this panel has to wear them.

Two shapes that break a naive one-to-one:

one operator may run several nodes under one reward preimage — the standings already disambiguate colliding names, and this join has the same problem one level down
a node reports for whatever preimages it mines, so the mapping is not a key in either direction

Scope

Decode the full NodeDetails + sysinfo + location + startup in telemetry.rs, keyed by peer id.
A migration and a store round-trip, overwrite-on-newer.
A DTO on MinerDetail, ts-rs exported.
A panel on the miner route, carrying the attribution caveat.

Worth doing in that order, with (1) and (2) landing before any UI, because the mapping above is the part most likely to be wrong against a future feed version and the tests should pin it first.

The feed carries far more about a node than we read. We decode two fields of eleven from `NodeDetails` — the name and the peer id — and drop the rest on the floor, including the hardware the miner is actually running on. ## The shapes, mapped from the live feed Captured from `wss://feed-telemetry.quantus.cat/feed` subscribed to mainnet, 75 seconds, 287 `AddedNode` frames. Recording it here because the substrate-telemetry wire format is positional, undocumented in any version-specific way, and differs between releases — this is what *this* feed sends today. **Action 3, `AddedNode`** — arity 8: ``` [0] node_id 0 [1] details (see below) [2] stats [peer_count, transactions_in_queue] [3] io [[state_cache_size samples…]] [4] hardware [[upload…], [download…], [timestamps…]] [5] block [height, hash, block_time_ms, timestamp_ms, propagation_ms] [6] location null | [latitude, longitude, city] [7] startup_time 1789225142469 (ms since epoch) ``` **`details`** — arity 11, and the interesting one: ``` [0] name "Wallace-Vast-ODB-G" [1] implementation "Quantus Node" [2] version "1.0.1-f1176cea6a6" [3] validator null [4] network_id "QmcjjPmnyM6KYXp1U4Rdy6v2stskYQnTpgk4FH1cMfhMpB" <- the only other field we read [5] target_os "linux" [6] target_arch "x86_64" [7] target_env "gnu" [8] (null in every sample) [9] sysinfo object, below [10] (null in every sample) ``` **`sysinfo`** is the one object in the whole format — keys stable across every sample: ```json { "cpu": "AMD Ryzen 5 5600 6-Core Processor", "memory": 65192095744, "core_count": 6, "linux_kernel": "7.0.0-31-generic", "linux_distro": "Ubuntu 24.04.4 LTS", "is_virtual_machine": false } ``` **The update messages**, which carry the volatile half: ``` 5 LocatedNode [node_id, lat, long, city] 8 NodeStatsUpdate [node_id, [peer_count, transactions_in_queue]] 9 NodeHardware [node_id, [[upload…], [download…], [timestamps…]]] 21 NodeIOUpdate [node_id, [[state_cache_size…]]] 23 (feed metadata) {"git_hash": ""} ``` Frame counts over those 75 seconds: `3`×287, `6`×3832, `7`×4138, `8`×750, `9`×310, `21`×310. Volume is dominated by block imports, which we already handle; the metadata frames are cheap. ## What to cache The slow-moving half is what belongs in Postgres, keyed by **`network_id`** (the peer id), overwritten whenever a newer frame arrives: name, implementation, version, target os/arch/env, the whole of `sysinfo`, location, and `startup_time`. The volatile half — peer count, transactions in queue, bandwidth, state cache — should stay in memory and be read live. Persisting a bandwidth series per node would be a lot of rows for something nobody would page back through, and it is already a rolling series in the feed. `startup_time` is worth storing rather than deriving: it makes "node uptime" a fact about the node rather than about how long we have been watching, and it survives our restarts. ## The part that needs care **A miner is joined to a node by inference, and the hardware inherits that.** `readme.md` is emphatic that an attributed name "is an inference, never a claim the miner made and never an identity check" — the join runs miner preimage → attribution → `NodeKey::Peer` → this metadata, so every CPU model shown on a miner page rests on the same telemetry voting that gave it its name. That has to be visible in the UI, not buried. A miner whose attribution is a single lucky vote must not display a CPU as though the chain asserted it. The existing `confidence` and `AttributionSource` already travel with the row; this panel has to wear them. Two shapes that break a naive one-to-one: - one operator may run several nodes under one reward preimage — the standings already disambiguate colliding names, and this join has the same problem one level down - a node reports for whatever preimages it mines, so the mapping is not a key in either direction ## Scope 1. Decode the full `NodeDetails` + `sysinfo` + location + startup in `telemetry.rs`, keyed by peer id. 2. A migration and a store round-trip, overwrite-on-newer. 3. A DTO on `MinerDetail`, ts-rs exported. 4. A panel on the miner route, carrying the attribution caveat. Worth doing in that order, with (1) and (2) landing before any UI, because the mapping above is the part most likely to be wrong against a future feed version and the tests should pin it first.

grenade referenced this issue from a commit

2026-09-13 14:26:33 +00:00

feat(telemetry): decode the whole of NodeDetails, and locate nodes by country

grenade referenced this issue from a commit

2026-09-13 14:26:33 +00:00

feat: show the node a miner's blocks were reported from

grenade closed this issue

2026-09-13 14:26:33 +00:00

Sign in to join this conversation.