feat: a network page — supply, capability and daily activity
deploy / build (push) Successful in 7m27s
deploy / deploy-web (push) Successful in 5s
deploy / deploy-api (push) Successful in 17s

The site answered "who is mining" thoroughly and "what is this economy doing"
not at all. Everything needed was already reachable and none of it was
assembled anywhere.

Measured against mainnet rather than assumed:

    total issuance      5,682,913.17 QTC
    endowed at genesis  5,670,000.00 QTC
    mined since          12,913.17 QTC   <- emission to date, printed nowhere else
    accounts                    1,925    counted, not inferred from indexed events
    locks/holds/freezes/reserves    0    all four, counted
    vesting schedules              48
    referenda                       0
    reversible transfers            0
    calls used                 7 of 58
    events fired              18 of 107

**No "circulating supply" field, deliberately.** With every immobilising map
empty it would equal total issuance exactly, and printing it as a separate
headline would assert a distinction this chain does not currently make. The page
states what was counted and lets that be read. Vesting is a count rather than a
sum for the same reason: this runtime's vesting does not touch the balances
locks, so what it holds and when it releases would be a guess.

**Signed extrinsics are separated from the total**, because three quarters of
this chain's extrinsics are inherents — 513 signed of 2,052 on the day measured
— and a single "transactions per day" line would be mostly clockwork, a number
that reads as adoption and is not.

Two kinds of certainty share the page and are kept apart. Supply and the map
sizes are chain state, read now. The daily activity is our index, and the
Activity panel prints the range it covered and says "the whole chain" or "a
partial index" — a chart whose axis claims a month over a week of data is the
failure this repository has already shipped twice.

One thing found while building it: a counter that has never moved is *absent*
from storage, not zero. `TechReferenda::ReferendumCount` and
`ReversibleTransfers::NextTransactionId` both read as nothing, and reporting
them unknown would have hidden the most interesting fact about them — that
governance and reversible transfers are shipped and have never been used.
`plain_value` falls back to the entry's declared default, which is only sound
because `StorageTarget` says whether it is `Default` or `Optional`.

Refs #17

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jp6a8EDar9ueEhAxzep4V5
This commit is contained in:
2026-09-14 08:43:13 +03:00
co-authored by Claude Opus 5
parent 3551b832a0
commit aaf37baaca
15 changed files with 1187 additions and 7 deletions
+274 -5
View File
@@ -18,11 +18,11 @@ use axum::{Json, Router};
use blackbeard_entities::{
AccountDetail, AccountEvent, AccountRoleEntry, AccountRow, ActivitySource, ApiError,
BigUintDec, BlockDetail, CallIndex, CallSummary, ChainInfo, ChainRoles, ChainSeries,
ChainState, ChainSummary, EventSummary, GenesisDetail, LeaderboardRow, MinerDetail, MinerId,
MinerSeriesPoint, NamedAccount, NodeActivity, NodeIndex, NodeInfo, NodeRow, PendingTransfer,
RecentBlock, ReversibleState, RewardSummary, RoleSource, RuntimeConstant, RuntimeDetail,
RuntimeField, RuntimePallet, RuntimeSignedExtension, RuntimeStorage, RuntimeSummary,
RuntimeVariant, StateEntry, Window,
ChainState, ChainSummary, DailyActivity, EventSummary, GenesisDetail, LeaderboardRow,
LockCounts, MinerDetail, MinerId, MinerSeriesPoint, NamedAccount, NetworkSummary, NodeActivity,
NodeIndex, NodeInfo, NodeRow, PendingTransfer, RecentBlock, ReversibleState, RewardSummary,
RoleSource, RuntimeConstant, RuntimeDetail, RuntimeField, RuntimePallet,
RuntimeSignedExtension, RuntimeStorage, RuntimeSummary, RuntimeVariant, StateEntry, Window,
};
use serde::{Deserialize, Serialize};
use tower_http::compression::CompressionLayer;
@@ -55,6 +55,7 @@ pub fn router(state: AppState, allowed_origins: &[String]) -> Router {
.route("/v1/chains/{chain}/series", get(series))
.route("/v1/chains/{chain}/blocks/{block}", get(block))
.route("/v1/chains/{chain}/miners/{miner}", get(miner))
.route("/v1/chains/{chain}/network", get(network))
.route("/v1/chains/{chain}/nodes", get(nodes))
.route("/v1/chains/{chain}/nodes/{peer}", get(node))
.route("/v1/chains/{chain}/accounts", get(accounts))
@@ -783,6 +784,274 @@ async fn miner(
}))
}
/// Days of activity the network page reports.
const NETWORK_DAYS: i32 = 30;
/// Ceiling on a map walk behind the network page.
///
/// `System::Account` is 1,914 on mainnet and the walk is two round trips. This
/// stops a chain nobody expected to be large from turning one page render into
/// a thousand requests; the count comes back flagged incomplete instead.
const MAP_WALK_CEILING: usize = 50_000;
/// Read one plain storage item and hand back its decoded value.
///
/// `None` for an item the runtime does not declare, a node that has pruned it,
/// or a value that will not decode — all absences, and all rendered as unknown
/// rather than as zero. A zero here would be a claim about the chain.
async fn plain_value(
runtime: &crate::state::ChainRuntime,
parsed: &blackbeard_core::runtime::Runtime,
pallet: &str,
item: &str,
) -> Option<serde_json::Value> {
let target = parsed.storage_key(pallet, item, &[]).ok()?;
let key = format!("0x{}", hex::encode(&target.key));
// **An absent `Default` entry decodes to its default, not to nothing.** The
// runtime writes a key only when something changes it, so a counter that
// has never moved is simply not there: `TechReferenda::ReferendumCount` and
// `ReversibleTransfers::NextTransactionId` both read absent on mainnet, and
// reporting them unknown would hide the most interesting fact about them —
// that governance and reversible transfers are shipped and never used. Zero
// is the answer; absence is how the chain says it.
//
// An `Optional` entry carries no default, and there nothing does mean
// nothing. `StorageTarget` is what tells the two apart.
let raw = match runtime.rpc.storage(&key, None).await.ok()? {
Some(raw) => raw,
None => target.default.clone()?,
};
parsed
.decode_typed(target.value_ty, &raw)
.ok()
.map(|(v, _)| v)
}
/// A decoded plain value as a decimal string, for the balance-shaped items.
fn as_decimal(value: Option<serde_json::Value>) -> Option<BigUintDec> {
match value? {
serde_json::Value::String(s) if s.chars().all(|c| c.is_ascii_digit()) => {
Some(BigUintDec(s))
}
serde_json::Value::Number(n) => Some(BigUintDec(n.to_string())),
_ => None,
}
}
/// The same, as a `u64`, for the counters.
fn as_count(value: Option<serde_json::Value>) -> Option<u64> {
match value? {
serde_json::Value::Number(n) => n.as_u64(),
serde_json::Value::String(s) => s.parse().ok(),
_ => None,
}
}
/// Count the keys of one map, by name.
async fn map_size(
runtime: &crate::state::ChainRuntime,
parsed: &blackbeard_core::runtime::Runtime,
pallet: &str,
item: &str,
) -> Option<(u32, bool)> {
let map = parsed.storage_map(pallet, item).ok()?;
let prefix = format!("0x{}", hex::encode(&map.prefix));
let (count, complete) = runtime
.rpc
.count_keys(&prefix, MAP_WALK_CEILING)
.await
.ok()?;
Some((count as u32, complete))
}
/// `GET /v1/chains/{chain}/network`
///
/// The figures an analyst needs in one place. Two kinds of number, and they are
/// not equally certain: the supply and the map sizes are chain state, read now;
/// the daily activity is this observer's index and is only as complete as the
/// range it reports beside itself.
async fn network(
State(state): State<AppState>,
Path(chain): Path<String>,
) -> Result<Json<NetworkSummary>, Failure> {
let runtime = state.chain(&chain).ok_or_else(|| unknown_chain(&chain))?;
let id = runtime.id();
let (parsed, spec_version, height) = {
let inner = runtime.read();
(inner.current_runtime(), inner.spec_version, inner.height)
};
let days = state
.store
.activity_by_day(&id, NETWORK_DAYS)
.await
.map_err(database_unavailable)?
.into_iter()
.map(|d| DailyActivity {
day: d.day,
blocks: d.blocks,
miners: d.miners,
extrinsics: d.extrinsics,
signed: d.signed,
signers: d.signers,
events: d.events,
new_accounts: d.new_accounts,
})
.collect();
let scan = state
.store
.event_scan(&id)
.await
.map_err(database_unavailable)?;
let Some(parsed) = parsed else {
// No metadata: the activity still stands, because it comes from the
// database rather than the chain. The supply does not, and an absent
// figure is better than a zero.
return Ok(Json(NetworkSummary {
chain: id,
spec_version,
total_issuance: None,
inactive_issuance: None,
genesis_endowment: None,
mined_since_genesis: None,
collected_fees: None,
accounts: None,
accounts_complete: false,
locks: None,
vesting_schedules: None,
referenda: None,
reversible_transfers: None,
wormhole_leaves: None,
calls_used: 0,
calls_declared: 0,
events_fired: 0,
events_declared: 0,
indexed_from: scan.map(|s| s.0),
indexed_to: scan.map(|s| s.1),
height,
days,
}));
};
let total_issuance =
as_decimal(plain_value(&runtime, &parsed, "Balances", "TotalIssuance").await);
let inactive_issuance =
as_decimal(plain_value(&runtime, &parsed, "Balances", "InactiveIssuance").await);
let collected_fees =
as_decimal(plain_value(&runtime, &parsed, "MiningRewards", "CollectedFees").await);
let vesting_schedules = map_size(&runtime, &parsed, "Vesting", "Schedules")
.await
.map(|(n, _)| n);
let referenda =
as_count(plain_value(&runtime, &parsed, "TechReferenda", "ReferendumCount").await)
.map(|n| n as u32);
let reversible_transfers = as_count(
plain_value(
&runtime,
&parsed,
"ReversibleTransfers",
"NextTransactionId",
)
.await,
)
.map(|n| n as u32);
let wormhole_leaves = as_count(plain_value(&runtime, &parsed, "ZkTree", "LeafCount").await);
let (accounts, accounts_complete) = match map_size(&runtime, &parsed, "System", "Account").await
{
Some((n, complete)) => (Some(n), complete),
None => (None, false),
};
// All four, because all four are ways supply can be immobile and the useful
// statement on this chain is that every one of them is empty. Counted, not
// assumed.
let locks = match (
map_size(&runtime, &parsed, "Balances", "Locks").await,
map_size(&runtime, &parsed, "Balances", "Holds").await,
map_size(&runtime, &parsed, "Balances", "Freezes").await,
map_size(&runtime, &parsed, "Balances", "Reserves").await,
) {
(Some(l), Some(h), Some(f), Some(r)) => Some(LockCounts {
locks: l.0,
holds: h.0,
freezes: f.0,
reserves: r.0,
}),
_ => None,
};
// What genesis handed out. The same walk `/genesis` makes, and the reason
// emission-to-date can be stated at all: issuance alone cannot distinguish
// what was minted from what was granted.
let (endowed, _) = genesis_accounts(&runtime, &parsed).await;
let genesis_endowment = endowed
.iter()
.filter_map(|(_, balance)| balance.parse::<u128>().ok())
.try_fold(0u128, |sum, b| sum.checked_add(b))
.map(|total| BigUintDec(total.to_string()));
let mined_since_genesis = match (&total_issuance, &genesis_endowment) {
(Some(issued), Some(endowed)) => issued
.0
.parse::<u128>()
.ok()
.zip(endowed.0.parse::<u128>().ok())
// Saturating rather than wrapping: an endowment larger than
// issuance would mean tokens were burned, and a huge positive
// number is a worse answer than zero.
.map(|(i, e)| BigUintDec(i.saturating_sub(e).to_string())),
_ => None,
};
let described = parsed.describe();
let calls_declared: u32 = described.pallets.iter().map(|p| p.calls.len() as u32).sum();
let events_declared: u32 = described
.pallets
.iter()
.map(|p| p.events.len() as u32)
.sum();
let calls_used = state
.store
.call_usage(&id)
.await
.map(|u| u.len() as u32)
.unwrap_or(0);
let events_fired = state
.store
.event_usage(&id)
.await
.map(|u| u.len() as u32)
.unwrap_or(0);
Ok(Json(NetworkSummary {
chain: id,
spec_version,
total_issuance,
inactive_issuance,
genesis_endowment,
mined_since_genesis,
collected_fees,
accounts,
accounts_complete,
locks,
vesting_schedules,
referenda,
reversible_transfers,
wormhole_leaves,
calls_used,
calls_declared,
events_fired,
events_declared,
indexed_from: scan.map(|s| s.0),
indexed_to: scan.map(|s| s.1),
height,
days,
}))
}
/// How many cached nodes a chain's index will reach back for.
///
/// Mainnet telemetry reports under three hundred nodes and the testnets fewer,
+33
View File
@@ -515,6 +515,39 @@ impl RpcClient {
.unwrap_or_default())
}
/// How many keys live under a prefix.
///
/// Pages through them and counts, because Substrate offers no `count` and a
/// map's size is a real fact about a chain — how many accounts exist, how
/// many balances are locked. At the node's 1,000-key ceiling, mainnet's
/// 1,914 accounts is two round trips.
///
/// `stop_at` bounds the walk so a map nobody expected to be large cannot
/// turn a page render into a thousand requests; the count is returned with
/// a flag saying whether it is complete.
pub async fn count_keys(
&self,
prefix: &str,
stop_at: usize,
) -> Result<(usize, bool), DataError> {
let mut total = 0usize;
let mut start: Option<String> = None;
loop {
let page = self
.storage_keys_paged(prefix, MAX_KEYS_PER_PAGE, start.as_deref(), None)
.await?;
let got = page.len();
total += got;
if got < MAX_KEYS_PER_PAGE as usize {
return Ok((total, true));
}
if total >= stop_at {
return Ok((total, false));
}
start = page.last().cloned();
}
}
/// The runtime's own description of itself, as of `hash`.
///
/// The node executes `Metadata_metadata` against the runtime code in that
+128
View File
@@ -60,6 +60,32 @@ pub struct StoreConfig {
pub max_connections: u32,
}
/// One day of a chain's activity.
///
/// Every count is over the chain's own clock and over whatever this observer
/// has indexed — two different limits, and the caller has to state the second.
#[derive(Debug, Clone, PartialEq)]
pub struct DailyActivity {
/// Midnight UTC beginning the day.
pub day: DateTime<Utc>,
/// Blocks authored.
pub blocks: u64,
/// Distinct miners that authored one.
pub miners: u32,
/// Extrinsics, inherents included.
pub extrinsics: u64,
/// Of those, the ones that carried a signature. The rest are inherents —
/// on this chain most traffic is the timestamp, so a single "transactions"
/// line would be mostly clockwork.
pub signed: u64,
/// Distinct accounts that signed one.
pub signers: u32,
/// Events emitted, of the kinds this observer indexes.
pub events: u64,
/// Accounts seen for the first time anywhere in the indexed record.
pub new_accounts: u64,
}
/// One node's cached telemetry, as a row.
///
/// The same facts as [`crate::telemetry::NodeMetadata`], with the two
@@ -1928,6 +1954,108 @@ impl Store {
}))
}
/// One day of the chain's activity, from this observer's index.
///
/// Bucketed by the *chain's* clock — `chain_extrinsic.at` and `block.authored_at`
/// — never by when this observer saw anything. A stretch caught up on carries
/// observation times minutes apart for blocks spanning days, and a daily
/// series built on that reports a fortnight of activity as one afternoon.
///
/// The scope of every figure is the indexed range, which the caller must
/// carry to the reader: on a chain still being walked backwards these counts
/// are of what has been read, not of what happened.
pub async fn activity_by_day(
&self,
chain: &ChainId,
days: i32,
) -> Result<Vec<DailyActivity>, DataError> {
let rows = sqlx::query!(
r#"
with span as (
select generate_series(
date_trunc('day', now()) - make_interval(days => $2::int - 1),
date_trunc('day', now()),
interval '1 day'
) as day
),
blocks as (
select date_trunc('day', authored_at) as day,
count(*) as blocks,
count(distinct miner) as miners
from block
where chain = $1 and authored_at is not null
group by 1
),
calls as (
select date_trunc('day', at) as day,
count(*) as extrinsics,
count(*) filter (where signer is not null) as signed,
count(distinct signer) as signers
from chain_extrinsic
where chain = $1 and at is not null
group by 1
),
events as (
select date_trunc('day', at) as day, count(*) as events
from chain_event
where chain = $1 and at is not null
group by 1
),
-- An account's first appearance anywhere in the indexed record. Only
-- honest on a chain indexed from genesis; on a partial index this is
-- "first seen by us", which is why the caller reports the range.
first_seen as (
select account, min(at) as first_at
from (
select unnest(accounts) as account, at from chain_event
where chain = $1 and at is not null
union all
select unnest(accounts) as account, at from chain_extrinsic
where chain = $1 and at is not null
) seen
group by account
),
arrivals as (
select date_trunc('day', first_at) as day, count(*) as new_accounts
from first_seen
group by 1
)
select span.day as "day!",
coalesce(blocks.blocks, 0) as "blocks!",
coalesce(blocks.miners, 0) as "miners!",
coalesce(calls.extrinsics, 0) as "extrinsics!",
coalesce(calls.signed, 0) as "signed!",
coalesce(calls.signers, 0) as "signers!",
coalesce(events.events, 0) as "events!",
coalesce(arrivals.new_accounts, 0) as "new_accounts!"
from span
left join blocks on blocks.day = span.day
left join calls on calls.day = span.day
left join events on events.day = span.day
left join arrivals on arrivals.day = span.day
order by span.day asc
"#,
chain.as_str(),
days,
)
.fetch_all(&self.pool)
.await?;
Ok(rows
.into_iter()
.map(|r| DailyActivity {
day: r.day,
blocks: r.blocks as u64,
miners: r.miners as u32,
extrinsics: r.extrinsics as u64,
signed: r.signed as u64,
signers: r.signers as u32,
events: r.events as u64,
new_accounts: r.new_accounts as u64,
})
.collect())
}
/// Every node cached for a chain, most recently heard first.
///
/// The half of a node index the live feed cannot supply: a node announced
+140
View File
@@ -187,3 +187,143 @@ pub struct ChainSummary {
/// When these numbers were computed.
pub updated_at: DateTime<Utc>,
}
/// The figures an analyst needs to judge an ecosystem, in one place.
///
/// Two kinds of number live here and they are not equally certain. The supply
/// and the map sizes are **chain state**, read now and true now. The per-day
/// activity is **this observer's index**, and is only as complete as
/// `indexed_from`..`indexed_to` — which on a chain still being walked backwards
/// is not the whole chain. Anything rendering these has to say which range it
/// covered, or it describes a period its axis does not.
#[derive(Debug, Clone, Serialize, Deserialize, TS)]
#[ts(export, export_to = "NetworkSummary.ts")]
pub struct NetworkSummary {
/// Which chain.
pub chain: ChainId,
/// The runtime these figures were read against.
#[ts(type = "number")]
pub spec_version: Option<u32>,
/// `Balances::TotalIssuance`, in the smallest unit.
pub total_issuance: Option<BigUintDec>,
/// `Balances::InactiveIssuance`.
pub inactive_issuance: Option<BigUintDec>,
/// What genesis handed out, summed from block zero's state.
pub genesis_endowment: Option<BigUintDec>,
/// Issuance minus the endowment: everything mined since launch, which is
/// the real emission to date and is not printed anywhere else.
pub mined_since_genesis: Option<BigUintDec>,
/// `MiningRewards::CollectedFees`.
pub collected_fees: Option<BigUintDec>,
/// `System::Account` entries — accounts that exist, counted rather than
/// inferred from the events we happen to have indexed.
#[ts(type = "number")]
pub accounts: Option<u32>,
/// Whether that count reached the end of the map rather than a ceiling.
pub accounts_complete: bool,
/// Entries in `Balances::Locks`, `Holds`, `Freezes` and `Reserves`.
///
/// All four are counted because all four are ways supply can be immobile,
/// and the useful statement on this chain is that every one of them is
/// empty. **That is why there is no "circulating supply" field**: with
/// nothing locked it would equal total issuance exactly, and printing it as
/// a separate headline would assert a distinction the chain does not
/// currently make.
pub locks: Option<LockCounts>,
/// `Vesting::Schedules` entries. Counted, not summed: this runtime's
/// vesting does not touch `Balances::Locks`, so what it holds and when it
/// releases would be a guess without reading the pallet's own maths.
#[ts(type = "number")]
pub vesting_schedules: Option<u32>,
/// `TechReferenda::ReferendumCount` — referenda ever opened.
#[ts(type = "number")]
pub referenda: Option<u32>,
/// `ReversibleTransfers::NextTransactionId` — reversible transfers ever
/// created.
#[ts(type = "number")]
pub reversible_transfers: Option<u32>,
/// `ZkTree::LeafCount` — wormhole leaves, one per mining reward.
#[ts(type = "number")]
pub wormhole_leaves: Option<u64>,
/// Dispatchables ever used, of those declared.
#[ts(type = "number")]
pub calls_used: u32,
/// Dispatchables declared by the runtime.
#[ts(type = "number")]
pub calls_declared: u32,
/// Event kinds ever fired, of those declared.
#[ts(type = "number")]
pub events_fired: u32,
/// Event kinds declared.
#[ts(type = "number")]
pub events_declared: u32,
/// Lowest block whose extrinsics this observer has read.
#[ts(type = "number")]
pub indexed_from: Option<u64>,
/// Highest.
#[ts(type = "number")]
pub indexed_to: Option<u64>,
/// The chain's own height, so a reader can see how much of it is indexed.
#[ts(type = "number")]
pub height: Option<u64>,
/// Activity per day, oldest first.
pub days: Vec<DailyActivity>,
}
/// How many accounts have supply immobilised, by each mechanism.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, TS)]
#[ts(export, export_to = "LockCounts.ts")]
pub struct LockCounts {
/// `Balances::Locks`.
#[ts(type = "number")]
pub locks: u32,
/// `Balances::Holds`.
#[ts(type = "number")]
pub holds: u32,
/// `Balances::Freezes`.
#[ts(type = "number")]
pub freezes: u32,
/// `Balances::Reserves`.
#[ts(type = "number")]
pub reserves: u32,
}
/// One day of a chain's activity, as this observer indexed it.
#[derive(Debug, Clone, Serialize, Deserialize, TS)]
#[ts(export, export_to = "DailyActivity.ts")]
pub struct DailyActivity {
/// Midnight UTC beginning the day.
pub day: DateTime<Utc>,
/// Blocks authored.
#[ts(type = "number")]
pub blocks: u64,
/// Distinct miners that authored one.
#[ts(type = "number")]
pub miners: u32,
/// Extrinsics, inherents included.
#[ts(type = "number")]
pub extrinsics: u64,
/// Of those, the ones carrying a signature.
///
/// Separated from the total because on this chain most traffic is the
/// timestamp inherent, and a single "transactions per day" line would be
/// mostly clockwork — a number that looks like adoption and is not.
#[ts(type = "number")]
pub signed: u64,
/// Distinct accounts that signed one.
#[ts(type = "number")]
pub signers: u32,
/// Events emitted, of the kinds this observer indexes.
#[ts(type = "number")]
pub events: u64,
/// Accounts seen for the first time anywhere in the indexed record.
#[ts(type = "number")]
pub new_accounts: u64,
}
+4 -1
View File
@@ -31,7 +31,10 @@ mod ws;
pub use account::{AccountDetail, AccountEvent, AccountRow, ActivitySource, RewardSummary};
pub use block::{BlockDetail, BlockEvent, BlockExtrinsic, BlockObservation, RecentBlock};
pub use call::{CallIndex, CallSummary, EventSummary};
pub use chain::{ChainId, ChainInfo, ChainStatus, ChainSummary, ClientVersion, Tracking};
pub use chain::{
ChainId, ChainInfo, ChainStatus, ChainSummary, ClientVersion, DailyActivity, LockCounts,
NetworkSummary, Tracking,
};
pub use error::{ApiError, EntityError};
pub use miner::{
AttributionSource, LeaderboardRow, MinerDetail, MinerId, MinerSeriesPoint, NodeActivity,