Map enumeration asks for more keys than Substrate allows, and swallows the error #4

New Issue

grenade · 2026-09-10T07:48:30Z

grenade commented

2026-09-10 07:48:30 +00:00

What

/heisenberg/reversible reports 0 pending. The chain has 2.

PENDING_LIMIT is 1,600 and state_getKeysPaged caps count at 1,000:

count=20   -> 2 keys
count=1000 -> 2 keys
count=1600 -> ERROR 4002 "count exceeds maximum value. value: 1600, max: 1000"

The route then does .unwrap_or_default(), so the error becomes an empty page.
It does not look broken — it looks like a chain with nothing in flight, which is
exactly what mainnet legitimately looks like. This would have shipped and stayed
wrong indefinitely.

Found only because ReversibleTransfers::NextTransactionId is a plain counter
and reads 5 on Heisenberg, which contradicted the page.

Why it did not show up in testing

GENESIS_LIMIT is 500, under the cap, so genesis enumeration worked and gave
false confidence in the machinery. The System::Account verification also used
a small count. Every path that was checked happened to be under 1,000.

Fix

Clamp count in storage_keys_paged — 1,000 is a protocol limit, not a
caller's preference, so it belongs in the one place that knows about the RPC
rather than in each caller's constant.
Stop swallowing the error. An enumeration that fails is not an empty map.
A chain whose runtime lacks the pallet already returns 404, so a failure here
is a real fault and should say so.
Page properly rather than truncating at one request, so a map larger than
1,000 entries is complete rather than quietly cut off.

Note

The same silent shape is worth grepping for: unwrap_or_default() on anything
that talks to a node turns "could not ask" into "the answer is nothing", and the
two are never the same.

## What `/heisenberg/reversible` reports **0 pending**. The chain has **2**. `PENDING_LIMIT` is 1,600 and `state_getKeysPaged` caps `count` at 1,000: ``` count=20 -> 2 keys count=1000 -> 2 keys count=1600 -> ERROR 4002 "count exceeds maximum value. value: 1600, max: 1000" ``` The route then does `.unwrap_or_default()`, so the error becomes an empty page. It does not look broken — it looks like a chain with nothing in flight, which is exactly what mainnet legitimately looks like. This would have shipped and stayed wrong indefinitely. Found only because `ReversibleTransfers::NextTransactionId` is a plain counter and reads `5` on Heisenberg, which contradicted the page. ## Why it did not show up in testing `GENESIS_LIMIT` is 500, under the cap, so genesis enumeration worked and gave false confidence in the machinery. The `System::Account` verification also used a small count. Every path that was checked happened to be under 1,000. ## Fix 1. Clamp `count` in `storage_keys_paged` — 1,000 is a protocol limit, not a caller's preference, so it belongs in the one place that knows about the RPC rather than in each caller's constant. 2. **Stop swallowing the error.** An enumeration that fails is not an empty map. A chain whose runtime lacks the pallet already returns 404, so a failure here is a real fault and should say so. 3. Page properly rather than truncating at one request, so a map larger than 1,000 entries is complete rather than quietly cut off. ## Note The same silent shape is worth grepping for: `unwrap_or_default()` on anything that talks to a node turns "could not ask" into "the answer is nothing", and the two are never the same.

grenade referenced this issue from a commit

2026-09-10 07:50:18 +00:00

fix: map enumeration asked for more keys than the node allows

grenade closed this issue

2026-09-10 07:50:18 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: blackbeard/observer#4