Decide whether @polkadot/types needs patching for DilithiumSignatureScheme, or whether registry overrides suffice #1
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Investigation first, code only if the answer says so. Context in quantus/extension#1.
The question
packages/types/src/interfaces/extrinsics/definitions.tshardcodes:which is 64/65 bytes. Quantus needs
DilithiumSignatureScheme— a two-variant enum carrying 5261 or 7219 bytes. But polkadot-js resolvesExtrinsicSignaturethrough the registry, and chains have historically overridden it from userland. So this may need no fork change at all:Test that before writing anything. If it works, this repo stays a plain mirror of upstream and the override moves into whatever dapp-side helper the extension ships — a much better outcome than carrying a patch.
What to actually check
U8aFixedaccept a 7219-byte length? Its bit-length handling is well outside normal use at this size; there may be an assertion or a practical limit.packages/extension-base/src/background/handlers/Extension.tsbuilds aTypeRegistryper request and callsregistry.register(types)from the storedMetadataDef, so the same override may be injectable there through metadata registration rather than code.ExtrinsicPayload.signpickswithTypefromregistry.createTypeUnsafe('ExtrinsicSignature', []) instanceof Enum.MultiSignatureis anEnumand so isDilithiumSignatureScheme, sowithTypestaystrueeither way — meaning the signing side is already correct without any override, and only extrinsic assembly is at issue. Confirm this, because it decides whether the extension needs this repo at all.0x84,MultiAddress::Id, variant byte, thensig ‖ pkas a fixed array with no compact length prefix.That last point is where a decoder most plausibly goes wrong.
qsafe.af's hand-rolled parser reads the signature as SCALE bytes with a compact length prefix, which is not what the runtime encodes; worth confirming which of the two is mistaken before trusting either as a reference.The precedent worth reading first
quantus-appshit this exact class of problem in polkadart and pinned a fork over it. Their note:A codec/metadata width disagreement is reachable here too, and the consequence is that a user approves what they did not sign. Whatever conclusion this issue reaches, pin it with a test that encodes a full signed extrinsic and compares bytes — not just one that checks the type constructs.
Outcome
Record the answer here either way. If no patch is needed, say so and leave this repo as a mirror; that is a result, not a non-result.
Narrowing this: the workaround that looked most promising is ruled out.
signedTransactiondoes not avoid the problemSignerPayloadJSONcarries awithSignedTransactionflag, and a signer may return a fully assembled extrinsic instead of a bare signature. That looked like a way to sidestepExtrinsicSignatureentirely — let the extension concatenate the bytes itself, hand back the finished transaction, and never ask@polkadot/typesto encode a 5261-byte signature.It does not work.
@polkadot/api'ssubmittable/createClass.jsaccepts the field and then immediately re-decodes it:That
createTypeUnsafe('Extrinsic', …)goes straight throughMultiSignature, andaddSignatureon the line below does too. So the registry still has to know the real signature type — which is this issue. The re-decode is a deliberate anti-tampering check (it re-derives aSignerPayloadfrom the decoded extrinsic and compares), so it is not something to disable.Worth noting the extension's own
signedTransactionpath has the same dependency:LedgerSign.tsxbuilds it withextrinsic.addSignature(...), i.e. via@polkadot/types, not by concatenation.Consequence
Anything on the
@polkadot/apistack — including polkadot-js/apps — needs the answer to this issue. There is no signer-side trick that avoids it.Contrast, and why it is a useful comparison
polkadot-api (papi) reaches the same goal without any of this.
getSignerTypereads theAddressandSignaturetypes from metadata rather than from a hardcoded definition, andcreateV4Txassembles by concatenation with no length assertion anywhere, so a 5261-byte signature needs no accommodation at all. Its only Quantus blocker is a variant-name whitelist bolted on top of the metadata lookup — quantus/papi-console#1, a two-line pnpm patch.That is the same doctrine blackbeard.observer settled on: decode the chain's data using the chain's own description of itself. Where this issue lands, prefer the solution that reads the type from metadata over one that registers another hardcoded name — the runtime already said what the type is, and
transactionVersionhas moved 2 → 3 → 6 across four Quantus upgrades, so anything pinned by name will rot.Practical effect on priority
quantus/papi-console gives a working signing path without waiting for this issue, so this is no longer on the critical path for "can we sign a real extrinsic". It stays required for polkadot-js/apps and for any dapp on
@polkadot/api— which is most of the existing ecosystem, so it is still worth doing, just not first.Answer: neither.
@polkadot/api's codec cannot be the encode/decode path at all.Tested directly against Heisenberg (spec 148, genesis
0xa5aa9e5c…c8d3b4f) with a tier-1 harness that signs with the forked keyring and submits. The registry-override hypothesis in the title is falsified, and so is the fallback ("patch@polkadot/types"). Recording the evidence because the failures are ordered, and only the first two are the ones you would predict.1. Registry overrides lose to metadata
User types are registered before
setMetadata, so the metadata-derived lookup type wins. The override is dead code.2.
@polkadot/typesrefuses fixed arrays longer than 2048Two hard-coded guards, both in this repo's upstream:
packages/types/src/metadata/PortableRegistry/PortableRegistry.ts→#extractArraypackages/types-create/src/util/getTypeDef.ts→_decodeFixedVecML-DSA-65 is
[u8;5261]and ML-DSA-87 is[u8;7219], so every Quantus signature trips it. The cap is arbitrary rather than structural:[u8;N]resolves toU8aFixed.with(N * 8), a singleUint8Arraywith no per-element codec, so nothing gets slower. Raising it locally to 65536 innode_moduleslet the extrinsic encode and reach the node.3. …and even then, polkadot-js cannot decode a Quantus block
Index 0 is the timestamp inherent. Its preamble byte is
0x05: the top two bits are a type tag (0b00bare,0b10signed,0b01general) and the low six are the version, so that is bare, v5, while signed transactions in the same block are0x84— signed, v4 — and the metadata declares extrinsic version 4. Three different numbers, all correct.blackbeard.observer'sruntime.rsdocuments this and has the regression test; polkadot-js reads the byte as a version and rejects the block.So
api.rpc.chain.getBlockthrows on every block of this chain, at the first extrinsic, before any Quantus-specific type is reached.4. The submission is still rejected, and that is the decisive part
With the cap raised, the extrinsic encodes (7220 bytes: variant
0x00+ 7219) and submits:Ruled out as causes:
qzk1Nxai…2vSn7, which holds 627410384707496 (627.41 HEI) at nonce 2760 on Heisenberg. Our Poseidon2 matches the chain'shash_bytes, andVerifychecksinto_account() == signerfirst.primitives/dilithium-crypto/src/signing_context.rs—EXTRINSIC = b"QUANTUS_EXTRINSIC", andpair.rs'stest_extrinsic_signature_rejects_other_contextspins it. We sign under exactly that.scheme_macro.rsstoressig ‖ pk(bytes[..SIGNATURE_LEN]is the signature). Ours matches.What is left is the signed payload's bytes, and here is the point: polkadot-js logs
"Treating them as no-effect" is an assumption, not a reading. The runtime's
TxExtensiontuple (runtime/src/lib.rs) has twelve members, two of which polkadot-js has never heard of, and it guesses zero bytes for both theirextraand their implicit. That guess happens to be right today — both arePhantomDatawithtype Implicit = ()— but the metadata declares those types, and a signer that guesses is wrong the moment a runtime upgrade gives either of them a field.That is not a hypothetical here. Testnet encoding has already changed between runtimes on this chain, and
transactionVersionhas gone 2 → 3 → 6 across four upgrades — each one an extrinsic-format change that silently breaks a signer written against the previous one. A wallet that produces a valid signature over the wrong payload fails asBadProof, indistinguishable from a wrong key.Decision
Do not fork
@polkadot/api. Encode and decode against the runtime's own metadata instead, the wayblackbeard.observeralready does for blocks —state_getMetadataat a block hash makes the node runMetadata_metadataagainst that block's runtime, so the runtime WASM is the oracle and the signed extensions are read, in order, with their declared types, rather than assumed.Filed as quantus/wasm#3 (
@quantus/codec). This issue stays open only to track removing@polkadot/api's codec from the extension's path; the "patch or override" question is closed — the answer is neither.Resolved, by removal
@quantus/codec(quantus/wasm#3) landed and quantus/extension#7 tier 1 now signs abalances.transfer_keep_alivethat Heisenberg accepts and includes — block 1050475, nonce 2761 → 2762 — with@polkadot/apientirely out of the path.WsProviderremains only as a JSON-RPC transport; no type is decoded by polkadot-js.One correction to the analysis above, for the record. Under §4 I listed the two unrecognised extensions as the remaining suspect for the
BadProof. Having read the metadata with the new codec, the registry says they are innocent:Both are empty on both halves, so polkadot-js's guess of zero bytes was correct here. It was correct by luck rather than by reading, which is the argument that stands — but it was not the defect.
I did not isolate what the defect was. The working submission differs from the failed one in using an immortal era where polkadot-js chose a mortal one, which makes
CheckMortality's implicit the genesis hash instead of a birth block both sides have to agree on; that is the obvious candidate and it is not evidence. Chasing it further would mean debugging a code path that is being removed, so it stays unknown and is recorded as unknown.Closing. The question in the title — patch
@polkadot/types, or override the registry — has the answer neither, and the reasons are §1–§3 above rather than theBadProof. Those three stand on their own: the 2048-byte cap, the preamble byte read as a version, and guessing at extensions the metadata declares.