feat: add @quantus/codec, driving encode and decode from runtime metadata

The extension has to build a signing payload, assemble an extrinsic and decode
a call well enough to show a user what they are approving. The obvious route was
@polkadot/api's codec. That is closed, and quantus/api#1 carries the tested
evidence:

  - @polkadot/types caps fixed arrays at 2048 bytes, and ML-DSA signatures are
    [u8;5261] and [u8;7219], so every Quantus extrinsic trips it
  - api.rpc.chain.getBlock throws on every block of this chain, at the timestamp
    inherent, because it reads the extrinsic preamble byte as a version when the
    top two bits are a type tag
  - it *guesses* that signed extensions it does not recognise contribute nothing
    to the signed payload

The third is why this is a package rather than a patch. The guess is right
today — the registry says ReversibleTransactionExtension and
WormholeProofRecorderExtension are empty on both halves — and it is right only
by luck. This chain's encoding has changed between runtimes, transactionVersion
has gone 2 -> 3 -> 6 across four upgrades, and when the guess stops holding the
wallet keeps signing: valid signatures over a payload missing bytes the runtime
put there, reported by the chain as BadProof, which is also what it reports for
a wrong key.

So nothing here names a pallet, a call, an extension or a signature scheme.
Every type id is read from metadata the node produced by running
Metadata_metadata against the runtime WASM in a given block's state, the same
oracle blackbeard.observer has been decoding against across four upgrade
boundaries. encode_extensions walks the declared extensions in order and refuses
to build a payload when one that encodes to something has no value supplied —
a wallet that cannot sign is a bug report, one that signs the wrong bytes is a
support case nobody diagnoses.

Proven end to end on Heisenberg at spec 148: a balances.transfer_keep_alive
built entirely here, signed by @quantus/crypto under QUANTUS_EXTRINSIC, included
at block 1050475 and read back from that block — inherent at index 0 included,
which is the block @polkadot/api cannot decode at all.

Two notes carried over from @quantus/crypto, both load-bearing: decode_checked
walks with scale_decode's IgnoreVisitor before scale_value touches the bytes,
because scale_value sizes a Vec from the length prefix before decoding an item
and an aborted allocation leaves no Err to catch; and the build needs binaryen
123, since 105 silently corrupts the output.

Closes #3

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012uDUodEcRbBwNRi3UCmw8f
This commit is contained in:
2026-09-15 14:14:34 +03:00
co-authored by Claude Opus 5
parent e8bf9e20c7
commit 3a1611a92e
23 changed files with 2839 additions and 6 deletions
+503
View File
@@ -0,0 +1,503 @@
// Copyright 2026 @quantus/codec authors & contributors
// SPDX-License-Identifier: Apache-2.0
//! Building the bytes that get signed, and the extrinsic that carries them.
use alloc::collections::BTreeMap;
use alloc::format;
use alloc::string::{String, ToString};
use alloc::vec::Vec;
use parity_scale_codec::Encode;
use scale_value::{Composite, Primitive, Value, ValueDef};
use crate::runtime::{CodecError, Runtime};
/// What the caller knows about one signed extension.
///
/// Both halves are optional because most extensions need neither: a zero-sized
/// `ty` contributes nothing to the extrinsic, and a zero-sized `additional`
/// contributes nothing to the payload. Supplying a value for a zero-sized type
/// is not an error; *omitting* one for a non-zero-sized type is.
#[derive(Debug, Clone, Default)]
pub struct ExtensionValue {
pub extra: Option<serde_json::Value>,
pub additional: Option<serde_json::Value>,
}
/// The two byte strings a signed extrinsic needs from its extensions.
#[derive(Debug, Clone)]
pub struct EncodedExtensions {
/// Goes into the extrinsic, after the signature.
pub extra: Vec<u8>,
/// Goes into the signed payload only — never on the wire.
pub additional: Vec<u8>,
}
impl Runtime {
/// Encode a call by name, against this runtime's own call type.
pub fn encode_call(
&self,
pallet: &str,
call: &str,
args: &serde_json::Value,
) -> Result<Vec<u8>, CodecError> {
// The outer `Call` enum is a variant per pallet, and each of those
// carries that pallet's own call enum. So a call is two nested variants
// by name, and the indices — which are what actually go on the wire —
// come from the registry rather than from a table in this crate.
let pallet_ty = self.extrinsic.call;
let inner = Value::variant(
call.to_string(),
self.composite_for_call(pallet, call, args)?,
);
let outer = Value::variant(
pallet.to_string(),
Composite::Unnamed(alloc::vec![inner]),
);
let mut out = Vec::new();
scale_value::scale::encode_as_type(&outer, pallet_ty, self.types(), &mut out)
.map_err(|e| CodecError::Encode(format!("{pallet}.{call}: {e}")))?;
Ok(out)
}
/// Build the argument composite for one call, interpreting `args` against
/// the types the runtime declares for it.
fn composite_for_call(
&self,
pallet: &str,
call: &str,
args: &serde_json::Value,
) -> Result<Composite<()>, CodecError> {
let calls_ty = self
.calls
.get(pallet)
.copied()
.ok_or_else(|| CodecError::NoSuchCall(pallet.to_string()))?;
let variant = match self.types().resolve(calls_ty).map(|t| &t.type_def) {
Some(scale_info::TypeDef::Variant(v)) => v
.variants
.iter()
.find(|v| v.name == call)
.ok_or_else(|| CodecError::NoSuchCall(format!("{pallet}.{call}")))?,
_ => return Err(CodecError::NoSuchCall(format!("{pallet}.{call}"))),
};
let mut fields = Vec::new();
for field in &variant.fields {
let name = field.name.clone().unwrap_or_default();
let supplied = args.get(&name).ok_or_else(|| {
CodecError::Encode(format!("{pallet}.{call}: no value for argument {name}"))
})?;
fields.push((name, self.json_to_value(supplied, field.ty.id)?));
}
Ok(Composite::Named(fields))
}
/// Encode every signed extension this runtime declares, in order.
///
/// ## Why an unsupplied extension is fatal
///
/// `@polkadot/api` logs `Unknown signed extensions … treating them as
/// no-effect` and writes zero bytes for anything it does not recognise. That
/// guess is correct only while every unrecognised extension happens to be
/// zero-sized, and when it stops being correct the wallet keeps signing —
/// valid signatures over a payload that is missing bytes the runtime put
/// there. The chain calls that `BadProof`, which is also what it calls a
/// wrong key, so the failure is silent, remote and indistinguishable from
/// the one thing it is not.
///
/// Here the registry decides. An extension whose declared type encodes to
/// nothing contributes nothing and needs no value; anything else must be
/// supplied by the caller or this refuses to build a payload at all. A
/// wallet that cannot sign is a bug report; a wallet that signs the wrong
/// bytes is a support case that never gets diagnosed.
pub fn encode_extensions(
&self,
values: &BTreeMap<String, ExtensionValue>,
) -> Result<EncodedExtensions, CodecError> {
let mut extra = Vec::new();
let mut additional = Vec::new();
for def in &self.extensions {
let supplied = values.get(&def.identifier);
self.encode_half(
def.ty,
supplied.and_then(|v| v.extra.as_ref()),
&def.identifier,
&mut extra,
)?;
self.encode_half(
def.additional,
supplied.and_then(|v| v.additional.as_ref()),
&def.identifier,
&mut additional,
)?;
}
Ok(EncodedExtensions { extra, additional })
}
fn encode_half(
&self,
ty: u32,
supplied: Option<&serde_json::Value>,
identifier: &str,
out: &mut Vec<u8>,
) -> Result<(), CodecError> {
if self.is_empty_ty(ty) {
// Encodes to nothing whether or not a value was supplied. Writing
// nothing here is a reading of the registry, not an assumption.
return Ok(());
}
let value = supplied.ok_or_else(|| CodecError::MissingExtension(identifier.to_string()))?;
let converted = self.json_to_value(value, ty)?;
scale_value::scale::encode_as_type(&converted, ty, self.types(), out)
.map_err(|e| CodecError::Encode(format!("{identifier}: {e}")))
}
/// The bytes a signer signs: `call ‖ extra ‖ additional`.
///
/// Substrate's own rule from `unchecked_extrinsic.rs` — a payload longer than
/// 256 bytes is signed as its BLAKE2b-256 hash — is **not** applied here.
/// That is the caller's, because the hash belongs with the signing code that
/// also chooses the FIPS 204 context, and splitting one rule across two
/// packages is how the halves drift apart.
pub fn signer_payload(
&self,
call: &[u8],
extensions: &EncodedExtensions,
) -> Vec<u8> {
let mut out = Vec::with_capacity(call.len() + extensions.extra.len() + extensions.additional.len());
out.extend_from_slice(call);
out.extend_from_slice(&extensions.extra);
out.extend_from_slice(&extensions.additional);
out
}
/// Assemble a signed extrinsic.
///
/// The preamble is `0b10 << 6 | version`: the type tag says signed and the
/// low six bits carry the version the metadata declares. Not a hard-coded
/// `0x84` — if this runtime ever declares a different extrinsic version, the
/// byte follows it.
///
/// `signature` is the already-encoded `Signature` type, variant byte
/// included: the signing side knows which ML-DSA scheme the key is, and
/// re-deriving it here from the byte length would be a second source of
/// truth. It is written raw — a fixed-size array takes **no compact length
/// prefix**, which is the detail that a `Vec<u8>`-shaped assumption gets
/// wrong by exactly two bytes.
pub fn encode_extrinsic(
&self,
address: &serde_json::Value,
signature: &[u8],
extra: &[u8],
call: &[u8],
) -> Result<Vec<u8>, CodecError> {
let mut body = Vec::new();
body.push(0b1000_0000 | (self.extrinsic_version() & 0b0011_1111));
let addr = self.json_to_value(address, self.extrinsic.address)?;
scale_value::scale::encode_as_type(&addr, self.extrinsic.address, self.types(), &mut body)
.map_err(|e| CodecError::Encode(format!("address: {e}")))?;
body.extend_from_slice(signature);
body.extend_from_slice(extra);
body.extend_from_slice(call);
// The node expects the extrinsic length-prefixed.
let mut out = parity_scale_codec::Compact(body.len() as u64).encode();
out.extend_from_slice(&body);
Ok(out)
}
/// Interpret a JSON value as a particular registry type.
///
/// Type-directed on purpose. The same JSON string `"0xa5aa…"` is an
/// `AccountId32`, an `H256` or a `Vec<u8>` depending only on what the runtime
/// says goes there, and JSON carries no way to tell them apart. Asking the
/// registry is the only way that stays right across an upgrade.
pub(crate) fn json_to_value(
&self,
json: &serde_json::Value,
ty: u32,
) -> Result<Value<()>, CodecError> {
use scale_info::TypeDef;
let def = self
.types()
.resolve(ty)
.map(|t| &t.type_def)
.ok_or_else(|| CodecError::Encode(format!("no registry type {ty}")))?;
match def {
TypeDef::Compact(c) => self.json_to_value(json, c.type_param.id),
TypeDef::Primitive(p) => primitive(json, p),
TypeDef::Array(a) => {
let inner = a.type_param.id;
let want = a.len as usize;
if let Some(bytes) = hex_bytes(json) {
if bytes.len() != want {
return Err(CodecError::Encode(format!(
"expected {want} bytes, got {}",
bytes.len()
)));
}
return Ok(byte_composite(&bytes));
}
self.unnamed(json, |_| inner)
}
TypeDef::Sequence(s) => {
if let Some(bytes) = hex_bytes(json) {
return Ok(byte_composite(&bytes));
}
self.unnamed(json, |_| s.type_param.id)
}
TypeDef::Tuple(t) => {
// An empty tuple is the unit type; JSON `null` and an empty
// array both mean it, and so does anything else, since it
// encodes to no bytes either way.
if t.fields.is_empty() {
return Ok(Value::unnamed_composite([]));
}
let ids: Vec<u32> = t.fields.iter().map(|f| f.id).collect();
self.unnamed(json, move |i| ids[i.min(ids.len() - 1)])
}
TypeDef::Composite(c) => {
// A single-field struct is transparent, named or not, unless the
// caller actually spelled the field out. `AccountId32(pub [u8;
// 32])` should take the hex string its inner array takes, and
// `CheckMetadataHash { mode }` should take `"Disabled"` — neither
// wrapper is something a caller should have to know about, and
// both are wrappers the *runtime* chose, so the registry is what
// tells us they are there.
if c.fields.len() == 1 {
let name = c.fields[0].name.clone();
let spelled_out = name
.as_ref()
.zip(json.as_object())
.is_some_and(|(n, map)| map.contains_key(n.as_str()));
if !spelled_out {
let inner = self.json_to_value(json, c.fields[0].ty.id)?;
return Ok(match name {
Some(n) => Value {
value: ValueDef::Composite(Composite::Named(alloc::vec![(n, inner)])),
context: (),
},
None => Value::unnamed_composite([inner]),
});
}
}
match json {
serde_json::Value::Object(map) => {
let mut fields = Vec::new();
for f in &c.fields {
let name = f.name.clone().unwrap_or_default();
let v = map.get(&name).ok_or_else(|| {
CodecError::Encode(format!("no value for field {name}"))
})?;
fields.push((name, self.json_to_value(v, f.ty.id)?));
}
Ok(Value {
value: ValueDef::Composite(Composite::Named(fields)),
context: (),
})
}
_ => {
let ids: Vec<u32> = c.fields.iter().map(|f| f.ty.id).collect();
self.unnamed(json, move |i| ids[i.min(ids.len().saturating_sub(1))])
}
}
}
TypeDef::Variant(v) => {
// Two spellings, both unambiguous: `"Immortal"` for a variant
// that carries nothing, `{"Id": "0x…"}` for one that does.
let (name, payload) = match json {
serde_json::Value::String(s) => (s.clone(), None),
serde_json::Value::Null => ("None".to_string(), None),
serde_json::Value::Object(map) if map.len() == 1 => {
let (k, v) = map.iter().next().expect("len == 1");
(k.clone(), Some(v))
}
_ => {
return Err(CodecError::Encode(format!(
"cannot read {json} as a variant"
)))
}
};
let variant = v
.variants
.iter()
.find(|x| x.name == name)
.ok_or_else(|| CodecError::Encode(format!("no variant {name}")))?;
let composite = match (payload, variant.fields.len()) {
(_, 0) => Composite::Unnamed(Vec::new()),
(Some(p), 1) => {
Composite::Unnamed(alloc::vec![self.json_to_value(p, variant.fields[0].ty.id)?])
}
(Some(p), _) => {
let ids: Vec<u32> = variant.fields.iter().map(|f| f.ty.id).collect();
match self.unnamed(p, move |i| ids[i.min(ids.len() - 1)])?.value {
ValueDef::Composite(c) => c,
_ => unreachable!("unnamed always returns a composite"),
}
}
(None, _) => {
return Err(CodecError::Encode(format!(
"variant {name} needs a payload"
)))
}
};
Ok(Value::variant(name, composite))
}
TypeDef::BitSequence(_) => Err(CodecError::Encode(
"encoding a bit sequence is not supported".to_string(),
)),
}
}
fn unnamed(
&self,
json: &serde_json::Value,
ty_at: impl Fn(usize) -> u32,
) -> Result<Value<()>, CodecError> {
let items = match json {
serde_json::Value::Array(a) => a,
_ => {
return Err(CodecError::Encode(format!(
"expected an array, got {json}"
)))
}
};
let mut out = Vec::with_capacity(items.len());
for (i, item) in items.iter().enumerate() {
out.push(self.json_to_value(item, ty_at(i))?);
}
Ok(Value::unnamed_composite(out))
}
}
/// Read a JSON value as a SCALE primitive.
///
/// Numbers arrive as JSON numbers when they fit and as **decimal strings** when
/// they do not: a `u128` balance loses precision above 2^53 in JSON, and this
/// chain's balances are 12 decimal places, so that boundary is reached by
/// ordinary amounts rather than exotic ones.
fn primitive(
json: &serde_json::Value,
p: &scale_info::TypeDefPrimitive,
) -> Result<Value<()>, CodecError> {
use scale_info::TypeDefPrimitive as P;
let as_u128 = || -> Result<u128, CodecError> {
match json {
serde_json::Value::Number(n) => n
.as_u64()
.map(u128::from)
.ok_or_else(|| CodecError::Encode(format!("{n} is not a whole number"))),
serde_json::Value::String(s) => {
let s = s.trim();
if let Some(h) = s.strip_prefix("0x") {
u128::from_str_radix(h, 16)
} else {
s.parse::<u128>()
}
.map_err(|e| CodecError::Encode(format!("{s} is not a number: {e}")))
}
_ => Err(CodecError::Encode(format!("{json} is not a number"))),
}
};
Ok(match p {
P::Bool => Value {
value: ValueDef::Primitive(Primitive::Bool(json.as_bool().ok_or_else(|| {
CodecError::Encode(format!("{json} is not a boolean"))
})?)),
context: (),
},
P::Str => Value {
value: ValueDef::Primitive(Primitive::String(
json.as_str()
.ok_or_else(|| CodecError::Encode(format!("{json} is not a string")))?
.to_string(),
)),
context: (),
},
P::U8 | P::U16 | P::U32 | P::U64 | P::U128 | P::U256 => Value {
value: ValueDef::Primitive(Primitive::U128(as_u128()?)),
context: (),
},
P::I8 | P::I16 | P::I32 | P::I64 | P::I128 | P::I256 => Value {
value: ValueDef::Primitive(Primitive::I128(as_u128()? as i128)),
context: (),
},
P::Char => Err(CodecError::Encode("char is not encodable".to_string()))?,
})
}
/// `0x…` as bytes, if this is a hex string.
fn hex_bytes(json: &serde_json::Value) -> Option<Vec<u8>> {
let s = json.as_str()?.strip_prefix("0x")?;
if s.len() % 2 != 0 {
return None;
}
(0..s.len() / 2)
.map(|i| u8::from_str_radix(&s[i * 2..i * 2 + 2], 16).ok())
.collect()
}
fn byte_composite(bytes: &[u8]) -> Value<()> {
Value::unnamed_composite(bytes.iter().map(|b| Value {
value: ValueDef::Primitive(Primitive::U128(u128::from(*b))),
context: (),
}))
}