feat: add @quantus/codec, driving encode and decode from runtime metadata
The extension has to build a signing payload, assemble an extrinsic and decode
a call well enough to show a user what they are approving. The obvious route was
@polkadot/api's codec. That is closed, and quantus/api#1 carries the tested
evidence:
- @polkadot/types caps fixed arrays at 2048 bytes, and ML-DSA signatures are
[u8;5261] and [u8;7219], so every Quantus extrinsic trips it
- api.rpc.chain.getBlock throws on every block of this chain, at the timestamp
inherent, because it reads the extrinsic preamble byte as a version when the
top two bits are a type tag
- it *guesses* that signed extensions it does not recognise contribute nothing
to the signed payload
The third is why this is a package rather than a patch. The guess is right
today — the registry says ReversibleTransactionExtension and
WormholeProofRecorderExtension are empty on both halves — and it is right only
by luck. This chain's encoding has changed between runtimes, transactionVersion
has gone 2 -> 3 -> 6 across four upgrades, and when the guess stops holding the
wallet keeps signing: valid signatures over a payload missing bytes the runtime
put there, reported by the chain as BadProof, which is also what it reports for
a wrong key.
So nothing here names a pallet, a call, an extension or a signature scheme.
Every type id is read from metadata the node produced by running
Metadata_metadata against the runtime WASM in a given block's state, the same
oracle blackbeard.observer has been decoding against across four upgrade
boundaries. encode_extensions walks the declared extensions in order and refuses
to build a payload when one that encodes to something has no value supplied —
a wallet that cannot sign is a bug report, one that signs the wrong bytes is a
support case nobody diagnoses.
Proven end to end on Heisenberg at spec 148: a balances.transfer_keep_alive
built entirely here, signed by @quantus/crypto under QUANTUS_EXTRINSIC, included
at block 1050475 and read back from that block — inherent at index 0 included,
which is the block @polkadot/api cannot decode at all.
Two notes carried over from @quantus/crypto, both load-bearing: decode_checked
walks with scale_decode's IgnoreVisitor before scale_value touches the bytes,
because scale_value sizes a Vec from the length prefix before decoding an item
and an aborted allocation leaves no Err to catch; and the build needs binaryen
123, since 105 silently corrupts the output.
Closes #3
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012uDUodEcRbBwNRi3UCmw8f
This commit is contained in:
@@ -0,0 +1,503 @@
|
||||
// Copyright 2026 @quantus/codec authors & contributors
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
//! Building the bytes that get signed, and the extrinsic that carries them.
|
||||
|
||||
use alloc::collections::BTreeMap;
|
||||
use alloc::format;
|
||||
use alloc::string::{String, ToString};
|
||||
use alloc::vec::Vec;
|
||||
|
||||
use parity_scale_codec::Encode;
|
||||
use scale_value::{Composite, Primitive, Value, ValueDef};
|
||||
|
||||
use crate::runtime::{CodecError, Runtime};
|
||||
|
||||
/// What the caller knows about one signed extension.
|
||||
///
|
||||
/// Both halves are optional because most extensions need neither: a zero-sized
|
||||
/// `ty` contributes nothing to the extrinsic, and a zero-sized `additional`
|
||||
/// contributes nothing to the payload. Supplying a value for a zero-sized type
|
||||
/// is not an error; *omitting* one for a non-zero-sized type is.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct ExtensionValue {
|
||||
pub extra: Option<serde_json::Value>,
|
||||
pub additional: Option<serde_json::Value>,
|
||||
}
|
||||
|
||||
/// The two byte strings a signed extrinsic needs from its extensions.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct EncodedExtensions {
|
||||
/// Goes into the extrinsic, after the signature.
|
||||
pub extra: Vec<u8>,
|
||||
/// Goes into the signed payload only — never on the wire.
|
||||
pub additional: Vec<u8>,
|
||||
}
|
||||
|
||||
impl Runtime {
|
||||
/// Encode a call by name, against this runtime's own call type.
|
||||
pub fn encode_call(
|
||||
&self,
|
||||
pallet: &str,
|
||||
call: &str,
|
||||
args: &serde_json::Value,
|
||||
) -> Result<Vec<u8>, CodecError> {
|
||||
// The outer `Call` enum is a variant per pallet, and each of those
|
||||
// carries that pallet's own call enum. So a call is two nested variants
|
||||
// by name, and the indices — which are what actually go on the wire —
|
||||
// come from the registry rather than from a table in this crate.
|
||||
let pallet_ty = self.extrinsic.call;
|
||||
let inner = Value::variant(
|
||||
call.to_string(),
|
||||
self.composite_for_call(pallet, call, args)?,
|
||||
);
|
||||
let outer = Value::variant(
|
||||
pallet.to_string(),
|
||||
Composite::Unnamed(alloc::vec![inner]),
|
||||
);
|
||||
|
||||
let mut out = Vec::new();
|
||||
|
||||
scale_value::scale::encode_as_type(&outer, pallet_ty, self.types(), &mut out)
|
||||
.map_err(|e| CodecError::Encode(format!("{pallet}.{call}: {e}")))?;
|
||||
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Build the argument composite for one call, interpreting `args` against
|
||||
/// the types the runtime declares for it.
|
||||
fn composite_for_call(
|
||||
&self,
|
||||
pallet: &str,
|
||||
call: &str,
|
||||
args: &serde_json::Value,
|
||||
) -> Result<Composite<()>, CodecError> {
|
||||
let calls_ty = self
|
||||
.calls
|
||||
.get(pallet)
|
||||
.copied()
|
||||
.ok_or_else(|| CodecError::NoSuchCall(pallet.to_string()))?;
|
||||
|
||||
let variant = match self.types().resolve(calls_ty).map(|t| &t.type_def) {
|
||||
Some(scale_info::TypeDef::Variant(v)) => v
|
||||
.variants
|
||||
.iter()
|
||||
.find(|v| v.name == call)
|
||||
.ok_or_else(|| CodecError::NoSuchCall(format!("{pallet}.{call}")))?,
|
||||
_ => return Err(CodecError::NoSuchCall(format!("{pallet}.{call}"))),
|
||||
};
|
||||
|
||||
let mut fields = Vec::new();
|
||||
|
||||
for field in &variant.fields {
|
||||
let name = field.name.clone().unwrap_or_default();
|
||||
let supplied = args.get(&name).ok_or_else(|| {
|
||||
CodecError::Encode(format!("{pallet}.{call}: no value for argument {name}"))
|
||||
})?;
|
||||
|
||||
fields.push((name, self.json_to_value(supplied, field.ty.id)?));
|
||||
}
|
||||
|
||||
Ok(Composite::Named(fields))
|
||||
}
|
||||
|
||||
/// Encode every signed extension this runtime declares, in order.
|
||||
///
|
||||
/// ## Why an unsupplied extension is fatal
|
||||
///
|
||||
/// `@polkadot/api` logs `Unknown signed extensions … treating them as
|
||||
/// no-effect` and writes zero bytes for anything it does not recognise. That
|
||||
/// guess is correct only while every unrecognised extension happens to be
|
||||
/// zero-sized, and when it stops being correct the wallet keeps signing —
|
||||
/// valid signatures over a payload that is missing bytes the runtime put
|
||||
/// there. The chain calls that `BadProof`, which is also what it calls a
|
||||
/// wrong key, so the failure is silent, remote and indistinguishable from
|
||||
/// the one thing it is not.
|
||||
///
|
||||
/// Here the registry decides. An extension whose declared type encodes to
|
||||
/// nothing contributes nothing and needs no value; anything else must be
|
||||
/// supplied by the caller or this refuses to build a payload at all. A
|
||||
/// wallet that cannot sign is a bug report; a wallet that signs the wrong
|
||||
/// bytes is a support case that never gets diagnosed.
|
||||
pub fn encode_extensions(
|
||||
&self,
|
||||
values: &BTreeMap<String, ExtensionValue>,
|
||||
) -> Result<EncodedExtensions, CodecError> {
|
||||
let mut extra = Vec::new();
|
||||
let mut additional = Vec::new();
|
||||
|
||||
for def in &self.extensions {
|
||||
let supplied = values.get(&def.identifier);
|
||||
|
||||
self.encode_half(
|
||||
def.ty,
|
||||
supplied.and_then(|v| v.extra.as_ref()),
|
||||
&def.identifier,
|
||||
&mut extra,
|
||||
)?;
|
||||
|
||||
self.encode_half(
|
||||
def.additional,
|
||||
supplied.and_then(|v| v.additional.as_ref()),
|
||||
&def.identifier,
|
||||
&mut additional,
|
||||
)?;
|
||||
}
|
||||
|
||||
Ok(EncodedExtensions { extra, additional })
|
||||
}
|
||||
|
||||
fn encode_half(
|
||||
&self,
|
||||
ty: u32,
|
||||
supplied: Option<&serde_json::Value>,
|
||||
identifier: &str,
|
||||
out: &mut Vec<u8>,
|
||||
) -> Result<(), CodecError> {
|
||||
if self.is_empty_ty(ty) {
|
||||
// Encodes to nothing whether or not a value was supplied. Writing
|
||||
// nothing here is a reading of the registry, not an assumption.
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let value = supplied.ok_or_else(|| CodecError::MissingExtension(identifier.to_string()))?;
|
||||
let converted = self.json_to_value(value, ty)?;
|
||||
|
||||
scale_value::scale::encode_as_type(&converted, ty, self.types(), out)
|
||||
.map_err(|e| CodecError::Encode(format!("{identifier}: {e}")))
|
||||
}
|
||||
|
||||
/// The bytes a signer signs: `call ‖ extra ‖ additional`.
|
||||
///
|
||||
/// Substrate's own rule from `unchecked_extrinsic.rs` — a payload longer than
|
||||
/// 256 bytes is signed as its BLAKE2b-256 hash — is **not** applied here.
|
||||
/// That is the caller's, because the hash belongs with the signing code that
|
||||
/// also chooses the FIPS 204 context, and splitting one rule across two
|
||||
/// packages is how the halves drift apart.
|
||||
pub fn signer_payload(
|
||||
&self,
|
||||
call: &[u8],
|
||||
extensions: &EncodedExtensions,
|
||||
) -> Vec<u8> {
|
||||
let mut out = Vec::with_capacity(call.len() + extensions.extra.len() + extensions.additional.len());
|
||||
|
||||
out.extend_from_slice(call);
|
||||
out.extend_from_slice(&extensions.extra);
|
||||
out.extend_from_slice(&extensions.additional);
|
||||
|
||||
out
|
||||
}
|
||||
|
||||
/// Assemble a signed extrinsic.
|
||||
///
|
||||
/// The preamble is `0b10 << 6 | version`: the type tag says signed and the
|
||||
/// low six bits carry the version the metadata declares. Not a hard-coded
|
||||
/// `0x84` — if this runtime ever declares a different extrinsic version, the
|
||||
/// byte follows it.
|
||||
///
|
||||
/// `signature` is the already-encoded `Signature` type, variant byte
|
||||
/// included: the signing side knows which ML-DSA scheme the key is, and
|
||||
/// re-deriving it here from the byte length would be a second source of
|
||||
/// truth. It is written raw — a fixed-size array takes **no compact length
|
||||
/// prefix**, which is the detail that a `Vec<u8>`-shaped assumption gets
|
||||
/// wrong by exactly two bytes.
|
||||
pub fn encode_extrinsic(
|
||||
&self,
|
||||
address: &serde_json::Value,
|
||||
signature: &[u8],
|
||||
extra: &[u8],
|
||||
call: &[u8],
|
||||
) -> Result<Vec<u8>, CodecError> {
|
||||
let mut body = Vec::new();
|
||||
|
||||
body.push(0b1000_0000 | (self.extrinsic_version() & 0b0011_1111));
|
||||
|
||||
let addr = self.json_to_value(address, self.extrinsic.address)?;
|
||||
|
||||
scale_value::scale::encode_as_type(&addr, self.extrinsic.address, self.types(), &mut body)
|
||||
.map_err(|e| CodecError::Encode(format!("address: {e}")))?;
|
||||
|
||||
body.extend_from_slice(signature);
|
||||
body.extend_from_slice(extra);
|
||||
body.extend_from_slice(call);
|
||||
|
||||
// The node expects the extrinsic length-prefixed.
|
||||
let mut out = parity_scale_codec::Compact(body.len() as u64).encode();
|
||||
|
||||
out.extend_from_slice(&body);
|
||||
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Interpret a JSON value as a particular registry type.
|
||||
///
|
||||
/// Type-directed on purpose. The same JSON string `"0xa5aa…"` is an
|
||||
/// `AccountId32`, an `H256` or a `Vec<u8>` depending only on what the runtime
|
||||
/// says goes there, and JSON carries no way to tell them apart. Asking the
|
||||
/// registry is the only way that stays right across an upgrade.
|
||||
pub(crate) fn json_to_value(
|
||||
&self,
|
||||
json: &serde_json::Value,
|
||||
ty: u32,
|
||||
) -> Result<Value<()>, CodecError> {
|
||||
use scale_info::TypeDef;
|
||||
|
||||
let def = self
|
||||
.types()
|
||||
.resolve(ty)
|
||||
.map(|t| &t.type_def)
|
||||
.ok_or_else(|| CodecError::Encode(format!("no registry type {ty}")))?;
|
||||
|
||||
match def {
|
||||
TypeDef::Compact(c) => self.json_to_value(json, c.type_param.id),
|
||||
|
||||
TypeDef::Primitive(p) => primitive(json, p),
|
||||
|
||||
TypeDef::Array(a) => {
|
||||
let inner = a.type_param.id;
|
||||
let want = a.len as usize;
|
||||
|
||||
if let Some(bytes) = hex_bytes(json) {
|
||||
if bytes.len() != want {
|
||||
return Err(CodecError::Encode(format!(
|
||||
"expected {want} bytes, got {}",
|
||||
bytes.len()
|
||||
)));
|
||||
}
|
||||
|
||||
return Ok(byte_composite(&bytes));
|
||||
}
|
||||
|
||||
self.unnamed(json, |_| inner)
|
||||
}
|
||||
|
||||
TypeDef::Sequence(s) => {
|
||||
if let Some(bytes) = hex_bytes(json) {
|
||||
return Ok(byte_composite(&bytes));
|
||||
}
|
||||
|
||||
self.unnamed(json, |_| s.type_param.id)
|
||||
}
|
||||
|
||||
TypeDef::Tuple(t) => {
|
||||
// An empty tuple is the unit type; JSON `null` and an empty
|
||||
// array both mean it, and so does anything else, since it
|
||||
// encodes to no bytes either way.
|
||||
if t.fields.is_empty() {
|
||||
return Ok(Value::unnamed_composite([]));
|
||||
}
|
||||
|
||||
let ids: Vec<u32> = t.fields.iter().map(|f| f.id).collect();
|
||||
|
||||
self.unnamed(json, move |i| ids[i.min(ids.len() - 1)])
|
||||
}
|
||||
|
||||
TypeDef::Composite(c) => {
|
||||
// A single-field struct is transparent, named or not, unless the
|
||||
// caller actually spelled the field out. `AccountId32(pub [u8;
|
||||
// 32])` should take the hex string its inner array takes, and
|
||||
// `CheckMetadataHash { mode }` should take `"Disabled"` — neither
|
||||
// wrapper is something a caller should have to know about, and
|
||||
// both are wrappers the *runtime* chose, so the registry is what
|
||||
// tells us they are there.
|
||||
if c.fields.len() == 1 {
|
||||
let name = c.fields[0].name.clone();
|
||||
let spelled_out = name
|
||||
.as_ref()
|
||||
.zip(json.as_object())
|
||||
.is_some_and(|(n, map)| map.contains_key(n.as_str()));
|
||||
|
||||
if !spelled_out {
|
||||
let inner = self.json_to_value(json, c.fields[0].ty.id)?;
|
||||
|
||||
return Ok(match name {
|
||||
Some(n) => Value {
|
||||
value: ValueDef::Composite(Composite::Named(alloc::vec![(n, inner)])),
|
||||
context: (),
|
||||
},
|
||||
None => Value::unnamed_composite([inner]),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
match json {
|
||||
serde_json::Value::Object(map) => {
|
||||
let mut fields = Vec::new();
|
||||
|
||||
for f in &c.fields {
|
||||
let name = f.name.clone().unwrap_or_default();
|
||||
let v = map.get(&name).ok_or_else(|| {
|
||||
CodecError::Encode(format!("no value for field {name}"))
|
||||
})?;
|
||||
|
||||
fields.push((name, self.json_to_value(v, f.ty.id)?));
|
||||
}
|
||||
|
||||
Ok(Value {
|
||||
value: ValueDef::Composite(Composite::Named(fields)),
|
||||
context: (),
|
||||
})
|
||||
}
|
||||
_ => {
|
||||
let ids: Vec<u32> = c.fields.iter().map(|f| f.ty.id).collect();
|
||||
|
||||
self.unnamed(json, move |i| ids[i.min(ids.len().saturating_sub(1))])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
TypeDef::Variant(v) => {
|
||||
// Two spellings, both unambiguous: `"Immortal"` for a variant
|
||||
// that carries nothing, `{"Id": "0x…"}` for one that does.
|
||||
let (name, payload) = match json {
|
||||
serde_json::Value::String(s) => (s.clone(), None),
|
||||
serde_json::Value::Null => ("None".to_string(), None),
|
||||
serde_json::Value::Object(map) if map.len() == 1 => {
|
||||
let (k, v) = map.iter().next().expect("len == 1");
|
||||
|
||||
(k.clone(), Some(v))
|
||||
}
|
||||
_ => {
|
||||
return Err(CodecError::Encode(format!(
|
||||
"cannot read {json} as a variant"
|
||||
)))
|
||||
}
|
||||
};
|
||||
|
||||
let variant = v
|
||||
.variants
|
||||
.iter()
|
||||
.find(|x| x.name == name)
|
||||
.ok_or_else(|| CodecError::Encode(format!("no variant {name}")))?;
|
||||
|
||||
let composite = match (payload, variant.fields.len()) {
|
||||
(_, 0) => Composite::Unnamed(Vec::new()),
|
||||
(Some(p), 1) => {
|
||||
Composite::Unnamed(alloc::vec![self.json_to_value(p, variant.fields[0].ty.id)?])
|
||||
}
|
||||
(Some(p), _) => {
|
||||
let ids: Vec<u32> = variant.fields.iter().map(|f| f.ty.id).collect();
|
||||
|
||||
match self.unnamed(p, move |i| ids[i.min(ids.len() - 1)])?.value {
|
||||
ValueDef::Composite(c) => c,
|
||||
_ => unreachable!("unnamed always returns a composite"),
|
||||
}
|
||||
}
|
||||
(None, _) => {
|
||||
return Err(CodecError::Encode(format!(
|
||||
"variant {name} needs a payload"
|
||||
)))
|
||||
}
|
||||
};
|
||||
|
||||
Ok(Value::variant(name, composite))
|
||||
}
|
||||
|
||||
TypeDef::BitSequence(_) => Err(CodecError::Encode(
|
||||
"encoding a bit sequence is not supported".to_string(),
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
fn unnamed(
|
||||
&self,
|
||||
json: &serde_json::Value,
|
||||
ty_at: impl Fn(usize) -> u32,
|
||||
) -> Result<Value<()>, CodecError> {
|
||||
let items = match json {
|
||||
serde_json::Value::Array(a) => a,
|
||||
_ => {
|
||||
return Err(CodecError::Encode(format!(
|
||||
"expected an array, got {json}"
|
||||
)))
|
||||
}
|
||||
};
|
||||
|
||||
let mut out = Vec::with_capacity(items.len());
|
||||
|
||||
for (i, item) in items.iter().enumerate() {
|
||||
out.push(self.json_to_value(item, ty_at(i))?);
|
||||
}
|
||||
|
||||
Ok(Value::unnamed_composite(out))
|
||||
}
|
||||
}
|
||||
|
||||
/// Read a JSON value as a SCALE primitive.
|
||||
///
|
||||
/// Numbers arrive as JSON numbers when they fit and as **decimal strings** when
|
||||
/// they do not: a `u128` balance loses precision above 2^53 in JSON, and this
|
||||
/// chain's balances are 12 decimal places, so that boundary is reached by
|
||||
/// ordinary amounts rather than exotic ones.
|
||||
fn primitive(
|
||||
json: &serde_json::Value,
|
||||
p: &scale_info::TypeDefPrimitive,
|
||||
) -> Result<Value<()>, CodecError> {
|
||||
use scale_info::TypeDefPrimitive as P;
|
||||
|
||||
let as_u128 = || -> Result<u128, CodecError> {
|
||||
match json {
|
||||
serde_json::Value::Number(n) => n
|
||||
.as_u64()
|
||||
.map(u128::from)
|
||||
.ok_or_else(|| CodecError::Encode(format!("{n} is not a whole number"))),
|
||||
serde_json::Value::String(s) => {
|
||||
let s = s.trim();
|
||||
|
||||
if let Some(h) = s.strip_prefix("0x") {
|
||||
u128::from_str_radix(h, 16)
|
||||
} else {
|
||||
s.parse::<u128>()
|
||||
}
|
||||
.map_err(|e| CodecError::Encode(format!("{s} is not a number: {e}")))
|
||||
}
|
||||
_ => Err(CodecError::Encode(format!("{json} is not a number"))),
|
||||
}
|
||||
};
|
||||
|
||||
Ok(match p {
|
||||
P::Bool => Value {
|
||||
value: ValueDef::Primitive(Primitive::Bool(json.as_bool().ok_or_else(|| {
|
||||
CodecError::Encode(format!("{json} is not a boolean"))
|
||||
})?)),
|
||||
context: (),
|
||||
},
|
||||
P::Str => Value {
|
||||
value: ValueDef::Primitive(Primitive::String(
|
||||
json.as_str()
|
||||
.ok_or_else(|| CodecError::Encode(format!("{json} is not a string")))?
|
||||
.to_string(),
|
||||
)),
|
||||
context: (),
|
||||
},
|
||||
P::U8 | P::U16 | P::U32 | P::U64 | P::U128 | P::U256 => Value {
|
||||
value: ValueDef::Primitive(Primitive::U128(as_u128()?)),
|
||||
context: (),
|
||||
},
|
||||
P::I8 | P::I16 | P::I32 | P::I64 | P::I128 | P::I256 => Value {
|
||||
value: ValueDef::Primitive(Primitive::I128(as_u128()? as i128)),
|
||||
context: (),
|
||||
},
|
||||
P::Char => Err(CodecError::Encode("char is not encodable".to_string()))?,
|
||||
})
|
||||
}
|
||||
|
||||
/// `0x…` as bytes, if this is a hex string.
|
||||
fn hex_bytes(json: &serde_json::Value) -> Option<Vec<u8>> {
|
||||
let s = json.as_str()?.strip_prefix("0x")?;
|
||||
|
||||
if s.len() % 2 != 0 {
|
||||
return None;
|
||||
}
|
||||
|
||||
(0..s.len() / 2)
|
||||
.map(|i| u8::from_str_radix(&s[i * 2..i * 2 + 2], 16).ok())
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn byte_composite(bytes: &[u8]) -> Value<()> {
|
||||
Value::unnamed_composite(bytes.iter().map(|b| Value {
|
||||
value: ValueDef::Primitive(Primitive::U128(u128::from(*b))),
|
||||
context: (),
|
||||
}))
|
||||
}
|
||||
Reference in New Issue
Block a user