[CI Skip] release/stable 4.9.1

skip-checks: true

.123trigger

@@ -1,4 +1,4 @@
-27
+29
 1.4.2
 1.6.2
 1.11.2
@@ -19,3 +19,6 @@
 3.10.2
 4.0.2
 4.0.3
+
+4.6.2
+4.7.2

.nvmrc

@@ -1 +1 @@
-12
+14

CHANGELOG.md

@@ -1,5 +1,148 @@
 # CHANGELOG
 
+## 4.9.1 May 6, 2021
+
+Upgrade priority: Medium. Required for the next v0.9.0 Polkadot/Kusama chain upgrades where parachains are used.
+
+- **Breaking change** The previously deprecated contracts interfaces, has been removed. Update to `<myContract>.query.*`, `<myContract>.tx.*` and `<myCode>.tx.*` interfaces as per the previous code usage hints.
+- **Important** By default the `isPedantic` mode is now switched on for storage queries. This means that the API will fail on any types where the input/output decoding doesn't match 100%
+
+Changes:
+
+- Re-add isPedantic check on storage decoding
+- `ConsensusEngineId` now implements `[u8;4]` (mapping closer to Rust source)
+- Additional Xcm types
+- Adjust parachain types
+- Remove error logging on createType (details passed in error)
+- Adjust `ContractInfo` type as per latest Substrate
+- Adjust `DigestItem` type as per latest Substrate
+- Adjust re-encoding of `BitVec` based on decoded length
+- Added `contracts_instantiate`, `state_traceBlock` & `system_reservedPeers` RPCs
+- Prepare for auto-capability detection with historic `.at` queries
+- Add known upgrade block for Westend 900 & 9000
+- Remove all deprecated interfaces inside `@polkadot/api-contracts`
+- Initial support for NMap storage keys
+- Support metadata v13 from Substrate (upcoming upgrade)
+- Optimize historic metadata retrieval from rpc core
+
+
+## 4.8.1 May 2, 2021
+
+Upgrade priority: Low. Recommended when using or developing for parachains.
+
+Changes:
+
+- De-alias xcm `Outcome` (used in multiple pallettes)
+- Adjust xcm types as per latest, e.g. `XcmError`
+- Fix generate of well-known storage keys (e.g. `:code`)
+- Optimize metadata unknown type validation
+- Allow metadata creation for `Map` or `Record`
+- Expose additional camelCase `method` on `AbiMessage` (maps to decorated name)
+- Added explicit double-byte ss58 `AccountId` encoding test
+- Add known upgrade block for Polkadot v30
+- Cleanup WebSocket handlers when close event has been received
+- Update to latest Substrate metadata
+
+
+## 4.7.2 Apr 26, 2021
+
+Upgrade priority: Low. Recommended for users of 4.7.1 using nvm
+
+Changes:
+
+- Update `EthBlock`/`EthHeader` types (as per Rust origin)
+- Updated Xcm `OriginKind` enum
+- Add derives for `membership`
+- Cleanups on encodedLength calculations
+- Update `.nvmrc` to reflect engines changes
+
+
+## 4.7.1 Apr 25, 2021
+
+Upgrade priority: Low. Recommended for users following Substrate master.
+
+Changes:
+
+- Adjust Struct decoding with better error messages on invalid input objects
+- Add support for new `BoundedVec` type
+- Expand `Data` type (as used in identity), with all is/as getters
+- Updated `Xcm` types (with Plularity)
+- Adjust `DispatchError` type as per current Substrate master
+- Adjusted Statemint `Proxy` type
+- Remove metadata generation dependency on well-known.json (dedupe)
+- Adjust WebSocket reconnection delay from 1s to 2.5s
+- Revert Beefy keys addition for Westend
+- Add explicit `engines` config to `package.json`
+- Support for `None` in `RewardDestination`
+- Align with latest substrate metadata
+
+
+## 4.6.2 Apr 19, 2021
+
+Upgrade priority: Low. Recommended for users of Kusama/Polkadot.
+
+Changes:
+
+- Correct Kusama/Polkadot runtime 30 session key definitions
+
+
+## 4.6.1 Apr 19, 2021
+
+Upgrade priority: Low. Recommended if used in a Node.js 15.x environment.
+
+Changes:
+
+- Assert keypair validity in signing operations
+- Expose `internalError` on submittable results (e.g. event decoding errors)
+- Simplify session derives for old/new
+- Add Statemint to known types
+- Add support for `rpc.beefy.subscribeJustifications`
+- Update Xcm types
+- Explicitly handle `isReadyOrError` with `Api.create`
+- Add Kusama upgrade block for 2030
+
+
+## 4.5.1 Apr 12, 2021
+
+Upgrade priority: Low. Recommended with usage with the Rococo testnet.
+
+Changes:
+
+- Update parachain types as per latest Polkadot
+- Cater for `Keys` with `BEEFY` session keys
+- Update `Keys` injection to allow for on-chain type detection
+- Add `Westend` known upgrade block for runtime 50
+- Align types with latest Substrate master
+- Adjust keepalive `system_health` timer
+
+
+## 4.4.1 Apr 5, 2021
+
+Upgrade priority: Low. Recommended with usage with the Rococo testnet.
+
+Changes:
+
+- Add `ServiceQuality` type for Cumulus
+- Adjust XCM MultiLocation type (X5 to X8 junctions)
+- Add Polkadot 29 upgrade block
+
+
+## 4.3.1 Mar 29, 2021
+
+Upgrade priority: Low.
+
+Contributed:
+
+- Fix documentation on ABI parameters (Thanks to https://github.com/vminkov)
+
+Changes:
+
+- Warn on usage of deprecated contract interfaces
+- Update `ParaInfo` type
+- Council should be optional in treasury derives
+- Allow extraction of session derives indexes with no staking
+
+
 ## 4.2.1 Mar 22, 2021
 
 Upgrade priority: Low. Recommended when tracking the latest updates from Substrate.

README.md

@@ -21,7 +21,7 @@ The API is split up into a number of internal packages -
 
 - [@polkadot/api](packages/api/) The API library, providing both Promise and RxJS Observable-based interfaces. This is the main user-facing entry point.
 - [@polkadot/api-derive](packages/api-derive/) Derived results that are injected into the API, allowing for combinations of various query results (only used internally and exposed on the Api instances via `api.derive.*`)
-- [@polkadot/api-metadata](packages/api-metadata/) Base extrinsic, storage and constant injectors for injection
+- [@polkadot/metadata](packages/metadata/) Base extrinsic, storage and constant injectors for injection
 - [@polkadot/rpc-core](packages/rpc-core/) Wrapper around all [JSON-RPC methods](https://polkadot.js.org/docs/substrate/rpc) exposed by a Polkadot network client
 - [@polkadot/rpc-provider](packages/rpc-provider/) Providers for connecting to nodes, including WebSockets and Http
 

package.json

@@ -8,7 +8,7 @@
     "packages/*"
   ],
   "resolutions": {
-    "typescript": "^4.2.3"
+    "typescript": "^4.2.4"
   },
   "scripts": {
     "build": "yarn build:interfaces && polkadot-dev-build-ts",
@@ -25,14 +25,14 @@
     "test:watch": "polkadot-dev-run-test --watch"
   },
   "devDependencies": {
-    "@babel/core": "^7.13.10",
-    "@babel/register": "^7.13.8",
-    "@babel/runtime": "^7.13.10",
-    "@polkadot/dev": "^0.62.6",
-    "@polkadot/ts": "^0.3.62",
+    "@babel/core": "^7.14.0",
+    "@babel/register": "^7.13.16",
+    "@babel/runtime": "^7.14.0",
+    "@polkadot/dev": "^0.62.12",
+    "@polkadot/ts": "^0.3.64",
     "@polkadot/typegen": "workspace:packages/typegen",
-    "@types/jest": "^26.0.21",
+    "@types/jest": "^26.0.23",
     "copyfiles": "^2.4.1"
   },
-  "version": "4.2.1"
+  "version": "4.9.1"
 }

packages/api-contract/README.md

@@ -8,7 +8,7 @@ import { Abi } from '@polkadot/api-contract';
 
 const wsProvider = new WsProvider(<...Node Url...>);
 const api = await ApiPromise.create({ provider: wsProvider });
-const abi = new Abi(api.registry, <...JSON ABI...>);
+const abi = new Abi(<...JSON ABI...>, api.registry.getChainProperties());
 
 api.tx.contracts
   .call(<contract addr>, <value>, <max gas>, abi.messages.<method name>(<...params...>))

packages/api-contract/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@polkadot/api-contract",
-  "version": "4.2.1",
+  "version": "4.9.1",
   "type": "module",
   "description": "Interfaces for interacting with contracts and contract ABIs",
   "main": "index.js",
@@ -8,6 +8,9 @@
     "./detectPackage.js",
     "./detectPackage.cjs"
   ],
+  "engines": {
+    "node": ">=14.0.0"
+  },
   "author": "Jaco Greeff <jacogr@gmail.com>",
   "maintainers": [],
   "contributors": [],
@@ -16,11 +19,11 @@
   "bugs": "https://github.com/polkadot-js/api/issues",
   "homepage": "https://github.com/polkadot-js/api/tree/master/packages/api-contract#readme",
   "dependencies": {
-    "@babel/runtime": "^7.13.10",
-    "@polkadot/api": "4.2.1",
-    "@polkadot/types": "4.2.1",
-    "@polkadot/util": "^6.0.5",
-    "@polkadot/x-rxjs": "^6.0.5",
+    "@babel/runtime": "^7.14.0",
+    "@polkadot/api": "4.9.1",
+    "@polkadot/types": "4.9.1",
+    "@polkadot/util": "^6.3.1",
+    "@polkadot/x-rxjs": "^6.3.1",
     "bn.js": "^4.11.9"
   }
 }

packages/api-contract/src/Abi.ts

@@ -6,7 +6,7 @@ import type { ChainProperties, ContractConstructorSpec, ContractEventSpec, Contr
 import type { AnyJson, Codec, CodecArg } from '@polkadot/types/types';
 import type { AbiConstructor, AbiEvent, AbiMessage, AbiParam, DecodedEvent, DecodedMessage } from './types';
 
-import { assert, assertReturn, compactAddLength, compactStripLength, isNumber, isObject, isString, logger, stringCamelCase, u8aConcat, u8aToHex } from '@polkadot/util';
+import { assert, assertReturn, compactAddLength, compactStripLength, isNumber, isObject, isString, logger, stringCamelCase, stringify, u8aConcat, u8aToHex } from '@polkadot/util';
 
 import { MetaRegistry } from './MetaRegistry';
 
@@ -21,7 +21,7 @@ function findMessage <T extends AbiMessage> (list: T[], messageOrId: T | string
       ? list.find(({ identifier }) => [identifier, stringCamelCase(identifier)].includes(messageOrId.toString()))
       : messageOrId;
 
-  return assertReturn(message, `Attempted to call an invalid contract interface, ${JSON.stringify(messageOrId)}`);
+  return assertReturn(message, `Attempted to call an invalid contract interface, ${stringify(messageOrId)}`);
 }
 
 export class Abi {
@@ -81,7 +81,7 @@ export class Abi {
     const index = data[0];
     const event = this.#events[index];
 
-    assert(event, `Unable to find event with index ${index}`);
+    assert(event, () => `Unable to find event with index ${index}`);
 
     return event.fromU8a(data.subarray(1));
   }
@@ -118,7 +118,7 @@ export class Abi {
           type: this.registry.getMetaTypeDef(arg.type)
         };
       } catch (error) {
-        l.error(`Error expanding argument ${index} in ${JSON.stringify(spec)}`);
+        l.error(`Error expanding argument ${index} in ${stringify(spec)}`);
 
         throw error;
       }
@@ -143,6 +143,7 @@ export class Abi {
 
   #createMessage = (spec: ContractMessageSpec | ContractConstructorSpec, index: number, add: Partial<AbiMessage> = {}): AbiMessage => {
     const args = this.#createArgs(spec.args, spec);
+    const identifier = spec.name.toString();
     const message = {
       ...add,
       args,
@@ -151,8 +152,9 @@ export class Abi {
         args: this.#decodeArgs(args, data),
         message
       }),
-      identifier: spec.name.toString(),
+      identifier,
       index,
+      method: stringCamelCase(identifier),
       selector: spec.selector,
       toU8a: (params: CodecArg[], additional?: Uint8Array) =>
         this.#encodeArgs(spec, args, params, additional)
@@ -186,7 +188,7 @@ export class Abi {
   }
 
   #encodeArgs = ({ name, selector }: ContractMessageSpec | ContractConstructorSpec, args: AbiParam[], data: CodecArg[], additional = EMPTY_U8A): Uint8Array => {
-    assert(data.length === args.length, `Expected ${args.length} arguments to contract message '${name.toString()}', found ${data.length}`);
+    assert(data.length === args.length, () => `Expected ${args.length} arguments to contract message '${name.toString()}', found ${data.length}`);
 
     return compactAddLength(
       u8aConcat(

packages/api-contract/src/MetaRegistry.ts

@@ -80,7 +80,7 @@ export class MetaRegistry extends TypeRegistry {
   #getMetaType = (id: SiLookupTypeId): SiType => {
     const type = this.#siTypes[getRegistryOffset(id)];
 
-    assert(!isUndefined(type), `getMetaType:: Unable to find ${id.toNumber()} in type values`);
+    assert(!isUndefined(type), () => `getMetaType:: Unable to find ${id.toNumber()} in type values`);
 
     return this.createType('SiType', type);
   }
@@ -184,7 +184,7 @@ export class MetaRegistry extends TypeRegistry {
   }
 
   #extractSequence = ({ type }: SiTypeDefSequence, id: SiLookupTypeId): Omit<TypeDef, 'type'> => {
-    assert(!!type, `ContractRegistry: Invalid sequence type found at id ${id.toString()}`);
+    assert(!!type, () => `ContractRegistry: Invalid sequence type found at id ${id.toString()}`);
 
     return {
       info: TypeDefInfo.Vec,

packages/api-contract/src/base/Blueprint.ts

@@ -6,17 +6,17 @@ import type { ApiTypes, DecorateMethod } from '@polkadot/api/types';
 import type { AccountId, EventRecord, Hash } from '@polkadot/types/interfaces';
 import type { AnyJson, CodecArg, ISubmittableResult } from '@polkadot/types/types';
 import type { AbiConstructor, BlueprintOptions } from '../types';
-import type { ContractGeneric, MapConstructorExec } from './types';
+import type { MapConstructorExec } from './types';
 
 import { SubmittableResult } from '@polkadot/api';
 import { ApiBase } from '@polkadot/api/base';
-import { isUndefined, stringCamelCase } from '@polkadot/util';
+import { isUndefined } from '@polkadot/util';
 
 import { Abi } from '../Abi';
 import { applyOnEvent } from '../util';
 import { Base } from './Base';
 import { Contract } from './Contract';
-import { createBluePrintTx, createBluePrintWithId, EMPTY_SALT, encodeSalt } from './util';
+import { createBluePrintTx, EMPTY_SALT, encodeSalt } from './util';
 
 export class BlueprintSubmittableResult<ApiType extends ApiTypes> extends SubmittableResult {
   public readonly contract?: Contract<ApiType>;
@@ -34,25 +34,16 @@ export class Blueprint<ApiType extends ApiTypes> extends Base<ApiType> {
    */
   public readonly codeHash: Hash;
 
-  /**
-   * @deprecated
-   * @description Deprecated. Use `.tx.<constructorName>`. Creates a contract in a non-deterministic way.
-   */
-  public readonly createContract: ContractGeneric<BlueprintOptions, SubmittableExtrinsic<ApiType, BlueprintSubmittableResult<ApiType>>>;
-
   readonly #tx: MapConstructorExec<ApiType> = {};
 
   constructor (api: ApiBase<ApiType>, abi: AnyJson | Abi, codeHash: string | Hash | Uint8Array, decorateMethod: DecorateMethod<ApiType>) {
     super(api, abi, decorateMethod);
 
     this.codeHash = this.registry.createType('Hash', codeHash);
-    this.createContract = createBluePrintWithId(this.#deploy);
 
     this.abi.constructors.forEach((c): void => {
-      const messageName = stringCamelCase(c.identifier);
-
-      if (isUndefined(this.#tx[messageName])) {
-        this.#tx[messageName] = createBluePrintTx((o, p) => this.#deploy(c, o, p));
+      if (isUndefined(this.#tx[c.method])) {
+        this.#tx[c.method] = createBluePrintTx((o, p) => this.#deploy(c, o, p));
       }
     });
   }

packages/api-contract/src/base/Code.ts

@@ -10,7 +10,7 @@ import type { MapConstructorExec } from './types';
 
 import { SubmittableResult } from '@polkadot/api';
 import { ApiBase } from '@polkadot/api/base';
-import { assert, compactAddLength, isFunction, isUndefined, isWasm, stringCamelCase, u8aToU8a } from '@polkadot/util';
+import { assert, compactAddLength, isFunction, isUndefined, isWasm, u8aToU8a } from '@polkadot/util';
 import { blake2AsU8a } from '@polkadot/util-crypto';
 
 import { Abi } from '../Abi';
@@ -47,10 +47,8 @@ export class Code<ApiType extends ApiTypes> extends Base<ApiType> {
     assert(isWasm(this.code), 'No WASM code provided');
 
     this.abi.constructors.forEach((c): void => {
-      const messageName = stringCamelCase(c.identifier);
-
-      if (isUndefined(this.#tx[messageName])) {
-        this.#tx[messageName] = createBluePrintTx((o, p) => this.#instantiate(c, o, p));
+      if (isUndefined(this.#tx[c.method])) {
+        this.#tx[c.method] = createBluePrintTx((o, p) => this.#instantiate(c, o, p));
       }
     });
   }
@@ -59,27 +57,21 @@ export class Code<ApiType extends ApiTypes> extends Base<ApiType> {
     return this.#tx;
   }
 
-  /**
-   * @description Deploy the code bundle and the contract, creating a Blueprint.
-   * @deprecated Use the `code.tx.<constructor>(...) format to put code and deploy
-   */
-  public createContract (constructorOrId: AbiConstructor | string | number, options: BlueprintOptions, params: CodecArg[]): SubmittableExtrinsic<ApiType, CodeSubmittableResult<ApiType>> {
-    return this.#instantiate(constructorOrId, options, params);
-  }
-
   #instantiate = (constructorOrId: AbiConstructor | string | number, options: BlueprintOptions, params: CodecArg[]): SubmittableExtrinsic<ApiType, CodeSubmittableResult<ApiType>> => {
     return isFunction(this.api.tx.contracts.instantiateWithCode)
-      ? this.#instantiateSingle(constructorOrId, options, params)
-      : this.#instantiateBatch(constructorOrId, options, params);
+      ? this.#instantiateCurrent(constructorOrId, options, params)
+      : this.#instantiatePrev(constructorOrId, options, params);
   }
 
-  #instantiateSingle = (constructorOrId: AbiConstructor | string | number, { gasLimit = 0, salt, value = 0 }: BlueprintOptions, params: CodecArg[]): SubmittableExtrinsic<ApiType, CodeSubmittableResult<ApiType>> => {
+  #instantiateCurrent = (constructorOrId: AbiConstructor | string | number, { gasLimit = 0, salt, value = 0 }: BlueprintOptions, params: CodecArg[]): SubmittableExtrinsic<ApiType, CodeSubmittableResult<ApiType>> => {
     return this.api.tx.contracts
       .instantiateWithCode(value, gasLimit, compactAddLength(this.code), this.abi.findConstructor(constructorOrId).toU8a(params), encodeSalt(salt))
       .withResultTransform(this.#transformEvents);
   }
 
-  #instantiateBatch = (constructorOrId: AbiConstructor | string | number, { gasLimit = 0, salt, value = 0 }: BlueprintOptions, params: CodecArg[]): SubmittableExtrinsic<ApiType, CodeSubmittableResult<ApiType>> => {
+  #instantiatePrev = (constructorOrId: AbiConstructor | string | number, { gasLimit = 0, salt, value = 0 }: BlueprintOptions, params: CodecArg[]): SubmittableExtrinsic<ApiType, CodeSubmittableResult<ApiType>> => {
+    assert(isFunction(this.api.tx.utility?.batch), 'Your chain does not include the utility pallet, for contracts v2 deployment, this is required');
+
     const encodedSalt = encodeSalt(salt);
     const withSalt = this.api.tx.contracts.instantiate.meta.args.length === 5;
     const encoded = this.abi.findConstructor(constructorOrId).toU8a(params, withSalt ? EMPTY_SALT : encodedSalt);

packages/api-contract/src/base/Contract.ts

@@ -7,13 +7,13 @@ import type { Bytes } from '@polkadot/types';
 import type { AccountId, ContractExecResult, EventRecord, Weight } from '@polkadot/types/interfaces';
 import type { AnyJson, CodecArg, ISubmittableResult, Registry } from '@polkadot/types/types';
 import type { AbiMessage, ContractCallOutcome, ContractOptions, DecodedEvent } from '../types';
-import type { ContractCallResult, ContractCallSend, ContractGeneric, ContractQuery, ContractTx, MapMessageQuery, MapMessageTx } from './types';
+import type { ContractCallResult, ContractCallSend, ContractQuery, ContractTx, MapMessageQuery, MapMessageTx } from './types';
 
 import BN from 'bn.js';
 
 import { SubmittableResult } from '@polkadot/api';
 import { ApiBase } from '@polkadot/api/base';
-import { assert, bnToBn, isFunction, isUndefined, logger, stringCamelCase } from '@polkadot/util';
+import { assert, bnToBn, isFunction, isUndefined, logger } from '@polkadot/util';
 import { map } from '@polkadot/x-rxjs/operators';
 
 import { Abi } from '../Abi';
@@ -23,6 +23,7 @@ import { Base } from './Base';
 // As per Rust, 5 * GAS_PER_SEC
 const MAX_CALL_GAS = new BN(5_000_000_000_000).subn(1);
 const ERROR_NO_CALL = 'Your node does not expose the contracts.call RPC. This is most probably due to a runtime configuration.';
+
 const l = logger('Contract');
 
 function createQuery <ApiType extends ApiTypes> (fn: (origin: string | AccountId | Uint8Array, options: ContractOptions, params: CodecArg[]) => ContractCallResult<ApiType, ContractCallOutcome>): ContractQuery<ApiType> {
@@ -39,13 +40,6 @@ function createTx <ApiType extends ApiTypes> (fn: (options: ContractOptions, par
       : fn(...extractOptions(options, params));
 }
 
-function createWithId <T> (fn: (messageOrId: AbiMessage | string | number, options: ContractOptions, params: CodecArg[]) => T): ContractGeneric<ContractOptions, T> {
-  return (messageOrId: AbiMessage | string | number, options: BigInt | string | number | BN | ContractOptions, ...params: CodecArg[]): T =>
-    isOptions(options)
-      ? fn(messageOrId, options, params)
-      : fn(messageOrId, ...extractOptions(options, params));
-}
-
 export class ContractSubmittableResult extends SubmittableResult {
   public readonly contractEvents?: DecodedEvent[];
 
@@ -89,18 +83,6 @@ export class Contract<ApiType extends ApiTypes> extends Base<ApiType> {
    */
   public readonly address: AccountId;
 
-  /**
-   * @deprecated
-   * @description Deprecated. Use `.tx.<messageName>` to send a transaction.
-   */
-  public readonly exec: ContractGeneric<ContractOptions, SubmittableExtrinsic<ApiType>>;
-
-  /**
-   * @deprecated
-   * @description Deprecated. Use `.tx.<messageName>` to send a transaction.
-   */
-  public readonly read: ContractGeneric<ContractOptions, ContractCallSend<ApiType>>;
-
   readonly #query: MapMessageQuery<ApiType> = {};
 
   readonly #tx: MapMessageTx<ApiType> = {};
@@ -109,18 +91,14 @@ export class Contract<ApiType extends ApiTypes> extends Base<ApiType> {
     super(api, abi, decorateMethod);
 
     this.address = this.registry.createType('AccountId', address);
-    this.exec = createWithId(this.#exec);
-    this.read = createWithId(this.#read);
 
     this.abi.messages.forEach((m): void => {
-      const messageName = stringCamelCase(m.identifier);
-
-      if (isUndefined(this.#tx[messageName])) {
-        this.#tx[messageName] = createTx((o, p) => this.#exec(m, o, p));
+      if (isUndefined(this.#tx[m.method])) {
+        this.#tx[m.method] = createTx((o, p) => this.#exec(m, o, p));
       }
 
-      if (isUndefined(this.#query[messageName])) {
-        this.#query[messageName] = createQuery((f, o, p) => this.#read(m, o, p).send(f));
+      if (isUndefined(this.#query[m.method])) {
+        this.#query[m.method] = createQuery((f, o, p) => this.#read(m, o, p).send(f));
       }
     });
   }

packages/api-contract/src/checkTypes.manual.ts

@@ -30,10 +30,6 @@ async function checkContract (api: ApiPromise, pairs: TestKeyringMap): Promise<v
   // execute
   await contract.tx.inc({ gasLimit: 1234 }, 123).signAndSend(pairs.eve);
   await contract.tx.inc(123, 456, 69).signAndSend(pairs.eve);
-
-  // deprecated
-  await contract.read('get', {}).send(pairs.eve.address);
-  await contract.exec('inc', {}, 123).signAndSend(pairs.dave);
 }
 
 async function main (): Promise<void> {

packages/api-contract/src/packageInfo.ts

@@ -3,4 +3,4 @@
 
 // Auto-generated by @polkadot/dev, do not edit
 
-export const packageInfo = { name: '@polkadot/api-contract', version: '4.2.1' };
+export const packageInfo = { name: '@polkadot/api-contract', version: '4.9.1' };

packages/api-contract/src/types.ts

@@ -39,6 +39,7 @@ export interface AbiMessage {
   isConstructor?: boolean;
   isMutating?: boolean;
   isPayable?: boolean;
+  method: string;
   returnType?: TypeDef | null;
   selector: ContractSelector;
   toU8a: (params: CodecArg[], additional?: Uint8Array) => Uint8Array;

packages/api-derive/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@polkadot/api-derive",
-  "version": "4.2.1",
+  "version": "4.9.1",
   "type": "module",
   "description": "Common functions used across Polkadot, derived from RPC calls and storage queries.",
   "main": "index.js",
@@ -8,6 +8,9 @@
     "./detectPackage.js",
     "./detectPackage.cjs"
   ],
+  "engines": {
+    "node": ">=14.0.0"
+  },
   "author": "Jaco Greeff <jacogr@gmail.com>",
   "maintainers": [],
   "contributors": [],
@@ -16,17 +19,17 @@
   "bugs": "https://github.com/polkadot-js/api/issues",
   "homepage": "https://github.com/polkadot-js/api/tree/master/packages/api-derive#readme",
   "dependencies": {
-    "@babel/runtime": "^7.13.10",
-    "@polkadot/api": "4.2.1",
-    "@polkadot/rpc-core": "4.2.1",
-    "@polkadot/types": "4.2.1",
-    "@polkadot/util": "^6.0.5",
-    "@polkadot/util-crypto": "^6.0.5",
-    "@polkadot/x-rxjs": "^6.0.5",
+    "@babel/runtime": "^7.14.0",
+    "@polkadot/api": "4.9.1",
+    "@polkadot/rpc-core": "4.9.1",
+    "@polkadot/types": "4.9.1",
+    "@polkadot/util": "^6.3.1",
+    "@polkadot/util-crypto": "^6.3.1",
+    "@polkadot/x-rxjs": "^6.3.1",
     "bn.js": "^4.11.9"
   },
   "devDependencies": {
-    "@polkadot/keyring": "^6.0.5",
-    "@polkadot/rpc-provider": "4.2.1"
+    "@polkadot/keyring": "^6.3.1",
+    "@polkadot/rpc-provider": "4.9.1"
   }
 }

packages/api-derive/src/bounties/bounties.ts

@@ -37,12 +37,16 @@ export function bounties (instanceId: string, api: ApiInterfaceRx): () => Observ
   return memo(instanceId, (): Observable<DeriveBounties> =>
     combineLatest([
       bountyBase.bountyCount<BountyIndex>(),
-      api.query.council ? api.query.council.proposalCount<ProposalIndex>() : of(0)
+      api.query.council
+        ? api.query.council.proposalCount<ProposalIndex>()
+        : of(0)
     ]).pipe(
       switchMap(() =>
         combineLatest([
           bountyBase.bounties.keys<[BountyIndex]>(),
-          api.derive.council ? api.derive.council.proposals() : of([])
+          api.derive.council
+            ? api.derive.council.proposals()
+            : of([])
         ])
       ),
       switchMap(([keys, proposals]): Observable<Result> => {

packages/api-derive/src/collective/proposals.ts

@@ -13,6 +13,8 @@ import { catchError, map, switchMap } from '@polkadot/x-rxjs/operators';
 
 import { memo } from '../util';
 
+type Collective = 'council' | 'membership' | 'technicalCommittee';
+
 type Result = [(Hash | Uint8Array | string)[], (Option<Proposal> | null)[], Option<Votes>[]];
 
 function parse (api: ApiInterfaceRx, [hashes, proposals, votes]: Result): DeriveCollectiveProposal[] {
@@ -29,7 +31,7 @@ function parse (api: ApiInterfaceRx, [hashes, proposals, votes]: Result): Derive
     .filter((proposal): proposal is DeriveCollectiveProposal => !!proposal);
 }
 
-function _proposalsFrom (instanceId: string, api: ApiInterfaceRx, section: 'council' | 'technicalCommittee' = 'council'): (hashes: (Hash | Uint8Array | string)[]) => Observable<DeriveCollectiveProposal[]> {
+function _proposalsFrom (instanceId: string, api: ApiInterfaceRx, section: Collective): (hashes: (Hash | Uint8Array | string)[]) => Observable<DeriveCollectiveProposal[]> {
   return memo(instanceId, (hashes: (Hash | Uint8Array | string)[]): Observable<DeriveCollectiveProposal[]> =>
     (isFunction(api.query[section]?.proposals) && hashes.length
       ? combineLatest<Result>([
@@ -49,19 +51,19 @@ function _proposalsFrom (instanceId: string, api: ApiInterfaceRx, section: 'coun
   );
 }
 
-export function proposals (instanceId: string, api: ApiInterfaceRx, section: 'council' | 'technicalCommittee' = 'council'): () => Observable<DeriveCollectiveProposal[]> {
+export function proposals (instanceId: string, api: ApiInterfaceRx, section: Collective): () => Observable<DeriveCollectiveProposal[]> {
   const proposalsFrom = _proposalsFrom(instanceId, api, section);
 
   return memo(instanceId, (): Observable<DeriveCollectiveProposal[]> =>
     isFunction(api.query[section]?.proposals)
-      ? api.query[section].proposals().pipe(
+      ? api.query[section as 'council'].proposals().pipe(
         switchMap(proposalsFrom)
       )
       : of([] as DeriveCollectiveProposal[])
   );
 }
 
-export function proposal (instanceId: string, api: ApiInterfaceRx, section: 'council' | 'technicalCommittee' = 'council'): (hash: Hash | Uint8Array | string) => Observable<DeriveCollectiveProposal | null> {
+export function proposal (instanceId: string, api: ApiInterfaceRx, section: Collective): (hash: Hash | Uint8Array | string) => Observable<DeriveCollectiveProposal | null> {
   const proposalsFrom = _proposalsFrom(instanceId, api, section);
 
   return memo(instanceId, (hash: Hash | Uint8Array | string): Observable<DeriveCollectiveProposal | null> =>

packages/api-derive/src/council/proposals.ts

@@ -10,9 +10,9 @@ import { proposal as collectiveProposal, proposals as collectiveProposals } from
 import { memo } from '../util';
 
 export function proposal (instanceId: string, api: ApiInterfaceRx): (hash: Hash | Uint8Array | string) => Observable<DeriveCollectiveProposal | null> {
-  return memo(instanceId, collectiveProposal(instanceId, api));
+  return memo(instanceId, collectiveProposal(instanceId, api, 'council'));
 }
 
 export function proposals (instanceId: string, api: ApiInterfaceRx): () => Observable<DeriveCollectiveProposal[]> {
-  return memo(instanceId, collectiveProposals(instanceId, api));
+  return memo(instanceId, collectiveProposals(instanceId, api, 'council'));
 }

packages/api-derive/src/index.ts

@@ -16,6 +16,7 @@ import * as council from './council';
 import * as democracy from './democracy';
 import * as elections from './elections';
 import * as imOnline from './imOnline';
+import * as membership from './membership';
 import * as parachains from './parachains';
 import * as session from './session';
 import * as society from './society';
@@ -27,7 +28,7 @@ import * as tx from './tx';
 export { packageInfo } from './packageInfo';
 export * from './type';
 
-export const derive = { accounts, balances, bounties, chain, contracts, council, democracy, elections, imOnline, parachains, session, society, staking, technicalCommittee, treasury, tx };
+export const derive = { accounts, balances, bounties, chain, contracts, council, democracy, elections, imOnline, membership, parachains, session, society, staking, technicalCommittee, treasury, tx };
 
 type DeriveSection<Section> = {
   [Method in keyof Section]: Section[Method] extends AnyFunction
@@ -49,6 +50,7 @@ const deriveAvail: Record<string, string[]> = {
   democracy: ['democracy'],
   elections: ['electionsPhragmen', 'elections'],
   imOnline: ['imOnline'],
+  membership: ['membership'],
   parachains: ['parachains', 'registrar'],
   session: ['session'],
   society: ['society'],

packages/api-derive/src/membership/index.ts

@@ -0,0 +1,4 @@
+// Copyright 2017-2021 @polkadot/api-derive authors & contributors
+// SPDX-License-Identifier: Apache-2.0
+
+export * from './proposals';

packages/api-derive/src/membership/proposals.ts

@@ -0,0 +1,13 @@
+// Copyright 2017-2021 @polkadot/api-derive authors & contributors
+// SPDX-License-Identifier: Apache-2.0
+
+import type { ApiInterfaceRx } from '@polkadot/api/types';
+import type { Observable } from '@polkadot/x-rxjs';
+import type { DeriveCollectiveProposal } from '../types';
+
+import { proposals as collectiveProposals } from '../collective';
+import { memo } from '../util';
+
+export function proposals (instanceId: string, api: ApiInterfaceRx): () => Observable<DeriveCollectiveProposal[]> {
+  return memo(instanceId, collectiveProposals(instanceId, api, 'membership'));
+}

packages/api-derive/src/packageInfo.ts

@@ -3,4 +3,4 @@
 
 // Auto-generated by @polkadot/dev, do not edit
 
-export const packageInfo = { name: '@polkadot/api-derive', version: '4.2.1' };
+export const packageInfo = { name: '@polkadot/api-derive', version: '4.9.1' };

packages/api-derive/src/session/indexes.ts

@@ -13,7 +13,7 @@ import { map } from '@polkadot/x-rxjs/operators';
 import { memo } from '../util';
 
 // parse into Indexes
-function parse ([activeEra, activeEraStart, currentEra, currentIndex, validatorCount]: [EraIndex, Option<Moment>, EraIndex, SessionIndex, u32]): DeriveSessionIndexes {
+function parse ([currentIndex, activeEra, activeEraStart, currentEra, validatorCount]: [SessionIndex, EraIndex, Option<Moment>, EraIndex, u32]): DeriveSessionIndexes {
   return {
     activeEra,
     activeEraStart,
@@ -24,42 +24,57 @@ function parse ([activeEra, activeEraStart, currentEra, currentIndex, validatorC
 }
 
 // query based on latest
-function query (api: ApiInterfaceRx): Observable<DeriveSessionIndexes> {
-  return api.queryMulti<[Option<ActiveEraInfo>, Option<EraIndex>, SessionIndex, u32]>([
+function queryStaking (api: ApiInterfaceRx): Observable<DeriveSessionIndexes> {
+  return api.queryMulti<[SessionIndex, Option<ActiveEraInfo>, Option<EraIndex>, u32]>([
+    api.query.session.currentIndex,
     api.query.staking.activeEra,
     api.query.staking.currentEra,
-    api.query.session.currentIndex,
     api.query.staking.validatorCount
   ]).pipe(
-    map(([activeOpt, currentEra, currentIndex, validatorCount]): DeriveSessionIndexes => {
+    map(([currentIndex, activeOpt, currentEra, validatorCount]): DeriveSessionIndexes => {
       const { index, start } = activeOpt.unwrapOrDefault();
 
       return parse([
+        currentIndex,
         index,
         start,
         currentEra.unwrapOrDefault(),
-        currentIndex,
         validatorCount
       ]);
     })
   );
 }
 
+// query based on latest
+function querySession (api: ApiInterfaceRx): Observable<DeriveSessionIndexes> {
+  return api.query.session.currentIndex().pipe(
+    map((currentIndex): DeriveSessionIndexes => parse([
+      currentIndex,
+      api.registry.createType('EraIndex'),
+      api.registry.createType('Option<Moment>'),
+      api.registry.createType('EraIndex'),
+      api.registry.createType('u32')
+    ]))
+  );
+}
+
 // empty set when none is available
 function empty (api: ApiInterfaceRx): Observable<DeriveSessionIndexes> {
   return of(parse([
+    api.registry.createType('SessionIndex', 1),
     api.registry.createType('EraIndex'),
     api.registry.createType('Option<Moment>'),
     api.registry.createType('EraIndex'),
-    api.registry.createType('SessionIndex', 1),
     api.registry.createType('u32')
   ]));
 }
 
 export function indexes (instanceId: string, api: ApiInterfaceRx): () => Observable<DeriveSessionIndexes> {
   return memo(instanceId, (): Observable<DeriveSessionIndexes> =>
-    api.query.session && api.query.staking
-      ? query(api)
+    api.query.session
+      ? api.query.staking
+        ? queryStaking(api)
+        : querySession(api)
       : empty(api)
   );
 }

packages/api-derive/src/session/info.ts

@@ -2,71 +2,31 @@
 // SPDX-License-Identifier: Apache-2.0
 
 import type { ApiInterfaceRx } from '@polkadot/api/types';
-import type { u64 } from '@polkadot/types';
-import type { SessionIndex } from '@polkadot/types/interfaces';
 import type { Observable } from '@polkadot/x-rxjs';
-import type { DeriveSessionIndexes, DeriveSessionInfo } from '../types';
+import type { DeriveSessionInfo } from '../types';
 
 import { map } from '@polkadot/x-rxjs/operators';
 
 import { memo } from '../util';
 
-type ResultType = [boolean, u64, SessionIndex];
-type Result = [ResultType, DeriveSessionIndexes];
-
-function createDerive (api: ApiInterfaceRx, [[hasBabe, epochDuration, sessionsPerEra], { activeEra, activeEraStart, currentEra, currentIndex, validatorCount }]: Result): DeriveSessionInfo {
-  return {
-    activeEra,
-    activeEraStart,
-    currentEra,
-    currentIndex,
-    eraLength: api.registry.createType('BlockNumber', sessionsPerEra.mul(epochDuration)),
-    isEpoch: hasBabe,
-    sessionLength: epochDuration,
-    sessionsPerEra,
-    validatorCount
-  };
-}
-
-function queryAura (api: ApiInterfaceRx): Observable<DeriveSessionInfo> {
-  return api.derive.session.indexes().pipe(
-    map((indexes): DeriveSessionInfo =>
-      createDerive(api, [
-        [
-          false,
-          api.registry.createType('u64', 1),
-          // we may have aura without staking (permissioned)
-          api.consts.staking?.sessionsPerEra || api.registry.createType('SessionIndex', 1)
-        ],
-        indexes
-      ])
-    )
-  );
-}
-
-function queryBabe (api: ApiInterfaceRx): Observable<DeriveSessionInfo> {
-  return api.derive.session.indexes().pipe(
-    map((indexes) =>
-      createDerive(api, [
-        [
-          true,
-          api.consts.babe.epochDuration,
-          // we may have babe without staking (permissioned)
-          api.consts.staking?.sessionsPerEra || api.registry.createType('SessionIndex', 1)
-        ],
-        indexes
-      ])
-    )
-  );
-}
-
 /**
  * @description Retrieves all the session and era query and calculates specific values on it as the length of the session and eras
  */
 export function info (instanceId: string, api: ApiInterfaceRx): () => Observable<DeriveSessionInfo> {
   return memo(instanceId, (): Observable<DeriveSessionInfo> =>
-    api.consts.babe
-      ? queryBabe(api)
-      : queryAura(api)
+    api.derive.session.indexes().pipe(
+      map((indexes) => {
+        const sessionLength = api.consts?.babe?.epochDuration || api.registry.createType('u64', 1);
+        const sessionsPerEra = api.consts?.staking?.sessionsPerEra || api.registry.createType('SessionIndex', 1);
+
+        return {
+          ...indexes,
+          eraLength: api.registry.createType('BlockNumber', sessionsPerEra.mul(sessionLength)),
+          isEpoch: !!api.query.babe,
+          sessionLength,
+          sessionsPerEra
+        };
+      })
+    )
   );
 }

packages/api-derive/src/session/progress.ts

@@ -44,7 +44,7 @@ function queryBabe (api: ApiInterfaceRx): Observable<[DeriveSessionInfo, ResultS
       combineLatest([
         of(info),
         // we may have no staking, but have babe (permissioned)
-        api.query.staking
+        api.query.staking?.erasStartSessionIndex
           ? api.queryMulti<ResultSlots>([
             api.query.babe.currentSlot,
             api.query.babe.epochIndex,
@@ -69,7 +69,7 @@ function queryBabe (api: ApiInterfaceRx): Observable<[DeriveSessionInfo, ResultS
  */
 export function progress (instanceId: string, api: ApiInterfaceRx): () => Observable<DeriveSessionProgress> {
   return memo(instanceId, (): Observable<DeriveSessionProgress> =>
-    api.consts.babe
+    api.query.babe
       ? queryBabe(api).pipe(
         map(([info, slots]: [DeriveSessionInfo, ResultSlotsFlat]): DeriveSessionProgress =>
           createDerive(api, info, slots)

packages/api-derive/src/treasury/proposals.ts

@@ -61,7 +61,9 @@ function retrieveProposals (api: ApiInterfaceRx, proposalCount: ProposalIndex, a
 
   return combineLatest([
     api.query.treasury.proposals.multi<Option<TreasuryProposal>>(allIds),
-    api.derive.council.proposals()
+    api.derive.council
+      ? api.derive.council.proposals()
+      : of([] as DeriveCollectiveProposal[])
   ]).pipe(
     map(([allProposals, councilProposals]: [Option<TreasuryProposal>[], DeriveCollectiveProposal[]]): DeriveTreasuryProposals =>
       parseResult(api, { allIds, allProposals, approvalIds, councilProposals, proposalCount })

packages/api/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@polkadot/api",
-  "version": "4.2.1",
+  "version": "4.9.1",
   "type": "module",
   "description": "Promise and RxJS wrappers around the Polkadot JS RPC",
   "main": "index.js",
@@ -8,6 +8,9 @@
     "./detectPackage.js",
     "./detectPackage.cjs"
   ],
+  "engines": {
+    "node": ">=14.0.0"
+  },
   "author": "Jaco Greeff <jacogr@gmail.com>",
   "maintainers": [],
   "contributors": [],
@@ -16,17 +19,17 @@
   "bugs": "https://github.com/polkadot-js/api/issues",
   "homepage": "https://github.com/polkadot-js/api/tree/master/packages/api#readme",
   "dependencies": {
-    "@babel/runtime": "^7.13.10",
-    "@polkadot/api-derive": "4.2.1",
-    "@polkadot/keyring": "^6.0.5",
-    "@polkadot/metadata": "4.2.1",
-    "@polkadot/rpc-core": "4.2.1",
-    "@polkadot/rpc-provider": "4.2.1",
-    "@polkadot/types": "4.2.1",
-    "@polkadot/types-known": "4.2.1",
-    "@polkadot/util": "^6.0.5",
-    "@polkadot/util-crypto": "^6.0.5",
-    "@polkadot/x-rxjs": "^6.0.5",
+    "@babel/runtime": "^7.14.0",
+    "@polkadot/api-derive": "4.9.1",
+    "@polkadot/keyring": "^6.3.1",
+    "@polkadot/metadata": "4.9.1",
+    "@polkadot/rpc-core": "4.9.1",
+    "@polkadot/rpc-provider": "4.9.1",
+    "@polkadot/types": "4.9.1",
+    "@polkadot/types-known": "4.9.1",
+    "@polkadot/util": "^6.3.1",
+    "@polkadot/util-crypto": "^6.3.1",
+    "@polkadot/x-rxjs": "^6.3.1",
     "bn.js": "^4.11.9",
     "eventemitter3": "^4.0.7"
   }

packages/api/src/augment/consts.ts

@@ -3,7 +3,7 @@
 
 import type { Vec, u16, u32, u64, u8 } from '@polkadot/types';
 import type { Codec } from '@polkadot/types/types';
-import type { Balance, BalanceOf, BlockNumber, LockIdentifier, ModuleId, Moment, Perbill, Percent, Permill, RuntimeDbWeight, Weight } from '@polkadot/types/interfaces/runtime';
+import type { Balance, BalanceOf, BlockNumber, LockIdentifier, Moment, PalletId, Perbill, Percent, Permill, RuntimeDbWeight, Weight } from '@polkadot/types/interfaces/runtime';
 import type { SessionIndex } from '@polkadot/types/interfaces/session';
 import type { EraIndex } from '@polkadot/types/interfaces/staking';
 import type { RuntimeVersion } from '@polkadot/types/interfaces/state';
@@ -184,6 +184,13 @@ declare module '@polkadot/api/types/consts' {
     };
     electionProviderMultiPhase: {
       [key: string]: Codec;
+      /**
+       * The repeat threshold of the offchain worker.
+       * 
+       * For example, if it is 5, that means that at least 5 blocks will elapse between attempts
+       * to submit the worker's solution.
+       **/
+      offchainRepeat: BlockNumber & AugmentedConst<ApiType>;
       /**
        * Duration of the signed phase.
        **/
@@ -200,12 +207,38 @@ declare module '@polkadot/api/types/consts' {
     };
     elections: {
       [key: string]: Codec;
+      /**
+       * How much should be locked up in order to submit one's candidacy.
+       **/
       candidacyBond: BalanceOf & AugmentedConst<ApiType>;
+      /**
+       * Number of members to elect.
+       **/
       desiredMembers: u32 & AugmentedConst<ApiType>;
+      /**
+       * Number of runners_up to keep.
+       **/
       desiredRunnersUp: u32 & AugmentedConst<ApiType>;
-      moduleId: LockIdentifier & AugmentedConst<ApiType>;
+      /**
+       * Identifier for the elections-phragmen pallet's lock
+       **/
+      palletId: LockIdentifier & AugmentedConst<ApiType>;
+      /**
+       * How long each seat is kept. This defines the next block number at which an election
+       * round will happen. If set to zero, no elections are ever triggered and the module will
+       * be in passive mode.
+       **/
       termDuration: BlockNumber & AugmentedConst<ApiType>;
+      /**
+       * Base deposit associated with voting.
+       * 
+       * This should be sensibly high to economically ensure the pallet cannot be attacked by
+       * creating a gigantic number of votes.
+       **/
       votingBondBase: BalanceOf & AugmentedConst<ApiType>;
+      /**
+       * The amount of bond that need to be locked for each vote (32 bytes).
+       **/
       votingBondFactor: BalanceOf & AugmentedConst<ApiType>;
     };
     gilt: {
@@ -257,7 +290,7 @@ declare module '@polkadot/api/types/consts' {
     identity: {
       [key: string]: Codec;
       /**
-       * The amount held on deposit for a registered identity.
+       * The amount held on deposit for a registered identity
        **/
       basicDeposit: BalanceOf & AugmentedConst<ApiType>;
       /**
@@ -295,7 +328,7 @@ declare module '@polkadot/api/types/consts' {
     lottery: {
       [key: string]: Codec;
       maxCalls: u32 & AugmentedConst<ApiType>;
-      moduleId: ModuleId & AugmentedConst<ApiType>;
+      palletId: PalletId & AugmentedConst<ApiType>;
     };
     multisig: {
       [key: string]: Codec;
@@ -381,6 +414,10 @@ declare module '@polkadot/api/types/consts' {
        * The number of blocks between membership challenges.
        **/
       challengePeriod: BlockNumber & AugmentedConst<ApiType>;
+      /**
+       * Maximum candidate intake per round.
+       **/
+      maxCandidateIntake: u32 & AugmentedConst<ApiType>;
       /**
        * The number of times a member may vote the wrong way (or not at all, when they are a skeptic)
        * before they become suspended.
@@ -389,7 +426,7 @@ declare module '@polkadot/api/types/consts' {
       /**
        * The societies's module id
        **/
-      moduleId: ModuleId & AugmentedConst<ApiType>;
+      palletId: PalletId & AugmentedConst<ApiType>;
       /**
        * The amount of incentive paid within each period. Doesn't include VoterTip.
        **/
@@ -410,6 +447,10 @@ declare module '@polkadot/api/types/consts' {
        * Number of eras that staked funds must remain bonded for.
        **/
       bondingDuration: EraIndex & AugmentedConst<ApiType>;
+      /**
+       * Maximum number of nominations per nominator.
+       **/
+      maxNominations: u32 & AugmentedConst<ApiType>;
       /**
        * The maximum number of nominators rewarded for each validator.
        * 
@@ -514,7 +555,7 @@ declare module '@polkadot/api/types/consts' {
       /**
        * The treasury's module id, used for deriving its sovereign account ID.
        **/
-      moduleId: ModuleId & AugmentedConst<ApiType>;
+      palletId: PalletId & AugmentedConst<ApiType>;
       /**
        * Fraction of a proposal's value that should be bonded in order to place the proposal.
        * An accepted proposal gets these back. A rejected proposal does not.
@@ -532,7 +573,7 @@ declare module '@polkadot/api/types/consts' {
     vesting: {
       [key: string]: Codec;
       /**
-       * The minimum amount to be transferred to create a new vesting schedule.
+       * The minimum amount transferred to call `vested_transfer`.
        **/
       minVestedTransfer: BalanceOf & AugmentedConst<ApiType>;
     };

packages/api/src/augment/errors.ts

@@ -512,6 +512,10 @@ declare module '@polkadot/api/types/errors' {
     };
     electionProviderMultiPhase: {
       [key: string]: AugmentedError<ApiType>;
+      /**
+       * OCW submitted solution for wrong round
+       **/
+      OcwCallWrongEra: AugmentedError<ApiType>;
       /**
        * Submission was too early.
        **/
@@ -743,6 +747,64 @@ declare module '@polkadot/api/types/errors' {
        **/
       InvalidKey: AugmentedError<ApiType>;
     };
+    indices: {
+      [key: string]: AugmentedError<ApiType>;
+      /**
+       * The index was not available.
+       **/
+      InUse: AugmentedError<ApiType>;
+      /**
+       * The index was not already assigned.
+       **/
+      NotAssigned: AugmentedError<ApiType>;
+      /**
+       * The index is assigned to another account.
+       **/
+      NotOwner: AugmentedError<ApiType>;
+      /**
+       * The source and destination accounts are identical.
+       **/
+      NotTransfer: AugmentedError<ApiType>;
+      /**
+       * The index is permanent and may not be freed/changed.
+       **/
+      Permanent: AugmentedError<ApiType>;
+    };
+    lottery: {
+      [key: string]: AugmentedError<ApiType>;
+      /**
+       * A lottery has already ended.
+       **/
+      AlreadyEnded: AugmentedError<ApiType>;
+      /**
+       * You are already participating in the lottery with this call.
+       **/
+      AlreadyParticipating: AugmentedError<ApiType>;
+      /**
+       * Failed to encode calls
+       **/
+      EncodingFailed: AugmentedError<ApiType>;
+      /**
+       * A lottery is already in progress.
+       **/
+      InProgress: AugmentedError<ApiType>;
+      /**
+       * The call is not valid for an open lottery.
+       **/
+      InvalidCall: AugmentedError<ApiType>;
+      /**
+       * A lottery has not been configured.
+       **/
+      NotConfigured: AugmentedError<ApiType>;
+      /**
+       * An overflow has occurred.
+       **/
+      Overflow: AugmentedError<ApiType>;
+      /**
+       * Too many calls for a single lottery.
+       **/
+      TooManyCalls: AugmentedError<ApiType>;
+    };
     multisig: {
       [key: string]: AugmentedError<ApiType>;
       /**
@@ -1185,6 +1247,17 @@ declare module '@polkadot/api/types/errors' {
        **/
       WrongProposalWeight: AugmentedError<ApiType>;
     };
+    technicalMembership: {
+      [key: string]: AugmentedError<ApiType>;
+      /**
+       * Already a member.
+       **/
+      AlreadyMember: AugmentedError<ApiType>;
+      /**
+       * Not a member.
+       **/
+      NotMember: AugmentedError<ApiType>;
+    };
     tips: {
       [key: string]: AugmentedError<ApiType>;
       /**
@@ -1222,6 +1295,10 @@ declare module '@polkadot/api/types/errors' {
        * No proposal or bounty at that index.
        **/
       InvalidIndex: AugmentedError<ApiType>;
+      /**
+       * Too many approvals in the queue.
+       **/
+      TooManyApprovals: AugmentedError<ApiType>;
     };
     vesting: {
       [key: string]: AugmentedError<ApiType>;

packages/api/src/augment/events.ts

@@ -418,10 +418,11 @@ declare module '@polkadot/api/types/events' {
        **/
       MemberKicked: AugmentedEvent<ApiType, [AccountId]>;
       /**
-       * A new term with \[new_members\]. This indicates that enough candidates existed to run the
-       * election, not that enough have has been elected. The inner value must be examined for
-       * this purpose. A `NewTerm(\[\])` indicates that some candidates got their bond slashed and
-       * none were elected, whilst `EmptyTerm` means that no candidates existed to begin with.
+       * A new term with \[new_members\]. This indicates that enough candidates existed to run
+       * the election, not that enough have has been elected. The inner value must be examined
+       * for this purpose. A `NewTerm(\[\])` indicates that some candidates got their bond
+       * slashed and none were elected, whilst `EmptyTerm` means that no candidates existed to
+       * begin with.
        **/
       NewTerm: AugmentedEvent<ApiType, [Vec<ITuple<[AccountId, Balance]>>]>;
       /**
@@ -589,11 +590,10 @@ declare module '@polkadot/api/types/events' {
       [key: string]: AugmentedEvent<ApiType>;
       /**
        * There is an offence reported of the given `kind` happened at the `session_index` and
-       * (kind-specific) time slot. This event is not deposited for duplicate slashes. last
-       * element indicates of the offence was applied (true) or queued (false)
-       * \[kind, timeslot, applied\].
+       * (kind-specific) time slot. This event is not deposited for duplicate slashes.
+       * \[kind, timeslot\].
        **/
-      Offence: AugmentedEvent<ApiType, [Kind, OpaqueTimeSlot, bool]>;
+      Offence: AugmentedEvent<ApiType, [Kind, OpaqueTimeSlot]>;
     };
     proxy: {
       [key: string]: AugmentedEvent<ApiType>;

packages/api/src/augment/query.ts

@@ -17,7 +17,7 @@ import type { SetId, StoredPendingChange, StoredState } from '@polkadot/types/in
 import type { RegistrarInfo, Registration } from '@polkadot/types/interfaces/identity';
 import type { AuthIndex } from '@polkadot/types/interfaces/imOnline';
 import type { CallIndex, LotteryConfig } from '@polkadot/types/interfaces/lottery';
-import type { DeferredOffenceOf, Kind, OffenceDetails, OpaqueTimeSlot, ReportIdOf } from '@polkadot/types/interfaces/offences';
+import type { Kind, OffenceDetails, OpaqueTimeSlot, ReportIdOf } from '@polkadot/types/interfaces/offences';
 import type { ProxyAnnouncement, ProxyDefinition } from '@polkadot/types/interfaces/proxy';
 import type { ActiveRecovery, RecoveryConfig } from '@polkadot/types/interfaces/recovery';
 import type { AccountId, AccountIndex, AssetId, Balance, BalanceOf, BlockNumber, Hash, KeyTypeId, Moment, OpaqueCall, Perbill, Releases, Slot, ValidatorId } from '@polkadot/types/interfaces/runtime';
@@ -600,11 +600,6 @@ declare module '@polkadot/api/types/storage' {
        * A vector of reports of the same kind that happened at the same time slot.
        **/
       concurrentReportsIndex: AugmentedQueryDoubleMap<ApiType, (key1: Kind | string | Uint8Array, key2: OpaqueTimeSlot | string | Uint8Array) => Observable<Vec<ReportIdOf>>, [Kind, OpaqueTimeSlot]> & QueryableStorageEntry<ApiType, [Kind, OpaqueTimeSlot]>;
-      /**
-       * Deferred reports that have been rejected by the offence handler and need to be submitted
-       * at a later time.
-       **/
-      deferredOffences: AugmentedQuery<ApiType, () => Observable<Vec<DeferredOffenceOf>>, []> & QueryableStorageEntry<ApiType, []>;
       /**
        * The primary structure that holds all offence records keyed by report identifiers.
        **/

packages/api/src/augment/rpc.ts

@@ -6,10 +6,11 @@ import type { Bytes, HashMap, Json, Null, Option, StorageKey, Text, U256, U64, V
 import type { AnyNumber, Codec, IExtrinsic, Observable } from '@polkadot/types/types';
 import type { ExtrinsicOrHash, ExtrinsicStatus } from '@polkadot/types/interfaces/author';
 import type { EpochAuthorship } from '@polkadot/types/interfaces/babe';
+import type { BeefySignedCommitment } from '@polkadot/types/interfaces/beefy';
 import type { BlockHash } from '@polkadot/types/interfaces/chain';
 import type { PrefixedStorageKey } from '@polkadot/types/interfaces/childstate';
 import type { AuthorityId } from '@polkadot/types/interfaces/consensus';
-import type { ContractCallRequest, ContractExecResult } from '@polkadot/types/interfaces/contracts';
+import type { ContractCallRequest, ContractExecResult, ContractInstantiateResult, InstantiateRequest } from '@polkadot/types/interfaces/contracts';
 import type { CreatedBlock } from '@polkadot/types/interfaces/engine';
 import type { EthAccount, EthCallRequest, EthFilter, EthFilterChanges, EthLog, EthReceipt, EthRichBlock, EthSubKind, EthSubParams, EthSyncStatus, EthTransaction, EthTransactionRequest, EthWork } from '@polkadot/types/interfaces/eth';
 import type { Extrinsic } from '@polkadot/types/interfaces/extrinsics';
@@ -19,7 +20,7 @@ import type { StorageKind } from '@polkadot/types/interfaces/offchain';
 import type { FeeDetails, RuntimeDispatchInfo } from '@polkadot/types/interfaces/payment';
 import type { RpcMethods } from '@polkadot/types/interfaces/rpc';
 import type { AccountId, BlockNumber, H160, H256, H64, Hash, Header, Index, Justification, KeyValue, SignedBlock, StorageData } from '@polkadot/types/interfaces/runtime';
-import type { ReadProof, RuntimeVersion } from '@polkadot/types/interfaces/state';
+import type { ReadProof, RuntimeVersion, TraceBlockResponse } from '@polkadot/types/interfaces/state';
 import type { ApplyExtrinsicResult, ChainProperties, ChainType, Health, NetworkState, NodeRole, PeerInfo, SyncState } from '@polkadot/types/interfaces/system';
 
 declare module '@polkadot/rpc-core/types.jsonrpc' {
@@ -64,6 +65,12 @@ declare module '@polkadot/rpc-core/types.jsonrpc' {
        **/
       epochAuthorship: AugmentedRpc<() => Observable<HashMap<AuthorityId, EpochAuthorship>>>;
     };
+    beefy: {
+      /**
+       * Returns the block most recently finalized by BEEFY, alongside side its justification.
+       **/
+      subscribeJustifications: AugmentedRpc<() => Observable<BeefySignedCommitment>>;
+    };
     chain: {
       /**
        * Get header and body of a relay chain block
@@ -121,6 +128,10 @@ declare module '@polkadot/rpc-core/types.jsonrpc' {
        * Returns the value under a specified storage key in a contract
        **/
       getStorage: AugmentedRpc<(address: AccountId | string | Uint8Array, key: H256 | string | Uint8Array, at?: BlockHash | string | Uint8Array) => Observable<Option<Bytes>>>;
+      /**
+       * Instantiate a new contract
+       **/
+      instantiate: AugmentedRpc<(request: InstantiateRequest | { origin?: any; endowment?: any; gasLimit?: any; code?: any; data?: any; salt?: any } | string | Uint8Array, at?: BlockHash | string | Uint8Array) => Observable<ContractInstantiateResult>>;
       /**
        * Returns the projected time a given contract will be able to sustain paying its rent
        **/
@@ -435,6 +446,10 @@ declare module '@polkadot/rpc-core/types.jsonrpc' {
        * Subscribes to storage changes for the provided keys
        **/
       subscribeStorage: AugmentedRpc<<T = Codec[]>(keys?: Vec<StorageKey> | (StorageKey | string | Uint8Array | any)[]) => Observable<T>>;
+      /**
+       * Provides a way to trace the re-execution of a single block
+       **/
+      traceBlock: AugmentedRpc<(block: Hash | string | Uint8Array, targets: Option<Text> | null | object | string | Uint8Array, storageKeys: Option<Text> | null | object | string | Uint8Array) => Observable<TraceBlockResponse>>;
     };
     syncstate: {
       /**
@@ -503,6 +518,10 @@ declare module '@polkadot/rpc-core/types.jsonrpc' {
        * Remove a reserved peer
        **/
       removeReservedPeer: AugmentedRpc<(peerId: Text | string) => Observable<Text>>;
+      /**
+       * Returns the list of reserved peers
+       **/
+      reservedPeers: AugmentedRpc<() => Observable<Vec<Text>>>;
       /**
        * Resets the log filter to Substrate defaults
        **/

packages/api/src/augment/tx.ts

@@ -108,8 +108,7 @@ declare module '@polkadot/api/types/submittable' {
        * 
        * The origin must be Signed and the sender must have sufficient funds free.
        * 
-       * Funds of sender are reserved according to the formula:
-       * `AssetDepositBase + AssetDepositPerZombie * max_zombies`.
+       * Funds of sender are reserved by `AssetDeposit`.
        * 
        * Parameters:
        * - `id`: The identifier of the new asset. This must not be currently in use to identify
@@ -713,7 +712,7 @@ declare module '@polkadot/api/types/submittable' {
        * * `gas_limit`: The gas limit enforced when executing the constructor.
        * * `code`: The contract code to deploy in raw bytes.
        * * `data`: The input data to pass to the contract constructor.
-       * * `salt`: Used for the address derivation. See [`Self::contract_address`].
+       * * `salt`: Used for the address derivation. See [`Pallet::contract_address`].
        * 
        * Instantiation is executed as follows:
        * 
@@ -1277,8 +1276,8 @@ declare module '@polkadot/api/types/submittable' {
        * origin is removed as a runner-up.
        * - `origin` is a current member. In this case, the deposit is unreserved and origin is
        * removed as a member, consequently not being a candidate for the next round anymore.
-       * Similar to [`remove_members`], if replacement runners exists, they are immediately used.
-       * If the prime is renouncing, then no prime will exist until the next round.
+       * Similar to [`remove_members`], if replacement runners exists, they are immediately
+       * used. If the prime is renouncing, then no prime will exist until the next round.
        * 
        * The dispatch origin of this call must be signed, and have one of the above roles.
        * 
@@ -2837,7 +2836,7 @@ declare module '@polkadot/api/types/submittable' {
        * - Write: Bonded, Payee, [Origin Account], Locks, Ledger
        * # </weight>
        **/
-      bond: AugmentedSubmittable<(controller: LookupSource | { Id: any } | { Index: any } | { Raw: any } | { Address32: any } | { Address20: any } | string | Uint8Array, value: Compact<BalanceOf> | AnyNumber | Uint8Array, payee: RewardDestination | { Staked: any } | { Stash: any } | { Controller: any } | { Account: any } | string | Uint8Array) => SubmittableExtrinsic<ApiType>, [LookupSource, Compact<BalanceOf>, RewardDestination]>;
+      bond: AugmentedSubmittable<(controller: LookupSource | { Id: any } | { Index: any } | { Raw: any } | { Address32: any } | { Address20: any } | string | Uint8Array, value: Compact<BalanceOf> | AnyNumber | Uint8Array, payee: RewardDestination | { Staked: any } | { Stash: any } | { Controller: any } | { Account: any } | { None: any } | string | Uint8Array) => SubmittableExtrinsic<ApiType>, [LookupSource, Compact<BalanceOf>, RewardDestination]>;
       /**
        * Add some extra amount that have appeared in the stash `free_balance` into the balance up
        * for staking.
@@ -3143,7 +3142,7 @@ declare module '@polkadot/api/types/submittable' {
        * - Write: Payee
        * # </weight>
        **/
-      setPayee: AugmentedSubmittable<(payee: RewardDestination | { Staked: any } | { Stash: any } | { Controller: any } | { Account: any } | string | Uint8Array) => SubmittableExtrinsic<ApiType>, [RewardDestination]>;
+      setPayee: AugmentedSubmittable<(payee: RewardDestination | { Staked: any } | { Stash: any } | { Controller: any } | { Account: any } | { None: any } | string | Uint8Array) => SubmittableExtrinsic<ApiType>, [RewardDestination]>;
       /**
        * Sets the ideal number of validators.
        * 
@@ -3875,7 +3874,7 @@ declare module '@polkadot/api/types/submittable' {
        * Unlock any vested funds of the sender account.
        * 
        * The dispatch origin for this call must be _Signed_ and the sender must have funds still
-       * locked under this module.
+       * locked under this pallet.
        * 
        * Emits either `VestingCompleted` or `VestingUpdated`.
        * 
@@ -3893,7 +3892,7 @@ declare module '@polkadot/api/types/submittable' {
        * The dispatch origin for this call must be _Signed_.
        * 
        * - `target`: The account whose vested funds should be unlocked. Must have funds still
-       * locked under this module.
+       * locked under this pallet.
        * 
        * Emits either `VestingCompleted` or `VestingUpdated`.
        * 

packages/api/src/base/Decorate.ts

@@ -262,7 +262,8 @@ export abstract class Decorate<ApiType extends ApiTypes> extends Events {
 
       return allKeys;
     }, []);
-    const unknown = exposed.filter((key) => !allKeys.includes(key));
+    const unknown = exposed.filter((k) => !allKeys.includes(k));
+    const deletion = allKnown.filter(([k]) => hasResults && !exposed.includes(k) && k !== 'rpc_methods');
 
     if (unknown.length) {
       l.warn(`RPC methods not decorated: ${unknown.join(', ')}`);
@@ -270,17 +271,10 @@ export abstract class Decorate<ApiType extends ApiTypes> extends Events {
 
     // loop through all entries we have (populated in decorate) and filter as required
     // only remove when we have results and method missing, or with no results if optional
-    allKnown
-      .filter(([key]): boolean => hasResults
-        ? !exposed.includes(key) && key !== 'rpc_methods' // rpc_methods doesn't appear, v1
-        : key === 'rpc_methods' // we didn't find this one, remove
-      )
-      // eslint-disable-next-line @typescript-eslint/no-unused-vars
-      .forEach(([_, { method, section }]): void => {
-        delete (this._rpc as Record<string, Record<string, unknown>>)[section][method];
-        delete (this._rpcCore as unknown as Record<string, Record<string, unknown>>)[section][method];
-        delete (this._rx.rpc as Record<string, Record<string, unknown>>)[section][method];
-      });
+    deletion.forEach(([, { method, section }]): void => {
+      delete (this._rpc as Record<string, Record<string, unknown>>)[section][method];
+      delete (this._rx.rpc as Record<string, Record<string, unknown>>)[section][method];
+    });
   }
 
   protected _decorateRpc<ApiType extends ApiTypes> (rpc: RpcCore & RpcInterface, decorateMethod: DecorateMethod<ApiType>, input: Partial<DecoratedRpc<ApiType, RpcInterface>> = {}): DecoratedRpc<ApiType, RpcInterface> {
@@ -378,8 +372,16 @@ export abstract class Decorate<ApiType extends ApiTypes> extends Events {
     decorated.is = <A extends AnyTuple> (key: IStorageKey<AnyTuple>): key is IStorageKey<A> =>
       key.section === creator.section && key.method === creator.method;
 
-    decorated.key = (arg1?: Arg, arg2?: Arg): string =>
-      u8aToHex(compactStripLength(creator(creator.meta.type.isDoubleMap ? [arg1, arg2] : arg1))[1]);
+    decorated.key = (...args: Arg[]): string =>
+      u8aToHex(compactStripLength(creator(
+        creator.meta.type.isPlain
+          ? undefined
+          : creator.meta.type.isMap
+            ? args[0]
+            : creator.meta.type.isDoubleMap
+              ? [args[0], args[1]]
+              : args
+      ))[1]);
 
     decorated.keyPrefix = (key1?: Arg): string =>
       u8aToHex(creator.keyPrefix(key1));
@@ -393,6 +395,7 @@ export abstract class Decorate<ApiType extends ApiTypes> extends Events {
     decorated.sizeAt = decorateMethod((hash: Hash | Uint8Array | string, arg1?: Arg, arg2?: Arg): Observable<u64> =>
       this._rpcCore.state.getStorageSize(getArgs(arg1, arg2), hash));
 
+    // FIXME NMap support
     // .keys() & .entries() only available on map types
     if (creator.iterKey && (creator.meta.type.isMap || creator.meta.type.isDoubleMap)) {
       decorated.entries = decorateMethod(

packages/api/src/base/Init.ts

@@ -13,15 +13,15 @@ import BN from 'bn.js';
 import { Metadata } from '@polkadot/metadata';
 import { TypeRegistry } from '@polkadot/types/create';
 import { getSpecAlias, getSpecExtensions, getSpecHasher, getSpecRpc, getSpecTypes, getUpgradeVersion } from '@polkadot/types-known';
-import { assert, BN_ZERO, logger, u8aEq, u8aToHex, u8aToU8a } from '@polkadot/util';
+import { assert, BN_ZERO, logger, stringify, u8aEq, u8aToHex, u8aToU8a } from '@polkadot/util';
 import { cryptoWaitReady } from '@polkadot/util-crypto';
 import { of } from '@polkadot/x-rxjs';
 import { map, switchMap } from '@polkadot/x-rxjs/operators';
 
+import { detectedCapabilities } from './capabilities';
 import { Decorate } from './Decorate';
-import { detectedCapabilities } from './util';
 
-const KEEPALIVE_INTERVAL = 15000;
+const KEEPALIVE_INTERVAL = 10000;
 const DEFAULT_BLOCKNUMBER = { unwrap: () => BN_ZERO };
 
 const l = logger('api/init');
@@ -31,7 +31,7 @@ export abstract class Init<ApiType extends ApiTypes> extends Decorate<ApiType> {
 
   #registries: VersionedRegistry[] = [];
 
-  #updateSub?: Subscription;
+  #updateSub?: Subscription | null = null;
 
   constructor (options: ApiOptions, type: ApiTypes, decorateMethod: DecorateMethod<ApiType>) {
     super(options, type, decorateMethod);
@@ -58,7 +58,7 @@ export abstract class Init<ApiType extends ApiTypes> extends Decorate<ApiType> {
 
     this._rx.signer = options.signer;
 
-    this._rpcCore.setRegistrySwap((hash: string | Uint8Array) => this.getBlockRegistry(hash));
+    this._rpcCore.setRegistrySwap((blockHash: Uint8Array) => this.getBlockRegistry(blockHash));
 
     if (this.hasSubscriptions) {
       this._rpcCore.provider.on('disconnected', this.#onProviderDisconnect);
@@ -102,10 +102,10 @@ export abstract class Init<ApiType extends ApiTypes> extends Decorate<ApiType> {
   /**
    * @description Sets up a registry based on the block hash defined
    */
-  public async getBlockRegistry (blockHash: string | Uint8Array): Promise<VersionedRegistry> {
-    // shortcut in the case where we have an immediate-same request
-    const lastBlockHash = u8aToU8a(blockHash);
-    const existingViaHash = this.#registries.find((r) => r.lastBlockHash && u8aEq(lastBlockHash, r.lastBlockHash));
+  public async getBlockRegistry (blockHash: Uint8Array): Promise<VersionedRegistry> {
+    const existingViaHash = this.#registries.find(({ lastBlockHash }) =>
+      lastBlockHash && u8aEq(lastBlockHash, blockHash)
+    );
 
     if (existingViaHash) {
       return existingViaHash;
@@ -129,10 +129,10 @@ export abstract class Init<ApiType extends ApiTypes> extends Decorate<ApiType> {
       : await this._rpcCore.state.getRuntimeVersion(header.parentHash).toPromise();
 
     // check for pre-existing registries
-    const existingViaVersion = this.#registries.find((r) => r.specVersion.eq(version.specVersion));
+    const existingViaVersion = this.#registries.find(({ specVersion }) => specVersion.eq(version.specVersion));
 
     if (existingViaVersion) {
-      existingViaVersion.lastBlockHash = lastBlockHash;
+      existingViaVersion.lastBlockHash = blockHash;
 
       return existingViaVersion;
     }
@@ -141,14 +141,14 @@ export abstract class Init<ApiType extends ApiTypes> extends Decorate<ApiType> {
     const metadata = await this._rpcCore.state.getMetadata(header.parentHash).toPromise();
     const registry = this._initRegistry(new TypeRegistry(blockHash), this._runtimeChain as Text, version, metadata);
 
-    // For now, since this is new we ignore the capability lookups (this could be useful once proven)
-    // this._detectCapabilities(registry, blockHash);
-
     // add our new registry
-    const result = { isDefault: false, lastBlockHash, metadata, metadataConsts: null, registry, specVersion: version.specVersion };
+    const result = { isDefault: false, lastBlockHash: blockHash, metadata, metadataConsts: null, registry, specVersion: version.specVersion };
 
     this.#registries.push(result);
 
+    // TODO This could be useful for historic, disabled due to cross-looping, i.e. .at queries
+    // this._detectCapabilities(registry, blockHash);
+
     return result;
   }
 
@@ -156,10 +156,10 @@ export abstract class Init<ApiType extends ApiTypes> extends Decorate<ApiType> {
     // on re-connection to the same chain, we don't want to re-do everything from chain again
     if (this._isReady) {
       return true;
-    } else if (this.#updateSub) {
-      this.#updateSub.unsubscribe();
     }
 
+    this._unsubscribeUpdates();
+
     // only load from on-chain if we are not a clone (default path), alternatively
     // just use the values from the source instance provided
     [this._genesisHash, this._runtimeMetadata] = this._options.source?._isReady
@@ -190,17 +190,19 @@ export abstract class Init<ApiType extends ApiTypes> extends Decorate<ApiType> {
     return [source.genesisHash, source.runtimeMetadata];
   }
 
-  private _detectCapabilities (registry?: Registry, blockHash?: string | Uint8Array): void {
+  private _detectCapabilities (registry: Registry, blockHash?: string | Uint8Array): boolean {
     detectedCapabilities(this._rx, blockHash)
       .toPromise()
       .then((types): void => {
         if (Object.keys(types).length) {
-          (registry || this.registry).register(types as Record<string, string>);
+          registry.register(types as Record<string, string>);
 
-          l.debug(`Capabilities detected${blockHash ? ` (${u8aToHex(u8aToU8a(blockHash))})` : ''}: ${JSON.stringify(types)}`);
+          l.debug(() => `Capabilities detected${blockHash ? ` (${u8aToHex(u8aToU8a(blockHash))})` : ''}: ${stringify(types)}`);
         }
       })
-      .catch(l.error);
+      .catch(undefined);
+
+    return true;
   }
 
   // subscribe to metadata updates, inject the types on changes
@@ -235,9 +237,8 @@ export abstract class Init<ApiType extends ApiTypes> extends Decorate<ApiType> {
               // clear the registry types to ensure that we override correctly
               this._initRegistry(thisRegistry.registry.init(), this._runtimeChain as Text, version, metadata);
               this.injectMetadata(metadata, false, thisRegistry.registry);
-              this._detectCapabilities(thisRegistry.registry);
 
-              return true;
+              return this._detectCapabilities(thisRegistry.registry);
             })
           )
       )
@@ -293,15 +294,36 @@ export abstract class Init<ApiType extends ApiTypes> extends Decorate<ApiType> {
 
     // inject metadata and adjust the types as detected
     this.injectMetadata(metadata, true);
-    this._detectCapabilities();
 
     // derive is last, since it uses the decorated rx
     this._rx.derive = this._decorateDeriveRx(this._rxDecorateMethod);
     this._derive = this._decorateDerive(this._decorateMethod);
 
+    // detect the on-chain capabilities
+    this._detectCapabilities(this.registry);
+
     return true;
   }
 
+  private _unsubscribeHealth (): void {
+    if (this.#healthTimer) {
+      clearInterval(this.#healthTimer);
+      this.#healthTimer = null;
+    }
+  }
+
+  private _unsubscribeUpdates (): void {
+    if (this.#updateSub) {
+      this.#updateSub.unsubscribe();
+      this.#updateSub = null;
+    }
+  }
+
+  protected _unsubscribe (): void {
+    this._unsubscribeHealth();
+    this._unsubscribeUpdates();
+  }
+
   #onProviderConnect = async (): Promise<void> => {
     this.emit('connected');
     this._isConnected.next(true);
@@ -321,7 +343,7 @@ export abstract class Init<ApiType extends ApiTypes> extends Decorate<ApiType> {
       }
 
       this.#healthTimer = setInterval((): void => {
-        this._rpcCore.system.health().toPromise().catch(() => null);
+        this._rpcCore.system.health().toPromise().catch((error) => l.warn(`Health keepalive check failed: ${(error as Error).message}`));
       }, KEEPALIVE_INTERVAL);
     } catch (_error) {
       const error = new Error(`FATAL: Unable to initialize the API: ${(_error as Error).message}`);
@@ -337,10 +359,7 @@ export abstract class Init<ApiType extends ApiTypes> extends Decorate<ApiType> {
     this.emit('disconnected');
     this._isConnected.next(false);
 
-    if (this.#healthTimer) {
-      clearInterval(this.#healthTimer);
-      this.#healthTimer = null;
-    }
+    this._unsubscribeHealth();
   };
 
   #onProviderError = (error: Error): void => {

packages/api/src/base/capabilities.ts

@@ -0,0 +1,190 @@
+// Copyright 2017-2021 @polkadot/api-derive authors & contributors
+// SPDX-License-Identifier: Apache-2.0
+
+import type { ApiInterfaceRx } from '@polkadot/api/types';
+import type { Raw, u32 } from '@polkadot/types';
+import type { Releases } from '@polkadot/types/interfaces';
+import type { InterfaceTypes } from '@polkadot/types/types';
+
+import { assert, compactFromU8a } from '@polkadot/util';
+import { combineLatest, Observable, of } from '@polkadot/x-rxjs';
+import { catchError, map, take } from '@polkadot/x-rxjs/operators';
+
+// the order and types needs to map with the all array setup below
+type ExtractedQ = [Releases | null];
+
+type ExtractedR = [Raw | null, Raw | null];
+
+type ExtractedC = [u32 | null, u32 | null];
+
+type DetectedKeys = keyof Pick<InterfaceTypes, 'AccountInfo' | 'ValidatorPrefs'>;
+
+type DetectedValues = keyof InterfaceTypes;
+
+interface DetectedTypes extends Record<DetectedKeys, DetectedValues> {
+  AccountInfo: 'AccountInfoWithRefCount' | 'AccountInfoWithDualRefCount' | 'AccountInfoWithTripleRefCount',
+  Keys: 'SessionKeys1' | 'SessionKeys2' | 'SessionKeys3' | 'SessionKeys4' | 'SessionKeys5' | 'SessionKeys6' | 'SessionKeys7' | 'SessionKeys8' | 'SessionKeys9' | 'SessionKeys10';
+  RefCount: 'u8' | 'u32',
+  SlotRange: Record<string, unknown>;
+  ValidatorPrefs: 'ValidatorPrefsWithBlocked' | 'ValidatorPrefsWithCommission';
+  WinningData: string;
+}
+
+interface AvailableMap<T> {
+  filtered: T[];
+  included: boolean[];
+  original: T[];
+}
+
+interface Constants {
+  accountIdLength: number;
+  refcount1Length: number;
+  refcount2Length: number;
+  refcount3Length: number;
+}
+
+const NumberMap = ['Zero', 'One', 'Two', 'Three', 'Four', 'Five', 'Six', 'Seven', 'Eight', 'Nine', 'Ten'];
+
+function mapCapabilities ({ accountIdLength, refcount1Length, refcount2Length, refcount3Length }: Constants, [leasePeriodsPerSlot, slotRangeCount]: ExtractedC, [stakingVersion]: ExtractedQ, [keys, accountInfo]: ExtractedR): Partial<DetectedTypes> {
+  const types: Partial<DetectedTypes> = {};
+
+  // AccountInfo
+  if (accountInfo) {
+    const length = accountInfo.length;
+
+    if (length === refcount1Length) {
+      types.AccountInfo = 'AccountInfoWithRefCount';
+    } else if (length === refcount2Length) {
+      types.AccountInfo = 'AccountInfoWithDualRefCount';
+    } else if (length === refcount3Length) {
+      types.AccountInfo = 'AccountInfoWithTripleRefCount';
+    }
+  }
+
+  // ValidatorPrefs
+  if (stakingVersion) {
+    if (stakingVersion.index >= 4) { // v1 = index 0, V5 = index 4
+      types.ValidatorPrefs = 'ValidatorPrefsWithBlocked';
+    } else {
+      types.ValidatorPrefs = 'ValidatorPrefsWithCommission';
+    }
+  }
+
+  // Keys
+  if (keys) {
+    try {
+      const [offset, numItems] = compactFromU8a(keys);
+      const tupleLength = (keys.length - offset) / numItems.toNumber();
+      const numIds = tupleLength / accountIdLength;
+      const numIdsRound = Math.floor(numIds);
+
+      assert(numIds >= 2 && numIds <= 11, () => `Detected ${numIds} in Keys, should be >= 2 and <= 11`);
+
+      if (numIdsRound !== numIds) {
+        // Beefy?
+        if ((((numIdsRound - 1) * accountIdLength) + 33) === tupleLength) {
+          // eslint-disable-next-line @typescript-eslint/ban-ts-comment
+          // @ts-ignore
+          types.Keys = `SessionKeys${numIdsRound - 1}B`;
+        } else {
+          assert(false, () => `Expected integer number of keys, found ${numIds.toFixed(2)}`);
+        }
+      } else {
+        // eslint-disable-next-line @typescript-eslint/ban-ts-comment
+        // @ts-ignore
+        types.Keys = `SessionKeys${numIds - 1}`;
+      }
+    } catch {
+      // ignore
+    }
+  }
+
+  // auctions
+  if (leasePeriodsPerSlot && slotRangeCount) {
+    const _enum: string[] = [];
+
+    for (let i = 0; leasePeriodsPerSlot.gtn(i); i++) {
+      for (let j = i; leasePeriodsPerSlot.gtn(j); j++) {
+        _enum.push(`${NumberMap[i]}${NumberMap[j]}`);
+      }
+    }
+
+    types.SlotRange = { _enum };
+    types.WinningData = `[WinningDataEntry; ${slotRangeCount.toNumber()}]`;
+  }
+
+  return types;
+}
+
+function filterEntries <T> (original: T[]): AvailableMap<T> {
+  const included = original.map((c) => !!c);
+
+  return {
+    filtered: original.filter((_, index) => included[index]),
+    included,
+    original
+  };
+}
+
+function extractResults <R, T = unknown> (results: unknown[], map: AvailableMap<T>): R {
+  let offset = -1;
+
+  return map.included.map((isIncluded) =>
+    isIncluded
+      ? results[++offset]
+      : null
+  ) as unknown as R;
+}
+
+/**
+ * @description Query the chain for the specific capabilities
+ */
+export function detectedCapabilities (api: ApiInterfaceRx, blockHash?: Uint8Array | string | undefined): Observable<Partial<DetectedTypes>> {
+  const emptyAccountId = api.registry.createType('AccountId');
+  const consts = filterEntries([
+    api.consts.auctions?.leasePeriodsPerSlot,
+    api.consts.auctions?.slotRangeCount
+  ]);
+  const queries = filterEntries([
+    api.query.staking?.storageVersion
+  ]);
+  const raws = filterEntries([
+    api.query.session?.queuedKeys.key(),
+    api.query.system?.account.key(emptyAccountId)
+  ]);
+
+  return combineLatest([
+    consts.filtered.length
+      ? blockHash
+        // FIXME consts don't have .at as of yet...
+        ? of([])
+        : of(consts.filtered)
+      : of([]),
+    queries.filtered.length
+      ? blockHash
+        ? combineLatest(queries.filtered.map((c) => c.at(blockHash)))
+        : api.queryMulti(queries.filtered)
+      : of([]),
+    raws.filtered.length
+      ? blockHash
+        ? combineLatest(raws.filtered.map((k) => api.rpc.state.getStorage.raw(k, blockHash)))
+        : combineLatest(raws.filtered.map((k) => api.rpc.state.getStorage.raw(k)))
+      : of([])
+  ]).pipe(
+    map(([cResults, qResults, rResults]): Partial<DetectedTypes> =>
+      mapCapabilities(
+        {
+          accountIdLength: emptyAccountId.encodedLength,
+          refcount1Length: api.registry.createType('AccountInfoWithRefCount').encodedLength,
+          refcount2Length: api.registry.createType('AccountInfoWithDualRefCount').encodedLength,
+          refcount3Length: api.registry.createType('AccountInfoWithTripleRefCount').encodedLength
+        },
+        extractResults<ExtractedC>(cResults, consts),
+        extractResults<ExtractedQ>(qResults, queries),
+        extractResults<ExtractedR>(rResults, raws)
+      )
+    ),
+    take(1),
+    catchError(() => of({}))
+  );
+}

packages/api/src/base/index.ts

@@ -50,6 +50,8 @@ export abstract class ApiBase<ApiType extends ApiTypes> extends Getters<ApiType>
    * @description Disconnect from the underlying provider, halting all network traffic
    */
   public disconnect (): Promise<void> {
+    this._unsubscribe();
+
     return this._rpcCore.disconnect();
   }
 

packages/api/src/base/util.ts

@@ -1,82 +0,0 @@
-// Copyright 2017-2021 @polkadot/api-derive authors & contributors
-// SPDX-License-Identifier: Apache-2.0
-
-import type { ApiInterfaceRx } from '@polkadot/api/types';
-import type { bool } from '@polkadot/types';
-import type { Releases } from '@polkadot/types/interfaces';
-import type { InterfaceTypes } from '@polkadot/types/types';
-
-import { combineLatest, Observable, of } from '@polkadot/x-rxjs';
-import { map, take } from '@polkadot/x-rxjs/operators';
-
-// the order and types needs to map with the all array setup below
-type Extracted = [bool | null, bool | null, bool | null, Releases | null];
-
-type DetectedKeys = keyof Pick<InterfaceTypes, 'AccountInfo' | 'ValidatorPrefs'>;
-
-type DetectedValues = keyof InterfaceTypes;
-
-interface DetectedTypes extends Record<DetectedKeys, DetectedValues> {
-  AccountInfo: 'AccountInfoWithRefCount' | 'AccountInfoWithDualRefCount' | 'AccountInfoWithTripleRefCount',
-  RefCount: 'u8' | 'u32',
-  ValidatorPrefs: 'ValidatorPrefsWithBlocked' | 'ValidatorPrefsWithCommission';
-}
-
-function mapCapabilities ([systemRefcount32, systemRefcountDual, systemRefcountTriple, stakingVersion]: Extracted): Partial<DetectedTypes> {
-  const types: Partial<DetectedTypes> = {};
-
-  // AccountInfo
-  if (systemRefcountTriple && systemRefcountTriple.isTrue) {
-    types.AccountInfo = 'AccountInfoWithTripleRefCount';
-  } else if (systemRefcountDual && systemRefcountDual.isTrue) {
-    types.AccountInfo = 'AccountInfoWithDualRefCount';
-  } else {
-    types.AccountInfo = 'AccountInfoWithRefCount';
-
-    if (!systemRefcount32 || systemRefcount32.isFalse) {
-      types.RefCount = 'u8';
-    }
-  }
-
-  // ValidatorPrefs
-  if (stakingVersion) {
-    if (stakingVersion.index >= 4) { // v1 = index 0, V5 = index 4
-      types.ValidatorPrefs = 'ValidatorPrefsWithBlocked';
-    } else {
-      types.ValidatorPrefs = 'ValidatorPrefsWithCommission';
-    }
-  }
-
-  return types;
-}
-
-/**
- * @description Query the chain for the specific capabilities
- */
-export function detectedCapabilities (api: ApiInterfaceRx, blockHash?: Uint8Array | string | undefined): Observable<Partial<DetectedTypes>> {
-  const all = [
-    api.query.system?.upgradedToU32RefCount,
-    api.query.system?.upgradedToDualRefCount,
-    api.query.system?.upgradedToTripleRefCount,
-    api.query.staking?.storageVersion
-  ];
-  const included = all.map((c) => !!c);
-  const filtered = all.filter((_, index) => included[index]);
-
-  return (
-    filtered.length
-      ? blockHash
-        ? combineLatest(filtered.map((c) => c.at(blockHash)))
-        : api.queryMulti(filtered)
-      : of([])
-  ).pipe(
-    map((results): Partial<DetectedTypes> => {
-      let offset = -1;
-
-      return mapCapabilities(
-        included.map((isIncluded) => isIncluded ? results[++offset] : null) as Extracted
-      );
-    }),
-    take(1)
-  );
-}

packages/api/src/packageInfo.ts

@@ -3,4 +3,4 @@
 
 // Auto-generated by @polkadot/dev, do not edit
 
-export const packageInfo = { name: '@polkadot/api', version: '4.2.1' };
+export const packageInfo = { name: '@polkadot/api', version: '4.9.1' };

packages/api/src/promise/index.spec.ts

@@ -1,7 +1,7 @@
 // Copyright 2017-2021 @polkadot/api authors & contributors
 // SPDX-License-Identifier: Apache-2.0
 
-import type { ApiOptions, SubmittableExtrinsic } from '../types';
+import type { SubmittableExtrinsic } from '../types';
 
 import { createPair } from '@polkadot/keyring/pair';
 import { createTestKeyring } from '@polkadot/keyring/testing';
@@ -44,7 +44,7 @@ describe('ApiPromise', (): void => {
   }
 
   beforeEach((): void => {
-    jest.setTimeout(3000000);
+    jest.setTimeout(10000);
     provider = new MockProvider(registry);
   });
 
@@ -53,14 +53,10 @@ describe('ApiPromise', (): void => {
       const rpcData = await provider.send('state_getMetadata', []);
       const genesisHash = registry.createType('Hash', await provider.send('chain_getBlockHash', [])).toHex();
       const specVersion = 0;
-      const metadata: any = {};
-      const key = `${genesisHash}-${specVersion}`;
-
-      // eslint-disable-next-line @typescript-eslint/no-unsafe-member-access
-      metadata[key] = rpcData;
+      const metadata = { [`${genesisHash}-${specVersion}`]: rpcData as string };
 
       // eslint-disable-next-line @typescript-eslint/no-unsafe-assignment
-      const api = await ApiPromise.create({ metadata, provider, registry } as ApiOptions);
+      const api = await ApiPromise.create({ metadata, provider, registry });
 
       expect(api.genesisHash).toBeDefined();
       expect(api.runtimeMetadata).toBeDefined();

packages/api/src/promise/index.ts

@@ -211,7 +211,19 @@ export class ApiPromise extends ApiBase<'promise'> {
    * ```
    */
   public static create (options?: ApiOptions): Promise<ApiPromise> {
-    return new ApiPromise(options).isReady;
+    const instance = new ApiPromise(options);
+
+    if (options && options.throwOnConnect) {
+      return instance.isReadyOrError;
+    }
+
+    // Swallow any rejections on isReadyOrError
+    // (in Node 15.x this creates issues, when not being looked at)
+    instance.isReadyOrError.catch(() => {
+      // ignore
+    });
+
+    return instance.isReady;
   }
 
   /**
@@ -235,18 +247,14 @@ export class ApiPromise extends ApiBase<'promise'> {
     super(options, 'promise', decorateMethod);
 
     this.#isReadyPromise = new Promise((resolve): void => {
-      super.once('ready', (): void => {
-        resolve(this);
-      });
+      super.once('ready', () => resolve(this));
     });
 
     this.#isReadyOrErrorPromise = new Promise((resolve, reject): void => {
-      super.once('ready', (): void => {
-        resolve(this);
-      });
-      super.once('error', (e): void => {
-        reject(e);
-      });
+      const tracker = promiseTracker(resolve, reject);
+
+      super.once('ready', () => tracker.resolve(this));
+      super.once('error', (e) => tracker.reject(e));
     });
   }
 

packages/api/src/rx/index.ts

@@ -165,9 +165,7 @@ export class ApiRx extends ApiBase<'rxjs'> {
     this.#isReadyRx = from<Promise<ApiRx>>(
       // You can create an observable from an event, however my mind groks this form better
       new Promise((resolve): void => {
-        super.on('ready', (): void => {
-          resolve(this);
-        });
+        super.on('ready', () => resolve(this));
       })
     );
   }

packages/api/src/submittable/Result.ts

@@ -32,14 +32,17 @@ export class SubmittableResult implements ISubmittableResult {
 
   public readonly dispatchInfo?: DispatchInfo;
 
+  public readonly internalError?: Error;
+
   public readonly events: EventRecord[];
 
   public readonly status: ExtrinsicStatus;
 
-  constructor ({ dispatchError, dispatchInfo, events, status }: SubmittableResultValue) {
+  constructor ({ dispatchError, dispatchInfo, events, internalError, status }: SubmittableResultValue) {
     this.dispatchError = dispatchError || extractError(events);
     this.dispatchInfo = dispatchInfo || extractInfo(events);
     this.events = events || [];
+    this.internalError = internalError;
     this.status = status;
   }
 
@@ -85,6 +88,7 @@ export class SubmittableResult implements ISubmittableResult {
       dispatchError: this.dispatchError?.toHuman(),
       dispatchInfo: this.dispatchInfo?.toHuman(),
       events: this.events.map((event) => event.toHuman(isExtended)),
+      internalError: this.internalError?.message.toString(),
       status: this.status.toHuman(isExtended)
     };
   }

packages/api/src/submittable/createClass.ts

@@ -11,7 +11,7 @@ import type { AddressOrPair, SignerOptions, SubmittableDryRunResult, Submittable
 
 import { assert, isBn, isFunction, isNumber, isString, isU8a } from '@polkadot/util';
 import { of } from '@polkadot/x-rxjs';
-import { first, map, mapTo, mergeMap, switchMap, tap } from '@polkadot/x-rxjs/operators';
+import { catchError, first, map, mapTo, mergeMap, switchMap, tap } from '@polkadot/x-rxjs/operators';
 
 import { ApiBase } from '../base';
 import { filterEvents, isKeyringPair } from '../util';
@@ -242,6 +242,9 @@ export function createClass <ApiType extends ApiTypes> ({ api, apiType, decorate
             events: filterEvents(hash, block, events, status),
             status
           }))
+        ),
+        catchError((internalError: Error) =>
+          of(this.#transformResult(new SubmittableResult({ internalError, status })))
         )
       );
     }

packages/api/src/submittable/types.ts

@@ -10,6 +10,7 @@ export interface SubmittableResultValue {
   dispatchError?: DispatchError;
   dispatchInfo?: DispatchInfo;
   events?: EventRecord[];
+  internalError?: Error;
   status: ExtrinsicStatus;
 }
 

packages/api/src/types/index.ts

@@ -69,6 +69,10 @@ export interface ApiOptions extends RegisteredTypes {
    * @description The source object to use for runtime information (only used when cloning)
    */
   source?: ApiBase<any>;
+  /**
+   * @description Throws an error when the initial connection fails (same as isReadyOrError)
+   */
+  throwOnConnect?: boolean;
 }
 
 // A smaller interface of ApiRx, used in derive and in SubmittableExtrinsic

packages/api/src/util/validate.spec.ts

@@ -66,14 +66,14 @@ describe('extractStorageArgs', (): void => {
     expect(
       (): any =>
         extractStorageArgs(storage.staking.erasStakers, [])
-    ).toThrow('staking.erasStakers(EraIndex, AccountId) is a doublemap, requiring 2 arguments, 0 found');
+    ).toThrow('staking.erasStakers(EraIndex, AccountId) is a double map, requiring 2 arguments, 0 found');
   });
 
   it('validates doublemap, 2 args (failing with 1 arg)', (): void => {
     expect(
       (): any =>
         extractStorageArgs(storage.staking.erasStakers, [123])
-    ).toThrow('staking.erasStakers(EraIndex, AccountId) is a doublemap, requiring 2 arguments, 1 found');
+    ).toThrow('staking.erasStakers(EraIndex, AccountId) is a double map, requiring 2 arguments, 1 found');
   });
 
   // Linked maps have been removed

packages/api/src/util/validate.ts

@@ -6,14 +6,14 @@ import type { StorageEntry } from '@polkadot/types/primitive/types';
 
 import { assert, isUndefined } from '@polkadot/util';
 
-function sig ({ method, section }: StorageEntry, ...args: Type[]): string {
+function sig ({ method, section }: StorageEntry, args: Type[]): string {
   return `${section}.${method}(${args.join(', ')})`;
 }
 
 function doDoubleMap (creator: StorageEntry, args: unknown[]): [StorageEntry, [any, any]] {
   const { key1, key2 } = creator.meta.type.asDoubleMap;
 
-  assert(args.length === 2, `${sig(creator, key1, key2)} is a doublemap, requiring 2 arguments, ${args.length} found`);
+  assert(args.length === 2, () => `${sig(creator, [key1, key2])} is a double map, requiring 2 arguments, ${args.length} found`);
 
   // pass as tuple
   return [creator, args as [any, any]];
@@ -22,7 +22,7 @@ function doDoubleMap (creator: StorageEntry, args: unknown[]): [StorageEntry, [a
 function doMap (creator: StorageEntry, args: unknown[]): [StorageEntry] | [StorageEntry, any] {
   const { key } = creator.meta.type.asMap;
 
-  assert(args.length === 1, `${sig(creator, key)} is a map, requiring 1 argument, ${args.length} found`);
+  assert(args.length === 1, () => `${sig(creator, [key])} is a map, requiring 1 argument, ${args.length} found`);
 
   // expand
   return args.length
@@ -30,18 +30,29 @@ function doMap (creator: StorageEntry, args: unknown[]): [StorageEntry] | [Stora
     : [creator];
 }
 
+function doNMap (creator: StorageEntry, args: unknown[]): [StorageEntry, any[]] {
+  const { keyVec } = creator.meta.type.asNMap;
+
+  assert(args.length === keyVec.length, () => `${sig(creator, keyVec)} is a multi map, requiring ${keyVec.length} arguments, ${args.length} found`);
+
+  // pass as tuple
+  return [creator, args];
+}
+
 // sets up the arguments in the form of [creator, args] ready to be used in a storage
 // call. Additionally, it verifies that the correct number of arguments have been passed
-export function extractStorageArgs (creator: StorageEntry, _args: unknown[]): [StorageEntry, [any, any]] | [StorageEntry] | [StorageEntry, any] {
+export function extractStorageArgs (creator: StorageEntry, _args: unknown[]): [StorageEntry, any[]] | [StorageEntry] | [StorageEntry, any] {
   const args = _args.filter((arg) => !isUndefined(arg));
 
   if (creator.meta.type.isDoubleMap) {
     return doDoubleMap(creator, args);
   } else if (creator.meta.type.isMap) {
     return doMap(creator, args);
+  } else if (creator.meta.type.isNMap) {
+    return doNMap(creator, args);
   }
 
-  assert(args.length === 0, `${sig(creator)} does not take any arguments, ${args.length} found`);
+  assert(args.length === 0, () => `${sig(creator, [])} does not take any arguments, ${args.length} found`);
 
   // no args
   return [creator];

packages/metadata/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@polkadot/metadata",
-  "version": "4.2.1",
+  "version": "4.9.1",
   "type": "module",
   "description": "Helpers to extract information from runtime metadata",
   "main": "index.js",
@@ -8,6 +8,9 @@
     "./detectPackage.js",
     "./detectPackage.cjs"
   ],
+  "engines": {
+    "node": ">=14.0.0"
+  },
   "repository": "github:polkadot-js/api",
   "author": "Jaco Greeff <jacogr@gmail.com>",
   "maintainers": [],
@@ -16,14 +19,14 @@
   "bugs": "https://github.com/polkadot-js/api/issues",
   "homepage": "https://github.com/polkadot-js/api/tree/master/packages/type-metadata#readme",
   "dependencies": {
-    "@babel/runtime": "^7.13.10",
-    "@polkadot/types": "4.2.1",
-    "@polkadot/types-known": "4.2.1",
-    "@polkadot/util": "^6.0.5",
-    "@polkadot/util-crypto": "^6.0.5",
+    "@babel/runtime": "^7.14.0",
+    "@polkadot/types": "4.9.1",
+    "@polkadot/types-known": "4.9.1",
+    "@polkadot/util": "^6.3.1",
+    "@polkadot/util-crypto": "^6.3.1",
     "bn.js": "^4.11.9"
   },
   "devDependencies": {
-    "@polkadot/keyring": "^6.0.5"
+    "@polkadot/keyring": "^6.3.1"
   }
 }

packages/metadata/src/MagicNumber.spec.ts

@@ -9,10 +9,20 @@ describe('MagicNumber', (): void => {
   const registry = new TypeRegistry();
 
   it('succeeds when the magic number matches', (): void => {
-    expect((): MagicNumber => new MagicNumber(registry, MAGIC_NUMBER)).not.toThrow();
+    expect(
+      () => new MagicNumber(registry, MAGIC_NUMBER)
+    ).not.toThrow();
+  });
+
+  it('succeeds when the magic number is empty', (): void => {
+    expect(
+      () => new MagicNumber(registry)
+    ).not.toThrow();
   });
 
   it('fails when the magic number mismatches', (): void => {
-    expect((): MagicNumber => new MagicNumber(registry, 0x12345)).toThrow(/MagicNumber/);
+    expect(
+      () => new MagicNumber(registry, 0x12345678)
+    ).toThrow(/MagicNumber mismatch/);
   });
 });

packages/metadata/src/MagicNumber.ts

@@ -13,9 +13,7 @@ export class MagicNumber extends U32 {
     super(registry, value);
 
     if (!this.isEmpty) {
-      const magic = registry.createType('u32', MAGIC_NUMBER);
-
-      assert(this.eq(magic), `MagicNumber mismatch: expected ${magic.toHex()}, found ${this.toHex()}`);
+      assert(this.eq(MAGIC_NUMBER), () => `MagicNumber mismatch: expected ${registry.createType('u32', MAGIC_NUMBER).toHex()}, found ${this.toHex()}`);
     }
   }
 }

packages/metadata/src/Metadata.ts

@@ -3,7 +3,7 @@
 
 import type { Registry } from '@polkadot/types/types';
 
-import { isString, u8aConcat, u8aToU8a } from '@polkadot/util';
+import { isHex, isU8a, u8aToU8a } from '@polkadot/util';
 
 import { MetadataVersioned } from './MetadataVersioned';
 
@@ -11,37 +11,39 @@ import { MetadataVersioned } from './MetadataVersioned';
 const VERSION_IDX = 4;
 
 // magic + lowest supported version
-const EMPTY_METADATA = u8aConcat(new Uint8Array([0x6d, 0x65, 0x74, 0x61, 9]));
+const EMPTY_METADATA = new Uint8Array([0x6d, 0x65, 0x74, 0x61, 9]);
 const EMPTY_U8A = new Uint8Array();
 
-function sanitizeInput (_value: Uint8Array | string = EMPTY_U8A): Uint8Array {
-  if (isString(_value)) {
-    return sanitizeInput(u8aToU8a(_value));
-  }
-
-  return _value.length === 0
-    ? EMPTY_METADATA
-    : _value;
+function toU8a (value: Uint8Array | string = EMPTY_U8A): Uint8Array {
+  return isHex(value)
+    ? toU8a(u8aToU8a(value))
+    : isU8a(value) && value.length === 0
+      ? EMPTY_METADATA
+      : value;
 }
 
-function decodeMetadata (registry: Registry, _value?: Uint8Array | string): MetadataVersioned {
-  const value = sanitizeInput(_value);
-  const version = value[VERSION_IDX];
+function decodeMetadata (registry: Registry, _value?: Uint8Array | string | Map<string, unknown> | Record<string, unknown>): MetadataVersioned {
+  if (!_value || isU8a(_value) || isHex(_value)) {
+    const value = toU8a(_value);
+    const version = value[VERSION_IDX];
 
-  try {
-    return new MetadataVersioned(registry, value);
-  } catch (error) {
-    // This is an f-ing hack as a follow-up to another ugly hack
-    // https://github.com/polkadot-js/api/commit/a9211690be6b68ad6c6dad7852f1665cadcfa5b2
-    // when we fail on V9, try to re-parse it as v10... yes... HACK
-    if (version === 9) {
-      value[VERSION_IDX] = 10;
+    try {
+      return new MetadataVersioned(registry, value);
+    } catch (error) {
+      // This is an f-ing hack as a follow-up to another ugly hack
+      // https://github.com/polkadot-js/api/commit/a9211690be6b68ad6c6dad7852f1665cadcfa5b2
+      // when we fail on V9, try to re-parse it as v10... yes... HACK
+      if (version === 9) {
+        value[VERSION_IDX] = 10;
 
-      return decodeMetadata(registry, value);
+        return decodeMetadata(registry, value);
+      }
+
+      throw error;
     }
-
-    throw error;
   }
+
+  return new MetadataVersioned(registry, _value);
 }
 
 /**
@@ -50,7 +52,7 @@ function decodeMetadata (registry: Registry, _value?: Uint8Array | string): Meta
  * The versioned runtime metadata as a decoded structure
  */
 export class Metadata extends MetadataVersioned {
-  constructor (registry: Registry, value?: Uint8Array | string) {
+  constructor (registry: Registry, value?: Uint8Array | string | Map<string, unknown> | Record<string, unknown>) {
     super(registry, decodeMetadata(registry, value));
   }
 }

packages/metadata/src/MetadataVersioned.ts

@@ -1,7 +1,7 @@
 // Copyright 2017-2021 @polkadot/metadata authors & contributors
 // SPDX-License-Identifier: Apache-2.0
 
-import type { MetadataAll, MetadataLatest, MetadataV9, MetadataV10, MetadataV11, MetadataV12 } from '@polkadot/types/interfaces/metadata';
+import type { MetadataAll, MetadataLatest, MetadataV9, MetadataV10, MetadataV11, MetadataV12, MetadataV13 } from '@polkadot/types/interfaces/metadata';
 import type { AnyJson, Registry } from '@polkadot/types/types';
 
 import { Struct } from '@polkadot/types/codec';
@@ -10,13 +10,16 @@ import { assert } from '@polkadot/util';
 import { toV10 } from './v9/toV10';
 import { toV11 } from './v10/toV11';
 import { toV12 } from './v11/toV12';
-import { toLatest } from './v12/toLatest';
+import { toV13 } from './v12/toV13';
+import { toLatest } from './v13/toLatest';
 import { MagicNumber } from './MagicNumber';
 import { getUniqTypes, toCallsOnly } from './util';
 
-type MetaMapped = MetadataV9 | MetadataV10 | MetadataV11 | MetadataV12;
-type MetaVersions = 9 | 10 | 11 | 12 | 13;
-type MetaAsX = 'asV9' | 'asV10' | 'asV11' | 'asV12';
+type MetaMapped = MetadataV9 | MetadataV10 | MetadataV11 | MetadataV12 | MetadataV13;
+type MetaAsX = 'asV9' | 'asV10' | 'asV11' | 'asV12' | 'asV13';
+type MetaVersions = 'latest' | 9 | 10 | 11 | 12 | 13;
+
+const LATEST_VERSION = 13;
 
 /**
  * @name MetadataVersioned
@@ -24,7 +27,7 @@ type MetaAsX = 'asV9' | 'asV10' | 'asV11' | 'asV12';
  * The versioned runtime metadata as a decoded structure
  */
 export class MetadataVersioned extends Struct {
-  readonly #converted = new Map<number, MetaMapped>();
+  readonly #converted = new Map<MetaVersions, MetaMapped>();
 
   constructor (registry: Registry, value?: unknown) {
     super(registry, {
@@ -34,16 +37,18 @@ export class MetadataVersioned extends Struct {
   }
 
   #assertVersion = (version: number): boolean => {
-    assert(this.version <= version, `Cannot convert metadata from v${this.version} to v${version}`);
+    assert(this.version <= version, () => `Cannot convert metadata from version ${this.version} to ${version}`);
 
     return this.version === version;
   };
 
   #getVersion = <T extends MetaMapped, F extends MetaMapped>(version: MetaVersions, fromPrev: (registry: Registry, input: F, metaVersion: number) => T): T => {
     const asCurr = `asV${version}` as MetaAsX;
-    const asPrev = `asV${version - 1}` as MetaAsX;
+    const asPrev = version === 'latest'
+      ? `asV${LATEST_VERSION}` as MetaAsX
+      : `asV${version - 1}` as MetaAsX;
 
-    if (this.#assertVersion(version)) {
+    if (version !== 'latest' && this.#assertVersion(version)) {
       return this.#metadata()[asCurr] as T;
     }
 
@@ -101,16 +106,23 @@ export class MetadataVersioned extends Struct {
     return this.#getVersion(12, toV12);
   }
 
+  /**
+   * @description Returns the wrapped values as a V13 object
+   */
+  public get asV13 (): MetadataV13 {
+    return this.#getVersion(13, toV13);
+  }
+
   /**
    * @description Returns the wrapped values as a latest version object
    */
   public get asLatest (): MetadataLatest {
     // This is non-existent & latest - applied here to do the module-specific type conversions
-    return this.#getVersion(13, toLatest);
+    return this.#getVersion('latest', toLatest);
   }
 
   /**
-   * @description
+   * @description The magicNumber for the Metadata (known constant)
    */
   public get magicNumber (): MagicNumber {
     return this.get('magicNumber') as MagicNumber;
@@ -131,8 +143,8 @@ export class MetadataVersioned extends Struct {
    * @description Converts the Object to JSON, typically used for RPC transfers
    */
   public toJSON (): Record<string, AnyJson> {
-    // HACK(y): ensure that we apply the aliasses if we have not done so already, this is
-    // needed to ensure we have the overrides as intended (only applied in toLatest)
+    // HACK(y): ensure that we apply the aliases if we have not done so already, this is
+    // needed to ensure we have the correct overrides (which is only applied in toLatest)
     // eslint-disable-next-line no-unused-expressions
     this.asLatest;
 

packages/metadata/src/decorate/constants/index.spec.ts

@@ -17,7 +17,7 @@ function init (meta: string): [Constants, TypeRegistry] {
 
   registry.setMetadata(metadata);
 
-  return [decorateConstants(registry, metadata.asLatest), registry];
+  return [decorateConstants(registry, metadata.asLatest, 12), registry];
 }
 
 describe('decorateConstants', (): void => {

packages/metadata/src/decorate/constants/index.ts

@@ -8,7 +8,8 @@ import type { ConstantCodec, Constants, ModuleConstants } from '../types';
 import { hexToU8a, stringCamelCase } from '@polkadot/util';
 
 /** @internal */
-export function decorateConstants (registry: Registry, { modules }: MetadataLatest): Constants {
+// eslint-disable-next-line @typescript-eslint/no-unused-vars
+export function decorateConstants (registry: Registry, { modules }: MetadataLatest, _metaVersion: number): Constants {
   return modules.reduce((result: Constants, { constants, name }): Constants => {
     if (constants.isEmpty) {
       return result;

packages/metadata/src/decorate/extrinsics/createUnchecked.ts

@@ -16,7 +16,7 @@ export function createUnchecked (registry: Registry, section: string, callIndex:
   const funcName = stringCamelCase(callMetadata.name);
 
   const extrinsicFn = (...args: any[]): Call => {
-    assert(expectedArgs.length === args.length, `Extrinsic ${section}.${funcName} expects ${expectedArgs.length.valueOf()} arguments, got ${args.length}.`);
+    assert(expectedArgs.length === args.length, () => `Extrinsic ${section}.${funcName} expects ${expectedArgs.length.valueOf()} arguments, got ${args.length}.`);
 
     return registry.createType('Call', {
       args,

packages/metadata/src/decorate/index.spec.ts

@@ -46,7 +46,7 @@ describe('Decorated', () => {
   });
 
   it('should return constants with the correct type and value', (): void => {
-    const consts = decorateConstants(registry, metadata.asLatest);
+    const consts = decorateConstants(registry, metadata.asLatest, metadata.version);
 
     expect(consts.democracy.cooloffPeriod).toBeInstanceOf(registry.createClass('BlockNumber'));
     expect(consts.democracy.cooloffPeriod.toHex()).toEqual('0x000c4e00');

packages/metadata/src/decorate/index.ts

@@ -20,13 +20,14 @@ export function expandMetadata (registry: Registry, metadata: Metadata): Decorat
   assert(metadata instanceof Metadata, 'You need to pass a valid Metadata instance to Decorated');
 
   const latest = metadata.asLatest;
+  const version = metadata.version;
 
   return {
-    consts: decorateConstants(registry, latest),
-    errors: decorateErrors(registry, latest, metadata.version),
-    events: decorateEvents(registry, latest, metadata.version),
-    query: decorateStorage(registry, latest, metadata.version),
-    tx: decorateExtrinsics(registry, latest, metadata.version)
+    consts: decorateConstants(registry, latest, version),
+    errors: decorateErrors(registry, latest, version),
+    events: decorateEvents(registry, latest, version),
+    query: decorateStorage(registry, latest, version),
+    tx: decorateExtrinsics(registry, latest, version)
   };
 }
 

packages/metadata/src/decorate/storage/createFunction.spec.ts

@@ -28,7 +28,7 @@ describe('createFunction', (): void => {
       method: 'NextKeys',
       prefix: 'Session',
       section: 'session'
-    }, { metaVersion: 9 });
+    }, {});
 
     expect(
       u8aToHex(
@@ -49,4 +49,52 @@ describe('createFunction', (): void => {
       '5eb36b60f0fc4b9177116eba3e5cd57fea6289a57f5f5b9ffeb0475c66e7a521' // blake2
     );
   });
+
+  it('allows creating of NMap keys (with Bytes)', (): void => {
+    const storageFn = createFunction(registry, {
+      // eslint-disable-next-line @typescript-eslint/no-unsafe-assignment
+      meta: {
+        type: {
+          asNMap: {
+            hashers: [
+              registry.createType('StorageHasher', 'Twox64Concat'),
+              registry.createType('StorageHasher', 'Blake2_256'),
+              registry.createType('StorageHasher', 'Blake2_256')
+            ],
+            keyVec: [
+              new Text(registry, 'Bytes'),
+              new Text(registry, 'AccountId'),
+              new Text(registry, 'AccountId')
+            ],
+            value: new Text(registry, 'SessionKeys5')
+          },
+          isNMap: true
+        }
+      } as any,
+      method: 'NextKeys',
+      prefix: 'Session',
+      section: 'session'
+    }, {});
+
+    expect(
+      u8aToHex(
+        storageFn([
+          // hex, without length prefix
+          '0x3a73657373696f6e3a6b657973',
+          // addresses
+          'DB2mp5nNhbFN86J9hxoAog8JALMhDXgwvWMxrRMLNUFMEY4',
+          'DB2mp5nNhbFN86J9hxoAog8JALMhDXgwvWMxrRMLNUFMEY4'
+        ])
+      )
+    ).toEqual(
+      '0x' +
+      'd901' + // length
+      'cec5070d609dd3497f72bde07fc96ba0' + // twox 128
+      '4c014e6bf8b8c2c011e7290b85696bb3' + // twox 128
+      '9fe6329cc0b39e09' + // twox 64
+      '343a73657373696f6e3a6b657973' + // twox 64 (concat, with length)
+      '5eb36b60f0fc4b9177116eba3e5cd57fea6289a57f5f5b9ffeb0475c66e7a521' + // blake2
+      '5eb36b60f0fc4b9177116eba3e5cd57fea6289a57f5f5b9ffeb0475c66e7a521' // blake2
+    );
+  });
 });

packages/metadata/src/decorate/storage/createFunction.ts

@@ -8,16 +8,15 @@ import type { Codec, Registry } from '@polkadot/types/types';
 import BN from 'bn.js';
 
 import { Raw } from '@polkadot/types/codec';
-import { StorageKey } from '@polkadot/types/primitive';
-import { assert, compactAddLength, compactStripLength, isNull, isUndefined, stringLowerFirst, u8aConcat } from '@polkadot/util';
+import { StorageKey, Type } from '@polkadot/types/primitive';
+import { assert, compactAddLength, compactStripLength, isNull, isUndefined, stringCamelCase, stringify, stringLowerFirst, u8aConcat, u8aToU8a } from '@polkadot/util';
 import { xxhashAsU8a } from '@polkadot/util-crypto';
 
-import { getHasher, HasherFunction } from './getHasher';
+import { getHasher } from './getHasher';
 
 export interface CreateItemOptions {
   key?: string;
-  metaVersion: number;
-  skipHashing?: boolean; // We don't hash the keys defined in ./substrate.ts
+  skipHashing?: boolean;
 }
 
 export interface CreateItemFn {
@@ -34,81 +33,35 @@ interface IterFn {
 
 type CreateArgType = boolean | string | number | null | BN | BigInt | Uint8Array | Codec;
 
-const EMPTY_U8A = new Uint8Array([]);
-const NULL_HASHER = (value: Uint8Array): Uint8Array => value;
+type U8aHasher = (input: Uint8Array) => Uint8Array;
 
-// get the hashers, the base (and  in the case of DoubleMap), the second key
-/** @internal */
-function getHashers ({ meta: { type } }: CreateItemFn): [HasherFunction, HasherFunction?] {
-  if (type.isDoubleMap) {
-    return [
-      getHasher(type.asDoubleMap.hasher),
-      getHasher(type.asDoubleMap.key2Hasher)
-    ];
-  } else if (type.isMap) {
-    return [getHasher(type.asMap.hasher)];
-  }
-
-  // the default
-  return [getHasher()];
-}
-
-// create a base prefixed key
-/** @internal */
 function createPrefixedKey ({ method, prefix }: CreateItemFn): Uint8Array {
   return u8aConcat(xxhashAsU8a(prefix, 128), xxhashAsU8a(method, 128));
 }
 
-// create a key for a DoubleMap type
-/** @internal */
-function createKeyDoubleMap (registry: Registry, itemFn: CreateItemFn, args: [CreateArgType, CreateArgType], [hasher1, hasher2]: [HasherFunction, HasherFunction?]): Uint8Array {
-  const { meta: { name, type } } = itemFn;
+function createKey (registry: Registry, itemFn: CreateItemFn, keys: Type[], hashers: U8aHasher[], args: CreateArgType[]): Uint8Array {
+  const { method, section } = itemFn;
 
-  // since we are passing an almost-unknown through, trust, but verify
-  assert(Array.isArray(args) && !isUndefined(args[0]) && !isNull(args[0]) && !isUndefined(args[1]) && !isNull(args[1]), `${(name || 'unknown').toString()} is a DoubleMap and requires two arguments`);
-
-  // if this fails, we have bigger issues
-  assert(!isUndefined(hasher2), '2 hashing functions should be defined for DoubleMaps');
-
-  const [key1, key2] = args;
-  const map = type.asDoubleMap;
-  const val1 = registry.createType(map.key1.toString() as 'Raw', key1).toU8a();
-  const val2 = registry.createType(map.key2.toString() as 'Raw', key2).toU8a();
+  assert(Array.isArray(args) && args.length === keys.length, () => `Call to ${stringCamelCase(section || 'unknown')}.${stringCamelCase(method || 'unknown')} needs ${keys.length} arguments, found ${stringify(args)}`);
 
   // as per createKey, always add the length prefix (underlying it is Bytes)
-  return compactAddLength(u8aConcat(
-    createPrefixedKey(itemFn),
-    hasher1(val1),
-    hasher2(val2)
-  ));
-}
+  return compactAddLength(
+    u8aConcat(
+      createPrefixedKey(itemFn),
+      ...keys.map((type, index): Uint8Array => {
+        const arg = args[index];
 
-// create a key for either a map or a plain value
-/** @internal */
-function createKey (registry: Registry, itemFn: CreateItemFn, arg: CreateArgType, hasher: (value: Uint8Array) => Uint8Array): Uint8Array {
-  const { meta: { name, type } } = itemFn;
-  let param: Uint8Array = EMPTY_U8A;
+        assert(!isUndefined(arg) && !isNull(arg), () => `Call to ${stringCamelCase(section || 'unknown')}.${stringCamelCase(method || 'unknown')} has a null or undefined argument at position ${index}`);
 
-  if (type.isMap) {
-    const map = type.asMap;
-
-    assert(!isUndefined(arg) && !isNull(arg), `${name.toString()} is a Map and requires one argument`);
-
-    param = registry.createType(map.key.toString() as 'Raw', arg).toU8a();
-  }
-
-  // StorageKey is a Bytes, so is length-prefixed
-  return compactAddLength(u8aConcat(
-    createPrefixedKey(itemFn),
-    param.length
-      ? hasher(param)
-      : EMPTY_U8A
-  ));
+        return hashers[index](registry.createType(type.toString() as 'Raw', arg).toU8a());
+      })
+    )
+  );
 }
 
 // attach the metadata to expand to a StorageFunction
 /** @internal */
-function expandWithMeta ({ meta, method, prefix, section }: CreateItemFn, _storageFn: (arg?: CreateArgType | [CreateArgType?, CreateArgType?]) => Uint8Array): StorageEntry {
+function expandWithMeta ({ meta, method, prefix, section }: CreateItemFn, _storageFn: (arg?: CreateArgType | CreateArgType[]) => Uint8Array): StorageEntry {
   const storageFn = _storageFn as StorageEntry;
 
   storageFn.meta = meta;
@@ -155,6 +108,7 @@ function extendHeadMeta (registry: Registry, { meta: { documentation, name, type
 function extendPrefixedMap (registry: Registry, itemFn: CreateItemFn, storageFn: StorageEntry): StorageEntry {
   const { meta: { type } } = itemFn;
 
+  // FIXME NMap support
   storageFn.iterKey = extendHeadMeta(registry, itemFn, storageFn, (arg?: any): Raw => {
     assert(type.isDoubleMap || isUndefined(arg), 'Filtering arguments for keys/entries are only valid on double maps');
 
@@ -177,24 +131,31 @@ function extendPrefixedMap (registry: Registry, itemFn: CreateItemFn, storageFn:
 /** @internal */
 export function createFunction (registry: Registry, itemFn: CreateItemFn, options: CreateItemOptions): StorageEntry {
   const { meta: { type } } = itemFn;
-  const [hasher, key2Hasher] = getHashers(itemFn);
 
   // Can only have zero or one argument:
   //   - storage.system.account(address)
   //   - storage.timestamp.blockPeriod()
-  // For doublemap queries the params is passed in as an tuple, [key1, key2]
-  const storageFn = expandWithMeta(itemFn, (arg?: CreateArgType | [CreateArgType?, CreateArgType?]): Uint8Array =>
-    type.isDoubleMap
-      ? createKeyDoubleMap(registry, itemFn, arg as [CreateArgType, CreateArgType], [hasher, key2Hasher])
-      : createKey(registry, itemFn, arg as CreateArgType, options.skipHashing ? NULL_HASHER : hasher)
+  // For higher-map queries the params are passed in as an tuple, [key1, key2]
+  const storageFn = expandWithMeta(itemFn, (arg?: CreateArgType | CreateArgType[]) =>
+    type.isPlain
+      ? options.skipHashing
+        ? compactAddLength(u8aToU8a(options.key))
+        : createKey(registry, itemFn, [], [], [])
+      : type.isMap
+        ? createKey(registry, itemFn, [type.asMap.key], [getHasher(type.asMap.hasher)], [arg as CreateArgType])
+        : type.isDoubleMap
+          ? createKey(registry, itemFn, [type.asDoubleMap.key1, type.asDoubleMap.key2], [getHasher(type.asDoubleMap.hasher), getHasher(type.asDoubleMap.key2Hasher)], arg as CreateArgType[])
+          : createKey(registry, itemFn, type.asNMap.keyVec, type.asNMap.hashers.map((h) => getHasher(h)), arg as CreateArgType[])
   );
 
+  // FIXME NMap support
   if (type.isMap || type.isDoubleMap) {
     extendPrefixedMap(registry, itemFn, storageFn);
   }
 
   storageFn.keyPrefix = (arg?: any): Uint8Array =>
-    (storageFn.iterKey && storageFn.iterKey(arg)) || compactStripLength(storageFn())[1];
+    (storageFn.iterKey && storageFn.iterKey(arg)) ||
+    compactStripLength(storageFn())[1];
 
   return storageFn;
 }

packages/metadata/src/decorate/storage/getHasher.ts

@@ -31,6 +31,6 @@ const HASHERS: Record<keyof typeof AllHashers, HasherFunction> = {
 };
 
 /** @internal */
-export function getHasher (hasher?: StorageHasher): HasherFunction {
-  return HASHERS[hasher?.type as 'Identity'] || DEFAULT_FN;
+export function getHasher (hasher: StorageHasher): HasherFunction {
+  return HASHERS[hasher.type as 'Identity'] || DEFAULT_FN;
 }

packages/metadata/src/decorate/storage/getStorage.spec.ts

@@ -2,19 +2,21 @@
 // SPDX-License-Identifier: Apache-2.0
 
 import { TypeRegistry } from '@polkadot/types/create';
+import { compactAddLength, u8aToU8a } from '@polkadot/util';
 
 import { getStorage } from './getStorage';
 
 describe('getSorage', (): void => {
   const registry = new TypeRegistry();
-  const storage = getStorage(registry, 8);
+  const storage = getStorage(registry);
 
   it('should return well known keys', (): void => {
     expect(typeof storage.substrate).toBe('object');
 
+    expect(storage.substrate.code()).toEqual(compactAddLength(u8aToU8a(':code')));
+
     expect(storage.substrate.changesTrieConfig).toBeTruthy();
     expect(storage.substrate.childStorageKeyPrefix).toBeTruthy();
-    expect(storage.substrate.code).toBeTruthy();
     expect(storage.substrate.extrinsicIndex).toBeTruthy();
     expect(storage.substrate.heapPages).toBeTruthy();
   });

packages/metadata/src/decorate/storage/getStorage.ts

@@ -5,15 +5,15 @@ import type { StorageEntry } from '@polkadot/types/primitive/types';
 import type { Registry } from '@polkadot/types/types';
 import type { Storage } from '../types';
 
-import * as substrate from './substrate';
+import { substrate } from './substrate';
 
 /** @internal */
-export function getStorage (registry: Registry, metaVersion: number): Storage {
+export function getStorage (registry: Registry): Storage {
   return {
     substrate: Object
       .entries(substrate)
       .reduce((storage: Record<string, StorageEntry>, [key, fn]): Record<string, StorageEntry> => {
-        (storage as Record<string, unknown>)[key] = fn(registry, metaVersion);
+        (storage as Record<string, unknown>)[key] = fn(registry);
 
         return storage;
       }, {})

packages/metadata/src/decorate/storage/index.spec.ts

@@ -21,7 +21,11 @@ describe('decorateStorage', (): void => {
     const query = decorateStorage(registry, metadata.asLatest, metadata.version);
 
     it('should throw if the storage function expects an argument', (): void => {
-      expect(() => query.balances.account()).toThrowError(/requires one argument/);
+      expect(() => query.balances.account()).toThrowError('Call to balances.account has a null or undefined argument at position 0');
+    });
+
+    it('should throw if the storage function expects multiple arguments', (): void => {
+      expect(() => query.staking.erasStakers([1])).toThrowError('Call to staking.erasStakers needs 2 arguments, found [1]');
     });
 
     it('should return a value if the storage function does not expect an argument', (): void => {

packages/metadata/src/decorate/storage/index.ts

@@ -11,7 +11,8 @@ import { createFunction } from './createFunction';
 import { getStorage } from './getStorage';
 
 /** @internal */
-export function decorateStorage (registry: Registry, { modules }: MetadataLatest, metaVersion: number): Storage {
+// eslint-disable-next-line @typescript-eslint/no-unused-vars
+export function decorateStorage (registry: Registry, { modules }: MetadataLatest, _metaVersion: number): Storage {
   return modules.reduce((result: Storage, moduleMetadata): Storage => {
     if (moduleMetadata.storage.isNone) {
       return result;
@@ -31,11 +32,11 @@ export function decorateStorage (registry: Registry, { modules }: MetadataLatest
         method,
         prefix,
         section
-      }, { metaVersion });
+      }, {});
 
       return newModule;
     }, {} as ModuleStorage);
 
     return result;
-  }, { ...getStorage(registry, metaVersion) });
+  }, { ...getStorage(registry) });
 }

packages/metadata/src/decorate/storage/substrate.spec.ts

@@ -0,0 +1,19 @@
+// Copyright 2017-2021 @polkadot/metadata authors & contributors
+// SPDX-License-Identifier: Apache-2.0
+
+import { TypeRegistry } from '@polkadot/types/create';
+import { compactAddLength, u8aToU8a } from '@polkadot/util';
+
+import { substrate } from './substrate';
+
+const registry = new TypeRegistry();
+
+describe('substrate', (): void => {
+  it('creates a well-known :code key', (): void => {
+    expect(
+      substrate.code(registry)()
+    ).toEqual(
+      compactAddLength(u8aToU8a(':code'))
+    );
+  });
+});

packages/metadata/src/decorate/storage/substrate.ts

@@ -12,44 +12,45 @@ interface SubstrateMetadata {
   type: string;
 }
 
+type Creator = (registry: Registry) => StorageEntry;
+
 // Small helper function to factorize code on this page.
 /** @internal */
-function createRuntimeFunction (method: string, key: string, { documentation, type }: SubstrateMetadata): (registry: Registry, metaVersion: number) => StorageEntry {
-  return (registry: Registry, metaVersion: number): StorageEntry =>
+function createRuntimeFunction (method: string, key: string, { documentation, type }: SubstrateMetadata): (registry: Registry) => StorageEntry {
+  return (registry: Registry): StorageEntry =>
     createFunction(registry, {
       meta: {
         documentation: registry.createType('Vec<Text>', [documentation]),
         modifier: registry.createType('StorageEntryModifierLatest', 1), // required
+        name: registry.createType('Text', method),
         toJSON: (): any => key,
         type: registry.createType('StorageEntryTypeLatest', type, 0)
       } as StorageEntryMetadataLatest,
       method,
       prefix: 'Substrate',
       section: 'substrate'
-    }, { key, metaVersion, skipHashing: true });
+    }, { key, skipHashing: true });
 }
 
-export const code = createRuntimeFunction('code', ':code', {
-  documentation: 'Wasm code of the runtime.',
-  type: 'Bytes'
-});
-
-export const heapPages = createRuntimeFunction('heapPages', ':heappages', {
-  documentation: 'Number of wasm linear memory pages required for execution of the runtime.',
-  type: 'u64'
-});
-
-export const extrinsicIndex = createRuntimeFunction('extrinsicIndex', ':extrinsic_index', {
-  documentation: 'Current extrinsic index (u32) is stored under this key.',
-  type: 'u32'
-});
-
-export const changesTrieConfig = createRuntimeFunction('changesTrieConfig', ':changes_trie', {
-  documentation: 'Changes trie configuration is stored under this key.',
-  type: 'u32'
-});
-
-export const childStorageKeyPrefix = createRuntimeFunction('childStorageKeyPrefix', ':child_storage:', {
-  documentation: 'Prefix of child storage keys.',
-  type: 'u32'
-});
+export const substrate: Record<string, Creator> = {
+  changesTrieConfig: createRuntimeFunction('changesTrieConfig', ':changes_trie', {
+    documentation: ' Changes trie configuration is stored under this key.',
+    type: 'u32'
+  }),
+  childStorageKeyPrefix: createRuntimeFunction('childStorageKeyPrefix', ':child_storage:', {
+    documentation: ' Prefix of child storage keys.',
+    type: 'u32'
+  }),
+  code: createRuntimeFunction('code', ':code', {
+    documentation: ' Wasm code of the runtime.',
+    type: 'Bytes'
+  }),
+  extrinsicIndex: createRuntimeFunction('extrinsicIndex', ':extrinsic_index', {
+    documentation: ' Current extrinsic index (u32) is stored under this key.',
+    type: 'u32'
+  }),
+  heapPages: createRuntimeFunction('heapPages', ':heappages', {
+    documentation: ' Number of wasm linear memory pages required for execution of the runtime.',
+    type: 'u64'
+  })
+};

packages/metadata/src/packageInfo.ts

@@ -3,4 +3,4 @@
 
 // Auto-generated by @polkadot/dev, do not edit
 
-export const packageInfo = { name: '@polkadot/metadata', version: '4.2.1' };
+export const packageInfo = { name: '@polkadot/metadata', version: '4.9.1' };

packages/metadata/src/static.ts

@@ -1,6 +1,6 @@
 // Copyright 2017-2021 @polkadot/metadata authors & contributors
 // SPDX-License-Identifier: Apache-2.0
 
-import metadata from './v12/static';
+import metadata from './v13/static';
 
 export default metadata;

packages/metadata/src/util/flattenUniq.ts

@@ -4,16 +4,16 @@
 type Types = string | Types[];
 
 /** @internal */
-export function flattenUniq (list: Types[]): string[] {
-  const flat = list.reduce((result: string[], entry): string[] => {
-    return result.concat(
-      Array.isArray(entry)
-        ? flattenUniq(entry)
-        : entry
-    );
-  }, []);
+export function flattenUniq (list: Types[], start: string[] = []): string[] {
+  return [...new Set(
+    list.reduce((result: string[], entry): string[] => {
+      if (Array.isArray(entry)) {
+        return flattenUniq(entry, result);
+      }
 
-  return [...new Set(flat)]
-    .filter((value: string) => value)
-    .sort();
+      result.push(entry);
+
+      return result;
+    }, start)
+  )];
 }

packages/metadata/src/util/getUniqTypes.ts

@@ -12,10 +12,11 @@ type Arg = { type: Type } & Codec;
 
 type Item = {
   type: {
-    isDoubleMap?: boolean;
+    isDoubleMap: boolean;
     isMap: boolean;
+    isNMap: boolean;
     isPlain: boolean;
-    asDoubleMap?: {
+    asDoubleMap: {
       key1: Text;
       key2: Text;
       value: Text;
@@ -24,16 +25,15 @@ type Item = {
       key: Text;
       value: Text;
     };
+    asNMap: {
+      keyVec: Text[];
+      value: Text;
+    },
     asPlain: Text;
   };
 } & Codec;
 
-type Storage = Option<Vec<Item> | {
-  // v0
-  functions?: Vec<Item>;
-  // V7+
-  items?: Vec<Item>;
-} & Codec>;
+type Storage = Option<{ items: Vec<Item> } & Codec>;
 
 type Call = { args: Vec<Arg> } & Codec;
 
@@ -44,12 +44,6 @@ type Event = { args: Vec<Type> } & Codec;
 type Events = Option<Vec<Event>>;
 
 type Module = {
-  // V0
-  module?: {
-    call: {
-      functions: Vec<Call>;
-    };
-  };
   // V1+
   calls?: Calls;
   // V6+
@@ -60,30 +54,25 @@ type Module = {
 
 interface ExtractionMetadata {
   modules: Vec<Module>;
-
-  // V0
-  outerEvent?: {
-    events: Vec<[Text, Vec<Event>] & Codec>;
-  };
 }
 
 /** @internal */
 function unwrapCalls (mod: Module): Call[] {
   return mod.calls
     ? mod.calls.unwrapOr([])
-    // V0
-    : mod.module
-      ? mod.module.call.functions
-      : [];
+    : [];
+}
+
+/** @internal */
+function typeToString ({ type }: Arg): string {
+  return type.toString();
 }
 
 /** @internal */
 function getCallNames ({ modules }: ExtractionMetadata): string[][][] {
   return modules.map((mod): string[][] =>
     unwrapCalls(mod).map(({ args }): string[] =>
-      args.map((arg): string =>
-        arg.type.toString()
-      )
+      args.map(typeToString)
     )
   );
 }
@@ -91,90 +80,66 @@ function getCallNames ({ modules }: ExtractionMetadata): string[][][] {
 /** @internal */
 function getConstantNames ({ modules }: ExtractionMetadata): string[][] {
   return modules.map(({ constants }): string[] =>
-    constants
-      ? constants.map((constant): string =>
-        constant.type.toString()
-      )
-      : []
+    (constants || []).map(typeToString)
   );
 }
 
 /** @internal */
 function unwrapEvents (events?: Events): Event[] {
-  if (!events) {
-    return [];
-  }
-
-  return events.unwrapOr([]);
+  return events
+    ? events.unwrapOr([])
+    : [];
 }
 
 /** @internal */
-function getEventNames ({ modules, outerEvent }: ExtractionMetadata): string[][][] {
-  const mapArg = ({ args }: Event): string[] =>
-    args.map((arg): string =>
-      arg.toString()
-    );
-
-  // V0
-  if (outerEvent) {
-    return outerEvent.events.map(([, events]): string[][] =>
-      events.map(mapArg)
-    );
-  }
-
-  // V1+
+function getEventNames ({ modules }: ExtractionMetadata): string[][][] {
   return modules.map(({ events }): string[][] =>
-    unwrapEvents(events).map(mapArg)
+    unwrapEvents(events).map(({ args }: Event): string[] =>
+      args.map((a) => a.toString())
+    )
   );
 }
 
 /** @internal */
 function unwrapStorage (storage?: Storage): Item[] {
-  if (!storage) {
-    return [];
-  }
-
-  const data = storage.unwrapOr([]);
-
-  return Array.isArray(data)
-    ? data
-    : (data.items || data.functions) as Item[];
+  return storage
+    ? storage.unwrapOr({ items: [] }).items
+    : [];
 }
 
 /** @internal */
 function getStorageNames ({ modules }: ExtractionMetadata): string[][][] {
   return modules.map(({ storage }): string[][] =>
-    unwrapStorage(storage).map(({ type }): string[] => {
-      if (type.isDoubleMap && type.asDoubleMap) {
-        return [
-          type.asDoubleMap.key1.toString(),
-          type.asDoubleMap.key2.toString(),
-          type.asDoubleMap.value.toString()
-        ];
-      } else if (type.isMap) {
-        return [
-          type.asMap.key.toString(),
-          type.asMap.value.toString()
-        ];
-      } else {
-        return [
+    unwrapStorage(storage).map(({ type }) =>
+      type.isPlain
+        ? [
           type.asPlain.toString()
-        ];
-      }
-    })
+        ]
+        : type.isMap
+          ? [
+            type.asMap.value.toString(),
+            type.asMap.key.toString()
+          ]
+          : type.isDoubleMap
+            ? [
+              type.asDoubleMap.value.toString(),
+              type.asDoubleMap.key1.toString(),
+              type.asDoubleMap.key2.toString()
+            ]
+            : [
+              type.asNMap.value.toString(),
+              ...type.asNMap.keyVec.map((k) => k.toString())
+            ]
+    )
   );
 }
 
 /** @internal */
 export function getUniqTypes (registry: Registry, meta: ExtractionMetadata, throwError: boolean): string[] {
-  const types = flattenUniq([
+  return validateTypes(registry, throwError, flattenUniq([
     getCallNames(meta),
     getConstantNames(meta),
     getEventNames(meta),
     getStorageNames(meta)
-  ]);
-
-  validateTypes(registry, types, throwError);
-
-  return types;
+  ]));
 }

packages/metadata/src/util/testUtil.ts

@@ -5,7 +5,7 @@ import type { Codec, Registry } from '@polkadot/types/types';
 import type { MetadataInterface } from '../types';
 
 import { unwrapStorageType } from '@polkadot/types/primitive/StorageKey';
-import { assert, hexToU8a, u8aToHex } from '@polkadot/util';
+import { assert, hexToU8a, stringify, u8aToHex } from '@polkadot/util';
 
 import { Metadata } from '../Metadata';
 import { getUniqTypes } from './getUniqTypes';
@@ -22,7 +22,7 @@ export function decodeLatestSubstrate<Modules extends Codec> (registry: Registry
       expect((metadata[`asV${version}` as keyof Metadata] as unknown as MetadataInterface<Modules>).modules.length).not.toBe(0);
       expect(metadata.toJSON()).toEqual(staticSubstrate);
     } catch (error) {
-      console.error(JSON.stringify(metadata.toJSON()));
+      console.error(stringify(metadata.toJSON()));
 
       throw error;
     }
@@ -65,7 +65,7 @@ export function defaultValues (registry: Registry, rpcData: string, withThrow =
               if (withFallbackCheck) {
                 const [hexType, hexOrig] = [u8aToHex(type.toU8a()), u8aToHex(fallback.toU8a(true))];
 
-                assert(hexType === hexOrig, `Fallback does not match (${((hexOrig.length - 2) / 2) - ((hexType.length - 2) / 2)} bytes missing): ${hexType} !== ${hexOrig}`);
+                assert(hexType === hexOrig, () => `Fallback does not match (${((hexOrig.length - 2) / 2) - ((hexType.length - 2) / 2)} bytes missing): ${hexType} !== ${hexOrig}`);
               }
             } catch (error) {
               const message = `${location}:: ${(error as Error).message}`;

packages/metadata/src/util/validateTypes.ts

@@ -11,8 +11,10 @@ import { flattenUniq } from './flattenUniq';
 const l = logger('metadata');
 
 /** @internal */
-export function validateTypes (registry: Registry, types: string[], throwError: boolean): void {
-  const missing = flattenUniq(extractTypes(types)).filter((type) => !registry.hasType(type));
+export function validateTypes (registry: Registry, throwError: boolean, types: string[]): string[] {
+  const missing = flattenUniq(extractTypes(types))
+    .filter((type) => !registry.hasType(type))
+    .sort();
 
   if (missing.length !== 0) {
     const message = `Unknown types found, no types for ${missing.join(', ')}`;
@@ -23,4 +25,6 @@ export function validateTypes (registry: Registry, types: string[], throwError:
       l.warn(message);
     }
   }
+
+  return types;
 }

packages/metadata/src/v12/static-substrate.json

@@ -499,7 +499,7 @@
             {
               "name": "Version",
               "type": "RuntimeVersion",
-              "value": "0x106e6f6465387375627374726174652d6e6f64650a000000090100000000000034df6acb689907609b0300000037e397fc7c91f5e40100000040fe3ad401f8959a04000000d2bc9897eed08f1502000000f78b278be53f454c02000000ed99c5acb25eedf502000000cbca25e39f14238702000000687ad44ad37f03c201000000bc9d89904f5b923f0100000068b66ba122c93fa70100000037c8bb1350a9a2a80100000091d5df18b0d2cf5801000000ab3c0572291feb8b0100000002000000",
+              "value": "0x106e6f6465387375627374726174652d6e6f64650a000000090100000100000034df6acb689907609b0300000037e397fc7c91f5e40100000040fe3ad401f8959a04000000d2bc9897eed08f1502000000f78b278be53f454c02000000ed99c5acb25eedf502000000cbca25e39f14238702000000687ad44ad37f03c201000000bc9d89904f5b923f0100000068b66ba122c93fa70100000037c8bb1350a9a2a80100000091d5df18b0d2cf5801000000ab3c0572291feb8b0100000002000000",
               "documentation": [
                 " Get the chain's current version."
               ]
@@ -947,7 +947,7 @@
             {
               "name": "EpochDuration",
               "type": "u64",
-              "value": "0x1400000000000000",
+              "value": "0xc800000000000000",
               "documentation": [
                 " The amount of time, in slots, that each epoch should last.",
                 " NOTE: Currently it is not possible to change the epoch duration after",
@@ -1392,7 +1392,38 @@
               ]
             }
           ],
-          "errors": [],
+          "errors": [
+            {
+              "name": "NotAssigned",
+              "documentation": [
+                " The index was not already assigned."
+              ]
+            },
+            {
+              "name": "NotOwner",
+              "documentation": [
+                " The index is assigned to another account."
+              ]
+            },
+            {
+              "name": "InUse",
+              "documentation": [
+                " The index was not available."
+              ]
+            },
+            {
+              "name": "NotTransfer",
+              "documentation": [
+                " The source and destination accounts are identical."
+              ]
+            },
+            {
+              "name": "Permanent",
+              "documentation": [
+                " The index is permanent and may not be freed/changed."
+              ]
+            }
+          ],
           "index": 5
         },
         {
@@ -1971,7 +2002,7 @@
             {
               "name": "UnsignedPhase",
               "type": "BlockNumber",
-              "value": "0x05000000",
+              "value": "0x32000000",
               "documentation": [
                 " Duration of the unsigned phase."
               ]
@@ -1979,7 +2010,7 @@
             {
               "name": "SignedPhase",
               "type": "BlockNumber",
-              "value": "0x05000000",
+              "value": "0x32000000",
               "documentation": [
                 " Duration of the signed phase."
               ]
@@ -3257,6 +3288,14 @@
                 " For each validator only the `$MaxNominatorRewardedPerValidator` biggest stakers can claim",
                 " their reward. This used to limit the i/o cost for the nominator payout."
               ]
+            },
+            {
+              "name": "MaxNominations",
+              "type": "u32",
+              "value": "0x10000000",
+              "documentation": [
+                " Maximum number of nominations per nominator."
+              ]
             }
           ],
           "errors": [
@@ -5747,7 +5786,7 @@
         {
           "name": "Elections",
           "storage": {
-            "prefix": "PhragmenElection",
+            "prefix": "Elections",
             "items": [
               {
                 "name": "Members",
@@ -5917,8 +5956,8 @@
                 "   origin is removed as a runner-up.",
                 " - `origin` is a current member. In this case, the deposit is unreserved and origin is",
                 "   removed as a member, consequently not being a candidate for the next round anymore.",
-                "   Similar to [`remove_members`], if replacement runners exists, they are immediately used.",
-                "   If the prime is renouncing, then no prime will exist until the next round.",
+                "   Similar to [`remove_members`], if replacement runners exists, they are immediately",
+                "   used. If the prime is renouncing, then no prime will exist until the next round.",
                 "",
                 " The dispatch origin of this call must be signed, and have one of the above roles.",
                 "",
@@ -5989,10 +6028,11 @@
                 "Vec<(AccountId,Balance)>"
               ],
               "documentation": [
-                " A new term with \\[new_members\\]. This indicates that enough candidates existed to run the",
-                " election, not that enough have has been elected. The inner value must be examined for",
-                " this purpose. A `NewTerm(\\[\\])` indicates that some candidates got their bond slashed and",
-                " none were elected, whilst `EmptyTerm` means that no candidates existed to begin with."
+                " A new term with \\[new_members\\]. This indicates that enough candidates existed to run",
+                " the election, not that enough have has been elected. The inner value must be examined",
+                " for this purpose. A `NewTerm(\\[\\])` indicates that some candidates got their bond",
+                " slashed and none were elected, whilst `EmptyTerm` means that no candidates existed to",
+                " begin with."
               ]
             },
             {
@@ -6054,47 +6094,66 @@
             }
           ],
           "constants": [
+            {
+              "name": "PalletId",
+              "type": "LockIdentifier",
+              "value": "0x706872656c656374",
+              "documentation": [
+                " Identifier for the elections-phragmen pallet's lock"
+              ]
+            },
             {
               "name": "CandidacyBond",
               "type": "BalanceOf",
               "value": "0x0080c6a47e8d03000000000000000000",
-              "documentation": []
+              "documentation": [
+                " How much should be locked up in order to submit one's candidacy."
+              ]
             },
             {
               "name": "VotingBondBase",
               "type": "BalanceOf",
               "value": "0x00f0436de36a01000000000000000000",
-              "documentation": []
+              "documentation": [
+                " Base deposit associated with voting.",
+                "",
+                " This should be sensibly high to economically ensure the pallet cannot be attacked by",
+                " creating a gigantic number of votes."
+              ]
             },
             {
               "name": "VotingBondFactor",
               "type": "BalanceOf",
               "value": "0x0000cc7b9fae00000000000000000000",
-              "documentation": []
+              "documentation": [
+                " The amount of bond that need to be locked for each vote (32 bytes)."
+              ]
             },
             {
               "name": "DesiredMembers",
               "type": "u32",
               "value": "0x0d000000",
-              "documentation": []
+              "documentation": [
+                " Number of members to elect."
+              ]
             },
             {
               "name": "DesiredRunnersUp",
               "type": "u32",
               "value": "0x07000000",
-              "documentation": []
+              "documentation": [
+                " Number of runners_up to keep."
+              ]
             },
             {
               "name": "TermDuration",
               "type": "BlockNumber",
               "value": "0x80130300",
-              "documentation": []
-            },
-            {
-              "name": "ModuleId",
-              "type": "LockIdentifier",
-              "value": "0x706872656c656374",
-              "documentation": []
+              "documentation": [
+                " How long each seat is kept. This defines the next block number at which an election",
+                " round will happen. If set to zero, no elections are ever triggered and the module will",
+                " be in passive mode."
+              ]
             }
           ],
           "errors": [
@@ -6383,7 +6442,20 @@
             }
           ],
           "constants": [],
-          "errors": [],
+          "errors": [
+            {
+              "name": "AlreadyMember",
+              "documentation": [
+                " Already a member."
+              ]
+            },
+            {
+              "name": "NotMember",
+              "documentation": [
+                " Not a member."
+              ]
+            }
+          ],
           "index": 15
         },
         {
@@ -6825,8 +6897,8 @@
               ]
             },
             {
-              "name": "ModuleId",
-              "type": "ModuleId",
+              "name": "PalletId",
+              "type": "PalletId",
               "value": "0x70792f7472737279",
               "documentation": [
                 " The treasury's module id, used for deriving its sovereign account ID."
@@ -6845,6 +6917,12 @@
               "documentation": [
                 " No proposal or bounty at that index."
               ]
+            },
+            {
+              "name": "TooManyApprovals",
+              "documentation": [
+                " Too many approvals in the queue."
+              ]
             }
           ],
           "index": 17
@@ -7026,7 +7104,7 @@
                 " * `gas_limit`: The gas limit enforced when executing the constructor.",
                 " * `code`: The contract code to deploy in raw bytes.",
                 " * `data`: The input data to pass to the contract constructor.",
-                " * `salt`: Used for the address derivation. See [`Self::contract_address`].",
+                " * `salt`: Used for the address derivation. See [`Pallet::contract_address`].",
                 "",
                 " Instantiation is executed as follows:",
                 "",
@@ -9569,12 +9647,20 @@
               ]
             },
             {
-              "name": "ModuleId",
-              "type": "ModuleId",
+              "name": "PalletId",
+              "type": "PalletId",
               "value": "0x70792f736f636965",
               "documentation": [
                 " The societies's module id"
               ]
+            },
+            {
+              "name": "MaxCandidateIntake",
+              "type": "u32",
+              "value": "0x0a000000",
+              "documentation": [
+                " Maximum candidate intake per round."
+              ]
             }
           ],
           "errors": [
@@ -10274,7 +10360,7 @@
                 " Unlock any vested funds of the sender account.",
                 "",
                 " The dispatch origin for this call must be _Signed_ and the sender must have funds still",
-                " locked under this module.",
+                " locked under this pallet.",
                 "",
                 " Emits either `VestingCompleted` or `VestingUpdated`.",
                 "",
@@ -10300,7 +10386,7 @@
                 " The dispatch origin for this call must be _Signed_.",
                 "",
                 " - `target`: The account whose vested funds should be unlocked. Must have funds still",
-                " locked under this module.",
+                " locked under this pallet.",
                 "",
                 " Emits either `VestingCompleted` or `VestingUpdated`.",
                 "",
@@ -10409,7 +10495,7 @@
               "type": "BalanceOf",
               "value": "0x0000c16ff28623000000000000000000",
               "documentation": [
-                " The minimum amount to be transferred to create a new vesting schedule."
+                " The minimum amount transferred to call `vested_transfer`."
               ]
             }
           ],
@@ -12614,8 +12700,7 @@
                 "",
                 " The origin must be Signed and the sender must have sufficient funds free.",
                 "",
-                " Funds of sender are reserved according to the formula:",
-                " `AssetDepositBase + AssetDepositPerZombie * max_zombies`.",
+                " Funds of sender are reserved by `AssetDeposit`.",
                 "",
                 " Parameters:",
                 " - `id`: The identifier of the new asset. This must not be currently in use to identify",
@@ -13902,8 +13987,8 @@
           ],
           "constants": [
             {
-              "name": "ModuleId",
-              "type": "ModuleId",
+              "name": "PalletId",
+              "type": "PalletId",
               "value": "0x70792f6c6f74746f",
               "documentation": []
             },
@@ -13914,7 +13999,56 @@
               "documentation": []
             }
           ],
-          "errors": [],
+          "errors": [
+            {
+              "name": "Overflow",
+              "documentation": [
+                " An overflow has occurred."
+              ]
+            },
+            {
+              "name": "NotConfigured",
+              "documentation": [
+                " A lottery has not been configured."
+              ]
+            },
+            {
+              "name": "InProgress",
+              "documentation": [
+                " A lottery is already in progress."
+              ]
+            },
+            {
+              "name": "AlreadyEnded",
+              "documentation": [
+                " A lottery has already ended."
+              ]
+            },
+            {
+              "name": "InvalidCall",
+              "documentation": [
+                " The call is not valid for an open lottery."
+              ]
+            },
+            {
+              "name": "AlreadyParticipating",
+              "documentation": [
+                " You are already participating in the lottery with this call."
+              ]
+            },
+            {
+              "name": "TooManyCalls",
+              "documentation": [
+                " Too many calls for a single lottery."
+              ]
+            },
+            {
+              "name": "EncodingFailed",
+              "documentation": [
+                " Failed to encode calls"
+              ]
+            }
+          ],
           "index": 36
         },
         {

packages/metadata/src/v12/toV13.ts

@@ -0,0 +1,12 @@
+// Copyright 2017-2021 @polkadot/metadata authors & contributors
+// SPDX-License-Identifier: Apache-2.0
+
+import type { MetadataV12, MetadataV13 } from '@polkadot/types/interfaces/metadata';
+import type { Registry } from '@polkadot/types/types';
+
+/**
+ * @internal
+ **/
+export function toV13 (registry: Registry, metadata: MetadataV12): MetadataV13 {
+  return registry.createType('MetadataV13', metadata);
+}

packages/metadata/src/v13/Metadata.spec.ts

@@ -0,0 +1,18 @@
+// Copyright 2017-2021 @polkadot/metadata authors & contributors
+// SPDX-License-Identifier: Apache-2.0
+
+import { TypeRegistry } from '@polkadot/types/create';
+
+import { decodeLatestSubstrate, defaultValues, toLatest } from '../util/testUtil';
+import substrateData from './static';
+import substrateJson from './static-substrate.json';
+
+describe('MetadataV13 (substrate)', (): void => {
+  const registry = new TypeRegistry();
+
+  decodeLatestSubstrate(registry, 13, substrateData, substrateJson);
+
+  toLatest(registry, 13, substrateData);
+
+  defaultValues(registry, substrateData, true, true);
+});

packages/metadata/src/v13/toLatest.ts

@@ -1,7 +1,7 @@
 // Copyright 2017-2021 @polkadot/metadata authors & contributors
 // SPDX-License-Identifier: Apache-2.0
 
-import type { EventMetadataLatest, EventMetadataV12, FunctionMetadataLatest, FunctionMetadataV12, MetadataLatest, MetadataV12, ModuleConstantMetadataLatest, ModuleConstantMetadataV12, ModuleMetadataLatest, ModuleMetadataV12, StorageEntryMetadataLatest, StorageMetadataLatest, StorageMetadataV12 } from '@polkadot/types/interfaces/metadata';
+import type { EventMetadataLatest, EventMetadataV13, FunctionMetadataLatest, FunctionMetadataV13, MetadataLatest, MetadataV13, ModuleConstantMetadataLatest, ModuleConstantMetadataV13, ModuleMetadataLatest, ModuleMetadataV13, StorageEntryMetadataLatest, StorageMetadataLatest, StorageMetadataV13 } from '@polkadot/types/interfaces/metadata';
 import type { OverrideModuleType, Registry } from '@polkadot/types/types';
 
 import { Type } from '@polkadot/types/primitive';
@@ -16,11 +16,13 @@ const KNOWN_ORIGINS: Record<string, string> = {
   TechnicalCommittee: 'CollectiveOrigin'
 };
 
+const BOXES = [['<', '>'], ['<', ','], [',', '>'], ['(', ')'], ['(', ','], [',', ','], [',', ')']];
+
 /**
  * Find and apply the correct type override
  * @internal
  **/
-function setTypeOverride (sectionTypes: OverrideModuleType, ...types: Type[]): void {
+function setTypeOverride (sectionTypes: OverrideModuleType, types: Type[]): void {
   types.forEach((type): void => {
     const override = Object.keys(sectionTypes).find((aliased) => type.eq(aliased));
 
@@ -29,9 +31,11 @@ function setTypeOverride (sectionTypes: OverrideModuleType, ...types: Type[]): v
     } else {
       // FIXME: NOT happy with this approach, but gets over the initial hump cased by (Vec<Announcement>,BalanceOf)
       const orig = type.toString();
-      const alias = Object.entries(sectionTypes).reduce((result: string, [from, to]): string =>
-        [['<', '>'], ['<', ','], [',', '>'], ['(', ')'], ['(', ','], [',', ','], [',', ')']].reduce((result, [one, two]): string =>
-          result.replace(`${one}${from}${two}`, `${one}${to}${two}`), result), orig);
+      const alias = Object
+        .entries(sectionTypes)
+        .reduce((result: string, [from, to]) =>
+          BOXES.reduce((result, [one, two]) =>
+            result.replace(`${one}${from}${two}`, `${one}${to}${two}`), result), orig);
 
       if (orig !== alias) {
         type.setOverride(alias);
@@ -44,9 +48,9 @@ function setTypeOverride (sectionTypes: OverrideModuleType, ...types: Type[]): v
  * Apply module-specific type overrides (always be done as part of toLatest)
  * @internal
  **/
-function convertCalls (registry: Registry, calls: FunctionMetadataV12[], sectionTypes: OverrideModuleType): FunctionMetadataLatest[] {
+function convertCalls (registry: Registry, calls: FunctionMetadataV13[], sectionTypes: OverrideModuleType): FunctionMetadataLatest[] {
   return calls.map((c): FunctionMetadataLatest => {
-    c.args.forEach(({ type }) => setTypeOverride(sectionTypes, type));
+    setTypeOverride(sectionTypes, c.args.map(({ type }) => type));
 
     return registry.createType('FunctionMetadataLatest', c);
   });
@@ -56,9 +60,9 @@ function convertCalls (registry: Registry, calls: FunctionMetadataV12[], section
  * Apply module-specific type overrides (always be done as part of toLatest)
  * @internal
  */
-function convertConstants (registry: Registry, constants: ModuleConstantMetadataV12[], sectionTypes: OverrideModuleType): ModuleConstantMetadataLatest[] {
+function convertConstants (registry: Registry, constants: ModuleConstantMetadataV13[], sectionTypes: OverrideModuleType): ModuleConstantMetadataLatest[] {
   return constants.map((c): ModuleConstantMetadataLatest => {
-    setTypeOverride(sectionTypes, c.type);
+    setTypeOverride(sectionTypes, [c.type]);
 
     return registry.createType('ModuleConstantMetadataLatest', c);
   });
@@ -68,9 +72,9 @@ function convertConstants (registry: Registry, constants: ModuleConstantMetadata
  * Apply module-specific type overrides (always be done as part of toLatest)
  * @internal
  **/
-function convertEvents (registry: Registry, events: EventMetadataV12[], sectionTypes: OverrideModuleType): EventMetadataLatest[] {
+function convertEvents (registry: Registry, events: EventMetadataV13[], sectionTypes: OverrideModuleType): EventMetadataLatest[] {
   return events.map((e): EventMetadataLatest => {
-    e.args.forEach((type) => setTypeOverride(sectionTypes, type));
+    setTypeOverride(sectionTypes, e.args.map((type) => type));
 
     return registry.createType('EventMetadataLatest', e);
   });
@@ -80,16 +84,18 @@ function convertEvents (registry: Registry, events: EventMetadataV12[], sectionT
  * Apply module-specific storage type overrides (always part of toLatest)
  * @internal
  **/
-function convertStorage (registry: Registry, { items, prefix }: StorageMetadataV12, sectionTypes: OverrideModuleType): StorageMetadataLatest {
+function convertStorage (registry: Registry, { items, prefix }: StorageMetadataV13, sectionTypes: OverrideModuleType): StorageMetadataLatest {
   return registry.createType('StorageMetadataLatest', {
     items: items.map((s): StorageEntryMetadataLatest => {
-      if (s.type.isDoubleMap) {
-        setTypeOverride(sectionTypes, s.type.asDoubleMap.value, s.type.asDoubleMap.key1, s.type.asDoubleMap.key2);
-      } else if (s.type.isMap) {
-        setTypeOverride(sectionTypes, s.type.asMap.value, s.type.asMap.key);
-      } else {
-        setTypeOverride(sectionTypes, s.type.asPlain);
-      }
+      setTypeOverride(sectionTypes,
+        s.type.isPlain
+          ? [s.type.asPlain]
+          : s.type.isMap
+            ? [s.type.asMap.value, s.type.asMap.key]
+            : s.type.isDoubleMap
+              ? [s.type.asDoubleMap.value, s.type.asDoubleMap.key1, s.type.asDoubleMap.key2]
+              : [s.type.asNMap.value, ...s.type.asNMap.keyVec]
+      );
 
       return registry.createType('StorageEntryMetadataLatest', s);
     }),
@@ -98,13 +104,15 @@ function convertStorage (registry: Registry, { items, prefix }: StorageMetadataV
 }
 
 // generate & register the OriginCaller type
-function registerOriginCaller (registry: Registry, modules: ModuleMetadataV12[], metaVersion: number): void {
+function registerOriginCaller (registry: Registry, modules: ModuleMetadataV13[], metaVersion: number): void {
   registry.register({
     OriginCaller: {
       _enum: modules
         .map((mod, index): [string, number] => [
           mod.name.toString(),
-          metaVersion >= 12 ? mod.index.toNumber() : index
+          metaVersion >= 12
+            ? mod.index.toNumber()
+            : index
         ])
         .sort((a, b) => a[1] - b[1])
         .reduce((result: Record<string, string>, [name, index]): Record<string, string> => {
@@ -121,7 +129,7 @@ function registerOriginCaller (registry: Registry, modules: ModuleMetadataV12[],
 }
 
 /** @internal */
-function createModule (registry: Registry, mod: ModuleMetadataV12, { calls, constants, events, storage }: { calls: FunctionMetadataV12[] | null, constants: ModuleConstantMetadataV12[], events: EventMetadataV12[] | null, storage: StorageMetadataV12 | null }): ModuleMetadataLatest {
+function createModule (registry: Registry, mod: ModuleMetadataV13, { calls, constants, events, storage }: { calls: FunctionMetadataV13[] | null, constants: ModuleConstantMetadataV13[], events: EventMetadataV13[] | null, storage: StorageMetadataV13 | null }): ModuleMetadataLatest {
   const sectionTypes = getModuleTypes(registry, stringCamelCase(mod.name));
 
   return registry.createType('ModuleMetadataLatest', {
@@ -138,7 +146,7 @@ function createModule (registry: Registry, mod: ModuleMetadataV12, { calls, cons
  * most-recent metadata, since it allows us a chance to actually apply call and storage specific type aliasses
  * @internal
  **/
-export function toLatest (registry: Registry, { extrinsic, modules }: MetadataV12, metaVersion: number): MetadataLatest {
+export function toLatest (registry: Registry, { extrinsic, modules }: MetadataV13, metaVersion: number): MetadataLatest {
   registerOriginCaller(registry, modules, metaVersion);
 
   return registry.createType('MetadataLatest', {

packages/rpc-core/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@polkadot/rpc-core",
-  "version": "4.2.1",
+  "version": "4.9.1",
   "type": "module",
   "description": "A JavaScript wrapper for the Polkadot JsonRPC interface",
   "main": "index.js",
@@ -8,6 +8,9 @@
     "./detectPackage.js",
     "./detectPackage.cjs"
   ],
+  "engines": {
+    "node": ">=14.0.0"
+  },
   "author": "Jaco Greeff <jacogr@gmail.com>",
   "maintainers": [],
   "contributors": [],
@@ -16,14 +19,14 @@
   "bugs": "https://github.com/polkadot-js/api/issues",
   "homepage": "https://github.com/polkadot-js/api/tree/master/packages/rpc-core#readme",
   "dependencies": {
-    "@babel/runtime": "^7.13.10",
-    "@polkadot/metadata": "4.2.1",
-    "@polkadot/rpc-provider": "4.2.1",
-    "@polkadot/types": "4.2.1",
-    "@polkadot/util": "^6.0.5",
-    "@polkadot/x-rxjs": "^6.0.5"
+    "@babel/runtime": "^7.14.0",
+    "@polkadot/metadata": "4.9.1",
+    "@polkadot/rpc-provider": "4.9.1",
+    "@polkadot/types": "4.9.1",
+    "@polkadot/util": "^6.3.1",
+    "@polkadot/x-rxjs": "^6.3.1"
   },
   "devDependencies": {
-    "@polkadot/keyring": "^6.0.5"
+    "@polkadot/keyring": "^6.3.1"
   }
 }

packages/rpc-core/src/index.ts

@@ -84,7 +84,7 @@ export class RpcCore {
 
   #registryDefault: Registry;
 
-  #getBlockRegistry?: (blockHash: string | Uint8Array) => Promise<{ registry: Registry }>;
+  #getBlockRegistry?: (blockHash: Uint8Array) => Promise<{ registry: Registry }>;
 
   readonly #storageCache = new Map<string, string | null>();
 
@@ -140,8 +140,10 @@ export class RpcCore {
   /**
    * @description Sets a registry swap (typically from Api)
    */
-  public setRegistrySwap (registrySwap: (blockHash: string | Uint8Array) => Promise<{ registry: Registry }>): void {
-    this.#getBlockRegistry = registrySwap;
+  public setRegistrySwap (registrySwap: (blockHash: Uint8Array) => Promise<{ registry: Registry }>): void {
+    this.#getBlockRegistry = memoize(registrySwap, {
+      getInstanceId: () => this.#instanceId
+    });
   }
 
   public addUserInterfaces<Section extends keyof RpcInterface> (userRpc: Record<string, Record<string, DefinitionRpc | DefinitionRpcSub>>): void {
@@ -211,9 +213,9 @@ export class RpcCore {
     const callWithRegistry = async (outputAs: OutputType, values: any[]): Promise<Codec | Codec[]> => {
       const blockHash = hashIndex === -1
         ? null
-        : values[hashIndex] as Uint8Array;
+        : values[hashIndex] as (Uint8Array | string | null | undefined);
       const { registry } = blockHash && this.#getBlockRegistry
-        ? await this.#getBlockRegistry(blockHash)
+        ? await this.#getBlockRegistry(u8aToU8a(blockHash))
         : { registry: this.#registryDefault };
       const params = this._formatInputs(registry, null, def, values);
       const data = await this.provider.send(rpcName, params.map((param): AnyJson => param.toJSON())) as AnyJson;
@@ -343,7 +345,7 @@ export class RpcCore {
       ? ''
       : ` (${def.params.length - reqArgCount} optional)`;
 
-    assert(inputs.length >= reqArgCount && inputs.length <= def.params.length, `Expected ${def.params.length} parameters${optText}, ${inputs.length} found instead`);
+    assert(inputs.length >= reqArgCount && inputs.length <= def.params.length, () => `Expected ${def.params.length} parameters${optText}, ${inputs.length} found instead`);
 
     return inputs.map((input, index): Codec =>
       createTypeUnsafe(registry, def.params[index].type, [input], { blockHash })
@@ -438,27 +440,23 @@ export class RpcCore {
       ? ''
       : ` entry ${entryIndex}:`;
 
-    if (meta.modifier.isOptional) {
-      let inner = null;
-
-      if (!isEmpty) {
-        try {
-          inner = createTypeUnsafe(registry, type, [input], { blockHash, isPedantic: true });
-        } catch (error) {
-          l.error(`Unable to decode storage ${key.section || 'unknown'}.${key.method || 'unknown'}:${entryNum}`, (error as Error).message);
-        }
-      }
-
-      const option = new Option(registry, createClass(registry, type), inner);
-
-      if (blockHash) {
-        option.createdAtHash = registry.createType('Hash', blockHash);
-      }
-
-      return option;
-    }
-
     try {
+      if (meta.modifier.isOptional) {
+        let inner = null;
+
+        if (!isEmpty) {
+          inner = createTypeUnsafe(registry, type, [input], { blockHash, isPedantic: true });
+        }
+
+        const option = new Option(registry, createClass(registry, type), inner);
+
+        if (blockHash) {
+          option.createdAtHash = registry.createType('Hash', blockHash);
+        }
+
+        return option;
+      }
+
       return createTypeUnsafe(registry, type, [
         isEmpty
           ? meta.fallback
@@ -467,9 +465,7 @@ export class RpcCore {
           : input
       ], { blockHash, isPedantic: true });
     } catch (error) {
-      l.error(`Unable to decode storage ${key.section || 'unknown'}.${key.method || 'unknown'}:${entryNum}`, (error as Error).message);
-
-      return createTypeUnsafe(registry, 'Raw', [input], { blockHash });
+      throw new Error(`Unable to decode storage ${key.section || 'unknown'}.${key.method || 'unknown'}:${entryNum}: ${(error as Error).message}`);
     }
   }
 }

packages/rpc-core/src/packageInfo.ts

@@ -3,4 +3,4 @@
 
 // Auto-generated by @polkadot/dev, do not edit
 
-export const packageInfo = { name: '@polkadot/rpc-core', version: '4.2.1' };
+export const packageInfo = { name: '@polkadot/rpc-core', version: '4.9.1' };

packages/rpc-core/src/util/drr.ts

@@ -3,7 +3,7 @@
 
 import type { Observable } from '@polkadot/x-rxjs';
 
-import { logger } from '@polkadot/util';
+import { logger, stringify } from '@polkadot/util';
 import { catchError, distinctUntilChanged, publishReplay, refCount, tap } from '@polkadot/x-rxjs/operators';
 
 import { refCountDelay } from './refCountDelay';
@@ -19,10 +19,10 @@ interface Options {
 const l = logger('drr');
 
 const CMP = (a: unknown, b: unknown): boolean =>
-  JSON.stringify({ t: a }) === JSON.stringify({ t: b });
+  stringify({ t: a }) === stringify({ t: b });
 
 const ERR = (error: Error): Observable<never> => {
-  l.error(error);
+  l.error(error.message);
 
   throw error;
 };

packages/rpc-provider/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@polkadot/rpc-provider",
-  "version": "4.2.1",
+  "version": "4.9.1",
   "type": "module",
   "description": "Transport providers for the API",
   "main": "index.js",
@@ -8,6 +8,9 @@
     "./detectPackage.js",
     "./detectPackage.cjs"
   ],
+  "engines": {
+    "node": ">=14.0.0"
+  },
   "author": "Jaco Greeff <jacogr@gmail.com>",
   "maintainers": [],
   "contributors": [],
@@ -16,19 +19,19 @@
   "bugs": "https://github.com/polkadot-js/api/issues",
   "homepage": "https://github.com/polkadot-js/api/tree/master/packages/rpc-provider#readme",
   "dependencies": {
-    "@babel/runtime": "^7.13.10",
-    "@polkadot/types": "4.2.1",
-    "@polkadot/util": "^6.0.5",
-    "@polkadot/util-crypto": "^6.0.5",
-    "@polkadot/x-fetch": "^6.0.5",
-    "@polkadot/x-global": "^6.0.5",
-    "@polkadot/x-ws": "^6.0.5",
+    "@babel/runtime": "^7.14.0",
+    "@polkadot/types": "4.9.1",
+    "@polkadot/util": "^6.3.1",
+    "@polkadot/util-crypto": "^6.3.1",
+    "@polkadot/x-fetch": "^6.3.1",
+    "@polkadot/x-global": "^6.3.1",
+    "@polkadot/x-ws": "^6.3.1",
     "bn.js": "^4.11.9",
     "eventemitter3": "^4.0.7"
   },
   "devDependencies": {
-    "@polkadot/keyring": "^6.0.5",
-    "@polkadot/metadata": "4.2.1",
+    "@polkadot/keyring": "^6.3.1",
+    "@polkadot/metadata": "4.9.1",
     "mock-socket": "^9.0.3",
     "nock": "^13.0.11"
   }

packages/rpc-provider/src/coder/index.ts

@@ -3,7 +3,7 @@
 
 import type { JsonRpcRequest, JsonRpcResponse, JsonRpcResponseBaseError } from '../types';
 
-import { assert, isNumber, isString, isUndefined } from '@polkadot/util';
+import { assert, isNumber, isString, isUndefined, stringify } from '@polkadot/util';
 
 function formatErrorData (data?: string | number): string {
   if (isUndefined(data)) {
@@ -12,7 +12,7 @@ function formatErrorData (data?: string | number): string {
 
   const formatted = `: ${isString(data)
     ? data.replace(/Error\("/g, '').replace(/\("/g, '(').replace(/"\)/g, ')').replace(/\(/g, ', ').replace(/\)/g, '')
-    : JSON.stringify(data)}`;
+    : stringify(data)}`;
 
   // We need some sort of cut-off here since these can be very large and
   // very nested, pick a number and trim the result display to it
@@ -47,7 +47,7 @@ export class RpcCoder {
   }
 
   public encodeJson (method: string, params: any | any[]): string {
-    return JSON.stringify(
+    return stringify(
       this.encodeObject(method, params)
     );
   }

packages/rpc-provider/src/http/index.ts

@@ -44,7 +44,7 @@ export class HttpProvider implements ProviderInterface {
    * @param {string} endpoint The endpoint url starting with http://
    */
   constructor (endpoint: string = defaults.HTTP_URL, headers: Record<string, string> = {}) {
-    assert(/^(https|http):\/\//.test(endpoint), `Endpoint should start with 'http://', received '${endpoint}'`);
+    assert(/^(https|http):\/\//.test(endpoint), () => `Endpoint should start with 'http://', received '${endpoint}'`);
 
     this.#coder = new RpcCoder();
     this.#endpoint = endpoint;
@@ -116,7 +116,7 @@ export class HttpProvider implements ProviderInterface {
       method: 'POST'
     });
 
-    assert(response.ok, `[${response.status}]: ${response.statusText}`);
+    assert(response.ok, () => `[${response.status}]: ${response.statusText}`);
 
     // eslint-disable-next-line @typescript-eslint/no-unsafe-assignment
     const result = await response.json();

packages/rpc-provider/src/mock/index.ts

@@ -60,7 +60,7 @@ export class MockProvider implements ProviderInterface {
     chain_getFinalizedHead: () => this.registry.createType('Header', rpcHeader.result).hash,
     // eslint-disable-next-line @typescript-eslint/no-unsafe-member-access
     chain_getHeader: () => this.registry.createType('Header', rpcHeader.result).toJSON(),
-    rpc_methods: (): string[] => [],
+    rpc_methods: () => this.registry.createType('RpcMethods').toJSON(),
     state_getKeys: (): string[] => [],
     state_getKeysPaged: (): string[] => [],
     state_getMetadata: (): string => rpcMetadata,
@@ -69,6 +69,7 @@ export class MockProvider implements ProviderInterface {
     system_chain: (): string => 'mockChain',
     system_name: (): string => 'mockClient',
     system_properties: (): Record<string, number | string> => ({ ss58Format: 42 }),
+    system_upgradedToTripleRefCount: () => this.registry.createType('bool', true),
     system_version: (): string => '9.8.7'
   };
 
@@ -121,16 +122,18 @@ export class MockProvider implements ProviderInterface {
 
   // eslint-disable-next-line @typescript-eslint/require-await
   public async send (method: string, params: any[]): Promise<unknown> {
-    assert(this.requests[method], `provider.send: Invalid method '${method}'`);
+    l.debug(() => ['send', method, params]);
+
+    assert(this.requests[method], () => `provider.send: Invalid method '${method}'`);
 
     return this.requests[method](this.db, params);
   }
 
   // eslint-disable-next-line @typescript-eslint/require-await
   public async subscribe (type: string, method: string, ...params: unknown[]): Promise<number> {
-    l.debug((): any => ['subscribe', method, params]);
+    l.debug(() => ['subscribe', method, params]);
 
-    assert(this.subscriptions[method], `provider.subscribe: Invalid method '${method}'`);
+    assert(this.subscriptions[method], () => `provider.subscribe: Invalid method '${method}'`);
 
     const callback = params.pop() as MockStateSubscriptionCallback;
     const id = ++this.subscriptionId;
@@ -151,7 +154,7 @@ export class MockProvider implements ProviderInterface {
 
     l.debug((): any => ['unsubscribe', id, sub]);
 
-    assert(sub, `Unable to find subscription for ${id}`);
+    assert(sub, () => `Unable to find subscription for ${id}`);
 
     delete this.subscriptionMap[id];
     delete this.subscriptions[sub].callbacks[id];

packages/rpc-provider/src/packageInfo.ts

@@ -3,4 +3,4 @@
 
 // Auto-generated by @polkadot/dev, do not edit
 
-export const packageInfo = { name: '@polkadot/rpc-provider', version: '4.2.1' };
+export const packageInfo = { name: '@polkadot/rpc-provider', version: '4.9.1' };

packages/rpc-provider/src/ws/index.ts

@@ -32,13 +32,13 @@ interface WsStateSubscription extends SubscriptionHandler {
   params: any[];
 }
 
-const ALIASSES: { [index: string]: string } = {
+const ALIASES: { [index: string]: string } = {
   chain_finalisedHead: 'chain_finalizedHead',
   chain_subscribeFinalisedHeads: 'chain_subscribeFinalizedHeads',
   chain_unsubscribeFinalisedHeads: 'chain_unsubscribeFinalizedHeads'
 };
 
-const RETRY_DELAY = 1000;
+const RETRY_DELAY = 2500;
 
 const l = logger('api-ws');
 
@@ -109,7 +109,7 @@ export class WsProvider implements ProviderInterface {
     assert(endpoints.length !== 0, 'WsProvider requires at least one Endpoint');
 
     endpoints.forEach((endpoint) => {
-      assert(/^(wss|ws):\/\//.test(endpoint), `Endpoint should start with 'ws://', received '${endpoint}'`);
+      assert(/^(wss|ws):\/\//.test(endpoint), () => `Endpoint should start with 'ws://', received '${endpoint}'`);
     });
 
     this.#eventemitter = new EventEmitter();
@@ -225,7 +225,6 @@ export class WsProvider implements ProviderInterface {
 
       // 1000 - Normal closure; the connection successfully completed
       this.#websocket.close(1000);
-      this.#websocket = null;
     } catch (error) {
       l.error(error);
 
@@ -351,9 +350,19 @@ export class WsProvider implements ProviderInterface {
     }
 
     this.#isConnected = false;
+
+    if (this.#websocket) {
+      this.#websocket.onclose = null;
+      this.#websocket.onerror = null;
+      this.#websocket.onmessage = null;
+      this.#websocket.onopen = null;
+      this.#websocket = null;
+    }
+
     this.#emit('disconnected');
+
     // reject all hanging requests
-    eraseRecord(this.#handlers, (handler) => handler.callback(error, undefined));
+    eraseRecord(this.#handlers, (h) => h.callback(error, undefined));
     eraseRecord(this.#waitingForId);
 
     if (this.#autoConnectMs > 0) {
@@ -419,7 +428,7 @@ export class WsProvider implements ProviderInterface {
   }
 
   #onSocketMessageSubscribe = (response: JsonRpcResponse): void => {
-    const method = ALIASSES[response.method as string] || response.method || 'invalid';
+    const method = ALIASES[response.method as string] || response.method || 'invalid';
     const subId = `${method}::${response.params.subscription}`;
     const handler = this.#subscriptions[subId];
 

packages/rpc-provider/test/mockWs.ts

@@ -3,6 +3,8 @@
 
 import { Server, WebSocket } from 'mock-socket';
 
+import { stringify } from '@polkadot/util';
+
 interface Scope {
   body: { [index: string]: Record<string, any> };
   requests: number;
@@ -79,7 +81,7 @@ export function mockWs (requests: ({ method: string } & ErrorDef)[], wsUrl: stri
       scope.body[request.method] = body;
       requestCount++;
 
-      socket.send(JSON.stringify(response));
+      socket.send(stringify(response));
     });
   });
 

packages/typegen/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@polkadot/typegen",
-  "version": "4.2.1",
+  "version": "4.9.1",
   "type": "module",
   "description": "Type generation scripts",
   "main": "index.js",
@@ -8,6 +8,9 @@
     "./detectPackage.js",
     "./detectPackage.cjs"
   ],
+  "engines": {
+    "node": ">=14.0.0"
+  },
   "bin": {
     "polkadot-types-chain-info": "./scripts/polkadot-types-chain-info.cjs",
     "polkadot-types-from-chain": "./scripts/polkadot-types-from-chain.cjs",
@@ -23,20 +26,20 @@
   "bugs": "https://github.com/polkadot-js/api/issues",
   "homepage": "https://github.com/polkadot-js/api/tree/master/packages/typegen#readme",
   "dependencies": {
-    "@babel/core": "^7.13.10",
-    "@babel/register": "^7.13.8",
-    "@babel/runtime": "^7.13.10",
-    "@polkadot/api": "4.2.1",
-    "@polkadot/metadata": "4.2.1",
-    "@polkadot/rpc-provider": "4.2.1",
-    "@polkadot/types": "4.2.1",
-    "@polkadot/util": "^6.0.5",
+    "@babel/core": "^7.14.0",
+    "@babel/register": "^7.13.16",
+    "@babel/runtime": "^7.14.0",
+    "@polkadot/api": "4.9.1",
+    "@polkadot/metadata": "4.9.1",
+    "@polkadot/rpc-provider": "4.9.1",
+    "@polkadot/types": "4.9.1",
+    "@polkadot/util": "^6.3.1",
     "handlebars": "^4.7.7",
-    "websocket": "^1.0.33",
-    "yargs": "^16.2.0"
+    "websocket": "^1.0.34",
+    "yargs": "^17.0.1"
   },
   "devDependencies": {
     "@types/websocket": "^1.0.2",
-    "@types/yargs": "^16.0.0"
+    "@types/yargs": "^16.0.1"
   }
 }

packages/typegen/src/generate/query.ts

@@ -25,38 +25,60 @@ function entrySignature (allDefs: Record<string, ModuleTypes>, registry: Registr
 
     return ['', '', formatType(allDefs, outputType, imports)];
   } else if (storageEntry.type.isMap) {
+    const map = storageEntry.type.asMap;
+
     // Find similar types of the `key` type
-    const similarTypes = getSimilarTypes(registry, allDefs, storageEntry.type.asMap.key.toString(), imports);
+    const similarTypes = getSimilarTypes(registry, allDefs, map.key.toString(), imports);
 
     setImports(allDefs, imports, [
       ...similarTypes,
-      storageEntry.type.asMap.value.toString()
+      map.value.toString()
     ]);
 
     return [
-      formatType(allDefs, storageEntry.type.asMap.key.toString(), imports),
+      formatType(allDefs, map.key.toString(), imports),
       `arg: ${similarTypes.join(' | ')}`,
       formatType(allDefs, outputType, imports)
     ];
   } else if (storageEntry.type.isDoubleMap) {
+    const dm = storageEntry.type.asDoubleMap;
+
     // Find similar types of `key1` and `key2` types
-    const similarTypes1 = getSimilarTypes(registry, allDefs, storageEntry.type.asDoubleMap.key1.toString(), imports);
-    const similarTypes2 = getSimilarTypes(registry, allDefs, storageEntry.type.asDoubleMap.key2.toString(), imports);
+    const similarTypes1 = getSimilarTypes(registry, allDefs, dm.key1.toString(), imports);
+    const similarTypes2 = getSimilarTypes(registry, allDefs, dm.key2.toString(), imports);
 
     setImports(allDefs, imports, [
       ...similarTypes1,
       ...similarTypes2,
-      storageEntry.type.asDoubleMap.value.toString()
+      dm.value.toString()
     ]);
 
     const key1Types = similarTypes1.join(' | ');
     const key2Types = similarTypes2.join(' | ');
 
     return [
-      [formatType(allDefs, storageEntry.type.asDoubleMap.key1.toString(), imports), formatType(allDefs, storageEntry.type.asDoubleMap.key2.toString(), imports)].join(', '),
+      [formatType(allDefs, dm.key1.toString(), imports), formatType(allDefs, dm.key2.toString(), imports)].join(', '),
       `key1: ${key1Types}, key2: ${key2Types}`,
       formatType(allDefs, outputType, imports)
     ];
+  } else if (storageEntry.type.isNMap) {
+    const nmap = storageEntry.type.asNMap;
+
+    // Find similar types for all keys
+    const similarTypes = nmap.keyVec.map((k) => getSimilarTypes(registry, allDefs, k.toString(), imports));
+
+    setImports(allDefs, imports, [
+      ...similarTypes.reduce<string[]>((all, t) => all.concat(t), []),
+      nmap.value.toString()
+    ]);
+
+    const keyTypes = similarTypes.map((t) => t.join(' | '));
+
+    return [
+      nmap.keyVec.map((k) => formatType(allDefs, k.toString(), imports)).join(', '),
+      keyTypes.map((t, i) => `key${i + 1}: ${t}`).join(', '),
+      formatType(allDefs, outputType, imports)
+    ];
   }
 
   throw new Error(`entryArgs: Cannot parse args of entry ${storageEntry.name.toString()}`);

packages/typegen/src/generate/tsDef.ts

@@ -10,7 +10,7 @@ import path from 'path';
 import { getTypeDef } from '@polkadot/types/create';
 import { TypeDefInfo } from '@polkadot/types/create/types';
 import * as defaultDefinitions from '@polkadot/types/interfaces/definitions';
-import { assert, isString, stringCamelCase, stringUpperFirst } from '@polkadot/util';
+import { assert, isString, stringCamelCase, stringify, stringUpperFirst } from '@polkadot/util';
 
 import { createImports, exportInterface, exportType, formatType, readTemplate, setImports, TypeImports, writeFile } from '../util';
 
@@ -192,7 +192,7 @@ function generateInterfaces (definitions: Record<string, ModuleTypes>, { types }
   };
 
   return Object.entries(types).map(([name, type]): [string, string] => {
-    const def = getTypeDef(isString(type) ? type : JSON.stringify(type), { name });
+    const def = getTypeDef(isString(type) ? type : stringify(type), { name });
 
     return [name, generators[def.info](definitions, def, imports)];
   });

packages/typegen/src/metadataMd.ts

@@ -7,6 +7,7 @@ import type { Codec, DefinitionRpcParam } from '@polkadot/types/types';
 import fs from 'fs';
 
 import { Metadata } from '@polkadot/metadata';
+import { getStorage as getSubstrateStorage } from '@polkadot/metadata/decorate/storage/getStorage';
 import rpcdata from '@polkadot/metadata/static';
 import { Vec } from '@polkadot/types/codec';
 import { TypeRegistry } from '@polkadot/types/create';
@@ -16,15 +17,17 @@ import { Text } from '@polkadot/types/primitive';
 import { unwrapStorageType } from '@polkadot/types/primitive/StorageKey';
 import { stringCamelCase, stringLowerFirst } from '@polkadot/util';
 
+interface SectionItem {
+  link?: string;
+  name: string;
+  [bullet: string]: undefined | string | Vec<Text>;
+}
+
 interface Section {
   link?: string;
   name: string;
   description?: string;
-  items: {
-    link?: string;
-    name: string;
-    [bullet: string]: undefined | string | Vec<Text>;
-  }[];
+  items: SectionItem[];
 }
 
 interface Page {
@@ -156,7 +159,7 @@ function addRpc (): string {
           });
 
         return all;
-      }, []).sort((a, b) => a.name.localeCompare(b.name)),
+      }, []).sort(sortByName),
     title: 'JSON-RPC'
   });
 }
@@ -192,10 +195,11 @@ function addConstants (metadata: MetadataLatest): string {
 
 /** @internal */
 function addStorage (metadata: MetadataLatest): string {
+  const { substrate } = getSubstrateStorage(metadata.registry);
   const moduleSections = metadata.modules
     .sort(sortByName)
     .filter((moduleMetadata) => !moduleMetadata.storage.isNone)
-    .map((moduleMetadata) => {
+    .map((moduleMetadata): Section => {
       const sectionName = stringLowerFirst(moduleMetadata.name);
 
       return {
@@ -206,7 +210,9 @@ function addStorage (metadata: MetadataLatest): string {
               ? ('`' + func.type.asMap.key.toString() + '`')
               : func.type.isDoubleMap
                 ? ('`' + func.type.asDoubleMap.key1.toString() + ', ' + func.type.asDoubleMap.key2.toString() + '`')
-                : '';
+                : func.type.isNMap
+                  ? ('`' + func.type.asNMap.keyVec.map((k) => k.toString()).join(', ') + '`')
+                  : '';
             const methodName = stringLowerFirst(func.name);
             const outputType = unwrapStorageType(func.type, func.modifier.isOptional);
 
@@ -220,12 +226,27 @@ function addStorage (metadata: MetadataLatest): string {
       };
     });
 
-  // eslint-disable-next-line @typescript-eslint/no-unsafe-assignment
-  const knownSection: any = JSON.parse(fs.readFileSync('docs/substrate/storage-known-section.json', 'utf8'));
-
   return renderPage({
     description: DESC_STORAGE,
-    sections: moduleSections.concat([knownSection]),
+    sections: moduleSections.concat([{
+      description: 'These are well-known keys that are always available to the runtime implementation of any Substrate-based network.',
+      items: Object.entries(substrate).map(([name, { meta }]) => {
+        const arg = meta.type.isMap
+          ? ('`' + meta.type.asMap.key.toString() + '`')
+          : meta.type.isDoubleMap
+            ? ('`' + meta.type.asDoubleMap.key1.toString() + ', ' + meta.type.asDoubleMap.key2.toString() + '`')
+            : '';
+        const methodName = stringLowerFirst(name);
+        const outputType = unwrapStorageType(meta.type, meta.modifier.isOptional);
+
+        return {
+          interface: '`' + `api.query.substrate.${methodName}` + '`',
+          name: `${methodName}(${arg}): ` + '`' + outputType + '`',
+          summary: meta.documentation
+        };
+      }),
+      name: 'substrate'
+    }]).sort(sortByName),
     title: 'Storage'
   });
 }

packages/typegen/src/packageInfo.ts

@@ -3,4 +3,4 @@
 
 // Auto-generated by @polkadot/dev, do not edit
 
-export const packageInfo = { name: '@polkadot/typegen', version: '4.2.1' };
+export const packageInfo = { name: '@polkadot/typegen', version: '4.9.1' };

packages/typegen/src/util/derived.ts

@@ -12,7 +12,7 @@ import { GenericAccountId, GenericLookupSource, GenericVote } from '@polkadot/ty
 import { AllConvictions } from '@polkadot/types/interfaces/democracy/definitions';
 import { Null } from '@polkadot/types/primitive';
 import * as primitiveClasses from '@polkadot/types/primitive';
-import { isChildClass } from '@polkadot/util';
+import { isChildClass, stringify } from '@polkadot/util';
 
 import { isCompactEncodable } from './class';
 import { formatType } from './formatting';
@@ -98,7 +98,7 @@ export function getSimilarTypes (registry: Registry, definitions: Record<string,
 
         possibleTypes.push(`([${subs.join(', ')}])[]`);
       } else {
-        throw new Error(`Unhandled subtype in Vec, ${JSON.stringify(subDef)}`);
+        throw new Error(`Unhandled subtype in Vec, ${stringify(subDef)}`);
       }
     }
   } else if (isChildClass(Enum, Clazz)) {

packages/typegen/src/util/formatting.ts

@@ -7,7 +7,7 @@ import Handlebars from 'handlebars';
 
 import { getTypeDef, paramsNotation } from '@polkadot/types/create';
 import { TypeDefInfo } from '@polkadot/types/create/types';
-import { isString } from '@polkadot/util';
+import { isString, stringify } from '@polkadot/util';
 
 import { readTemplate } from './file';
 import { ModuleTypes, setImports, TypeImports } from './imports';
@@ -284,7 +284,7 @@ export function formatType (definitions: Record<string, ModuleTypes>, type: stri
     }
 
     default: {
-      throw new Error(`Cannot format ${JSON.stringify(type)}`);
+      throw new Error(`Cannot format ${stringify(type)}`);
     }
   }
 }

packages/types-known/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@polkadot/types-known",
-  "version": "4.2.1",
+  "version": "4.9.1",
   "type": "module",
   "description": "A JavaScript wrapper for the Polkadot JsonRPC interface",
   "main": "index.js",
@@ -8,6 +8,9 @@
     "./detectPackage.js",
     "./detectPackage.cjs"
   ],
+  "engines": {
+    "node": ">=14.0.0"
+  },
   "author": "Jaco Greeff <jacogr@gmail.com>",
   "maintainers": [],
   "contributors": [],
@@ -16,10 +19,10 @@
   "bugs": "https://github.com/polkadot-js/api/issues",
   "homepage": "https://github.com/polkadot-js/api/tree/master/packages/types-known#readme",
   "dependencies": {
-    "@babel/runtime": "^7.13.10",
-    "@polkadot/networks": "^6.0.5",
-    "@polkadot/types": "4.2.1",
-    "@polkadot/util": "^6.0.5",
+    "@babel/runtime": "^7.14.0",
+    "@polkadot/networks": "^6.3.1",
+    "@polkadot/types": "4.9.1",
+    "@polkadot/util": "^6.3.1",
     "bn.js": "^4.11.9"
   },
   "devDependencies": {