[CI Skip] 0.98.1

skip-checks: true

.123trigger

@@ -1 +1 @@
-8
+13

.github/workflows/push-master.yml

@@ -53,3 +53,15 @@ jobs:
       run: |
         yarn install --frozen-lockfile
         yarn polkadot-dev-ghact-docs
+
+  dummy:
+    name: Dummy
+    if: "contains(github.event.head_commit.message, '[CI Skip]')"
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        node-version: [12.x]
+    steps:
+    - name: dummy
+      run: |
+        echo "Dummy skip step"

CHANGELOG.md

@@ -1,3 +1,24 @@
+# 0.98.1
+
+- Make the TypeScript generation script mor re-usable (Thanks to @xlc)
+- Add `.entries()` for all map-types, returning storage keys and values
+- Add `.signAsync` to submittable extrinsics (Thanks to @c410-f3r)
+- Cleanup circular references between internal dependencies
+- Support for new Substrate democracy with preimages in derive
+- Alignment of types with Substrate/Polkadot master branches
+
+# 0.97.1
+
+- **Breaking change** Add the passing on an explicit `Registry` to all calls to `createType` as well as all codec classes. If you are upgrading form an older version, use the [migration guide](UPGRADING.md) to walk through the changes required.
+- **Breaking change** The metadata classes have been moved from `@polkadot/types` into `@polkadot/metadata`. If you explicitly create `Metadata` classes, you now would need to do so with the `import Metadata from '@polkadot/metadata';` This is a decorated class, where the previous metadata is now available on `Metadata.metadata`, alongside decorated `consts.*`', `tx.*` and `query.*`.
+- **Breaking change** Session `Keys` defaults to 4 keys now (Substrate master), depending on your node config, you would want to override with e.g. `Keys: 'SessionKeys3'` (1, 2, 3, 4, 5 & 6)
+- **Breaking change** `ValidatorPrefs` now default to the new percentage commission model, to use the previous `validatorPayment`, you can override with `ValidatorPrefs: 'ValidatorPrefsTo196'`
+- `api.derive.*` now has built-in memomization
+- Various fixes and cleanups to `api.derive.*` including era length progress and vesting calculations
+- Aligned all types with the latest Polkadot & Substrate, including Kusama CC3
+- Support for Metadata V9, which adjusts the hashing methods of storage entries
+- Support for Metadata V10 (still un-released), which includes prefixed map support
+
 # 0.96.1
 
 - Updated types for latest Kusama

UPGRADING.md

@@ -2,7 +2,91 @@
 
 This is an upgrade guide for users of the API. It does not attempt to detail each version (the [CHANGELOG](CHANGELOG.md) has all the changes between versions), but rather tries to explain the rationale behind major breaking changes and how users of the API should handle this.
 
-While we try to keep the user-facing interfaces as stable as possible, sometimes you just need to make additions to move forward and improve things down the road, as painful as they may be. Like you, we are also users of the API, and eat our own dogfood - and as such, feel any pains introduced first.
+While we try to keep the user-facing interfaces as stable as possible, sometimes you just need to make additions to move forward and improve things down the road, as painful as they may be. Like you, we are also users of the API, and eat our own dog food - and as such, feel any pains introduced first.
+
+
+## 0.97.1 (and newer)
+
+The 0.97 series lays the groundwork to allow type registration  to be ties to a specific chain and a specific Api instance. In the past, 2 Api instances in the same process would share types, which mean that you could not connect to 2 independent chains with different types. This is very problematic for Polkadot chains, where the idea is to connect to multiple chains.
+
+When using the Api, a new `Registry` will be created on using `new Api(...)` or `Api.create(...)` and this will be transparently passed when creating types. In the cases where you create type instances explicitly or create type classes for injection, you would need to make adjustments.
+
+### Type classes
+
+In a number of instances, developers are creating classes and making these available for interacting with their chains. For instance, an example of a custom type could be -
+
+```js
+import { Struct, Text, u32 } from '@polkadot/types';
+
+export class Preferences extends Struct {
+  constructor (value?: ahy) {
+    super({
+      name: Text,
+      id: u32
+    }, value);
+  }
+
+  ...
+}
+```
+
+In the current iteration, the underlying `@polkadot/types` bases structures now require a `Registry` to be passed as the first parameter. This means that the above signature would be adjusted to -
+
+```js
+// the next import is only required for TypeScript
+import { Registry } from '@polkadot/types/types';
+import { Struct, Text, u32 } from '@polkadot/types';
+
+export class Preferences extends Struct {
+  constructor (registry: Registry, value?: ahy) {
+    super(registry, {
+      name: Text,
+      id: u32
+    }, value);
+  }
+
+  ...
+}
+```
+
+Where the type is used or returned from the API, the `Registry` will be automatically passed to class creation.
+
+### createType
+
+Previously, when creating a type instance such as `BlockNumber`, you would do `api.createType('BlockNumber', <initValue>)`, this is unchanged. In the cases where you directly import from `@polkadot/types`, the following pattern is required -
+
+```js
+import { createType } from '@polkadot/types';
+
+...
+const blockNumber = createType(api.registry, 'BlockNumber', 12345);
+```
+
+In some cases, you would want to explicitly pass a `Registry` interface to the API, instead of relying on it explicitly. This is generally applicable in the cases where you want to use the `createType` independently from the API -
+
+```js
+import { ApiPromise } from '@polkadot/api';
+import { TypeRegistry, createType } from '@polkadot/types';
+
+...
+const registry = new TypeRegistry();
+const blockNumber = createType(registry, 'BlockNumber', 12345);
+const api = await ApiPromise.create({ registry });
+```
+
+### Extrinsic metadata
+
+In some applications, the undocumented `findFunction` has been used to determine the Api has the metadata for a specific extrinsic. The has been exposed on top of `GenericCall`, and it typically used in applications such as signers. Along with the compulsory registry, the above functions have been moved to the `Registry` itself, so if you previously had -
+
+```js
+const { meta, method, section } = GenericCall.findFunction(extrinsic.callIndex);
+```
+
+You need to change it to -
+
+```js
+const { meta, method, section } = registry.findMetaCall(extrinsic.callIndex);
+```
 
 ## 0.90.1 (and newer), from 0.81.1 (and older)
 
@@ -29,10 +113,10 @@ To better align with the actual types from the metadata, and avoid (too much) co
 
 The [@polkadot/api](packages/api) has always handled the conversion of types for parameters when making calls or queries. For example, when making a transfer to `BOB` (address), any of the following is valid -
 
-- `api.tx.balances.transfer(BOB, 12345)` - value specied as a number
-- `api.tx.balances.transfer(BOB, '12345')` - value specied as a string
-- `api.tx.balances.transfer(BOB, '0x3039')` - value specied as a hex
-- `api.tx.balances.transfer(BOB, new BN(12345))` - value specied as a [BN](https://github.com/indutny/bn.js/)
+- `api.tx.balances.transfer(BOB, 12345)` - value specified as a number
+- `api.tx.balances.transfer(BOB, '12345')` - value specified as a string
+- `api.tx.balances.transfer(BOB, '0x3039')` - value specified as a hex
+- `api.tx.balances.transfer(BOB, new BN(12345))` - value specified as a [BN](https://github.com/indutny/bn.js/)
 
 Internally the API will take the input and convert the value into a `Balance`, serialize it using the SCALE codec and transfer it to the node. In some cases users would construct the `Balance` type manually, by importing the class and calling `new` on it. This last approach has now been removed, and where classes are still available (limited reach), discouraged.
 
@@ -108,7 +192,7 @@ export class MyStruct extends Struct {
 }
 ```
 
-Internally the [@polkadot/types](packages/types) package now only defines classes where there are specific encoding logic applied. For all other types, the definitions are done via a JSON-like format and then the TypeScript definitions are generated from these. (In a world where nodes inject types and the type definitions are not needed, this functionality will be useful to allow TS devs to auto-generate type definitions based on what the node defines.)
+Internally the [@polkadot/types](packages/types) package now only defines classes where there are specific encoding logic applied. For all other types, the definitions are done via a JSON-like format and then the TypeScript definitions are generated from these. (In a world where nodes inject types and the type definitions are not needed, this functionality will be useful to allow TS developers to auto-generate type definitions based on what the node defines.)
 
 ### Signing transactions (Signer interface)
 

copy-to-apps.sh

@@ -15,7 +15,7 @@ yarn polkadot-dev-build-ts
 copy_folder "api" "api"
 copy_folder "api-contract" "api-contract"
 copy_folder "api-derive" "api-derive"
-copy_folder "api-metadata" "api-metadata"
+copy_folder "metadata" "metadata"
 copy_folder "rpc-core" "rpc-core"
 copy_folder "rpc-provider" "rpc-provider"
 copy_folder "type-jsonrpc" "jsonrpc"

jest.config.js

@@ -6,8 +6,8 @@ module.exports = Object.assign({}, config, {
   moduleNameMapper: {
     '@polkadot/api-contract(.*)$': '<rootDir>/packages/api-contract/src/$1',
     '@polkadot/api-derive(.*)$': '<rootDir>/packages/api-derive/src/$1',
-    '@polkadot/api-metadata(.*)$': '<rootDir>/packages/api-metadata/src/$1',
     '@polkadot/api(.*)$': '<rootDir>/packages/api/src/$1',
+    '@polkadot/metadata(.*)$': '<rootDir>/packages/metadata/src/$1',
     '@polkadot/rpc-(core|provider)(.*)$': '<rootDir>/packages/rpc-$1/src/$2',
     '@polkadot/jsonrpc(.*)$': '<rootDir>/packages/type-jsonrpc/src/$1',
     '@polkadot/types(.*)$': '<rootDir>/packages/types/src/$1'
@@ -16,7 +16,7 @@ module.exports = Object.assign({}, config, {
     '<rootDir>/packages/api/build',
     '<rootDir>/packages/api-derive/build',
     '<rootDir>/packages/api-contract/build',
-    '<rootDir>/packages/api-metadata/build',
+    '<rootDir>/packages/metadata/build',
     '<rootDir>/packages/rpc-core/build',
     '<rootDir>/packages/rpc-provider/build',
     '<rootDir>/packages/type-jsonrpc/build',

lerna.json

@@ -9,5 +9,5 @@
   "packages": [
     "packages/*"
   ],
-  "version": "0.96.1"
+  "version": "0.98.1"
 }

package.json

@@ -1,5 +1,5 @@
 {
-  "version": "0.96.1",
+  "version": "0.98.1",
   "private": true,
   "engines": {
     "yarn": "^1.10.1"
@@ -14,6 +14,7 @@
   "scripts": {
     "build": "yarn build:interfaces && polkadot-dev-build-ts",
     "build:interfaces": "node packages/types/src/scripts/interfacesTsWrapper.js",
+    "circular": "madge --circular --extensions ts ./",
     "docs": "node packages/types/src/scripts/MetadataMdWrapper.js && polkadot-dev-build-docs",
     "chain:info": "node packages/types/src/scripts/extractChainWrapper.js",
     "lint": "eslint --ext .js,.jsx,.ts,.tsx . && tsc --noEmit --pretty",
@@ -28,10 +29,10 @@
     "test:watch": "jest --coverage --watch"
   },
   "devDependencies": {
-    "@babel/core": "^7.7.0",
-    "@babel/register": "^7.7.0",
-    "@babel/runtime": "^7.6.3",
-    "@polkadot/dev": "^0.32.0-beta.13",
-    "@polkadot/ts": "^0.1.84"
+    "@babel/core": "^7.7.4",
+    "@babel/register": "^7.7.4",
+    "@babel/runtime": "^7.7.4",
+    "@polkadot/dev": "^0.32.0-beta.15",
+    "@polkadot/ts": "^0.1.86"
   }
 }

packages/api-contract/README.md

@@ -3,9 +3,12 @@
 Interfaces to allow for the encoding and decoding of Substrate contract ABIs.
 
 ```js
+import {ApiPromise, WsProvider } from '@polkadot/api';
 import { Abi } from '@polkadot/api-contract';
 
-const abi = new Abi(<...JSON ABI...>);
+const wsProvider = new WsProvider(<...Node Url...>);
+const api = await ApiPromise.create({ provider: wsProvider });
+const abi = new Abi(api.registry, <...JSON ABI...>);
 
 api.tx.contracts
   .call(<contract addr>, <value>, <max gas>, abi.messages.<method name>(<...params...>))

packages/api-contract/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@polkadot/api-contract",
-  "version": "0.96.1",
+  "version": "0.98.1",
   "description": "Interfaces for interacting with contracts and contract ABIs",
   "main": "index.js",
   "keywords": [
@@ -26,7 +26,7 @@
   },
   "homepage": "https://github.com/polkadot-js/api/tree/master/packages/api-contract#readme",
   "dependencies": {
-    "@babel/runtime": "^7.7.1",
-    "@polkadot/types": "^0.96.1"
+    "@babel/runtime": "^7.7.4",
+    "@polkadot/types": "^0.98.1"
   }
 }

packages/api-contract/src/Abi.spec.ts

@@ -2,16 +2,19 @@
 // This software may be modified and distributed under the terms
 // of the Apache-2.0 license. See the LICENSE file for details.
 
-import incrementerAbi from '../test/contracts/incrementer.json';
+import { TypeRegistry } from '@polkadot/types';
 
+import incrementerAbi from '../test/contracts/incrementer.json';
 import { Abi } from '.';
 
 describe('Abi', (): void => {
+  const registry = new TypeRegistry();
+
   describe('incrementer', (): void => {
     let abi: Abi;
 
     beforeEach((): void => {
-      abi = new Abi(incrementerAbi);
+      abi = new Abi(registry, incrementerAbi);
     });
 
     it('has the attached methods', (): void => {

packages/api-contract/src/Abi.ts

@@ -2,7 +2,9 @@
 // This software may be modified and distributed under the terms
 // of the Apache-2.0 license. See the LICENSE file for details.
 
+import { Registry } from '@polkadot/types/types';
 import { AbiConstructors, AbiMessages, ContractABI, ContractABIPre, ContractABIFn, InterfaceAbi } from './types';
+
 import { stringCamelCase } from '@polkadot/util';
 
 import ContractRegistry from './ContractRegistry';
@@ -14,8 +16,8 @@ export default class ContractAbi extends ContractRegistry implements InterfaceAb
 
   public readonly messages: AbiMessages;
 
-  constructor (abi: ContractABIPre) {
-    super(abi);
+  constructor (registry: Registry, abi: ContractABIPre) {
+    super(registry, abi);
     [this.abi, this.constructors, this.messages] = this.decodeAbi(abi);
   }
 

packages/api-contract/src/ContractRegistry.ts

@@ -2,28 +2,28 @@
 // This software may be modified and distributed under the terms
 // of the Apache-2.0 license. See the LICENSE file for details.
 
-import { CodecArg, Constructor, TypeDef } from '@polkadot/types/types';
+import { CodecArg, Constructor, Registry, TypeDef } from '@polkadot/types/types';
 import { ContractABIArgBasePre, ContractABIContract, ContractABIContractPre, ContractABIEvent, ContractABIEventPre, ContractABIFn, ContractABIFnArg, ContractABIMessage, ContractABIMessageBase, ContractABIMessagePre, ContractABI, ContractABIMessageCommon, ContractABIPre, ContractABIRange, ContractABIRangePre, ContractABIStorage, ContractABIStorageLayout, ContractABIStorageLayoutPre, ContractABIStoragePre, ContractABIStorageStruct, ContractABIStorageStructPre, ContractABITypePre } from './types';
-import { Compact, u32, createClass, createType, encodeType } from '@polkadot/types';
 
+import { Compact, u32, createClass, createType, encodeType } from '@polkadot/types';
 import { assert, hexToU8a, isNumber, isString, isNull, isObject, isUndefined, stringCamelCase, isHex, hexToNumber } from '@polkadot/util';
 
 import MetaRegistry from './MetaRegistry';
 
 // parse a selector, this can be a number (older) or of [<hex>, <hex>, ...]. However,
 // just catch everything (since this is now non-standard for u32 anyway)
-function parseSelector (fnname: string, input: ContractABIMessageCommon['selector']): u32 {
+function parseSelector (registry: Registry, fnname: string, input: ContractABIMessageCommon['selector']): u32 {
   if (isNumber(input)) {
-    return createType('u32', input);
+    return createType(registry, 'u32', input);
   } else if (isHex(input)) {
-    return createType('u32', hexToU8a(input));
+    return createType(registry, 'u32', hexToU8a(input));
   } else if (typeof input === 'string') {
     try {
       const array = JSON.parse(input);
 
       assert(array.length === 4, `${fnname}: Invalid selector length`);
 
-      return createType('u32', Uint8Array.from(
+      return createType(registry, 'u32', Uint8Array.from(
         // the as number[] is here to pacify TS, it doesn't quite know how to handle the cb
         (array as number[]).map((value: string | number): number =>
           isHex(value)
@@ -39,8 +39,9 @@ function parseSelector (fnname: string, input: ContractABIMessageCommon['selecto
   throw new Error(`${fnname}: Unable to parse selector`);
 }
 
-function createArgClass (args: ContractABIFnArg[], baseDef: Record<string, string>): Constructor {
+function createArgClass (registry: Registry, args: ContractABIFnArg[], baseDef: Record<string, string>): Constructor {
   return createClass(
+    registry,
     JSON.stringify(
       args.reduce((base: Record<string, any>, { name, type }): Record<string, any> => {
         base[name] = type.displayName || encodeType(type);
@@ -87,7 +88,7 @@ export default class ContractRegistry extends MetaRegistry {
       assert(isNumber(name) && isString(this.stringAt(name)), `Expected name for ${fnname}`);
       assert(isNull(returnType) || (isNumber(returnType.ty) && isObject(this.typeDefAt(returnType.ty))), `Expected return_type for ${fnname}`);
 
-      parseSelector(fnname, selector);
+      parseSelector(this.registry, fnname, selector);
       this.validateArgs(fnname, message.args);
     });
   }
@@ -113,12 +114,13 @@ export default class ContractRegistry extends MetaRegistry {
         type
       };
     });
-    const Clazz = createArgClass(args, isUndefined(message.selector) ? {} : { __selector: 'u32' });
-    const baseStruct: { [index: string]: any } = { __selector: isUndefined(message.selector) ? undefined : parseSelector(name, message.selector) };
+    const Clazz = createArgClass(this.registry, args, isUndefined(message.selector) ? {} : { __selector: 'u32' });
+    const baseStruct: { [index: string]: any } = { __selector: isUndefined(message.selector) ? undefined : parseSelector(this.registry, name, message.selector) };
     const encoder = (...params: CodecArg[]): Uint8Array => {
       assert(params.length === args.length, `Expected ${args.length} arguments to contract ${name}, found ${params.length}`);
 
       const u8a = new Clazz(
+        this.registry,
         args.reduce((mapped, { name }, index): Record<string, CodecArg> => {
           mapped[name] = params[index];
 

packages/api-contract/src/MetaRegistry.spec.ts

@@ -2,32 +2,41 @@
 // This software may be modified and distributed under the terms
 // of the Apache-2.0 license. See the LICENSE file for details.
 
-import fs from 'fs';
-import path from 'path';
+import { TypeRegistry } from '@polkadot/types';
 
 import MetaRegistry from './MetaRegistry';
 
 import erc20Abi from '../test/contracts/Erc20.json';
+import erc20Cmp from '../test/compare/erc20.test.json';
 import sharedVecAbi from '../test/contracts/SharedVecV2.json';
+import sharedVecCmp from '../test/compare/shared-vec.test.json';
+import test001Abi from '../test/abi/test001.json';
+import test001Cmp from '../test/compare/test001.cmp.json';
+
+function compare (meta: MetaRegistry, other: any): void {
+  try {
+    expect(meta.typeDefs).toEqual(other);
+  } catch (error) {
+    console.error(JSON.stringify(meta.typeDefs));
+
+    throw error;
+  }
+}
 
 describe('MetaRegistry', (): void => {
+  const registry = new TypeRegistry();
+
   describe('construction', (): void => {
     it('initializes from a contract ABI (ERC20)', (): void => {
-      const metaRegistry = new MetaRegistry(erc20Abi);
-      fs.writeFile(path.join(__dirname, '../test/compare/erc20.test.json'), JSON.stringify(metaRegistry.typeDefs, null, 2), function (err): void {
-        if (err) throw err;
-      });
-
-      expect(true).toBe(true);
+      compare(new MetaRegistry(registry, erc20Abi), erc20Cmp);
     });
 
     it('initializes from a contract ABI (SharedVec)', (): void => {
-      const metaRegistry = new MetaRegistry(sharedVecAbi);
-      fs.writeFile(path.join(__dirname, '../test/compare/shared-vec.test.json'), JSON.stringify(metaRegistry.typeDefs, null, 2), function (err): void {
-        if (err) throw err;
-      });
+      compare(new MetaRegistry(registry, sharedVecAbi), sharedVecCmp);
+    });
 
-      expect(true).toBe(true);
+    it('initializes from a contract ABI (Other, test001)', (): void => {
+      compare(new MetaRegistry(registry, test001Abi), test001Cmp);
     });
   });
 });

packages/api-contract/src/MetaRegistry.ts

@@ -2,7 +2,7 @@
 // This software may be modified and distributed under the terms
 // of the Apache-2.0 license. See the LICENSE file for details.
 
-import { MetaRegistryItem, MetaRegistryJson, MetaTypeDefClikeEnum, MetaType, MetaTypeDefEnum, MetaTypeDefEnumVariant, MetaTypeDefEnumVariantStruct, MetaTypeDefEnumVariantTupleStruct, MetaTypeDefEnumVariantUnit, MetaTypeDefStruct, MetaTypeDefStructField, MetaTypeDefTupleStruct, MetaTypeDefUnion, MetaTypeIdCustom, MetaTypeIdVec, MetaTypeIdVecFixed, MetaTypeInfo, StringIndex, TypeDef, TypeDefInfo, TypeIndex } from '@polkadot/types/types';
+import { MetaRegistryItem, MetaRegistryJson, MetaTypeDefClikeEnum, MetaType, MetaTypeDefEnum, MetaTypeDefEnumVariant, MetaTypeDefEnumVariantStruct, MetaTypeDefEnumVariantTupleStruct, MetaTypeDefEnumVariantUnit, MetaTypeDefStruct, MetaTypeDefStructField, MetaTypeDefTupleStruct, MetaTypeDefUnion, MetaTypeIdCustom, MetaTypeIdVec, MetaTypeIdVecFixed, MetaTypeInfo, Registry, StringIndex, TypeDef, TypeDefInfo, TypeIndex } from '@polkadot/types/types';
 
 import { assert } from '@polkadot/util';
 import { displayType, withTypeString } from '@polkadot/types';
@@ -34,13 +34,16 @@ function detectedType ({ def, id }: MetaType): MetaTypeInfo {
 }
 
 class MetadataRegistryLookup {
+  public readonly registry: Registry;
+
   protected _strings: string[] = [];
 
   protected _types: MetaType[] = [];
 
   public typeDefs: TypeDef[] = [];
 
-  constructor ({ registry: { strings, types } }: MetaRegistryJson) {
+  constructor (registry: Registry, { registry: { strings, types } }: MetaRegistryJson) {
+    this.registry = registry;
     this._strings = strings;
     this._types = types;
   }
@@ -102,8 +105,8 @@ class MetadataRegistryLookup {
 }
 
 export default class MetaRegistry extends MetadataRegistryLookup {
-  constructor (json: MetaRegistryJson) {
-    super(json);
+  constructor (registry: Registry, json: MetaRegistryJson) {
+    super(registry, json);
 
     // Generate TypeDefs for each provided registry type
     this._types.forEach((_, index: number): void => this.setTypeDefAtIndex(index + 1));

packages/api-contract/src/base/Blueprint.ts

@@ -41,7 +41,7 @@ export default class Blueprint<ApiType extends ApiTypes> extends BaseWithTx<ApiT
   constructor (api: ApiObject<ApiType>, abi: ContractABIPre | Abi, decorateMethod: DecorateMethod<ApiType>, codeHash: string | Hash) {
     super(api, abi, decorateMethod);
 
-    this.codeHash = createType('Hash', codeHash);
+    this.codeHash = createType(this.registry, 'Hash', codeHash);
   }
 
   public deployContract (constructorIndex = 0, endowment: number | BN, maxGas: number | BN, ...params: any[]): BlueprintCreate<ApiType> {

packages/api-contract/src/base/Contract.ts

@@ -44,7 +44,7 @@ export default class Contract<ApiType extends ApiTypes> extends BaseWithTxAndRpc
         as === 'rpc'
           ? (account: IKeyringPair | string | AccountId | Address): ContractCallResult<'rpc'> => {
             return this.rpcContractsCall(
-              createType('ContractCallRequest', {
+              createType(this.registry, 'ContractCallRequest', {
                 origin: account,
                 dest: this.address.toString(),
                 value,
@@ -54,7 +54,7 @@ export default class Contract<ApiType extends ApiTypes> extends BaseWithTxAndRpc
             )
               .pipe(
                 map((result: ContractExecResult): ContractCallOutcome =>
-                  this.createOutcome(result, createType('AccountId', account), def, params)
+                  this.createOutcome(result, createType(this.registry, 'AccountId', account), def, params)
                 )
               );
           }
@@ -74,8 +74,8 @@ export default class Contract<ApiType extends ApiTypes> extends BaseWithTxAndRpc
       const { data } = result.asSuccess;
 
       output = message.returnType
-        ? formatData(data, message.returnType)
-        : createType('Data', data);
+        ? formatData(this.registry, data, message.returnType)
+        : createType(this.registry, 'Data', data);
     }
 
     return {
@@ -92,6 +92,6 @@ export default class Contract<ApiType extends ApiTypes> extends BaseWithTxAndRpc
   constructor (api: ApiObject<ApiType>, abi: ContractABIPre | Abi, decorateMethod: DecorateMethod<ApiType>, address: string | AccountId | Address) {
     super(api, abi, decorateMethod);
 
-    this.address = createType('Address', address);
+    this.address = createType(this.registry, 'Address', address);
   }
 }

packages/api-contract/src/base/util.ts

@@ -5,6 +5,7 @@
 import { ApiTypes, DecorateMethod, DecoratedRpc, SubmittableModuleExtrinsics } from '@polkadot/api/types';
 import { ApiObject, ContractABIMessage, ContractABIPre, ContractBase, ContractMessage } from '../types';
 import { RpcInterface } from '@polkadot/rpc-core/jsonrpc.types';
+import { Registry } from '@polkadot/types/types';
 
 import { assert, stringCamelCase } from '@polkadot/util';
 import Abi from '../Abi';
@@ -16,10 +17,13 @@ export abstract class Base<ApiType extends ApiTypes> implements ContractBase<Api
 
   public readonly decorateMethod: DecorateMethod<ApiType>;
 
+  public readonly registry: Registry;
+
   constructor (api: ApiObject<ApiType>, abi: ContractABIPre | Abi, decorateMethod: DecorateMethod<ApiType>) {
     this.abi = abi instanceof Abi
       ? abi
-      : new Abi(abi);
+      : new Abi(api.registry, abi);
+    this.registry = api.registry;
     this.api = api;
     this.decorateMethod = decorateMethod;
   }

packages/api-contract/src/util.ts

@@ -2,20 +2,21 @@
 // This software may be modified and distributed under the terms
 // of the Apache-2.0 license. See the LICENSE file for details.
 
-import { Codec, TypeDefInfo, TypeDef } from '@polkadot/types/types';
+import { Codec, Registry, TypeDefInfo, TypeDef } from '@polkadot/types/types';
 
 import { Data, Option, createClass } from '@polkadot/types';
 import { createTypeUnsafe } from '@polkadot/types/codec';
 
-export function formatData (data: Data, { info, type }: TypeDef): Codec {
+export function formatData (registry: Registry, data: Data, { info, type }: TypeDef): Codec {
   const u8a = data.toU8a();
 
   if (info === TypeDefInfo.Option) {
     return new Option(
-      createClass(type),
-      createTypeUnsafe(type, [u8a], true)
+      registry,
+      createClass(registry, type),
+      createTypeUnsafe(registry, type, [u8a], true)
     );
   }
 
-  return createTypeUnsafe(type, [u8a], true);
+  return createTypeUnsafe(registry, type, [u8a], true);
 }

packages/api-contract/test/abi/test001.json

@@ -0,0 +1,467 @@
+{
+  "registry": {
+    "strings": [
+      "StorageAndEnv",
+      "app",
+      "__ink_private",
+      "__ink_storage",
+      "__storage",
+      "Storage",
+      "value",
+      "Value",
+      "ink_core",
+      "storage",
+      "cell",
+      "SyncCell",
+      "sync_cell",
+      "Key",
+      "key",
+      "name",
+      "vec",
+      "Vec",
+      "elems",
+      "__env",
+      "EnvAccess",
+      "env2",
+      "env_access",
+      "immutable",
+      "TestEnv",
+      "test",
+      "accessor",
+      "DefaultSrmlTypes",
+      "types",
+      "access",
+      "EnvAccessMut",
+      "mutable",
+      "env",
+      "PhantomData",
+      "buffer",
+      "has_interacted",
+      "has_returned_value",
+      "new",
+      "init_value",
+      "bool",
+      "default",
+      "flip",
+      "get"
+    ],
+    "types": [
+      {
+        "id": {
+          "custom.name": 1,
+          "custom.namespace": [
+            2,
+            2,
+            3,
+            4
+          ],
+          "custom.params": []
+        },
+        "def": {
+          "struct.fields": [
+            {
+              "name": 5,
+              "type": 2
+            },
+            {
+              "name": 20,
+              "type": 14
+            }
+          ]
+        }
+      },
+      {
+        "id": {
+          "custom.name": 6,
+          "custom.namespace": [
+            2,
+            2,
+            3,
+            4
+          ],
+          "custom.params": []
+        },
+        "def": {
+          "struct.fields": [
+            {
+              "name": 7,
+              "type": 3
+            },
+            {
+              "name": 16,
+              "type": 9
+            }
+          ]
+        }
+      },
+      {
+        "id": {
+          "custom.name": 8,
+          "custom.namespace": [
+            9,
+            10,
+            7
+          ],
+          "custom.params": [
+            4
+          ]
+        },
+        "def": {
+          "struct.fields": [
+            {
+              "name": 11,
+              "type": 5
+            }
+          ]
+        }
+      },
+      {
+        "id": "bool",
+        "def": "builtin"
+      },
+      {
+        "id": {
+          "custom.name": 12,
+          "custom.namespace": [
+            9,
+            10,
+            11,
+            13
+          ],
+          "custom.params": [
+            4
+          ]
+        },
+        "def": {
+          "struct.fields": [
+            {
+              "name": 11,
+              "type": 6
+            }
+          ]
+        }
+      },
+      {
+        "id": {
+          "custom.name": 14,
+          "custom.namespace": [
+            9,
+            10,
+            15
+          ],
+          "custom.params": []
+        },
+        "def": {
+          "tuple_struct.types": [
+            7
+          ]
+        }
+      },
+      {
+        "id": {
+          "array.len": 32,
+          "array.type": 8
+        },
+        "def": "builtin"
+      },
+      {
+        "id": "u8",
+        "def": "builtin"
+      },
+      {
+        "id": {
+          "custom.name": 8,
+          "custom.namespace": [
+            9,
+            10,
+            7
+          ],
+          "custom.params": [
+            10
+          ]
+        },
+        "def": {
+          "struct.fields": [
+            {
+              "name": 11,
+              "type": 13
+            }
+          ]
+        }
+      },
+      {
+        "id": "str",
+        "def": {
+          "struct.fields": [
+            {
+              "name": 17,
+              "type": 11
+            }
+          ]
+        }
+      },
+      {
+        "id": {
+          "custom.name": 18,
+          "custom.namespace": [],
+          "custom.params": [
+            8
+          ]
+        },
+        "def": {
+          "struct.fields": [
+            {
+              "name": 19,
+              "type": 12
+            }
+          ]
+        }
+      },
+      {
+        "id": {
+          "slice.type": 8
+        },
+        "def": "builtin"
+      },
+      {
+        "id": {
+          "custom.name": 12,
+          "custom.namespace": [
+            9,
+            10,
+            11,
+            13
+          ],
+          "custom.params": [
+            10
+          ]
+        },
+        "def": {
+          "struct.fields": [
+            {
+              "name": 11,
+              "type": 6
+            }
+          ]
+        }
+      },
+      {
+        "id": {
+          "custom.name": 21,
+          "custom.namespace": [
+            9,
+            22,
+            23,
+            24
+          ],
+          "custom.params": [
+            15
+          ]
+        },
+        "def": {
+          "struct.fields": [
+            {
+              "name": 30,
+              "type": 17
+            }
+          ]
+        }
+      },
+      {
+        "id": {
+          "custom.name": 25,
+          "custom.namespace": [
+            9,
+            22,
+            26,
+            27
+          ],
+          "custom.params": [
+            16
+          ]
+        },
+        "def": {
+          "struct.fields": []
+        }
+      },
+      {
+        "id": {
+          "custom.name": 28,
+          "custom.namespace": [
+            9,
+            22,
+            29
+          ],
+          "custom.params": []
+        },
+        "def": {
+          "clike_enum.variants": []
+        }
+      },
+      {
+        "id": {
+          "custom.name": 31,
+          "custom.namespace": [
+            9,
+            22,
+            23,
+            32
+          ],
+          "custom.params": [
+            15
+          ]
+        },
+        "def": {
+          "struct.fields": [
+            {
+              "name": 33,
+              "type": 18
+            },
+            {
+              "name": 35,
+              "type": 11
+            },
+            {
+              "name": 36,
+              "type": 4
+            },
+            {
+              "name": 37,
+              "type": 4
+            }
+          ]
+        }
+      },
+      {
+        "id": {
+          "custom.name": 34,
+          "custom.namespace": [],
+          "custom.params": [
+            15
+          ]
+        },
+        "def": {
+          "tuple_struct.types": []
+        }
+      }
+    ]
+  },
+  "storage": {
+    "struct.type": 1,
+    "struct.fields": [
+      {
+        "name": 10,
+        "layout": {
+          "struct.type": 2,
+          "struct.fields": [
+            {
+              "name": 7,
+              "layout": {
+                "struct.type": 3,
+                "struct.fields": [
+                  {
+                    "name": 11,
+                    "layout": {
+                      "range.offset": "0x0000000000000000000000000000000000000000000000000000000000000000",
+                      "range.len": 1,
+                      "range.elem_type": 4
+                    }
+                  }
+                ]
+              }
+            },
+            {
+              "name": 16,
+              "layout": {
+                "struct.type": 9,
+                "struct.fields": [
+                  {
+                    "name": 11,
+                    "layout": {
+                      "range.offset": "0x0000000000000000000000000000000000000000000000000000000000000001",
+                      "range.len": 1,
+                      "range.elem_type": 10
+                    }
+                  }
+                ]
+              }
+            }
+          ]
+        }
+      }
+    ]
+  },
+  "contract": {
+    "name": 2,
+    "constructors": [
+      {
+        "name": 38,
+        "selector": "[\"0x5E\",\"0xBD\",\"0x88\",\"0xD6\"]",
+        "args": [
+          {
+            "name": 16,
+            "type": {
+              "ty": 11,
+              "display_name": [
+                18
+              ]
+            }
+          },
+          {
+            "name": 39,
+            "type": {
+              "ty": 4,
+              "display_name": [
+                40
+              ]
+            }
+          }
+        ],
+        "docs": [
+          "Constructor that initializes the `bool` value to the given `init_value`."
+        ]
+      },
+      {
+        "name": 41,
+        "selector": "[\"0x02\",\"0x22\",\"0xFF\",\"0x18\"]",
+        "args": [],
+        "docs": [
+          "Constructor that initializes the `bool` value to `false`.",
+          "",
+          "Constructors can delegate to other constructors."
+        ]
+      }
+    ],
+    "messages": [
+      {
+        "name": 42,
+        "selector": "[\"0x8C\",\"0x97\",\"0xDB\",\"0x39\"]",
+        "mutates": true,
+        "args": [],
+        "return_type": null,
+        "docs": [
+          "A message that can be called on instantiated contracts.",
+          "This one flips the value of the stored `bool` from `true`",
+          "to `false` and vice versa."
+        ]
+      },
+      {
+        "name": 43,
+        "selector": "[\"0x25\",\"0x44\",\"0x4A\",\"0xFE\"]",
+        "mutates": false,
+        "args": [],
+        "return_type": {
+          "ty": 4,
+          "display_name": [
+            40
+          ]
+        },
+        "docs": [
+          "Simply returns the current value of our `bool`."
+        ]
+      }
+    ],
+    "events": [],
+    "docs": []
+  }
+}

packages/api-derive/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@polkadot/api-derive",
-  "version": "0.96.1",
+  "version": "0.98.1",
   "description": "Common functions used across Polkadot, derived from RPC calls and storage queries.",
   "main": "index.js",
   "keywords": [
@@ -27,11 +27,11 @@
   },
   "homepage": "https://github.com/polkadot-js/api/tree/master/packages/api-derive#readme",
   "dependencies": {
-    "@babel/runtime": "^7.7.1",
-    "@polkadot/api": "^0.96.1",
-    "@polkadot/types": "^0.96.1"
+    "@babel/runtime": "^7.7.4",
+    "@polkadot/api": "^0.98.1",
+    "@polkadot/types": "^0.98.1"
   },
   "devDependencies": {
-    "@polkadot/keyring": "^1.7.0-beta.5"
+    "@polkadot/keyring": "^1.7.1"
   }
 }

packages/api-derive/src/accounts/idAndIndex.ts

@@ -7,11 +7,11 @@ import { AccountId, AccountIndex, Address } from '@polkadot/types/interfaces';
 import { Observable, of } from 'rxjs';
 import { map } from 'rxjs/operators';
 import { ApiInterfaceRx } from '@polkadot/api/types';
+import { createType } from '@polkadot/types';
 import { isU8a } from '@polkadot/util';
 import { decodeAddress } from '@polkadot/util-crypto';
-import { createType } from '@polkadot/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 export type AccountIdAndIndex = [AccountId?, AccountIndex?];
 
@@ -23,14 +23,14 @@ function retrieve (api: ApiInterfaceRx, address: Address | AccountId | AccountIn
       : decodeAddress((address || '').toString());
 
     if (decoded.length === 32) {
-      const accountId = createType('AccountId', decoded);
+      const accountId = createType(api.registry, 'AccountId', decoded);
 
       return api.derive.accounts.idToIndex(accountId).pipe(
         map((accountIndex): AccountIdAndIndex => [accountId, accountIndex] as AccountIdAndIndex)
       );
     }
 
-    const accountIndex = createType('AccountIndex', decoded);
+    const accountIndex = createType(api.registry, 'AccountIndex', decoded);
 
     return api.derive.accounts.indexToId(accountIndex).pipe(
       map((accountId): AccountIdAndIndex => [accountId, accountIndex] as AccountIdAndIndex)
@@ -54,6 +54,6 @@ function retrieve (api: ApiInterfaceRx, address: Address | AccountId | AccountIn
  * ```
  */
 export function idAndIndex (api: ApiInterfaceRx): (address?: Address | AccountId | AccountIndex | string | null) => Observable<AccountIdAndIndex> {
-  return (address?: Address | AccountId | AccountIndex | string | null): Observable<AccountIdAndIndex> =>
-    retrieve(api, address).pipe(drr());
+  return memo((address?: Address | AccountId | AccountIndex | string | null): Observable<AccountIdAndIndex> =>
+    retrieve(api, address));
 }

packages/api-derive/src/accounts/idToIndex.ts

@@ -9,7 +9,7 @@ import { Observable } from 'rxjs';
 import { map, startWith } from 'rxjs/operators';
 import { ApiInterfaceRx } from '@polkadot/api/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 /**
  * @name idToIndex
@@ -26,12 +26,11 @@ import { drr } from '../util';
  * ```
  */
 export function idToIndex (api: ApiInterfaceRx): (accountId: AccountId | string) => Observable<AccountIndex | undefined> {
-  return (accountId: AccountId | string): Observable<AccountIndex | undefined> =>
+  return memo((accountId: AccountId | string): Observable<AccountIndex | undefined> =>
     api.derive.accounts.indexes().pipe(
       startWith({}),
       map((indexes: AccountIndexes): AccountIndex | undefined =>
         (indexes || {})[accountId.toString()]
-      ),
-      drr()
-    );
+      )
+    ));
 }

packages/api-derive/src/accounts/indexToId.ts

@@ -8,9 +8,9 @@ import { Observable } from 'rxjs';
 import { map, startWith } from 'rxjs/operators';
 import { ApiInterfaceRx } from '@polkadot/api/types';
 import { ENUMSET_SIZE } from '@polkadot/types/primitive/Generic/AccountIndex';
-import { createType, ClassOf, Vec } from '@polkadot/types';
+import { ClassOf, Vec, createType } from '@polkadot/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 /**
  * @name indexToId
@@ -28,17 +28,16 @@ import { drr } from '../util';
 export function indexToId (api: ApiInterfaceRx): (accountIndex: AccountIndex | string) => Observable<AccountId | undefined> {
   const querySection = api.query.indices || api.query.balances;
 
-  return (_accountIndex: AccountIndex | string): Observable<AccountId | undefined> => {
-    const accountIndex = _accountIndex instanceof ClassOf('AccountIndex')
+  return memo((_accountIndex: AccountIndex | string): Observable<AccountId | undefined> => {
+    const accountIndex = _accountIndex instanceof ClassOf(api.registry, 'AccountIndex')
       ? _accountIndex
-      : createType('AccountIndex', _accountIndex);
+      : createType(api.registry, 'AccountIndex', _accountIndex);
 
     return querySection.enumSet<Vec<AccountId>>(accountIndex.div(ENUMSET_SIZE)).pipe(
       startWith([]),
       map((accounts): AccountId | undefined =>
         (accounts || [])[accountIndex.mod(ENUMSET_SIZE).toNumber()]
-      ),
-      drr()
+      )
     );
-  };
+  });
 }

packages/api-derive/src/accounts/indexes.ts

@@ -9,9 +9,9 @@ import { AccountIndexes } from '../types';
 import { Observable } from 'rxjs';
 import { map, switchMap } from 'rxjs/operators';
 import { ENUMSET_SIZE } from '@polkadot/types/primitive/Generic/AccountIndex';
-import { createType } from '@polkadot/types';
+import { Vec, createType } from '@polkadot/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 const enumsetSize = ENUMSET_SIZE.toNumber();
 
@@ -31,27 +31,26 @@ const enumsetSize = ENUMSET_SIZE.toNumber();
  * ```
  */
 export function indexes (api: ApiInterfaceRx): () => Observable<AccountIndexes> {
-  return (): Observable<AccountIndexes> =>
+  return memo((): Observable<AccountIndexes> =>
     api.query.indices.nextEnumSet<AccountIndex>().pipe(
       // use the nextEnumSet (which is a counter of the number of sets) to construct
       // a range of values to query [0, 1, 2, ...]. Retrieve the full enum set for the
       // specific index - each query can return up to ENUMSET_SIZE (64) records, each
       // containing an AccountId
-      switchMap((next: AccountIndex): Observable<any> =>
-        api.query.indices.enumSet.multi([...Array(next.toNumber() + 1).keys()]) as Observable<any>
+      switchMap((next: AccountIndex): Observable<Vec<AccountId>[]> =>
+        api.query.indices.enumSet.multi<Vec<AccountId>>([...Array(next.toNumber() + 1).keys()])
       ),
-      map((all: (AccountId[] | undefined)[]): AccountIndexes =>
-        (all || []).reduce((result, list, outerIndex): AccountIndexes => {
+      map((all: AccountId[][]): AccountIndexes =>
+        all.reduce((result: AccountIndexes, list, outerIndex): AccountIndexes => {
           (list || []).forEach((accountId, innerIndex): void => {
             // re-create the index based on position 0 is [0][0] and likewise
             // 64 (0..63 in first) is [1][0] (the first index value in set 2)
             const index = (outerIndex * enumsetSize) + innerIndex;
 
-            result[accountId.toString()] = createType('AccountIndex', index);
+            result[accountId.toString()] = createType(api.registry, 'AccountIndex', index);
           });
 
           return result;
-        }, {} as AccountIndexes)),
-      drr()
-    );
+        }, {}))
+    ));
 }

packages/api-derive/src/accounts/info.ts

@@ -4,7 +4,7 @@
 
 import { ApiInterfaceRx } from '@polkadot/api/types';
 import { AccountId, AccountIndex, Address, Balance } from '@polkadot/types/interfaces';
-import { Codec } from '@polkadot/types/types';
+import { ITuple } from '@polkadot/types/types';
 import { DeriveAccountInfo } from '../types';
 
 import { Observable, combineLatest, of } from 'rxjs';
@@ -12,20 +12,20 @@ import { map, switchMap } from 'rxjs/operators';
 import { Bytes, Option, u32 } from '@polkadot/types';
 import { u8aToString } from '@polkadot/util';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 function retrieveNick (api: ApiInterfaceRx, accountId?: AccountId): Observable<string | undefined> {
-  return accountId && api.query.nicks
-    ? api.query.nicks
-      .nameOf<Option<[Bytes, Balance] & Codec>>(accountId)
-      .pipe(
-        map((nameOf): string | undefined =>
-          nameOf?.isSome
-            ? u8aToString(nameOf.unwrap()[0]).substr(0, (api.consts.nicks.maxLength as u32).toNumber())
-            : undefined
-        )
-      )
-    : of(undefined);
+  return ((
+    accountId && api.query.nicks
+      ? api.query.nicks.nameOf<Option<ITuple<[Bytes, Balance]>>>(accountId)
+      : of(undefined)
+  ) as Observable<Option<ITuple<[Bytes, Balance]>> | undefined>).pipe(
+    map((nameOf): string | undefined =>
+      nameOf?.isSome
+        ? u8aToString(nameOf.unwrap()[0]).substr(0, (api.consts.nicks.maxLength as u32).toNumber())
+        : undefined
+    )
+  );
 }
 
 /**
@@ -33,7 +33,7 @@ function retrieveNick (api: ApiInterfaceRx, accountId?: AccountId): Observable<s
  * @description Returns aux. info with regards to an account, current that includes the accountId, accountIndex and nickname
  */
 export function info (api: ApiInterfaceRx): (address?: AccountIndex | AccountId | Address | string | null) => Observable<DeriveAccountInfo> {
-  return (address?: AccountIndex | AccountId | Address | string | null): Observable<DeriveAccountInfo> =>
+  return memo((address?: AccountIndex | AccountId | Address | string | null): Observable<DeriveAccountInfo> =>
     api.derive.accounts.idAndIndex(address).pipe(
       switchMap(([accountId, accountIndex]): Observable<[DeriveAccountInfo, string?]> =>
         combineLatest([
@@ -43,7 +43,6 @@ export function info (api: ApiInterfaceRx): (address?: AccountIndex | AccountId
       ),
       map(([{ accountId, accountIndex }, nickname]): DeriveAccountInfo => ({
         accountId, accountIndex, nickname
-      })),
-      drr()
-    );
+      }))
+    ));
 }

packages/api-derive/src/balances/all.ts

@@ -12,13 +12,13 @@ import { ApiInterfaceRx } from '@polkadot/api/types';
 import { Option, Vec, createType } from '@polkadot/types';
 import { bnMax } from '@polkadot/util';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 type ResultBalance = [Balance, Balance, BalanceLock[], Option<VestingSchedule>];
 type Result = [AccountId, BlockNumber, ResultBalance, Index];
 
-function calcBalances ([accountId, bestNumber, [freeBalance, reservedBalance, locks, vesting], accountNonce]: Result): DerivedBalances {
-  let lockedBalance = createType('Balance');
+function calcBalances (api: ApiInterfaceRx, [accountId, bestNumber, [freeBalance, reservedBalance, locks, vesting], accountNonce]: Result): DerivedBalances {
+  let lockedBalance = createType(api.registry, 'Balance');
   let lockedBreakdown: BalanceLock[] = [];
 
   if (Array.isArray(locks)) {
@@ -27,27 +27,40 @@ function calcBalances ([accountId, bestNumber, [freeBalance, reservedBalance, lo
 
     // get the maximum of the locks according to https://github.com/paritytech/substrate/blob/master/srml/balances/src/lib.rs#L699
     if (lockedBreakdown.length) {
-      lockedBalance = createType('Balance', bnMax(...lockedBreakdown.map(({ amount }): Balance => amount)));
+      lockedBalance = createType(api.registry, 'Balance', bnMax(...lockedBreakdown.map(({ amount }): Balance => amount)));
     }
   }
 
-  // offset = balance locked at genesis, perBlock is the unlock amount
-  const { offset: vestingTotal, perBlock } = vesting.unwrapOr(createType('VestingSchedule'));
-  const vestedNow = createType('Balance', perBlock.mul(bestNumber));
-  const vestedBalance = createType('Balance', vestedNow.gt(vestingTotal) ? new BN(0) : bnMax(new BN(0), freeBalance.sub(vestingTotal).add(vestedNow)));
-  const availableBalance = createType('Balance', bnMax(new BN(0), (vestedBalance.gtn(0) ? vestedBalance : freeBalance).sub(lockedBalance)));
+  // Calculate the vesting balances,
+  //  - offset = balance locked at genesis,
+  //  - perBlock is the unlock amount
+  const { offset: vestingTotal, perBlock } = vesting.unwrapOr(createType(api.registry, 'VestingSchedule'));
+  const vestedBalance = createType(api.registry, 'Balance', perBlock.mul(bestNumber));
+  const isVesting = vestedBalance.lt(vestingTotal);
+
+  // The available balance & vested has an interplay here
+  // "
+  // vesting is a guarantee that the account's balance will never go below a certain amount. so it functions in the opposite way, a bit like a lock that is monotonically decreasing rather than a liquid amount that is monotonically increasing.
+  // locks function as the same guarantee - that a balance will not be lower than a particular amount.
+  // because of this you can see that if there is a "vesting lock" that guarantees the balance cannot go below 200, and a "staking lock" that guarantees the balance cannot drop below 300, then we just have two guarantees of which the first is irrelevant.
+  // i.e. (balance >= 200 && balance >= 300) == (balance >= 300)
+  // ""
+  const floating = freeBalance.sub(lockedBalance);
+  const extraReceived = isVesting ? freeBalance.sub(vestingTotal) : new BN(0);
+  const availableBalance = createType(api.registry, 'Balance', bnMax(new BN(0), isVesting && floating.gt(vestedBalance) ? vestedBalance.add(extraReceived) : floating));
 
   return {
     accountId,
     accountNonce,
     availableBalance,
     freeBalance,
+    isVesting,
     lockedBalance,
     lockedBreakdown,
     reservedBalance,
     vestedBalance,
     vestingTotal,
-    votingBalance: createType('Balance', freeBalance.add(reservedBalance))
+    votingBalance: createType(api.registry, 'Balance', freeBalance.add(reservedBalance))
   };
 }
 
@@ -76,7 +89,7 @@ function queryBalances (api: ApiInterfaceRx, accountId: AccountId): Observable<R
  * ```
  */
 export function all (api: ApiInterfaceRx): (address: AccountIndex | AccountId | Address | string) => Observable<DerivedBalances> {
-  return (address: AccountIndex | AccountId | Address | string): Observable<DerivedBalances> =>
+  return memo((address: AccountIndex | AccountId | Address | string): Observable<DerivedBalances> =>
     api.derive.accounts.info(address).pipe(
       switchMap(({ accountId }): Observable<Result> =>
         (accountId
@@ -91,10 +104,9 @@ export function all (api: ApiInterfaceRx): (address: AccountIndex | AccountId |
             //   : api.query.system.accountNonce<Index>(accountId)
             api.query.system.accountNonce<Index>(accountId)
           ])
-          : of([createType('AccountId'), createType('BlockNumber'), [createType('Balance'), createType('Balance'), createType('Vec<BalanceLock>'), createType('Option<VestingSchedule>', null)], createType('Index')])
+          : of([createType(api.registry, 'AccountId'), createType(api.registry, 'BlockNumber'), [createType(api.registry, 'Balance'), createType(api.registry, 'Balance'), createType(api.registry, 'Vec<BalanceLock>'), createType(api.registry, 'Option<VestingSchedule>', null)], createType(api.registry, 'Index')])
         )
       ),
-      map(calcBalances),
-      drr()
-    );
+      map((result): DerivedBalances => calcBalances(api, result))
+    ));
 }

packages/api-derive/src/balances/fees.ts

@@ -9,7 +9,7 @@ import { DerivedFees } from '../types';
 import { Observable, of } from 'rxjs';
 import { map } from 'rxjs/operators';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 type Result = [Balance, Balance, Balance, Balance, Balance];
 
@@ -19,9 +19,9 @@ function queryV2 (api: ApiInterfaceRx): Observable<Result> {
   return of([
     // get values from api.const for substrate versions post spec_version: 101
     // https://github.com/paritytech/substrate/pull/2883/files#diff-5e5e1c3aec9ddfde0a9054d062ab3db9R131
-    api.consts.balances.creationFee as Balance,
-    api.consts.balances.existentialDeposit as Balance,
-    api.consts.balances.transferFee as Balance,
+    api.consts.balances.creationFee,
+    api.consts.balances.existentialDeposit,
+    api.consts.balances.transferFee,
     paymentBase.transactionBaseFee as Balance,
     paymentBase.transactionByteFee as Balance
   ]);
@@ -56,7 +56,7 @@ export function fees (api: ApiInterfaceRx): () => Observable<DerivedFees> {
     ? queryV2
     : queryV1;
 
-  return (): Observable<DerivedFees> =>
+  return memo((): Observable<DerivedFees> =>
     query(api).pipe(
       map(([creationFee, existentialDeposit, transferFee, transactionBaseFee, transactionByteFee]): DerivedFees => ({
         creationFee,
@@ -64,7 +64,6 @@ export function fees (api: ApiInterfaceRx): () => Observable<DerivedFees> {
         transactionBaseFee,
         transactionByteFee,
         transferFee
-      })),
-      drr()
-    );
+      }))
+    ));
 }

packages/api-derive/src/balances/votingBalances.ts

@@ -3,23 +3,20 @@
 // of the Apache-2.0 license. See the LICENSE file for details.
 
 import { AccountId, AccountIndex, Address } from '@polkadot/types/interfaces';
+import { DerivedBalances } from '../types';
 
 import { combineLatest, Observable, of } from 'rxjs';
 import { ApiInterfaceRx } from '@polkadot/api/types';
 
-import { DerivedBalances } from '../types';
-import { drr } from '../util';
+import { memo } from '../util';
 
 export function votingBalances (api: ApiInterfaceRx): (addresses?: (AccountId | AccountIndex | Address | string)[]) => Observable<DerivedBalances[]> {
-  return (addresses?: (AccountId | AccountIndex | Address | string)[]): Observable<DerivedBalances[]> =>
-    (
-      !addresses || !addresses.length
-        ? of([] as DerivedBalances[])
-        : combineLatest(
-          addresses.map((accountId): Observable<DerivedBalances> =>
-            api.derive.balances.all(accountId))
-        )
-    ).pipe(
-      drr()
-    );
+  return memo((addresses?: (AccountId | AccountIndex | Address | string)[]): Observable<DerivedBalances[]> =>
+    !addresses || !addresses.length
+      ? of([] as DerivedBalances[])
+      : combineLatest(
+        addresses.map((accountId): Observable<DerivedBalances> =>
+          api.derive.balances.all(accountId))
+      )
+  );
 }

packages/api-derive/src/balances/votingBalancesNominatorsFor.ts

@@ -10,10 +10,10 @@ import { switchMap } from 'rxjs/operators';
 import { ApiInterfaceRx } from '@polkadot/api/types';
 import { Vec } from '@polkadot/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 export function votingBalancesNominatorsFor (api: ApiInterfaceRx): (address: AccountId | AccountIndex | Address | string) => Observable<DerivedBalances[]> {
-  return (address: AccountId | AccountIndex | Address | string): Observable<DerivedBalances[]> =>
+  return memo((address: AccountId | AccountIndex | Address | string): Observable<DerivedBalances[]> =>
     api.derive.accounts.info(address).pipe(
       switchMap(({ accountId }): Observable<AccountId[]> =>
         accountId
@@ -22,7 +22,6 @@ export function votingBalancesNominatorsFor (api: ApiInterfaceRx): (address: Acc
       ),
       switchMap((accounts): Observable<DerivedBalances[]> =>
         api.derive.balances.votingBalances(accounts)
-      ),
-      drr()
-    );
+      )
+    ));
 }

packages/api-derive/src/chain/bestNumber.ts

@@ -8,7 +8,7 @@ import { Observable } from 'rxjs';
 import { map } from 'rxjs/operators';
 import { ApiInterfaceRx } from '@polkadot/api/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 /**
  * @name bestNumber
@@ -23,9 +23,8 @@ import { drr } from '../util';
  * ```
  */
 export function bestNumber (api: ApiInterfaceRx): () => Observable<BlockNumber> {
-  return (): Observable<BlockNumber> =>
+  return memo((): Observable<BlockNumber> =>
     api.derive.chain.subscribeNewHeads().pipe(
-      map((header: Header): BlockNumber => header.number.unwrap()),
-      drr()
-    );
+      map((header: Header): BlockNumber => header.number.unwrap())
+    ));
 }

packages/api-derive/src/chain/bestNumberFinalized.ts

@@ -8,7 +8,7 @@ import { Observable } from 'rxjs';
 import { map } from 'rxjs/operators';
 import { ApiInterfaceRx } from '@polkadot/api/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 /**
  * @name bestNumberFinalized
@@ -24,9 +24,8 @@ import { drr } from '../util';
  * ```
  */
 export function bestNumberFinalized (api: ApiInterfaceRx): () => Observable<BlockNumber> {
-  return (): Observable<BlockNumber> =>
+  return memo((): Observable<BlockNumber> =>
     api.rpc.chain.subscribeFinalizedHeads().pipe(
-      map((header): BlockNumber => header.number.unwrap()),
-      drr()
-    );
+      map((header): BlockNumber => header.number.unwrap())
+    ));
 }

packages/api-derive/src/chain/bestNumberLag.ts

@@ -9,7 +9,7 @@ import { map } from 'rxjs/operators';
 import { ApiInterfaceRx } from '@polkadot/api/types';
 import { createType } from '@polkadot/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 /**
  * @name bestNumberLag
@@ -25,14 +25,13 @@ import { drr } from '../util';
  * ```
  */
 export function bestNumberLag (api: ApiInterfaceRx): () => Observable<BlockNumber> {
-  return (): Observable<BlockNumber> =>
+  return memo((): Observable<BlockNumber> =>
     combineLatest([
       api.derive.chain.bestNumber(),
       api.derive.chain.bestNumberFinalized()
     ]).pipe(
       map(([bestNumber, bestNumberFinalized]): BlockNumber =>
-        createType('BlockNumber', bestNumber.sub(bestNumberFinalized))
-      ),
-      drr()
-    );
+        createType(api.registry, 'BlockNumber', bestNumber.sub(bestNumberFinalized))
+      )
+    ));
 }

packages/api-derive/src/chain/getHeader.ts

@@ -11,7 +11,7 @@ import { catchError, map } from 'rxjs/operators';
 import { Vec } from '@polkadot/types';
 
 import { HeaderExtended } from '../type';
-import { drr } from '../util';
+import { memo } from '../util';
 
 /**
  * @name bestNumberFinalized
@@ -28,7 +28,7 @@ import { drr } from '../util';
  * ```
  */
 export function getHeader (api: ApiInterfaceRx): (hash: Uint8Array | string) => Observable<HeaderExtended | undefined> {
-  return (hash: Uint8Array | string): Observable<HeaderExtended | undefined> =>
+  return memo((hash: Uint8Array | string): Observable<HeaderExtended | undefined> =>
     combineLatest([
       api.rpc.chain.getHeader(hash),
       api.query.session
@@ -36,14 +36,13 @@ export function getHeader (api: ApiInterfaceRx): (hash: Uint8Array | string) =>
         : of([])
     ]).pipe(
       map(([header, validators]): HeaderExtended =>
-        new HeaderExtended(header, validators)
+        new HeaderExtended(api.registry, header, validators)
       ),
       catchError((): Observable<undefined> =>
         // where rpc.chain.getHeader throws, we will land here - it can happen that
         // we supplied an invalid hash. (Due to defaults, storeage will have an
         // empty value, so only the RPC is affected). So return undefined
         of()
-      ),
-      drr()
-    );
+      )
+    ));
 }

packages/api-derive/src/chain/subscribeNewHeads.ts

@@ -8,7 +8,7 @@ import { Observable, combineLatest } from 'rxjs';
 import { map } from 'rxjs/operators';
 
 import { HeaderExtended } from '../type';
-import { drr } from '../util';
+import { memo } from '../util';
 
 /**
  * @name subscribeNewHeads
@@ -24,14 +24,13 @@ import { drr } from '../util';
  * ```
  */
 export function subscribeNewHeads (api: ApiInterfaceRx): () => Observable<HeaderExtended> {
-  return (): Observable<HeaderExtended> =>
+  return memo((): Observable<HeaderExtended> =>
     combineLatest([
       api.rpc.chain.subscribeNewHeads(),
       api.derive.staking.validators()
     ]).pipe(
       map(([header, { validators }]): HeaderExtended =>
-        new HeaderExtended(header, validators)
-      ),
-      drr()
-    );
+        new HeaderExtended(api.registry, header, validators)
+      )
+    ));
 }

packages/api-derive/src/contracts/fees.ts

@@ -9,7 +9,7 @@ import BN from 'bn.js';
 import { Observable, of } from 'rxjs';
 import { map } from 'rxjs/operators';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 type ResultV2 = [BN, BN, BN, BN, BN, BN, BN, BN, BN, BN];
 
@@ -31,49 +31,6 @@ function parseResult ([callBaseFee, contractFee, createBaseFee, creationFee, ren
   };
 }
 
-// Only for 1.0 support. rentByteFee, rentDepositOffset, tombstoneDeposit are not available in substrate 1.0.
-// TODO remove this once 1.0 support is dropped
-function queryV1 (api: ApiInterfaceRx): Observable<DerivedContractFees> {
-  return (
-    api.queryMulti([
-      api.query.contract.callBaseFee,
-      api.query.contract.contractFee,
-      api.query.contract.createBaseFee,
-      api.query.contract.creationFee,
-      api.query.contract.transactionBaseFee,
-      api.query.contract.transactionByteFee,
-      api.query.contract.transferFee
-    ]) as unknown as Observable<BN[]>
-  ).pipe(
-    map(([callBaseFee, contractFee, createBaseFee, creationFee, transactionBaseFee, transactionByteFee, transferFee]): DerivedContractFees =>
-      // We've done this on purpose, i.e. so we can  just copy the name/order from the parse above and
-      // see gaps, in this case those are filled with `ZERO`
-      parseResult([
-        callBaseFee, contractFee, createBaseFee, creationFee, ZERO, ZERO, ZERO, transactionBaseFee, transactionByteFee, transferFee
-      ])
-    ),
-    drr()
-  );
-}
-
-// query via query (early v2, non-current, support to be dropped])
-function queryQuery (api: ApiInterfaceRx): Observable<ResultV2> {
-  const queryBase = api.query.contracts || api.query.contract;
-
-  return api.queryMulti([
-    queryBase.callBaseFee,
-    queryBase.contractFee,
-    queryBase.createBaseFee,
-    queryBase.creationFee,
-    queryBase.rentByteFee,
-    queryBase.rentDepositOffset,
-    queryBase.tombstoneDeposit,
-    queryBase.transactionBaseFee,
-    queryBase.transactionByteFee,
-    queryBase.transferFee
-  ]) as unknown as Observable<ResultV2>;
-}
-
 // query via constants (current applicable path)
 function queryConstants (api: ApiInterfaceRx): Observable<ResultV2> {
   return of([
@@ -104,23 +61,18 @@ function queryConstants (api: ApiInterfaceRx): Observable<ResultV2> {
  * ```
  */
 export function fees (api: ApiInterfaceRx): () => Observable<DerivedContractFees> {
-  return (): Observable<DerivedContractFees> => {
-    if (api.query.contract && !api.query.contract.rentByteFee) {
-      return queryV1(api);
-    }
-
+  return memo((): Observable<DerivedContractFees> => {
     return (
       api.consts.contracts
         ? queryConstants(api)
-        : queryQuery(api)
+        : of([ZERO, ZERO, ZERO, ZERO, ZERO, ZERO, ZERO, ZERO, ZERO, ZERO])
     ).pipe(
       map(([callBaseFee, contractFee, createBaseFee, creationFee, rentByteFee, rentDepositOffset, tombstoneDeposit, transactionBaseFee, transactionByteFee, transferFee]): DerivedContractFees =>
         // We've done this on purpose, i.e. so we can  just copy the name/order from the parse above and see gaps
         parseResult([
           callBaseFee, contractFee, createBaseFee, creationFee, rentByteFee, rentDepositOffset, tombstoneDeposit, transactionBaseFee, transactionByteFee, transferFee
         ])
-      ),
-      drr()
+      )
     );
-  };
+  });
 }

packages/api-derive/src/democracy/index.ts

@@ -2,6 +2,7 @@
 // This software may be modified and distributed under the terms
 // of the Apache-2.0 license. See the LICENSE file for details.
 
+export * from './proposals';
 export * from './referendumInfos';
 export * from './referendums';
 export * from './referendumVotesFor';

packages/api-derive/src/democracy/proposals.ts

@@ -0,0 +1,75 @@
+// Copyright 2017-2019 @polkadot/api-derive authors & contributors
+// This software may be modified and distributed under the terms
+// of the Apache-2.0 license. See the LICENSE file for details.
+
+import { AccountId, Balance, BlockNumber, Hash, PropIndex } from '@polkadot/types/interfaces';
+import { ITuple } from '@polkadot/types/types';
+import { DeriveProposal } from '../types';
+
+import { Observable, combineLatest, of } from 'rxjs';
+import { map, switchMap } from 'rxjs/operators';
+import { ApiInterfaceRx } from '@polkadot/api/types';
+import { Bytes, Option, Vec, createType } from '@polkadot/types';
+
+import { memo } from '../util';
+
+export type PreImage = Option<ITuple<[Bytes, AccountId, Balance, BlockNumber]>>;
+type Depositors = Option<ITuple<[Balance, Vec<AccountId>]>>;
+
+export function proposals (api: ApiInterfaceRx): () => Observable<DeriveProposal[]> {
+  return memo((): Observable<DeriveProposal[]> =>
+    api.query.democracy
+      .publicProps<Vec<ITuple<[PropIndex, Hash, AccountId]>>>()
+      .pipe(
+        switchMap((proposals) =>
+          combineLatest([
+            of(proposals),
+            combineLatest(
+              ...proposals.map(([, hash]) =>
+                api.query.democracy.preimages<PreImage>(hash)
+              )
+            )
+          ])
+        ),
+        switchMap(([proposals, preimages]) =>
+          combineLatest([
+            of(proposals),
+            of(preimages),
+            combineLatest(
+              ...proposals.map(([index]) =>
+                api.query.democracy.depositOf<Depositors>(index)
+              )
+            )
+          ])
+        ),
+        map(([proposals, _preImages, _depositors]): DeriveProposal[] =>
+          proposals
+            .filter(([, , proposer], index): boolean =>
+              !!(_preImages[index]?.isSome) && !!(_depositors[index]?.isSome) && !proposer.isEmpty
+            )
+            .map(([propIndex, hash, proposer], index): DeriveProposal => {
+              const preImage = _preImages[index].unwrapOr(null);
+              const depositors = _depositors[index].unwrap();
+
+              return {
+                balance: depositors[0],
+                hash,
+                index: propIndex,
+                preimage: preImage
+                  ? {
+                    at: preImage[3],
+                    balance: preImage[2],
+                    proposer: preImage[1]
+                  }
+                  : undefined,
+                proposal: preImage
+                  ? createType(api.registry, 'Proposal', preImage[0].toU8a(true))
+                  : undefined,
+                proposer,
+                seconds: depositors[1]
+              };
+            })
+        )
+      )
+  );
+}

packages/api-derive/src/democracy/referendumInfo.ts

@@ -3,36 +3,66 @@
 // of the Apache-2.0 license. See the LICENSE file for details.
 
 import { ReferendumInfo } from '@polkadot/types/interfaces/democracy';
+import { DerivedReferendum } from '../types';
+import { PreImage } from './proposals';
 
 import BN from 'bn.js';
-import { Observable } from 'rxjs';
-import { map } from 'rxjs/operators';
+import { Observable, of } from 'rxjs';
+import { map, switchMap } from 'rxjs/operators';
 import { ApiInterfaceRx } from '@polkadot/api/types';
-import { Option } from '@polkadot/types';
-import { isNull } from '@polkadot/util';
+import { Option, createType } from '@polkadot/types';
 
-import { ReferendumInfoExtended } from '../type';
-import { drr } from '../util';
+import { memo } from '../util';
 
-export function constructInfo (index: BN | number, optionInfo?: Option<ReferendumInfo>): Option<ReferendumInfoExtended> {
-  const info = optionInfo
-    ? optionInfo.unwrapOr(null)
+function constructInfo (api: ApiInterfaceRx, index: BN | number, _info: Option<ReferendumInfo>, _preimage?: PreImage): DerivedReferendum | null {
+  console.log(JSON.stringify(_info), JSON.stringify(_preimage));
+
+  const preImage = _preimage?.isSome
+    ? _preimage.unwrap()
     : null;
+  const info = _info.unwrapOr(null);
 
-  return new Option<ReferendumInfoExtended>(
-    ReferendumInfoExtended,
-    isNull(info)
-      ? null
-      : new ReferendumInfoExtended(info, index)
+  if (!info) {
+    return null;
+  }
+
+  return {
+    index: createType(api.registry, 'PropIndex', index),
+    info,
+    hash: info.proposalHash,
+    proposal: preImage
+      ? createType(api.registry, 'Proposal', preImage[0].toU8a(true))
+      : undefined,
+    preimage: preImage
+      ? {
+        at: preImage[3],
+        balance: preImage[2],
+        proposer: preImage[1]
+      }
+      : undefined
+  };
+}
+
+export function retrieveInfo (api: ApiInterfaceRx, index: BN | number, info: Option<ReferendumInfo>): Observable<DerivedReferendum | null> {
+  console.log(JSON.stringify(info));
+
+  return ((
+    info?.isSome
+      ? api.query.democracy.preimages<PreImage>(info.unwrap().proposalHash)
+      : of(undefined)
+  ) as Observable<PreImage | undefined>).pipe(
+    map((preimage?: PreImage): DerivedReferendum | null =>
+      constructInfo(api, index, info, preimage)
+    )
   );
 }
 
-export function referendumInfo (api: ApiInterfaceRx): (index: BN | number) => Observable<Option<ReferendumInfoExtended>> {
-  return (index: BN | number): Observable<Option<ReferendumInfoExtended>> =>
+export function referendumInfo (api: ApiInterfaceRx): (index: BN | number) => Observable<DerivedReferendum | null> {
+  return memo((index: BN | number): Observable<DerivedReferendum | null> =>
     api.query.democracy.referendumInfoOf<Option<ReferendumInfo>>(index).pipe(
-      map((optionInfo): Option<ReferendumInfoExtended> =>
-        constructInfo(index, optionInfo)
-      ),
-      drr()
-    );
+      switchMap((info): Observable<DerivedReferendum | null> =>
+        retrieveInfo(api, index, info)
+      )
+    )
+  );
 }

packages/api-derive/src/democracy/referendumInfos.ts

@@ -3,29 +3,33 @@
 // of the Apache-2.0 license. See the LICENSE file for details.
 
 import { ReferendumInfo } from '@polkadot/types/interfaces/democracy';
+import { DerivedReferendum } from '../types';
 
 import BN from 'bn.js';
-import { Observable, of } from 'rxjs';
-import { map } from 'rxjs/operators';
+import { Observable, combineLatest, of } from 'rxjs';
+import { map, switchMap } from 'rxjs/operators';
 import { ApiInterfaceRx } from '@polkadot/api/types';
 import { Option, Vec } from '@polkadot/types';
 
-import { ReferendumInfoExtended } from '../type';
-import { drr } from '../util';
-import { constructInfo } from './referendumInfo';
+import { memo } from '../util';
+import { retrieveInfo } from './referendumInfo';
 
-export function referendumInfos (api: ApiInterfaceRx): (ids?: (BN | number)[]) => Observable<Option<ReferendumInfoExtended>[]> {
-  return (ids: (BN | number)[] = []): Observable<Option<ReferendumInfoExtended>[]> =>
+export function referendumInfos (api: ApiInterfaceRx): (ids?: (BN | number)[]) => Observable<DerivedReferendum[]> {
+  return memo((ids: (BN | number)[] = []): Observable<DerivedReferendum[]> =>
     (
       !ids || !ids.length
         ? of([] as Option<ReferendumInfo>[])
         : api.query.democracy.referendumInfoOf.multi(ids) as Observable<Vec<Option<ReferendumInfo>>>
     ).pipe(
-      map((infos): Option<ReferendumInfoExtended>[] =>
-        ids.map((id, index): Option<ReferendumInfoExtended> =>
-          constructInfo(id, infos[index])
+      switchMap((infos): Observable<(DerivedReferendum | null)[]> =>
+        combineLatest(
+          ...ids.map((id, index): Observable<DerivedReferendum | null> =>
+            retrieveInfo(api, id, infos[index])
+          )
         )
       ),
-      drr()
-    );
+      map((infos): DerivedReferendum[] =>
+        infos.filter((referendum): boolean => !!referendum) as DerivedReferendum[]
+      )
+    ));
 }

packages/api-derive/src/democracy/referendumVotesFor.ts

@@ -11,10 +11,10 @@ import { ApiInterfaceRx } from '@polkadot/api/types';
 import { Vec, createType } from '@polkadot/types';
 
 import { DerivedBalances, DerivedReferendumVote } from '../types';
-import { drr } from '../util';
+import { memo } from '../util';
 
 export function referendumVotesFor (api: ApiInterfaceRx): (referendumId: BN | number) => Observable<DerivedReferendumVote[]> {
-  return (referendumId: BN | number): Observable<DerivedReferendumVote[]> =>
+  return memo((referendumId: BN | number): Observable<DerivedReferendumVote[]> =>
     api.query.democracy.votersFor<Vec<AccountId>>(referendumId).pipe(
       switchMap((votersFor): Observable<[Vec<AccountId>, Vote[], DerivedBalances[]]> =>
         combineLatest([
@@ -26,10 +26,9 @@ export function referendumVotesFor (api: ApiInterfaceRx): (referendumId: BN | nu
       map(([votersFor, votes, balances]): DerivedReferendumVote[] =>
         votersFor.map((accountId, index): DerivedReferendumVote => ({
           accountId,
-          balance: balances[index].votingBalance || createType('Balance'),
-          vote: votes[index] || createType('Vote')
-        } as unknown as DerivedReferendumVote))
-      ),
-      drr()
-    );
+          balance: balances[index].votingBalance || createType(api.registry, 'Balance'),
+          vote: votes[index] || createType(api.registry, 'Vote')
+        } as DerivedReferendumVote))
+      )
+    ));
 }

packages/api-derive/src/democracy/referendums.ts

@@ -3,23 +3,22 @@
 // of the Apache-2.0 license. See the LICENSE file for details.
 
 import { ReferendumIndex } from '@polkadot/types/interfaces/democracy';
+import { DerivedReferendum } from '../types';
 
 import BN from 'bn.js';
 import { Observable, of } from 'rxjs';
 import { switchMap } from 'rxjs/operators';
 import { ApiInterfaceRx } from '@polkadot/api/types';
-import { Option } from '@polkadot/types';
 
-import { ReferendumInfoExtended } from '../type';
-import { drr } from '../util';
+import { memo } from '../util';
 
-export function referendums (api: ApiInterfaceRx): () => Observable<Option<ReferendumInfoExtended>[]> {
-  return (): Observable<Option<ReferendumInfoExtended>[]> =>
+export function referendums (api: ApiInterfaceRx): () => Observable<DerivedReferendum[]> {
+  return memo((): Observable<DerivedReferendum[]> =>
     api.queryMulti<[ReferendumIndex, ReferendumIndex]>([
       api.query.democracy.nextTally,
       api.query.democracy.referendumCount
     ]).pipe(
-      switchMap(([nextTally, referendumCount]): Observable<Option<ReferendumInfoExtended>[]> =>
+      switchMap(([nextTally, referendumCount]): Observable<DerivedReferendum[]> =>
         referendumCount && nextTally && referendumCount.gt(nextTally) && referendumCount.gtn(0)
           ? api.derive.democracy.referendumInfos(
             [...Array(referendumCount.sub(nextTally).toNumber())].map((_, i): BN =>
@@ -27,7 +26,6 @@ export function referendums (api: ApiInterfaceRx): () => Observable<Option<Refer
             )
           )
           : of([])
-      ),
-      drr()
-    );
+      )
+    ));
 }

packages/api-derive/src/democracy/votes.ts

@@ -8,17 +8,16 @@ import BN from 'bn.js';
 import { Observable, of } from 'rxjs';
 import { ApiInterfaceRx } from '@polkadot/api/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 export function votes (api: ApiInterfaceRx): (referendumId: BN, accountIds?: AccountId[]) => Observable<Vote[]> {
-  return (referendumId: BN, accountIds: AccountId[] = []): Observable<Vote[]> =>
-    (
-      !accountIds || !accountIds.length
-        ? of([] as Vote[])
-        : api.query.democracy.voteOf.multi<Vote>(
-          accountIds.map((accountId): [BN, AccountId] =>
-            [referendumId, accountId]
-          )
+  return memo((referendumId: BN, accountIds: AccountId[] = []): Observable<Vote[]> =>
+    !accountIds || !accountIds.length
+      ? of([] as Vote[])
+      : api.query.democracy.voteOf.multi<Vote>(
+        accountIds.map((accountId): [BN, AccountId] =>
+          [referendumId, accountId]
         )
-    ).pipe(drr());
+      )
+  );
 }

packages/api-derive/src/elections/approvalsOf.ts

@@ -5,11 +5,28 @@
 import { AccountId, ApprovalFlag, SetIndex } from '@polkadot/types/interfaces';
 
 import { ApiInterfaceRx } from '@polkadot/api/types';
-import { Observable } from 'rxjs';
+import { Observable, of } from 'rxjs';
 import { Vec } from '@polkadot/types';
 import { switchMap, map } from 'rxjs/operators';
 import { approvalFlagsToBools } from '../util/approvalFlagsToBools';
-import { drr } from '../util';
+import { memo } from '../util';
+
+function queryElections (api: ApiInterfaceRx, who: AccountId | string): Observable<boolean[][]> {
+  return api.query.elections.nextVoterSet<SetIndex>().pipe(
+    switchMap((nextVoterSet: SetIndex): Observable<Vec<ApprovalFlag>[]> =>
+      api.query.elections.approvalsOf.multi(
+        [...Array(nextVoterSet.toNumber() + 1).keys()].map((i): [string, number] =>
+          [who.toString(), i]
+        )
+      ) as any as Observable<Vec<ApprovalFlag>[]>
+    ),
+    map((votes: Vec<ApprovalFlag>[]): boolean[][] =>
+      votes.map((flags: Vec<ApprovalFlag>): boolean[] =>
+        approvalFlagsToBools(flags)
+      )
+    )
+  );
+}
 
 /**
  * @name approvalsOf
@@ -23,21 +40,10 @@ import { drr } from '../util';
  * });
  * ```
  */
-export function approvalsOf (api: ApiInterfaceRx): (who: AccountId) => Observable<boolean[][]> {
-  return (who: AccountId | string): Observable<boolean[][]> =>
-    api.query.elections.nextVoterSet<SetIndex>().pipe(
-      switchMap((nextVoterSet: SetIndex): Observable<Vec<ApprovalFlag>[]> =>
-        api.query.elections.approvalsOf.multi(
-          [...Array(nextVoterSet.toNumber() + 1).keys()].map((i): [string, number] => [
-            who.toString(), i]
-          )
-        ) as any as Observable<Vec<ApprovalFlag>[]>
-      ),
-      map((votes: Vec<ApprovalFlag>[]): boolean[][] =>
-        votes.map((flags: Vec<ApprovalFlag>): boolean[] =>
-          approvalFlagsToBools(flags)
-        )
-      ),
-      drr()
-    );
+export function approvalsOf (api: ApiInterfaceRx): (who: AccountId | string) => Observable<boolean[][]> {
+  return memo((who: AccountId | string): Observable<boolean[][]> =>
+    api.query.elections
+      ? queryElections(api, who)
+      : of([])
+  );
 }

packages/api-derive/src/elections/approvalsOfAt.ts

@@ -5,13 +5,19 @@
 import { AccountId, ApprovalFlag, SetIndex } from '@polkadot/types/interfaces';
 
 import BN from 'bn.js';
-import { Observable } from 'rxjs';
+import { Observable, of } from 'rxjs';
 import { map } from 'rxjs/operators';
 import { ApiInterfaceRx } from '@polkadot/api/types';
 import { Vec } from '@polkadot/types';
 
 import { approvalFlagsToBools } from '../util/approvalFlagsToBools';
-import { drr } from '../util';
+import { memo } from '../util';
+
+function queryElections (api: ApiInterfaceRx, who: AccountId | string, at: SetIndex | BN | number): Observable<boolean[]> {
+  return api.query.elections.approvalsOf<Vec<ApprovalFlag>>([who.toString(), at]).pipe(
+    map((flags: Vec<ApprovalFlag>): boolean[] => approvalFlagsToBools(flags))
+  );
+}
 
 /**
  * @name approvalsOfAt
@@ -26,9 +32,9 @@ import { drr } from '../util';
  * ```
  */
 export function approvalsOfAt (api: ApiInterfaceRx): (who: AccountId, at: SetIndex) => Observable<boolean[]> {
-  return (who: AccountId | string, at: SetIndex | BN | number): Observable<boolean[]> =>
-    api.query.elections.approvalsOf<Vec<ApprovalFlag>>([who.toString(), at]).pipe(
-      map((flags: Vec<ApprovalFlag>): boolean[] => approvalFlagsToBools(flags)),
-      drr()
-    );
+  return memo((who: AccountId | string, at: SetIndex | BN | number): Observable<boolean[]> =>
+    api.query.elections
+      ? queryElections(api, who, at)
+      : of([])
+  );
 }

packages/api-derive/src/elections/info.ts

@@ -3,7 +3,7 @@
 // of the Apache-2.0 license. See the LICENSE file for details.
 
 import { AccountId, Balance, BlockNumber, SetIndex, VoteIndex } from '@polkadot/types/interfaces';
-import { Codec } from '@polkadot/types/types';
+import { ITuple } from '@polkadot/types/types';
 
 import { Observable, combineLatest } from 'rxjs';
 import { map } from 'rxjs/operators';
@@ -11,18 +11,18 @@ import { ApiInterfaceRx } from '@polkadot/api/types';
 import { createType, Vec, u32 } from '@polkadot/types';
 
 import { DerivedElectionsInfo } from '../types';
-import { drr } from '../util';
+import { memo } from '../util';
 
-type ResultElectionsInner = [u32, u32, Vec<[AccountId, BlockNumber] & Codec>, SetIndex, BlockNumber, VoteIndex, SetIndex];
+type ResultElectionsInner = [u32, u32, Vec<ITuple<[AccountId, BlockNumber]>>, SetIndex, BlockNumber, VoteIndex, SetIndex];
 type ResultElections = [Vec<AccountId>, ResultElectionsInner];
 
-function deriveElections ([candidates, [candidateCount, desiredSeats, members, nextVoterSet, termDuration, voteCount, voterCount]]: ResultElections): DerivedElectionsInfo {
+function deriveElections (api: ApiInterfaceRx, [candidates, [candidateCount, desiredSeats, members, nextVoterSet, termDuration, voteCount, voterCount]]: ResultElections): DerivedElectionsInfo {
   return {
     candidates,
     candidateCount,
     desiredSeats,
     nextVoterSet,
-    members: members.map(([accountId]): [AccountId, Balance] => [accountId, createType('Balance')]),
+    members: members.map(([accountId]): [AccountId, Balance] => [accountId, createType(api.registry, 'Balance')]),
     runnersUp: [],
     termDuration,
     voteCount,
@@ -43,17 +43,17 @@ function queryElections (api: ApiInterfaceRx): Observable<DerivedElectionsInfo>
       api.query.elections.voteCount,
       api.query.elections.voterCount
     ])
-  ]).pipe(map(deriveElections));
+  ]).pipe(map((result): DerivedElectionsInfo => deriveElections(api, result)));
 }
 
-function derivePhragmen (candidates: AccountId[], members: [AccountId, Balance][], runnersUp: [AccountId, Balance][], candidacyBond: Balance, desiredSeats: u32, termDuration: BlockNumber, votingBond: Balance): DerivedElectionsInfo {
+function derivePhragmen (api: ApiInterfaceRx, candidates: AccountId[], members: [AccountId, Balance][], runnersUp: [AccountId, Balance][], candidacyBond: Balance, desiredSeats: u32, termDuration: BlockNumber, votingBond: Balance): DerivedElectionsInfo {
   return {
     candidates,
-    candidateCount: createType('u32', candidates.length),
+    candidateCount: createType(api.registry, 'u32', candidates.length),
     candidacyBond,
     desiredSeats,
-    members,
-    runnersUp,
+    members: members.sort((a, b): number => b[1].cmp(a[1])),
+    runnersUp: runnersUp.sort((a, b): number => b[1].cmp(a[1])),
     termDuration,
     votingBond
   };
@@ -64,10 +64,11 @@ function queryPhragmen (api: ApiInterfaceRx): Observable<DerivedElectionsInfo> {
   // we are not using multi queries here, so empty array is empty (instead of space-filled)
   return combineLatest([
     api.query.electionsPhragmen.candidates<Vec<AccountId>>(),
-    api.query.electionsPhragmen.members<Vec<[AccountId, Balance] & Codec>>(),
-    api.query.electionsPhragmen.runnersUp<Vec<[AccountId, Balance] & Codec>>()
+    api.query.electionsPhragmen.members<Vec<ITuple<[AccountId, Balance]>>>(),
+    api.query.electionsPhragmen.runnersUp<Vec<ITuple<[AccountId, Balance]>>>()
   ]).pipe(
     map(([candidates, members, runnersUp]): DerivedElectionsInfo => derivePhragmen(
+      api,
       candidates,
       members,
       runnersUp,
@@ -93,10 +94,9 @@ function queryPhragmen (api: ApiInterfaceRx): Observable<DerivedElectionsInfo> {
  * ```
  */
 export function info (api: ApiInterfaceRx): () => Observable<DerivedElectionsInfo> {
-  const query = api.query.electionsPhragmen
-    ? queryPhragmen
-    : queryElections;
-
-  return (): Observable<DerivedElectionsInfo> =>
-    query(api).pipe(drr());
+  return memo((): Observable<DerivedElectionsInfo> =>
+    api.query.electionsPhragmen
+      ? queryPhragmen(api)
+      : queryElections(api)
+  );
 }

packages/api-derive/src/elections/voterPositions.ts

@@ -4,14 +4,41 @@
 
 import { AccountId, SetIndex } from '@polkadot/types/interfaces';
 import { ApiInterfaceRx } from '@polkadot/api/types';
-import { Vec, Option, createType } from '@polkadot/types';
+import { Vec, Option, createType, u32 } from '@polkadot/types';
 import { DerivedVoterPositions } from '../types';
 
 import BN from 'bn.js';
-import { of, combineLatest, Observable } from 'rxjs';
+import { of, Observable } from 'rxjs';
 import { switchMap, map } from 'rxjs/operators';
 
-import { drr } from '../util';
+import { memo } from '../util';
+
+function queryElections (api: ApiInterfaceRx): Observable<DerivedVoterPositions> {
+  return api.query.elections.nextVoterSet<SetIndex>().pipe(
+    switchMap((nextVoterSet: SetIndex): Observable<Vec<Option<AccountId>>[]> =>
+      api.query.elections.voters.multi<Vec<Option<AccountId>>>([...Array(+nextVoterSet + 1).keys()].map((_, i): [number] => [i]))
+    ),
+    map((voters: Vec<Option<AccountId>>[]): DerivedVoterPositions => {
+      return voters.reduce((result: DerivedVoterPositions, vec, setIndex): DerivedVoterPositions => {
+        vec.forEach((e, index): void => {
+          // re-create the index based on position 0 is [0][0] and likewise
+          // 64 (0..63 in first) is [1][0] (the first index value in set 2)
+          const accountId: AccountId | null = e.unwrapOr(null);
+
+          if (accountId) {
+            result[accountId.toString()] = {
+              globalIndex: (api.consts.elections.voterSetSize as u32).muln(setIndex).addn(index),
+              index: new BN(index),
+              setIndex: createType(api.registry, 'SetIndex', setIndex)
+            };
+          }
+        });
+
+        return result;
+      }, {});
+    })
+  );
+}
 
 /**
  * @name voterPositions
@@ -27,33 +54,9 @@ import { drr } from '../util';
  * ```
  */
 export function voterPositions (api: ApiInterfaceRx): () => Observable<DerivedVoterPositions> {
-  return (): Observable<DerivedVoterPositions> =>
-    api.query.elections.nextVoterSet<SetIndex>().pipe(
-      switchMap((nextVoterSet: SetIndex): Observable<[BN, Vec<Option<AccountId>>[]]> => combineLatest(
-        of(api.consts.elections.voterSetSize) as any as Observable<BN>,
-        api.query.elections.voters.multi([...Array(+nextVoterSet + 1).keys()].map((_, i): [number] => [i])) as any as Observable<Vec<Option<AccountId>>[]>
-      )),
-      map((result: [BN, Vec<Option<AccountId>>[]]): DerivedVoterPositions => {
-        const [setSize, voters] = result;
-
-        return voters.reduce((result: DerivedVoterPositions, vec, setIndex): DerivedVoterPositions => {
-          vec.forEach((e, index): void => {
-            // re-create the index based on position 0 is [0][0] and likewise
-            // 64 (0..63 in first) is [1][0] (the first index value in set 2)
-            const accountId: AccountId | null = e.unwrapOr(null);
-
-            if (accountId) {
-              result[accountId.toString()] = {
-                globalIndex: setSize.muln(setIndex).addn(index),
-                index: new BN(index),
-                setIndex: createType('SetIndex', setIndex)
-              };
-            }
-          });
-
-          return result;
-        }, {});
-      }),
-      drr()
-    );
+  return memo((): Observable<DerivedVoterPositions> =>
+    api.query.elections
+      ? queryElections(api)
+      : of({})
+  );
 }

packages/api-derive/src/elections/voters.ts

@@ -6,12 +6,25 @@ import { ApiInterfaceRx } from '@polkadot/api/types';
 import { AccountId } from '@polkadot/types/interfaces';
 import { DerivedVoterPositions } from '../types';
 
-import { Observable } from 'rxjs';
+import { Observable, of } from 'rxjs';
 import { map } from 'rxjs/operators';
 import { createType, Vec } from '@polkadot/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
+function queryElections (api: ApiInterfaceRx): Observable<AccountId[]> {
+  return api.derive.elections.voterPositions().pipe(
+    map((voterPositions: DerivedVoterPositions): Vec<AccountId> =>
+      createType(
+        api.registry,
+        'Vec<AccountId>',
+        Object.entries(voterPositions)
+          .sort((a, b): number => a[1].globalIndex.cmp(b[1].globalIndex))
+          .map(([accountId]): AccountId => createType(api.registry, 'AccountId', accountId))
+      )
+    )
+  );
+}
 /**
  * @name voters
  * @returns An array of all current voters from all sets.
@@ -24,17 +37,10 @@ import { drr } from '../util';
  * });
  * ```
  */
-export function voters (api: ApiInterfaceRx): () => Observable<Vec<AccountId>> {
-  return (): Observable<Vec<AccountId>> =>
-    api.derive.elections.voterPositions().pipe(
-      map((voterPositions: DerivedVoterPositions): Vec<AccountId> =>
-        createType(
-          'Vec<AccountId>',
-          Object.entries(voterPositions)
-            .sort((a, b): number => a[1].globalIndex.cmp(b[1].globalIndex))
-            .map(([accountId]): AccountId => createType('AccountId', accountId))
-        )
-      ),
-      drr()
-    );
+export function voters (api: ApiInterfaceRx): () => Observable<AccountId[]> {
+  return memo((): Observable<AccountId[]> =>
+    api.query.elections
+      ? queryElections(api)
+      : of([])
+  );
 }

packages/api-derive/src/imOnline/receivedHeartbeats.ts

@@ -10,13 +10,13 @@ import { map, switchMap } from 'rxjs/operators';
 import { ApiInterfaceRx } from '@polkadot/api/types';
 import { Bytes, Option, u32 } from '@polkadot/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 /**
  * @description Return a boolean array indicating whether the passed accounts had received heartbeats in the current session
  */
 export function receivedHeartbeats (api: ApiInterfaceRx): () => Observable<DerivedHeartbeats> {
-  return (): Observable<DerivedHeartbeats> =>
+  return memo((): Observable<DerivedHeartbeats> =>
     api.query.imOnline?.receivedHeartbeats && api.query.imOnline.authoredBlocks
       ? api.derive.staking.overview().pipe(
         switchMap(({ currentIndex, validators }): Observable<[AccountId[], Option<Bytes>[], u32[]]> =>
@@ -35,8 +35,7 @@ export function receivedHeartbeats (api: ApiInterfaceRx): () => Observable<Deriv
               isOnline: !heartbeats[index].isEmpty || numBlocks[index].gtn(0)
             }
           }), {})
-        ),
-        drr()
+        )
       )
-      : of({});
+      : of({}));
 }

packages/api-derive/src/index.spec.ts

@@ -5,6 +5,7 @@
 import { Observable, from } from 'rxjs';
 import ApiRx from '@polkadot/api/rx/Api';
 import MockProvider from '@polkadot/rpc-provider/mock';
+import { TypeRegistry } from '@polkadot/types';
 
 import { ExactDerive } from '.';
 
@@ -26,8 +27,10 @@ const testFunction = (api: ApiRx): any => {
 };
 
 describe('derive', (): void => {
+  const registry = new TypeRegistry();
+
   describe('builtin', (): void => {
-    const api = new ApiRx({ provider: new MockProvider() });
+    const api = new ApiRx({ provider: new MockProvider(registry), registry });
 
     beforeAll((done): void => {
       api.isReady.subscribe((): void => done());
@@ -77,7 +80,8 @@ describe('derive', (): void => {
           test: (): any => (): Observable<any> => from([1, 2, 3])
         }
       },
-      provider: new MockProvider()
+      provider: new MockProvider(registry),
+      registry
     });
 
     beforeAll((done): void => {

packages/api-derive/src/session/eraLength.ts

@@ -9,12 +9,11 @@ import { Observable } from 'rxjs';
 import { map } from 'rxjs/operators';
 import { ApiInterfaceRx } from '@polkadot/api/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 export function eraLength (api: ApiInterfaceRx): () => Observable<BlockNumber> {
-  return (): Observable<BlockNumber> =>
+  return memo((): Observable<BlockNumber> =>
     api.derive.session.info().pipe(
-      map(({ eraLength }: DerivedSessionInfo): BlockNumber => eraLength),
-      drr()
-    );
+      map(({ eraLength }: DerivedSessionInfo): BlockNumber => eraLength)
+    ));
 }

packages/api-derive/src/session/eraProgress.ts

@@ -9,12 +9,11 @@ import { Observable } from 'rxjs';
 import { map } from 'rxjs/operators';
 import { ApiInterfaceRx } from '@polkadot/api/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 export function eraProgress (api: ApiInterfaceRx): () => Observable<BlockNumber> {
-  return (): Observable<BlockNumber> =>
+  return memo((): Observable<BlockNumber> =>
     api.derive.session.info().pipe(
-      map(({ eraProgress }: DerivedSessionInfo): BlockNumber => eraProgress),
-      drr()
-    );
+      map(({ eraProgress }: DerivedSessionInfo): BlockNumber => eraProgress)
+    ));
 }

packages/api-derive/src/session/indexes.ts

@@ -6,22 +6,26 @@ import { ApiInterfaceRx } from '@polkadot/api/types';
 import { EraIndex, SessionIndex } from '@polkadot/types/interfaces';
 import { DeriveSessionIndexes } from '../types';
 
-import { Observable } from 'rxjs';
+import { Observable, of } from 'rxjs';
 import { map } from 'rxjs/operators';
-import { u32 } from '@polkadot/types';
+import { createType, u32 as U32 } from '@polkadot/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 export function indexes (api: ApiInterfaceRx): () => Observable<DeriveSessionIndexes> {
-  return (): Observable<DeriveSessionIndexes> =>
-    api.queryMulti<[SessionIndex, EraIndex, u32]>([
-      api.query.session.currentIndex,
-      api.query.staking.currentEra,
-      api.query.staking.validatorCount
-    ]).pipe(
+  return memo((): Observable<DeriveSessionIndexes> =>
+    (
+      // Some chains (eg. very limited node-template), does not have session
+      api.query.session && api.query.staking
+        ? api.queryMulti<[SessionIndex, EraIndex, U32]>([
+          api.query.session.currentIndex,
+          api.query.staking.currentEra,
+          api.query.staking.validatorCount
+        ])
+        : of([createType(api.registry, 'SessionIndex', 1), createType(api.registry, 'EraIndex', 1), createType(api.registry, 'u32')])
+    ).pipe(
       map(([currentIndex, currentEra, validatorCount]): DeriveSessionIndexes => ({
         currentIndex, currentEra, validatorCount
-      })),
-      drr()
-    );
+      }))
+    ));
 }

packages/api-derive/src/session/info.ts

@@ -10,7 +10,7 @@ import { map } from 'rxjs/operators';
 import { ApiInterfaceRx } from '@polkadot/api/types';
 import { Option, u64, createType } from '@polkadot/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 type ResultV1Session = [Option<BlockNumber>, BlockNumber, BlockNumber, SessionIndex];
 type ResultV1 = [BlockNumber, DeriveSessionIndexes, ResultV1Session];
@@ -20,8 +20,8 @@ type ResultType = [boolean, u64, SessionIndex];
 type Result = [ResultType, DeriveSessionIndexes, ResultSlots];
 
 // internal helper to just split the logic - take all inputs, do the calculations and combine
-function createDerivedV1 ([bestNumber, { currentIndex, validatorCount }, [_lastLengthChange, sessionLength, lastEraLengthChange, sessionsPerEra]]: ResultV1): DerivedSessionInfo {
-  const lastLengthChange = _lastLengthChange?.unwrapOr(null) || createType('BlockNumber');
+function createDerivedV1 (api: ApiInterfaceRx, [bestNumber, { currentIndex, validatorCount }, [_lastLengthChange, sessionLength, lastEraLengthChange, sessionsPerEra]]: ResultV1): DerivedSessionInfo {
+  const lastLengthChange = _lastLengthChange?.unwrapOr(null) || createType(api.registry, 'BlockNumber');
   const sessionProgress = bestNumber
     .sub(lastLengthChange)
     .add(sessionLength)
@@ -34,38 +34,32 @@ function createDerivedV1 ([bestNumber, { currentIndex, validatorCount }, [_lastL
     .add(sessionProgress);
 
   return {
-    currentEra: createType('EraIndex', currentEra),
+    currentEra: createType(api.registry, 'EraIndex', currentEra),
     currentIndex,
-    eraLength: createType('BlockNumber', sessionLength.mul(sessionsPerEra)),
-    eraProgress: createType('BlockNumber', eraProgress),
+    eraLength: createType(api.registry, 'BlockNumber', sessionLength.mul(sessionsPerEra)),
+    eraProgress: createType(api.registry, 'BlockNumber', eraProgress),
     isEpoch: false,
-    lastEraLengthChange,
-    lastLengthChange,
     sessionLength,
     sessionsPerEra,
-    sessionProgress: createType('BlockNumber', sessionProgress),
+    sessionProgress: createType(api.registry, 'BlockNumber', sessionProgress),
     validatorCount
   };
 }
 
-function createDerivedLatest ([[hasBabe, epochDuration, sessionsPerEra], { currentIndex, currentEra, validatorCount }, [currentSlot, epochIndex, epochOrGenesisStartSlot, currentEraStartSessionIndex]]: Result): DerivedSessionInfo {
+function createDerivedLatest (api: ApiInterfaceRx, [[hasBabe, epochDuration, sessionsPerEra], { currentIndex, currentEra, validatorCount }, [currentSlot, epochIndex, epochOrGenesisStartSlot, currentEraStartSessionIndex]]: Result): DerivedSessionInfo {
   const epochStartSlot = epochIndex.mul(epochDuration).add(epochOrGenesisStartSlot);
   const sessionProgress = currentSlot.sub(epochStartSlot);
-  const eraProgress = currentIndex.sub(currentEraStartSessionIndex).add(sessionProgress);
-
-  // console.log(sessionProgress);
+  const eraProgress = currentIndex.sub(currentEraStartSessionIndex).mul(epochDuration).add(sessionProgress);
 
   return {
     currentEra,
     currentIndex,
-    eraLength: createType('BlockNumber', sessionsPerEra.mul(epochDuration)),
-    eraProgress: createType('BlockNumber', eraProgress),
+    eraLength: createType(api.registry, 'BlockNumber', sessionsPerEra.mul(epochDuration)),
+    eraProgress: createType(api.registry, 'BlockNumber', eraProgress),
     isEpoch: hasBabe,
-    lastEraLengthChange: createType('BlockNumber'),
-    lastLengthChange: createType('BlockNumber', epochStartSlot),
     sessionLength: epochDuration,
     sessionsPerEra,
-    sessionProgress: createType('BlockNumber', sessionProgress),
+    sessionProgress: createType(api.registry, 'BlockNumber', sessionProgress),
     validatorCount
   };
 }
@@ -81,17 +75,17 @@ function infoV1 (api: ApiInterfaceRx): Observable<DerivedSessionInfo> {
       api.query.staking.sessionsPerEra
     ])
   ]).pipe(
-    map(createDerivedV1)
+    map((result): DerivedSessionInfo => createDerivedV1(api, result))
   );
 }
 
 function infoLatestAura (api: ApiInterfaceRx): Observable<DerivedSessionInfo> {
   return api.derive.session.indexes().pipe(
     map((indexes): DerivedSessionInfo =>
-      createDerivedLatest([
-        [false, createType('u64', 1), api.consts.staking.sessionsPerEra as SessionIndex],
+      createDerivedLatest(api, [
+        [false, createType(api.registry, 'u64', 1), api.consts.staking?.sessionsPerEra || createType(api.registry, 'SessionIndex', 1)],
         indexes,
-        [createType('u64', 1), createType('u64', 1), createType('u64', 1), createType('SessionIndex', 1)]
+        [createType(api.registry, 'u64', 1), createType(api.registry, 'u64', 1), createType(api.registry, 'u64', 1), createType(api.registry, 'SessionIndex', 1)]
       ])
     )
   );
@@ -108,8 +102,8 @@ function infoLatestBabe (api: ApiInterfaceRx): Observable<DerivedSessionInfo> {
     ])
   ]).pipe(
     map(([indexes, slots]: [DeriveSessionIndexes, ResultSlots]): DerivedSessionInfo =>
-      createDerivedLatest([
-        [true, api.consts.babe.epochDuration as u64, api.consts.staking.sessionsPerEra as SessionIndex],
+      createDerivedLatest(api, [
+        [true, api.consts.babe.epochDuration, api.consts.staking.sessionsPerEra],
         indexes,
         slots
       ])
@@ -121,12 +115,12 @@ function infoLatestBabe (api: ApiInterfaceRx): Observable<DerivedSessionInfo> {
  * @description Retrieves all the session and era info and calculates specific values on it as the length of the session and eras
  */
 export function info (api: ApiInterfaceRx): () => Observable<DerivedSessionInfo> {
-  const query = api.consts.staking
+  const query = api.consts.timestamp || api.consts.babe || api.consts.aura
     ? api.consts.babe
       ? infoLatestBabe // 2.x with Babe
       : infoLatestAura // 2.x with Aura (not all info there)
     : infoV1;
 
-  return (): Observable<DerivedSessionInfo> =>
-    query(api).pipe(drr());
+  return memo((): Observable<DerivedSessionInfo> =>
+    query(api));
 }

packages/api-derive/src/session/sessionProgress.ts

@@ -9,12 +9,11 @@ import { Observable } from 'rxjs';
 import { map } from 'rxjs/operators';
 import { ApiInterfaceRx } from '@polkadot/api/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 export function sessionProgress (api: ApiInterfaceRx): () => Observable<BlockNumber> {
-  return (): Observable<BlockNumber> =>
+  return memo((): Observable<BlockNumber> =>
     api.derive.session.info().pipe(
-      map(({ sessionProgress }: DerivedSessionInfo): BlockNumber => sessionProgress),
-      drr()
-    );
+      map(({ sessionProgress }: DerivedSessionInfo): BlockNumber => sessionProgress)
+    ));
 }

packages/api-derive/src/staking/controllers.ts

@@ -3,22 +3,24 @@
 // of the Apache-2.0 license. See the LICENSE file for details.
 
 import { ApiInterfaceRx } from '@polkadot/api/types';
-import { AccountId } from '@polkadot/types/interfaces';
-import { Codec } from '@polkadot/types/types';
+import { AccountId, ValidatorPrefs } from '@polkadot/types/interfaces';
+import { ITuple } from '@polkadot/types/types';
 
 import { Observable, combineLatest, of } from 'rxjs';
 import { switchMap } from 'rxjs/operators';
-import { Option } from '@polkadot/types';
+import { Option, Vec } from '@polkadot/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
+
+type DeriveControllers = [AccountId[], Option<AccountId>[]];
 
 /**
  * @description From the list of stash accounts, retrieve the list of controllers
  */
-export function controllers (api: ApiInterfaceRx): () => Observable<[AccountId[], Option<AccountId>[]]> {
-  return (): Observable<[AccountId[], Option<AccountId>[]]> =>
-    api.query.staking.validators<[AccountId[]] & Codec>().pipe(
-      switchMap(([stashIds]): Observable<[AccountId[], Option<AccountId>[]]> =>
+export function controllers (api: ApiInterfaceRx): () => Observable<DeriveControllers> {
+  return memo((): Observable<[AccountId[], Option<AccountId>[]]> =>
+    api.query.staking.validators<ITuple<[Vec<AccountId>, Vec<ValidatorPrefs>]>>().pipe(
+      switchMap(([stashIds]): Observable<DeriveControllers> =>
         combineLatest([
           of(stashIds),
           // for V2, don't return all the controllers, we call bonded at a later point
@@ -26,7 +28,6 @@ export function controllers (api: ApiInterfaceRx): () => Observable<[AccountId[]
             ? of([])
             : api.query.staking.bonded.multi<Option<AccountId>>(stashIds)
         ])
-      ),
-      drr()
-    );
+      )
+    ));
 }

packages/api-derive/src/staking/electedInfo.ts

@@ -0,0 +1,30 @@
+// Copyright 2017-2019 @polkadot/api-derive authors & contributors
+// This software may be modified and distributed under the terms
+// of the Apache-2.0 license. See the LICENSE file for details.
+
+import { ApiInterfaceRx } from '@polkadot/api/types';
+import { AccountId } from '@polkadot/types/interfaces';
+import { DerivedStaking, DerivedStakingElected } from '../types';
+
+import { Observable, combineLatest, of } from 'rxjs';
+import { map, switchMap } from 'rxjs/operators';
+
+import { memo } from '../util';
+
+export function electedInfo (api: ApiInterfaceRx): () => Observable<DerivedStakingElected> {
+  return memo((): Observable<DerivedStakingElected> =>
+    api.derive.staking.validators().pipe(
+      switchMap(({ currentElected }): Observable<[AccountId[], DerivedStaking[]]> =>
+        combineLatest([
+          of(currentElected),
+          combineLatest(currentElected.map((accountId): Observable<DerivedStaking> =>
+            api.derive.staking.info(accountId)
+          ))
+        ])
+      ),
+      map(([currentElected, info]): DerivedStakingElected => ({
+        currentElected, info
+      }))
+    )
+  );
+}

packages/api-derive/src/staking/index.ts

@@ -3,6 +3,7 @@
 // of the Apache-2.0 license. See the LICENSE file for details.
 
 export * from './controllers';
+export * from './electedInfo';
 export * from './info';
 export * from './overview';
 export * from './recentlyOffline';

packages/api-derive/src/staking/info.ts

@@ -2,11 +2,10 @@
 // This software may be modified and distributed under the terms
 // of the Apache-2.0 license. See the LICENSE file for details.
 
-import { AccountId, Balance, BlockNumber, Exposure, Keys, RewardDestination, StakingLedger, UnlockChunk, ValidatorPrefs } from '@polkadot/types/interfaces';
-import { Codec } from '@polkadot/types/types';
-
 import { ApiInterfaceRx } from '@polkadot/api/types';
-import { DerivedRecentlyOffline, DerivedStaking, DerivedUnlocking } from '../types';
+import { AccountId, Balance, BlockNumber, Exposure, Keys, Nominations, RewardDestination, StakingLedger, UnlockChunk, ValidatorPrefs } from '@polkadot/types/interfaces';
+import { ITuple } from '@polkadot/types/types';
+import { DerivedRecentlyOffline, DerivedSessionInfo, DerivedStaking, DerivedUnlocking } from '../types';
 
 import BN from 'bn.js';
 import { combineLatest, Observable, of } from 'rxjs';
@@ -15,17 +14,16 @@ import { createType, Option, Vec } from '@polkadot/types';
 
 import { isUndefined } from '@polkadot/util';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 interface ParseInput {
   accountId: AccountId;
-  bestNumber: BlockNumber;
   controllerId: AccountId;
-  eraLength: BlockNumber;
   recentlyOffline?: DerivedRecentlyOffline;
   nominators: AccountId[];
   rewardDestination: RewardDestination;
   queuedKeys?: [AccountId, Keys][];
+  sessionInfo: DerivedSessionInfo;
   stakers: Exposure;
   stakingLedger: Option<StakingLedger>;
   stashId: AccountId;
@@ -47,46 +45,50 @@ function groupByEra (list: UnlockChunk[]): Record<string, BN> {
   }, {});
 }
 
-// calculate the remining blocks in a specific unlock era
-function remainingBlocks (era: BN, eraLength: BN, bestNumber: BlockNumber): BlockNumber {
-  const remaining = eraLength.mul(era).sub(bestNumber);
+// calculate the remaining blocks in a specific unlock era
+function remainingBlocks (api: ApiInterfaceRx, era: BN, sessionInfo: DerivedSessionInfo): BlockNumber {
+  const remaining = era.sub(sessionInfo.currentEra);
 
-  return createType('BlockNumber', remaining.lten(0)
-    ? new BN(0)
-    : remaining
+  // on the Rust side the current-era >= era-for-unlock (removal done on >)
+  return createType(api.registry, 'BlockNumber', remaining.gtn(0)
+    ? remaining
+      .subn(1)
+      .mul(sessionInfo.eraLength)
+      .add(sessionInfo.eraLength.sub(sessionInfo.eraProgress))
+    : 0
   );
 }
 
-function calculateUnlocking (stakingLedger: StakingLedger | undefined, eraLength: BN, bestNumber: BlockNumber): DerivedUnlocking[] | undefined {
+function calculateUnlocking (api: ApiInterfaceRx, stakingLedger: StakingLedger | undefined, sessionInfo: DerivedSessionInfo): DerivedUnlocking[] | undefined {
   if (isUndefined(stakingLedger)) {
     return undefined;
   }
 
   const unlockingChunks = stakingLedger.unlocking.filter(({ era }): boolean =>
-    remainingBlocks(era.unwrap(), eraLength, bestNumber).gtn(0)
+    remainingBlocks(api, era.unwrap(), sessionInfo).gtn(0)
   );
 
   if (!unlockingChunks.length) {
     return undefined;
   }
 
-  // group the Unlockchunks that have the same era and sum their values
+  // group the unlock chunks that have the same era and sum their values
   const groupedResult = groupByEra(unlockingChunks);
   const results = Object.entries(groupedResult).map(([eraString, value]): DerivedUnlocking => ({
-    value: createType('Balance', value),
-    remainingBlocks: remainingBlocks(createType('BlockNumber', eraString), eraLength, bestNumber)
+    value: createType(api.registry, 'Balance', value),
+    remainingBlocks: remainingBlocks(api, new BN(eraString), sessionInfo)
   }));
 
   return results.length ? results : undefined;
 }
 
-function redeemableSum (stakingLedger: StakingLedger | undefined, eraLength: BN, bestNumber: BlockNumber): Balance {
+function redeemableSum (api: ApiInterfaceRx, stakingLedger: StakingLedger | undefined, sessionInfo: DerivedSessionInfo): Balance {
   if (isUndefined(stakingLedger)) {
-    return createType('Balance');
+    return createType(api.registry, 'Balance');
   }
 
-  return createType('Balance', stakingLedger.unlocking.reduce((total, { era, value }): BN => {
-    return remainingBlocks(era.unwrap(), eraLength, bestNumber).eqn(0)
+  return createType(api.registry, 'Balance', stakingLedger.unlocking.reduce((total, { era, value }): BN => {
+    return remainingBlocks(api, era.unwrap(), sessionInfo).eqn(0)
       ? total.add(value.unwrap())
       : total;
   }, new BN(0)));
@@ -117,31 +119,32 @@ function unwrapSessionIds (stashId: AccountId, queuedKeys: Option<AccountId> | [
   };
 }
 
-function parseResult ({ accountId, controllerId, stashId, eraLength, bestNumber, recentlyOffline = {}, nextKeyFor = createType('Option<AccountId>', null), queuedKeys, stakingLedger, nominators, rewardDestination, stakers, validatorPrefs, nextKeys = createType('Option<Keys>', null) }: ParseInput): DerivedStaking {
+function parseResult (api: ApiInterfaceRx, { accountId, controllerId, stashId, sessionInfo, recentlyOffline = {}, nextKeyFor, queuedKeys, stakingLedger, nominators, rewardDestination, stakers, validatorPrefs, nextKeys }: ParseInput): DerivedStaking {
   const _stakingLedger = stakingLedger.unwrapOr(undefined);
+  const _nextKeyFor = nextKeyFor || createType(api.registry, 'Option<AccountId>', null);
+  const _nextKeys = nextKeys || createType(api.registry, 'Option<Keys>', null);
 
   return {
     accountId,
     controllerId,
     nominators,
     offline: recentlyOffline[stashId.toString()],
-    redeemable: redeemableSum(_stakingLedger, eraLength, bestNumber),
+    redeemable: redeemableSum(api, _stakingLedger, sessionInfo),
     rewardDestination,
     stakers,
     stakingLedger: _stakingLedger,
     stashId,
-    unlocking: calculateUnlocking(_stakingLedger, eraLength, bestNumber),
+    unlocking: calculateUnlocking(api, _stakingLedger, sessionInfo),
     validatorPrefs,
-    ...unwrapSessionIds(stashId, queuedKeys || nextKeyFor, nextKeys)
+    ...unwrapSessionIds(stashId, queuedKeys || _nextKeyFor, _nextKeys)
   };
 }
 
-type MultiResultV1 = [Option<AccountId>, Option<StakingLedger>, [Vec<AccountId>], RewardDestination, Exposure, [ValidatorPrefs] & Codec];
+type MultiResultV1 = [Option<AccountId>, Option<StakingLedger>, [Vec<AccountId>], RewardDestination, Exposure, ITuple<[ValidatorPrefs]>];
 
 function retrieveInfoV1 (api: ApiInterfaceRx, accountId: AccountId, stashId: AccountId, controllerId: AccountId): Observable<DerivedStaking> {
   return combineLatest([
-    api.derive.chain.bestNumber(),
-    api.derive.session.eraLength(),
+    api.derive.session.info(),
     api.derive.staking.recentlyOffline(),
     api.queryMulti([
       [api.query.session.nextKeyFor, controllerId],
@@ -151,63 +154,64 @@ function retrieveInfoV1 (api: ApiInterfaceRx, accountId: AccountId, stashId: Acc
       [api.query.staking.stakers, stashId],
       [api.query.staking.validators, stashId]
     ]) as Observable<MultiResultV1>
-  ]).pipe(map(([
-    bestNumber, eraLength, recentlyOffline,
-    [nextKeyFor, stakingLedger, [nominators], rewardDestination, stakers, [validatorPrefs]]
-  ]: [BlockNumber, BlockNumber, DerivedRecentlyOffline, MultiResultV1]): DerivedStaking =>
-    parseResult({
-      accountId, controllerId, stashId, eraLength, bestNumber, recentlyOffline, nextKeyFor, stakingLedger, nominators, rewardDestination, stakers, validatorPrefs
+  ]).pipe(map(([sessionInfo, recentlyOffline, [nextKeyFor, stakingLedger, [nominators], rewardDestination, stakers, [validatorPrefs]]]: [DerivedSessionInfo, DerivedRecentlyOffline, MultiResultV1]): DerivedStaking =>
+    parseResult(api, {
+      accountId, controllerId, stashId, sessionInfo, recentlyOffline, nextKeyFor, stakingLedger, nominators, rewardDestination, stakers, validatorPrefs
     })
   ));
 }
 
-type MultiResultV2 = [Option<StakingLedger>, [Vec<AccountId>], RewardDestination, Exposure, [ValidatorPrefs], Option<Keys>];
+type MultiResultV2 = [[Vec<AccountId>] | Option<ITuple<[Nominations]>>, RewardDestination, Exposure, [ValidatorPrefs], Option<Keys>, Option<StakingLedger>];
 
 function retrieveInfoV2 (api: ApiInterfaceRx, accountId: AccountId, stashId: AccountId, controllerId: AccountId): Observable<DerivedStaking> {
   return combineLatest([
-    api.derive.chain.bestNumber(),
-    api.derive.session.eraLength(),
-    api.query.session.queuedKeys<Vec<[AccountId, Keys] & Codec>>(),
+    api.derive.session.info(),
+    api.query.session.queuedKeys<Vec<ITuple<[AccountId, Keys]>>>(),
     api.queryMulti([
-      [api.query.staking.ledger, controllerId],
       [api.query.staking.nominators, stashId],
       [api.query.staking.payee, stashId],
       [api.query.staking.stakers, stashId],
       [api.query.staking.validators, stashId],
-      [api.query.session.nextKeys, [api.consts.session.dedupKeyPrefix, stashId]]
+      [api.query.session.nextKeys, [api.consts.session.dedupKeyPrefix, stashId]],
+      [api.query.staking.ledger, controllerId]
     ]) as Observable<MultiResultV2>
-  ]).pipe(map(([
-    bestNumber, eraLength, queuedKeys,
-    [stakingLedger, [nominators], rewardDestination, stakers, [validatorPrefs], nextKeys]
-  ]: [BlockNumber, BlockNumber, [AccountId, Keys][], MultiResultV2]): DerivedStaking =>
-    parseResult({
-      accountId, controllerId, stashId, eraLength, bestNumber, queuedKeys, stakingLedger, nominators, rewardDestination, stakers, validatorPrefs, nextKeys
-    })
-  ));
+  ]).pipe(map(([sessionInfo, queuedKeys, [_nominators, rewardDestination, stakers, [validatorPrefs], nextKeys, stakingLedger]]: [DerivedSessionInfo, [AccountId, Keys][], MultiResultV2]): DerivedStaking => {
+    // if we have staking.storageVersion it indicates the new structure, unwrap as needed
+    // FIXME We really want to be pulling all the new (valuable) info along
+    const nominators: AccountId[] = api.query.staking.storageVersion
+      ? (_nominators as Option<ITuple<[Nominations]>>).isSome
+        ? (_nominators as Option<ITuple<[Nominations]>>).unwrap()[0].targets
+        : []
+      : (_nominators as [Vec<AccountId>])[0];
+
+    return parseResult(api, {
+      accountId, controllerId, stashId, sessionInfo, queuedKeys, stakingLedger, nominators, rewardDestination, stakers, validatorPrefs, nextKeys
+    });
+  }));
 }
 
-function retrieveV1 (api: ApiInterfaceRx, controllerId: AccountId): Observable<DerivedStaking> {
-  return api.query.staking
-    .ledger<Option<StakingLedger>>(controllerId)
-    .pipe(
-      switchMap((stakingLedger): Observable<DerivedStaking> =>
-        stakingLedger.isSome
-          ? retrieveInfoV1(api, controllerId, stakingLedger.unwrap().stash, controllerId)
-          : of({ accountId: controllerId, nextSessionIds: [], sessionIds: [] })
-      )
-    );
+function retrieveV1 (api: ApiInterfaceRx, accountId: AccountId): Observable<DerivedStaking> {
+  // depending on where we come from, this may be a controller or stash
+  return combineLatest([
+    api.query.staking.bonded<Option<AccountId>>(accountId),
+    api.query.staking.ledger<Option<StakingLedger>>(accountId)
+  ]).pipe(switchMap(([bonded, stakingLedger]): Observable<DerivedStaking> =>
+    stakingLedger.isSome
+      ? retrieveInfoV1(api, accountId, stakingLedger.unwrap().stash, accountId)
+      : bonded.isSome
+        ? retrieveInfoV1(api, accountId, accountId, bonded.unwrap())
+        : of({ accountId: accountId, nextSessionIds: [], sessionIds: [] })
+  ));
 }
 
 function retrieveV2 (api: ApiInterfaceRx, stashId: AccountId): Observable<DerivedStaking> {
   return api.query.staking
     .bonded<Option<AccountId>>(stashId)
-    .pipe(
-      switchMap((controllerId): Observable<DerivedStaking> =>
-        controllerId.isSome
-          ? retrieveInfoV2(api, stashId, stashId, controllerId.unwrap())
-          : of({ accountId: stashId, nextSessionIds: [], sessionIds: [] })
-      )
-    );
+    .pipe(switchMap((controllerId): Observable<DerivedStaking> =>
+      controllerId.isSome
+        ? retrieveInfoV2(api, stashId, stashId, controllerId.unwrap())
+        : of({ accountId: stashId, nextSessionIds: [], sessionIds: [] })
+    ));
 }
 
 /**
@@ -218,6 +222,6 @@ export function info (api: ApiInterfaceRx): (_accountId: Uint8Array | string) =>
     ? retrieveV2
     : retrieveV1;
 
-  return (accountId: Uint8Array | string): Observable<DerivedStaking> =>
-    query(api, createType('AccountId', accountId)).pipe(drr());
+  return memo((accountId: Uint8Array | string): Observable<DerivedStaking> =>
+    query(api, createType(api.registry, 'AccountId', accountId)));
 }

packages/api-derive/src/staking/infoStash.ts

@@ -11,7 +11,7 @@ import { Observable } from 'rxjs';
 import { map } from 'rxjs/operators';
 import { Option, Vec } from '@polkadot/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 // NOTE Unused ATM, preparing for V2-only
 
@@ -57,9 +57,8 @@ export function infoStash (api: ApiInterfaceRx): (stashId: AccountId) => Observa
     ? retrieveV2
     : retrieveV1;
 
-  return (stashId: AccountId): Observable<DerivedStakingStash> =>
+  return memo((stashId: AccountId): Observable<DerivedStakingStash> =>
     query(api, stashId).pipe(
-      map((result): DerivedStakingStash => parse(stashId, result)),
-      drr()
-    );
+      map((result): DerivedStakingStash => parse(stashId, result))
+    ));
 }

packages/api-derive/src/staking/overview.ts

@@ -10,13 +10,13 @@ import { Observable, combineLatest, of } from 'rxjs';
 import { map, switchMap } from 'rxjs/operators';
 import { createType } from '@polkadot/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 /**
  * @description Retrieve the staking overview, including elected and points earned
  */
 export function overview (api: ApiInterfaceRx): () => Observable<DerivedStakingOverview> {
-  return (): Observable<DerivedStakingOverview> =>
+  return memo((): Observable<DerivedStakingOverview> =>
     combineLatest([
       api.derive.session.indexes(),
       api.derive.staking.validators()
@@ -25,14 +25,13 @@ export function overview (api: ApiInterfaceRx): () => Observable<DerivedStakingO
         combineLatest([
           of({ currentElected, currentEra, currentIndex, validators, validatorCount }),
           // this will change on a per block basis, keep it innermost (and it needs eraIndex)
-          api.query.staking.currentEraPointsEarned
+          api.query.staking?.currentEraPointsEarned
             ? api.query.staking.currentEraPointsEarned<EraPoints>(currentEra)
-            : of(createType('EraPoints'))
+            : of(createType(api.registry, 'EraPoints'))
         ])
       ),
       map(([{ currentElected, currentEra, currentIndex, validators, validatorCount }, eraPoints]): DerivedStakingOverview => ({
         currentElected, currentEra, currentIndex, eraPoints, validators, validatorCount
-      })),
-      drr()
-    );
+      }))
+    ));
 }

packages/api-derive/src/staking/recentlyOffline.ts

@@ -2,18 +2,18 @@
 // This software may be modified and distributed under the terms
 // of the Apache-2.0 license. See the LICENSE file for details.
 
-import { AccountId, BlockNumber } from '@polkadot/types/interfaces';
-import { Codec } from '@polkadot/types/types';
-
-import BN from 'bn.js';
-import { Observable, of } from 'rxjs';
-import { map } from 'rxjs/operators';
 import { ApiInterfaceRx } from '@polkadot/api/types';
+import { AccountId, BlockNumber } from '@polkadot/types/interfaces';
+import { ITuple } from '@polkadot/types/types';
 import { DerivedRecentlyOffline } from '../types';
 
-import { drr } from '../util';
+import { Observable, of } from 'rxjs';
+import { map } from 'rxjs/operators';
+import { Vec, u32 } from '@polkadot/types';
 
-type OfflineResult = [AccountId, BlockNumber, BN][] & Codec;
+import { memo } from '../util';
+
+type OfflineResult = Vec<ITuple<[AccountId, BlockNumber, u32]>>;
 
 function expandDerived (recentlyOffline: OfflineResult): DerivedRecentlyOffline {
   return recentlyOffline.reduce((result: DerivedRecentlyOffline, [accountId, blockNumber, count]): DerivedRecentlyOffline => {
@@ -36,7 +36,7 @@ function expandDerived (recentlyOffline: OfflineResult): DerivedRecentlyOffline
  * @description Retrieve a keyed record of accounts recently reported to be offline
  */
 export function recentlyOffline (api: ApiInterfaceRx): () => Observable<DerivedRecentlyOffline> {
-  return (): Observable<DerivedRecentlyOffline> =>
+  return memo((): Observable<DerivedRecentlyOffline> =>
     (
       // TODO recentlyOffline  has been dropped for 2.x and replaced, figure out the
       // replacement as actually use and implement it
@@ -44,7 +44,6 @@ export function recentlyOffline (api: ApiInterfaceRx): () => Observable<DerivedR
         ? api.query.staking.recentlyOffline<OfflineResult>()
         : of([] as unknown as OfflineResult)
     ).pipe(
-      map(expandDerived),
-      drr()
-    );
+      map(expandDerived)
+    ));
 }

packages/api-derive/src/staking/validators.ts

@@ -6,24 +6,29 @@ import { AccountId } from '@polkadot/types/interfaces';
 import { ApiInterfaceRx } from '@polkadot/api/types';
 import { DeriveStakingValidators } from '../types';
 
-import { Observable } from 'rxjs';
+import { Observable, of } from 'rxjs';
 import { map } from 'rxjs/operators';
 import { Vec } from '@polkadot/types';
 
-import { drr } from '../util';
+import { memo } from '../util';
 
 /**
  * @description Retrieve latest list of validators
  */
 export function validators (api: ApiInterfaceRx): () => Observable<DeriveStakingValidators> {
-  return (): Observable<DeriveStakingValidators> =>
-    api.queryMulti<[Vec<AccountId>, Vec<AccountId>]>([
-      api.query.session.validators,
-      api.query.staking.currentElected
-    ]).pipe(
+  return memo((): Observable<DeriveStakingValidators> =>
+    (
+      // Sadly the node-template is (for some obscure reason) not comprehensive, so while the derive works
+      // in all actual real-world deployed chains, it does create some confusion for limited template chains
+      api.query.session && api.query.staking
+        ? api.queryMulti<[Vec<AccountId>, Vec<AccountId>]>([
+          api.query.session.validators,
+          api.query.staking.currentElected
+        ])
+        : of([[], []] as [AccountId[], AccountId[]])
+    ).pipe(
       map(([validators, currentElected]): DeriveStakingValidators => ({
         currentElected, validators
-      })),
-      drr()
-    );
+      }))
+    ));
 }

packages/api-derive/src/type/HeaderExtended.ts

@@ -3,7 +3,7 @@
 // of the Apache-2.0 license. See the LICENSE file for details.
 
 import { AccountId, Header } from '@polkadot/types/interfaces';
-import { AnyJsonObject, Constructor } from '@polkadot/types/types';
+import { AnyJsonObject, Constructor, Registry } from '@polkadot/types/types';
 
 import runtimeTypes from '@polkadot/types/interfaces/runtime/definitions';
 import { Struct } from '@polkadot/types';
@@ -19,8 +19,8 @@ const _Header: Constructor<Header> = Struct.with(runtimeTypes.types.Header as an
 export default class HeaderExtended extends _Header {
   private _author?: AccountId;
 
-  constructor (header?: Header, sessionValidators?: AccountId[]) {
-    super(header);
+  constructor (registry: Registry, header?: Header, sessionValidators?: AccountId[]) {
+    super(registry, header);
 
     this._author = this.extractAuthor(sessionValidators);
   }

packages/api-derive/src/type/ReferendumInfoExtended.ts

@@ -3,7 +3,7 @@
 // of the Apache-2.0 license. See the LICENSE file for details.
 
 import { ReferendumIndex, ReferendumInfo } from '@polkadot/types/interfaces/democracy';
-import { AnyJsonObject, Constructor } from '@polkadot/types/types';
+import { AnyJsonObject, Constructor, Registry } from '@polkadot/types/types';
 
 import BN from 'bn.js';
 import democracyTypes from '@polkadot/types/interfaces/democracy/definitions';
@@ -20,12 +20,12 @@ const _ReferendumInfo: Constructor<ReferendumInfo> = Struct.with(democracyTypes.
 export default class ReferendumInfoExtended extends _ReferendumInfo {
   private _index: ReferendumIndex;
 
-  constructor (value: ReferendumInfo | ReferendumInfoExtended, index?: BN | number) {
-    super(value);
+  constructor (registry: Registry, value: ReferendumInfo | ReferendumInfoExtended, index?: BN | number) {
+    super(registry, value);
 
     this._index = value instanceof ReferendumInfoExtended
       ? value.index
-      : createType('ReferendumIndex', index);
+      : createType(registry, 'ReferendumIndex', index);
   }
 
   /**

packages/api-derive/src/types.ts

@@ -2,10 +2,10 @@
 // This software may be modified and distributed under the terms
 // of the Apache-2.0 license. See the LICENSE file for details.
 
-import { AccountId, AccountIndex, Balance, BalanceLock, BlockNumber, EraIndex, EraPoints, Exposure, Index, Keys, RewardDestination, SessionIndex, SetIndex, StakingLedger, ValidatorPrefs, Vote, VoteIndex } from '@polkadot/types/interfaces';
+import { AccountId, AccountIndex, Balance, BalanceLock, BlockNumber, EraIndex, EraPoints, Exposure, Hash, Index, Keys, Proposal, PropIndex, ReferendumInfo, RewardDestination, SessionIndex, SetIndex, StakingLedger, ValidatorPrefs, Vote, VoteIndex } from '@polkadot/types/interfaces';
 
 import BN from 'bn.js';
-import { u32 } from '@polkadot/types';
+import { u32, Vec } from '@polkadot/types';
 
 export type AccountIndexes = Record<string, AccountIndex>;
 
@@ -19,6 +19,7 @@ export interface DerivedBalances {
   accountId: AccountId;
   accountNonce: Index;
   freeBalance: Balance;
+  isVesting: boolean;
   lockedBalance: Balance;
   lockedBreakdown: BalanceLock[];
   availableBalance: Balance;
@@ -78,6 +79,30 @@ export interface RecentlyOffline {
   count: BN;
 }
 
+export interface DeriveProposalPreImage {
+  at: BlockNumber;
+  balance: Balance;
+  proposer: AccountId;
+}
+
+export interface DeriveProposal {
+  balance?: Balance;
+  hash: Hash;
+  index: PropIndex;
+  preimage?: DeriveProposalPreImage;
+  proposal?: Proposal;
+  proposer: AccountId;
+  seconds: Vec<AccountId>;
+}
+
+export interface DerivedReferendum {
+  hash: Hash;
+  index: PropIndex;
+  info: ReferendumInfo;
+  preimage?: DeriveProposalPreImage;
+  proposal?: Proposal;
+}
+
 export type DerivedRecentlyOffline = Record<string, RecentlyOffline[]>;
 
 export interface DerivedReferendumVote {
@@ -96,8 +121,6 @@ export interface DerivedSessionInfo extends DeriveSessionIndexes {
   eraLength: BlockNumber;
   eraProgress: BlockNumber;
   isEpoch: boolean;
-  lastEraLengthChange: BlockNumber;
-  lastLengthChange: BlockNumber;
   sessionLength: BlockNumber;
   sessionsPerEra: SessionIndex;
   sessionProgress: BlockNumber;
@@ -107,6 +130,11 @@ export type DerivedStakingAccount = [AccountId, DerivedStakingOnlineStatus];
 
 export type DerivedStakingAccounts = DerivedStakingAccount[];
 
+export interface DerivedStakingElected {
+  currentElected: AccountId[];
+  info: DerivedStaking[];
+}
+
 export interface DerivedStakingOnlineStatus {
   online?: {
     isOnline: boolean;

packages/api-derive/src/util/approvalFlagToBools.spec.ts

@@ -4,11 +4,13 @@
 
 import { ApprovalFlag } from '@polkadot/types/interfaces';
 
-import { createType } from '@polkadot/types';
+import { createType, TypeRegistry } from '@polkadot/types';
 
 import { approvalFlagsToBools } from './approvalFlagsToBools';
 
 describe('approvalFlagsToBools', (): void => {
+  const registry = new TypeRegistry();
+
   it('translates and empty array to empty', (): void => {
     expect(
       approvalFlagsToBools([] as ApprovalFlag[])
@@ -18,7 +20,7 @@ describe('approvalFlagsToBools', (): void => {
   it('translates a single input', (): void => {
     expect(
       approvalFlagsToBools([
-        createType('ApprovalFlag', 0b1010)
+        createType(registry, 'ApprovalFlag', 0b1010)
       ])
     ).toEqual([false, true, false, true]);
   });
@@ -26,8 +28,8 @@ describe('approvalFlagsToBools', (): void => {
   it('translates multiple inputs', (): void => {
     expect(
       approvalFlagsToBools([
-        createType('ApprovalFlag', 0b0000),
-        createType('ApprovalFlag', 0b1100)
+        createType(registry, 'ApprovalFlag', 0b0000),
+        createType(registry, 'ApprovalFlag', 0b1100)
       ])
     ).toEqual([false, false, false, true, true]);
   });

packages/api-derive/src/util/drr.ts

@@ -1,8 +0,0 @@
-// Copyright 2017-2019 @polkadot/api-derive authors & contributors
-// This software may be modified and distributed under the terms
-// of the Apache-2.0 license. See the LICENSE file for details.
-
-import { drr as drrBase, DrrResult } from '@polkadot/rpc-core/rxjs';
-
-export const drr = (): DrrResult =>
-  drrBase({ skipTimeout: true });

packages/api-derive/src/util/index.ts

@@ -3,5 +3,4 @@
 // of the Apache-2.0 license. See the LICENSE file for details.
 
 export * from './approvalFlagsToBools';
-export * from './drr';
 export * from './memo';

packages/api-derive/src/util/memo.ts

@@ -3,12 +3,31 @@
 // of the Apache-2.0 license. See the LICENSE file for details.v
 
 import createMemo from 'memoizee';
+import { Observable, Observer } from 'rxjs';
+import { drr } from '@polkadot/rpc-core/rxjs';
 
-export function memo <T extends Function> (fn: T, withoutJSON?: boolean): T {
-  return withoutJSON
-    ? createMemo(fn)
-    : createMemo(fn, {
-      // eslint-disable-next-line @typescript-eslint/unbound-method
-      normalizer: JSON.stringify
-    });
+type ObsFn <T> = (...params: any[]) => Observable<T>;
+
+// Normalize via JSON.stringify, allow e.g. AccountId -> ss58
+// eslint-disable-next-line @typescript-eslint/unbound-method
+const normalizer = JSON.stringify;
+
+// Wraps a derive, doing 2 things to optimize calls -
+//   1. creates a memo of the inner fn -> Observable, removing when unsubscribed
+//   2. wraps the observable in a drr() (which includes an unsub delay)
+export function memo <T> (inner: ObsFn<T>): ObsFn<T> {
+  const cached = createMemo(
+    (...params: any[]): Observable<T> =>
+      Observable.create((observer: Observer<T>): VoidCallback => {
+        const sub = inner(...params).subscribe(observer);
+
+        return (): void => {
+          cached.delete(...params);
+          sub.unsubscribe();
+        };
+      }).pipe(drr()),
+    { normalizer }
+  );
+
+  return cached;
 }

packages/api-metadata/README.md

@@ -1 +0,0 @@
-# @polkadot/api-metadata

packages/api-metadata/src/consts/README.md

@@ -1,3 +0,0 @@
-# @polkadot/api-metadata/consts
-
-A base interface to the parameter types

packages/api-metadata/src/consts/static.ts

@@ -1,12 +0,0 @@
-// Copyright 2017-2019 @polkadot/api-metadata authors & contributors
-// This software may be modified and distributed under the terms
-// of the Apache-2.0 license. See the LICENSE file for details.
-
-import { Metadata } from '@polkadot/types';
-import metadataRpc from '@polkadot/types/Metadata/static';
-
-import fromMetadata from './fromMetadata';
-
-export default fromMetadata(
-  new Metadata(metadataRpc)
-);

packages/api-metadata/src/consts/types.ts

@@ -1,14 +0,0 @@
-// Copyright 2017-2019 @polkadot/api-metadata authors & contributors
-// This software may be modified and distributed under the terms
-// of the Apache-2.0 license. See the LICENSE file for details.
-
-import { Codec } from '@polkadot/types/types';
-import { ModuleConstantMetadataLatest } from '@polkadot/types/interfaces';
-
-export interface ConstantCodec extends Codec {
-  meta: ModuleConstantMetadataLatest;
-}
-
-export type ModuleConstants = Record<string, Codec>;
-
-export type Constants = Record<string, ModuleConstants>;

packages/api-metadata/src/extrinsics/README.md

@@ -1,3 +0,0 @@
-# @polkadot/api-metadata/extrinsics
-
-Base extrinsic definitions & codecs

packages/api-metadata/src/extrinsics/static.ts

@@ -1,15 +0,0 @@
-// Copyright 2017-2019 @polkadot/api-metadata authors & contributors
-// This software may be modified and distributed under the terms
-// of the Apache-2.0 license. See the LICENSE file for details.
-
-import { ModulesWithCalls } from '@polkadot/types/types';
-
-import Metadata from '@polkadot/types/Metadata';
-import metadataRpc from '@polkadot/types/Metadata/static';
-
-import fromMetadata from './fromMetadata';
-
-const staticMetadata: ModulesWithCalls = fromMetadata(
-  new Metadata(metadataRpc)
-);
-export default staticMetadata;

packages/api-metadata/src/storage/README.md

@@ -1,3 +0,0 @@
-# @polkadot/api-metadata/storage
-
-A base interface to the storage layer

packages/api-metadata/src/storage/fromMetadata/createFunction.ts

@@ -1,170 +0,0 @@
-// Copyright 2017-2019 @polkadot/api-metadata authors & contributors
-// This software may be modified and distributed under the terms
-// of the Apache-2.0 license. See the LICENSE file for details.
-
-import { Codec } from '@polkadot/types/types';
-
-import BN from 'bn.js';
-import { Compact, U8a, createType, createTypeUnsafe } from '@polkadot/types/codec';
-import { StorageEntryMetadata, StorageEntryType } from '@polkadot/types/Metadata/v8/Storage';
-import { StorageEntry } from '@polkadot/types/primitive/StorageKey';
-import { assert, isNull, isUndefined, stringLowerFirst, stringToU8a, u8aConcat } from '@polkadot/util';
-
-import getHasher, { HasherFunction } from './getHasher';
-
-export interface CreateItemOptions {
-  key?: string;
-  skipHashing?: boolean; // We don't hash the keys defined in ./substrate.ts
-}
-
-export interface CreateItemFn {
-  meta: StorageEntryMetadata;
-  method: string;
-  prefix: string;
-  section: string;
-}
-
-type CreateArgType = boolean | string | number | null | BN | Uint8Array | Codec;
-
-const NULL_HASHER = (value: Uint8Array): Uint8Array => value;
-
-// with the prefix, method & options, create both the string & raw keys
-function createKeys ({ method, prefix }: CreateItemFn, options: CreateItemOptions): [string, Uint8Array] {
-  const stringKey = options.key
-    ? options.key
-    : `${prefix} ${method}`;
-
-  return [
-    stringKey,
-    stringToU8a(stringKey)
-  ];
-}
-
-// get the hashers, the base (and  in the case of DoubleMap), the second key
-function getHashers ({ meta: { type } }: CreateItemFn): [HasherFunction, HasherFunction?] {
-  if (type.isDoubleMap) {
-    return [
-      getHasher(type.asDoubleMap.hasher),
-      getHasher(type.asDoubleMap.key2Hasher)
-    ];
-  } else if (type.isMap) {
-    return [getHasher(type.asMap.hasher)];
-  }
-
-  // the default
-  return [getHasher()];
-}
-
-// create a key for a DoubleMap type
-function createKeyDoubleMap ({ meta: { name, type } }: CreateItemFn, rawKey: Uint8Array, args: [CreateArgType, CreateArgType], [hasher, key2Hasher]: [HasherFunction, HasherFunction?]): Uint8Array {
-  // since we are passing an almost-unknown through, trust, but verify
-  assert(
-    Array.isArray(args) && !isUndefined(args[0]) && !isNull(args[0]) && !isUndefined(args[1]) && !isNull(args[1]),
-    `${name} is a DoubleMap and requires two arguments`
-  );
-
-  const [key1, key2] = args;
-  const type1 = type.asDoubleMap.key1.toString();
-  const type2 = type.asDoubleMap.key2.toString();
-  const param1Encoded = u8aConcat(rawKey, createTypeUnsafe(type1, [key1]).toU8a(true));
-  const param1Hashed = hasher(param1Encoded);
-
-  // If this fails it means the getHashers function failed - and we have much bigger issues
-  const param2Hashed = (key2Hasher as HasherFunction)(createTypeUnsafe(type2, [key2]).toU8a(true));
-
-  // as per createKey, always add the length prefix (underlying it is Bytes)
-  return Compact.addLengthPrefix(u8aConcat(param1Hashed, param2Hashed));
-}
-
-// create a key for either a map or a plain value
-function createKey ({ meta: { name, type } }: CreateItemFn, rawKey: Uint8Array, arg: CreateArgType, hasher: (value: Uint8Array) => Uint8Array): Uint8Array {
-  let key = rawKey;
-
-  if (type.isMap) {
-    assert(!isUndefined(arg) && !isNull(arg), `${name} is a Map and requires one argument`);
-
-    const mapType = type.asMap.key.toString();
-    const param = createTypeUnsafe(mapType, [arg]).toU8a();
-
-    key = u8aConcat(key, param);
-  }
-
-  // StorageKey is a Bytes, so is length-prefixed
-  return Compact.addLengthPrefix(hasher(key));
-}
-
-// attach the metadata to expsnd to a StorageFunction
-function expandWithMeta ({ meta, method, prefix, section }: CreateItemFn, storageFn: StorageEntry): StorageEntry {
-  storageFn.meta = meta;
-  storageFn.method = stringLowerFirst(method);
-  storageFn.prefix = prefix;
-  storageFn.section = section;
-
-  // explicitly add the actual method in the toJSON, this gets used to determine caching and without it
-  // instances (e.g. collective) will not work since it is only matched on param meta
-  storageFn.toJSON = (): any => ({
-    ...(meta.toJSON() as any),
-    storage: { method, prefix, section }
-  });
-
-  return storageFn;
-}
-
-// attch the head key hashing for linked maps
-function extendLinkedMap ({ meta: { documentation, name, type } }: CreateItemFn, storageFn: StorageEntry, stringKey: string, hasher: HasherFunction): StorageEntry {
-  const headHash = new U8a(hasher(`head of ${stringKey}`));
-  const headFn: any = (): U8a =>
-    headHash;
-
-  // metadata with a fallback value using the type of the key, the normal
-  // meta fallback only applies to actual entry values, create one for head
-  headFn.meta = new StorageEntryMetadata({
-    name,
-    modifier: createType('StorageEntryModifierLatest', 1), // required
-    type: new StorageEntryType(createType('PlainTypeLatest', type.asMap.key), 0),
-    fallback: createType('Bytes', createTypeUnsafe(type.asMap.key.toString()).toHex()),
-    documentation
-  });
-
-  // here we pass the section/method through as well - these are not on
-  // the function itself, so specify these explicitly to the constructor
-  storageFn.headKey = createType('StorageKey', headFn, {
-    method: storageFn.method,
-    section: `head of ${storageFn.section}`
-  });
-
-  return storageFn;
-}
-
-/**
- * From the schema of a function in the module's storage, generate the function
- * that will return the correct storage key.
- *
- * @param item - The function's definition schema to create the function from.
- * The schema is taken from state_getMetadata.
- * @param options - Additional options when creating the function. These options
- * are not known at runtime (from state_getMetadata), they need to be supplied
- * by us manually at compile time.
- */
-export default function createFunction (item: CreateItemFn, options: CreateItemOptions = {}): StorageEntry {
-  const { meta: { type } } = item;
-  const [stringKey, rawKey] = createKeys(item, options);
-  const [hasher, key2Hasher] = getHashers(item);
-
-  // Can only have zero or one argument:
-  //   - storage.balances.freeBalance(address)
-  //   - storage.timestamp.blockPeriod()
-  // For doublemap queries the params is passed in as an tuple, [key1, key2]
-  const _storageFn = (arg?: CreateArgType | [CreateArgType?, CreateArgType?]): Uint8Array =>
-    type.isDoubleMap
-      ? createKeyDoubleMap(item, rawKey, arg as [CreateArgType, CreateArgType], [hasher, key2Hasher])
-      : createKey(item, rawKey, arg as CreateArgType, options.skipHashing ? NULL_HASHER : hasher);
-
-  const storageFn = expandWithMeta(item, _storageFn as StorageEntry);
-
-  if (type.isMap && type.asMap.linked.isTrue) {
-    extendLinkedMap(item, storageFn, stringKey, hasher);
-  }
-
-  return storageFn;
-}

packages/api-metadata/src/storage/fromMetadata/storage.ts

@@ -1,10 +0,0 @@
-// Copyright 2017-2019 @polkadot/api-metadata authors & contributors
-// This software may be modified and distributed under the terms
-// of the Apache-2.0 license. See the LICENSE file for details.
-
-import { Storage } from '../types';
-import * as substrate from './substrate';
-
-export const storage: Storage = {
-  substrate // Prefill storage with well known keys, as not returned by state_getMetadata
-};

packages/api-metadata/src/storage/index.ts

@@ -1,7 +0,0 @@
-// Copyright 2017-2019 @polkadot/api-metadata authors & contributors
-// This software may be modified and distributed under the terms
-// of the Apache-2.0 license. See the LICENSE file for details.
-
-import { storage } from './fromMetadata/storage';
-
-export default storage;

packages/api-metadata/src/storage/static.ts

@@ -1,12 +0,0 @@
-// Copyright 2017-2019 @polkadot/api-metadata authors & contributors
-// This software may be modified and distributed under the terms
-// of the Apache-2.0 license. See the LICENSE file for details.
-
-import { Metadata } from '@polkadot/types';
-import metadataRpc from '@polkadot/types/Metadata/static';
-
-import fromMetadata from './fromMetadata';
-
-export default fromMetadata(
-  new Metadata(metadataRpc)
-);

packages/api/README.md

@@ -94,6 +94,7 @@ Some of the users of the API (let us know if you are missing from the list), inc
 - [Polkadot-JS UI](https://github.com/polkadot-js/apps) A user-interface that allows you to make transactions, query the network or participate in actions on the network such as referendums and staking
 - [Polkabot](https://gitlab.com/Polkabot) Polkabot is a Matrix chatbot that keeps an eye on the Polkadot network. You can see Polkabot in action in https://matrix.to/#/#polkadot-network-status:matrix.org
 - [Polkawallet.io](https://polkawallet.io) and [Polkawallet (Github)](https://github.com/polkawallet-io/polkawallet-RN/) A mobile wallet for the Polkadot network to manage funds and make transactions, available on both Androind and iOS
+- [PolkaStats.io](https://polkastats.io), [PolkaStats frontend GitHub repository](https://github.com/Colm3na/polkastats-v2) and [PolkaStats backend GitHub repository](https://github.com/Colm3na/polkastats-backend-v2) Polkadot network statistics (currently Kusama and Alexander). Shows network information and staking details from validators and intentions.
 
 ## Classes
 

packages/api/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@polkadot/api",
-  "version": "0.96.1",
+  "version": "0.98.1",
   "description": "Promise and RxJS wrappers around the Polkadot JS RPC",
   "main": "index.js",
   "keywords": [
@@ -26,16 +26,16 @@
   },
   "homepage": "https://github.com/polkadot-js/api/tree/master/packages/api#readme",
   "dependencies": {
-    "@babel/runtime": "^7.7.1",
-    "@polkadot/api-derive": "^0.96.1",
-    "@polkadot/api-metadata": "^0.96.1",
-    "@polkadot/keyring": "^1.7.0-beta.5",
-    "@polkadot/rpc-core": "^0.96.1",
-    "@polkadot/rpc-provider": "^0.96.1",
-    "@polkadot/types": "^0.96.1",
-    "@polkadot/util-crypto": "^1.7.0-beta.5"
+    "@babel/runtime": "^7.7.4",
+    "@polkadot/api-derive": "^0.98.1",
+    "@polkadot/keyring": "^1.7.1",
+    "@polkadot/metadata": "^0.98.1",
+    "@polkadot/rpc-core": "^0.98.1",
+    "@polkadot/rpc-provider": "^0.98.1",
+    "@polkadot/types": "^0.98.1",
+    "@polkadot/util-crypto": "^1.7.1"
   },
   "devDependencies": {
-    "@polkadot/keyring": "^1.7.0-beta.5"
+    "@polkadot/keyring": "^1.7.1"
   }
 }

packages/api/src/base/Decorate.ts

@@ -2,9 +2,10 @@
 // This software may be modified and distributed under the terms
 // of the Apache-2.0 license. See the LICENSE file for details.
 
+import { Constants, Storage } from '@polkadot/metadata/Decorated/types';
 import { RpcInterface } from '@polkadot/rpc-core/jsonrpc.types';
 import { Call, Hash, RuntimeVersion } from '@polkadot/types/interfaces';
-import { AnyFunction, CallFunction, Codec, CodecArg as Arg, ModulesWithCalls } from '@polkadot/types/types';
+import { AnyFunction, CallFunction, Codec, CodecArg as Arg, ModulesWithCalls, Registry } from '@polkadot/types/types';
 import { SubmittableExtrinsic } from '../submittable/types';
 import {
   ApiInterfaceRx, ApiOptions, ApiTypes,
@@ -14,18 +15,16 @@ import {
 } from '../types';
 
 import BN from 'bn.js';
-import { BehaviorSubject, Observable } from 'rxjs';
+import { BehaviorSubject, Observable, combineLatest, of } from 'rxjs';
 import { map, switchMap } from 'rxjs/operators';
 import decorateDerive, { ExactDerive } from '@polkadot/api-derive';
-import { Constants } from '@polkadot/api-metadata/consts/types';
-import { Storage } from '@polkadot/api-metadata/storage/types';
 import RpcCore from '@polkadot/rpc-core';
 import { WsProvider } from '@polkadot/rpc-provider';
-import { Metadata, Null, u64 } from '@polkadot/types';
+import { Metadata, Null, TypeRegistry, u64, Vec } from '@polkadot/types';
 import Linkage, { LinkageResult } from '@polkadot/types/codec/Linkage';
 import { DEFAULT_VERSION as EXTRINSIC_DEFAULT_VERSION } from '@polkadot/types/primitive/Extrinsic/constants';
-import { StorageEntry } from '@polkadot/types/primitive/StorageKey';
-import { compactStripLength, u8aToHex } from '@polkadot/util';
+import StorageKey, { StorageEntry } from '@polkadot/types/primitive/StorageKey';
+import { assert, compactStripLength, u8aToHex } from '@polkadot/util';
 
 import { createSubmittable } from '../submittable';
 import { decorateSections, DeriveAllSections } from '../util/decorate';
@@ -40,6 +39,8 @@ interface MetaDecoration {
 }
 
 export default abstract class Decorate<ApiType extends ApiTypes> extends Events {
+  public readonly registry: Registry;
+
   // HACK Use BN import so decorateDerive works... yes, wtf.
   protected __phantom = new BN(0);
 
@@ -89,7 +90,7 @@ export default abstract class Decorate<ApiType extends ApiTypes> extends Events
    *
    * We could easily imagine other user-facing interfaces, which are simply
    * implemented by transforming the Observable to Stream/Iterator/Kefir/Bacon
-   * via `deocrateMethod`.
+   * via `decorateMethod`.
    */
   protected decorateMethod: DecorateMethod<ApiType>;
 
@@ -114,6 +115,8 @@ export default abstract class Decorate<ApiType extends ApiTypes> extends Events
   constructor (options: ApiOptions, type: ApiTypes, decorateMethod: DecorateMethod<ApiType>) {
     super();
 
+    this.registry = options.registry || new TypeRegistry();
+
     const thisProvider = options.source
       ? options.source._rpcCore.provider.clone()
       : (options.provider || new WsProvider());
@@ -121,9 +124,10 @@ export default abstract class Decorate<ApiType extends ApiTypes> extends Events
     this.decorateMethod = decorateMethod;
     this._options = options;
     this._type = type;
-    this._rpcCore = new RpcCore(thisProvider, this._options.rpc);
+    this._rpcCore = new RpcCore(this.registry, thisProvider, this._options.rpc);
     this._isConnected = new BehaviorSubject(this._rpcCore.provider.isConnected());
     this._rx.hasSubscriptions = this._rpcCore.provider.hasSubscriptions;
+    this._rx.registry = this.registry;
   }
 
   /**
@@ -164,7 +168,7 @@ export default abstract class Decorate<ApiType extends ApiTypes> extends Events
     this.filterRpcMethods(methods);
   }
 
-  protected filterRpcMethods (methods: string[]): void{
+  protected filterRpcMethods (methods: string[]): void {
     // this is true when the RPC has entries
     const hasResults = methods.length !== 0;
 
@@ -195,10 +199,10 @@ export default abstract class Decorate<ApiType extends ApiTypes> extends Events
         }
 
         return section;
-      }, {} as unknown as DecoratedRpcSection<ApiType, RpcInterface[typeof sectionName]>);
+      }, {} as DecoratedRpcSection<ApiType, RpcInterface[typeof sectionName]>);
 
       return out;
-    }, {} as unknown as DecoratedRpc<ApiType, RpcInterface>);
+    }, {} as DecoratedRpc<ApiType, RpcInterface>);
   }
 
   protected decorateMulti<ApiType extends ApiTypes> (decorateMethod: DecorateMethod<ApiType>): QueryableStorageMulti<ApiType> {
@@ -219,10 +223,10 @@ export default abstract class Decorate<ApiType extends ApiTypes> extends Events
         out[name] = this.decorateExtrinsicEntry(method, creator);
 
         return out;
-      }, {} as unknown as SubmittableModuleExtrinsics<ApiType>);
+      }, {} as SubmittableModuleExtrinsics<ApiType>);
 
       return out;
-    }, creator as unknown as SubmittableExtrinsics<ApiType>);
+    }, creator as SubmittableExtrinsics<ApiType>);
   }
 
   private decorateExtrinsicEntry<ApiType extends ApiTypes> (method: CallFunction, creator: (value: Call | Uint8Array | string) => SubmittableExtrinsic<ApiType>): SubmittableExtrinsicFunction<ApiType> {
@@ -238,10 +242,10 @@ export default abstract class Decorate<ApiType extends ApiTypes> extends Events
         out[name] = this.decorateStorageEntry(method, decorateMethod);
 
         return out;
-      }, {} as unknown as QueryableModuleStorage<ApiType>);
+      }, {} as QueryableModuleStorage<ApiType>);
 
       return out;
-    }, {} as unknown as QueryableStorage<ApiType>);
+    }, {} as QueryableStorage<ApiType>);
   }
 
   private decorateStorageEntry<ApiType extends ApiTypes> (creator: StorageEntry, decorateMethod: DecorateMethod<ApiType>): QueryableStorageEntry<ApiType> {
@@ -251,8 +255,10 @@ export default abstract class Decorate<ApiType extends ApiTypes> extends Events
       : [creator, ...args];
 
     // FIXME We probably want to be able to query the full list with non-subs as well
-    const decorated = this.hasSubscriptions && creator.headKey
-      ? this.decorateStorageEntryLinked(creator, decorateMethod)
+    const decorated = this.hasSubscriptions && creator.iterKey && (creator.meta.type.asMap.kind.isLinkedMap || creator.meta.type.asMap.kind.isPrefixedMap)
+      ? creator.meta.type.asMap.kind.isLinkedMap
+        ? this.decorateStorageLinked(creator, decorateMethod)
+        : this.decorateStoragePrefixed(creator, decorateMethod)
       : decorateMethod((...args: any[]): Observable<Codec> => (
         this.hasSubscriptions
           ? this._rpcCore.state
@@ -269,6 +275,9 @@ export default abstract class Decorate<ApiType extends ApiTypes> extends Events
     decorated.at = decorateMethod((hash: Hash, arg1?: Arg, arg2?: Arg): Observable<Codec> =>
       this._rpcCore.state.getStorage(getArgs(arg1, arg2), hash));
 
+    decorated.entries = decorateMethod((): Observable<[StorageKey, Codec][]> =>
+      this.retrieveMapEntries(creator));
+
     decorated.hash = decorateMethod((arg1?: Arg, arg2?: Arg): Observable<Hash> =>
       this._rpcCore.state.getStorageHash(getArgs(arg1, arg2)));
 
@@ -288,7 +297,7 @@ export default abstract class Decorate<ApiType extends ApiTypes> extends Events
     return this.decorateFunctionMeta(creator, decorated) as unknown as QueryableStorageEntry<ApiType>;
   }
 
-  private decorateStorageEntryLinked<ApiType extends ApiTypes> (creator: StorageEntry, decorateMethod: DecorateMethod<ApiType>): ReturnType<DecorateMethod<ApiType>> {
+  private decorateStorageLinked<ApiType extends ApiTypes> (creator: StorageEntry, decorateMethod: DecorateMethod<ApiType>): ReturnType<DecorateMethod<ApiType>> {
     const result: Map<Codec, [Codec, Linkage<Codec>]> = new Map();
     let subject: BehaviorSubject<LinkageResult>;
     let head: Codec | null = null;
@@ -329,8 +338,8 @@ export default abstract class Decorate<ApiType extends ApiTypes> extends Events
           }
 
           const nextResult = vals.length
-            ? new LinkageResult([keys[0].constructor as any, keys], [vals[0].constructor as any, vals])
-            : new LinkageResult([Null, []], [Null, []]);
+            ? new LinkageResult(this.registry, [keys[0].constructor as any, keys], [vals[0].constructor as any, vals])
+            : new LinkageResult(this.registry, [Null, []], [Null, []]);
 
           // we set our result into a subject so we have a single observable for
           // which the value changes over time. Initially create, follow-up next
@@ -351,11 +360,51 @@ export default abstract class Decorate<ApiType extends ApiTypes> extends Events
           .subscribeStorage<[[Codec, Linkage<Codec>]]>([[creator, ...args]])
           .pipe(map(([data]): [Codec, Linkage<Codec>] => data))
         : this._rpcCore.state
-          .subscribeStorage<[LinkageResult]>([creator.headKey])
+          .subscribeStorage<[LinkageResult]>([creator.iterKey])
           .pipe(switchMap(([key]): Observable<LinkageResult> => getNext(head = key)))
     );
   }
 
+  private retrieveMapData (creator: StorageEntry): Observable<[StorageKey[], Vec<Codec>]> {
+    assert(creator.meta.type.isMap, 'entries can only be retrieved on maps');
+
+    const outputType = creator.meta.type.asMap.value.toString();
+
+    return this._rpcCore.state
+      // FIXME This should really be some sort of subscription, so we can get stuff as
+      // it changes (as of now it is a one-shot query). Not sure how to do this though...
+      .getKeys(creator.iterKey)
+      .pipe(
+        switchMap((keys): Observable<[StorageKey[], Vec<Codec>]> =>
+          combineLatest([
+            of(keys),
+            // Not 100% sure about this, surely this is just a vec?
+            this._rpcCore.state.subscribeStorage<Vec<Codec>>(
+              keys.map((key) => key.setOutputType(outputType))
+            )
+          ])
+        )
+      );
+  }
+
+  private retrieveMapEntries (creator: StorageEntry): Observable<[StorageKey, Codec][]> {
+    return this.retrieveMapData(creator).pipe(
+      map(([keys, values]): [StorageKey, Codec][] =>
+        keys.map((key, index): [StorageKey, Codec] => [key, values[index]])
+      )
+    );
+  }
+
+  private decorateStoragePrefixed<ApiType extends ApiTypes> (creator: StorageEntry, decorateMethod: DecorateMethod<ApiType>): ReturnType<DecorateMethod<ApiType>> {
+    return decorateMethod((...args: any[]): Observable<Codec> =>
+      args.length
+        ? this._rpcCore.state
+          .subscribeStorage<[Codec]>([[creator, ...args]])
+          .pipe(map(([data]): Codec => data))
+        : this.retrieveMapData(creator).pipe(map(([, values]): Vec<Codec> => values))
+    );
+  }
+
   protected decorateDeriveRx (decorateMethod: DecorateMethod<ApiType>): DeriveAllSections<'rxjs', ExactDerive> {
     // Pull in derive from api-derive
     const derive = decorateDerive(this._rx as ApiInterfaceRx, this._options.derives);

packages/api/src/base/Init.ts

@@ -6,50 +6,17 @@ import { SignedBlock } from '@polkadot/types/interfaces';
 import { RegistryTypes } from '@polkadot/types/types';
 import { ApiBase, ApiOptions, ApiTypes, DecorateMethod } from '../types';
 
-import constantsFromMeta from '@polkadot/api-metadata/consts/fromMetadata';
-import extrinsicsFromMeta from '@polkadot/api-metadata/extrinsics/fromMetadata';
-import storageFromMeta from '@polkadot/api-metadata/storage/fromMetadata';
-import { GenericCall, GenericEvent, Metadata, u32 as U32 } from '@polkadot/types';
+import DecoratedMeta from '@polkadot/metadata/Decorated';
+import { Metadata, u32 as U32 } from '@polkadot/types';
 import { LATEST_EXTRINSIC_VERSION } from '@polkadot/types/primitive/Extrinsic/Extrinsic';
 import { logger } from '@polkadot/util';
 import { cryptoWaitReady, setSS58Format } from '@polkadot/util-crypto';
 import addressDefaults from '@polkadot/util-crypto/address/defaults';
 
 import Decorate from './Decorate';
+import { getChainTypes, getMetadataTypes } from './typeInjector';
 
 const KEEPALIVE_INTERVAL = 15000;
-const DEFAULT_SS58 = new U32(addressDefaults.prefix);
-
-// these are override types for polkadot chains
-// NOTE The SessionKeys definition for Polkadot and Substrate (OpaqueKeys
-// implementation) are different. Detect Polkadot and inject the `Keys`
-// definition as applicable. (3 keys in substrate vs 4 in Polkadot).
-const TYPES_FOR_POLKADOT: Record<string, string> = {
-  Keys: 'SessionKeysPolkadot'
-};
-
-// NOTE this is for support of old, e.g. Alex, old metadata and BlockNumber/Index
-// This is detected based on metadata version, since this is what we have up-front
-const TYPES_SUBSTRATE_1 = {
-  BlockNumber: 'u64',
-  Index: 'u64',
-  EventRecord: 'EventRecord0to76',
-  ValidatorPrefs: 'ValidatorPrefs0to145'
-};
-
-// Type overrides based on specific nodes
-const TYPES_CHAIN: Record<string, Record<string, string>> = {
-  // TODO Remove this once it is not needed, i.e. upgraded
-  'Kusama CC1': {
-    RawBabePreDigest: 'RawBabePreDigest0to159'
-  }
-};
-
-// Type overrides for specific spec types as given in runtimeVersion
-const TYPES_SPEC: Record<string, Record<string, string>> = {
-  kusama: TYPES_FOR_POLKADOT,
-  polkadot: TYPES_FOR_POLKADOT
-};
 
 const l = logger('api/decorator');
 
@@ -119,23 +86,16 @@ export default abstract class Init<ApiType extends ApiTypes> extends Decorate<Ap
   }
 
   private async metaFromChain (optMetadata: Record<string, string>): Promise<Metadata> {
-    const { typesChain = {}, typesSpec = {} } = this._options;
+    const { typesChain, typesSpec } = this._options;
     const [genesisHash, runtimeVersion, chain, chainProps] = await Promise.all([
       this._rpcCore.chain.getBlockHash(0).toPromise(),
       this._rpcCore.state.getRuntimeVersion().toPromise(),
       this._rpcCore.system.chain().toPromise(),
       this._rpcCore.system.properties().toPromise()
     ]);
-    const specName = runtimeVersion.specName.toString();
-    const chainName = chain.toString();
 
     // based on the node spec & chain, inject specific type overrides
-    this.registerTypes({
-      ...(TYPES_SPEC[specName] || {}),
-      ...(TYPES_CHAIN[chainName] || {}),
-      ...(typesSpec[specName] || {}),
-      ...(typesChain[chainName] || {})
-    });
+    this.registerTypes(getChainTypes(chain, runtimeVersion, typesChain, typesSpec));
 
     // filter the RPC methods (this does an rpc-methods call)
     await this.filterRpc();
@@ -143,7 +103,7 @@ export default abstract class Init<ApiType extends ApiTypes> extends Decorate<Ap
     // retrieve metadata, either from chain  or as pass-in via options
     const metadataKey = `${genesisHash}-${runtimeVersion.specVersion}`;
     const metadata = metadataKey in optMetadata
-      ? new Metadata(optMetadata[metadataKey])
+      ? new Metadata(this.registry, optMetadata[metadataKey])
       : await this._rpcCore.state.getMetadata().toPromise();
 
     // set our chain version & genesisHash as returned
@@ -151,7 +111,7 @@ export default abstract class Init<ApiType extends ApiTypes> extends Decorate<Ap
     this._runtimeVersion = runtimeVersion;
 
     // set the global ss58Format as detected by the chain
-    setSS58Format(chainProps.ss58Format.unwrapOr(DEFAULT_SS58).toNumber());
+    setSS58Format(chainProps.ss58Format.unwrapOr(new U32(this.registry, addressDefaults.prefix)).toNumber());
 
     // get unique types & validate
     metadata.getUniqTypes(false);
@@ -160,22 +120,13 @@ export default abstract class Init<ApiType extends ApiTypes> extends Decorate<Ap
   }
 
   private async initFromMeta (metadata: Metadata): Promise<boolean> {
-    // HACK-ish Old EventRecord, BlockNumber & Indexes for e.g. Alex, based on metadata version
-    //   v3 = Alex
-    //   v4 = v1.0 branch
-    if (metadata.version <= 4) {
-      this.registerTypes(TYPES_SUBSTRATE_1);
-    }
+    // inject types based on metadata, if applicable
+    this.registerTypes(getMetadataTypes(metadata.version));
 
-    const extrinsics = extrinsicsFromMeta(metadata);
-    const storage = storageFromMeta(metadata);
-    const constants = constantsFromMeta(metadata);
+    const decoratedMeta = new DecoratedMeta(this.registry, metadata);
 
     // only inject if we are not a clone (global init)
     if (!this._options.source) {
-      GenericEvent.injectMetadata(metadata);
-      GenericCall.injectMetadata(metadata);
-
       // detect the extrinsic version in-use based on the last block
       const { block: { extrinsics: [firstTx] } }: SignedBlock = await this._rpcCore.chain.getBlock().toPromise();
 
@@ -183,16 +134,16 @@ export default abstract class Init<ApiType extends ApiTypes> extends Decorate<Ap
       this._extrinsicType = firstTx ? firstTx.type : LATEST_EXTRINSIC_VERSION;
     }
 
-    this._extrinsics = this.decorateExtrinsics(extrinsics, this.decorateMethod);
-    this._query = this.decorateStorage(storage, this.decorateMethod);
-    this._consts = constants;
+    this._extrinsics = this.decorateExtrinsics(decoratedMeta.tx, this.decorateMethod);
+    this._query = this.decorateStorage(decoratedMeta.query, this.decorateMethod);
+    this._consts = decoratedMeta.consts;
 
     this._rx.extrinsicType = this._extrinsicType;
     this._rx.genesisHash = this._genesisHash;
     this._rx.runtimeVersion = this._runtimeVersion;
-    this._rx.tx = this.decorateExtrinsics(extrinsics, this.rxDecorateMethod);
-    this._rx.query = this.decorateStorage(storage, this.rxDecorateMethod);
-    this._rx.consts = constants;
+    this._rx.tx = this.decorateExtrinsics(decoratedMeta.tx, this.rxDecorateMethod);
+    this._rx.query = this.decorateStorage(decoratedMeta.query, this.rxDecorateMethod);
+    this._rx.consts = decoratedMeta.consts;
 
     // derive is last, since it uses the decorated rx
     this._rx.derive = this.decorateDeriveRx(this.rxDecorateMethod);

packages/api/src/base/index.ts

@@ -2,14 +2,14 @@
 // This software may be modified and distributed under the terms
 // of the Apache-2.0 license. See the LICENSE file for details.
 
+import { Constants } from '@polkadot/metadata/Decorated/types';
 import { RpcInterface } from '@polkadot/rpc-core/jsonrpc.types';
 import { Hash, RuntimeVersion } from '@polkadot/types/interfaces';
 import { InterfaceRegistry } from '@polkadot/types/interfaceRegistry';
 import { CallFunction, InterfaceTypes, RegistryTypes, SignerPayloadRawBase } from '@polkadot/types/types';
 import { ApiInterfaceRx, ApiOptions, ApiTypes, DecoratedRpc, DecorateMethod, QueryableStorage, QueryableStorageMulti, SubmittableExtrinsics, Signer } from '../types';
 
-import { Constants } from '@polkadot/api-metadata/consts/types';
-import { GenericCall, Metadata, createType, getTypeRegistry } from '@polkadot/types';
+import { Metadata, createType } from '@polkadot/types';
 import { assert, isString, isUndefined, u8aToHex, u8aToU8a } from '@polkadot/util';
 
 import Init from './Init';
@@ -30,7 +30,7 @@ try {
 function assertResult<T> (value: T | undefined): T {
   assert(!isUndefined(value), 'Api needs to be initialized before using, listen on \'ready\'');
 
-  return value as T;
+  return value;
 }
 
 export default abstract class ApiBase<ApiType extends ApiTypes> extends Init<ApiType> {
@@ -227,8 +227,8 @@ export default abstract class ApiBase<ApiType extends ApiTypes> extends Init<Api
   /**
    * @description Creates an instance of a type as registered
    */
-  public createType<K extends InterfaceTypes> (type: K, ...params: any[]): InterfaceRegistry[K] {
-    return createType(type, ...params);
+  public createType = <K extends InterfaceTypes> (type: K, ...params: any[]): InterfaceRegistry[K] => {
+    return createType(this.registry, type, ...params);
   }
 
   /**
@@ -242,14 +242,14 @@ export default abstract class ApiBase<ApiType extends ApiTypes> extends Init<Api
    * @description Finds the definition for a specific [[Call]] based on the index supplied
    */
   public findCall (callIndex: Uint8Array | string): CallFunction {
-    return GenericCall.findFunction(u8aToU8a(callIndex));
+    return this.registry.findMetaCall(u8aToU8a(callIndex));
   }
 
   /**
    * @description Register additional user-defined of chain-specific types in the type registry
    */
   public registerTypes (types?: RegistryTypes): void {
-    types && getTypeRegistry().register(types);
+    types && this.registry.register(types);
   }
 
   /**
@@ -265,9 +265,7 @@ export default abstract class ApiBase<ApiType extends ApiTypes> extends Init<Api
   public async sign (signer: KeyringSigner | string, data: SignerPayloadRawBase): Promise<string> {
     // NOTE Do we really want to do this? Or turn it into an observable for rxjs?
     if (isString(signer)) {
-      if (!this._rx.signer || !this._rx.signer.signRaw) {
-        throw new Error('No signer exists with a signRaw interface');
-      }
+      assert(this._rx.signer?.signRaw, 'No signer exists with a signRaw interface');
 
       return (
         await this._rx.signer.signRaw({

packages/api/src/base/typeInjector.ts

@@ -0,0 +1,92 @@
+// Copyright 2017-2019 @polkadot/api authors & contributors
+// This software may be modified and distributed under the terms
+// of the Apache-2.0 license. See the LICENSE file for details.
+
+import { RuntimeVersion } from '@polkadot/types/interfaces';
+import { RegistryTypes } from '@polkadot/types/types';
+
+import { Text } from '@polkadot/types';
+import { isUndefined } from '@polkadot/util';
+
+interface VersionedType {
+  minmax: [number?, number?]; // min (v >= min) and max (v <= max)
+  types: RegistryTypes;
+}
+
+// these are override types for polkadot chains
+// NOTE The SessionKeys definition for Polkadot and Substrate (OpaqueKeys
+// implementation) are different. Detect Polkadot and inject the `Keys`
+// definition as applicable. (4 keys in substrate vs 5 in Polkadot/CC3).
+const TYPES_POLKADOT_VERSIONED: VersionedType[] = [
+  {
+    minmax: [0, 1018], // CC2
+    types: {
+      Keys: 'SessionKeys4',
+      ValidatorPrefs: 'ValidatorPrefsTo196'
+    }
+  },
+  {
+    minmax: [1019, undefined], // CC3
+    types: {
+      Keys: 'SessionKeys5'
+    }
+  }
+];
+
+// Type overrides based on specific nodes
+const TYPES_CHAIN: Record<string, VersionedType[]> = {};
+
+// Type overrides based on  metadata versions
+const TYPES_META: VersionedType[] = [
+  {
+    // NOTE this is for support of old, e.g. Alex, old metadata and BlockNumber/Index
+    // This is detected based on metadata version, since this is what we have up-front
+    //   v3 = Alex
+    //   v4 = v1.0 branch
+    minmax: [0, 4],
+    types: {
+      BlockNumber: 'u64',
+      Index: 'u64',
+      EventRecord: 'EventRecordTo76',
+      ValidatorPrefs: 'ValidatorPrefsTo145'
+    }
+  }
+];
+
+// Type overrides for specific spec types & versions as given in runtimeVersion
+const TYPES_SPEC: Record<string, VersionedType[]> = {
+  kusama: TYPES_POLKADOT_VERSIONED,
+  polkadot: TYPES_POLKADOT_VERSIONED
+};
+
+// flatten a VersionedType[] into a Record<string, string>
+function filterVersions (versions: VersionedType[] = [], version: number): RegistryTypes {
+  return versions
+    .filter(({ minmax: [min, max] }): boolean =>
+      (isUndefined(min) || version >= min) &&
+      (isUndefined(max) || version <= max)
+    )
+    .reduce((result: RegistryTypes, { types }): RegistryTypes => ({
+      ...result,
+      ...types
+    }), {});
+}
+
+// based on the metadata version, return the registry types
+export function getMetadataTypes (version: number): RegistryTypes {
+  return filterVersions(TYPES_META, version);
+}
+
+// based on the chain and runtimeVersion, get the applicable types (ready for registration)
+export function getChainTypes (chainName: Text, { specName, specVersion }: RuntimeVersion, typesChain: Record<string, RegistryTypes> = {}, typesSpec: Record<string, RegistryTypes> = {}): RegistryTypes {
+  const _chainName = chainName.toString();
+  const _specName = specName.toString();
+  const _specVersion = specVersion.toNumber();
+
+  return {
+    ...filterVersions(TYPES_SPEC[_specName], _specVersion),
+    ...filterVersions(TYPES_CHAIN[_chainName], _specVersion),
+    ...(typesSpec[_specName] || {}),
+    ...(typesChain[_chainName] || {})
+  };
+}

packages/api/src/checkTypes.manual.ts

@@ -4,22 +4,24 @@
 
 // Simple non-runnable checks to test type definitions in the editor itself
 
-import { ConstantCodec } from '@polkadot/api-metadata/consts/types';
 import { Balance, Header, Index } from '@polkadot/types/interfaces';
 import { IExtrinsic, IMethod } from '@polkadot/types/types';
 
 import { ApiPromise } from '@polkadot/api';
 import { HeaderExtended } from '@polkadot/api-derive';
 import testKeyring, { TestKeyringMap } from '@polkadot/keyring/testingPairs';
-import { createType, createTypeUnsafe } from '@polkadot/types/codec';
+import { createType, createTypeUnsafe, TypeRegistry } from '@polkadot/types/codec';
 
 import { SubmittableResult } from './';
 
+const registry = new TypeRegistry();
+
 function consts (api: ApiPromise): void {
   // constants has actual value & metadata
   console.log(
-    api.consts.balances.creationFee.toHex(),
-    (api.consts.balances.creationFee as ConstantCodec).meta.documentation.map((s): string => s.toString()).join('')
+    api.consts.foo.bar,
+    api.consts.balances.creationFee.toNumber(),
+    api.consts.balances.creationFee.meta.documentation.map((s): string => s.toString()).join('')
   );
 }
 
@@ -70,13 +72,13 @@ async function rpc (api: ApiPromise): Promise<void> {
 
 function types (): void {
   // check correct types with `createType`
-  const balance = createType('Balance', 2);
-  const gas = createType('Gas', 2);
-  const compact = createType('Compact<u32>', 2);
-  // const random = createType('RandomType', 2); // This one should deliberately show a TS error
+  const balance = createType(registry, 'Balance', 2);
+  const gas = createType(registry, 'Gas', 2);
+  const compact = createType(registry, 'Compact<u32>', 2);
+  // const random = createType(registry, 'RandomType', 2); // This one should deliberately show a TS error
 
-  const gasUnsafe = createTypeUnsafe('Gas', [2]);
-  const overriddenUnsafe = createTypeUnsafe<Header>('Gas', [2]);
+  const gasUnsafe = createTypeUnsafe(registry, 'Gas', [2]);
+  const overriddenUnsafe = createTypeUnsafe<Header>(registry, 'Gas', [2]);
 
   console.log(balance, gas, compact, gasUnsafe, overriddenUnsafe);
 }

packages/api/src/consts.types.ts

@@ -0,0 +1,97 @@
+// Auto-generated via `yarn build:interfaces`, do not edit
+/* eslint-disable @typescript-eslint/no-empty-interface */
+
+import { Bytes, u32, u64 } from '@polkadot/types';
+import { Balance, BalanceOf, BlockNumber, Moment, Permill } from '@polkadot/types/interfaces/runtime';
+import { Gas } from '@polkadot/types/interfaces/contracts';
+import { SessionIndex } from '@polkadot/types/interfaces/session';
+import { EraIndex } from '@polkadot/types/interfaces/staking';
+import { Codec } from '@polkadot/types/types';
+
+declare module '@polkadot/metadata/Decorated/types' {
+  export interface Constants {
+    babe: {
+      [index: string]: Codec;
+      epochDuration: u64 & ConstantCodec;
+      expectedBlockTime: Moment & ConstantCodec;
+    };
+    timestamp: {
+      [index: string]: Codec;
+      minimumPeriod: Moment & ConstantCodec;
+    };
+    balances: {
+      [index: string]: Codec;
+      existentialDeposit: Balance & ConstantCodec;
+      transferFee: Balance & ConstantCodec;
+      creationFee: Balance & ConstantCodec;
+    };
+    transactionPayment: {
+      [index: string]: Codec;
+      transactionBaseFee: BalanceOf & ConstantCodec;
+      transactionByteFee: BalanceOf & ConstantCodec;
+    };
+    staking: {
+      [index: string]: Codec;
+      sessionsPerEra: SessionIndex & ConstantCodec;
+      bondingDuration: EraIndex & ConstantCodec;
+    };
+    session: {
+      [index: string]: Codec;
+      dedupKeyPrefix: Bytes & ConstantCodec;
+    };
+    democracy: {
+      [index: string]: Codec;
+      enactmentPeriod: BlockNumber & ConstantCodec;
+      launchPeriod: BlockNumber & ConstantCodec;
+      votingPeriod: BlockNumber & ConstantCodec;
+      minimumDeposit: BalanceOf & ConstantCodec;
+      emergencyVotingPeriod: BlockNumber & ConstantCodec;
+      cooloffPeriod: BlockNumber & ConstantCodec;
+    };
+    elections: {
+      [index: string]: Codec;
+      candidacyBond: BalanceOf & ConstantCodec;
+      votingBond: BalanceOf & ConstantCodec;
+      desiredMembers: u32 & ConstantCodec;
+      desiredRunnersUp: u32 & ConstantCodec;
+      termDuration: BlockNumber & ConstantCodec;
+    };
+    finalityTracker: {
+      [index: string]: Codec;
+      windowSize: BlockNumber & ConstantCodec;
+      reportLatency: BlockNumber & ConstantCodec;
+    };
+    treasury: {
+      [index: string]: Codec;
+      proposalBond: Permill & ConstantCodec;
+      proposalBondMinimum: BalanceOf & ConstantCodec;
+      spendPeriod: BlockNumber & ConstantCodec;
+      burn: Permill & ConstantCodec;
+    };
+    contracts: {
+      [index: string]: Codec;
+      signedClaimHandicap: BlockNumber & ConstantCodec;
+      tombstoneDeposit: BalanceOf & ConstantCodec;
+      storageSizeOffset: u32 & ConstantCodec;
+      rentByteFee: BalanceOf & ConstantCodec;
+      rentDepositOffset: BalanceOf & ConstantCodec;
+      surchargeReward: BalanceOf & ConstantCodec;
+      transferFee: BalanceOf & ConstantCodec;
+      creationFee: BalanceOf & ConstantCodec;
+      transactionBaseFee: BalanceOf & ConstantCodec;
+      transactionByteFee: BalanceOf & ConstantCodec;
+      contractFee: BalanceOf & ConstantCodec;
+      callBaseFee: Gas & ConstantCodec;
+      instantiateBaseFee: Gas & ConstantCodec;
+      maxDepth: u32 & ConstantCodec;
+      maxValueSize: u32 & ConstantCodec;
+      blockGasLimit: Gas & ConstantCodec;
+    };
+    nicks: {
+      [index: string]: Codec;
+      reservationFee: BalanceOf & ConstantCodec;
+      minLength: u32 & ConstantCodec;
+      maxLength: u32 & ConstantCodec;
+    };
+  }
+}

packages/api/src/promise/Api.spec.ts

@@ -4,30 +4,31 @@
 
 import testingPairs from '@polkadot/keyring/testingPairs';
 import Mock from '@polkadot/rpc-provider/mock/index';
-import { createType } from '@polkadot/types';
+import { createType, TypeRegistry } from '@polkadot/types';
 
 import { ApiPromise } from '..';
 import { SingleAccountSigner } from '../../test/util';
 import { ApiOptions } from '../types';
 
 describe('ApiPromise', (): void => {
+  const registry = new TypeRegistry();
   const keyring = testingPairs({ type: 'ed25519' });
   let provider: Mock;
 
   beforeEach((): void => {
     jest.setTimeout(3000000);
-    provider = new Mock();
+    provider = new Mock(registry);
   });
 
   describe('initialization', (): void => {
     it('Create API instance with metadata map and makes the runtime, rpc, state & extrinsics available', async (): Promise<void> => {
       const rpcData = await provider.send('state_getMetadata', []);
-      const genesisHash = createType('Hash', await provider.send('chain_getBlockHash', [])).toHex();
+      const genesisHash = createType(registry, 'Hash', await provider.send('chain_getBlockHash', [])).toHex();
       const specVersion = 0;
       const metadata: any = {};
       const key = `${genesisHash}-${specVersion}`;
       metadata[key] = rpcData;
-      const api = await ApiPromise.create({ provider, metadata } as unknown as ApiOptions);
+      const api = await ApiPromise.create({ provider, metadata, registry } as ApiOptions);
 
       expect(api.genesisHash).toBeDefined();
       expect(api.runtimeMetadata).toBeDefined();
@@ -40,7 +41,7 @@ describe('ApiPromise', (): void => {
 
     it('Create API instance without metadata and makes the runtime, rpc, state & extrinsics available', async (): Promise<void> => {
       const metadata = {};
-      const api = await ApiPromise.create({ provider, metadata });
+      const api = await ApiPromise.create({ provider, metadata, registry });
 
       expect(api.genesisHash).toBeDefined();
       expect(api.runtimeMetadata).toBeDefined();
@@ -58,7 +59,7 @@ describe('ApiPromise', (): void => {
     const SIG = '0x659effefbbe5ab4d7136ebb5084b959eb424e32b862307371be4721ac2c46334245af4f1476c36c5e5aff04396c2fdd2ce561ec90382821d4aa071b559b1db0f';
 
     it('signs data using a specified keyring', async (): Promise<void> => {
-      const api = new ApiPromise({ provider });
+      const api = new ApiPromise({ provider, registry });
 
       expect(
         await api.sign(keyring.alice_session, TEST)
@@ -68,7 +69,8 @@ describe('ApiPromise', (): void => {
     it('signs data using an external signer', async (): Promise<void> => {
       const api = new ApiPromise({
         provider,
-        signer: new SingleAccountSigner(keyring.alice_session)
+        registry,
+        signer: new SingleAccountSigner(registry, keyring.alice_session)
       });
 
       expect(
@@ -76,4 +78,16 @@ describe('ApiPromise', (): void => {
       ).toEqual(SIG);
     });
   });
+
+  describe('decorator.signAsync', (): void => {
+    it('signs a transfer using an external signer', async (): Promise<void> => {
+      const api = await ApiPromise.create({ provider, registry });
+      api.setSigner(new SingleAccountSigner(registry, keyring.alice_session));
+      const transfer = api.tx.balances.transfer(keyring.eve.address, 12345);
+      const dummySignature = '0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000';
+      expect(transfer.signature.toHex()).toEqual(dummySignature);
+      await transfer.signAsync(keyring.alice_session, {});
+      expect(transfer.signature.toHex()).not.toEqual(dummySignature);
+    });
+  });
 });

packages/api/src/query.types.ts

@@ -3,8 +3,8 @@
 
 import { Observable } from 'rxjs';
 import { Option, U8a, Vec } from '@polkadot/types/codec';
-import { Bytes, Null, bool, i8, u32, u64 } from '@polkadot/types';
-import { AccountId, AccountIndex, Balance, BalanceOf, BlockNumber, Hash, Index, KeyTypeId, Moment, Perbill, ValidatorId, Weight, WeightMultiplier } from '@polkadot/types/interfaces/runtime';
+import { Bytes, Null, bool, u32, u64 } from '@polkadot/types';
+import { AccountId, AccountIndex, Balance, BalanceOf, BlockNumber, Hash, Index, KeyTypeId, Moment, Perbill, ValidatorId, Weight } from '@polkadot/types/interfaces/runtime';
 import { UncleEntryItem } from '@polkadot/types/interfaces/authorship';
 import { BabeAuthorityWeight, MaybeVrf } from '@polkadot/types/interfaces/babe';
 import { BalanceLock, VestingSchedule } from '@polkadot/types/interfaces/balances';
@@ -12,14 +12,15 @@ import { ProposalIndex, Votes } from '@polkadot/types/interfaces/collective';
 import { AuthorityId } from '@polkadot/types/interfaces/consensus';
 import { CodeHash, ContractInfo, Gas, PrefabWasmModule, Schedule } from '@polkadot/types/interfaces/contracts';
 import { Conviction, PropIndex, Proposal, ReferendumIndex, ReferendumInfo } from '@polkadot/types/interfaces/democracy';
-import { ApprovalFlag, SetIndex, Vote, VoteIndex, VoteThreshold, VoterInfo } from '@polkadot/types/interfaces/elections';
-import { AuthorityWeight, SetId, StoredPendingChange, StoredState } from '@polkadot/types/interfaces/grandpa';
+import { Vote, VoteThreshold } from '@polkadot/types/interfaces/elections';
+import { AuthorityList, SetId, StoredPendingChange, StoredState } from '@polkadot/types/interfaces/grandpa';
 import { AuthIndex } from '@polkadot/types/interfaces/imOnline';
 import { Kind, OffenceDetails, OpaqueTimeSlot, ReportIdOf } from '@polkadot/types/interfaces/offences';
 import { Keys, SessionIndex } from '@polkadot/types/interfaces/session';
-import { EraIndex, EraPoints, Exposure, Forcing, MomentOf, RewardDestination, SlashJournalEntry, StakingLedger, ValidatorPrefs } from '@polkadot/types/interfaces/staking';
+import { EraIndex, EraPoints, Exposure, Forcing, MomentOf, Nominations, RewardDestination, SlashingSpans, SpanIndex, SpanRecord, StakingLedger, UnappliedSlash, ValidatorPrefs } from '@polkadot/types/interfaces/staking';
 import { DigestOf, EventIndex, EventRecord } from '@polkadot/types/interfaces/system';
 import { TreasuryProposal } from '@polkadot/types/interfaces/treasury';
+import { Multiplier } from '@polkadot/types/interfaces/txpayment';
 import { ITuple } from '@polkadot/types/types';
 
 declare module './types' {
@@ -27,13 +28,11 @@ declare module './types' {
     system: {
       [index: string]: QueryableStorageEntry<ApiType>;
       accountNonce: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<Index>> & QueryableStorageEntry<ApiType>;
-      extrinsicCount: StorageEntryExact<ApiType, () => Observable<u32>> & QueryableStorageEntry<ApiType>;
-      allExtrinsicsWeight: StorageEntryExact<ApiType, () => Observable<Weight>> & QueryableStorageEntry<ApiType>;
-      allExtrinsicsLen: StorageEntryExact<ApiType, () => Observable<u32>> & QueryableStorageEntry<ApiType>;
-      nextWeightMultiplier: StorageEntryExact<ApiType, () => Observable<WeightMultiplier>> & QueryableStorageEntry<ApiType>;
+      extrinsicCount: StorageEntryExact<ApiType, () => Observable<Option<u32>>> & QueryableStorageEntry<ApiType>;
+      allExtrinsicsWeight: StorageEntryExact<ApiType, () => Observable<Option<Weight>>> & QueryableStorageEntry<ApiType>;
+      allExtrinsicsLen: StorageEntryExact<ApiType, () => Observable<Option<u32>>> & QueryableStorageEntry<ApiType>;
       blockHash: StorageEntryExact<ApiType, (arg: BlockNumber | Uint8Array | number | string) => Observable<Hash>> & QueryableStorageEntry<ApiType>;
       extrinsicData: StorageEntryExact<ApiType, (arg: u32 | Uint8Array | number | string) => Observable<Bytes>> & QueryableStorageEntry<ApiType>;
-      randomMaterial: StorageEntryExact<ApiType, () => Observable<ITuple<[i8, Vec<Hash>]>>> & QueryableStorageEntry<ApiType>;
       number: StorageEntryExact<ApiType, () => Observable<BlockNumber>> & QueryableStorageEntry<ApiType>;
       parentHash: StorageEntryExact<ApiType, () => Observable<Hash>> & QueryableStorageEntry<ApiType>;
       extrinsicsRoot: StorageEntryExact<ApiType, () => Observable<Hash>> & QueryableStorageEntry<ApiType>;
@@ -52,7 +51,7 @@ declare module './types' {
       nextRandomness: StorageEntryExact<ApiType, () => Observable<U8a>> & QueryableStorageEntry<ApiType>;
       segmentIndex: StorageEntryExact<ApiType, () => Observable<u32>> & QueryableStorageEntry<ApiType>;
       underConstruction: StorageEntryExact<ApiType, (arg: u32 | Uint8Array | number | string) => Observable<Vec<U8a>>> & QueryableStorageEntry<ApiType>;
-      initialized: StorageEntryExact<ApiType, () => Observable<MaybeVrf>> & QueryableStorageEntry<ApiType>;
+      initialized: StorageEntryExact<ApiType, () => Observable<Option<MaybeVrf>>> & QueryableStorageEntry<ApiType>;
     };
     timestamp: {
       [index: string]: QueryableStorageEntry<ApiType>;
@@ -62,7 +61,7 @@ declare module './types' {
     authorship: {
       [index: string]: QueryableStorageEntry<ApiType>;
       uncles: StorageEntryExact<ApiType, () => Observable<Vec<UncleEntryItem>>> & QueryableStorageEntry<ApiType>;
-      author: StorageEntryExact<ApiType, () => Observable<AccountId>> & QueryableStorageEntry<ApiType>;
+      author: StorageEntryExact<ApiType, () => Observable<Option<AccountId>>> & QueryableStorageEntry<ApiType>;
       didSetUncles: StorageEntryExact<ApiType, () => Observable<bool>> & QueryableStorageEntry<ApiType>;
     };
     indices: {
@@ -73,21 +72,25 @@ declare module './types' {
     balances: {
       [index: string]: QueryableStorageEntry<ApiType>;
       totalIssuance: StorageEntryExact<ApiType, () => Observable<Balance>> & QueryableStorageEntry<ApiType>;
-      vesting: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<VestingSchedule>> & QueryableStorageEntry<ApiType>;
+      vesting: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<Option<VestingSchedule>>> & QueryableStorageEntry<ApiType>;
       freeBalance: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<Balance>> & QueryableStorageEntry<ApiType>;
       reservedBalance: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<Balance>> & QueryableStorageEntry<ApiType>;
       locks: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<Vec<BalanceLock>>> & QueryableStorageEntry<ApiType>;
     };
+    transactionPayment: {
+      [index: string]: QueryableStorageEntry<ApiType>;
+      nextFeeMultiplier: StorageEntryExact<ApiType, () => Observable<Multiplier>> & QueryableStorageEntry<ApiType>;
+    };
     staking: {
       [index: string]: QueryableStorageEntry<ApiType>;
       validatorCount: StorageEntryExact<ApiType, () => Observable<u32>> & QueryableStorageEntry<ApiType>;
       minimumValidatorCount: StorageEntryExact<ApiType, () => Observable<u32>> & QueryableStorageEntry<ApiType>;
       invulnerables: StorageEntryExact<ApiType, () => Observable<Vec<AccountId>>> & QueryableStorageEntry<ApiType>;
-      bonded: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<AccountId>> & QueryableStorageEntry<ApiType>;
-      ledger: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<StakingLedger>> & QueryableStorageEntry<ApiType>;
+      bonded: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<Option<AccountId>>> & QueryableStorageEntry<ApiType>;
+      ledger: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<Option<StakingLedger>>> & QueryableStorageEntry<ApiType>;
       payee: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<RewardDestination>> & QueryableStorageEntry<ApiType>;
       validators: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<ValidatorPrefs>> & QueryableStorageEntry<ApiType>;
-      nominators: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<Vec<AccountId>>> & QueryableStorageEntry<ApiType>;
+      nominators: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<Option<Nominations>>> & QueryableStorageEntry<ApiType>;
       stakers: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<Exposure>> & QueryableStorageEntry<ApiType>;
       currentElected: StorageEntryExact<ApiType, () => Observable<Vec<AccountId>>> & QueryableStorageEntry<ApiType>;
       currentEra: StorageEntryExact<ApiType, () => Observable<EraIndex>> & QueryableStorageEntry<ApiType>;
@@ -97,8 +100,15 @@ declare module './types' {
       slotStake: StorageEntryExact<ApiType, () => Observable<BalanceOf>> & QueryableStorageEntry<ApiType>;
       forceEra: StorageEntryExact<ApiType, () => Observable<Forcing>> & QueryableStorageEntry<ApiType>;
       slashRewardFraction: StorageEntryExact<ApiType, () => Observable<Perbill>> & QueryableStorageEntry<ApiType>;
+      canceledSlashPayout: StorageEntryExact<ApiType, () => Observable<BalanceOf>> & QueryableStorageEntry<ApiType>;
+      unappliedSlashes: StorageEntryExact<ApiType, (arg: EraIndex | Uint8Array | number | string) => Observable<Vec<UnappliedSlash>>> & QueryableStorageEntry<ApiType>;
       bondedEras: StorageEntryExact<ApiType, () => Observable<Vec<ITuple<[EraIndex, SessionIndex]>>>> & QueryableStorageEntry<ApiType>;
-      eraSlashJournal: StorageEntryExact<ApiType, (arg: EraIndex | Uint8Array | number | string) => Observable<Vec<SlashJournalEntry>>> & QueryableStorageEntry<ApiType>;
+      validatorSlashInEra: StorageEntryExact<ApiType, (key1: EraIndex | Uint8Array | number | string, key2: AccountId | Uint8Array | string) => Observable<Option<ITuple<[Perbill, BalanceOf]>>>> & QueryableStorageEntry<ApiType>;
+      nominatorSlashInEra: StorageEntryExact<ApiType, (key1: EraIndex | Uint8Array | number | string, key2: AccountId | Uint8Array | string) => Observable<Option<BalanceOf>>> & QueryableStorageEntry<ApiType>;
+      slashingSpans: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<Option<SlashingSpans>>> & QueryableStorageEntry<ApiType>;
+      spanSlash: StorageEntryExact<ApiType, (arg: ITuple<[AccountId, SpanIndex]>) => Observable<SpanRecord>> & QueryableStorageEntry<ApiType>;
+      earliestUnappliedSlash: StorageEntryExact<ApiType, () => Observable<Option<EraIndex>>> & QueryableStorageEntry<ApiType>;
+      storageVersion: StorageEntryExact<ApiType, () => Observable<u32>> & QueryableStorageEntry<ApiType>;
     };
     session: {
       [index: string]: QueryableStorageEntry<ApiType>;
@@ -107,61 +117,52 @@ declare module './types' {
       queuedChanged: StorageEntryExact<ApiType, () => Observable<bool>> & QueryableStorageEntry<ApiType>;
       queuedKeys: StorageEntryExact<ApiType, () => Observable<Vec<ITuple<[ValidatorId, Keys]>>>> & QueryableStorageEntry<ApiType>;
       disabledValidators: StorageEntryExact<ApiType, () => Observable<Vec<u32>>> & QueryableStorageEntry<ApiType>;
-      nextKeys: StorageEntryExact<ApiType, (key1: Bytes | Uint8Array | string, key2: ValidatorId | Uint8Array | string) => Observable<Keys>> & QueryableStorageEntry<ApiType>;
-      keyOwner: StorageEntryExact<ApiType, (key1: Bytes | Uint8Array | string, key2: ITuple<[KeyTypeId, Bytes]>) => Observable<ValidatorId>> & QueryableStorageEntry<ApiType>;
+      nextKeys: StorageEntryExact<ApiType, (key1: Bytes | Uint8Array | string, key2: ValidatorId | Uint8Array | string) => Observable<Option<Keys>>> & QueryableStorageEntry<ApiType>;
+      keyOwner: StorageEntryExact<ApiType, (key1: Bytes | Uint8Array | string, key2: ITuple<[KeyTypeId, Bytes]>) => Observable<Option<ValidatorId>>> & QueryableStorageEntry<ApiType>;
     };
     democracy: {
       [index: string]: QueryableStorageEntry<ApiType>;
       publicPropCount: StorageEntryExact<ApiType, () => Observable<PropIndex>> & QueryableStorageEntry<ApiType>;
       publicProps: StorageEntryExact<ApiType, () => Observable<Vec<ITuple<[PropIndex, Proposal, AccountId]>>>> & QueryableStorageEntry<ApiType>;
-      depositOf: StorageEntryExact<ApiType, (arg: PropIndex | Uint8Array | number | string) => Observable<ITuple<[BalanceOf, Vec<AccountId>]>>> & QueryableStorageEntry<ApiType>;
+      depositOf: StorageEntryExact<ApiType, (arg: PropIndex | Uint8Array | number | string) => Observable<Option<ITuple<[BalanceOf, Vec<AccountId>]>>>> & QueryableStorageEntry<ApiType>;
       referendumCount: StorageEntryExact<ApiType, () => Observable<ReferendumIndex>> & QueryableStorageEntry<ApiType>;
       nextTally: StorageEntryExact<ApiType, () => Observable<ReferendumIndex>> & QueryableStorageEntry<ApiType>;
-      referendumInfoOf: StorageEntryExact<ApiType, (arg: ReferendumIndex | Uint8Array | number | string) => Observable<ReferendumInfo>> & QueryableStorageEntry<ApiType>;
+      referendumInfoOf: StorageEntryExact<ApiType, (arg: ReferendumIndex | Uint8Array | number | string) => Observable<Option<ReferendumInfo>>> & QueryableStorageEntry<ApiType>;
       dispatchQueue: StorageEntryExact<ApiType, (arg: BlockNumber | Uint8Array | number | string) => Observable<Vec<Option<ITuple<[Proposal, ReferendumIndex]>>>>> & QueryableStorageEntry<ApiType>;
       votersFor: StorageEntryExact<ApiType, (arg: ReferendumIndex | Uint8Array | number | string) => Observable<Vec<AccountId>>> & QueryableStorageEntry<ApiType>;
       voteOf: StorageEntryExact<ApiType, (arg: ITuple<[ReferendumIndex, AccountId]>) => Observable<Vote>> & QueryableStorageEntry<ApiType>;
-      proxy: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<AccountId>> & QueryableStorageEntry<ApiType>;
+      proxy: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<Option<AccountId>>> & QueryableStorageEntry<ApiType>;
       delegations: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<ITuple<[AccountId, Conviction]>>> & QueryableStorageEntry<ApiType>;
       lastTabledWasExternal: StorageEntryExact<ApiType, () => Observable<bool>> & QueryableStorageEntry<ApiType>;
-      nextExternal: StorageEntryExact<ApiType, () => Observable<ITuple<[Proposal, VoteThreshold]>>> & QueryableStorageEntry<ApiType>;
-      blacklist: StorageEntryExact<ApiType, (arg: Hash | Uint8Array | string) => Observable<ITuple<[BlockNumber, Vec<AccountId>]>>> & QueryableStorageEntry<ApiType>;
+      nextExternal: StorageEntryExact<ApiType, () => Observable<Option<ITuple<[Proposal, VoteThreshold]>>>> & QueryableStorageEntry<ApiType>;
+      blacklist: StorageEntryExact<ApiType, (arg: Hash | Uint8Array | string) => Observable<Option<ITuple<[BlockNumber, Vec<AccountId>]>>>> & QueryableStorageEntry<ApiType>;
       cancellations: StorageEntryExact<ApiType, (arg: Hash | Uint8Array | string) => Observable<bool>> & QueryableStorageEntry<ApiType>;
     };
     council: {
       [index: string]: QueryableStorageEntry<ApiType>;
       proposals: StorageEntryExact<ApiType, () => Observable<Vec<Hash>>> & QueryableStorageEntry<ApiType>;
-      proposalOf: StorageEntryExact<ApiType, (arg: Hash | Uint8Array | string) => Observable<Proposal>> & QueryableStorageEntry<ApiType>;
-      voting: StorageEntryExact<ApiType, (arg: Hash | Uint8Array | string) => Observable<Votes>> & QueryableStorageEntry<ApiType>;
+      proposalOf: StorageEntryExact<ApiType, (arg: Hash | Uint8Array | string) => Observable<Option<Proposal>>> & QueryableStorageEntry<ApiType>;
+      voting: StorageEntryExact<ApiType, (arg: Hash | Uint8Array | string) => Observable<Option<Votes>>> & QueryableStorageEntry<ApiType>;
       proposalCount: StorageEntryExact<ApiType, () => Observable<u32>> & QueryableStorageEntry<ApiType>;
       members: StorageEntryExact<ApiType, () => Observable<Vec<AccountId>>> & QueryableStorageEntry<ApiType>;
     };
     technicalCommittee: {
       [index: string]: QueryableStorageEntry<ApiType>;
       proposals: StorageEntryExact<ApiType, () => Observable<Vec<Hash>>> & QueryableStorageEntry<ApiType>;
-      proposalOf: StorageEntryExact<ApiType, (arg: Hash | Uint8Array | string) => Observable<Proposal>> & QueryableStorageEntry<ApiType>;
-      voting: StorageEntryExact<ApiType, (arg: Hash | Uint8Array | string) => Observable<Votes>> & QueryableStorageEntry<ApiType>;
+      proposalOf: StorageEntryExact<ApiType, (arg: Hash | Uint8Array | string) => Observable<Option<Proposal>>> & QueryableStorageEntry<ApiType>;
+      voting: StorageEntryExact<ApiType, (arg: Hash | Uint8Array | string) => Observable<Option<Votes>>> & QueryableStorageEntry<ApiType>;
       proposalCount: StorageEntryExact<ApiType, () => Observable<u32>> & QueryableStorageEntry<ApiType>;
       members: StorageEntryExact<ApiType, () => Observable<Vec<AccountId>>> & QueryableStorageEntry<ApiType>;
     };
     elections: {
       [index: string]: QueryableStorageEntry<ApiType>;
-      presentationDuration: StorageEntryExact<ApiType, () => Observable<BlockNumber>> & QueryableStorageEntry<ApiType>;
-      termDuration: StorageEntryExact<ApiType, () => Observable<BlockNumber>> & QueryableStorageEntry<ApiType>;
-      desiredSeats: StorageEntryExact<ApiType, () => Observable<u32>> & QueryableStorageEntry<ApiType>;
-      members: StorageEntryExact<ApiType, () => Observable<Vec<ITuple<[AccountId, BlockNumber]>>>> & QueryableStorageEntry<ApiType>;
-      voteCount: StorageEntryExact<ApiType, () => Observable<VoteIndex>> & QueryableStorageEntry<ApiType>;
-      approvalsOf: StorageEntryExact<ApiType, (arg: ITuple<[AccountId, SetIndex]>) => Observable<Vec<ApprovalFlag>>> & QueryableStorageEntry<ApiType>;
-      registerInfoOf: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<ITuple<[VoteIndex, u32]>>> & QueryableStorageEntry<ApiType>;
-      voterInfoOf: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<VoterInfo>> & QueryableStorageEntry<ApiType>;
-      voters: StorageEntryExact<ApiType, (arg: SetIndex | Uint8Array | number | string) => Observable<Vec<Option<AccountId>>>> & QueryableStorageEntry<ApiType>;
-      nextVoterSet: StorageEntryExact<ApiType, () => Observable<SetIndex>> & QueryableStorageEntry<ApiType>;
-      voterCount: StorageEntryExact<ApiType, () => Observable<SetIndex>> & QueryableStorageEntry<ApiType>;
+      members: StorageEntryExact<ApiType, () => Observable<Vec<ITuple<[AccountId, BalanceOf]>>>> & QueryableStorageEntry<ApiType>;
+      runnersUp: StorageEntryExact<ApiType, () => Observable<Vec<ITuple<[AccountId, BalanceOf]>>>> & QueryableStorageEntry<ApiType>;
+      electionRounds: StorageEntryExact<ApiType, () => Observable<u32>> & QueryableStorageEntry<ApiType>;
+      votesOf: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<Vec<AccountId>>> & QueryableStorageEntry<ApiType>;
+      stakeOf: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<BalanceOf>> & QueryableStorageEntry<ApiType>;
       candidates: StorageEntryExact<ApiType, () => Observable<Vec<AccountId>>> & QueryableStorageEntry<ApiType>;
-      candidateCount: StorageEntryExact<ApiType, () => Observable<u32>> & QueryableStorageEntry<ApiType>;
-      nextFinalize: StorageEntryExact<ApiType, () => Observable<ITuple<[BlockNumber, u32, Vec<AccountId>]>>> & QueryableStorageEntry<ApiType>;
-      leaderboard: StorageEntryExact<ApiType, () => Observable<Vec<ITuple<[BalanceOf, AccountId]>>>> & QueryableStorageEntry<ApiType>;
-      proxy: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<AccountId>> & QueryableStorageEntry<ApiType>;
+      didMigrate: StorageEntryExact<ApiType, () => Observable<bool>> & QueryableStorageEntry<ApiType>;
     };
     technicalMembership: {
       [index: string]: QueryableStorageEntry<ApiType>;
@@ -169,28 +170,28 @@ declare module './types' {
     };
     grandpa: {
       [index: string]: QueryableStorageEntry<ApiType>;
-      authorities: StorageEntryExact<ApiType, () => Observable<Vec<ITuple<[AuthorityId, AuthorityWeight]>>>> & QueryableStorageEntry<ApiType>;
+      authorities: StorageEntryExact<ApiType, () => Observable<AuthorityList>> & QueryableStorageEntry<ApiType>;
       state: StorageEntryExact<ApiType, () => Observable<StoredState>> & QueryableStorageEntry<ApiType>;
-      pendingChange: StorageEntryExact<ApiType, () => Observable<StoredPendingChange>> & QueryableStorageEntry<ApiType>;
-      nextForced: StorageEntryExact<ApiType, () => Observable<BlockNumber>> & QueryableStorageEntry<ApiType>;
-      stalled: StorageEntryExact<ApiType, () => Observable<ITuple<[BlockNumber, BlockNumber]>>> & QueryableStorageEntry<ApiType>;
+      pendingChange: StorageEntryExact<ApiType, () => Observable<Option<StoredPendingChange>>> & QueryableStorageEntry<ApiType>;
+      nextForced: StorageEntryExact<ApiType, () => Observable<Option<BlockNumber>>> & QueryableStorageEntry<ApiType>;
+      stalled: StorageEntryExact<ApiType, () => Observable<Option<ITuple<[BlockNumber, BlockNumber]>>>> & QueryableStorageEntry<ApiType>;
       currentSetId: StorageEntryExact<ApiType, () => Observable<SetId>> & QueryableStorageEntry<ApiType>;
-      setIdSession: StorageEntryExact<ApiType, (arg: SetId | Uint8Array | number | string) => Observable<SessionIndex>> & QueryableStorageEntry<ApiType>;
+      setIdSession: StorageEntryExact<ApiType, (arg: SetId | Uint8Array | number | string) => Observable<Option<SessionIndex>>> & QueryableStorageEntry<ApiType>;
     };
     treasury: {
       [index: string]: QueryableStorageEntry<ApiType>;
       proposalCount: StorageEntryExact<ApiType, () => Observable<ProposalIndex>> & QueryableStorageEntry<ApiType>;
-      proposals: StorageEntryExact<ApiType, (arg: ProposalIndex | Uint8Array | number | string) => Observable<TreasuryProposal>> & QueryableStorageEntry<ApiType>;
+      proposals: StorageEntryExact<ApiType, (arg: ProposalIndex | Uint8Array | number | string) => Observable<Option<TreasuryProposal>>> & QueryableStorageEntry<ApiType>;
       approvals: StorageEntryExact<ApiType, () => Observable<Vec<ProposalIndex>>> & QueryableStorageEntry<ApiType>;
     };
     contracts: {
       [index: string]: QueryableStorageEntry<ApiType>;
       gasSpent: StorageEntryExact<ApiType, () => Observable<Gas>> & QueryableStorageEntry<ApiType>;
       currentSchedule: StorageEntryExact<ApiType, () => Observable<Schedule>> & QueryableStorageEntry<ApiType>;
-      pristineCode: StorageEntryExact<ApiType, (arg: CodeHash | Uint8Array | string) => Observable<Bytes>> & QueryableStorageEntry<ApiType>;
-      codeStorage: StorageEntryExact<ApiType, (arg: CodeHash | Uint8Array | string) => Observable<PrefabWasmModule>> & QueryableStorageEntry<ApiType>;
+      pristineCode: StorageEntryExact<ApiType, (arg: CodeHash | Uint8Array | string) => Observable<Option<Bytes>>> & QueryableStorageEntry<ApiType>;
+      codeStorage: StorageEntryExact<ApiType, (arg: CodeHash | Uint8Array | string) => Observable<Option<PrefabWasmModule>>> & QueryableStorageEntry<ApiType>;
       accountCounter: StorageEntryExact<ApiType, () => Observable<u64>> & QueryableStorageEntry<ApiType>;
-      contractInfoOf: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<ContractInfo>> & QueryableStorageEntry<ApiType>;
+      contractInfoOf: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<Option<ContractInfo>>> & QueryableStorageEntry<ApiType>;
       gasPrice: StorageEntryExact<ApiType, () => Observable<BalanceOf>> & QueryableStorageEntry<ApiType>;
     };
     sudo: {
@@ -201,13 +202,22 @@ declare module './types' {
       [index: string]: QueryableStorageEntry<ApiType>;
       gossipAt: StorageEntryExact<ApiType, () => Observable<BlockNumber>> & QueryableStorageEntry<ApiType>;
       keys: StorageEntryExact<ApiType, () => Observable<Vec<AuthorityId>>> & QueryableStorageEntry<ApiType>;
-      receivedHeartbeats: StorageEntryExact<ApiType, (key1: SessionIndex | Uint8Array | number | string, key2: AuthIndex | Uint8Array | number | string) => Observable<Bytes>> & QueryableStorageEntry<ApiType>;
+      receivedHeartbeats: StorageEntryExact<ApiType, (key1: SessionIndex | Uint8Array | number | string, key2: AuthIndex | Uint8Array | number | string) => Observable<Option<Bytes>>> & QueryableStorageEntry<ApiType>;
+      authoredBlocks: StorageEntryExact<ApiType, (key1: SessionIndex | Uint8Array | number | string, key2: ValidatorId | Uint8Array | string) => Observable<u32>> & QueryableStorageEntry<ApiType>;
     };
     offences: {
       [index: string]: QueryableStorageEntry<ApiType>;
-      reports: StorageEntryExact<ApiType, (arg: ReportIdOf | Uint8Array | string) => Observable<OffenceDetails>> & QueryableStorageEntry<ApiType>;
+      reports: StorageEntryExact<ApiType, (arg: ReportIdOf | Uint8Array | string) => Observable<Option<OffenceDetails>>> & QueryableStorageEntry<ApiType>;
       concurrentReportsIndex: StorageEntryExact<ApiType, (key1: Kind | Uint8Array | string, key2: OpaqueTimeSlot | Uint8Array | string) => Observable<Vec<ReportIdOf>>> & QueryableStorageEntry<ApiType>;
       reportsByKindIndex: StorageEntryExact<ApiType, (arg: Kind | Uint8Array | string) => Observable<Bytes>> & QueryableStorageEntry<ApiType>;
     };
+    randomnessCollectiveFlip: {
+      [index: string]: QueryableStorageEntry<ApiType>;
+      randomMaterial: StorageEntryExact<ApiType, () => Observable<Vec<Hash>>> & QueryableStorageEntry<ApiType>;
+    };
+    nicks: {
+      [index: string]: QueryableStorageEntry<ApiType>;
+      nameOf: StorageEntryExact<ApiType, (arg: AccountId | Uint8Array | string) => Observable<Option<ITuple<[Bytes, BalanceOf]>>>> & QueryableStorageEntry<ApiType>;
+    };
   }
 }

packages/api/src/submittable/Submittable.ts

@@ -1,259 +0,0 @@
-/* eslint-disable no-dupe-class-members */
-// Copyright 2017-2019 @polkadot/api authors & contributors
-// This software may be modified and distributed under the terms
-// of the Apache-2.0 license. See the LICENSE file for details.
-
-import { AccountId, Address, Call, Extrinsic, ExtrinsicEra, ExtrinsicStatus, Hash, Header, Index } from '@polkadot/types/interfaces';
-import { Callback, Codec, Constructor, IKeyringPair, SignatureOptions } from '@polkadot/types/types';
-import { ApiInterfaceRx, ApiTypes, SignerResult } from '../types';
-import { SignerOptions, SubmittableExtrinsic, SubmittableResultImpl, SubmittableResultResult, SubmittableResultSubscription } from './types';
-
-import { Observable, combineLatest, of } from 'rxjs';
-import { first, map, mergeMap, switchMap, tap } from 'rxjs/operators';
-import { createType, ClassOf } from '@polkadot/types';
-import { isBn, isFunction, isNumber, isUndefined } from '@polkadot/util';
-
-import { filterEvents, isKeyringPair } from '../util';
-import ApiBase from '../base';
-import SubmittableResult from './Result';
-
-interface SubmittableOptions<ApiType extends ApiTypes> {
-  api: ApiInterfaceRx;
-  decorateMethod: ApiBase<ApiType>['decorateMethod'];
-  type: ApiTypes;
-}
-
-// The default for 6s allowing for 5min eras. When translating this to faster blocks -
-//   - 4s = (10 / 15) * 5 = 3.33m
-//   - 2s = (10 / 30) * 5 = 1.66m
-const BLOCKTIME = 6;
-const ONE_MINUTE = 60 / BLOCKTIME;
-const DEFAULT_MORTAL_LENGTH = 5 * ONE_MINUTE;
-
-const _Extrinsic: Constructor<Extrinsic> = ClassOf('Extrinsic');
-
-export default class Submittable<ApiType extends ApiTypes> extends _Extrinsic implements SubmittableExtrinsic<ApiType> {
-  private readonly _api: ApiInterfaceRx;
-
-  private readonly _decorateMethod: ApiBase<ApiType>['decorateMethod'];
-
-  private readonly _ignoreStatusCb: boolean;
-
-  constructor (extrinsic: Call | Uint8Array | string, { api, decorateMethod, type }: SubmittableOptions<ApiType>) {
-    super(extrinsic, { version: api.extrinsicType });
-
-    this._api = api;
-    this._decorateMethod = decorateMethod;
-    this._ignoreStatusCb = type === 'rxjs';
-  }
-
-  // sign a transaction, returning the this to allow cianing, i.e. .sign(...).send()
-  public sign (account: IKeyringPair, optionsOrNonce: Partial<SignerOptions>): this {
-    // NOTE here we actually override nonce if it was specified (backwards compat for
-    // the previous signature - don't let userspace break, but allow then time to upgrade)
-    const options: Partial<SignerOptions> = isBn(optionsOrNonce) || isNumber(optionsOrNonce)
-      ? { nonce: optionsOrNonce }
-      : optionsOrNonce;
-
-    super.sign(account, this._makeSignOptions(options, {}));
-
-    return this;
-  }
-
-  // signAndSend with an immediate Hash result
-  public signAndSend (account: IKeyringPair | string | AccountId | Address, options?: Partial<SignerOptions>): SubmittableResultResult<ApiType>;
-
-  // signAndSend with a subscription, i.e. callback provided
-  public signAndSend (account: IKeyringPair | string | AccountId | Address, statusCb: Callback<SubmittableResultImpl>): SubmittableResultSubscription<ApiType>;
-
-  // signAndSend with options and a callback
-  public signAndSend (account: IKeyringPair | string | AccountId | Address, options: Partial<SignerOptions>, statusCb?: Callback<SubmittableResultImpl>): SubmittableResultSubscription<ApiType>;
-
-  // signAndSend implementation for all 3 cases above
-  public signAndSend (account: IKeyringPair | string | AccountId | Address, optionsOrStatus?: Partial<SignerOptions> | Callback<SubmittableResultImpl>, optionalStatusCb?: Callback<SubmittableResultImpl>): SubmittableResultResult<ApiType> | SubmittableResultSubscription<ApiType> {
-    const [options, statusCb] = this._makeSignAndSendOptions(optionsOrStatus, optionalStatusCb);
-    const isSubscription = this._api.hasSubscriptions && (this._ignoreStatusCb || !!statusCb);
-    const address = isKeyringPair(account) ? account.address : account.toString();
-    let updateId: number | undefined;
-
-    return this._decorateMethod(
-      (): Observable<Codec> => (
-        this._getPrelimState(address, options).pipe(
-          first(),
-          mergeMap(async ([nonce, header]): Promise<void> => {
-            const eraOptions = this._makeEraOptions(options, { header, nonce });
-
-            if (isKeyringPair(account)) {
-              this.sign(account, eraOptions);
-            } else {
-              updateId = await this._signViaSigner(address, eraOptions, header);
-            }
-          }),
-          switchMap((): Observable<SubmittableResultImpl> | Observable<Hash> => {
-            return isSubscription
-              ? this._subscribeObservable(updateId)
-              : this._sendObservable(updateId);
-          })
-        ) as Observable<Codec>) // FIXME This is wrong, SubmittableResult is _not_ a codec
-    )(statusCb) as SubmittableResultResult<ApiType> | SubmittableResultSubscription<ApiType>;
-  }
-
-  // send with an immediate Hash result
-  public send (): SubmittableResultResult<ApiType>;
-
-  // send with a status callback
-  public send (statusCb: Callback<SubmittableResultImpl>): SubmittableResultSubscription<ApiType>;
-
-  // send implementation for both immediate Hash and statusCb variants
-  public send (statusCb?: Callback<SubmittableResultImpl>): SubmittableResultResult<ApiType> | SubmittableResultSubscription<ApiType> {
-    const isSubscription = this._api.hasSubscriptions && (this._ignoreStatusCb || !!statusCb);
-
-    return this._decorateMethod(
-      isSubscription
-        ? this._subscribeObservable
-        : this._sendObservable
-    )(statusCb);
-  }
-
-  private _makeSignAndSendOptions (optionsOrStatus?: Partial<SignerOptions> | Callback<SubmittableResultImpl>, statusCb?: Callback<SubmittableResultImpl>): [Partial<SignerOptions>, Callback<SubmittableResultImpl>?] {
-    let options: Partial<SignerOptions> = {};
-
-    if (isFunction(optionsOrStatus)) {
-      statusCb = optionsOrStatus;
-    } else {
-      options = { ...optionsOrStatus };
-    }
-
-    return [options, statusCb];
-  }
-
-  private async _signViaSigner (address: string, optionsWithEra: SignatureOptions, header: Header | null): Promise<number> {
-    if (!this._api.signer) {
-      throw new Error('no signer attached');
-    }
-
-    const payload = createType('SignerPayload', {
-      ...optionsWithEra,
-      address,
-      method: this.method,
-      blockNumber: header ? header.number : 0
-    });
-    let result: SignerResult;
-
-    if (this._api.signer.signPayload) {
-      result = await this._api.signer.signPayload(payload.toPayload());
-    } else if (this._api.signer.signRaw) {
-      result = await this._api.signer.signRaw(payload.toRaw());
-    } else {
-      throw new Error('Invalid signer interface, it should implement either signPayload or signRaw (or both)');
-    }
-
-    // Here we explicitly call `toPayload()` again instead of working with an object
-    // (reference) as passed to the signer. This means that we are sure that the
-    // payload data is not modified from our inputs, but the signer
-    super.addSignature(address, result.signature, payload.toPayload());
-
-    return result.id;
-  }
-
-  private _makeSignOptions (options: Partial<SignerOptions>, extras: { blockHash?: Hash; era?: ExtrinsicEra; nonce?: Index }): SignatureOptions {
-    return {
-      blockHash: this._api.genesisHash,
-      ...options,
-      ...extras,
-      genesisHash: this._api.genesisHash,
-      runtimeVersion: this._api.runtimeVersion,
-      version: this._api.extrinsicType
-    } as unknown as SignatureOptions;
-  }
-
-  private _makeEraOptions (options: Partial<SignerOptions>, { header, nonce }: { header: Header | null; nonce: Index }): SignatureOptions {
-    if (!header) {
-      if (isNumber(options.era)) {
-        // since we have no header, it is immortal, remove any option overrides
-        // so we only supply the genesisHash and no era to the construction
-        delete options.era;
-        delete options.blockHash;
-      }
-
-      return this._makeSignOptions(options, { nonce });
-    }
-
-    return this._makeSignOptions(options, {
-      blockHash: header.hash,
-      era: createType('ExtrinsicEra', {
-        current: header.number,
-        period: options.era || DEFAULT_MORTAL_LENGTH
-      }),
-      nonce
-    });
-  }
-
-  private _getPrelimState (address: string, options: Partial<SignerOptions>): Observable<[Index, Header | null]> {
-    return combineLatest([
-      // if we have a nonce already, don't retrieve the latest, use what is there
-      isUndefined(options.nonce)
-        // FIXME This apparently is having issues on latest Kusama for nonce retrieval,
-        // hence we are using the accountNonce only
-        // ? this._api.rpc.account.nextIndex
-        //   ? this._api.rpc.account.nextIndex(address)
-        //   : this._api.query.system.accountNonce(address)
-        ? this._api.query.system.accountNonce(address)
-        : of(createType('Index', options.nonce)),
-      // if we have an era provided already or eraLength is <= 0 (immortal)
-      // don't get the latest block, just pass null, handle in mergeMap
-      (isUndefined(options.era) || (isNumber(options.era) && options.era > 0))
-        ? this._api.rpc.chain.getHeader()
-        : of(null)
-    ]);
-  }
-
-  private _updateSigner (updateId: number, status: Hash | SubmittableResultImpl): void {
-    if ((updateId !== -1) && this._api.signer && this._api.signer.update) {
-      this._api.signer.update(updateId, status);
-    }
-  }
-
-  private _statusObservable (status: ExtrinsicStatus): Observable<SubmittableResultImpl> {
-    if (!status.isFinalized) {
-      return of(new SubmittableResult({ status }));
-    }
-
-    const blockHash = status.asFinalized;
-
-    return combineLatest([
-      this._api.rpc.chain.getBlock(blockHash),
-      this._api.query.system.events.at(blockHash)
-    ]).pipe(
-      map(([signedBlock, allEvents]): SubmittableResultImpl =>
-        new SubmittableResult({
-          events: filterEvents(this.hash, signedBlock, allEvents),
-          status
-        })
-      )
-    );
-  }
-
-  private _sendObservable = (updateId = -1): Observable<Hash> => {
-    return this._api.rpc.author
-      .submitExtrinsic(this)
-      .pipe(
-        tap((hash): void => {
-          this._updateSigner(updateId, hash);
-        })
-      );
-  }
-
-  private _subscribeObservable = (updateId = -1): Observable<SubmittableResultImpl> => {
-    return this._api.rpc.author
-      .submitAndWatchExtrinsic(this)
-      .pipe(
-        switchMap((status): Observable<SubmittableResultImpl> =>
-          this._statusObservable(status)
-        ),
-        tap((status): void => {
-          this._updateSigner(updateId, status);
-        })
-      );
-  }
-}

packages/api/src/submittable/createClass.ts

@@ -0,0 +1,278 @@
+/* eslint-disable no-dupe-class-members */
+// Copyright 2017-2019 @polkadot/api authors & contributors
+// This software may be modified and distributed under the terms
+// of the Apache-2.0 license. See the LICENSE file for details.
+
+import { AccountId, Address, Call, ExtrinsicEra, ExtrinsicStatus, Hash, Header, Index } from '@polkadot/types/interfaces';
+import { Callback, Codec, Constructor, IKeyringPair, Registry, SignatureOptions } from '@polkadot/types/types';
+import { ApiInterfaceRx, ApiTypes, SignerResult } from '../types';
+import { SignerOptions, SubmittableExtrinsic, SubmittableResultImpl, SubmittableResultResult, SubmittableResultSubscription } from './types';
+
+import { Observable, combineLatest, of } from 'rxjs';
+import { first, map, mergeMap, switchMap, tap } from 'rxjs/operators';
+import { createType, ClassOf } from '@polkadot/types';
+import { isBn, isFunction, isNumber, isUndefined } from '@polkadot/util';
+
+import { filterEvents, isKeyringPair } from '../util';
+import ApiBase from '../base';
+import SubmittableResult from './Result';
+
+interface SubmittableOptions<ApiType extends ApiTypes> {
+  api: ApiInterfaceRx;
+  apiType: ApiTypes;
+  decorateMethod: ApiBase<ApiType>['decorateMethod'];
+}
+
+// The default for 6s allowing for 5min eras. When translating this to faster blocks -
+//   - 4s = (10 / 15) * 5 = 3.33m
+//   - 2s = (10 / 30) * 5 = 1.66m
+const BLOCKTIME = 6;
+const ONE_MINUTE = 60 / BLOCKTIME;
+const DEFAULT_MORTAL_LENGTH = 5 * ONE_MINUTE;
+
+export default function createClass <ApiType extends ApiTypes> ({ api, apiType, decorateMethod }: SubmittableOptions<ApiType>): Constructor<SubmittableExtrinsic<ApiType>> {
+  // an instance of the base extrinsic for us to extend
+  const Extrinsic = ClassOf(api.registry, 'Extrinsic');
+
+  return class Submittable extends Extrinsic implements SubmittableExtrinsic<ApiType> {
+    private readonly _api: ApiInterfaceRx;
+
+    private readonly _decorateMethod: ApiBase<ApiType>['decorateMethod'];
+
+    private readonly _ignoreStatusCb: boolean;
+
+    constructor (registry: Registry, extrinsic: Call | Uint8Array | string) {
+      super(registry, extrinsic, { version: api.extrinsicType });
+
+      this._api = api;
+      this._decorateMethod = decorateMethod;
+      this._ignoreStatusCb = apiType === 'rxjs';
+    }
+
+    // sign a transaction, returning the this to allow chaining, i.e. .sign(...).send()
+    public sign (account: IKeyringPair, optionsOrNonce: Partial<SignerOptions>): this {
+      super.sign(account, this._makeSignOptions(this._optionsOrNonce(optionsOrNonce), {}));
+
+      return this;
+    }
+
+    // signs a transaction, returning `this` to allow chaining. E.g.: `sign(...).send()`
+    //
+    // also supports signing through external signers
+    public async signAsync (account: IKeyringPair, optionsOrNonce: Partial<SignerOptions>): Promise<this> {
+      if (this._api.signer) {
+        const options = this._makeSignOptions(this._optionsOrNonce(optionsOrNonce), {});
+        await this._signViaSigner(account.address, options, null);
+      } else {
+        this.sign(account, optionsOrNonce);
+      }
+
+      return this;
+    }
+
+    // signAndSend with an immediate Hash result
+    public signAndSend (account: IKeyringPair | string | AccountId | Address, options?: Partial<SignerOptions>): SubmittableResultResult<ApiType>;
+
+    // signAndSend with a subscription, i.e. callback provided
+    public signAndSend (account: IKeyringPair | string | AccountId | Address, statusCb: Callback<SubmittableResultImpl>): SubmittableResultSubscription<ApiType>;
+
+    // signAndSend with options and a callback
+    public signAndSend (account: IKeyringPair | string | AccountId | Address, options: Partial<SignerOptions>, statusCb?: Callback<SubmittableResultImpl>): SubmittableResultSubscription<ApiType>;
+
+    // signAndSend implementation for all 3 cases above
+    public signAndSend (account: IKeyringPair | string | AccountId | Address, optionsOrStatus?: Partial<SignerOptions> | Callback<SubmittableResultImpl>, optionalStatusCb?: Callback<SubmittableResultImpl>): SubmittableResultResult<ApiType> | SubmittableResultSubscription<ApiType> {
+      const [options, statusCb] = this._makeSignAndSendOptions(optionsOrStatus, optionalStatusCb);
+      const isSubscription = this._api.hasSubscriptions && (this._ignoreStatusCb || !!statusCb);
+      const address = isKeyringPair(account) ? account.address : account.toString();
+      let updateId: number | undefined;
+
+      return this._decorateMethod(
+        (): Observable<Codec> => (
+          this._getPrelimState(address, options).pipe(
+            first(),
+            mergeMap(async ([nonce, header]): Promise<void> => {
+              const eraOptions = this._makeEraOptions(options, { header, nonce });
+
+              if (isKeyringPair(account)) {
+                this.sign(account, eraOptions);
+              } else {
+                updateId = await this._signViaSigner(address, eraOptions, header);
+              }
+            }),
+            switchMap((): Observable<SubmittableResultImpl> | Observable<Hash> => {
+              return isSubscription
+                ? this._subscribeObservable(updateId)
+                : this._sendObservable(updateId);
+            })
+          ) as Observable<Codec>) // FIXME This is wrong, SubmittableResult is _not_ a codec
+      )(statusCb) as SubmittableResultResult<ApiType> | SubmittableResultSubscription<ApiType>;
+    }
+
+    // send with an immediate Hash result
+    public send (): SubmittableResultResult<ApiType>;
+
+    // send with a status callback
+    public send (statusCb: Callback<SubmittableResultImpl>): SubmittableResultSubscription<ApiType>;
+
+    // send implementation for both immediate Hash and statusCb variants
+    public send (statusCb?: Callback<SubmittableResultImpl>): SubmittableResultResult<ApiType> | SubmittableResultSubscription<ApiType> {
+      const isSubscription = this._api.hasSubscriptions && (this._ignoreStatusCb || !!statusCb);
+
+      return this._decorateMethod(
+        isSubscription
+          ? this._subscribeObservable
+          : this._sendObservable
+      )(statusCb);
+    }
+
+    private _makeSignAndSendOptions (optionsOrStatus?: Partial<SignerOptions> | Callback<SubmittableResultImpl>, statusCb?: Callback<SubmittableResultImpl>): [Partial<SignerOptions>, Callback<SubmittableResultImpl>?] {
+      let options: Partial<SignerOptions> = {};
+
+      if (isFunction(optionsOrStatus)) {
+        statusCb = optionsOrStatus;
+      } else {
+        options = { ...optionsOrStatus };
+      }
+
+      return [options, statusCb];
+    }
+
+    private async _signViaSigner (address: string, options: SignatureOptions, header: Header | null): Promise<number> {
+      if (!this._api.signer) {
+        throw new Error('no signer attached');
+      }
+
+      const payload = createType(this.registry, 'SignerPayload', {
+        ...options,
+        address,
+        method: this.method,
+        blockNumber: header ? header.number : 0
+      });
+      let result: SignerResult;
+
+      if (this._api.signer.signPayload) {
+        result = await this._api.signer.signPayload(payload.toPayload());
+      } else if (this._api.signer.signRaw) {
+        result = await this._api.signer.signRaw(payload.toRaw());
+      } else {
+        throw new Error('Invalid signer interface, it should implement either signPayload or signRaw (or both)');
+      }
+
+      // Here we explicitly call `toPayload()` again instead of working with an object
+      // (reference) as passed to the signer. This means that we are sure that the
+      // payload data is not modified from our inputs, but the signer
+      super.addSignature(address, result.signature, payload.toPayload());
+
+      return result.id;
+    }
+
+    private _makeSignOptions (options: Partial<SignerOptions>, extras: { blockHash?: Hash; era?: ExtrinsicEra; nonce?: Index }): SignatureOptions {
+      return {
+        blockHash: this._api.genesisHash,
+        ...options,
+        ...extras,
+        genesisHash: this._api.genesisHash,
+        runtimeVersion: this._api.runtimeVersion,
+        version: this._api.extrinsicType
+      } as SignatureOptions;
+    }
+
+    private _makeEraOptions (options: Partial<SignerOptions>, { header, nonce }: { header: Header | null; nonce: Index }): SignatureOptions {
+      if (!header) {
+        if (isNumber(options.era)) {
+          // since we have no header, it is immortal, remove any option overrides
+          // so we only supply the genesisHash and no era to the construction
+          delete options.era;
+          delete options.blockHash;
+        }
+
+        return this._makeSignOptions(options, { nonce });
+      }
+
+      return this._makeSignOptions(options, {
+        blockHash: header.hash,
+        era: createType(this.registry, 'ExtrinsicEra', {
+          current: header.number,
+          period: options.era || DEFAULT_MORTAL_LENGTH
+        }),
+        nonce
+      });
+    }
+
+    private _getPrelimState (address: string, options: Partial<SignerOptions>): Observable<[Index, Header | null]> {
+      return combineLatest([
+        // if we have a nonce already, don't retrieve the latest, use what is there
+        isUndefined(options.nonce)
+          // FIXME This apparently is having issues on latest Kusama for nonce retrieval,
+          // hence we are using the accountNonce only
+          // ? this._api.rpc.account.nextIndex
+          //   ? this._api.rpc.account.nextIndex(address)
+          //   : this._api.query.system.accountNonce(address)
+          ? this._api.query.system.accountNonce(address)
+          : of(createType(this.registry, 'Index', options.nonce)),
+        // if we have an era provided already or eraLength is <= 0 (immortal)
+        // don't get the latest block, just pass null, handle in mergeMap
+        (isUndefined(options.era) || (isNumber(options.era) && options.era > 0))
+          ? this._api.rpc.chain.getHeader()
+          : of(null)
+      ]);
+    }
+
+    private _updateSigner (updateId: number, status: Hash | SubmittableResultImpl): void {
+      if ((updateId !== -1) && this._api.signer && this._api.signer.update) {
+        this._api.signer.update(updateId, status);
+      }
+    }
+
+    private _statusObservable (status: ExtrinsicStatus): Observable<SubmittableResultImpl> {
+      if (!status.isFinalized) {
+        return of(new SubmittableResult({ status }));
+      }
+
+      const blockHash = status.asFinalized;
+
+      return combineLatest([
+        this._api.rpc.chain.getBlock(blockHash),
+        this._api.query.system.events.at(blockHash)
+      ]).pipe(
+        map(([signedBlock, allEvents]): SubmittableResultImpl =>
+          new SubmittableResult({
+            events: filterEvents(this.hash, signedBlock, allEvents),
+            status
+          })
+        )
+      );
+    }
+
+    private _sendObservable = (updateId = -1): Observable<Hash> => {
+      return this._api.rpc.author
+        .submitExtrinsic(this)
+        .pipe(
+          tap((hash): void => {
+            this._updateSigner(updateId, hash);
+          })
+        );
+    }
+
+    private _subscribeObservable = (updateId = -1): Observable<SubmittableResultImpl> => {
+      return this._api.rpc.author
+        .submitAndWatchExtrinsic(this)
+        .pipe(
+          switchMap((status): Observable<SubmittableResultImpl> =>
+            this._statusObservable(status)
+          ),
+          tap((status): void => {
+            this._updateSigner(updateId, status);
+          })
+        );
+    }
+
+    // NOTE here we actually override nonce if it was specified (backwards compat for
+    // the previous signature - don't let userspace break, but allow then time to upgrade)
+    private _optionsOrNonce = (optionsOrNonce: Partial<SignerOptions>): Partial<SignerOptions> => {
+      return isBn(optionsOrNonce) || isNumber(optionsOrNonce)
+        ? { nonce: optionsOrNonce }
+        : optionsOrNonce;
+    }
+  };
+}

packages/api/src/submittable/createSubmittable.ts

@@ -8,19 +8,20 @@ import { ApiInterfaceRx, ApiTypes } from '../types';
 import { SubmittableExtrinsic } from './types';
 
 import ApiBase from '../base';
+import createClass from './createClass';
 
 type Creator<ApiType extends ApiTypes> = (extrinsic: Call | Uint8Array | string) => SubmittableExtrinsic<ApiType>;
 
 let Submittable: Constructor<SubmittableExtrinsic<any>>;
 
-export default function createSubmittable<ApiType extends ApiTypes> (type: ApiTypes, api: ApiInterfaceRx, decorateMethod: ApiBase<ApiType>['decorateMethod']): Creator<ApiType> {
+export default function createSubmittable<ApiType extends ApiTypes> (apiType: ApiTypes, api: ApiInterfaceRx, decorateMethod: ApiBase<ApiType>['decorateMethod']): Creator<ApiType> {
   return (extrinsic: Call | Uint8Array | string): SubmittableExtrinsic<ApiType> => {
-    // HACK This is not great, but basically what we do here is to lazily only require the class
-    // right at the point it is actually needed - delaying initialization
+    // This is not great, but basically what we do here is to only require the
+    // class right at the point it is actually needed - delaying initialization
     if (!Submittable) {
-      Submittable = require('./Submittable').default;
+      Submittable = createClass<ApiType>({ api, apiType, decorateMethod });
     }
 
-    return new Submittable(extrinsic, { api, decorateMethod, type });
+    return new Submittable(api.registry, extrinsic);
   };
 }

packages/api/src/submittable/types.ts

@@ -48,6 +48,8 @@ export interface SubmittableExtrinsic<ApiType extends ApiTypes> extends IExtrins
 
   sign(account: IKeyringPair, _options: Partial<SignatureOptions>): this;
 
+  signAsync(account: IKeyringPair, _options: Partial<SignatureOptions>): Promise<this>;
+
   signAndSend(account: IKeyringPair | string | AccountId | Address, options?: Partial<SignerOptions>): SubmittableResultResult<ApiType>;
 
   signAndSend(account: IKeyringPair | string | AccountId | Address, statusCb: Callback<SubmittableResultImpl>): SubmittableResultSubscription<ApiType>;

packages/api/src/types.ts

@@ -2,20 +2,25 @@
 // This software may be modified and distributed under the terms
 // of the Apache-2.0 license. See the LICENSE file for details.
 
+// These two files add module augmentation so that `api.{query,consts}.*.*`
+// get typed. Import them to make these types effective.
+import './consts.types';
+import './query.types';
+
+import { Constants } from '@polkadot/metadata/Decorated/types';
 import { UserRpc } from '@polkadot/rpc-core/types';
 import { Hash, RuntimeVersion } from '@polkadot/types/interfaces';
-import { AnyFunction, Callback, CallFunction, Codec, CodecArg, RegistryTypes, SignatureOptions, SignerPayloadJSON, SignerPayloadRaw } from '@polkadot/types/types';
+import { AnyFunction, Callback, CallFunction, Codec, CodecArg, RegistryTypes, SignatureOptions, SignerPayloadJSON, SignerPayloadRaw, Registry } from '@polkadot/types/types';
 import { SubmittableResultImpl, SubmittableExtrinsic } from './submittable/types';
 import { DeriveAllSections } from './util/decorate';
 
 import BN from 'bn.js';
 import { Observable } from 'rxjs';
 import { DeriveCustom, ExactDerive } from '@polkadot/api-derive';
-import { Constants } from '@polkadot/api-metadata/consts/types';
 import { RpcInterface } from '@polkadot/rpc-core/jsonrpc.types';
 import { ProviderInterface, ProviderInterfaceEmitted } from '@polkadot/rpc-provider/types';
 import { Metadata, u64 } from '@polkadot/types';
-import { StorageEntry } from '@polkadot/types/primitive/StorageKey';
+import StorageKey, { StorageEntry } from '@polkadot/types/primitive/StorageKey';
 
 import ApiBase from './base';
 
@@ -105,6 +110,7 @@ interface StorageEntryPromiseMulti {
 export interface StorageEntryBase<ApiType extends ApiTypes, F extends AnyFunction> {
   at: (hash: Hash | Uint8Array | string, ...args: Parameters<F>) => PromiseOrObs<ApiType, ObsInnerType<ReturnType<F>>>;
   creator: StorageEntry;
+  entries: () => PromiseOrObs<ApiType, [StorageKey, ObsInnerType<ReturnType<F>>][]>;
   hash: (...args: Parameters<F>) => PromiseOrObs<ApiType, Hash>;
   key: (...args: Parameters<F>) => string;
   size: (...args: Parameters<F>) => PromiseOrObs<ApiType, u64>;
@@ -187,6 +193,10 @@ export interface ApiOptions {
    * connecting to a WsProvider connecting localhost with the default port, i.e. `ws://127.0.0.1:9944`
    */
   provider?: ProviderInterface;
+  /**
+   * @description A type registry to use along with this instance
+   */
+  registry?: Registry;
   /**
    * @description User-defined RPC methods
    */
@@ -222,6 +232,7 @@ export interface ApiInterfaceRx {
   extrinsicType: number;
   genesisHash: Hash;
   hasSubscriptions: boolean;
+  registry: Registry;
   runtimeMetadata: Metadata;
   runtimeVersion: RuntimeVersion;
   query: QueryableStorage<'rxjs'>;

packages/api/test/e2e/api-derive/rx.spec.ts

@@ -9,7 +9,7 @@ import BN from 'bn.js';
 import ApiRx from '@polkadot/api/rx/Api';
 import { HeaderExtended } from '@polkadot/api-derive';
 import { DerivedBalances, DerivedContractFees, DerivedElectionsInfo, DerivedFees, DerivedSessionInfo } from '@polkadot/api-derive/types';
-import { createType, ClassOf } from '@polkadot/types';
+import { createType, ClassOf, TypeRegistry } from '@polkadot/types';
 import { WsProvider } from '@polkadot/rpc-provider';
 
 import { describeE2E } from '../../util';
@@ -24,6 +24,7 @@ describeE2E({
     'remote-substrate-1.0'
   ]
 })('Api-RX derive e2e', (wsUrl: string): void => {
+  const registry = new TypeRegistry();
   let api: ApiRx;
 
   beforeAll((): void => {
@@ -76,7 +77,7 @@ describeE2E({
         api.derive.accounts.indexToId(IX).subscribe((accountId): void => {
           // The first emitted value for accountId is undefined when passing the IX
           if (accountId) {
-            expect(accountId instanceof ClassOf('AccountId')).toBe(true);
+            expect(accountId instanceof ClassOf(registry, 'AccountId')).toBe(true);
             expect(accountId.toString()).toEqual(ID);
           } else {
             expect(accountId).toEqual(undefined);
@@ -91,7 +92,7 @@ describeE2E({
         api.derive.accounts.idToIndex(ID).subscribe((accountIndex): void => {
           // The first emitted value for AccountIndex is undefined when passing the ID
           if (accountIndex) {
-            expect(accountIndex instanceof ClassOf('AccountIndex')).toBe(true);
+            expect(accountIndex instanceof ClassOf(registry, 'AccountIndex')).toBe(true);
             expect(accountIndex.toString()).toEqual(IX);
           } else {
             expect(accountIndex).toEqual(undefined);
@@ -107,7 +108,7 @@ describeE2E({
           // A local dev chain should have the AccountIndex of Alice
           expect(accountIndexes).toHaveProperty(
             ID,
-            createType('AccountIndex', IX)
+            createType(registry, 'AccountIndex', IX)
           );
           done();
         });
@@ -122,8 +123,8 @@ describeE2E({
       it('It returns an object with all relevant balance information of an account', (done): void => {
         api.derive.balances.all(ID).subscribe((balances: DerivedBalances): void => {
           expect(balances).toEqual(expect.objectContaining({
-            accountId: expect.any(ClassOf('AccountId')),
-            accountNonce: expect.any(ClassOf('Index')),
+            accountId: expect.any(ClassOf(registry, 'AccountId')),
+            accountNonce: expect.any(ClassOf(registry, 'Index')),
             availableBalance: expect.any(BN),
             freeBalance: expect.any(BN),
             lockedBalance: expect.any(BN),
@@ -156,7 +157,7 @@ describeE2E({
     describe('bestNumber', (): void => {
       it('Get the latest block number', (done): void => {
         api.derive.chain.bestNumber().subscribe((blockNumber: BlockNumber): void => {
-          expect(blockNumber instanceof ClassOf('BlockNumber')).toBe(true);
+          expect(blockNumber instanceof ClassOf(registry, 'BlockNumber')).toBe(true);
           expect(blockNumber.gten(0)).toBe(true);
           done();
         });
@@ -166,7 +167,7 @@ describeE2E({
     describe('bestNumberFinalized', (): void => {
       it('Get the latest finalised block number', (done): void => {
         api.derive.chain.bestNumberFinalized().subscribe((blockNumber: BlockNumber): void => {
-          expect(blockNumber instanceof ClassOf('BlockNumber')).toBe(true);
+          expect(blockNumber instanceof ClassOf(registry, 'BlockNumber')).toBe(true);
           expect(blockNumber.gten(0)).toBe(true);
           done();
         });
@@ -176,7 +177,7 @@ describeE2E({
     describe('bestNumberLag', (): void => {
       it('lag between finalised head and best head', (done): void => {
         api.derive.chain.bestNumberLag().subscribe((numberLag: BlockNumber): void => {
-          expect(numberLag instanceof ClassOf('BlockNumber')).toBe(true);
+          expect(numberLag instanceof ClassOf(registry, 'BlockNumber')).toBe(true);
           expect(numberLag.gten(0)).toBe(true);
           done();
         });