perf: fused-PTX field arithmetic, +54..66% on every card #23

grenade · 2026-09-13T19:30:06Z

grenade commented

2026-09-13 19:30:06 +00:00

Closes the bulk of the gap to the closed-source qpow-cuda kernel (quantus/miner#3).

Why they were 2x faster. Same 1472 multiplies per hash; their PTX (the fatbin ships compute_120 PTX) shows every field operation as one self-contained inline-PTX block. The 128-bit product is reduced with mad.lo.cc.u32/madc.hi.cc.u32, which ptxas emits as one IMAD.WIDE with a carry-out predicate, plus three ALU ops folding the 2^96 = -1 term. The internal layer's row sum rides in the mad.wide.u32 partial products for free. Accumulators are 32-bit limb chains (IADD, IADD.X, merged IADD3.X carries). Ours paid a ~14-op compare-and-select reduction, a 64-bit add-with-carry per diagonal element, and five instructions per accumulator add on sm_120.

What this does. Ports those primitives behind LAIR_FUSED_MUL (default 1): gf_mul, gf_sqr (three-product squaring), gf_mul_add with the addend folded into the partial products, acc_add/acc_add2 as limb chains, a four-op acc_reduce; the round constant joins the unreduced sum in the internal layer.

LAIR_EXACT_REDUCE (default 0): with 0 the reduction drops its final borrow as their kernel does. That is wrong only when the product's bits 64..95 are zero and its low 64 bits are below its top 32 bits, about 2^-64 per multiply (emulated against a reference multiply: 0/3M random mismatches, and the constructed 2^48*2^48 fails as predicted; the other apparent hazard, hh+carry wrapping 32 bits, is impossible). Exactness measured at -15%, so it is off.

Measured, three interleaved rounds, parity 40/40 against the CPU on each host:

host	main	this PR
beast 2x5090	1356	2146 MH/s	+58% (1073/card vs their 1171)
benjy 4090	460	711 MH/s	+54%
quadbrat 3060	84	140 MH/s	+66%

Static sm_120 31,280 -> 20,888 instructions, sm_86/89 39,208 -> 22,576, no spills, 110 registers on sm_120. ncu: 22,913 instructions per nonce against their 22,233; the residual 8% per 5090 is cycles (2.08 vs 1.87 SM-cycles/nonce, more dispatch stalls), not instructions or instruction cache.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Ue5ZZm1Hiv5zPnucykKKuF

Closes the bulk of the gap to the closed-source qpow-cuda kernel (quantus/miner#3). **Why they were 2x faster.** Same 1472 multiplies per hash; their PTX (the fatbin ships compute_120 PTX) shows every field operation as one self-contained inline-PTX block. The 128-bit product is reduced with `mad.lo.cc.u32`/`madc.hi.cc.u32`, which ptxas emits as one IMAD.WIDE with a carry-out predicate, plus three ALU ops folding the 2^96 = -1 term. The internal layer's row sum rides in the `mad.wide.u32` partial products for free. Accumulators are 32-bit limb chains (IADD, IADD.X, merged IADD3.X carries). Ours paid a ~14-op compare-and-select reduction, a 64-bit add-with-carry per diagonal element, and five instructions per accumulator add on sm_120. **What this does.** Ports those primitives behind `LAIR_FUSED_MUL` (default 1): `gf_mul`, `gf_sqr` (three-product squaring), `gf_mul_add` with the addend folded into the partial products, `acc_add`/`acc_add2` as limb chains, a four-op `acc_reduce`; the round constant joins the unreduced sum in the internal layer. `LAIR_EXACT_REDUCE` (default 0): with 0 the reduction drops its final borrow as their kernel does. That is wrong only when the product's bits 64..95 are zero and its low 64 bits are below its top 32 bits, about 2^-64 per multiply (emulated against a reference multiply: 0/3M random mismatches, and the constructed 2^48*2^48 fails as predicted; the other apparent hazard, hh+carry wrapping 32 bits, is impossible). Exactness measured at -15%, so it is off. **Measured**, three interleaved rounds, parity 40/40 against the CPU on each host: | host | main | this PR | | |---|---|---|---| | beast 2x5090 | 1356 | 2146 MH/s | +58% (1073/card vs their 1171) | | benjy 4090 | 460 | 711 MH/s | +54% | | quadbrat 3060 | 84 | 140 MH/s | +66% | Static sm_120 31,280 -> 20,888 instructions, sm_86/89 39,208 -> 22,576, no spills, 110 registers on sm_120. ncu: 22,913 instructions per nonce against their 22,233; the residual 8% per 5090 is cycles (2.08 vs 1.87 SM-cycles/nonce, more dispatch stalls), not instructions or instruction cache. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01Ue5ZZm1Hiv5zPnucykKKuF

grenade added 1 commit 2026-09-13 19:30:08 +00:00

engine-cuda: fused-PTX field arithmetic, +54..66% on every card

ci / fmt (pull_request) Successful in 20s

Details

bench / build (pull_request) Successful in 1m16s

Details

ci / clippy (pull_request) Successful in 1m44s

Details

bench / measure (pull_request) Successful in 2m52s

Details

ci / doc (pull_request) Successful in 2m1s

Details

ci / test (pull_request) Successful in 6m59s

Details

3763996ae6

The closed-source qpow-cuda kernel runs 2x our hashrate on the same
silicon with the same 1472 multiplies per hash. Its PTX shows why: every
field operation is one self-contained inline-PTX block, the 128-bit
product is reduced with mad.lo.cc/madc.hi.cc (one IMAD.WIDE with a
carry-out) plus three ALU ops, the internal layer's row sum rides in the
mad.wide partial products for free, and accumulators are 32-bit limb
chains that ptxas turns into IADD/IADD.X with merged carries. Ours paid
a compare-and-select reduction (~14 ops), a 64-bit add-with-carry per
diagonal element, and five instructions per accumulator add on sm_120.

This ports those primitives behind LAIR_FUSED_MUL (default 1): gf_mul,
gf_sqr (three-product squaring), gf_mul_add with the addend folded into
the partial products, acc_add/acc_add2 as limb chains, and a four-op
acc_reduce. The round constant now joins the unreduced sum in the
internal layer.

LAIR_EXACT_REDUCE (default 0) keeps the reduction bit-exact for three
more ops per multiply. With 0 the final borrow is dropped, as in their
kernel: wrong only when the product's bits 64..95 are zero and its low
64 bits are below its top 32 bits, about 2^-64 per multiply (emulated
against a reference: 0/3M random mismatches, 2^48*2^48 fails as
predicted). Exactness measured at -15% on beast, so it is off.

Measured, three interleaved rounds, parity 40/40 on each host:
  beast 2x5090  1356 -> 2146 MH/s  (+58%)
  benjy 4090     460 ->  711 MH/s  (+54%)
  quadbrat 3060   84 ->  140 MH/s  (+66%)
Static sm_120 31,280 -> 20,888 instructions, sm_86/89 39,208 -> 22,576,
no spills. ncu: 22,913 instructions per nonce against their 22,233.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ue5ZZm1Hiv5zPnucykKKuF

gitea-actionsbot commented

2026-09-13 19:34:49 +00:00

quantus-bench `3763996ae6` on benjy.hanzalova.internal

worker	median MH/s	spread	windows
0	708.79	0.1%	708.5, 708.8, 709.3, 709.4, 708.7

total median: 708.79 MH/s (batch 1000000, 5 x 30s, engine gpu-cuda)

gpu 0 NVIDIA GeForce RTX 4090: driver 580.173.02, limit 250 W, draw 250 W, sm 1980 MHz, mem 810 MHz, 60 C
parity: OK (25/25 jobs found solutions, all verified against CPU)
--- resume miner ---
active

## quantus-bench 3763996ae698c96be111c8711a17bc0914bf5be5 on benjy.hanzalova.internal | worker | median MH/s | spread | windows | | --- | --- | --- | --- | | 0 | 708.79 | 0.1% | 708.5, 708.8, 709.3, 709.4, 708.7 | total median: **708.79 MH/s** (batch 1000000, 5 x 30s, engine gpu-cuda) - gpu 0 NVIDIA GeForce RTX 4090: driver 580.173.02, limit 250 W, draw 250 W, sm 1980 MHz, mem 810 MHz, 60 C - parity: OK (25/25 jobs found solutions, all verified against CPU) --- resume miner --- active

grenade merged commit dc6c58ceb6 into main

2026-09-13 19:40:28 +00:00

grenade referenced this issue from a commit

2026-09-13 19:40:30 +00:00

Merge pull request 'perf: fused-PTX field arithmetic, +54..66% on every card' (#23) from perf/fused-field-ptx into main

grenade deleted branch perf/fused-field-ptx

2026-09-13 19:40:31 +00:00

grenade referenced this pull request

2026-09-13 19:55:57 +00:00

research: Goldilocks multiplies on the INT8 tensor cores #24

grenade referenced this pull request

2026-09-13 20:18:38 +00:00