engine-cuda: loop-structured mining_loop kernel, opt-in via MINER_CUDA_KERNEL=loop #29

Merged
grenade merged 2 commits from perf/loop-kernel into main 2026-09-14 13:44:56 +00:00
Owner

First step of the rewrite in quantus/miner#27, as a gated path so the deployed kernel is untouched.

What it adds. A second entry point mining_loop in the same fatbin. It shares every arithmetic primitive with mining_main; the per-nonce body is one template used by both. Its round loops are rolled with the round constants fetched from constant memory by index (the shape of the fastest closed kernel), with its own launch-bound knobs (LAIR_LOOP_TPB, LAIR_LOOP_MINBLOCKS, LAIR_LOOP_INT_PAIR). MINER_CUDA_KERNEL=loop selects it at run time, defaults the block size to 512, and reports kernel id cuda-loop on the per-device metrics; unset keeps mining_main and the cuda id the deploy validate asserts. No default changes: sm_120 mining_main is still 21,120 instructions at 110 registers, byte for byte.

Measured, beast, three interleaved rounds, parity 500/500 on both kernels:

kernel MH/s static instrs regs spill
unrolled (deployed) 2206 21,120 110 0
loop, 512 threads 2130 (-3.4%) 11,840 128 88 B

Shapes tried for the loop kernel: 192 registers (one block per SM) 1976, clocks up to 2600 MHz because the card is underfilled; 80 registers 2005 with 424 B of spill; 64 registers 373 to 532 with 716 to 872 B of spill; two internal rounds per iteration 1976 against one at 2105; 256- and 128-thread blocks below 512. On sm_86/89 the loop kernel compiles to 88 registers with no spill where the unrolled one spills 8 bytes; benjy and quadbrat numbers to follow in a comment.

What this says about #27. The loop structure by itself does not lower this arithmetic's register footprint the way the quanpool kernel's is: 64 registers is only reachable with heavy spill. Its loop bodies are 778 / 254 / 789 instructions (external-initial, internal, external-terminal) against their 741 / 199 / 662, so the internal round's diagonal multiply-add is where the instructions are. One attempt at that, folding the whole 64-bit row sum into the first partial product with a carry-out the way their SASS does, measured worse (unrolled 2208 -> 2172, loop internal body 254 -> 268) and was reverted: ptxas does not turn that PTX into their instruction sequence. Next steps happen behind this switch: the round arithmetic as one carry chain with 32-bit halves throughout, screened by the loop body's static count before any bench.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Ue5ZZm1Hiv5zPnucykKKuF

First step of the rewrite in quantus/miner#27, as a gated path so the deployed kernel is untouched. **What it adds.** A second entry point `mining_loop` in the same fatbin. It shares every arithmetic primitive with `mining_main`; the per-nonce body is one template used by both. Its round loops are rolled with the round constants fetched from constant memory by index (the shape of the fastest closed kernel), with its own launch-bound knobs (`LAIR_LOOP_TPB`, `LAIR_LOOP_MINBLOCKS`, `LAIR_LOOP_INT_PAIR`). `MINER_CUDA_KERNEL=loop` selects it at run time, defaults the block size to 512, and reports kernel id `cuda-loop` on the per-device metrics; unset keeps `mining_main` and the `cuda` id the deploy validate asserts. **No default changes**: sm_120 `mining_main` is still 21,120 instructions at 110 registers, byte for byte. **Measured**, beast, three interleaved rounds, parity 500/500 on both kernels: | kernel | MH/s | static instrs | regs | spill | |---|---|---|---|---| | unrolled (deployed) | 2206 | 21,120 | 110 | 0 | | loop, 512 threads | 2130 (-3.4%) | 11,840 | 128 | 88 B | Shapes tried for the loop kernel: 192 registers (one block per SM) 1976, clocks up to 2600 MHz because the card is underfilled; 80 registers 2005 with 424 B of spill; 64 registers 373 to 532 with 716 to 872 B of spill; two internal rounds per iteration 1976 against one at 2105; 256- and 128-thread blocks below 512. On sm_86/89 the loop kernel compiles to 88 registers with no spill where the unrolled one spills 8 bytes; benjy and quadbrat numbers to follow in a comment. **What this says about #27.** The loop structure by itself does not lower this arithmetic's register footprint the way the quanpool kernel's is: 64 registers is only reachable with heavy spill. Its loop bodies are 778 / 254 / 789 instructions (external-initial, internal, external-terminal) against their 741 / 199 / 662, so the internal round's diagonal multiply-add is where the instructions are. One attempt at that, folding the whole 64-bit row sum into the first partial product with a carry-out the way their SASS does, measured worse (unrolled 2208 -> 2172, loop internal body 254 -> 268) and was reverted: ptxas does not turn that PTX into their instruction sequence. Next steps happen behind this switch: the round arithmetic as one carry chain with 32-bit halves throughout, screened by the loop body's static count before any bench. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01Ue5ZZm1Hiv5zPnucykKKuF
grenade added 1 commit 2026-09-14 13:30:42 +00:00
engine-cuda: loop-structured mining_loop kernel behind MINER_CUDA_KERNEL
Some checks failed
ci / fmt (pull_request) Successful in 19s
bench / build (pull_request) Successful in 1m13s
bench / measure (pull_request) Failing after 14s
ci / clippy (pull_request) Successful in 1m49s
ci / doc (pull_request) Successful in 2m25s
ci / test (pull_request) Successful in 8m23s
816b1efb61
A second entry point in the same fatbin, sharing every arithmetic
primitive with mining_main, with the round loops rolled and the round
constants fetched from constant memory by index (quantus/miner#27). The
per-nonce body is one template used by both entries, so the deployed
kernel is byte-identical (sm_120 21,120 instructions, 110 registers).

Selected at run time: MINER_CUDA_KERNEL=loop loads mining_loop, reports
kernel id cuda-loop on the per-device metrics, and defaults the block
size to 512 (LAIR_LOOP_TPB); unset or unrolled keeps mining_main and the
cuda kernel id the deploy validate asserts.

Measured on beast, three interleaved rounds, parity 500/500 on both:
unrolled 2206, loop 2130 MH/s (-3.4%). The loop kernel is 11,840 static
instructions at 128 registers (88 bytes of spill) on sm_120; 88 registers
with no spill on sm_86/89. Shapes tried for it: 192 registers 1976 (one
block per SM, card underfilled), 80 registers 2005 (424 B spill), 64
registers 373-532 (716-872 B spill), two internal rounds per iteration
1976 against one at 2105, 256/128-thread blocks below 512.

Not the win #27 hoped for yet: the loop structure alone does not lower
this arithmetic's register footprint the way the quanpool kernel's is,
and its internal round is 254 instructions to their 199. Kept as an
opt-in path so the next steps (the round arithmetic as one carry chain)
can be measured against the deployed kernel without touching it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ue5ZZm1Hiv5zPnucykKKuF
Author
Owner

benjy and quadbrat, same binary, three interleaved rounds, parity 200/200 on the loop kernel on each:

host unrolled loop
benjy RTX 4090 (sm_89) 716-718 MH/s at 1935-1950 MHz 727-736 MH/s at 2010-2055 MHz +1.7%
quadbrat RTX 3060 (sm_86) 141.4-142.0 138.9-139.1 -1.8%
beast RTX 5090 (sm_120) 2206 2130 -3.4%

On the 4090 the 88-register no-spill build runs at a higher clock and wins. So the loop kernel is the better default for benjy today; the switch is per host in the deploy matrix (MINER_CUDA_KERNEL=loop in its environment, kernel: cuda-loop for validate), a follow-up once this merges.

benjy and quadbrat, same binary, three interleaved rounds, parity 200/200 on the loop kernel on each: | host | unrolled | loop | | |---|---|---|---| | benjy RTX 4090 (sm_89) | 716-718 MH/s at 1935-1950 MHz | 727-736 MH/s at 2010-2055 MHz | **+1.7%** | | quadbrat RTX 3060 (sm_86) | 141.4-142.0 | 138.9-139.1 | -1.8% | | beast RTX 5090 (sm_120) | 2206 | 2130 | -3.4% | On the 4090 the 88-register no-spill build runs at a higher clock and wins. So the loop kernel is the better default for benjy today; the switch is per host in the deploy matrix (`MINER_CUDA_KERNEL=loop` in its environment, `kernel: cuda-loop` for validate), a follow-up once this merges.
grenade added 1 commit 2026-09-14 13:36:09 +00:00
deploy: per-host CUDA kernel choice; benjy runs the loop kernel
All checks were successful
ci / fmt (pull_request) Successful in 21s
bench / build (pull_request) Successful in 1m0s
ci / clippy (pull_request) Successful in 2m42s
ci / doc (pull_request) Successful in 2m46s
bench / measure (pull_request) Successful in 2m56s
ci / test (pull_request) Successful in 7m54s
12bd36aee2
The matrix gains `cuda_kernel` (rendered into miner.env as
MINER_CUDA_KERNEL) next to `kernel`, the id validate asserts. benjy's
4090 measured +1.7% on the loop kernel (727-736 vs 716-718 MH/s, parity
200/200), so it runs loop / cuda-loop; quadbrat (-1.8%) and beast
(-3.4%) stay on unrolled / cuda. A rollback to a binary that predates
the switch is safe: it ignores the variable.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ue5ZZm1Hiv5zPnucykKKuF
Author
Owner

Second commit (12bd36a) adds the per-host switch to the deploy matrix: cuda_kernel is rendered into miner.env as MINER_CUDA_KERNEL, kernel stays the id validate asserts. benjy runs loop / cuda-loop on the measurement above; quadbrat and beast stay unrolled / cuda. The earlier bench-workflow failure on this PR (run 67) was the harness refusing a card my own A/B was occupying at that minute, not a build problem.

Second commit (12bd36a) adds the per-host switch to the deploy matrix: `cuda_kernel` is rendered into miner.env as `MINER_CUDA_KERNEL`, `kernel` stays the id validate asserts. benjy runs `loop` / `cuda-loop` on the measurement above; quadbrat and beast stay `unrolled` / `cuda`. The earlier bench-workflow failure on this PR (run 67) was the harness refusing a card my own A/B was occupying at that minute, not a build problem.

quantus-bench 12bd36aee2 on benjy.hanzalova.internal

worker median MH/s spread windows
0 718.48 0.3% 719.8, 719.8, 718.1, 718.5, 717.3

total median: 718.48 MH/s (batch 1000000, 5 x 30s, engine gpu-cuda)

  • gpu 0 NVIDIA GeForce RTX 4090: driver 580.173.02, limit 250 W, draw 250 W, sm 1920 MHz, mem 810 MHz, 57 C
  • parity: OK (25/25 jobs found solutions, all verified against CPU)
    --- resume miner ---
    active
## quantus-bench 12bd36aee216a6342da2cde9c3e865d3b48b324b on benjy.hanzalova.internal | worker | median MH/s | spread | windows | | --- | --- | --- | --- | | 0 | 718.48 | 0.3% | 719.8, 719.8, 718.1, 718.5, 717.3 | total median: **718.48 MH/s** (batch 1000000, 5 x 30s, engine gpu-cuda) - gpu 0 NVIDIA GeForce RTX 4090: driver 580.173.02, limit 250 W, draw 250 W, sm 1920 MHz, mem 810 MHz, 57 C - parity: OK (25/25 jobs found solutions, all verified against CPU) --- resume miner --- active
grenade merged commit ad77d99dfa into main 2026-09-14 13:44:56 +00:00
grenade deleted branch perf/loop-kernel 2026-09-14 13:44:58 +00:00
Sign in to join this conversation.
No Reviewers
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: blackbeard/miner#29