blackbeard/miner
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases 2 Wiki Activity

engine-gpu: double-buffered submission so the GPU never drains between batches #5

New Issue
Closed
opened 2026-09-03 09:09:01 +00:00 by grenade · 2 comments
grenade commented 2026-09-03 09:09:01 +00:00
Owner
Copy Link

Part of #1. The proper fix for the same bubble the workers-per-device issue papers over. Applies to origin's wgpu engine, which stays our fallback on any host where engine-cuda is unavailable and is the engine on non-NVIDIA hardware.

Current loop

Per batch in run_single_batch (crates/engine-gpu/src/lib.rs):

  1. four queue.write_buffer calls (dispatch config, start nonce, midstate, results reset)
  2. encoder, compute pass, copy results to staging, queue.submit
  3. map_async on staging, device.poll(Wait) until the callback fires
  4. read 33 u32, unmap

The GPU is idle from the end of step 2's execution until the next iteration's step 2 reaches the hardware. With a 2.7 ms kernel that gap is the loss.

Design

Two resource sets per worker (GpuResources A and B), each with its own results and staging buffer. The loop becomes:

submit(A, batch 0)
loop:
  submit(B, batch n+1)          // GPU now has work queued behind A
  wait_and_read(A, batch n)     // blocks only until A's fence; B is running
  if found or cancelled: break
  swap(A, B)
  • map_async for A is issued right after A's submit, so by the time we wait, it is usually already complete.
  • Cancellation latency becomes two batches instead of one. At 1M that is ~5 ms. Acceptable; note it in the batch-size issue's stale-work budget.
  • Midstate for batch n+1 is computed on the CPU while batch n runs, which it effectively already is.
  • Solution handling: if A reports found, B may also find one; we take A's and discard B's when it lands (the worker pool already discards later results for a job). If B is found first it just waits one iteration.

Where the code goes

This one has to touch origin's engine-gpu; there is no way around it since the loop is the thing. Keep the diff mergeable:

  • New module crates/engine-gpu/src/pipelined.rs (ours) containing the double-buffered search_range.
  • GpuEngine::search_range dispatches to it behind a pipelined: bool set from a new flag --gpu-pipelined (default on once measured, off until then). Origin's run_single_batch stays byte-identical so origin fixes to it merge clean; we call it with per-set resources.
  • GpuResources gains a constructor for a second set. create_resources is already a method on GpuContext, so this is a second call, not a change.

Estimated diff in origin-owned code: the dispatch branch and the resources call, marked // lair:.

Measure

Harness: pipelined off vs on, one 5090, at 1M and 16M batch, workers-per-device = 1. Then pipelined on with workers-per-device = 2 to confirm they do not stack (they should not; if they do, the bubble was bigger than modelled and both stay).

Origin coupling

Moderate. If origin restructures search_range the merge conflicts in the one branch point. If origin ships its own pipelining, prefer theirs and delete pipelined.rs.

Part of #1. The proper fix for the same bubble the workers-per-device issue papers over. Applies to origin's wgpu engine, which stays our fallback on any host where `engine-cuda` is unavailable and is the engine on non-NVIDIA hardware. ## Current loop Per batch in `run_single_batch` (`crates/engine-gpu/src/lib.rs`): 1. four `queue.write_buffer` calls (dispatch config, start nonce, midstate, results reset) 2. encoder, compute pass, copy results to staging, `queue.submit` 3. `map_async` on staging, `device.poll(Wait)` until the callback fires 4. read 33 u32, unmap The GPU is idle from the end of step 2's execution until the next iteration's step 2 reaches the hardware. With a 2.7 ms kernel that gap is the loss. ## Design Two resource sets per worker (`GpuResources` A and B), each with its own results and staging buffer. The loop becomes: ``` submit(A, batch 0) loop: submit(B, batch n+1) // GPU now has work queued behind A wait_and_read(A, batch n) // blocks only until A's fence; B is running if found or cancelled: break swap(A, B) ``` - `map_async` for A is issued right after A's submit, so by the time we wait, it is usually already complete. - Cancellation latency becomes two batches instead of one. At 1M that is ~5 ms. Acceptable; note it in the batch-size issue's stale-work budget. - Midstate for batch n+1 is computed on the CPU while batch n runs, which it effectively already is. - Solution handling: if A reports found, B may also find one; we take A's and discard B's when it lands (the worker pool already discards later results for a job). If B is found first it just waits one iteration. ## Where the code goes This one has to touch origin's `engine-gpu`; there is no way around it since the loop is the thing. Keep the diff mergeable: - New module `crates/engine-gpu/src/pipelined.rs` (ours) containing the double-buffered `search_range`. - `GpuEngine::search_range` dispatches to it behind a `pipelined: bool` set from a new flag `--gpu-pipelined` (default on once measured, off until then). Origin's `run_single_batch` stays byte-identical so origin fixes to it merge clean; we call it with per-set resources. - `GpuResources` gains a constructor for a second set. `create_resources` is already a method on `GpuContext`, so this is a second call, not a change. Estimated diff in origin-owned code: the dispatch branch and the resources call, marked `// lair:`. ## Measure Harness: pipelined off vs on, one 5090, at 1M and 16M batch, workers-per-device = 1. Then pipelined on with workers-per-device = 2 to confirm they do not stack (they should not; if they do, the bubble was bigger than modelled and both stay). ## Origin coupling Moderate. If origin restructures `search_range` the merge conflicts in the one branch point. If origin ships its own pipelining, prefer theirs and delete `pipelined.rs`.
grenade added the needs-benchmarkorigin-couplingperf labels 2026-09-03 09:10:18 +00:00
grenade commented 2026-09-03 11:24:27 +00:00
Author
Owner
Copy Link

Measured ceiling from #9's batch phase histogram on benjy (4090, 1M batch): the host side of a batch is 0.13 ms against 6.79 ms on the GPU, 1.9% of batch time. That is the most double-buffering can recover on this card at the default batch size; about 5% on a 5090 where batches are 2.7 ms. Same ceiling applies to #4. #6 captures most of it by itself. Priority of this issue drops accordingly; it stays worth doing only as part of the wgpu fallback engine's tidy-up, after #3.

Measured ceiling from #9's batch phase histogram on benjy (4090, 1M batch): the host side of a batch is 0.13 ms against 6.79 ms on the GPU, 1.9% of batch time. That is the most double-buffering can recover on this card at the default batch size; about 5% on a 5090 where batches are 2.7 ms. Same ceiling applies to #4. #6 captures most of it by itself. Priority of this issue drops accordingly; it stays worth doing only as part of the wgpu fallback engine's tidy-up, after #3.
grenade commented 2026-09-14 12:28:41 +00:00
Author
Owner
Copy Link

Closing as superseded. The bubble this was meant to hide was measured at 1.9% of batch time on the 4090 and about 5% on a 5090, and #26 recovered what was recoverable on the engine that actually runs: batches are now whole grids, two per launch, which took beast from 2146 to 2205 MH/s (+2.8%) and benjy +1%; going to four or eight nonces per thread measured neutral to -1.2%, so the remaining gap between launches is not worth a second stream on the CUDA engine either. The quanpool kernel does run two streams, and its profile (#27) puts its advantage in instruction count and register footprint, not in launch overlap.

For the wgpu fallback this remains true in principle, but that engine now only runs on NVIDIA hosts without a CUDA driver, and touching origin's search_range for a fallback path is exactly the coupling #1 says to avoid.

Closing as superseded. The bubble this was meant to hide was measured at 1.9% of batch time on the 4090 and about 5% on a 5090, and #26 recovered what was recoverable on the engine that actually runs: batches are now whole grids, two per launch, which took beast from 2146 to 2205 MH/s (+2.8%) and benjy +1%; going to four or eight nonces per thread measured neutral to -1.2%, so the remaining gap between launches is not worth a second stream on the CUDA engine either. The quanpool kernel does run two streams, and its profile (#27) puts its advantage in instruction count and register footprint, not in launch overlap. For the wgpu fallback this remains true in principle, but that engine now only runs on NVIDIA hosts without a CUDA driver, and touching origin's `search_range` for a fallback path is exactly the coupling #1 says to avoid.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
needs-benchmark
Gain is estimated, not measured; needs a run on a 5090 host
 needs-chain
May require a matching change in quantus/chain
 origin-coupling
Touches how the lair fork tracks Quantus-Network/quantus-miner
 perf
Mining throughput workstream
No Label needs-benchmark origin-coupling perf
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
grenade
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: blackbeard/miner#5
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?