Batch size: raise the default and derive it from measured hashrate and a stale-work budget #6
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Part of #1. Available today by flag, so the deployment repo (
lair/quantus) can set it before any code changes. This issue is about making the miner pick well on its own.Current behaviour
DEFAULT_GPU_BATCH_SIZE = 1_000_000incrates/miner-cli/src/main.rs. Origin's commit58615edmeasured micro-optimisations at 16M and saw gains that were noise at 1M, so they know larger batches help and kept 1M anyway, presumably for cancellation latency on small cards.Two things happen at 1M on a 5090 that do not happen at 16M:
get_vendor_specific_dispatchcomputes 10,922 workgroups for a Blackwell flagship (2.8M threads), butrun_single_batchclamps logical threads to the batch, so 1M nonces means ~3,900 workgroups and exactly one nonce per thread. Every thread pays the per-thread loads (midstate, target, nonce base, ~44 storage reads) for one hash. At 16M the hint binds, each thread loops six nonces, and those loads amortise. That is what the 1.3% in58615edwas.Cost of a bigger batch
A running dispatch cannot be cancelled; the cancel flag is checked between batches only. On a new job the in-flight batch is wasted. That waste per job is one batch (two, once double-buffered):
Second cost:
results[0]early-exit means a found solution stops the rest of the batch, but the worker still waits for the whole dispatch to retire. Latency from found to submitted grows with batch size. At 64M that is up to 174 ms added to seal propagation, which is a real orphan-risk cost on a fast chain. This is the actual constraint, not cancellation.Design
--gpu-batch-size 16000000on the 5090 host,8000000on the 4090. Measured, not guessed, once the harness exists.--gpu-batch-size auto. The engine measures its own MH/s over the first few batches (it already computeshash_ratefor logging) and sizes subsequent batches to a target duration,--gpu-batch-target-ms, default 20 ms. On a 5090 that is ~7M; on a 3060 ~1M; on Apple ~1M. The stale-work and found-latency costs then stay bounded by wall-clock on every card, which is the right invariant, instead of by a nonce count that means 2.7 ms on one card and 100 ms on another.[256k, 64M]. Keep the high-256-bit carry clamp that already exists.Where: batch sizing lives in the worker's
search_rangeloop. Forengine-cudait is ours. Forengine-gpuit goes in the pipelined module from the double-buffering issue, so origin'srun_single_batchstill receives a plainbatch_size: u32and is untouched.Measure
Harness sweep on one 5090: 1M, 4M, 16M, 64M, fixed workers = 1, pipelining off. Then the same with pipelining on to see how much of the batch-size gain pipelining already captures. Record found-to-submitted latency as well as MH/s; the sweet spot is where MH/s flattens, not the top.
Origin coupling
Flag plus a sizing function in our modules. Origin's default stays origin's.
Closing: settled by the sweep on the 4090 (#3, #17). Batch size only appeared to matter because the grid cap forced multiple nonces per thread above 1M; with the cap raised to keep one nonce per thread, 1M to 16M all measure 304 to 306 MH/s on the wgpu-era kernel and 1,109 to 1,121 on the 5090s with the tuned one. The default stays at 1M, which is the smallest stale cost (one batch per job switch, now about 2.4 ms on the 4090). The auto-sizing idea has nothing left to optimise.