bench: quantus-bench harness and a workflow that measures on a mining host #11

Merged
grenade merged 3 commits from bench/harness into main 2026-09-03 10:19:34 +00:00

3 Commits

Author SHA1 Message Date
ca6642960c bench: serialise measurements per host with a flock, per-run file names
All checks were successful
ci / fmt (pull_request) Successful in 31s
bench / build (pull_request) Successful in 1m1s
ci / clippy (pull_request) Successful in 1m55s
bench / measure (pull_request) Successful in 2m59s
ci / test (pull_request) Successful in 9m7s
ci / doc (pull_request) Successful in 1m42s
Gitea's concurrency group did not serialise two runs on benjy: the second
run's scp failed with ETXTBSY on the binary the first was executing (safe,
it happened before the miner was touched, but the run was lost). The pause,
lock, measure and resume sequence now lives in bench-on-host.sh, executed
over one ssh call under flock on the host, with per-run binary and record
names so staging never collides.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CBgs2nSi4H2mdh8kD8vMX5
2026-09-03 13:06:45 +03:00
5f3c51c5cb bench: allow more worker threads than cards to measure submission overlap
Some checks failed
ci / fmt (pull_request) Successful in 21s
bench / build (pull_request) Successful in 56s
bench / measure (pull_request) Failing after 8s
ci / clippy (pull_request) Successful in 1m49s
ci / doc (pull_request) Successful in 1m59s
ci / test (pull_request) Successful in 7m0s
Two threads on one card overlap one thread's readback with the other's
dispatch, which is the bubble quantus/miner#4 and #5 target. Measuring it in
the harness first means #4 lands with a number instead of a hypothesis.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CBgs2nSi4H2mdh8kD8vMX5
2026-09-03 13:03:21 +03:00
bad02fbc33 bench: quantus-bench harness and a workflow that measures on a mining host
All checks were successful
ci / fmt (pull_request) Successful in 20s
bench / build (pull_request) Successful in 57s
ci / doc (pull_request) Successful in 1m46s
ci / clippy (pull_request) Successful in 2m16s
bench / measure (pull_request) Successful in 2m49s
ci / test (pull_request) Successful in 10m33s
quantus/miner#2. The measurement gate every kernel and submission change
passes before merge.

crates/bench-harness (lair-owned, never touched by origin merges):
- drives GpuEngine through MinerEngine, one worker thread per card, with
  fixed timed windows after a warm-up; reports per-worker median MH/s and
  spread and refuses (exit 5) when spread exceeds 2%
- captures nvidia-smi state (driver, power limit, draw, clock, temperature)
  before and after, and refuses to run (exit 3) when the enforced power limit
  is not the expected one, since that is the largest confound on these cards
- GPU/CPU parity on random jobs, same shape as engine-gpu's gpu_cpu_parity
  example, exit 4 on any mismatch
- one JSON record per run (schema 1) plus a markdown summary on stdout
- build.rs embeds the commit like miner-cli's

.gitea/workflows/bench.yaml:
- runner containers on the GPU hosts have no device passthrough, so the
  binary is built on cuda-13.0 (Fedora 43, matching the hosts; the rust
  image is Fedora 44 and its binaries need a newer glibc) and executed on
  the host over ssh as gitea_ci
- stops quantus-miner.service for the window and starts it again under a
  trap; refuses to measure a card that is busy with the miner stopped
- benjy (4090, dedicated) by default; host, duration, runs, batch size and
  workers are dispatch inputs; PRs touching engine or service crates trigger
  it and get the summary as a comment
- one measurement per host at a time (concurrency group)

The stop/start sudoers grants were added to lair/quantus infra-setup.sh
(lair/quantus#5) and applied to benjy and quadbrat.

Origin coupling: one line in the workspace members list, marked.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CBgs2nSi4H2mdh8kD8vMX5
2026-09-03 12:58:48 +03:00