# CPU Mining Algorithm and Optimisation Roadmap This document serves as the authoritative reference for: - The CPU mining algorithm and miner job lifecycle (as originally implemented in the baseline miner). - A staged roadmap for performance optimisation (CPU/GPU) and the test/metrics infrastructure supporting this refactor. Context: - The “CPU mining algorithm” section reflects the baseline implementation before any optimisation work. - The “Optimisation roadmap” section captures the goals and scope for the refactor, guiding incremental improvements. The implementation reflects the repository’s workspace structure: - crates/miner-service: HTTP API, job orchestration, scheduling. - crates/engine-cpu: CPU engines (baseline and fast) behind a trait. - crates/pow-core: QPoW math core (compat API + precompute/fast-path helpers). - crates/metrics: Prometheus metrics and optional HTTP exporter. It preserves compatibility with the node-facing HTTP API defined by `resonance-miner-api`. --- ## Table of Contents 1. Overview 2. Core Data Types and Concepts 3. Job Lifecycle 4. Concurrency and Memory Model 5. Performance Characteristics 6. API Mapping (Node ↔ Miner) 7. CPU Optimisation Roadmap - Phase 1 (Precompute + Incremental) - Phase 2 (Montgomery Arithmetic) - Phase 3 (Micro-optimisations) - Phase 4 (Cleaner Math API Surface) 8. GPU Offload Roadmap 9. Testing, Verification, and Metrics 10. Rollout and Risk Management 11. Concrete Task Checklist 12. Open Questions 13. Expected Outcomes --- ## 1) Overview At a high level, the miner: 1. Receives a mining job from a node via `POST /mine` (header hash, threshold, nonce range). 2. Validates the request and initializes a job. 3. Partitions the nonce range across N worker threads (or device workgroups). 4. Each worker linearly scans its assigned sub-range, evaluating nonces under QPoW rules. 5. The first acceptable solution triggers an early stop via a shared cancel flag. 6. The miner aggregates progress and reports results via `GET /result/{job_id}`. 7. Jobs can be cancelled via `POST /cancel/{job_id}`; stale jobs are periodically cleaned up. Key dependencies: - U512 for the 512-bit nonce space. - crossbeam-channel for per-thread results/progress. - std::thread for CPU engines (Tokio handles HTTP and the job loop). - pow-core for QPoW math: validity and distance functions; precompute and fast path helpers. --- ## 2) Core Data Types and Concepts - MiningService (crates/miner-service) - Owns the job registry (Arc>>). - Configures the engine backend (CPU baseline/fast; GPU future). - Runs the mining loop to poll/update job status. - MiningJob (crates/miner-service) - Inputs: `header_hash: [u8; 32]`, `distance_threshold: U512`, `nonce_start: U512`, `nonce_end: U512`. - State: `status`, `start_time`, `total_hash_count`, `best_result`, `cancel_flag`, `result_receiver`, `thread_handles`, per-thread stats. - Behavior: - `start_mining(engine, num_cores)`: partitions the range, spawns workers. - `cancel()`: flips cancel flag; joins threads; transitions to Cancelled. - `update_from_results()`: drains per-thread messages; aggregates counters; picks best result; advances status. - MinerEngine trait (crates/engine-cpu) - Abstracts the compute backend (baseline or optimised CPU; GPU in future). - `prepare_context(header, threshold) -> JobContext` - `search_range(ctx, range, cancel) -> EngineStatus` (synchronous for now) - JobContext (crates/pow-core) - Precomputed per-job constants: `(m, n)`, `target`, `threshold`, `header_int`. - Supports optimised incremental evaluation via: - `init_worker_y0(ctx, start_nonce) -> y0` (one-time modular exponentiation) - `step_mul(ctx, y) -> y * m mod n` (O(1) per nonce) - `distance_from_y(ctx, y)` and `distance_for_nonce(ctx, nonce)` - ThreadResult (crates/miner-service) - Per-thread message with fields: - `thread_id`, `hash_count` (delta), optional `result`, and `completed` flag. - MiningJobResult (crates/miner-service) - The winning candidate: `nonce: U512`, `work: [u8; 64]` (big-endian nonce), `distance: U512`. - EngineStatus (crates/engine-cpu) - `Found { candidate, hash_count } | Exhausted { hash_count } | Cancelled { hash_count }` - Carries hash_count for accurate metrics, especially for the winning thread. - QPoWSeal (concept) - Encodes the sealing payload returned to the node; includes the winning `nonce` bytes. --- ## 3) Job Lifecycle 1) Submission (node → miner) - `POST /mine` → validation → construct `MiningJob`. - `MiningService::add_job` inserts the job, prepares context, starts workers. 2) Execution (internal) - `start_mining`: - Computes total range and partitions into contiguous, inclusive sub-ranges. - Spawns threads that call `mine_range_with_engine`, which delegates to the selected `engine.search_range`. - Worker threads: - Emit `ThreadResult` on completion (with `hash_count`) and with `result` if a solution is found. 3) Orchestration - The mining loop periodically: - Drains thread messages, aggregates `hash_count`, selects the best result by smallest `distance`. - Sets `status`: - Completed if a solution exists. - Failed if all threads exhausted with no solution. - Stays Running otherwise. - Cleans up stale jobs after a retention period (e.g., 5 minutes). 4) Cancellation (node → miner) - `POST /cancel/{job_id}` → flip cancel flag → join threads → `status = Cancelled`. 5) Retrieval (node → miner) - `GET /result/{job_id}` returns: - status ∈ {running, completed, failed, cancelled} - `nonce` (hex U512) and `work` ([u8; 64] hex) when completed - `hash_count` total and `elapsed_time` in all cases 6) Cleanup - Jobs not Running and older than retention are removed to bound memory use. --- ## 4) Concurrency and Memory Model - Shared state: - Jobs map guarded by a Tokio Mutex. - Per-job cancel flag (Arc, relaxed ordering). - Communication: - crossbeam-channel for per-thread results and completion statuses. - Threads: - Joined on cancellation or when a job is explicitly removed. - Bounded channels: - Sized to `num_cores * 2` to avoid unbounded memory during bursts. --- ## 5) Performance Characteristics - Baseline engine: - Per-nonce modular exponentiation dominates CPU time. - Fast engine: - Replaces per-nonce exponentiation with: - One initial exponentiation per worker (`y0 = m^(h + start_nonce)`), - Then O(1) modular multiplication per nonce (`y = y * m mod n`). - Computes `nonce_element = SHA3_512(y)` and `distance = target XOR nonce_element`. - Preserves exact correctness with dramatically reduced per-nonce cost. - Hot spots: - Modular arithmetic (multiplication and exponentiation). - SHA3/Keccak per 64-byte input. - Big-endian conversion and loop overhead. - Factors: - Work scales with `num_cores`. - Early cancel reduces waste after the first solution. --- ## 6) API Mapping (Node ↔ Miner) - `POST /mine` → `accepted | error` - `GET /result/{job_id}` → `running | completed | failed | cancelled | not_found` - `POST /cancel/{job_id}` → `cancelled | not_found` Returned fields (when applicable): - `nonce`: U512 hex (no 0x). - `work`: 128-hex chars (nonce bytes). - `hash_count`: total nonces tested for the job. - `elapsed_time`: seconds. --- ## 7) CPU Optimisation Roadmap Goal: reduce per-nonce cost from O(log exponent) to O(1) and then improve constants. Phase 1: Precompute + Incremental Evaluation - Compute `(m, n)` and `target` once per job (CPU). - For each worker/thread, compute `y0 = m^(h + start_nonce) mod n` once (sliding-window exponentiation). - Replace per-nonce `mod_pow` with `y = (y * m) mod n`. - Compute `nonce_element = sha3_512(y)` and `distance = target XOR nonce_element`; compare with `threshold`. - Benefits: - Identical correctness with massive speedup (orders of magnitude). - Status: - Architected and implemented via `pow-core::JobContext`, `init_worker_y0`, `step_mul`, `distance_from_y`. - Exposed through `engine-cpu::FastCpuEngine`. Phase 2: Fast Modular Arithmetic (Montgomery) - Rationale: per-nonce modular multiplication is now the hot path; fixed `n` per job enables Montgomery form. - Tasks: - Implement 512-bit Montgomery multiplication (e.g., 8×64-bit limbs) or swap to a fixed-limb bigint library (crypto-bigint/fiat). - Precompute Montgomery params: `R`, `R^2 mod n`, `n'`. - Keep `y` and `m` in Montgomery domain; reduce for SHA3 if required by consensus. - Expected: - >2× speedup over Phase 1 (hardware-dependent). - Status: - Planned. Phase 3: Micro-optimisations - Use fast SHA3/Keccak implementation with SIMD for 64-byte input. - Loop unrolling; batch a few iterations to reduce branch overhead. - Batch hash_count updates; reduce crossbeam traffic. - Thread affinity and NUMA awareness on multi-socket systems. - Build flags (release/LTO/CGU/panic) and tuning. - Status: - Planned (some flags applied at workspace level). Phase 4: Cleaner Math API Surface (Optional but Recommended) - `pow-core`: - Keep a `compat` API mirroring the original. - Provide fast-path APIs as first-class. - Potentially route legacy functions to the fast path internally (with care). - Status: - Partially implemented (compat + fast-path APIs exist; compatibility preserved). --- ## 8) GPU Offload Roadmap Objective: Offload the per-nonce inner loop to GPU for massive parallelism; keep orchestration and job setup on CPU. CPU-side per job: - `get_random_rsa(header)` and Miller–Rabin loop to select `n` (done once). - Precompute `target`, `threshold`, and (if used) Montgomery parameters. - Orchestrate ranges and collect results; maintain HTTP API. GPU kernel per worker: - Option A: compute `y0 = m^(h + start_nonce)` in the kernel; Option B: provide precomputed `y0`. - For each nonce: - `y = montgomery_mul(y, m, n, n')` - `nonce_element = sha3_512(y)` (convert out of Montgomery domain if necessary) - `distance = target XOR nonce_element`; compare to `threshold` - If valid, set a global flag and report solution; early-exit other threads. Implementation plan: - G1 (Prototype): - Choose backend (CUDA via cust/rustacuda or OpenCL via ocl). - Implement 512-bit Montgomery mul/reduction; integrate a fast Keccak. - Kernel inputs: `m`, `n`, Montgomery params, `target`, `threshold`, per-thread `start_nonce`, `range_len`. - Host integration: coarse-grained range partitioning, device transfers, early-cancel. - G2 (Optimise): - Occupancy tuning, register pressure, block size. - Batch multiple steps per thread; use constant memory for shared parameters. - Minimise global memory traffic; efficient found-flag handling. - G3 (Integrate/Fallback): - Feature flags; auto-detect; clean cancellation semantics; multi-GPU support. - Unified JobContext across CPU/GPU paths. --- ## 9) Testing, Verification, and Metrics Correctness - Golden tests: fixed header/threshold and small ranges; ensure baseline and fast engines yield identical winners/failures and distances. - Randomised: compare across many random jobs for small ranges. - GPU (future): cross-check GPU vs CPU fast path. Benchmarks - Measure nonces/sec per engine; time-to-first-solution across thresholds. - Resource utilisation (CPU%, memory, GPU occupancy). Metrics (Prometheus) - Global: - `miner_jobs_total{status}` - `miner_hashes_total` - `miner_hash_rate` - Engine/job-aware: - `miner_job_hashes_total{engine,job_id}` - `miner_job_hash_rate{engine,job_id}` - `miner_job_status{engine,job_id,status}` (IntGauge) - `miner_jobs_by_engine_total{engine,status}` - Engine/job/thread-aware: - `miner_thread_hashes_total{engine,job_id,thread_id}` - `miner_thread_hash_rate{engine,job_id,thread_id}` (EMA smoothed via service) - Exporter: `/metrics` is enabled only if a metrics port is configured; otherwise metrics are disabled. Dashboards (Grafana) - Operator-focused (“Miner Overview”). - Engine-focused (“Engines Comparison”). --- ## 10) Rollout and Risk Management Staged Release - Keep HTTP API unchanged; engines behind a trait. - Gate new backends behind flags; keep a baseline engine for reference. - Ship the CPU fast path first; validate; then iterate with Montgomery and/or GPU. Risk Mitigations - Big-int correctness: - Property tests; cross-check with `num-bigint` reference. - Montgomery domain boundaries: - Clear conversions; isolated code; unit tests. - SHA3/Keccak differences: - Use vetted implementations; vector test suites. - Early cancel races: - Atomic flags; careful polling; stress tests. --- ## 11) Concrete Task Checklist CPU — Phase 1 (High impact, low complexity) - [ ] Add per-job precompute: `(m, n)`, `target`, `threshold`. - [ ] Implement `init_worker_y0(header, m, n, start_nonce)` (sliding-window exponentiation). - [ ] Replace per-nonce `mod_pow` with `y = (y * m) mod n`. - [ ] Compute `nonce_element = sha3_512(y)`; `distance = target XOR nonce_element`. - [ ] Keep miner API identical; validate correctness and benchmark. CPU — Phase 2 (Montgomery) - [ ] Introduce Montgomery params: `R`, `R^2 mod n`, `n'`. - [ ] Implement 512-bit Montgomery mul/reduction. - [ ] Keep `y` and `m` in Montgomery domain; convert for SHA3 if required. - [ ] Benchmark vs Phase 1; ensure correctness. CPU — Phase 3 (Micro-tuning) - [ ] Faster SHA3 path (SIMD). - [ ] Loop unrolling and batching. - [ ] Thread affinity and NUMA tuning. - [ ] Build flags: LTO, opt-level, codegen-units, panic strategy. - [ ] Measure and document gains. Math API Surface - [ ] Add/solidify `JobContext` and helpers for precompute, init, step, distance. - [ ] Keep legacy API functional while enabling fast paths. GPU — Phases G1/G2/G3 - [ ] Choose CUDA/OpenCL; scaffolding. - [ ] Implement Montgomery mul; integrate SHA3 in kernel. - [ ] Kernel for inner loop with early-exit signal. - [ ] Host integration: ranges, transfers, cancellation. - [ ] Occupancy/memory tuning; multi-GPU; feature flags. - [ ] Cross-validate vs CPU fast path; benchmarks. Metrics and Testing - [ ] Deterministic golden tests. - [ ] Randomised cross-checks. - [ ] Benchmarks and telemetry (dashboards). Note: Some Phase 1 items may already be implemented depending on the current branch state (e.g., `pow-core::JobContext`, `init_worker_y0`, `step_mul`, and `engine-cpu::FastCpuEngine` exist). Treat this checklist as a living plan; tick off items as verified by tests/benchmarks. --- ## 12) Open Questions - qpow-math parity: - Confirm any semantic differences vs upstream (e.g., return shapes, logging). - SHA3 input domain: - Ensure consensus requires SHA3 of the standard (non-Montgomery) result of `m^(h + nonce) mod n`; apply correct domain conversion if Montgomery is used. - Threshold semantics: - Confirm endian/formatting alignment for `distance` vs `threshold` across engines. --- ## 13) Expected Outcomes - CPU fast path: large speedup by eliminating per-nonce exponentiation. - Montgomery arithmetic: further significant gains on CPU. - GPU offload: orders-of-magnitude throughput improvements where available. - Robust telemetry: engine/job/thread-level metrics for accurate comparisons and operational insight. - Safe rollout: stable HTTP API and staged engine evolution behind a clean abstraction. ---