Files
miner/docs/implementation/00-cpu-algorithm-and-optimisation-roadmap.md
T
2025-09-07 21:48:07 +08:00

15 KiB
Raw Blame History

CPU Mining Algorithm and Optimisation Roadmap

This document serves as the authoritative reference for:

  • The CPU mining algorithm and miner job lifecycle (as originally implemented in the baseline miner).
  • A staged roadmap for performance optimisation (CPU/GPU) and the test/metrics infrastructure supporting this refactor.

Context:

  • The “CPU mining algorithm” section reflects the baseline implementation before any optimisation work.
  • The “Optimisation roadmap” section captures the goals and scope for the refactor, guiding incremental improvements.

The implementation reflects the repository’s workspace structure:

  • crates/miner-service: HTTP API, job orchestration, scheduling.
  • crates/engine-cpu: CPU engines (baseline and fast) behind a trait.
  • crates/pow-core: QPoW math core (compat API + precompute/fast-path helpers).
  • crates/metrics: Prometheus metrics and optional HTTP exporter.

It preserves compatibility with the node-facing HTTP API defined by resonance-miner-api.


Table of Contents

  1. Overview
  2. Core Data Types and Concepts
  3. Job Lifecycle
  4. Concurrency and Memory Model
  5. Performance Characteristics
  6. API Mapping (Node ↔ Miner)
  7. CPU Optimisation Roadmap
    • Phase 1 (Precompute + Incremental)
    • Phase 2 (Montgomery Arithmetic)
    • Phase 3 (Micro-optimisations)
    • Phase 4 (Cleaner Math API Surface)
  8. GPU Offload Roadmap
  9. Testing, Verification, and Metrics
  10. Rollout and Risk Management
  11. Concrete Task Checklist
  12. Open Questions
  13. Expected Outcomes

1) Overview

At a high level, the miner:

  1. Receives a mining job from a node via POST /mine (header hash, threshold, nonce range).
  2. Validates the request and initializes a job.
  3. Partitions the nonce range across N worker threads (or device workgroups).
  4. Each worker linearly scans its assigned sub-range, evaluating nonces under QPoW rules.
  5. The first acceptable solution triggers an early stop via a shared cancel flag.
  6. The miner aggregates progress and reports results via GET /result/{job_id}.
  7. Jobs can be cancelled via POST /cancel/{job_id}; stale jobs are periodically cleaned up.

Key dependencies:

  • U512 for the 512-bit nonce space.
  • crossbeam-channel for per-thread results/progress.
  • std::thread for CPU engines (Tokio handles HTTP and the job loop).
  • pow-core for QPoW math: validity and distance functions; precompute and fast path helpers.

2) Core Data Types and Concepts

  • MiningService (crates/miner-service)

    • Owns the job registry (Arc<Mutex<HashMap<job_id, MiningJob>>>).
    • Configures the engine backend (CPU baseline/fast; GPU future).
    • Runs the mining loop to poll/update job status.
  • MiningJob (crates/miner-service)

    • Inputs: header_hash: [u8; 32], distance_threshold: U512, nonce_start: U512, nonce_end: U512.
    • State: status, start_time, total_hash_count, best_result, cancel_flag, result_receiver, thread_handles, per-thread stats.
    • Behavior:
      • start_mining(engine, num_cores): partitions the range, spawns workers.
      • cancel(): flips cancel flag; joins threads; transitions to Cancelled.
      • update_from_results(): drains per-thread messages; aggregates counters; picks best result; advances status.
  • MinerEngine trait (crates/engine-cpu)

    • Abstracts the compute backend (baseline or optimised CPU; GPU in future).
    • prepare_context(header, threshold) -> JobContext
    • search_range(ctx, range, cancel) -> EngineStatus (synchronous for now)
  • JobContext (crates/pow-core)

    • Precomputed per-job constants: (m, n), target, threshold, header_int.
    • Supports optimised incremental evaluation via:
      • init_worker_y0(ctx, start_nonce) -> y0 (one-time modular exponentiation)
      • step_mul(ctx, y) -> y * m mod n (O(1) per nonce)
      • distance_from_y(ctx, y) and distance_for_nonce(ctx, nonce)
  • ThreadResult (crates/miner-service)

    • Per-thread message with fields:
      • thread_id, hash_count (delta), optional result, and completed flag.
  • MiningJobResult (crates/miner-service)

    • The winning candidate: nonce: U512, work: [u8; 64] (big-endian nonce), distance: U512.
  • EngineStatus (crates/engine-cpu)

    • Found { candidate, hash_count } | Exhausted { hash_count } | Cancelled { hash_count }
    • Carries hash_count for accurate metrics, especially for the winning thread.
  • QPoWSeal (concept)

    • Encodes the sealing payload returned to the node; includes the winning nonce bytes.

3) Job Lifecycle

  1. Submission (node → miner)
  • POST /mine → validation → construct MiningJob.
  • MiningService::add_job inserts the job, prepares context, starts workers.
  1. Execution (internal)
  • start_mining:
    • Computes total range and partitions into contiguous, inclusive sub-ranges.
    • Spawns threads that call mine_range_with_engine, which delegates to the selected engine.search_range.
  • Worker threads:
    • Emit ThreadResult on completion (with hash_count) and with result if a solution is found.
  1. Orchestration
  • The mining loop periodically:
    • Drains thread messages, aggregates hash_count, selects the best result by smallest distance.
    • Sets status:
      • Completed if a solution exists.
      • Failed if all threads exhausted with no solution.
      • Stays Running otherwise.
    • Cleans up stale jobs after a retention period (e.g., 5 minutes).
  1. Cancellation (node → miner)
  • POST /cancel/{job_id} → flip cancel flag → join threads → status = Cancelled.
  1. Retrieval (node → miner)
  • GET /result/{job_id} returns:
    • status ∈ {running, completed, failed, cancelled}
    • nonce (hex U512) and work ([u8; 64] hex) when completed
    • hash_count total and elapsed_time in all cases
  1. Cleanup
  • Jobs not Running and older than retention are removed to bound memory use.

4) Concurrency and Memory Model

  • Shared state:
    • Jobs map guarded by a Tokio Mutex.
    • Per-job cancel flag (Arc, relaxed ordering).
  • Communication:
    • crossbeam-channel for per-thread results and completion statuses.
  • Threads:
    • Joined on cancellation or when a job is explicitly removed.
  • Bounded channels:
    • Sized to num_cores * 2 to avoid unbounded memory during bursts.

5) Performance Characteristics

  • Baseline engine:
    • Per-nonce modular exponentiation dominates CPU time.
  • Fast engine:
    • Replaces per-nonce exponentiation with:
      • One initial exponentiation per worker (y0 = m^(h + start_nonce)),
      • Then O(1) modular multiplication per nonce (y = y * m mod n).
    • Computes nonce_element = SHA3_512(y) and distance = target XOR nonce_element.
    • Preserves exact correctness with dramatically reduced per-nonce cost.
  • Hot spots:
    • Modular arithmetic (multiplication and exponentiation).
    • SHA3/Keccak per 64-byte input.
    • Big-endian conversion and loop overhead.
  • Factors:
    • Work scales with num_cores.
    • Early cancel reduces waste after the first solution.

6) API Mapping (Node ↔ Miner)

  • POST /mine → accepted | error
  • GET /result/{job_id} → running | completed | failed | cancelled | not_found
  • POST /cancel/{job_id} → cancelled | not_found

Returned fields (when applicable):

  • nonce: U512 hex (no 0x).
  • work: 128-hex chars (nonce bytes).
  • hash_count: total nonces tested for the job.
  • elapsed_time: seconds.

7) CPU Optimisation Roadmap

Goal: reduce per-nonce cost from O(log exponent) to O(1) and then improve constants.

Phase 1: Precompute + Incremental Evaluation

  • Compute (m, n) and target once per job (CPU).
  • For each worker/thread, compute y0 = m^(h + start_nonce) mod n once (sliding-window exponentiation).
  • Replace per-nonce mod_pow with y = (y * m) mod n.
  • Compute nonce_element = sha3_512(y) and distance = target XOR nonce_element; compare with threshold.
  • Benefits:
    • Identical correctness with massive speedup (orders of magnitude).
  • Status:
    • Architected and implemented via pow-core::JobContext, init_worker_y0, step_mul, distance_from_y.
    • Exposed through engine-cpu::FastCpuEngine.

Phase 2: Fast Modular Arithmetic (Montgomery)

  • Rationale: per-nonce modular multiplication is now the hot path; fixed n per job enables Montgomery form.
  • Tasks:
    • Implement 512-bit Montgomery multiplication (e.g., 8×64-bit limbs) or swap to a fixed-limb bigint library (crypto-bigint/fiat).
    • Precompute Montgomery params: R, R^2 mod n, n'.
    • Keep y and m in Montgomery domain; reduce for SHA3 if required by consensus.
  • Expected:
    • 2× speedup over Phase 1 (hardware-dependent).

  • Status:
    • Planned.

Phase 3: Micro-optimisations

  • Use fast SHA3/Keccak implementation with SIMD for 64-byte input.
  • Loop unrolling; batch a few iterations to reduce branch overhead.
  • Batch hash_count updates; reduce crossbeam traffic.
  • Thread affinity and NUMA awareness on multi-socket systems.
  • Build flags (release/LTO/CGU/panic) and tuning.
  • Status:
    • Planned (some flags applied at workspace level).

Phase 4: Cleaner Math API Surface (Optional but Recommended)

  • pow-core:
    • Keep a compat API mirroring the original.
    • Provide fast-path APIs as first-class.
    • Potentially route legacy functions to the fast path internally (with care).
  • Status:
    • Partially implemented (compat + fast-path APIs exist; compatibility preserved).

8) GPU Offload Roadmap

Objective: Offload the per-nonce inner loop to GPU for massive parallelism; keep orchestration and job setup on CPU.

CPU-side per job:

  • get_random_rsa(header) and Miller–Rabin loop to select n (done once).
  • Precompute target, threshold, and (if used) Montgomery parameters.
  • Orchestrate ranges and collect results; maintain HTTP API.

GPU kernel per worker:

  • Option A: compute y0 = m^(h + start_nonce) in the kernel; Option B: provide precomputed y0.
  • For each nonce:
    • y = montgomery_mul(y, m, n, n')
    • nonce_element = sha3_512(y) (convert out of Montgomery domain if necessary)
    • distance = target XOR nonce_element; compare to threshold
    • If valid, set a global flag and report solution; early-exit other threads.

Implementation plan:

  • G1 (Prototype):
    • Choose backend (CUDA via cust/rustacuda or OpenCL via ocl).
    • Implement 512-bit Montgomery mul/reduction; integrate a fast Keccak.
    • Kernel inputs: m, n, Montgomery params, target, threshold, per-thread start_nonce, range_len.
    • Host integration: coarse-grained range partitioning, device transfers, early-cancel.
  • G2 (Optimise):
    • Occupancy tuning, register pressure, block size.
    • Batch multiple steps per thread; use constant memory for shared parameters.
    • Minimise global memory traffic; efficient found-flag handling.
  • G3 (Integrate/Fallback):
    • Feature flags; auto-detect; clean cancellation semantics; multi-GPU support.
    • Unified JobContext across CPU/GPU paths.

9) Testing, Verification, and Metrics

Correctness

  • Golden tests: fixed header/threshold and small ranges; ensure baseline and fast engines yield identical winners/failures and distances.
  • Randomised: compare across many random jobs for small ranges.
  • GPU (future): cross-check GPU vs CPU fast path.

Benchmarks

  • Measure nonces/sec per engine; time-to-first-solution across thresholds.
  • Resource utilisation (CPU%, memory, GPU occupancy).

Metrics (Prometheus)

  • Global:
    • miner_jobs_total{status}
    • miner_hashes_total
    • miner_hash_rate
  • Engine/job-aware:
    • miner_job_hashes_total{engine,job_id}
    • miner_job_hash_rate{engine,job_id}
    • miner_job_status{engine,job_id,status} (IntGauge)
    • miner_jobs_by_engine_total{engine,status}
  • Engine/job/thread-aware:
    • miner_thread_hashes_total{engine,job_id,thread_id}
    • miner_thread_hash_rate{engine,job_id,thread_id} (EMA smoothed via service)
  • Exporter: /metrics is enabled only if a metrics port is configured; otherwise metrics are disabled.

Dashboards (Grafana)

  • Operator-focused (“Miner Overview”).
  • Engine-focused (“Engines Comparison”).

10) Rollout and Risk Management

Staged Release

  • Keep HTTP API unchanged; engines behind a trait.
  • Gate new backends behind flags; keep a baseline engine for reference.
  • Ship the CPU fast path first; validate; then iterate with Montgomery and/or GPU.

Risk Mitigations

  • Big-int correctness:
    • Property tests; cross-check with num-bigint reference.
  • Montgomery domain boundaries:
    • Clear conversions; isolated code; unit tests.
  • SHA3/Keccak differences:
    • Use vetted implementations; vector test suites.
  • Early cancel races:
    • Atomic flags; careful polling; stress tests.

11) Concrete Task Checklist

CPU — Phase 1 (High impact, low complexity)

  • Add per-job precompute: (m, n), target, threshold.
  • Implement init_worker_y0(header, m, n, start_nonce) (sliding-window exponentiation).
  • Replace per-nonce mod_pow with y = (y * m) mod n.
  • Compute nonce_element = sha3_512(y); distance = target XOR nonce_element.
  • Keep miner API identical; validate correctness and benchmark.

CPU — Phase 2 (Montgomery)

  • Introduce Montgomery params: R, R^2 mod n, n'.
  • Implement 512-bit Montgomery mul/reduction.
  • Keep y and m in Montgomery domain; convert for SHA3 if required.
  • Benchmark vs Phase 1; ensure correctness.

CPU — Phase 3 (Micro-tuning)

  • Faster SHA3 path (SIMD).
  • Loop unrolling and batching.
  • Thread affinity and NUMA tuning.
  • Build flags: LTO, opt-level, codegen-units, panic strategy.
  • Measure and document gains.

Math API Surface

  • Add/solidify JobContext and helpers for precompute, init, step, distance.
  • Keep legacy API functional while enabling fast paths.

GPU — Phases G1/G2/G3

  • Choose CUDA/OpenCL; scaffolding.
  • Implement Montgomery mul; integrate SHA3 in kernel.
  • Kernel for inner loop with early-exit signal.
  • Host integration: ranges, transfers, cancellation.
  • Occupancy/memory tuning; multi-GPU; feature flags.
  • Cross-validate vs CPU fast path; benchmarks.

Metrics and Testing

  • Deterministic golden tests.
  • Randomised cross-checks.
  • Benchmarks and telemetry (dashboards).

Note: Some Phase 1 items may already be implemented depending on the current branch state (e.g., pow-core::JobContext, init_worker_y0, step_mul, and engine-cpu::FastCpuEngine exist). Treat this checklist as a living plan; tick off items as verified by tests/benchmarks.


12) Open Questions

  • qpow-math parity:
    • Confirm any semantic differences vs upstream (e.g., return shapes, logging).
  • SHA3 input domain:
    • Ensure consensus requires SHA3 of the standard (non-Montgomery) result of m^(h + nonce) mod n; apply correct domain conversion if Montgomery is used.
  • Threshold semantics:
    • Confirm endian/formatting alignment for distance vs threshold across engines.

13) Expected Outcomes

  • CPU fast path: large speedup by eliminating per-nonce exponentiation.
  • Montgomery arithmetic: further significant gains on CPU.
  • GPU offload: orders-of-magnitude throughput improvements where available.
  • Robust telemetry: engine/job/thread-level metrics for accurate comparisons and operational insight.
  • Safe rollout: stable HTTP API and staged engine evolution behind a clean abstraction.