07: stable toolchain, rust validations

- Switch toolchain to stable with clippy and rustfmt components - Clean
up Clippy warnings with inline format args and hygiene fixes - Add GPU
engine placeholders (gpu-cuda, gpu-opencl) in CLI and service - Return
clear runtime error and exit for unimplemented GPU engines - Format all
TOML files with Taplo and ensure taplo fmt --check passes - Improve
mining nonce partitioning for safe range splits - Update tests to pass
cleanly on stable without warnings
This commit is contained in:
rob thijssen
2025-09-09 21:39:01 +08:00
parent 242bbe71f6
commit 49efb4ebc5
15 changed files with 717 additions and 138 deletions
+12 -21
View File
@@ -20,37 +20,28 @@ jobs:
quality-control:
runs-on: ubuntu-latest
steps:
-
uses: actions/checkout@v4
-
name: setup rust
- uses: actions/checkout@v4
- name: setup rust
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
toolchain: ${{ env.RUSTUP_TOOLCHAIN }}
components: rustfmt, clippy
-
name: install taplo
- name: install taplo
run: cargo install taplo-cli --locked
-
name: taplo
- name: taplo
run: taplo format --check --config taplo.toml
-
name: format
- name: format
run: cargo fmt --all -- --check
timeout-minutes: 5
-
name: compile
- name: compile
run: cargo build --locked --workspace
timeout-minutes: 90
-
name: clippy
run: SKIP_WASM_BUILD=1 cargo clippy --locked --workspace
- name: clippy
run: cargo clippy --locked --workspace
timeout-minutes: 30
-
name: test
run: SKIP_WASM_BUILD=1 cargo test --locked --workspace
- name: test
run: cargo test --locked --workspace
timeout-minutes: 15
-
name: doc
run: SKIP_WASM_BUILD=1 cargo doc --locked --workspace --no-deps
- name: doc
run: cargo doc --locked --workspace --no-deps
timeout-minutes: 15
+17 -17
View File
@@ -1,12 +1,12 @@
[workspace]
members = [
"crates/miner-cli",
"crates/miner-service",
"crates/pow-core",
"crates/engine-cpu",
"crates/engine-gpu-cuda",
"crates/engine-gpu-opencl",
"crates/metrics",
"crates/engine-cpu",
"crates/engine-gpu-cuda",
"crates/engine-gpu-opencl",
"crates/metrics",
"crates/miner-cli",
"crates/miner-service",
"crates/pow-core",
]
default-members = ["crates/miner-cli"]
resolver = "2"
@@ -17,28 +17,28 @@ authors = ["Quantus Network"]
description = "Quantus External Miner Workspace"
[workspace.dependencies]
tokio = { version = "1.36", features = ["full"] }
warp = "0.3"
serde = { version = "1.0", features = ["derive"] }
serde_json = { version = "1.0.132", default-features = false }
hex = { version = "0.4.3", features = ["alloc"] }
primitive-types = { version = "0.13.1", default-features = false }
log = { version = "0.4.22", default-features = false }
env_logger = "0.10"
codec = { version = "3.6.12", default-features = false, package = "parity-scale-codec", features = ["derive", "std"] }
env_logger = "0.10"
hex = { version = "0.4.3", features = ["alloc"] }
log = { version = "0.4.22", default-features = false }
num-bigint = { version = "0.4", features = ["rand"] }
num-traits = "0.2"
primitive-types = { version = "0.13.1", default-features = false }
rand = { version = "0.8.5", default-features = false }
serde = { version = "1.0", features = ["derive"] }
serde_json = { version = "1.0.132", default-features = false }
tokio = { version = "1.36", features = ["full"] }
warp = "0.3"
# Use local pow-core crate instead of the external qpow-math; keep API compatibility at the engine layer.
# qpow-math intentionally omitted here.
resonance-miner-api = { git = "https://github.com/Quantus-Network/chain.git", package = "resonance-miner-api" }
anyhow = "1"
clap = { version = "4.5", features = ["derive", "env"] }
crossbeam-channel = "0.5"
num_cpus = "1.16"
resonance-miner-api = { git = "https://github.com/Quantus-Network/chain.git", package = "resonance-miner-api" }
sha2 = { version = "0.10", default-features = false }
sha3 = { version = "0.10", default-features = false }
thiserror = "1"
anyhow = "1"
# GPU backend dependencies are declared in individual engine crates (no optional deps in workspace)
+8 -1
View File
@@ -85,4 +85,11 @@ The miner previously used a flag named `--num-cores`. To better reflect intent,
* `POST /mine`: Submits a new mining job.
* `GET /result/{job_id}`: Retrieves the status and result of a specific mining job.
* `POST /cancel/{job_id}`: Cancels an ongoing mining job.
* `POST /cancel/{job_id}`: Cancels an ongoing mining job.
## Implementation and PR review docs
These documents provide reviewers with the authoritative context for changes. Commits and pull requests should link to the relevant prompt/response entry.
- Reviewer index: docs/implementation/readme.md
- Authoring/process guide: agents.md
+215
View File
@@ -0,0 +1,215 @@
# Agents and Authors Guide
This document defines the authoring workflow and process conventions for proposal/response documentation and how commits and PRs must link to those docs. It exists to make PR review fast, predictable, and auditable.
If you’re reviewing a PR, the canonical technical narrative lives in:
- docs/implementation/readme.md (index of iterations)
- The prompt/response pair referenced by the PR and its commits
If you’re authoring changes or operating as an agent, follow this guide.
---
## Purpose
- Keep changes small, explicit, and justified.
- Capture the plan (prompt) and the actual implementation (response) for each iterative step.
- Ensure every commit and PR references the relevant doc(s), so reviewers have full context without hunting.
---
## Where things live
- Iteration docs (prompts and responses): docs/implementation/
- Indexed in docs/implementation/readme.md (for reviewers)
- This process guide: /agents.md (for authors/agents)
- Repo overview and user-facing guidance: /README.md
---
## Numbering and naming conventions
Each iteration is a prompt/response pair using a two-digit sequence:
- 08-prompt-<short-topic>.md
- 08-response-<short-topic>.md
Conventions:
- Two-digit sequence numbers (00, 01, …, 10, 11, …) in chronological order.
- short-topic is lowercase, words separated by hyphens.
- The index is docs/implementation/readme.md (lowercase).
Examples:
- 06-prompt-manipulator-and-difficulty-overlay.md
- 06-response-manipulator-and-difficulty-overlay.md
- 07-prompt-stable-clippy-taplo-and-gpu-enum.md
- 07-response-stable-clippy-taplo-and-gpu-enum.md
---
## Authoring workflow
1) Plan the change (create a Prompt)
- File name: NN-prompt-<short-topic>.md
- Include:
- Background/context
- Objectives and acceptance criteria
- Scope of work (what will change, by crate/file if known)
- Non-goals (explicitly out of scope)
- Risks and mitigations
- Validation plan (tests, metrics, manual checks)
- CI expectations (what must pass)
2) Implement the change
- Commit in small, reviewable steps.
- Keep messages crisp and reference the prompt.
3) Document the outcome (create a Response)
- File name: NN-response-<short-topic>.md
- Include:
- Summary of changes vs. the prompt’s plan
- Key diffs by area (runtime, API, engine, metrics, tests)
- Any deviations from the prompt (and why)
- Validation results (what passed, what you measured)
- Residual risks / follow-ups
4) Update the index
- Add links to both files in docs/implementation/readme.md (reviewer index).
---
## Commit and PR linkage
Every commit must reference the iteration doc. Use this template:
Commit message template
- Title: imperative, max ~72 chars
- Body: details and motivation
- Footer: links to the prompt/response
Example:
Fix clippy lints and inline format args across miner-service
- Inline {var} format args in logs
- Remove redundant locals/fields
- No behavior changes
Docs:
- Prompt: docs/implementation/07-prompt-stable-clippy-taplo-and-gpu-enum.md
- Response: docs/implementation/07-response-stable-clippy-taplo-and-gpu-enum.md
PR description template
- What and why (summary)
- Validation (what you ran, results)
- Risks and mitigations
- Links:
- Prompt: docs/implementation/NN-prompt-<topic>.md
- Response: docs/implementation/NN-response-<topic>.md
---
## Local validation checklist (pre-PR)
Use the stable toolchain.
- Toolchain and lints
- rustup show active-toolchain -> stable
- cargo fmt --all -- --check
- cargo clippy --workspace --all-targets -- -D warnings
- TOML formatting (Taplo)
- taplo fmt --check
- Build and tests
- cargo build --workspace --locked
- cargo test --workspace --locked
- Runtime sanity (typical)
- cargo run -p miner-cli -- --engine cpu-fast --workers <n>
- Check logs and metrics if enabled (see below)
Notes:
- We standardize on stable (rust-toolchain sets channel = "stable").
- We keep components = ["clippy", "rustfmt"].
- Taplo formatting is authoritative per taplo.toml.
---
## CI expectations (what reviewers look for)
- Stable toolchain
- taplo fmt --check
- cargo fmt --all -- --check
- cargo clippy --workspace --all-targets -- -D warnings
- cargo test --workspace --locked
If PRs introduce new feature flags or targets, document how CI should build them, or gate them until CI can.
---
## Engine selection policy (naming and runtime)
Naming:
- CPU engines use Cpu-prefixed variants in the CLI to avoid future collisions and clarify behavior:
- cpu-baseline, cpu-fast, cpu-chain-manipulator
- GPU placeholders are exposed for UX and planning but currently unimplemented:
- gpu-cuda, gpu-opencl
Runtime behavior:
- If a GPU engine is selected, the service logs a clear error and exits non-zero:
- “engine 'gpu-cuda' is not implemented yet; use cpu-fast or cpu-baseline.”
- Reviewers expect this behavior to remain until real GPU engines are implemented.
---
## Metrics and observability conventions
- Metrics are feature-gated (metrics feature) and can be toggled at runtime with --metrics-port.
- Per-job and per-thread hash-rate gauges:
- Prefer remove-on-end semantics to avoid scrape-timing artifacts.
- Per-thread EMAs are summed to derive per-job hash rate.
- When changing metrics:
- Update prompt/response docs with metric names and label sets.
- Consider dashboards and alerting downstream.
---
## Style and lint conventions
- Use inline format args in log macros, e.g., "value = {value}".
- Prefer saturating arithmetic for U512 ranges; avoid panics on partitioning.
- Keep tests deterministic; avoid flakiness due to implicit ranges or 0 nonces.
- Apply rustfmt and follow Clippy guidance unless it harms readability/intent.
- TOML is formatted via Taplo per taplo.toml (tabs, key ordering for workspace.dependencies, etc.).
---
## Change management notes
- Avoid renaming CLI options unless clearly justified; when renaming, provide an alias window and document migration in the response doc.
- When adding features (e.g., new engines), default to conservative runtime behavior and explicit logs on unsupported paths.
- For system-level advice (affinity, priorities), prefer documenting in docs/implementation and cross-linking from README as needed.
---
## FAQ
Q: Why do we keep Cpu-prefixed names in the CLI?
- To prevent ambiguity once GPU engines land and to keep the mental model consistent. We suppress the enum_variant_names lint locally in the CLI.
Q: Why stable and not nightly?
- Consistency and CI portability. Stable + Clippy + Taplo cover our current needs.
Q: Why remove series on job end rather than setting 0?
- To avoid scrape-timing artifacts and enable cleaner aggregations. This is intentional and should be maintained.
Q: Where do I put process/authoring details?
- In this file (/agents.md). The docs/implementation/readme.md is optimized for PR reviewers and links here for process details.
---
## Quick links
- Reviewer index: docs/implementation/readme.md
- Process guide (this file): /agents.md
- Workspace README: /README.md
+4 -6
View File
@@ -251,7 +251,6 @@ impl MinerEngine for ChainManipulatorEngine {
// Per-block throttling: do NOT increment here. We increment on Found (i.e., when a block is solved).
let ctx = JobContext::new(header_hash, threshold);
// Debug: log current throttle state at job start
#[cfg(feature = "std")]
log::debug!(
target: "miner",
"manipulator throttle start: solved_blocks={}, sleep_ns_per_batch={}, step_batch={}",
@@ -303,18 +302,17 @@ impl MinerEngine for ChainManipulatorEngine {
if is_valid_distance(ctx, distance) {
let work = current.to_big_endian();
// Increment solved-block counter so the NEXT block throttles more.
let new_idx = self.job_index.fetch_add(1, AtomicOrdering::Relaxed) + 1;
#[cfg(feature = "std")]
let _new_idx = self.job_index.fetch_add(1, AtomicOrdering::Relaxed) + 1;
{
let capped = if let Some(cap) = self.throttle_cap {
std::cmp::min(new_idx, cap)
std::cmp::min(_new_idx, cap)
} else {
new_idx
_new_idx
};
log::debug!(
target: "miner",
"manipulator throttle increment: solved_blocks={} (next sleep_ns_per_batch={}, cap={:?})",
new_idx,
_new_idx,
self.base_delay_ns.saturating_mul(capped),
self.throttle_cap
);
+1 -1
View File
@@ -10,7 +10,7 @@ description = "Metrics and Prometheus exporter utilities for the Quantus Externa
default = ["registry"]
registry = []
# Enable a Warp-based HTTP exporter (used when --metrics-port is provided).
http-exporter = ["dep:warp", "dep:tokio", "dep:serde", "dep:serde_json"]
http-exporter = ["dep:serde", "dep:serde_json", "dep:tokio", "dep:warp"]
[dependencies]
anyhow = { workspace = true }
-1
View File
@@ -124,7 +124,6 @@ static HTTP_REQUESTS_TOTAL: Lazy<IntCounterVec> = Lazy::new(|| {
/// Access the global Prometheus registry used by this crate.
// Engine-aware, per-job, and per-thread labeled metrics
static JOB_HASHES_TOTAL: Lazy<IntCounterVec> = Lazy::new(|| {
let c = IntCounterVec::new(
opts!(
+12 -7
View File
@@ -39,11 +39,14 @@ struct Args {
#[arg(long = "manip-throttle-cap", env = "MINER_MANIP_THROTTLE_CAP")]
manip_throttle_cap: Option<u64>,
/// Mining engine to use (default: cpu-fast). Options: cpu-baseline, cpu-fast, cpu-chain-manipulator
/// Mining engine to use (default: cpu-fast).
/// Options: cpu-baseline, cpu-fast, cpu-chain-manipulator, gpu-cuda, gpu-opencl
/// Note: GPU engines are currently unimplemented and will return a clear error at runtime.
#[arg(long, env = "MINER_ENGINE", value_enum, default_value_t = EngineCli::CpuFast)]
engine: EngineCli,
}
#[allow(clippy::enum_variant_names)]
#[derive(Copy, Clone, Debug, Eq, PartialEq, ValueEnum)]
enum EngineCli {
/// Baseline CPU engine (reference implementation)
@@ -52,8 +55,10 @@ enum EngineCli {
CpuFast,
/// Throttling CPU engine that slows per block to help reduce difficulty
CpuChainManipulator,
// Cuda, // planned: CUDA GPU engine
// Opencl, // planned: OpenCL GPU engine
/// CUDA GPU engine (unimplemented; selecting will return an error)
GpuCuda,
/// OpenCL GPU engine (unimplemented; selecting will return an error)
GpuOpencl,
}
impl From<EngineCli> for EngineSelection {
@@ -62,8 +67,8 @@ impl From<EngineCli> for EngineSelection {
EngineCli::CpuBaseline => EngineSelection::CpuBaseline,
EngineCli::CpuFast => EngineSelection::CpuFast,
EngineCli::CpuChainManipulator => EngineSelection::CpuChainManipulator,
// EngineCli::Cuda => EngineSelection::Cuda,
// EngineCli::Opencl => EngineSelection::OpenCl,
EngineCli::GpuCuda => EngineSelection::GpuCuda,
EngineCli::GpuOpencl => EngineSelection::GpuOpenCl,
}
}
}
@@ -83,14 +88,14 @@ async fn main() {
log::info!("Starting external miner service...");
log::info!("API listening port: {}", args.port);
match args.workers {
Some(n) if n > 0 => log::info!("Using specified number of workers: {}", n),
Some(n) if n > 0 => log::info!("Using specified number of workers: {n}"),
Some(_) => {
log::warn!("Workers must be positive. Defaulting to all available logical CPUs.")
}
None => log::info!("Using all available logical CPUs (auto-detected)."),
}
match args.metrics_port {
Some(p) => log::info!("Metrics enabled on port {}", p),
Some(p) => log::info!("Metrics enabled on port {p}"),
None => log::info!("Metrics disabled (no --metrics-port provided)"),
}
log::info!("Selected engine: {:?}", args.engine);
+160 -61
View File
@@ -44,8 +44,8 @@ pub enum EngineSelection {
CpuBaseline,
CpuFast,
CpuChainManipulator,
// Cuda,
// OpenCl,
GpuCuda,
GpuOpenCl,
}
impl Default for ServiceConfig {
@@ -90,7 +90,7 @@ impl MiningService {
pub async fn add_job(&self, job_id: String, mut job: MiningJob) -> Result<(), String> {
let mut jobs = self.jobs.lock().await;
if jobs.contains_key(&job_id) {
log::warn!("Attempted to add duplicate job ID: {}", job_id);
log::warn!("Attempted to add duplicate job ID: {job_id}");
return Err("Job already exists".to_string());
}
@@ -122,7 +122,7 @@ impl MiningService {
pub async fn remove_job(&self, job_id: &str) -> Option<MiningJob> {
let mut jobs = self.jobs.lock().await;
if let Some(mut job) = jobs.remove(job_id) {
log::debug!(target: "miner", "Removing job: {}", job_id);
log::debug!(target: "miner", "Removing job: {job_id}");
job.cancel();
Some(job)
} else {
@@ -164,7 +164,7 @@ impl MiningService {
let retain = job.status == JobStatus::Running
|| job.start_time.elapsed().as_secs() < 300;
if !retain {
log::debug!(target: "miner", "Cleaning up old job {}", job_id);
log::debug!(target: "miner", "Cleaning up old job {job_id}");
}
retain
});
@@ -306,37 +306,25 @@ impl MiningJob {
self.result_receiver = Some(receiver);
self.engine_name = engine.name();
let total_range = (self.nonce_end - self.nonce_start).saturating_add(U512::from(1));
let range_per_worker = total_range / U512::from(workers as u64);
let remainder = total_range % U512::from(workers as u64);
// Partition nonce range safely
let partitions = compute_partitions(self.nonce_start, self.nonce_end, workers);
log::debug!(
target: "miner",
"Starting mining with {} workers, total range: {}, range per worker: {}",
"Starting mining with {} workers, total range: {} ({} partitions)",
workers,
total_range,
range_per_worker
partitions.total_range,
partitions.ranges.len()
);
// Prepare shared job context once per job.
let ctx = engine.prepare_context(self.header_hash, self.distance_threshold);
for thread_id in 0..workers {
let start = self.nonce_start + range_per_worker * U512::from(thread_id as u64);
let mut end = start + range_per_worker - U512::from(1);
if thread_id == workers - 1 {
end += remainder;
}
if end > self.nonce_end {
end = self.nonce_end;
}
for (thread_id, (start, end)) in partitions.ranges.into_iter().enumerate() {
let cancel_flag = self.cancel_flag.clone();
let sender = sender.clone();
let ctx = ctx.clone();
let engine = engine.clone();
let progress_chunk_ms = progress_chunk_ms;
let handle = thread::spawn(move || {
mine_range_with_engine(
@@ -601,7 +589,7 @@ fn mine_range_with_engine(
completed: true,
};
if sender.send(final_result).is_err() {
log::warn!("Thread {} failed to send final result", thread_id);
log::warn!("Thread {thread_id} failed to send final result");
}
done = true;
break;
@@ -615,7 +603,7 @@ fn mine_range_with_engine(
completed: false,
};
if sender.send(update).is_err() {
log::warn!("Thread {} failed to send progress update", thread_id);
log::warn!("Thread {thread_id} failed to send progress update");
break;
}
}
@@ -628,7 +616,7 @@ fn mine_range_with_engine(
completed: false,
};
if sender.send(update).is_err() {
log::warn!("Thread {} failed to send cancel update", thread_id);
log::warn!("Thread {thread_id} failed to send cancel update");
}
done = true;
break;
@@ -653,14 +641,11 @@ fn mine_range_with_engine(
completed: true,
};
if sender.send(final_result).is_err() {
log::warn!(
"Thread {} failed to send completion status after chunked search",
thread_id
);
log::warn!("Thread {thread_id} failed to send completion status after chunked search");
}
}
log::debug!("Thread {} completed.", thread_id);
log::debug!("Thread {thread_id} completed.");
}
/// Validates incoming mining requests for structural correctness.
@@ -704,7 +689,7 @@ pub async fn handle_mine_request(
request: MiningRequest,
state: MiningService,
) -> Result<impl Reply, Rejection> {
log::debug!("Received mine request: {:?}", request);
log::debug!("Received mine request: {request:?}");
if let Err(e) = validate_mining_request(&request) {
log::warn!("Invalid mine request ({}): {}", request.job_id, e);
#[cfg(feature = "metrics")]
@@ -775,12 +760,12 @@ pub async fn handle_result_request(
job_id: String,
state: MiningService,
) -> Result<impl Reply, Rejection> {
log::debug!("Received result request for job: {}", job_id);
log::debug!("Received result request for job: {job_id}");
let job = match state.get_job(&job_id).await {
Some(job) => job,
None => {
log::warn!("Result request for unknown job: {}", job_id);
log::warn!("Result request for unknown job: {job_id}");
return Ok(warp::reply::with_status(
warp::reply::json(&resonance_miner_api::MiningResult {
status: ApiResponseStatus::NotFound,
@@ -814,7 +799,7 @@ pub async fn handle_result_request(
Ok(warp::reply::with_status(
warp::reply::json(&resonance_miner_api::MiningResult {
status: status,
status,
job_id,
nonce,
work,
@@ -830,10 +815,10 @@ pub async fn handle_cancel_request(
job_id: String,
state: MiningService,
) -> Result<impl Reply, Rejection> {
log::debug!("Received cancel request for job: {}", job_id);
log::debug!("Received cancel request for job: {job_id}");
if state.cancel_job(&job_id).await {
log::debug!(target: "miner", "Successfully cancelled job: {}", job_id);
log::debug!(target: "miner", "Successfully cancelled job: {job_id}");
Ok(warp::reply::with_status(
warp::reply::json(&MiningResponse {
status: ApiResponseStatus::Cancelled,
@@ -843,7 +828,7 @@ pub async fn handle_cancel_request(
warp::http::StatusCode::OK,
))
} else {
log::warn!("Cancel request for unknown job: {}", job_id);
log::warn!("Cancel request for unknown job: {job_id}");
Ok(warp::reply::with_status(
warp::reply::json(&MiningResponse {
status: ApiResponseStatus::NotFound,
@@ -883,6 +868,44 @@ pub fn build_routes(
mine_route.or(result_route).or(cancel_route)
}
/// Helper structures and functions for safe range partitioning (placed before use)
#[derive(Debug, Clone)]
struct Partitions {
total_range: U512,
ranges: Vec<(U512, U512)>,
}
/// Compute safe, inclusive partitions of [start, end] into `workers` slices.
/// Guarantees coverage without overflow and clamps to `end`.
fn compute_partitions(start: U512, end: U512, workers: usize) -> Partitions {
// total inclusive range
let total_range = end.saturating_sub(start).saturating_add(U512::from(1u64));
let workers = workers.max(1);
let divisor = U512::from(workers as u64).max(U512::from(1u64));
let range_per = total_range / divisor;
let remainder = total_range % divisor;
let mut ranges = Vec::with_capacity(workers);
for i in 0..workers {
let idx = U512::from(i as u64);
let s = start.saturating_add(range_per.saturating_mul(idx));
let mut e = s.saturating_add(range_per).saturating_sub(U512::from(1u64));
if i == workers - 1 {
e = e.saturating_add(remainder);
}
if e > end {
e = end;
}
ranges.push((s, e));
}
Partitions {
total_range,
ranges,
}
}
/// Start the miner service with the given configuration.
/// - Spawns the mining loop.
/// - Optionally exposes a metrics endpoint if `metrics_port` is provided and the `metrics` feature is enabled.
@@ -932,6 +955,7 @@ pub async fn run(config: ServiceConfig) -> anyhow::Result<()> {
}
Some(count)
}
// keep run(config) open; do not close here
// Detect effective CPU pool for this process (cpuset if available).
let effective_cpus = detect_effective_cpus();
@@ -948,10 +972,7 @@ pub async fn run(config: ServiceConfig) -> anyhow::Result<()> {
Some(n) if n > 0 => {
if n > effective_cpus {
log::warn!(
"Requested {} workers exceeds available logical CPUs in cpuset ({}). Clamping to {}.",
n,
effective_cpus,
effective_cpus
"Requested {n} workers exceeds available logical CPUs in cpuset ({effective_cpus}). Clamping to {effective_cpus}."
);
effective_cpus
} else {
@@ -979,9 +1000,7 @@ pub async fn run(config: ServiceConfig) -> anyhow::Result<()> {
}
log::info!(
"Using {} worker thread(s) for mining (effective logical CPUs available: {})",
workers,
effective_cpus
"Using {workers} worker thread(s) for mining (effective logical CPUs available: {effective_cpus})"
);
// Select engine
@@ -993,24 +1012,34 @@ pub async fn run(config: ServiceConfig) -> anyhow::Result<()> {
let mut eng = engine_cpu::ChainEngine::new();
// Apply optional throttle parameters if provided.
if let Some(base) = config.manip_base_delay_ns {
log::debug!(target: "miner", "Manipulator base_delay_ns overridden via config: {} ns", base);
log::debug!(target: "miner", "Manipulator base_delay_ns overridden via config: {base} ns");
eng.base_delay_ns = base;
}
if let Some(step) = config.manip_step_batch {
log::debug!(target: "miner", "Manipulator step_batch overridden via config: {}", step);
log::debug!(target: "miner", "Manipulator step_batch overridden via config: {step}");
eng.step_batch = step;
}
if let Some(cap) = config.manip_throttle_cap {
log::debug!(target: "miner", "Manipulator throttle_cap set via config: {}", cap);
log::debug!(target: "miner", "Manipulator throttle_cap set via config: {cap}");
eng.throttle_cap = Some(cap);
}
// If a starting throttle index is provided, set it here for "pick up where we left off".
if let Some(n) = config.manip_solved_blocks {
log::debug!(target: "miner", "Manipulator starting throttle index (solved_blocks) set via config: {}", n);
log::debug!(target: "miner", "Manipulator starting throttle index (solved_blocks) set via config: {n}");
eng.job_index.store(n, std::sync::atomic::Ordering::Relaxed);
}
Arc::new(eng)
}
EngineSelection::GpuCuda => {
log::error!("Requested engine gpu-cuda is not implemented yet. Use a CPU engine (cpu-fast or cpu-baseline) for now.");
return Err(anyhow::anyhow!("engine 'gpu-cuda' is not implemented yet"));
}
EngineSelection::GpuOpenCl => {
log::error!("Requested engine gpu-opencl is not implemented yet. Use a CPU engine (cpu-fast or cpu-baseline) for now.");
return Err(anyhow::anyhow!(
"engine 'gpu-opencl' is not implemented yet"
));
}
};
log::info!("Using engine: {}", engine.name());
@@ -1024,14 +1053,13 @@ pub async fn run(config: ServiceConfig) -> anyhow::Result<()> {
if let Some(port) = config.metrics_port {
#[cfg(feature = "metrics")]
{
log::info!("Starting metrics endpoint on 0.0.0.0:{}", port);
log::info!("Starting metrics endpoint on 0.0.0.0:{port}");
metrics::start_http_exporter(port).await?;
}
#[cfg(not(feature = "metrics"))]
{
log::warn!(
"Metrics port provided ({}), but 'metrics' feature is not enabled. Skipping.",
port
"Metrics port provided ({port}), but 'metrics' feature is not enabled. Skipping."
);
}
} else {
@@ -1044,7 +1072,7 @@ pub async fn run(config: ServiceConfig) -> anyhow::Result<()> {
// Start server
let addr = ([0, 0, 0, 0], config.port);
let socket = std::net::SocketAddr::from(addr);
log::info!("Server starting on {}", socket);
log::info!("Server starting on {socket}");
warp::serve(routes).run(socket).await;
Ok(())
@@ -1052,13 +1080,17 @@ pub async fn run(config: ServiceConfig) -> anyhow::Result<()> {
#[cfg(test)]
mod tests {
use super::*;
use super::{JobStatus, MiningJob, MiningService};
use engine_cpu::MinerEngine;
use primitive_types::U512;
use std::sync::Arc;
use tokio::time::{sleep, Duration};
#[tokio::test]
async fn test_mining_state_add_get_remove() {
let engine: Arc<dyn MinerEngine> = Arc::new(engine_cpu::BaselineCpuEngine::new());
let state = MiningService::new(2, engine);
let state = MiningService::new(2, engine, 2000);
let job = MiningJob::new(
[1u8; 32],
@@ -1074,15 +1106,16 @@ mod tests {
#[tokio::test]
async fn test_job_lifecycle_fail() {
// Test that a job fails if no nonce is found (threshold too strict)
// Test that a job fails if no nonce is found (threshold too strict).
// To make this deterministic, avoid nonce=0, which some math paths treat as special.
let engine: Arc<dyn MinerEngine> = Arc::new(engine_cpu::BaselineCpuEngine::new());
let state = MiningService::new(2, engine);
let state = MiningService::new(2, engine, 2000);
state.start_mining_loop().await;
// Impossible threshold
// Impossible threshold with a nonce range that excludes 0
let header_hash = [1u8; 32];
let distance_threshold = U512::zero();
let nonce_start = U512::from(0);
let nonce_start = U512::from(1);
let nonce_end = U512::from(100);
let job = MiningJob::new(header_hash, distance_threshold, nonce_start, nonce_end);
@@ -1101,14 +1134,80 @@ mod tests {
let finished_job = finished_job.expect("Job did not finish in time");
assert_eq!(finished_job.status, JobStatus::Failed);
assert!(finished_job.total_hash_count >= 0);
}
#[cfg(test)]
mod partition_tests {
use crate::compute_partitions;
use primitive_types::U512;
fn u(x: u64) -> U512 {
U512::from(x)
}
#[test]
fn partitions_cover_entire_range_even_workers() {
// Range [0, 99] split into 4 workers
let p = compute_partitions(u(0), u(99), 4);
assert_eq!(p.total_range, u(100));
// Check coverage and ordering
let mut covered = 0u64;
let mut prev_end = u(0);
for (i, (s, e)) in p.ranges.iter().enumerate() {
if i == 0 {
assert_eq!(*s, u(0));
} else {
assert_eq!(*s, prev_end.saturating_add(u(1)));
}
assert!(e >= s);
covered += (e.saturating_sub(*s).saturating_add(u(1))).as_u64();
prev_end = *e;
}
assert_eq!(covered, 100);
assert_eq!(prev_end, u(99));
}
#[test]
fn partitions_cover_entire_range_odd_workers() {
// Range [0, 99] split into 3 workers
let p = compute_partitions(u(0), u(99), 3);
assert_eq!(p.total_range, u(100));
// Ensure last range takes remainder and we end exactly at 99
assert_eq!(p.ranges.len(), 3);
assert_eq!(p.ranges[0], (u(0), u(32)));
assert_eq!(p.ranges[1], (u(33), u(65)));
assert_eq!(p.ranges[2], (u(66), u(99)));
}
#[test]
fn partitions_handles_start_eq_end() {
// Range [42, 42] split into any workers -> only last partition reaches the end
let p = compute_partitions(u(42), u(42), 5);
assert_eq!(p.total_range, u(1));
assert_eq!(p.ranges.len(), 5);
// All but the last partition may have end < start (empty), but no overflow, and last ends at 42
assert_eq!(p.ranges[4].1, u(42));
}
#[test]
fn partitions_huge_values_no_overflow() {
// Use near-max U512 values to ensure no overflow
let start = U512::from(0);
let end = U512::MAX.saturating_sub(u(10_000)); // keep a finite total_range
let p = compute_partitions(start, end, 3);
// Check that each end <= original end
for (s, e) in p.ranges {
assert!(e >= s);
assert!(e <= end);
}
}
}
#[tokio::test]
async fn test_job_lifecycle_success() {
// Test that a job completes successfully
let engine: Arc<dyn MinerEngine> = Arc::new(engine_cpu::BaselineCpuEngine::new());
let state = MiningService::new(2, engine);
let state = MiningService::new(2, engine, 2000);
state.start_mining_loop().await;
// Easy threshold
+6 -6
View File
@@ -4,8 +4,8 @@ version = "0.1.0"
edition = "2021"
description = "Local QPoW math core for the Quantus external miner (reference + optimized paths)"
license = "MIT OR Apache-2.0"
keywords = ["quantus", "qpow", "pow", "crypto", "bigint"]
categories = ["cryptography", "algorithms", "no-std"]
keywords = ["bigint", "crypto", "pow", "qpow", "quantus"]
categories = ["algorithms", "cryptography", "no-std"]
[lib]
path = "src/lib.rs"
@@ -13,13 +13,13 @@ crate-type = ["rlib"]
[features]
# Enable std by default; allows logging and faster hash crates.
default = ["std", "baseline"]
default = ["baseline", "std"]
# std feature wires through to dependencies that have separate std/no_std builds.
std = [
"primitive-types/std",
"sha2/std",
"sha3/std",
"primitive-types/std",
"sha2/std",
"sha3/std",
]
# Baseline/reference implementation using BigUint-backed modular exponentiation.
+5 -15
View File
@@ -44,20 +44,14 @@ pub mod compat {
if nonce == [0u8; 64] {
#[cfg(feature = "std")]
error!(
"is_valid_nonce should not be called with 0 nonce, but was for header: {:?}",
header
"is_valid_nonce should not be called with 0 nonce, but was for header: {header:?}"
);
return (false, U512::zero());
}
let distance_achieved = get_nonce_distance(header, nonce);
#[cfg(feature = "std")]
debug!(
target: "pow-core",
"distance = {}..., threshold = {}...",
distance_achieved,
threshold
);
debug!(target: "pow-core", "distance = {distance_achieved}..., threshold = {threshold}...");
(distance_achieved <= threshold, distance_achieved)
}
@@ -190,7 +184,7 @@ fn get_nonce_distance_impl(header: [u8; 32], nonce: [u8; 64]) -> U512 {
let distance = target.bitxor(nonce_element);
#[cfg(feature = "std")]
debug!(target: "pow-core", "distance = {}", distance);
debug!(target: "pow-core", "distance = {distance}");
distance
}
@@ -198,7 +192,7 @@ fn get_nonce_distance_impl(header: [u8; 32], nonce: [u8; 64]) -> U512 {
///
/// - m: 256-bit derived via SHA2-256(header)
/// - n: 512-bit derived via SHA3-512(header), iteratively rehashed until valid:
/// (odd, composite, coprime with m, and n > m)
/// (odd, composite, coprime with m, and n > m)
fn get_random_rsa_impl(header: &[u8; 32]) -> (U512, U512) {
use sha2::{Digest, Sha256};
use sha3::Sha3_512;
@@ -383,12 +377,8 @@ fn sha3_512_impl(input: U512) -> U512 {
#[cfg(test)]
mod tests {
use super::*;
use hex::FromHex;
fn u512_from_hex(s: &str) -> U512 {
let bytes = <[u8; 64]>::from_hex(s).unwrap();
U512::from_big_endian(&bytes)
}
// removed unused helper u512_from_hex
#[test]
fn compat_distance_matches_context_distance() {
@@ -0,0 +1,126 @@
# 07 Prompt: Move to Stable, Clippy/Taplo Hygiene, and GPU Engine Enum
This prompt follows from 06 (Manipulator + Difficulty Overlay). It focuses on getting the repository clean and future-proofed for CI by standardizing the toolchain, enforcing static checks, and aligning engine naming for upcoming GPU work.
## Background
- We have mature CPU engines (baseline and fast), metrics, and a throttling engine.
- CI currently runs Taplo (TOML formatter) checks; we also want to rely on stable Rust and Clippy hygiene.
- The engine naming convention uses CPU prefixes; we will soon add GPU offloading. We want to preserve that naming and add GPU variants in a forward-compatible way.
## Objectives
1) Standardize the toolchain on stable
- Switch the repo’s toolchain to stable (rust-toolchain: channel = "stable").
- Keep useful components in the toolchain (clippy, rustfmt).
- Remove the unused wasm target from toolchain configuration.
2) Enforce Clippy hygiene across the workspace
- Run cargo clippy --workspace --all-targets and fix warnings (prefer inline format args, etc.).
- Keep Cpu-prefixed CLI variants (e.g., CpuFast) to preserve clarity for future GPU engines; suppress the enum_variant_names lint locally (not globally).
- Avoid code “simplifications” that would reduce maintainability; target idiomatic fixes only.
3) Ensure Taplo formatting checks pass
- Taplo is already configured via taplo.toml.
- Run taplo fmt and taplo fmt --check to ensure all TOML files comply (workspace Cargo.toml and all crate Cargo.toml files).
- Accept any Taplo-driven reordering or indentation changes as authoritative.
4) Add GPU engine variants (forward-compatible placeholders)
- Extend CLI engine enum with gpu-cuda and gpu-opencl variants.
- Extend service EngineSelection with GpuCuda and GpuOpenCl.
- If a user selects a GPU engine, fail fast at runtime with a clear error message and non-zero exit, explaining the engine is unimplemented and to use CPU engines for now.
- Update CLI help text to indicate GPU options are unimplemented.
5) Fix warnings and tighten tests
- Address any outstanding compiler warnings in tests (unused imports/helpers, useless comparisons).
- Ensure cargo test --locked --workspace passes cleanly on stable.
6) CI alignment (Taplo + Clippy + fmt + tests)
- Ensure CI pipeline covers:
- taplo fmt --check
- cargo fmt -- --check (if used)
- cargo clippy --workspace --all-targets -- -D warnings
- cargo test --locked --workspace
- Confirm the pipeline uses stable toolchain.
## Scope of Work
- Toolchain:
- Update rust-toolchain to stable and remove unused wasm target(s).
- Code hygiene:
- Apply clippy-driven edits (inline format args, remove redundant locals/fields, fix docs indentation, remove unused test helpers/imports).
- Preserve CPU engine variant names and add #[allow(clippy::enum_variant_names)] on the CLI enum.
- GPU placeholders:
- CLI: add GpuCuda, GpuOpencl variants with descriptive help; maintain Cpu-prefixed variants.
- Service: add EngineSelection::GpuCuda, ::GpuOpenCl; in run(), return a descriptive error when selected.
- Taplo:
- Format all TOML files to pass taplo fmt --check.
- Tests:
- Ensure all unit tests pass on stable.
- Documentation:
- Add this prompt (07-prompt-stable-clippy-taplo-and-gpu-enum.md).
- Add a corresponding response document (07-response-stable-clippy-taplo-and-gpu-enum.md) summarizing changes applied and their outcomes.
## Acceptance Criteria
- rust-toolchain uses stable with clippy and rustfmt; no wasm targets remain unless justified.
- cargo test --locked --workspace passes on stable with zero warnings from rustc.
- cargo clippy --workspace --all-targets returns clean (ideally with -D warnings in CI).
- taplo fmt --check passes.
- CLI supports engine values:
- cpu-baseline, cpu-fast, cpu-chain-manipulator (existing, fully functional)
- gpu-cuda, gpu-opencl (new, unimplemented; selecting them yields a clear error and non-zero exit)
- Service run(config) surfaces a meaningful error message (and exit) for unimplemented engines.
- Documentation pair (07 prompt/response) added under docs/implementation.
## Non-Goals
- Implementing the actual GPU engines.
- Changing engine behavior or metrics pipelines beyond hygiene/UX improvements.
- Introducing new config options unrelated to the scope above.
## Risks and Mitigations
- Risk: Clippy or Taplo changes could clash with future code-gen or formatting preferences.
- Mitigation: Keep changes minimal and idiomatic; follow Taplo’s configuration as the source of truth.
- Risk: Users expect GPU to work once they see options.
- Mitigation: Explicitly label GPU engines as unimplemented in CLI help and fail with clear runtime errors.
## Suggested Developer Workflow
1) Toolchain
- Update rust-toolchain to stable, keeping clippy/rustfmt, and remove wasm target(s).
2) Hygiene & Formatting
- cargo fmt (if used)
- cargo clippy --workspace --all-targets
- Fix issues (focus on inline format args, remove unused imports, doc indentation).
- taplo fmt, then taplo fmt --check
3) Tests & Runtime
- cargo test --locked --workspace
- Run the CLI:
- quantus-miner --engine cpu-fast (runs fine)
- quantus-miner --engine gpu-cuda (clear error message and non-zero exit)
4) CI
- Ensure pipeline includes taplo fmt --check, clippy -D warnings, and tests on stable.
## Deliverables
- Code changes:
- Updated rust-toolchain
- Clippy-driven fixes across crates
- GPU engine placeholders in CLI and service with descriptive runtime errors
- Taplo-formatted TOMLs
- Documentation:
- docs/implementation/07-prompt-stable-clippy-taplo-and-gpu-enum.md (this file)
- docs/implementation/07-response-stable-clippy-taplo-and-gpu-enum.md (companion response)
- CI (if updated in this iteration):
- Workflow(s) set to stable, with taplo fmt --check, clippy, fmt check, and tests
## References
- 06-prompt-manipulator-and-difficulty-overlay.md
- 06-response-manipulator-and-difficulty-overlay.md
- taplo.toml (TOML formatting rules)
@@ -0,0 +1,106 @@
# 07 Response: Stable Toolchain, Clippy/Taplo Hygiene, and GPU Engine Enum
This document summarizes the work completed after 06 (Manipulator + Difficulty Overlay) to standardize the toolchain on stable, clean up Clippy warnings, enforce Taplo formatting, and prepare the CLI/service for future GPU engines with clear user-facing behavior.
## Summary
- Moved the repository from nightly to stable toolchain, retaining useful components (clippy, rustfmt) and removing an unused wasm target.
- Cleaned up Clippy warnings across the workspace with minimal, idiomatic changes.
- Ensured Taplo formatting passes for all TOML files according to the existing `taplo.toml`.
- Preserved the CPU-prefixed engine variant naming (important for future GPU parity) and added GPU engine placeholders to both CLI and service.
- Selecting a GPU engine now produces a clear runtime error and exits non-zero, making the current status explicit to users.
- Verified that tests, Clippy, and Taplo checks pass on stable.
---
## Changes Made
### 1) Toolchain
- Switched to stable toolchain:
- `channel = "stable"`
- `components = ["clippy", "rustfmt"]`
- Removed the unused wasm target.
- Verified: `cargo +stable test --locked --workspace` completes successfully.
### 2) Clippy Hygiene
- Converted logging to inline format arguments across crates (miner-service, miner-cli, pow-core).
- Removed redundant locals and redundant field names in responses.
- Fixed doc-list indentation in pow-core for Clippy’s doc lints.
- Removed unused helper/imports in tests that caused warnings.
- Preserved CPU-prefixed CLI variants but silenced `enum_variant_names` locally on the enum, not globally:
- CLI enum variants: `CpuBaseline`, `CpuFast`, `CpuChainManipulator` retained.
- Verified: `cargo clippy --workspace --all-targets` runs cleanly on stable.
### 3) Taplo Formatting
- Ran `taplo fmt` across workspace TOML files and verified `taplo fmt --check` passes.
- Accepted Taplo-driven reordering/indentation as authoritative per `taplo.toml`.
### 4) GPU Engine Placeholders and UX
- CLI additions:
- Added `GpuCuda` and `GpuOpencl` as enum variants.
- Updated CLI help text to list gpu engines and note they are unimplemented.
- Kept CPU-prefixed variants to maintain forward compatibility with GPU engine naming.
- Service changes:
- Added `EngineSelection::GpuCuda` and `EngineSelection::GpuOpenCl`.
- In `run(config)`, selecting a GPU engine logs a clear error and returns `Err(anyhow!(...))`. The CLI then exits with a non-zero code and the descriptive message.
- Example user experience:
- `--engine gpu-cuda` -> logs “engine 'gpu-cuda' is not implemented yet, use cpu-fast or cpu-baseline” and exits non-zero.
- `--engine cpu-fast` -> normal operation.
### 5) Tests
- Adjusted tests to remove useless comparisons and unused imports.
- Ensured all tests pass on stable: `cargo +stable test --locked --workspace` is green.
---
## Impact
- The project now builds and tests cleanly on stable Rust with Clippy and Taplo checks, aligning with typical CI constraints.
- CPU engine naming convention is retained, preventing churn when GPU engines land, and keeping UX consistent.
- Users receive explicit, actionable errors when selecting a not-yet-implemented GPU engine, reducing confusion.
---
## Usage Notes
- CLI engine options:
- Implemented: `cpu-baseline`, `cpu-fast`, `cpu-chain-manipulator`
- Unimplemented (placeholder): `gpu-cuda`, `gpu-opencl`
- Behavior:
- Selecting `gpu-cuda` or `gpu-opencl` will return a descriptive runtime error and exit non-zero.
- Selecting `cpu-fast` (default) or `cpu-baseline` runs as expected.
---
## CI Notes
If not already configured, consider adding or verifying the following steps against the stable toolchain:
- Taplo: `taplo fmt --check`
- Rustfmt: `cargo fmt --all -- --check` (if enforcing Rust formatting)
- Clippy: `cargo clippy --workspace --all-targets -- -D warnings`
- Tests: `cargo test --locked --workspace`
This ensures reproducible formatting, lint hygiene, and test coverage in CI.
---
## Future Work
- Implement GPU engines (CUDA/OpenCL) behind feature flags and add device selection/configuration.
- Add GPU-safe metrics and engine-specific instrumentation.
- Benchmarks and performance monitoring across engine types (CPU vs GPU) with comparable worker/affinity settings.
- Optional: stricter lints and Clippy deny rules in CI for selected crates.
---
## Outcome
- Stable toolchain with Clippy and Taplo passing.
- Clear CLI/service behavior for not-yet-implemented GPU engines.
- Minimal, targeted code hygiene changes that improve maintainability and CI readiness without altering engine semantics.
+44
View File
@@ -0,0 +1,44 @@
# Implementation docs index
This directory is intended for PR reviewers as the definitive context for evaluating changes. Each entry captures a discrete iteration as a “prompt” (plan/spec) and a matching “response” (implementation and outcomes). Commits and PRs should link to the relevant entry; authoring/process guidance lives in /agents.md.
Conventions
- Numbering: entries are prefixed with a two‑digit sequence (00, 01, …) to indicate order.
- Pairing: most entries come as a prompt/response pair. Some early entries may be standalone context docs.
- Scope: each entry should be narrowly scoped and actionable to ease review and onboarding.
Table of contents
- 00
- [00-cpu-algorithm-and-optimisation-roadmap.md](./00-cpu-algorithm-and-optimisation-roadmap.md)
- 01
- [01-prompt-metrics-toggle-and-soc-restructure.md](./01-prompt-metrics-toggle-and-soc-restructure.md)
- [01-response-metrics-toggle-and-soc-restructure.md](./01-response-metrics-toggle-and-soc-restructure.md)
- 02
- [02-prompt-cpu-fast-engine-and-metrics.md](./02-prompt-cpu-fast-engine-and-metrics.md)
- [02-response-cpu-fast-engine-and-metrics.md](./02-response-cpu-fast-engine-and-metrics.md)
- 03
- [03-prompt-engine-comparability.md](./03-prompt-engine-comparability.md)
- [03-response-engine-comparability.md](./03-response-engine-comparability.md)
- 04
- [04-prompt-metrics-chunking-and-dashboards.md](./04-prompt-metrics-chunking-and-dashboards.md)
- [04-response-metrics-chunking-and-dashboards.md](./04-response-metrics-chunking-and-dashboards.md)
- 05
- [05-prompt-workers-and-dashboards-sync.md](./05-prompt-workers-and-dashboards-sync.md)
- [05-response-workers-and-dashboards-sync.md](./05-response-workers-and-dashboards-sync.md)
- 06
- [06-prompt-manipulator-and-difficulty-overlay.md](./06-prompt-manipulator-and-difficulty-overlay.md)
- [06-response-manipulator-and-difficulty-overlay.md](./06-response-manipulator-and-difficulty-overlay.md)
- 07
- [07-prompt-stable-clippy-taplo-and-gpu-enum.md](./07-prompt-stable-clippy-taplo-and-gpu-enum.md)
- [07-response-stable-clippy-taplo-and-gpu-enum.md](./07-response-stable-clippy-taplo-and-gpu-enum.md)
For authors and agents
Authoring workflow, conventions, and commit/PR linkage requirements are documented in /agents.md.
Cross‑references
- Authors and agents: see /agents.md for authoring workflow and process details. The repo README.md also links to review-critical docs.
- Workspace structure, crate boundaries, and feature gates are described inline in prompt/response docs.
- Dashboards and metrics: see entries 04–07 for metrics chunking, per‑thread/job gauges, and exporter behavior.
Last updated
- 07: Stable toolchain, Clippy/Taplo hygiene, GPU engine enum placeholders with user‑friendly errors.
+1 -2
View File
@@ -1,5 +1,4 @@
[toolchain]
channel = "nightly-2025-06-06"
channel = "stable"
components = ["clippy", "rustfmt"]
targets = ["wasm32v1-none"]
profile = "minimal"