feat(neuron): honour HF_HUB_CACHE / HF_HOME for the candle harness cache
Some checks failed
build-prerelease / Resolve version stamps (push) Successful in 31s
build-prerelease / Build neuron-blackwell (push) Successful in 3m39s
build-prerelease / Build cortex binary (push) Successful in 4m17s
build-prerelease / Package cortex RPM (push) Successful in 1m22s
CI / Format (push) Successful in 32s
CI / Test (push) Failing after 51s
CI / Clippy (push) Successful in 2m17s
build-prerelease / Build neuron-ampere (push) Successful in 4m58s
CI / Build cortex SRPM (push) Has been skipped
CI / Build neuron SRPM (push) Has been skipped
CI / Publish cortex to COPR (push) Has been skipped
CI / Publish neuron to COPR (push) Has been skipped
CI / Bump version in source (push) Has been skipped
build-prerelease / Build neuron-ada (push) Successful in 5m1s
build-prerelease / Package helexa-neuron-ada RPM (push) Successful in 3m0s
build-prerelease / Package helexa-neuron-ampere RPM (push) Successful in 3m4s
build-prerelease / Package helexa-neuron-blackwell RPM (push) Successful in 3m37s
build-prerelease / Publish to rpm.lair.cafe (unstable) (push) Successful in 1m3s

Resolves the candle harness's HuggingFace cache directory with the
following precedence (first hit wins):

1. Explicit `hf_cache` in `[harness.candle]` from neuron.toml.
2. `HF_HUB_CACHE` env var — the Python `huggingface_hub` convention.
   The Rust hf-hub crate doesn't read this natively, so we bridge here.
3. `HF_HOME` env var (`$HF_HOME/hub` per the canonical layout).
4. None — falls through to hf-hub's own default.

Honouring HF_HUB_CACHE lets a neuron host reuse an existing cache
directory shared with Python tooling or other harnesses on the same
host without per-tool config. The canonical per-host setup is a
systemd drop-in:

    /etc/systemd/system/neuron.service.d/local.conf
    [Service]
    Environment=HF_HUB_CACHE=/archive/hf-cache

neuron.example.toml documents the resolution chain inline.

script/validate-neuron.sh: bump LOAD_TIMEOUT from 600s to 3600s and
expose both load/infer timeouts via env (NEURON_LOAD_TIMEOUT,
NEURON_INFER_TIMEOUT). A Qwen3.6-class dense model is ~54 GB and was
hitting the 10-min ceiling cold-downloading on a residential link.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-20 07:52:50 +03:00
parent 62ca125a68
commit b400e8b704
3 changed files with 61 additions and 7 deletions

View File

@@ -128,6 +128,39 @@ const REPEAT_PENALTY: f32 = 1.1;
/// penalty. Matches the candle quantized-qwen3 example default.
const REPEAT_LAST_N: usize = 64;
/// Resolve the effective HuggingFace cache directory for the candle
/// harness. Precedence (first hit wins):
///
/// 1. Explicit `hf_cache` from `[harness.candle]` in `neuron.toml`.
/// Operator's wishes always win.
/// 2. `HF_HUB_CACHE` env var. The Python `huggingface_hub` library
/// points at the cache root directly with this var; the Rust
/// `hf-hub` crate doesn't read it natively, so we bridge here.
/// Honouring it lets a neuron host share a cache directory with
/// Python tooling and other harnesses without per-tool config.
/// 3. `HF_HOME` env var. Canonical HuggingFace base directory; the
/// cache lives at `$HF_HOME/hub`. Hf-hub respects this on its own,
/// but we resolve it here too so the resulting path shows up in
/// logs alongside the explicit/HF_HUB_CACHE cases.
/// 4. `None`. Falls through to `hf-hub`'s default
/// (`~/.cache/huggingface/hub`).
fn resolve_hf_cache(explicit: Option<PathBuf>) -> Option<PathBuf> {
if let Some(p) = explicit {
return Some(p);
}
if let Ok(v) = std::env::var("HF_HUB_CACHE")
&& !v.is_empty()
{
return Some(PathBuf::from(v));
}
if let Ok(v) = std::env::var("HF_HOME")
&& !v.is_empty()
{
return Some(PathBuf::from(v).join("hub"));
}
None
}
/// Apply the repetition penalty (if any) to the prediction logits and
/// then sample. Centralises the prefill / generation-loop call sites
/// so they share identical sampling behaviour.
@@ -147,6 +180,10 @@ fn sample_with_penalty(
impl CandleHarness {
pub fn new(bind_url: String, hf_cache: Option<PathBuf>) -> Self {
let hf_cache = resolve_hf_cache(hf_cache);
if let Some(p) = &hf_cache {
tracing::info!(path = %p.display(), "candle harness using HuggingFace cache");
}
Self {
models: Arc::new(RwLock::new(HashMap::new())),
hf_cache,