fix(neuron/candle): source tokenizer.json from base repo when GGUF
Some checks failed
build-prerelease / Resolve version stamps (push) Successful in 31s
CI / Format (push) Successful in 37s
CI / Clippy (push) Failing after 50s
CI / Test (push) Failing after 49s
CI / Build cortex SRPM (push) Has been skipped
CI / Publish cortex to COPR (push) Has been skipped
CI / Build neuron SRPM (push) Has been skipped
CI / Publish neuron to COPR (push) Has been skipped
CI / Bump version in source (push) Has been skipped
build-prerelease / Build neuron-blackwell (push) Successful in 3m32s
build-prerelease / Build cortex binary (push) Successful in 4m34s
build-prerelease / Package cortex RPM (push) Successful in 1m21s
build-prerelease / Build neuron-ampere (push) Successful in 5m9s
build-prerelease / Build neuron-ada (push) Successful in 4m52s
build-prerelease / Package helexa-neuron-ada RPM (push) Successful in 2m56s
build-prerelease / Package helexa-neuron-ampere RPM (push) Successful in 2m54s
build-prerelease / Package helexa-neuron-blackwell RPM (push) Successful in 3m36s
build-prerelease / Publish to rpm.lair.cafe (unstable) (push) Successful in 59s
Some checks failed
build-prerelease / Resolve version stamps (push) Successful in 31s
CI / Format (push) Successful in 37s
CI / Clippy (push) Failing after 50s
CI / Test (push) Failing after 49s
CI / Build cortex SRPM (push) Has been skipped
CI / Publish cortex to COPR (push) Has been skipped
CI / Build neuron SRPM (push) Has been skipped
CI / Publish neuron to COPR (push) Has been skipped
CI / Bump version in source (push) Has been skipped
build-prerelease / Build neuron-blackwell (push) Successful in 3m32s
build-prerelease / Build cortex binary (push) Successful in 4m34s
build-prerelease / Package cortex RPM (push) Successful in 1m21s
build-prerelease / Build neuron-ampere (push) Successful in 5m9s
build-prerelease / Build neuron-ada (push) Successful in 4m52s
build-prerelease / Package helexa-neuron-ada RPM (push) Successful in 2m56s
build-prerelease / Package helexa-neuron-ampere RPM (push) Successful in 2m54s
build-prerelease / Package helexa-neuron-blackwell RPM (push) Successful in 3m36s
build-prerelease / Publish to rpm.lair.cafe (unstable) (push) Successful in 59s
GGUF-only HF repos (unsloth/Qwen3-*-GGUF, Qwen/Qwen3-*-GGUF) ship the .gguf file but not tokenizer.json — the tokenizer data is embedded in the GGUF metadata itself, and the standalone tokenizer.json lives in the base non-GGUF repo (unsloth/Qwen3-0.6B, Qwen/Qwen3-0.6B, etc.). Live validation against quadbrat hit: HTTP 400 fetch tokenizer.json from unsloth/Qwen3-0.6B-GGUF: HTTP status client error (404 Not Found) resolve_files now derives the tokenizer repo by stripping a `-GGUF` or `-gguf` suffix from the model_id; non-GGUF ids fall through to fetching from the same repo. The error message includes the attempted tokenizer repo id so the next failure (e.g. base repo doesn't exist) is unambiguous. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -121,10 +121,33 @@ impl CandleHarness {
|
|||||||
.get(&gguf_filename)
|
.get(&gguf_filename)
|
||||||
.await
|
.await
|
||||||
.with_context(|| format!("fetch GGUF {gguf_filename}"))?;
|
.with_context(|| format!("fetch GGUF {gguf_filename}"))?;
|
||||||
let tokenizer_path = repo
|
|
||||||
|
// GGUF-only HF repos (unsloth/Qwen3-*-GGUF, Qwen/Qwen3-*-GGUF,
|
||||||
|
// etc.) ship the .gguf file but not tokenizer.json — the
|
||||||
|
// tokenizer.json lives in the base non-GGUF repo. Derive the
|
||||||
|
// base repo id by stripping a `-GGUF` / `-gguf` suffix; if
|
||||||
|
// there's no such suffix the same repo is used (works for
|
||||||
|
// non-GGUF model_ids).
|
||||||
|
let tokenizer_repo_id = spec
|
||||||
|
.model_id
|
||||||
|
.strip_suffix("-GGUF")
|
||||||
|
.or_else(|| spec.model_id.strip_suffix("-gguf"))
|
||||||
|
.unwrap_or(spec.model_id.as_str())
|
||||||
|
.to_string();
|
||||||
|
let tokenizer_repo = if tokenizer_repo_id == spec.model_id {
|
||||||
|
repo
|
||||||
|
} else {
|
||||||
|
tracing::debug!(
|
||||||
|
from = %spec.model_id,
|
||||||
|
to = %tokenizer_repo_id,
|
||||||
|
"tokenizer.json sourced from base repo (GGUF suffix stripped)"
|
||||||
|
);
|
||||||
|
api.model(tokenizer_repo_id.clone())
|
||||||
|
};
|
||||||
|
let tokenizer_path = tokenizer_repo
|
||||||
.get("tokenizer.json")
|
.get("tokenizer.json")
|
||||||
.await
|
.await
|
||||||
.context("fetch tokenizer.json")?;
|
.with_context(|| format!("fetch tokenizer.json from {tokenizer_repo_id}"))?;
|
||||||
Ok((gguf_path, tokenizer_path))
|
Ok((gguf_path, tokenizer_path))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user