fix(neuron): render HF chat templates via minijinja pycompat
All checks were successful
build-prerelease / Resolve version stamps (push) Successful in 29s
CI / Format (push) Successful in 34s
CI / CUDA type-check (push) Successful in 39s
CI / Clippy (push) Successful in 2m35s
build-prerelease / Build cortex binary (push) Successful in 4m21s
build-prerelease / Build neuron-blackwell (push) Successful in 6m4s
CI / Test (push) Successful in 6m47s
CI / Build cortex SRPM (push) Has been skipped
CI / Publish cortex to COPR (push) Has been skipped
CI / Build neuron SRPM (push) Has been skipped
CI / Publish neuron to COPR (push) Has been skipped
CI / Bump version in source (push) Has been skipped
build-prerelease / Build neuron-ampere (push) Successful in 7m43s
build-prerelease / Package cortex RPM (push) Successful in 1m21s
build-prerelease / Build neuron-ada (push) Successful in 5m41s
build-prerelease / Package helexa-neuron-ada RPM (push) Successful in 3m5s
build-prerelease / Package helexa-neuron-ampere RPM (push) Successful in 3m6s
build-prerelease / Package helexa-neuron-blackwell RPM (push) Successful in 3m52s
build-prerelease / Publish to rpm.lair.cafe (unstable) (push) Successful in 1m3s

The Qwen3.6 chat_template.jinja (now loaded after the precedence fix)
failed to render in minijinja: it uses Python str methods
(content.startswith/endswith/split/rstrip/lstrip) and the raise_exception
global that HF transformers patches into its Jinja env but minijinja
doesn't provide. The render error tripped the text-only fallback, so
image requests still produced zero <|image_pad|> tokens.

Wire the standard bridge into render_chat_template:
- minijinja-contrib `pycompat::unknown_method_callback` supplies the
  Python string/list/dict methods;
- a `raise_exception` global maps to a render error (so malformed inputs
  — e.g. an image in a system message — surface cleanly).

Add the real Qwen3.6-27B chat_template.jinja (verbatim from beast's HF
cache) as a test fixture and assert it renders one <|image_pad|> for a
text+image turn — the end-to-end check that would have caught this
before deploy.

Refs #16 / TP-vision.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-04 16:32:23 +03:00
parent 7ad56c6a86
commit c8bcaabc38
4 changed files with 224 additions and 9 deletions

View File

@@ -43,7 +43,7 @@
use anyhow::{Context, Result};
use cortex_core::openai::{ChatMessage, MessageContent};
use minijinja::Environment;
use minijinja::{Environment, Error as MjError, ErrorKind as MjErrorKind, Value as MjValue};
use serde_json::Value;
use std::path::Path;
@@ -191,6 +191,25 @@ pub fn render_chat_template(
kwargs: &Value,
) -> Result<String> {
let mut env = Environment::new();
// HF chat templates are authored against Python's Jinja2 with its
// string semantics. Bridge the two so real model templates render:
//
// - `pycompat::unknown_method_callback` supplies Python str/list/dict
// methods minijinja lacks natively (`startswith`, `endswith`,
// `split`, `rstrip`, `lstrip`, …) — the Qwen3.6 template uses
// several in its think-block and tool-response handling.
// - `raise_exception` is the global HF templates call to reject
// malformed inputs (e.g. an image in a system message). Map it to
// a render error so the caller falls back / surfaces it.
env.set_unknown_method_callback(minijinja_contrib::pycompat::unknown_method_callback);
env.add_function(
"raise_exception",
|msg: String| -> Result<MjValue, MjError> {
Err(MjError::new(MjErrorKind::InvalidOperation, msg))
},
);
// Compile the template against a fixed name so error messages
// surface "chat_template" rather than `<template>`.
env.add_template("chat_template", template)
@@ -334,6 +353,33 @@ mod tests {
assert_eq!(got.as_deref(), Some("FROM_CONFIG"));
}
/// The *actual* Qwen3.6-27B `chat_template.jinja` (verbatim from
/// beast's HF cache) must render in minijinja and emit exactly one
/// `<|image_pad|>` for a text+image user turn. This is the real
/// end-to-end check the unit tests above only approximate — it
/// catches any minijinja incompatibility (namespace, macros,
/// reverse slice, string methods) before it reaches production.
#[test]
fn real_qwen3_6_template_renders_one_image_pad() {
let template = include_str!("testdata/qwen3_6_chat_template.jinja");
let messages = vec![ChatMessage {
role: "user".into(),
content: MessageContent::Parts(vec![
json!({"type": "text", "text": "what is this?"}),
json!({"type": "image_url", "image_url": {"url": "data:image/png;base64,AAA="}}),
]),
extra: Value::Object(Default::default()),
}];
let out = render_chat_template(template, &messages, &Value::Null, &Value::Null)
.expect("real Qwen3.6 template should render in minijinja");
let pads = out.matches("<|image_pad|>").count();
assert_eq!(
pads, 1,
"expected exactly one <|image_pad|>; rendered:\n{out}"
);
assert!(out.contains("<|vision_start|>") && out.contains("<|vision_end|>"));
}
fn user_msg(text: &str) -> ChatMessage {
ChatMessage {
role: "user".into(),