fix(beast): 3 GiB of prefix-cache snapshots, not 1

`cached_tokens` was pinned at 2,016 for ten consecutive turns of a live
agentic session while the prompt grew to 52,869 — every turn
re-prefilling ~50k tokens at ~37 s, against ~20 s of actual decode. The
cache was not missing the prefix; the snapshot could not be stored.

At the measured `kv_bytes_per_token_per_card = 32768`, the 1 GiB default
cannot hold more than ~32k tokens of K/V, and less once `snapshot.rs`
deep-copies the GatedDeltaNet conv/recurrent state on top — 48 of 64
layers, in f32 per `mamba_ssm_dtype`. Sessions on this host run well
past that, so the deeper the session the more of every turn was spent
re-doing work.

Sized from this deployment rather than rounded: free at load 9,285 MB,
less the 4,713 MB KV budget, leaves 4,572 MB that snapshots share. A
52,869-token snapshot is ~1,652 MB of K/V, so 3 GiB holds a full deep
session with slack. 4 GiB would take 90% of the headroom, which is not
a trade worth making on a host that also cold-swaps image models.

One variable: `max_entries` stays at 8. The byte budget binds first
here, so the count cap is not what forces snapshots to be small, and
changing both would make the result unattributable.

Config-only, so this deploys without a binary rotation (#291) — the
classifier treats `asset/neuron/*.toml` as inert.

Expected: `cached_tokens` tracks the prompt instead of freezing, and
per-turn prefill falls from ~37 s toward the low seconds. If it stays
frozen at 2,016, the budget is not the cap and #292 moves to the
retention policy.

.gitea/workflows/ci.yml

@@ -101,14 +101,34 @@ jobs:
           echo "changed files:"
           printf '%s\n' "${files}" | sed 's/^/  /'
 
-          # Inert for Rust: the SPA, and documentation. Markdown is
-          # matched only at the repo root and under doc/ — a .md inside
-          # crates/ is left to fall through, since `include_str!` could
-          # make one load-bearing without this file knowing.
+          # Inert for Rust: the SPA, documentation, and fleet config.
+          # Markdown is matched only at the repo root and under doc/ — a
+          # .md inside crates/ is left to fall through, since
+          # `include_str!` could make one load-bearing without this file
+          # knowing.
+          #
+          # Fleet config (models.toml, asset/neuron/<host>.toml,
+          # asset/helexa-bench/<host>.toml) is read by a *running*
+          # daemon, never by a build script or an `include_str!`, so it
+          # cannot change a Rust result. Deploying one is a config rsync
+          # plus a service restart — neuron re-reads it and reloads the
+          # model — so spending ~25 minutes on fmt, clippy, test and a
+          # CUDA type-check to prove a one-line TOML edit compiles is
+          # pure latency (#291).
+          #
+          # The `config` job below stays ungated and still validates
+          # these files, so narrowing here removes the meritless work
+          # without removing the check that matters. build-prerelease
+          # already reaches the same conclusion on main: none of its
+          # per-package path regexes match these files, so a config-only
+          # push builds no binaries.
           rust_relevant="$(printf '%s\n' "${files}" \
             | grep -v '^helexa\.ai/' \
             | grep -v '^doc/' \
             | grep -vE '^[^/]+\.md$' \
+            | grep -vE '^asset/neuron/[^/]+\.toml$' \
+            | grep -vE '^asset/helexa-bench/[^/]+\.toml$' \
+            | grep -vE '^models\.toml$' \
             || true)"
           if [ -n "${rust_relevant}" ]; then
             echo "rust=true" >> "$GITHUB_OUTPUT"

asset/neuron/beast.toml

@@ -29,7 +29,47 @@ name = "candle"
 # count. NEURON_BATCHING=0 (systemd drop-in) is the kill switch;
 # removing this section reverts to batch-1 serialization.
 [harness.candle.admission]
-max_in_flight = 8
+# 2, not 8 (#291). max_in_flight is both the admission bound and the
+# batch engine's slot count, and the KV budget is divided by it at load.
+# At 8 that gave each slot 18,852 tokens out of 150,816 — against a
+# 131,072-token advertised context, which needs 4 GB of KV per card on
+# its own and 32 GB across eight slots, on a 31 GB card already holding
+# ~21 GB of weights. The advertised context and the slot count were
+# incompatible by roughly 7x, and an agentic session ran in a ~19k
+# window while seven idle slots held reservations.
+#
+# Seven days of paired request spans on this host: 89.6% of busy time
+# was ONE request in flight, 93.6% was two or fewer, and utilisation was
+# 4.9%. Most of what little concurrency exists is helexa-bench's own
+# synthetic bursts. Two slots cover 93.6% of observed load and give each
+# ~75k tokens; the batch engine is still worth 1.79x there (5.24x at
+# eight is throughput nobody is asking this host for). Anything above
+# two queues on admission (#53) with Retry-After rather than being
+# refused.
+max_in_flight = 2
+
+# Snapshot budget for the cross-request prefix cache (#292).
+#
+# 3 GiB, not the 1 GiB default. At the measured 32 KiB/token/card, a
+# 1 GiB budget cannot hold a snapshot beyond ~32k tokens of K/V alone —
+# less once the GatedDeltaNet conv/recurrent state is deep-copied on top,
+# which on this arch is 48 of 64 layers in f32. Agentic sessions here run
+# well past that: `cached_tokens` was observed pinned at 2,016 for ten
+# consecutive turns while the prompt grew to 52,869, so every turn
+# re-prefilled ~50k tokens — 37 s each, against ~20 s of actual decode.
+# The cache was not missing; the snapshot would not fit.
+#
+# Sized from this deployment's numbers rather than rounded: free at load
+# 9,285 MB, less the 4,713 MB KV budget, leaves 4,572 MB that snapshots
+# share with everything else. A 52,869-token snapshot is ~1,652 MB of
+# K/V, so 3 GiB holds a full deep session with slack. 4 GiB would take
+# 90% of the headroom, which is not a trade worth making on a host that
+# also cold-swaps image models.
+#
+# max_entries stays at the default 8: the byte budget binds first here,
+# so the count cap is not what is forcing snapshots to be small.
+[harness.candle.prefix_cache]
+budget_mb = 3072
 
 [[default_models]]
 # Qwen3.8-27B is the resident flagship (#251). Architecturally identical

bench/src/api.ts

@@ -1,4 +1,10 @@
-import type { Dimensions, ReportRow, RunRow, SeriesPoint } from "./types";
+import type {
+  Dimensions,
+  MeasurementRegime,
+  ReportRow,
+  RunRow,
+  SeriesPoint,
+} from "./types";
 
 // Empty default → `fetch('/api/...')` hits the dev proxy (vite.config.ts)
 // or the same origin. For a separately-hosted build, set VITE_API_BASE to
@@ -15,6 +21,7 @@ async function getJson<T>(path: string): Promise<T> {
 
 export const getDimensions = () => getJson<Dimensions>("/api/dimensions");
 export const getSummary = () => getJson<ReportRow[]>("/api/summary");
+export const getRegimes = () => getJson<MeasurementRegime[]>("/api/regimes");
 
 // host is resolved server-side (each model maps to one host today), so the
 // public UI selects by model + scenario alone.

bench/src/pages/Trends.tsx

@@ -11,8 +11,8 @@ import {
   XAxis,
   YAxis,
 } from "recharts";
-import { getDimensions, getSeries } from "../api";
-import type { Dimensions, SeriesPoint } from "../types";
+import { getDimensions, getRegimes, getSeries } from "../api";
+import type { Dimensions, MeasurementRegime, SeriesPoint } from "../types";
 import { BASELINE_SOURCE, baselineFor } from "../baseline";
 
 function Picker({
@@ -52,19 +52,22 @@ type SeriesDef = {
  * Every panel draws the same x-axis and the same regime divider, so a
  * reader can line a change up across metrics — which is the whole point
  * of having more than two of them. */
+/** A vertical rule at a build where the metric changed meaning. */
+type Rule = { at: string; label: string; detail?: string };
+
 function MetricChart({
   title,
   hint,
   data,
   lines,
-  divider,
+  rules,
   unit,
 }: {
   title: string;
   hint?: string;
   data: Record<string, unknown>[];
   lines: SeriesDef[];
-  divider?: string;
+  rules?: Rule[];
   unit?: string;
 }) {
   const hasAny = data.some((d) => lines.some((l) => d[l.key] != null));
@@ -83,19 +86,25 @@ function MetricChart({
           <YAxis />
           <Tooltip />
           <Legend />
-          {divider && (
-            <ReferenceLine
-              x={divider}
-              stroke="#bbb"
-              strokeDasharray="3 3"
-              label={{
-                value: "bench.py → helexa-bench",
-                position: "top",
-                fill: "#999",
-                fontSize: 11,
-              }}
-            />
-          )}
+          {(rules ?? [])
+            // Only draw a rule whose build is actually on this x-axis;
+            // recharts would otherwise place it at the origin and imply
+            // the boundary sits before all the data.
+            .filter((r) => data.some((d) => d.label === r.at))
+            .map((r) => (
+              <ReferenceLine
+                key={`${r.at}-${r.label}`}
+                x={r.at}
+                stroke="#bbb"
+                strokeDasharray="3 3"
+                label={{
+                  value: r.label,
+                  position: "top",
+                  fill: "#999",
+                  fontSize: 11,
+                }}
+              />
+            ))}
           {lines.map((l) => (
             <Line
               key={l.key}
@@ -119,6 +128,7 @@ export default function Trends() {
   const [model, setModel] = useState("");
   const [scenario, setScenario] = useState("");
   const [series, setSeries] = useState<SeriesPoint[]>([]);
+  const [regimes, setRegimes] = useState<MeasurementRegime[]>([]);
   const [err, setErr] = useState<string | null>(null);
 
   useEffect(() => {
@@ -129,6 +139,10 @@ export default function Trends() {
         if (d.scenarios[0]) setScenario(d.scenarios[0]);
       })
       .catch((e) => setErr(String(e)));
+    // A missing/older API just means no rules — not a page failure.
+    getRegimes()
+      .then(setRegimes)
+      .catch(() => setRegimes([]));
   }, []);
 
   useEffect(() => {
@@ -173,7 +187,43 @@ export default function Trends() {
   // first live build, with baseline points to its left).
   const firstLive = series[0]?.git_sha;
   const showDivider = base.length > 0 && series.length > 0;
-  const divider = showDivider ? firstLive : undefined;
+
+  // The build where the measuring identity changed, derived from the
+  // data rather than declared — a constant would go stale the moment
+  // the principal is reconfigured (#288).
+  const identityShift = useMemo(() => {
+    for (let i = 1; i < series.length; i++) {
+      const prev = series[i - 1].principal ?? "anonymous";
+      const cur = series[i].principal ?? "anonymous";
+      if (prev !== cur) {
+        return {
+          at: series[i].git_sha,
+          label: `measured as ${cur === "anonymous" ? "anonymous" : "identified"}`,
+          detail:
+            "The identity bench measured under changed here. Anonymous " +
+            "samples are subject to the #262 yield policy and understate " +
+            "capacity, so points either side are not comparable (#288).",
+        } as Rule;
+      }
+    }
+    return null;
+  }, [series]);
+
+  /** Declared boundaries touching `metric`, plus the regime-independent
+   *  ones that apply to every panel. */
+  const rulesFor = (metric: string): Rule[] => {
+    const out: Rule[] = [];
+    if (showDivider && firstLive) {
+      out.push({ at: firstLive, label: "bench.py → helexa-bench" });
+    }
+    for (const r of regimes) {
+      if (r.affects.includes(metric)) {
+        out.push({ at: r.first_sha, label: r.label, detail: r.detail });
+      }
+    }
+    if (identityShift) out.push(identityShift);
+    return out;
+  };
 
   // Anonymous and identified samples are not comparable once #262 is in
   // the build: an anonymous caller is capped below max_in_flight and
@@ -249,7 +299,7 @@ export default function Trends() {
             title="decode tok/s"
             unit="higher is better"
             data={data}
-            divider={divider}
+            rules={rulesFor("decode")}
             lines={[
               { key: "decode", name: "decode tok/s", stroke: "#0d6efd" },
               ...(base.length > 0
@@ -270,7 +320,7 @@ export default function Trends() {
             unit="higher is better"
             hint="The other half of serving speed, derived from prefill_tokens / prefill_ms. A prefix-cache hit shortens prefill_ms while the token count stays whole, so a high rate here is itself the cache-hit signal."
             data={data}
-            divider={divider}
+            rules={rulesFor("prefillTps")}
             lines={[
               { key: "prefillTps", name: "prefill tok/s", stroke: "#20c997" },
             ]}
@@ -281,7 +331,7 @@ export default function Trends() {
             unit="seconds, lower is better"
             hint="Median and p95 together on purpose. Under concurrency the median is dominated by whichever streams were admitted immediately; the p95 is the one that moves when a caller is made to wait. Charting only the median is how a 0.53 s → 11.77 s tail went unnoticed for a week."
             data={data}
-            divider={divider}
+            rules={rulesFor("ttft")}
             lines={[
               { key: "ttft", name: "TTFT median (s)", stroke: "#dc3545" },
               { key: "ttftP95", name: "TTFT p95 (s)", stroke: "#fd7e14" },
@@ -303,7 +353,7 @@ export default function Trends() {
             unit="ms, lower is better"
             hint="Stream smoothness. decode tok/s is a mean over the whole window, so a stream that stalls and then catches up is indistinguishable from one that never stalled — this is the number a user feels."
             data={data}
-            divider={divider}
+            rules={rulesFor("tpot")}
             lines={[
               { key: "tpot", name: "inter-token p95 (ms)", stroke: "#6f42c1" },
             ]}
@@ -314,7 +364,7 @@ export default function Trends() {
             unit="queue wait ms · requests shed"
             hint="Separates “the server is slow” from “you were queued behind someone”. Queue wait is TTFT minus server-measured prefill; rejected counts honest backpressure rather than silent failures."
             data={data}
-            divider={divider}
+            rules={rulesFor("queueWait")}
             lines={[
               { key: "queueWait", name: "queue wait (ms)", stroke: "#d63384" },
               { key: "rejected", name: "rejected (count)", stroke: "#adb5bd" },
@@ -326,13 +376,46 @@ export default function Trends() {
             unit="counts"
             hint="Cost, not speed. Reasoning tokens are the dominant driver on a reasoning model and move independently of every rate above — a template or sampling change can double what the model thinks before answering while the speed charts stay flat. Cached tokens are why prefill timing varies between otherwise identical samples."
             data={data}
-            divider={divider}
+            rules={rulesFor("completion")}
             lines={[
               { key: "completion", name: "completion tokens", stroke: "#0dcaf0" },
               { key: "reasoning", name: "reasoning tokens", stroke: "#ffc107" },
               { key: "cached", name: "cached prompt tokens", stroke: "#198754" },
             ]}
           />
+
+          {(() => {
+            // Explain every rule actually drawn for this selection. The
+            // chart label is a name; without the reason, a reader still
+            // has to go and rediscover what it meant — which is the cost
+            // this whole feature exists to remove.
+            const shown = [
+              ...regimes
+                .filter((r) => series.some((p) => p.git_sha === r.first_sha))
+                .map((r) => ({ at: r.first_sha, label: r.label, detail: r.detail })),
+              ...(identityShift ? [identityShift] : []),
+            ];
+            if (shown.length === 0) return null;
+            return (
+              <div className="mt-4 pt-3 border-top">
+                <p className="text-muted small mb-2">
+                  <strong>Dashed rules mark measurement-regime changes</strong> —
+                  builds where a number changed meaning rather than value. A step
+                  across one is the instrument, not the engine.
+                </p>
+                <dl className="row small text-muted mb-0">
+                  {shown.map((r) => (
+                    <div key={`${r.at}-${r.label}`} className="mb-2">
+                      <dt className="fw-semibold">
+                        <code>{r.at}</code> — {r.label}
+                      </dt>
+                      <dd className="mb-0">{r.detail}</dd>
+                    </div>
+                  ))}
+                </dl>
+              </div>
+            );
+          })()}
         </>
       )}
     </>

bench/src/types.ts

@@ -78,3 +78,13 @@ export interface RunRow {
   ok: boolean;
   error: string | null;
 }
+
+/** A build at which a metric changed meaning rather than value (#288).
+ *  Served from the API so the UI and `report` cite one list. */
+export interface MeasurementRegime {
+  first_sha: string;
+  label: string;
+  detail: string;
+  /** Series keys this affects, as Trends.tsx names them. */
+  affects: string[];
+}

crates/cortex-core/src/catalogue.rs

@@ -67,6 +67,13 @@ pub struct ModelProfile {
     /// model, with nothing forcing them to agree.
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub sampling: Option<SamplingOverride>,
+    /// The model's own `preserve_thinking` template control, forwarded
+    /// on cold load so a model this gateway loads behaves the same as
+    /// one the host was configured to hold resident. Two config files
+    /// describing one model is how #252 happened; this rides along for
+    /// the same reason `sampling` does.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub preserve_thinking: Option<bool>,
 
     // ── Enrichment (issue #62) ────────────────────────────────
     /// Per-model token budget. When present, advertised in `/v1/models`
@@ -251,6 +258,7 @@ mod tests {
             residency_priority: None,
             source: None,
             sampling: None,
+            preserve_thinking: None,
             limit: None,
             cost: None,
             capabilities: vec![],

crates/cortex-core/src/harness.rs

@@ -86,6 +86,27 @@ pub struct ModelSpec {
     /// still round-trips the spec.
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub sampling: Option<SamplingOverride>,
+    /// Whether prior assistant turns keep their reasoning when the chat
+    /// template re-renders the conversation.
+    ///
+    /// This is the model's own `preserve_thinking` template control, not
+    /// an invention of ours. `Qwen/Qwen3.8-27B` defaults it to *true* —
+    /// the whole transcript's think blocks are replayed — and `false`
+    /// keeps reasoning only for turns after the last user query, i.e.
+    /// the turn in progress.
+    ///
+    /// `None` leaves the kwarg unset so the template's own default
+    /// applies, which is the only safe default: the model's authors
+    /// chose it, and overriding that fleet-wide on a hunch is how you
+    /// ship a regression you cannot see.
+    ///
+    /// Exposed as operator config so the premise can be A/B'd on a real
+    /// workload — full replay is the larger prompt and pushes against
+    /// the prefix-cache budget and the throughput-derived context
+    /// ceiling, but whether it *helps* the model is a measurement, not
+    /// a belief. A request's own `chat_template_kwargs` still wins.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub preserve_thinking: Option<bool>,
 }
 
 /// Per-model token budget advertised by the catalogue or neuron.
@@ -141,9 +162,38 @@ fn is_zero(v: &usize) -> bool {
 #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
 pub struct ReasoningBudgetRung {
     /// The effort level as the caller spells it on the wire.
+    ///
+    /// Since #290 these are the rungs the **model's own chat template**
+    /// accepts, discovered by rendering it, rather than a ladder the
+    /// operator invented. `Qwen/Qwen3.8-27B` accepts exactly `low`,
+    /// `medium` and `xhigh` and raises on anything else; we previously
+    /// advertised `minimal`/`low`/`medium`/`high`, two of which the
+    /// template rejects and which omitted the model's own default.
+    ///
+    /// Advertise only rungs the template accepts: an invented name is a
+    /// name a caller can send and the model will reject. Note that a
+    /// client need not read this at all — pi-ai takes its levels from a
+    /// static `thinkingLevelMap` in the operator's config — so this is
+    /// a statement of truth, not a control surface we can rely on.
     pub effort: String,
-    /// Reasoning tokens that level is allowed to spend.
-    pub tokens: usize,
+    /// Backstop reasoning-token cap for this rung, when the deployment
+    /// sets one.
+    ///
+    /// `None` is the normal case and means "named rung, no cap": effort
+    /// is expressed to the model through its template, and the number of
+    /// tokens it then spends is the model's business. A cap is a safety
+    /// net against a runaway think block (#223), not the mechanism by
+    /// which effort is selected — enforcing effort by truncation is what
+    /// #290 fixed.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub tokens: Option<usize>,
+    /// Whether the model applies this rung when the caller names none.
+    ///
+    /// Taken from the template's own `|default(...)`. Without it a
+    /// caller that says nothing cannot tell what it is getting — on
+    /// Qwen3.8 that silently means `xhigh`.
+    #[serde(default, skip_serializing_if = "std::ops::Not::not")]
+    pub default: bool,
 }
 
 /// Operator-set pricing, **USD per 1,000,000 tokens, as JSON numbers**

crates/cortex-core/src/node.rs

@@ -194,10 +194,19 @@ pub struct CortexModelEntry {
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub max_model_len: Option<usize>,
     /// The same window under the two names generic OpenAI clients look
-    /// for (#278). pi-ai's provider discovery, for one, reads
-    /// `context_window` then `context_length` and nothing else — so a
-    /// client probing cortex found no window at all and fell back to a
-    /// number its operator had to invent by hand.
+    /// for (#278) — a client that probes for one of these rather than
+    /// `max_model_len` would otherwise find no window at all and fall
+    /// back to a number its operator had to invent by hand.
+    ///
+    /// Publishing them is cheap and harmless; do not assume any
+    /// *specific* client consumes them. An earlier version of this
+    /// comment credited pi-ai's provider discovery, which was wrong:
+    /// pi requires `contextWindow` as a static field on its model type
+    /// and reads none of these keys at runtime (checked against 0.82.1
+    /// and 0.84.3). The misattribution mattered — it made an operator's
+    /// hand-set 80k window look like our advertisement failing to land,
+    /// when in fact nothing was reading it and the number had to be
+    /// maintained by hand regardless.
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub context_window: Option<usize>,
     #[serde(default, skip_serializing_if = "Option::is_none")]
@@ -227,12 +236,16 @@ impl CortexModelEntry {
         self.context_window = self.limit.as_ref().map(|l| l.context);
         self.context_length = self.limit.as_ref().map(|l| l.context);
         self.max_input_tokens = self.limit.as_ref().and_then(|l| l.input);
-        // The ceiling, not the reserve (#278): this field is what
-        // OpenAI-ecosystem clients read to decide what to ask for, and
-        // pi-ai/DSH turns it straight into a per-request cap. Advertising
-        // the reserve here handed a reasoning model less budget than its
-        // own think block needs. Falls back to the reserve for a neuron
-        // too old to publish a ceiling.
+        // The ceiling, not the reserve (#278): a client that reads this
+        // field turns it into the cap it sends on each request, so
+        // advertising the reserve handed a reasoning model less budget
+        // than its own think block needs. Falls back to the reserve for
+        // a neuron too old to publish a ceiling.
+        //
+        // Observed with dsh. pi is *not* an example: its `maxTokens` is
+        // a required static field in models.json and it reads nothing
+        // from here — an operator has to copy this number across by
+        // hand, and gets no error if they copy it wrong.
         self.max_output_tokens = self.limit.as_ref().map(|l| {
             if l.output_ceiling > 0 {
                 l.output_ceiling

crates/cortex-core/src/responses.rs

@@ -215,11 +215,46 @@ pub struct ResponsesResponse {
     pub status: String,
     pub model: String,
     pub output: Vec<ResponsesOutputItem>,
+    /// Why a `status: "incomplete"` response stopped short. Required in
+    /// practice, not optional decoration: a client cannot tell an
+    /// honest truncation from a protocol fault without it, and at least
+    /// one treats the difference as fatal.
+    ///
+    /// pi-ai's `mapStopReason` maps
+    /// `incomplete` + `reason == "max_output_tokens"` to
+    /// `stopReason: "length"`, and `incomplete` with no reason to
+    /// `stopReason: "error"` ("Response incomplete without a provider
+    /// reason"). Its agent loop then either continues — failing any
+    /// truncated tool calls safely and letting the model recover on the
+    /// next turn — or halts outright. Omitting this field cost a live
+    /// agentic session its whole run.
+    ///
+    /// `None` for `completed` and `in_progress`.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub incomplete_details: Option<IncompleteDetails>,
     /// Populated on completion; `None` while streaming.
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub usage: Option<ResponsesUsage>,
 }
 
+/// Why a response is `incomplete`.
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
+pub struct IncompleteDetails {
+    /// OpenAI spells the output-cap case `"max_output_tokens"`; that is
+    /// the only reason we produce today. `content_filter` is the other
+    /// value the upstream API defines.
+    pub reason: String,
+}
+
+impl IncompleteDetails {
+    /// The response ran into the caller's output budget.
+    pub fn max_output_tokens() -> Self {
+        Self {
+            reason: "max_output_tokens".into(),
+        }
+    }
+}
+
 #[derive(Debug, Clone, Serialize, Deserialize)]
 #[serde(tag = "type", rename_all = "snake_case")]
 pub enum ResponsesOutputItem {
@@ -327,6 +362,24 @@ pub mod events {
     pub const FUNCTION_CALL_ARGUMENTS_DELTA: &str = "response.function_call_arguments.delta";
     pub const FUNCTION_CALL_ARGUMENTS_DONE: &str = "response.function_call_arguments.done";
     pub const COMPLETED: &str = "response.completed";
+    /// Terminal frame for a response that stopped short. Carries the
+    /// same `response` payload as [`COMPLETED`], with
+    /// `status: "incomplete"` and `incomplete_details.reason`.
+    ///
+    /// A client keyed on the event name — rather than on
+    /// `response.status` — sees no terminal event at all if a truncated
+    /// response is announced as `response.completed`, and waits out its
+    /// idle timeout.
+    pub const INCOMPLETE: &str = "response.incomplete";
+    /// Terminal frame for a response that did not finish because the
+    /// server failed — a poisoned device, an OOM, a producer that died
+    /// mid-stream. Carries `status: "failed"` and an `error` object.
+    ///
+    /// The alternative is worse than it looks: defaulting a missing
+    /// finish to a clean stop reports a crash as a complete answer, and
+    /// the caller believes a truncated or empty reply was the model's
+    /// considered response.
+    pub const FAILED: &str = "response.failed";
 }
 
 #[cfg(test)]
@@ -531,6 +584,7 @@ mod tests {
                 }],
                 status: "completed".into(),
             }],
+            incomplete_details: None,
             usage: Some(ResponsesUsage {
                 input_tokens: 5,
                 output_tokens: 3,
@@ -543,5 +597,36 @@ mod tests {
         let parsed: ResponsesResponse = serde_json::from_str(&json).unwrap();
         assert_eq!(parsed.id, "resp_1");
         assert_eq!(parsed.output.len(), 1);
+        assert!(
+            !json.contains("incomplete_details"),
+            "a completed response must not carry an incomplete reason"
+        );
+    }
+
+    /// The field pi-ai's `mapStopReason` keys on. Absent, it maps
+    /// `incomplete` to `stopReason: "error"` and its agent loop halts;
+    /// present with `max_output_tokens`, it maps to `"length"` and the
+    /// loop continues.
+    #[test]
+    fn an_incomplete_response_serialises_its_reason() {
+        let r = ResponsesResponse {
+            id: "resp_2".into(),
+            object: "response".into(),
+            created_at: 1700,
+            status: "incomplete".into(),
+            model: "m".into(),
+            output: vec![],
+            incomplete_details: Some(IncompleteDetails::max_output_tokens()),
+            usage: None,
+        };
+        let v: serde_json::Value = serde_json::to_value(&r).unwrap();
+        assert_eq!(v["status"], "incomplete");
+        assert_eq!(v["incomplete_details"]["reason"], "max_output_tokens");
+
+        let parsed: ResponsesResponse = serde_json::from_value(v).unwrap();
+        assert_eq!(
+            parsed.incomplete_details,
+            Some(IncompleteDetails::max_output_tokens())
+        );
     }
 }

crates/cortex-gateway/src/router.rs

@@ -539,6 +539,7 @@ async fn profile_to_spec(
         // (#283), so a cold-loaded model samples the same way as one
         // this host was configured to hold resident.
         sampling: profile.sampling.clone(),
+        preserve_thinking: profile.preserve_thinking,
     }
 }
 
@@ -647,6 +648,7 @@ mod tests {
             residency_priority: None,
             source: source.map(String::from),
             sampling: None,
+            preserve_thinking: None,
             limit: None,
             cost: None,
             capabilities: vec![],

crates/cortex-gateway/tests/model_limits.rs

@@ -69,17 +69,20 @@ capabilities = ["text"]
         let mut nodes = fleet.nodes.write().await;
         let node = nodes.get_mut("mock-node").expect("node exists");
         node.healthy = true;
-        // The ladder the neuron reported at poll time (#223) — host
-        // configuration, held once per node.
-        node.reasoning_budget = ["minimal", "low", "medium", "high"]
+        // The rungs the neuron reported at poll time. Since #290 these
+        // come from the model's own chat template rather than operator
+        // configuration, so the fixture uses the shape a real reasoning
+        // model ships: Qwen3.8-27B accepts exactly low/medium/xhigh and
+        // defaults to xhigh.
+        node.reasoning_budget = ["low", "medium", "xhigh"]
             .into_iter()
-            .zip([1024usize, 4096, 12288, 32768])
-            .map(
-                |(effort, tokens)| cortex_core::harness::ReasoningBudgetRung {
-                    effort: effort.into(),
-                    tokens,
-                },
-            )
+            .map(|effort| cortex_core::harness::ReasoningBudgetRung {
+                default: effort == "xhigh",
+                effort: effort.into(),
+                // No cap: effort is expressed to the model through its
+                // template, not enforced by truncating its thinking.
+                tokens: None,
+            })
             .collect();
         node.models.insert(
             "test-model".into(),
@@ -162,14 +165,16 @@ capabilities = ["text"]
     // vLLM-convention probes (Hermes Agent) auto-detect the window.
     assert_eq!(entry["max_model_len"], 49152);
     assert_eq!(entry["max_input_tokens"], 40960);
-    // The ceiling, not the reserve (#278). pi-ai's discovery — and every
-    // client like it — turns this field into the cap it sends on each
-    // request; advertising the 8192 reserve told a reasoning model's
-    // harness to stop below the cost of a single think block.
+    // The ceiling, not the reserve (#278). A client that reads this
+    // field turns it into the cap it sends on each request; advertising
+    // the 8192 reserve told a reasoning model's harness to stop below
+    // the cost of a single think block. Observed with dsh — pi takes
+    // `maxTokens` from static config and reads nothing here.
     assert_eq!(entry["max_output_tokens"], 32768);
-    // What each effort level buys (#223). A client can only send the
-    // rung names — it has no way to express a token count — so a ladder
-    // it cannot see is a ladder it has to guess at.
+    // The rungs the model itself accepts (#290). A caller can only send
+    // a rung name, so a name we invent is one the template will reject.
+    // Publishing the true set is correct regardless of who reads it —
+    // pi-ai, for one, takes its levels from static config instead.
     let rungs = entry["reasoning_budget"]
         .as_array()
         .expect("reasoning_budget advertised for a reasoning model");
@@ -177,17 +182,34 @@ capabilities = ["text"]
         .iter()
         .map(|r| r["effort"].as_str().expect("effort"))
         .collect();
-    assert_eq!(names, ["minimal", "low", "medium", "high"]);
+    assert_eq!(names, ["low", "medium", "xhigh"]);
+    // Which rung applies when the caller names none. Without this a
+    // silent caller cannot tell what it is getting — on Qwen3.8 that is
+    // xhigh, the most expensive rung on the ladder.
+    let default_rung: Vec<&str> = rungs
+        .iter()
+        .filter(|r| r["default"].as_bool().unwrap_or(false))
+        .map(|r| r["effort"].as_str().expect("effort"))
+        .collect();
+    assert_eq!(
+        default_rung,
+        ["xhigh"],
+        "the model's own default must be visible"
+    );
+    // A named rung carries no token count: effort reaches the model
+    // through its template, and enforcing it by truncation is the defect
+    // #290 fixed.
     assert!(
-        rungs
-            .iter()
-            .all(|r| r["tokens"].as_u64().is_some_and(|t| t > 0)),
-        "every advertised rung must name a token count"
+        rungs.iter().all(|r| r.get("tokens").is_none()),
+        "a named rung must not masquerade as a token budget"
     );
 
-    // The window under the two names generic clients actually read.
-    // pi-ai looks for `context_window` then `context_length` and gives
-    // up if neither is present — `max_model_len` is invisible to it.
+    // The window under the two names a generic client may read, for one
+    // that probes these rather than `max_model_len`. Publish all three;
+    // do not assume a particular client consumes any of them. pi, in
+    // particular, reads none — `contextWindow` is a required static
+    // field on its model type (checked against 0.82.1 and 0.84.3), so
+    // its window is hand-maintained whatever we advertise.
     assert_eq!(entry["context_window"], 49152);
     assert_eq!(entry["context_length"], 49152);
 

crates/helexa-bench/src/api.rs

@@ -34,6 +34,7 @@ pub fn api_routes(state: ApiState) -> Router {
     Router::new()
         .route("/api/health", get(health))
         .route("/api/dimensions", get(dimensions))
+        .route("/api/regimes", get(regimes))
         .route("/api/summary", get(summary))
         .route("/api/scaling", get(scaling))
         .route("/api/concurrency", get(concurrency))
@@ -65,6 +66,14 @@ async fn health(State(s): State<ApiState>) -> Result<Json<serde_json::Value>, Ap
     Ok(Json(json!({ "status": "ok", "run_count": count })))
 }
 
+/// Declared measurement-regime boundaries (#288).
+///
+/// Static — no store access — but served from the API rather than
+/// hardcoded in the frontend so `report` and the UI cite one list.
+async fn regimes() -> Json<&'static [crate::regime::MeasurementRegime]> {
+    Json(crate::regime::REGIMES)
+}
+
 async fn dimensions(State(s): State<ApiState>) -> Result<Json<crate::store::Dimensions>, ApiError> {
     let store = s.lock().await;
     store.dimensions().map(Json).map_err(err500)

crates/helexa-bench/src/client.rs

@@ -217,6 +217,7 @@ impl TargetClient {
             // entry supplies the override; it would matter if this were
             // ever used to load a model the host has no config for.
             sampling: None,
+            preserve_thinking: None,
         })
     }
 

crates/helexa-bench/src/lib.rs

@@ -7,6 +7,7 @@
 pub mod api;
 pub mod client;
 pub mod config;
+pub mod regime;
 pub mod report;
 pub mod scenario;
 pub mod store;

crates/helexa-bench/src/regime.rs

@@ -0,0 +1,148 @@
+//! Measurement-regime boundaries — builds where a number changed
+//! meaning rather than value.
+//!
+//! A trend line silently spans them, and every reader re-derives the
+//! same thing: that a cliff was the instrument, not the engine. Three
+//! have already cost real investigation time:
+//!
+//! - beast's `concurrency:8` fell 168.9 → 104.6 tok/s with `ttft_p95`
+//!   rising 0.53 → 11.77 s, and it read as a 38% throughput regression
+//!   until the one commit in the window turned out to be an admission
+//!   *policy* change that bench, being anonymous, was subject to (#288).
+//! - quadbrat's `capability:debug-reason` TTFT fell 88.8 → 0.1 s, which
+//!   read as a latency win and was a redefinition of when the clock
+//!   stops.
+//! - the same redefinition put decode rates at ~250 tok/s on a 3060
+//!   beforehand, by dividing reasoning-inclusive token counts by a
+//!   visible-content-only window.
+//!
+//! So the boundaries are declared, not remembered. Declared here rather
+//! than in the frontend because `report` renders the same series into
+//! `benchmarks.md` and needs the same caveats.
+//!
+//! ## What belongs here
+//!
+//! Only changes to **what a metric means**. A change that made the
+//! server genuinely faster, slower, or differently behaved does *not*
+//! belong — that is the signal these lines exist to keep readable. #280
+//! (fetch the model's own chat template) moved quadbrat's
+//! `completion_tokens` from 4096 to 1053 by ending a truncation loop;
+//! that is a real behavioural change and stays unmarked.
+//!
+//! The identified/anonymous split is deliberately absent too: it is
+//! recorded per-row in `runs.principal`, so the UI derives it from the
+//! data instead of trusting a constant to stay true.
+
+use serde::Serialize;
+
+/// A build at which one or more metrics changed meaning.
+#[derive(Debug, Clone, Serialize)]
+pub struct MeasurementRegime {
+    /// Short SHA of the first build measured under the new semantics.
+    /// Matched against `SeriesPoint::git_sha`, so a boundary whose build
+    /// never appears in a selection simply does not draw.
+    pub first_sha: &'static str,
+    /// Terse label for the chart rule.
+    pub label: &'static str,
+    /// What changed, and why the step is not a performance result.
+    pub detail: &'static str,
+    /// Series keys the change affects, as the API/UI name them. A
+    /// boundary only draws on panels carrying one of these — marking
+    /// every panel would train people to ignore the rules.
+    pub affects: &'static [&'static str],
+}
+
+/// The declared boundaries, oldest first.
+pub const REGIMES: &[MeasurementRegime] = &[
+    MeasurementRegime {
+        first_sha: "92dddc0",
+        label: "#262 anonymous callers yield",
+        detail: "Anonymous traffic became servable only from capacity left over \
+                 once identified traffic is satisfied — capped below max_in_flight \
+                 and parked at the class gate. An anonymous bench therefore \
+                 characterises the yield policy rather than serving capacity. \
+                 Steps here are admission policy, not the engine (#288).",
+        affects: &["decode", "ttft", "ttftP95", "queueWait", "rejected"],
+    },
+    MeasurementRegime {
+        first_sha: "0fe7aa3",
+        label: "#117 reasoning counts as liveness",
+        detail: "TTFT previously waited for the first *visible content* chunk, so \
+                 on a thinking model it measured the whole reasoning span (88.8 s \
+                 on quadbrat). It now stops at the first delta of any kind. The \
+                 same change fixed decode rates computed by dividing \
+                 reasoning-inclusive token counts by a content-only window, which \
+                 had produced ~250 tok/s on a 3060.",
+        affects: &["decode", "ttft"],
+    },
+];
+
+/// Boundaries affecting `metric`, in declaration order.
+pub fn regimes_for(metric: &str) -> Vec<&'static MeasurementRegime> {
+    REGIMES
+        .iter()
+        .filter(|r| r.affects.contains(&metric))
+        .collect()
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    /// A boundary that names no metric would draw nowhere and mislead by
+    /// omission; one naming an unknown key would draw nowhere and look
+    /// fine. Both are typos this catches.
+    #[test]
+    fn every_regime_affects_a_known_series_key() {
+        // The keys `Trends.tsx` maps `SeriesPoint` onto.
+        const KNOWN: &[&str] = &[
+            "decode",
+            "ttft",
+            "ttftP95",
+            "queueWait",
+            "rejected",
+            "prefillTps",
+            "reasoning",
+            "cached",
+            "completion",
+            "tpot",
+        ];
+        for r in REGIMES {
+            assert!(
+                !r.affects.is_empty(),
+                "{} affects nothing, so it would never draw",
+                r.first_sha
+            );
+            for key in r.affects {
+                assert!(
+                    KNOWN.contains(key),
+                    "{} names unknown series key {key:?}",
+                    r.first_sha
+                );
+            }
+        }
+    }
+
+    #[test]
+    fn regimes_for_selects_by_metric() {
+        assert!(regimes_for("ttft").iter().any(|r| r.first_sha == "0fe7aa3"));
+        assert!(
+            regimes_for("completion").is_empty(),
+            "#280 changed behaviour, not meaning — it must not be marked"
+        );
+    }
+
+    /// Short SHAs are matched against `SeriesPoint::git_sha`, which the
+    /// store records at the width `/version` reports.
+    #[test]
+    fn declared_shas_are_short_form() {
+        for r in REGIMES {
+            assert_eq!(
+                r.first_sha.len(),
+                7,
+                "{} is not a 7-char short sha; it would never match a series point",
+                r.first_sha
+            );
+        }
+    }
+}

crates/helexa-bench/src/scenario.rs

@@ -458,9 +458,38 @@ impl Scenario for ConcurrencyScenario {
             total_s: burst_window,
             prompt_tokens: streams.iter().find_map(|m| m.prompt_tokens),
             completion_tokens: total_tokens,
-            prefill_ms: None,
-            decode_ms: None,
-            prefill_tokens: None,
+            // Median server-side phase timing across the burst. These
+            // were `None`, which left the prefill-tok/s panel blank for
+            // exactly the cells where queue behaviour matters most, and
+            // made `queue_wait_ms` (TTFT minus server prefill, #85)
+            // uncomputable for a burst — the one number that separates
+            // "the server is slow" from "you were queued".
+            //
+            // Median rather than sum: every stream prefills its own copy
+            // of the same prompt, so the representative per-stream cost
+            // is the meaningful figure and it composes with `ttft_s`,
+            // which is also a median.
+            prefill_ms: median(
+                &streams
+                    .iter()
+                    .filter_map(|m| m.prefill_ms.map(|v| v as f64))
+                    .collect::<Vec<_>>(),
+            )
+            .map(|v| v as u64),
+            decode_ms: median(
+                &streams
+                    .iter()
+                    .filter_map(|m| m.decode_ms.map(|v| v as f64))
+                    .collect::<Vec<_>>(),
+            )
+            .map(|v| v as u64),
+            prefill_tokens: median(
+                &streams
+                    .iter()
+                    .filter_map(|m| m.prefill_tokens.map(|v| v as f64))
+                    .collect::<Vec<_>>(),
+            )
+            .map(|v| v as u64),
             // Summed across the burst, like `completion_tokens`: the cell
             // describes the whole burst, so a per-stream figure here
             // would not compose with it.
@@ -854,6 +883,52 @@ async fn stream_and_measure_inner(
 mod tests {
     use super::*;
 
+    /// A burst must report server-side phase timing, or `queue_wait_ms`
+    /// (TTFT minus server prefill, #85) cannot be computed for the very
+    /// cells it was designed for, and the prefill-tok/s panel stays
+    /// blank under concurrency.
+    #[test]
+    fn a_burst_reports_median_phase_timing_not_none() {
+        let s = |prefill_ms, decode_ms, prefill_tokens| ScenarioMetrics {
+            ttft_s: 0.1,
+            decode_tps: Some(10.0),
+            total_s: 1.0,
+            prompt_tokens: Some(100),
+            completion_tokens: 10,
+            prefill_ms,
+            decode_ms,
+            prefill_tokens,
+            reasoning_tokens: None,
+            cached_tokens: None,
+            tpot_p95_ms: Some(5.0),
+            concurrency: None,
+            ttft_p95_s: None,
+            queue_wait_ms_median: None,
+            rejected: None,
+            artifact: None,
+            image_units: None,
+        };
+        let streams = [
+            s(Some(10), Some(100), Some(512)),
+            s(Some(20), Some(200), Some(512)),
+            s(Some(30), Some(300), Some(512)),
+        ];
+        let med = |f: fn(&ScenarioMetrics) -> Option<u64>| {
+            median(
+                &streams
+                    .iter()
+                    .filter_map(|m| f(m).map(|v| v as f64))
+                    .collect::<Vec<_>>(),
+            )
+            .map(|v| v as u64)
+        };
+        assert_eq!(med(|m| m.prefill_ms), Some(20));
+        assert_eq!(med(|m| m.decode_ms), Some(200));
+        // Every stream prefills the same prompt, so the median is the
+        // prompt size rather than a sum across the burst.
+        assert_eq!(med(|m| m.prefill_tokens), Some(512));
+    }
+
     /// #288/#54: a burst must present one identity *per stream*.
     ///
     /// Fair-share bounds a single principal to `max_per_principal`

crates/neuron/src/harness/arch/qwen3_5/linear_attn.rs

@@ -380,7 +380,23 @@ impl GatedDeltaNet {
             self.head_v_dim,
         )?;
         // Stash the updated recurrent state for the next call.
-        self.state.recurrent_state = Some(new_state.to_dtype(dtype)?);
+        //
+        // The delta rule ran in f32 and `state_init` upcasts back to f32
+        // on entry, so storing at the model dtype is a round trip that
+        // buys only memory (#284). Decode takes it once per token.
+        //
+        // Storing f32 keeps `new_state` as-is, which on the CUDA decode
+        // path is the same buffer the kernel updated in place — so the
+        // stored state aliases it, where the `to_dtype` branch always
+        // allocated afresh. That is sound: the pre-update value is dead
+        // by this point, and prefix snapshots deep-copy
+        // (`snapshot_state`) precisely so they never share storage with
+        // a buffer a later forward will overwrite.
+        self.state.recurrent_state = Some(if gdn_state_f32() {
+            new_state
+        } else {
+            new_state.to_dtype(dtype)?
+        });
 
         // core_attn_out: (B, H, L, D_v) → (B, L, H, D_v) → (B*L*H, D_v).
         let core_attn_out = core_attn_out.transpose(1, 2)?.contiguous()?; // (B, L, H, D_v)
@@ -464,6 +480,33 @@ fn chunked_prefill_enabled() -> bool {
     })
 }
 
+/// `NEURON_GDN_STATE_F32=1` keeps the recurrent state in f32 between
+/// forwards instead of storing it at the model's activation dtype (#284).
+///
+/// The delta rule computes in f32 throughout, so storing the result at
+/// BF16 and upcasting it again next call is a pure round-trip loss.
+/// Measured on the chunked path: bit-exact carrying f32 at every chunk
+/// size, against 1.3e-3 relative error after 16 prefill chunks and
+/// **1.0e-2 after 4096 decode steps** — decode round-trips once per
+/// token, across 48 of this arch's 64 layers.
+///
+/// Off by default on purpose. Every #283 measurement was taken against
+/// the bf16 baseline, and flipping this silently would move the ground
+/// underneath the sampling A/B and the decode-rate work (#282, #285)
+/// at the same time. Turn it on deliberately, measure, then decide the
+/// default.
+///
+/// Costs ~2× the recurrent slab: +37.7 MB per GPU per sequence on the
+/// deployed 27B under TP=2, so +302 MB/GPU at `max_in_flight = 8`.
+fn gdn_state_f32() -> bool {
+    static ENABLED: std::sync::OnceLock<bool> = std::sync::OnceLock::new();
+    *ENABLED.get_or_init(|| {
+        std::env::var("NEURON_GDN_STATE_F32")
+            .map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
+            .unwrap_or(false)
+    })
+}
+
 /// Chunk-parallel gated delta rule — a faithful port of the HF
 /// reference `torch_chunk_gated_delta_rule` (chunk_size = 64) in
 /// `transformers/models/qwen3_5/modeling_qwen3_5.py`, minus the steps
@@ -1097,6 +1140,94 @@ mod tests {
         assert!(d_state < 2e-4, "final state diverged: {d_state}");
     }
 
+    /// #284: the state carried *between* forwards must not lose
+    /// precision. Every other parity test here is single-call and
+    /// all-f32, so none of them crosses the boundary where
+    /// `GatedDeltaNet::forward` stores the state at the model's
+    /// activation dtype — which is why the defect lived.
+    ///
+    /// Two properties, one test:
+    ///
+    /// 1. carrying f32, an N-chunk run equals a single call **exactly**,
+    ///    so the chunk-boundary math itself is sound (this kills the
+    ///    "chunked prefill corrupts state" hypothesis, #283 H1);
+    /// 2. carrying the model dtype loses a measurable, chunk-count-
+    ///    dependent amount — the defect the `NEURON_GDN_STATE_F32` flag
+    ///    exists to switch off.
+    ///
+    /// Asserting (2) is deliberate: if a future change makes the bf16
+    /// carry lossless this test fails loudly and the flag can go, rather
+    /// than the flag quietly outliving its reason.
+    #[test]
+    fn state_carried_across_chunks_is_exact_in_f32_and_lossy_in_bf16() {
+        let (b, h, dk, dv) = (1, 2, 32, 32);
+        let dev = Device::Cpu;
+        let l = 1024;
+        // Long-memory decay: g near zero keeps the state alive across
+        // every boundary, which is both the realistic regime for
+        // long-context retrieval and the one that accumulates most.
+        let scale = 1.0 / (dk as f64).sqrt();
+        let q = Tensor::randn(0f32, 1.0, (b, h, l, dk), &dev).unwrap();
+        let q = (l2norm(&q, 1e-6).unwrap() * scale).unwrap();
+        let k = Tensor::randn(0f32, 1.0, (b, h, l, dk), &dev).unwrap();
+        let k = l2norm(&k, 1e-6).unwrap();
+        let v = (Tensor::randn(0f32, 1.0, (b, h, l, dv), &dev).unwrap() * 0.5).unwrap();
+        let g = (Tensor::rand(0f32, 1f32, (b, h, l), &dev).unwrap() * -0.02).unwrap();
+        let beta = Tensor::rand(0.05f32, 0.95f32, (b, h, l), &dev).unwrap();
+        let zeros = || Tensor::zeros((b, h, dk, dv), DType::F32, &dev).unwrap();
+
+        // Ground truth: the whole sequence in one call, f32 throughout.
+        let (out_ref, _) = run_chunk_gated_delta_rule(&q, &k, &v, &g, &beta, zeros()).unwrap();
+
+        // Replay it in `chunk`-sized pieces, carrying the state the way
+        // `forward` does.
+        let replay = |chunk: usize, carry: Option<DType>| {
+            let mut state = zeros();
+            let mut outs = Vec::new();
+            let mut pos = 0;
+            while pos < l {
+                let n = chunk.min(l - pos);
+                let take = |t: &Tensor| t.narrow(2, pos, n).unwrap().contiguous().unwrap();
+                let (o, s) = run_chunk_gated_delta_rule(
+                    &take(&q),
+                    &take(&k),
+                    &take(&v),
+                    &take(&g),
+                    &take(&beta),
+                    state,
+                )
+                .unwrap();
+                state = match carry {
+                    Some(dt) => s.to_dtype(dt).unwrap().to_dtype(DType::F32).unwrap(),
+                    None => s,
+                };
+                outs.push(o);
+                pos += n;
+            }
+            Tensor::cat(&outs, 2).unwrap()
+        };
+
+        for chunk in [128usize, 256, 512] {
+            let exact = max_abs_diff(&out_ref, &replay(chunk, None));
+            assert_eq!(
+                exact, 0.0,
+                "chunk={chunk}: an f32 state carry must reproduce the \
+                 single-call result bit for bit, got {exact:e}"
+            );
+        }
+
+        // And the dtype round trip is what costs us. Compared on the
+        // final chunk, where the accumulated state has done the most work.
+        let tail = |t: &Tensor| t.narrow(2, l - 128, 128).unwrap().contiguous().unwrap();
+        let lossy = max_abs_diff(&tail(&out_ref), &tail(&replay(128, Some(DType::BF16))));
+        assert!(
+            lossy > 1e-5,
+            "storing the state at BF16 between chunks is expected to lose \
+             precision (#284); if this now passes losslessly the flag can \
+             be retired, got {lossy:e}"
+        );
+    }
+
     /// Adversarially correlated inputs: near-identical keys with
     /// beta ≈ 1 and negligible decay make the UT-transform matrix T
     /// maximally coherent — raw powers of T grow combinatorially

crates/neuron/src/harness/candle.rs

@@ -429,6 +429,16 @@ impl LoadedHandle {
         }
     }
 
+    /// Effort rungs the model's template accepts (#290).
+    pub fn reasoning_efforts(&self) -> super::reasoning_effort::SupportedEfforts {
+        match self {
+            LoadedHandle::Single(m) => m.reasoning_efforts.clone(),
+            #[cfg(feature = "cuda")]
+            LoadedHandle::Tp(m) => m.reasoning_efforts.clone(),
+            LoadedHandle::Image(_) => Default::default(),
+        }
+    }
+
     /// Self-derived `limit{context,input,output}` (#67) from this model's
     /// captured physics + live tightest-card free VRAM + its self-measured
     /// prefill rate (the configured bootstrap estimate until the first
@@ -590,22 +600,49 @@ impl LoadedImageModel {
 /// token count wins over a ladder rung — it is the more specific thing
 /// the caller reached for.
 ///
-/// `None` means unbounded, which is the default and exactly how the
-/// model behaved before this existed.
+/// An explicit `reasoning.max_tokens` wins. Failing that, the budget is
+/// the caller's own output budget less `answer_reserve_tokens`, so an
+/// OpenAI-shaped client — which has no way to send a token count — still
+/// gets an answer rather than a think block that eats the lot.
+///
+/// Effort levels used to be mapped onto a token ladder here and enforced
+/// by force-injecting `</think>` at the limit. That made effort mean two
+/// incompatible things at once: a prompt instruction to the model, and a
+/// guillotine applied to it. On `Qwen/Qwen3.8-27B` the model was told to
+/// reason at `xhigh` (the template default, because the caller's effort
+/// never reached the template) and cut off at `medium`'s 12,288 tokens —
+/// twice, to the token, in one agentic session — leaving it to answer
+/// from an unfinished plan.
+///
+/// Effort now reaches the model through its own template, where it is a
+/// trained control. The governor stays as a backstop for a caller that
+/// asks for a hard cap, which is what #223's runaway-think-block concern
+/// actually needs.
+///
+/// `None` means unbounded — reachable by configuring
+/// `answer_reserve_tokens = 0`, or when the reserve is not smaller than
+/// the whole output budget (a budget so small that reserving from it
+/// would leave no room to think at all; better to think freely and let
+/// the output cap bite than to emit a stub).
 fn requested_reasoning_budget(
     request: &ChatCompletionRequest,
     ladder: &super::reasoning_budget::ReasoningBudgetLadder,
+    max_new: usize,
 ) -> Option<usize> {
-    let reasoning = request.extra.get("reasoning")?;
-    if let Some(n) = reasoning
-        .get("max_tokens")
+    if let Some(explicit) = request
+        .extra
+        .get("reasoning")
+        .and_then(|r| r.get("max_tokens"))
         .and_then(|v| v.as_u64())
         .filter(|n| *n > 0)
     {
-        return Some(n as usize);
+        return Some(explicit as usize);
     }
-    let effort = reasoning.get("effort").and_then(|v| v.as_str())?;
-    ladder.for_effort(effort)
+    let reserve = ladder.answer_reserve_tokens;
+    if reserve == 0 || reserve >= max_new {
+        return None;
+    }
+    Some(max_new - reserve)
 }
 
 fn requested_sampling(request: &ChatCompletionRequest) -> super::sampling::RequestedSampling {
@@ -684,6 +721,14 @@ pub struct LoadedModel {
     /// [`InferenceEvent::ReasoningDelta`] at the token boundary;
     /// when `None` everything is `TextDelta`.
     pub reasoning_tokens: Option<ReasoningTokenPair>,
+    /// Effort rungs this model's own chat template accepts (#290),
+    /// discovered at load by rendering it. Empty when the template has
+    /// no effort control. Drives both what `/models` advertises and
+    /// which value we forward on a request.
+    pub reasoning_efforts: super::reasoning_effort::SupportedEfforts,
+    /// Operator's `preserve_thinking` for this model, or `None` to let
+    /// the template decide (#H7 A/B). See `ModelSpec::preserve_thinking`.
+    pub preserve_thinking: Option<bool>,
     /// Open/close token IDs for the model's tool-call marker
     /// pair (`<tool_call>` / `</tool_call>` on Qwen3-Coder / Hermes
     /// / DeepSeek / gpt-oss). `None` for models that don't emit
@@ -837,6 +882,14 @@ pub struct TpLoadedModel {
     /// load time. `None` when the model declares no reasoning
     /// markers.
     pub reasoning_tokens: Option<ReasoningTokenPair>,
+    /// Effort rungs this model's own chat template accepts (#290),
+    /// discovered at load by rendering it. Empty when the template has
+    /// no effort control. Drives both what `/models` advertises and
+    /// which value we forward on a request.
+    pub reasoning_efforts: super::reasoning_effort::SupportedEfforts,
+    /// Operator's `preserve_thinking` for this model, or `None` to let
+    /// the template decide (#H7 A/B). See `ModelSpec::preserve_thinking`.
+    pub preserve_thinking: Option<bool>,
     /// Same shape as [`LoadedModel::tool_call_tokens`].
     pub tool_call_tokens: Option<ToolCallTokenPair>,
     /// Same shape as [`LoadedModel::chat_template`].
@@ -2893,7 +2946,12 @@ impl CandleHarness {
             }
         };
 
-        let prompt = build_prompt_for_request(loaded.chat_template.as_deref(), &request)?;
+        let prompt = build_prompt_for_request(
+            loaded.chat_template.as_deref(),
+            &request,
+            &loaded.reasoning_efforts,
+            loaded.preserve_thinking,
+        )?;
         let encoding = loaded
             .tokenizer
             .encode(prompt.as_str(), true)
@@ -2955,23 +3013,30 @@ impl CandleHarness {
             &loaded.generation_defaults,
             unix_subsec_nanos(),
         );
-        // Bound the think block if the caller asked for it (#223).
+        // The output budget the request declared, under either
+        // spelling (#254), and the fallback when it declares none.
+        // One value drives generation, KV pricing and admission — they
+        // disagreed before, and the gate priced a budget nobody asked
+        // for (#278).
+        //
+        // Resolved before the governor because the governor's budget is
+        // now carved out of it.
+        let requested_output = request.effective_max_tokens().map(|t| t as usize);
+        let cfg_output_reserve = self.context_limit_cfg.output_reserve_tokens;
+        let max_new = requested_output.unwrap_or(cfg_output_reserve);
+
+        // Bound the think block so some of the caller's budget survives
+        // for the answer (#223).
+        let reasoning_budget =
+            requested_reasoning_budget(&request, &self.reasoning_budget_cfg, max_new);
         let governor = super::reasoning_budget::ReasoningGovernor::new(
-            requested_reasoning_budget(&request, &self.reasoning_budget_cfg),
+            reasoning_budget,
             loaded
                 .reasoning_tokens
                 .as_ref()
                 .map(|p| (p.open_id, p.close_id)),
             prompt_opens_reasoning(&prompt_tokens, loaded.reasoning_tokens.as_ref()),
         );
-        // The output budget the request declared, under either
-        // spelling (#254), and the fallback when it declares none.
-        // One value drives generation, KV pricing and admission — they
-        // disagreed before, and the gate priced a budget nobody asked
-        // for (#278).
-        let requested_output = request.effective_max_tokens().map(|t| t as usize);
-        let cfg_output_reserve = self.context_limit_cfg.output_reserve_tokens;
-        let max_new = requested_output.unwrap_or(cfg_output_reserve);
 
         let eos_id = loaded
             .tokenizer
@@ -3093,10 +3158,7 @@ impl CandleHarness {
                     sampling: sampling.clone(),
                     // The engine builds its own per-slot governor from
                     // this, since slots in a batch differ (#223).
-                    reasoning_budget: requested_reasoning_budget(
-                        &request,
-                        &self.reasoning_budget_cfg,
-                    ),
+                    reasoning_budget,
                     eos_id,
                     tool_schemas,
                     tx,
@@ -3463,24 +3525,27 @@ impl Harness for CandleHarness {
                 tool_call: h.has_tool_call(),
                 reasoning: h.has_reasoning(),
                 servable: h.servability(&self.context_limit_cfg),
-                // What each effort level buys (#223). Only meaningful
-                // for a model that reasons — advertising rungs on a
-                // model that never opens a think block would describe a
-                // control that does nothing.
-                reasoning_budget: if h.has_reasoning() {
-                    self.reasoning_budget_cfg
-                        .advertised()
-                        .into_iter()
-                        .map(
-                            |(effort, tokens)| cortex_core::harness::ReasoningBudgetRung {
-                                effort,
-                                tokens,
-                            },
-                        )
-                        .collect()
-                } else {
-                    Vec::new()
-                },
+                // The rungs this model's own template accepts (#290),
+                // discovered at load rather than invented. Advertising a
+                // rung the template rejects hands callers a name that
+                // errors, and omitting the model's default hides its
+                // strongest setting. (Whether a given client reads this
+                // is another matter — pi-ai does not; see
+                // `reasoning_effort`'s module doc.)
+                reasoning_budget: h
+                    .reasoning_efforts()
+                    .levels
+                    .iter()
+                    .map(|effort| cortex_core::harness::ReasoningBudgetRung {
+                        default: h.reasoning_efforts().default.as_deref() == Some(effort.as_str()),
+                        effort: effort.clone(),
+                        // No cap: effort reaches the model through its
+                        // template, and how many tokens it then spends
+                        // is the model's business. Enforcing effort by
+                        // truncation is the defect #290 fixed.
+                        tokens: None,
+                    })
+                    .collect(),
             });
         }
         // Models mid-recovery whose registry slot is absent (the
@@ -3779,9 +3844,20 @@ impl Harness for CandleHarness {
                 .unwrap_or_else(|| std::path::Path::new(".")),
         )
         .with_operator_override(&spec.model_id, spec.sampling.as_ref());
+        // Which effort rungs this model actually accepts (#290), asked
+        // of the template rather than assumed. Empty when it has no
+        // effort control, which is the pre-#290 behaviour.
+        let reasoning_efforts = chat_template
+            .as_deref()
+            .map(|t| super::reasoning_effort::probe(t, &spec.model_id))
+            .transpose()
+            .unwrap_or_default()
+            .unwrap_or_default();
         let loaded = Arc::new(LoadedModel {
             model_id: spec.model_id.clone(),
             generation_defaults,
+            reasoning_efforts,
+            preserve_thinking: spec.preserve_thinking,
             arch: arch_local,
             tokenizer,
             device,
@@ -4422,9 +4498,20 @@ impl CandleHarness {
                 .unwrap_or_else(|| std::path::Path::new(".")),
         )
         .with_operator_override(&spec.model_id, spec.sampling.as_ref());
+        // Which effort rungs this model actually accepts (#290), asked
+        // of the template rather than assumed. Empty when it has no
+        // effort control, which is the pre-#290 behaviour.
+        let reasoning_efforts = chat_template
+            .as_deref()
+            .map(|t| super::reasoning_effort::probe(t, &spec.model_id))
+            .transpose()
+            .unwrap_or_default()
+            .unwrap_or_default();
         let tp_loaded = StdArc::new(TpLoadedModel {
             model_id: spec.model_id.clone(),
             generation_defaults,
+            reasoning_efforts,
+            preserve_thinking: spec.preserve_thinking,
             tokenizer,
             devices: devices.clone(),
             pool: StdArc::new(TMutex::new(pool)),
@@ -4561,7 +4648,12 @@ impl CandleHarness {
             });
         }
 
-        let prompt = build_prompt_for_request(tp.chat_template.as_deref(), &request)?;
+        let prompt = build_prompt_for_request(
+            tp.chat_template.as_deref(),
+            &request,
+            &tp.reasoning_efforts,
+            tp.preserve_thinking,
+        )?;
         let encoding = tp
             .tokenizer
             .encode(prompt.as_str(), true)
@@ -4627,23 +4719,33 @@ impl CandleHarness {
             &tp.generation_defaults,
             unix_subsec_nanos(),
         );
-        // Bound the think block if the caller asked for it (#223).
-        let mut governor = super::reasoning_budget::ReasoningGovernor::new(
-            requested_reasoning_budget(&request, &self.reasoning_budget_cfg),
-            tp.reasoning_tokens
-                .as_ref()
-                .map(|p| (p.open_id, p.close_id)),
-            prompt_opens_reasoning(&prompt_tokens, tp.reasoning_tokens.as_ref()),
-        );
         // The output budget the request declared, under either
         // spelling (#254), and the fallback when it declares none.
         // One value drives generation, KV pricing and admission — they
         // disagreed before, and the gate priced a budget nobody asked
         // for (#278).
+        //
+        // Resolved before the governor because the governor's budget is
+        // now carved out of it.
         let requested_output = request.effective_max_tokens().map(|t| t as usize);
         let cfg_output_reserve = self.context_limit_cfg.output_reserve_tokens;
         let max_new = requested_output.unwrap_or(cfg_output_reserve);
 
+        // Bound the think block so some of the caller's budget survives
+        // for the answer (#223). Resolved into a local because the
+        // batched-engine path below needs the same value — the two must
+        // not diverge, or a request bounds differently depending on
+        // whether it was batched.
+        let reasoning_budget =
+            requested_reasoning_budget(&request, &self.reasoning_budget_cfg, max_new);
+        let mut governor = super::reasoning_budget::ReasoningGovernor::new(
+            reasoning_budget,
+            tp.reasoning_tokens
+                .as_ref()
+                .map(|p| (p.open_id, p.close_id)),
+            prompt_opens_reasoning(&prompt_tokens, tp.reasoning_tokens.as_ref()),
+        );
+
         let eos_id = tp
             .tokenizer
             .token_to_id("<|im_end|>")
@@ -4748,10 +4850,7 @@ impl CandleHarness {
                     sampling: sampling.clone(),
                     // The engine builds its own per-slot governor from
                     // this, since slots in a batch differ (#223).
-                    reasoning_budget: requested_reasoning_budget(
-                        &request,
-                        &self.reasoning_budget_cfg,
-                    ),
+                    reasoning_budget,
                     eos_id,
                     tool_schemas,
                     tx,
@@ -5748,9 +5847,114 @@ impl From<super::admission::AdmissionRejection> for InferenceError {
 ///
 /// Failures are logged at `warn` so an operator running with
 /// `RUST_LOG=neuron=debug` sees which path each request took.
+/// Resolve the request's reasoning effort onto a rung the model accepts
+/// and set it as a `chat_template_kwargs.reasoning_effort` (#290).
+///
+/// Precedence, most specific first:
+///
+/// 1. an explicit `chat_template_kwargs.reasoning_effort` from the
+///    caller — they reached for the model-native control, so it passes
+///    through untouched even if we would have mapped it elsewhere;
+/// 2. `reasoning.effort`, mapped through
+///    [`SupportedEfforts::nearest`](super::reasoning_effort::SupportedEfforts::nearest);
+/// 3. nothing, leaving the template to apply its own default.
+///
+/// `none`/`off` are deliberately not handled here: they are not an
+/// effort but a request to disable thinking entirely, which
+/// `apply_reasoning_effort` already turns into
+/// `enable_thinking = false` before this runs.
+fn apply_effort_kwarg(
+    kwargs: serde_json::Value,
+    request: &ChatCompletionRequest,
+    efforts: &super::reasoning_effort::SupportedEfforts,
+) -> serde_json::Value {
+    if efforts.is_empty() {
+        return kwargs;
+    }
+    if kwargs.get("reasoning_effort").is_some() {
+        return kwargs;
+    }
+    let Some(requested) = request
+        .extra
+        .get("reasoning")
+        .and_then(|r| r.get("effort"))
+        .and_then(|e| e.as_str())
+        .map(|e| e.trim().to_ascii_lowercase())
+    else {
+        return kwargs;
+    };
+    if matches!(requested.as_str(), "none" | "off") {
+        return kwargs;
+    }
+    let Some(resolved) = efforts.nearest(&requested) else {
+        // An effort name this model has no rung for and no neighbour to
+        // fall back to. Passing it through would make the template
+        // raise; dropping it lets the model use its own default, which
+        // is the conservative choice.
+        tracing::debug!(
+            requested = %requested,
+            available = %efforts.levels.join(","),
+            "reasoning effort: no comparable rung; leaving the template default"
+        );
+        return kwargs;
+    };
+    if resolved != requested {
+        tracing::info!(
+            requested = %requested,
+            resolved = %resolved,
+            available = %efforts.levels.join(","),
+            "reasoning effort: mapped to the nearest rung this model accepts (#290)"
+        );
+    }
+    let mut obj = match kwargs {
+        serde_json::Value::Object(o) => o,
+        _ => serde_json::Map::new(),
+    };
+    obj.insert(
+        "reasoning_effort".into(),
+        serde_json::Value::String(resolved.to_string()),
+    );
+    serde_json::Value::Object(obj)
+}
+
+/// Apply the operator's `preserve_thinking`, if they set one.
+///
+/// Precedence is request > operator > the model's own template default.
+/// A caller that sends `chat_template_kwargs.preserve_thinking` has
+/// said something specific and keeps it; an operator who sets nothing
+/// changes nothing, so the template's `|default` still governs.
+///
+/// Deliberately *not* defaulted by us. On `Qwen/Qwen3.8-27B` the
+/// template preserves every turn's reasoning unless told otherwise —
+/// its authors' choice — and `false` narrows replay to the turn in
+/// progress. Both are defensible and the difference is measurable, so
+/// this exists to run that measurement rather than to encode a guess.
+fn apply_preserve_thinking_kwarg(
+    kwargs: serde_json::Value,
+    configured: Option<bool>,
+) -> serde_json::Value {
+    let Some(configured) = configured else {
+        return kwargs;
+    };
+    if kwargs.get("preserve_thinking").is_some() {
+        return kwargs;
+    }
+    let mut obj = match kwargs {
+        serde_json::Value::Object(o) => o,
+        _ => serde_json::Map::new(),
+    };
+    obj.insert(
+        "preserve_thinking".into(),
+        serde_json::Value::Bool(configured),
+    );
+    serde_json::Value::Object(obj)
+}
+
 fn build_prompt_for_request(
     chat_template: Option<&str>,
     request: &ChatCompletionRequest,
+    efforts: &super::reasoning_effort::SupportedEfforts,
+    preserve_thinking: Option<bool>,
 ) -> Result<String, InferenceError> {
     if !super::chat_template::chat_templates_enabled() {
         return Ok(format_qwen3_prompt(&request.messages));
@@ -5768,6 +5972,16 @@ fn build_prompt_for_request(
         .get("chat_template_kwargs")
         .cloned()
         .unwrap_or(serde_json::Value::Null);
+    // Tell the model which rung it is on (#290). Until this existed the
+    // caller's effort was consumed for our token governor and never
+    // reached the template, so the template always took its own
+    // `|default(...)` — on Qwen3.8 that is `xhigh`, meaning every
+    // request was instructed to think maximally and then truncated at
+    // whatever budget the caller had actually asked for.
+    let kwargs = apply_effort_kwarg(kwargs, request, efforts);
+    // Whether prior turns keep their think blocks. Unset unless the
+    // operator chose, so the template's own default stands.
+    let kwargs = apply_preserve_thinking_kwarg(kwargs, preserve_thinking);
     let tools = request
         .extra
         .get("tools")
@@ -7125,6 +7339,251 @@ fn unix_subsec_nanos() -> u64 {
 #[cfg(test)]
 mod tests {
     use super::super::context_limit::ContextProfile;
+    use super::{
+        ChatCompletionRequest, ChatMessage, MessageContent, apply_effort_kwarg,
+        requested_reasoning_budget,
+    };
+
+    // ── Reasoning-effort forwarding (#290) ────────────────────────
+
+    fn qwen38_efforts() -> super::super::reasoning_effort::SupportedEfforts {
+        super::super::reasoning_effort::SupportedEfforts {
+            levels: vec!["low".into(), "medium".into(), "xhigh".into()],
+            default: Some("xhigh".into()),
+        }
+    }
+
+    fn req_with_effort(effort: Option<&str>) -> ChatCompletionRequest {
+        let mut extra = serde_json::Map::new();
+        if let Some(e) = effort {
+            extra.insert("reasoning".into(), serde_json::json!({ "effort": e }));
+        }
+        ChatCompletionRequest {
+            model: "m".into(),
+            messages: vec![ChatMessage {
+                role: "user".into(),
+                content: MessageContent::Text("hi".into()),
+                extra: serde_json::Value::Null,
+            }],
+            temperature: None,
+            top_p: None,
+            top_k: None,
+            seed: None,
+            repetition_penalty: None,
+            repeat_last_n: None,
+            max_tokens: None,
+            max_completion_tokens: None,
+            stream: None,
+            extra: serde_json::Value::Object(extra),
+        }
+    }
+
+    fn effort_in(kwargs: &serde_json::Value) -> Option<String> {
+        kwargs
+            .get("reasoning_effort")
+            .and_then(|v| v.as_str())
+            .map(str::to_string)
+    }
+
+    /// The core of #290: a caller's effort must reach the template. It
+    /// never did, so the template always took its own default (`xhigh`)
+    /// while the governor truncated at the caller's rung.
+    #[test]
+    fn a_requested_effort_reaches_the_template() {
+        let out = apply_effort_kwarg(
+            serde_json::Value::Null,
+            &req_with_effort(Some("medium")),
+            &qwen38_efforts(),
+        );
+        assert_eq!(effort_in(&out).as_deref(), Some("medium"));
+    }
+
+    /// `high` is not a rung on this model; the template raises on it, so
+    /// it must be mapped down rather than passed through.
+    #[test]
+    fn an_unsupported_effort_is_mapped_to_a_rung_the_template_accepts() {
+        let out = apply_effort_kwarg(
+            serde_json::Value::Null,
+            &req_with_effort(Some("high")),
+            &qwen38_efforts(),
+        );
+        assert_eq!(effort_in(&out).as_deref(), Some("medium"));
+    }
+
+    /// Saying nothing must leave the template's own default alone rather
+    /// than us choosing one on the model's behalf.
+    #[test]
+    fn no_requested_effort_leaves_the_template_default_untouched() {
+        let out = apply_effort_kwarg(
+            serde_json::Value::Null,
+            &req_with_effort(None),
+            &qwen38_efforts(),
+        );
+        assert_eq!(effort_in(&out), None);
+    }
+
+    /// `none`/`off` is not an effort — it disables thinking upstream.
+    /// Forwarding it would make the template raise.
+    #[test]
+    fn none_is_not_forwarded_as_an_effort() {
+        let out = apply_effort_kwarg(
+            serde_json::Value::Null,
+            &req_with_effort(Some("none")),
+            &qwen38_efforts(),
+        );
+        assert_eq!(effort_in(&out), None);
+    }
+
+    /// Unset by default: the model's authors chose the template's
+    /// default and we do not override it fleet-wide on a hunch.
+    #[test]
+    fn no_operator_setting_leaves_preserve_thinking_alone() {
+        let out = apply_preserve_thinking_kwarg(serde_json::Value::Null, None);
+        assert!(out.get("preserve_thinking").is_none());
+        assert!(out.is_null(), "an untouched Null must stay Null");
+    }
+
+    /// The A/B knob: an operator can narrow replay to the turn in
+    /// progress, or pin full replay, without a rebuild.
+    #[test]
+    fn an_operator_setting_reaches_the_template() {
+        for configured in [true, false] {
+            let out = apply_preserve_thinking_kwarg(serde_json::Value::Null, Some(configured));
+            assert_eq!(out["preserve_thinking"], serde_json::json!(configured));
+        }
+    }
+
+    /// A caller that named it has said something specific; the
+    /// operator's default must not overwrite it.
+    #[test]
+    fn a_requested_preserve_thinking_outranks_the_operator() {
+        let out = apply_preserve_thinking_kwarg(
+            serde_json::json!({ "preserve_thinking": true }),
+            Some(false),
+        );
+        assert_eq!(out["preserve_thinking"], serde_json::json!(true));
+    }
+
+    /// It must not clobber the effort kwarg sharing the same object —
+    /// both are applied to one `chat_template_kwargs`.
+    #[test]
+    fn it_preserves_kwargs_already_present() {
+        let out = apply_preserve_thinking_kwarg(
+            serde_json::json!({ "reasoning_effort": "medium", "custom": 1 }),
+            Some(false),
+        );
+        assert_eq!(out["reasoning_effort"], "medium");
+        assert_eq!(out["custom"], 1);
+        assert_eq!(out["preserve_thinking"], serde_json::json!(false));
+    }
+
+    /// A model whose template has no effort control must be left alone.
+    #[test]
+    fn a_model_without_rungs_gets_no_effort_kwarg() {
+        let out = apply_effort_kwarg(
+            serde_json::Value::Null,
+            &req_with_effort(Some("medium")),
+            &Default::default(),
+        );
+        assert_eq!(effort_in(&out), None);
+    }
+
+    /// Effort must not select a *rung's* budget — that was the
+    /// guillotine (#290): the model was told to reason at `xhigh` and
+    /// cut off at `medium`'s 12,288. The reserve below is not that; it
+    /// does not vary with effort.
+    #[test]
+    fn effort_no_longer_selects_a_rungs_token_budget() {
+        let ladder = super::super::reasoning_budget::ReasoningBudgetLadder::default();
+        let medium = requested_reasoning_budget(&req_with_effort(Some("medium")), &ladder, 32_768);
+        let xhigh = requested_reasoning_budget(&req_with_effort(Some("xhigh")), &ladder, 32_768);
+        assert_eq!(medium, xhigh, "the bound must not depend on effort");
+        assert_ne!(
+            medium,
+            Some(ladder.medium),
+            "medium must not become a {}-token guillotine",
+            ladder.medium
+        );
+    }
+
+    /// The regression this reserve exists to prevent (#223, reopened by
+    /// #290): an OpenAI-shaped client cannot send `reasoning.max_tokens`,
+    /// so removing the effort→ladder mapping left it with no bound at
+    /// all. Qwen3.8-27B then spent the entire 32,768-token budget
+    /// thinking and returned no answer — twice, on real sessions.
+    #[test]
+    fn a_caller_that_names_no_budget_still_keeps_room_to_answer() {
+        let ladder = super::super::reasoning_budget::ReasoningBudgetLadder::default();
+        let budget = requested_reasoning_budget(&req_with_effort(Some("xhigh")), &ladder, 32_768)
+            .expect("an unbounded think block is what broke");
+        assert_eq!(budget, 32_768 - ladder.answer_reserve_tokens);
+        assert!(
+            budget < 32_768,
+            "some of the caller's own budget must survive for the answer"
+        );
+    }
+
+    /// An explicit count is the more specific thing the caller reached
+    /// for, so it wins over the reserve in both directions.
+    #[test]
+    fn an_explicit_max_tokens_still_wins() {
+        let ladder = super::super::reasoning_budget::ReasoningBudgetLadder::default();
+        let mut req = req_with_effort(None);
+        if let serde_json::Value::Object(o) = &mut req.extra {
+            o.insert("reasoning".into(), serde_json::json!({ "max_tokens": 500 }));
+        }
+        assert_eq!(requested_reasoning_budget(&req, &ladder, 32_768), Some(500));
+
+        // Above the reserve-derived value, too — not clamped to it.
+        if let serde_json::Value::Object(o) = &mut req.extra {
+            o.insert(
+                "reasoning".into(),
+                serde_json::json!({ "max_tokens": 31_000 }),
+            );
+        }
+        assert_eq!(
+            requested_reasoning_budget(&req, &ladder, 32_768),
+            Some(31_000)
+        );
+    }
+
+    /// Reserving from a budget too small to think in would emit a stub.
+    /// Better to think freely and let the output cap bite.
+    #[test]
+    fn a_budget_smaller_than_the_reserve_stays_unbounded() {
+        let ladder = super::super::reasoning_budget::ReasoningBudgetLadder::default();
+        for max_new in [0, 1, ladder.answer_reserve_tokens] {
+            assert_eq!(
+                requested_reasoning_budget(&req_with_effort(None), &ladder, max_new),
+                None,
+                "max_new={max_new} must not produce a bound"
+            );
+        }
+        assert_eq!(
+            requested_reasoning_budget(
+                &req_with_effort(None),
+                &ladder,
+                ladder.answer_reserve_tokens + 1
+            ),
+            Some(1),
+            "one token above the reserve is the first bounded case"
+        );
+    }
+
+    /// The operator's escape hatch, and proof the reserve is the only
+    /// thing creating a bound.
+    #[test]
+    fn a_zero_reserve_restores_unbounded_reasoning() {
+        let ladder = super::super::reasoning_budget::ReasoningBudgetLadder {
+            answer_reserve_tokens: 0,
+            ..Default::default()
+        };
+        assert_eq!(
+            requested_reasoning_budget(&req_with_effort(Some("xhigh")), &ladder, 32_768),
+            None
+        );
+    }
+
     use super::super::sampling::SamplingParams;
     use super::sample_with_penalty;
     use candle_core::{Device, Tensor};
@@ -7144,8 +7603,16 @@ mod tests {
         }
     }
 
-    /// The three inlets converge on one field (#223): an explicit token
-    /// count, Anthropic's normalised `budget_tokens`, or an effort rung.
+    /// Two inlets converge on one field: an explicit token count, or
+    /// Anthropic's normalised `budget_tokens`, which translation folds
+    /// into the same shape.
+    ///
+    /// An effort rung deliberately no longer produces a budget (#290).
+    /// It used to, and that made effort mean two incompatible things at
+    /// once — a prompt instruction to the model, and a guillotine
+    /// applied to it. Effort now reaches the model through its own chat
+    /// template, where it is a trained control; the governor remains as
+    /// a backstop for a caller that asks for a hard cap.
     #[test]
     fn the_reasoning_budget_resolves_from_every_inlet() {
         use cortex_core::openai::{ChatCompletionRequest, MessageContent};
@@ -7166,43 +7633,59 @@ mod tests {
         };
         let _ = MessageContent::Null;
 
+        // Every case below shares one output budget, so the only thing
+        // that varies is the caller's reasoning control.
+        const OUT: usize = 32_768;
+        let reserved = OUT - ladder.answer_reserve_tokens;
+
         // Explicit count — also the shape Anthropic's `budget_tokens`
         // is normalised into during translation.
         assert_eq!(
             requested_reasoning_budget(
                 &req(serde_json::json!({"reasoning": {"max_tokens": 5000}})),
-                &ladder
+                &ladder,
+                OUT
             ),
             Some(5000)
         );
-        // Effort rung, resolved against the operator's configuration.
+        // An effort rung is NOT a budget (#290). `medium` used to resolve
+        // to 12,288 here and force-close the think block at that count —
+        // observed twice, to the token, in one agentic session, leaving
+        // the model to answer from an unfinished plan. It now falls back
+        // to the reserve, like every other caller that names no count.
         assert_eq!(
             requested_reasoning_budget(
                 &req(serde_json::json!({"reasoning": {"effort": "medium"}})),
-                &ladder
+                &ladder,
+                OUT
             ),
-            Some(12_288)
+            Some(reserved)
         );
-        // An explicit count beats a rung — it is the more specific ask.
+        // An explicit count still caps, whatever rung was named.
         assert_eq!(
             requested_reasoning_budget(
                 &req(serde_json::json!({"reasoning": {"effort": "low", "max_tokens": 999}})),
-                &ladder
+                &ladder,
+                OUT
             ),
             Some(999)
         );
-        // No control at all stays unbounded, exactly as before #223.
+        // No control at all: bounded by the reserve, not unbounded. This
+        // is the case pi sends, and the one that returned no answer at
+        // all while it was unbounded.
         assert_eq!(
-            requested_reasoning_budget(&req(serde_json::json!({})), &ladder),
-            None
+            requested_reasoning_budget(&req(serde_json::json!({})), &ladder, OUT),
+            Some(reserved)
         );
-        // `none` disables thinking via the template; it is not a budget.
+        // `none` disables thinking via the template, so the governor
+        // never fires; the reserve it reports is harmless either way.
         assert_eq!(
             requested_reasoning_budget(
                 &req(serde_json::json!({"reasoning": {"effort": "none"}})),
-                &ladder
+                &ladder,
+                OUT
             ),
-            None
+            Some(reserved)
         );
     }
 
@@ -8152,7 +8635,8 @@ mod tests {
             "tools": [{"type": "function", "function": {"name": "x"}}]
         }))
         .unwrap();
-        let err = build_prompt_for_request(Some(bad), &with_tools).unwrap_err();
+        let err = build_prompt_for_request(Some(bad), &with_tools, &Default::default(), None)
+            .unwrap_err();
         assert!(matches!(err, InferenceError::TemplateRenderFailed { .. }));
 
         // No tools → falling back is harmless, so it stays Ok.
@@ -8161,6 +8645,6 @@ mod tests {
             "messages": [{"role": "user", "content": "hi"}]
         }))
         .unwrap();
-        assert!(build_prompt_for_request(Some(bad), &no_tools).is_ok());
+        assert!(build_prompt_for_request(Some(bad), &no_tools, &Default::default(), None).is_ok());
     }
 }

crates/neuron/src/harness/chat_template.rs

@@ -438,6 +438,75 @@ mod tests {
         assert!(out.contains("<|vision_start|>") && out.contains("<|vision_end|>"));
     }
 
+    /// Blank lines inside a replayed tool-call argument must survive
+    /// rendering intact.
+    ///
+    /// A model that writes a source file and then re-reads it from its
+    /// own replayed tool call is comparing what it remembers writing
+    /// against what the prompt now shows. Any whitespace the pipeline
+    /// eats makes those disagree, and the model has no way to tell a
+    /// rendering artifact from a mistake of its own — which is a
+    /// plausible route into the announce-then-stop stall (#296).
+    ///
+    /// The JSON string→object normalisation above is the step that
+    /// could plausibly lose them, since it round-trips the arguments
+    /// through a parser.
+    #[test]
+    fn a_replayed_tool_call_keeps_the_blank_lines_in_its_arguments() {
+        let template = include_str!("../../tests/data/qwen3.8-27b.chat_template.jinja");
+        // Blank lines between sections, exactly as a written source file
+        // carries them.
+        let file_body = "'use strict';\n\n// ---- constants ----\nconst A = 1;\n\n\nfunction f() {\n  return A;\n}\n";
+        let args = json!({ "path": "/tmp/game.js", "content": file_body });
+        let mut assistant_extras = serde_json::Map::new();
+        assistant_extras.insert(
+            "tool_calls".into(),
+            json!([{
+                "id": "call_0", "type": "function",
+                "function": {
+                    "name": "write",
+                    // The OpenAI wire form: arguments as a JSON *string*.
+                    "arguments": serde_json::to_string(&args).unwrap()
+                }
+            }]),
+        );
+        let mut tool_extras = serde_json::Map::new();
+        tool_extras.insert("tool_call_id".into(), json!("call_0"));
+
+        let messages = vec![
+            user_msg("write the file"),
+            ChatMessage {
+                role: "assistant".into(),
+                content: MessageContent::Null,
+                extra: Value::Object(assistant_extras),
+            },
+            ChatMessage {
+                role: "tool".into(),
+                content: MessageContent::Text("Successfully wrote 84 bytes".into()),
+                extra: Value::Object(tool_extras),
+            },
+            user_msg("now check it parses"),
+        ];
+        let out =
+            render_chat_template(template, &messages, &Value::Null, &Value::Null).expect("render");
+
+        assert!(
+            out.contains(file_body),
+            "the file body must appear verbatim in the rendered prompt"
+        );
+        let blanks_in = file_body.lines().filter(|l| l.is_empty()).count();
+        assert_eq!(blanks_in, 3, "fixture sanity");
+        // And specifically the double newline the model would notice.
+        assert!(
+            out.contains("'use strict';\n\n// ---- constants ----"),
+            "blank line after the prologue was eaten"
+        );
+        assert!(
+            out.contains("const A = 1;\n\n\nfunction f()"),
+            "consecutive blank lines were collapsed"
+        );
+    }
+
     /// The second turn of an OpenAI-native agentic loop: the client
     /// replays its own assistant turn, which carries `tool_calls` and
     /// `"content": null`. That must render as the tool-call block —

crates/neuron/src/harness/mod.rs

@@ -13,6 +13,7 @@ pub mod prefix_cache;
 pub mod preflight;
 pub mod preprocess;
 pub mod reasoning_budget;
+pub mod reasoning_effort;
 pub mod sampling;
 pub mod speculative;
 pub mod tp;

crates/neuron/src/harness/preflight.rs

@@ -402,6 +402,7 @@ mod tests {
             tensor_parallel: tp,
             devices: None,
             sampling: None,
+            preserve_thinking: None,
         }
     }
 

crates/neuron/src/harness/reasoning_budget.rs

@@ -42,6 +42,31 @@ pub struct ReasoningBudgetLadder {
     pub xhigh: Option<usize>,
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub max: Option<usize>,
+    /// Tokens of the caller's **own** output budget held back for the
+    /// answer, when the caller names no explicit `reasoning.max_tokens`.
+    ///
+    /// This is what an OpenAI-shaped client gets instead of the rungs
+    /// above, and it is deliberately not derived from effort. Effort is
+    /// a prompt instruction the model is trained to follow (#290);
+    /// making it *also* set a guillotine is what told Qwen3.8-27B to
+    /// reason at `xhigh` and then cut it off at `medium`'s 12,288.
+    ///
+    /// A reserve has no such double meaning. The model thinks as long
+    /// as it likes within the budget the caller declared, minus enough
+    /// to answer — so the caller always gets an answer, and the size of
+    /// the think block stays the model's decision rather than ours.
+    ///
+    /// Set to `0` to disable, restoring fully unbounded reasoning.
+    #[serde(default = "default_answer_reserve")]
+    pub answer_reserve_tokens: usize,
+}
+
+/// Enough for a substantive answer with tool calls, not so much that it
+/// meaningfully shortens a think block: at the 32,768 budget pi sends,
+/// this leaves ~28.7 k for reasoning — above the ~27.2 k a full agentic
+/// design turn was measured to use.
+fn default_answer_reserve() -> usize {
+    4_096
 }
 
 impl Default for ReasoningBudgetLadder {
@@ -58,6 +83,7 @@ impl Default for ReasoningBudgetLadder {
             high: 32_768,
             xhigh: None,
             max: None,
+            answer_reserve_tokens: default_answer_reserve(),
         }
     }
 }

crates/neuron/src/harness/reasoning_effort.rs

@@ -0,0 +1,302 @@
+//! Which reasoning-effort rungs a model actually supports, discovered
+//! from the model itself (#290).
+//!
+//! ## Why this is discovered rather than configured
+//!
+//! `Qwen/Qwen3.8-27B` accepts exactly three efforts and hard-fails on
+//! anything else:
+//!
+//! ```jinja
+//! {%- set resolved_reasoning_effort = reasoning_effort|default('xhigh') %}
+//! {%- if resolved_reasoning_effort not in ('xhigh', 'medium', 'low') %}
+//!     {{- raise_exception('Unexpected reasoning effort ...') }}
+//! ```
+//!
+//! We previously advertised an invented ladder — `minimal`, `low`,
+//! `medium`, `high` — of which two values the template rejects outright
+//! and which omits `xhigh`, the model's own default and strongest rung.
+//!
+//! A caution about *why* that mattered, because the obvious story is
+//! wrong. It is tempting to say clients read our advertisement and were
+//! misled by it. pi-ai does not: its `getSupportedThinkingLevels` reads
+//! `model.thinkingLevelMap`, a static field in the operator's
+//! `models.json`, and treats `xhigh`/`max` as unavailable unless that
+//! map names them explicitly. So a pi user saw exactly
+//! `minimal, low, medium, high` — the invented ladder, to the value —
+//! for a reason that had nothing to do with what we published.
+//!
+//! The coincidence is the trap: our wrong advertisement and pi's
+//! default produced the same symptom, which made the advertisement look
+//! causal. Publishing rungs the model actually accepts is still correct
+//! — it is the truth, other clients may read it, and it is what makes
+//! `apply_effort_kwarg` right — but do not assume a given client
+//! consumes it. For pi the operator must also set `thinkingLevelMap`,
+//! by hand, and nothing validates that they match.
+//!
+//! A hardcoded per-model table would drift the moment a model ships a
+//! new template, which is exactly the class of defect #280 fixed by
+//! fetching the template instead of assuming it. So the rungs come from
+//! the artifact.
+//!
+//! ## How the probe works
+//!
+//! `raise_exception` is already wired to a render error, so the template
+//! answers the question directly: render it once per candidate effort
+//! and see which ones survive.
+//!
+//! Two ambiguities the probe has to resolve, or it would report
+//! confident nonsense:
+//!
+//! 1. **A template that ignores `reasoning_effort` entirely** renders
+//!    successfully for every candidate. Supporting "all six" would be a
+//!    lie. So a candidate only counts when the rendered prompt actually
+//!    *differs* across efforts — if every render is identical the model
+//!    has no effort control and we advertise none.
+//! 2. **The default** is whichever supported candidate renders
+//!    identically to a render with no effort supplied. Qwen3.8 defaults
+//!    to `xhigh`; without detecting that, a caller who says nothing gets
+//!    the strongest rung while we believe it got nothing.
+
+use anyhow::Result;
+use cortex_core::openai::{ChatMessage, MessageContent};
+use serde_json::{Value, json};
+
+/// Effort names in ascending order of thinking, as clients spell them.
+///
+/// Deliberately a superset of what any one model supports: this is the
+/// *candidate* set the probe tries, and the conventional vocabulary
+/// OpenAI-compatible clients use (pi-ai's `EXTENDED_THINKING_LEVELS`
+/// minus `off`, which is not an effort but a separate switch handled by
+/// `enable_thinking`).
+pub const CANDIDATE_EFFORTS: &[&str] = &["minimal", "low", "medium", "high", "xhigh", "max"];
+
+/// What one model's template accepts.
+#[derive(Debug, Clone, Default, PartialEq, Eq)]
+pub struct SupportedEfforts {
+    /// Accepted efforts, ascending, as the template spells them.
+    pub levels: Vec<String>,
+    /// The effort applied when the caller names none — the template's
+    /// own `|default(...)`. `None` when it cannot be determined.
+    pub default: Option<String>,
+}
+
+impl SupportedEfforts {
+    pub fn is_empty(&self) -> bool {
+        self.levels.is_empty()
+    }
+
+    pub fn supports(&self, effort: &str) -> bool {
+        self.levels.iter().any(|l| l == effort)
+    }
+
+    /// Map a caller's effort onto the nearest rung this model has.
+    ///
+    /// Exact match wins. Otherwise walk **down** from the requested
+    /// level first, then up — a caller asking for more thinking than the
+    /// model offers should get the most it has, and a caller asking for
+    /// less should not be silently upgraded into a slower, costlier
+    /// response than they asked for.
+    ///
+    /// Descending-first is the opposite of pi-ai's `clampThinkingLevel`,
+    /// which searches upward first. That is deliberate: pi is choosing
+    /// among levels it believes exist, whereas we are the ones who know
+    /// what exists, and over-delivering thinking is the failure that
+    /// costs the caller money and latency.
+    ///
+    /// `None` when the model has no rungs at all.
+    pub fn nearest(&self, requested: &str) -> Option<&str> {
+        if self.levels.is_empty() {
+            return None;
+        }
+        if let Some(exact) = self.levels.iter().find(|l| *l == requested) {
+            return Some(exact.as_str());
+        }
+        let idx = CANDIDATE_EFFORTS.iter().position(|c| *c == requested)?;
+        for c in CANDIDATE_EFFORTS[..idx].iter().rev() {
+            if let Some(hit) = self.levels.iter().find(|l| l.as_str() == *c) {
+                return Some(hit.as_str());
+            }
+        }
+        for c in &CANDIDATE_EFFORTS[idx + 1..] {
+            if let Some(hit) = self.levels.iter().find(|l| l.as_str() == *c) {
+                return Some(hit.as_str());
+            }
+        }
+        None
+    }
+}
+
+/// Render a probe prompt at `effort`, or `None` if the template rejects
+/// it.
+fn probe_render(template: &str, effort: Option<&str>) -> Option<String> {
+    let messages = [ChatMessage {
+        role: "user".into(),
+        content: MessageContent::Text("probe".into()),
+        extra: Value::Null,
+    }];
+    let kwargs = match effort {
+        Some(e) => json!({ "reasoning_effort": e }),
+        None => Value::Null,
+    };
+    super::chat_template::render_chat_template(template, &messages, &Value::Null, &kwargs).ok()
+}
+
+/// Discover which efforts `template` accepts (#290).
+///
+/// Errors are not propagated: a template that cannot render even the
+/// baseline probe yields no rungs, and the caller behaves as it did
+/// before this existed.
+pub fn probe(template: &str, model_id: &str) -> Result<SupportedEfforts> {
+    let Some(baseline) = probe_render(template, None) else {
+        tracing::debug!(
+            model = %model_id,
+            "reasoning effort: template does not render a baseline probe; advertising no rungs"
+        );
+        return Ok(SupportedEfforts::default());
+    };
+
+    let mut rendered: Vec<(&str, String)> = Vec::new();
+    for candidate in CANDIDATE_EFFORTS {
+        if let Some(out) = probe_render(template, Some(candidate)) {
+            rendered.push((candidate, out));
+        }
+    }
+
+    // A template that never reads `reasoning_effort` renders the same
+    // bytes whatever we pass. Advertising every candidate in that case
+    // would be the invented-ladder mistake again, in a new place.
+    let effort_aware = rendered.windows(2).any(|w| w[0].1 != w[1].1)
+        || rendered.iter().any(|(_, out)| *out != baseline);
+    if !effort_aware {
+        tracing::debug!(
+            model = %model_id,
+            "reasoning effort: template ignores reasoning_effort; advertising no rungs"
+        );
+        return Ok(SupportedEfforts::default());
+    }
+
+    let default = rendered
+        .iter()
+        .find(|(_, out)| *out == baseline)
+        .map(|(name, _)| (*name).to_string());
+    let levels: Vec<String> = rendered
+        .iter()
+        .map(|(name, _)| (*name).to_string())
+        .collect();
+
+    tracing::info!(
+        model = %model_id,
+        levels = %levels.join(","),
+        default = ?default,
+        "reasoning effort: rungs discovered from the model's chat template (#290)"
+    );
+    Ok(SupportedEfforts { levels, default })
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    /// The probe against the **real** deployed Qwen3.8-27B template,
+    /// not a hand-written approximation of it. A synthetic template
+    /// proves the probe logic; only the shipped artifact proves the
+    /// answer, and the artifact is what #290 is about.
+    #[test]
+    fn the_real_qwen38_template_yields_low_medium_xhigh_defaulting_to_xhigh() {
+        let path = concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/data/qwen3.8-27b.chat_template.jinja"
+        );
+        let Ok(template) = std::fs::read_to_string(path) else {
+            // The fixture is checked in; if it is ever removed this test
+            // should fail loudly rather than silently pass.
+            panic!("missing fixture {path}");
+        };
+        let s = probe(&template, "Qwen/Qwen3.8-27B").expect("probe");
+        assert_eq!(
+            s.levels,
+            vec!["low", "medium", "xhigh"],
+            "the shipped template accepts exactly these"
+        );
+        assert_eq!(s.default.as_deref(), Some("xhigh"));
+        // The two rungs we used to advertise are rejected outright.
+        assert!(!s.supports("minimal"));
+        assert!(!s.supports("high"));
+    }
+
+    /// The shape Qwen3.8-27B actually ships: three rungs, `xhigh`
+    /// default, `raise_exception` on anything else.
+    const QWEN38: &str = r#"
+{%- set resolved = reasoning_effort|default('xhigh') %}
+{%- if resolved not in ('xhigh', 'medium', 'low') %}
+{{- raise_exception('Unexpected reasoning effort ' ~ resolved) }}
+{%- endif %}
+{%- if resolved == 'xhigh' %}EFFORT-XHIGH{% elif resolved == 'low' %}EFFORT-LOW{% else %}EFFORT-MEDIUM{% endif %}
+{%- for m in messages %}<|im_start|>{{ m.role }}
+{{ m.content }}<|im_end|>
+{%- endfor %}
+"#;
+
+    /// A template with no effort control at all.
+    const PLAIN: &str = r#"
+{%- for m in messages %}<|im_start|>{{ m.role }}
+{{ m.content }}<|im_end|>
+{%- endfor %}
+"#;
+
+    #[test]
+    fn discovers_exactly_the_rungs_the_template_accepts() {
+        let s = probe(QWEN38, "test").expect("probe");
+        assert_eq!(s.levels, vec!["low", "medium", "xhigh"]);
+        assert!(!s.supports("minimal"), "template raises on minimal");
+        assert!(!s.supports("high"), "template raises on high");
+    }
+
+    /// Without this, a caller who names no effort silently receives the
+    /// strongest rung while we believe it received nothing — which is
+    /// how the xhigh instruction ended up on every request in #290.
+    #[test]
+    fn detects_the_templates_own_default() {
+        let s = probe(QWEN38, "test").expect("probe");
+        assert_eq!(s.default.as_deref(), Some("xhigh"));
+    }
+
+    /// A template that ignores the kwarg must advertise nothing rather
+    /// than appearing to support all six.
+    #[test]
+    fn a_template_without_effort_control_advertises_no_rungs() {
+        let s = probe(PLAIN, "test").expect("probe");
+        assert!(s.is_empty(), "got {:?}", s.levels);
+        assert_eq!(s.nearest("medium"), None);
+    }
+
+    #[test]
+    fn an_exact_match_is_returned_unchanged() {
+        let s = probe(QWEN38, "test").expect("probe");
+        assert_eq!(s.nearest("medium"), Some("medium"));
+        assert_eq!(s.nearest("xhigh"), Some("xhigh"));
+    }
+
+    /// `high` does not exist on this model. Descending first means it
+    /// resolves to `medium`, not `xhigh`: a caller is never given more
+    /// thinking (slower, dearer) than it asked for.
+    #[test]
+    fn an_unsupported_level_resolves_downward_not_upward() {
+        let s = probe(QWEN38, "test").expect("probe");
+        assert_eq!(s.nearest("high"), Some("medium"));
+    }
+
+    /// `minimal` is below every rung the model has, so there is nothing
+    /// to descend to and the nearest is the weakest available.
+    #[test]
+    fn a_level_below_every_rung_resolves_up_to_the_weakest() {
+        let s = probe(QWEN38, "test").expect("probe");
+        assert_eq!(s.nearest("minimal"), Some("low"));
+    }
+
+    #[test]
+    fn an_unknown_level_name_resolves_to_nothing_rather_than_a_guess() {
+        let s = probe(QWEN38, "test").expect("probe");
+        assert_eq!(s.nearest("turbo"), None);
+    }
+}

crates/neuron/src/wire/openai_responses.rs

@@ -176,6 +176,29 @@ pub fn request_to_chat(req: ResponsesRequest) -> Result<ChatCompletionRequest, T
                     {
                         attach_reasoning(&mut msg, text);
                     }
+                    // A `function_call` item belongs to the assistant
+                    // turn that precedes it, not to a turn of its own.
+                    //
+                    // Responses splits one turn across several items —
+                    // `reasoning`, then `message`, then `function_call`
+                    // — while chat carries all three on a single
+                    // assistant message. Pushing each item separately
+                    // produced *consecutive assistant turns*: one that
+                    // deliberates and then ends without acting, and one
+                    // that acts with an empty `<think></think>`.
+                    //
+                    // Neither shape exists in training, and the model
+                    // imitates what it is shown: after four such
+                    // examples in its own replayed context it produced
+                    // a think block and a preamble, then ended the turn
+                    // without the tool call it had just announced
+                    // (#296). Observed on a real session; the model was
+                    // copying our transcript, not misbehaving.
+                    if let Some(prev) = messages.last_mut()
+                        && merge_tool_call_into(prev, &mut msg)
+                    {
+                        continue;
+                    }
                     messages.push(msg);
                 }
             }
@@ -376,6 +399,57 @@ fn apply_reasoning_effort(extra: &mut serde_json::Map<String, Value>) {
     }
 }
 
+/// Fold a `function_call`-derived assistant message into the assistant
+/// turn it belongs to.
+///
+/// Returns `true` when the call was merged and the caller should drop
+/// its message. Only ever merges a *pure* tool-call message (empty
+/// content, no reasoning of its own) into a preceding assistant turn,
+/// so a genuine two-turn sequence is never collapsed and a reasoning
+/// item that arrived with the call keeps its own turn.
+///
+/// Consecutive calls accumulate, which is what parallel tool calling
+/// looks like on the wire: one assistant turn, several `tool_calls`.
+fn merge_tool_call_into(prev: &mut ChatMessage, msg: &mut ChatMessage) -> bool {
+    if prev.role != "assistant" || msg.role != "assistant" {
+        return false;
+    }
+    // Only a bare tool-call carrier folds in.
+    if !matches!(&msg.content, MessageContent::Text(t) if t.is_empty()) {
+        return false;
+    }
+    if msg.extra.get("reasoning_content").is_some() {
+        return false;
+    }
+    let Some(incoming) = msg
+        .extra
+        .get_mut("tool_calls")
+        .and_then(|v| v.as_array_mut())
+        .map(std::mem::take)
+    else {
+        return false;
+    };
+    // Everything else the item carried must survive the fold, or this
+    // trades one silent drop for another.
+    if msg
+        .extra
+        .as_object()
+        .is_some_and(|o| o.keys().any(|k| k != "tool_calls"))
+    {
+        return false;
+    }
+    let Some(obj) = prev.extra.as_object_mut() else {
+        return false;
+    };
+    match obj.get_mut("tool_calls").and_then(|v| v.as_array_mut()) {
+        Some(existing) => existing.extend(incoming),
+        None => {
+            obj.insert("tool_calls".into(), Value::Array(incoming));
+        }
+    }
+    true
+}
+
 /// The text of a replayed reasoning item, under either spelling.
 ///
 /// `summary` is what neuron emits (`summary_text` parts) and `content`
@@ -747,6 +821,31 @@ async fn run_projection(
         return;
     }
 
+    // No `Finish` means the producer died rather than finished — the
+    // poisoned-model path named above, an OOM, a dropped worker. This
+    // used to default to `FinishReason::Stop`, which told the caller a
+    // crash was a complete answer: pi-ai maps `status: "completed"` to
+    // `stopReason: "stop"` and treats whatever partial text arrived as
+    // the model's considered reply.
+    //
+    // Decided here, before the message item is opened, so a failed
+    // stream does not leave an item open that nothing will ever close.
+    let Some(reason) = finish else {
+        let _ = tx
+            .send(ResponseStreamFrame {
+                event_name: events::FAILED,
+                data: json!({
+                    "response": failed_response_shell(
+                        &meta,
+                        "server_error",
+                        "inference ended before the model finished",
+                    )
+                }),
+            })
+            .await;
+        return;
+    };
+
     // A stream can end while still thinking — a reasoning model that
     // exhausts max_output_tokens mid-block never emits visible text.
     // Close the item so the client sees a finished response rather than
@@ -761,7 +860,6 @@ async fn run_projection(
         return;
     }
 
-    let reason = finish.unwrap_or(FinishReason::Stop);
     let _ = emit_finish_frames(
         &tx,
         &meta,
@@ -777,6 +875,23 @@ async fn run_projection(
     .await;
 }
 
+/// The `response` payload for a [`events::FAILED`] frame.
+///
+/// pi-ai renders this as `` `${error.code}: ${error.message}` `` and
+/// throws, which is the correct outcome — the request genuinely failed
+/// — and is strictly better than a silent success.
+fn failed_response_shell(meta: &ResponseMeta, code: &str, message: &str) -> Value {
+    json!({
+        "id": meta.response_id,
+        "object": "response",
+        "created_at": meta.created_at,
+        "status": "failed",
+        "model": meta.model_id,
+        "output": [],
+        "error": { "code": code, "message": message },
+    })
+}
+
 /// Streaming state for a reasoning model's think block (#267).
 ///
 /// The block is its own output item, ahead of the message, so the
@@ -1043,7 +1158,14 @@ async fn emit_finish_frames(
             }),
         },
         ResponseStreamFrame {
-            event_name: events::COMPLETED,
+            // `response.incomplete` when we stopped short, so a client
+            // keyed on the event name rather than on `response.status`
+            // still sees a terminal frame instead of timing out.
+            event_name: if status == "incomplete" {
+                events::INCOMPLETE
+            } else {
+                events::COMPLETED
+            },
             data: json!({
                 "response": response_shell(meta, status, &output_items, usage)
             }),
@@ -1068,6 +1190,15 @@ fn response_shell(
     obj.insert("object".into(), Value::String("response".into()));
     obj.insert("created_at".into(), json!(meta.created_at));
     obj.insert("status".into(), Value::String(status.into()));
+    // Why we stopped short. A client cannot distinguish an honest
+    // truncation from a protocol fault without this, and pi-ai treats
+    // the difference as continue-vs-halt.
+    if status == "incomplete" {
+        obj.insert(
+            "incomplete_details".into(),
+            json!({ "reason": "max_output_tokens" }),
+        );
+    }
     obj.insert("model".into(), Value::String(meta.model_id.clone()));
     obj.insert("output".into(), Value::Array(output.to_vec()));
     if let Some(u) = usage {
@@ -1138,6 +1269,10 @@ pub fn build_response(
             }],
             status,
         }],
+        // Same contract as the streaming shell: a truncated response
+        // says so, or the caller cannot tell it from a fault.
+        incomplete_details: matches!(reason, FinishReason::Length)
+            .then(cortex_core::responses::IncompleteDetails::max_output_tokens),
         usage,
     }
 }
@@ -1147,6 +1282,80 @@ mod tests {
     use super::*;
     use cortex_core::openai::MessageContent;
 
+    /// One agentic step must render as ONE assistant turn.
+    ///
+    /// Responses splits a turn into `reasoning` + `message` +
+    /// `function_call` items. Emitting a message per item produced
+    /// *consecutive assistant turns* — one that deliberates and then
+    /// ends without acting, one that acts with an empty think block.
+    /// Shown four of those in its own replayed context, the model
+    /// copied the pattern: it produced a think block and a preamble,
+    /// then ended the turn without the tool call it had just announced
+    /// (#296).
+    #[test]
+    fn one_agentic_step_becomes_one_assistant_turn() {
+        let req: ResponsesRequest = serde_json::from_value(serde_json::json!({
+            "model": "m",
+            "input": [
+                {"type": "message", "role": "user",
+                 "content": [{"type": "input_text", "text": "build it"}]},
+                {"type": "reasoning",
+                 "content": [{"type": "reasoning_text", "text": "planning the file"}]},
+                {"type": "message", "role": "assistant",
+                 "content": [{"type": "output_text", "text": "Writing index.html:"}]},
+                {"type": "function_call", "call_id": "c1",
+                 "name": "write", "arguments": "{\"path\":\"a.html\"}"},
+                {"type": "function_call_output", "call_id": "c1", "output": "wrote 10 bytes"},
+                {"type": "message", "role": "user",
+                 "content": [{"type": "input_text", "text": "now check it"}]}
+            ]
+        }))
+        .unwrap();
+        let chat = request_to_chat(req).expect("translate");
+        let roles: Vec<&str> = chat.messages.iter().map(|m| m.role.as_str()).collect();
+        assert_eq!(
+            roles,
+            vec!["user", "assistant", "tool", "user"],
+            "no two assistant turns may be adjacent: got {roles:?}"
+        );
+
+        let a = &chat.messages[1];
+        assert!(
+            a.extra.get("reasoning_content").is_some(),
+            "reasoning must ride the turn it describes"
+        );
+        assert!(
+            matches!(&a.content, MessageContent::Text(t) if t.contains("Writing index.html")),
+            "the assistant's own text must survive the fold"
+        );
+        let calls = a.extra["tool_calls"].as_array().expect("tool_calls");
+        assert_eq!(calls.len(), 1);
+        assert_eq!(calls[0]["function"]["name"], "write");
+    }
+
+    /// Parallel tool calls are several calls on one turn, not several
+    /// turns.
+    #[test]
+    fn consecutive_function_calls_accumulate_on_one_turn() {
+        let req: ResponsesRequest = serde_json::from_value(serde_json::json!({
+            "model": "m",
+            "input": [
+                {"type": "message", "role": "user",
+                 "content": [{"type": "input_text", "text": "go"}]},
+                {"type": "message", "role": "assistant",
+                 "content": [{"type": "output_text", "text": "doing both:"}]},
+                {"type": "function_call", "call_id": "c1", "name": "a", "arguments": "{}"},
+                {"type": "function_call", "call_id": "c2", "name": "b", "arguments": "{}"}
+            ]
+        }))
+        .unwrap();
+        let chat = request_to_chat(req).expect("translate");
+        let roles: Vec<&str> = chat.messages.iter().map(|m| m.role.as_str()).collect();
+        assert_eq!(roles, vec!["user", "assistant"]);
+        let calls = chat.messages[1].extra["tool_calls"].as_array().unwrap();
+        assert_eq!(calls.len(), 2, "both calls belong to the same turn");
+    }
+
     /// Wrap typed items as `input` elements. Most translator tests
     /// exercise the typed path; the bare easy-message and unknown-item
     /// paths have dedicated tests below.
@@ -2003,9 +2212,19 @@ mod tests {
             names.contains(&events::CONTENT_PART_ADDED),
             "message item must still be opened so the finish frames refer to something"
         );
-        assert_eq!(names.last(), Some(&events::COMPLETED));
+        // This is the live failure in miniature: the model spent its
+        // whole budget thinking and produced almost no visible text.
+        // The stream must still close, and must say *why* it stopped —
+        // this frame used to be named `response.completed` while
+        // carrying `status: "incomplete"`, an inconsistency that cost a
+        // real agentic session its run.
+        assert_eq!(names.last(), Some(&events::INCOMPLETE));
         let last = frames.last().unwrap();
         assert_eq!(last.data["response"]["status"], "incomplete");
+        assert_eq!(
+            last.data["response"]["incomplete_details"]["reason"],
+            "max_output_tokens"
+        );
     }
 
     /// Tool calls sit after the message, which sits after any reasoning
@@ -2299,29 +2518,93 @@ mod tests {
         .unwrap();
         drop(tx);
         let frames = collect(out).await;
+        // The terminal frame is `response.incomplete`, not
+        // `response.completed` — a client keyed on the event name must
+        // still see an end to the stream.
+        assert!(
+            !frames.iter().any(|f| f.event_name == events::COMPLETED),
+            "a truncated response must not announce itself as completed"
+        );
+        let terminal = frames
+            .iter()
+            .find(|f| f.event_name == events::INCOMPLETE)
+            .expect("a terminal frame is required");
+        assert_eq!(terminal.data["response"]["status"], "incomplete");
+        // The field that decides whether a client recovers or halts.
+        assert_eq!(
+            terminal.data["response"]["incomplete_details"]["reason"], "max_output_tokens",
+            "pi-ai maps `incomplete` with no reason to stopReason:error \
+             and ends the agent loop; with max_output_tokens it maps to \
+             stopReason:length, fails truncated tool calls safely, and \
+             takes another turn"
+        );
+    }
+
+    /// The converse: a normal finish must stay `response.completed`
+    /// and carry no reason, or every client sees phantom truncation.
+    #[tokio::test]
+    async fn a_normal_finish_carries_no_incomplete_reason() {
+        let (tx, rx) = mpsc::channel::<InferenceEvent>(8);
+        let out = project_responses_stream(rx, meta());
+        tx.send(InferenceEvent::Start).await.unwrap();
+        tx.send(InferenceEvent::Finish {
+            reason: FinishReason::Stop,
+            prompt_tokens: 0,
+            completion_tokens: 0,
+            reasoning_tokens: 0,
+            cached_tokens: 0,
+            timing: None,
+        })
+        .await
+        .unwrap();
+        drop(tx);
+        let frames = collect(out).await;
         let completed = frames
             .iter()
             .find(|f| f.event_name == events::COMPLETED)
-            .unwrap();
-        assert_eq!(completed.data["response"]["status"], "incomplete");
+            .expect("a normal finish stays `response.completed`");
+        assert_eq!(completed.data["response"]["status"], "completed");
+        assert!(
+            completed.data["response"]
+                .get("incomplete_details")
+                .is_none(),
+            "a completed response must not carry an incomplete reason"
+        );
     }
 
+    /// A producer that drops without ever sending `Finish` did not
+    /// finish — it died (poisoned model, OOM, dropped worker). The
+    /// stream must still terminate, and must terminate as a *failure*.
+    ///
+    /// This previously asserted `COMPLETED`, encoding the defect: a
+    /// crashed inference was announced as a complete response, so a
+    /// client took whatever partial text had arrived to be the model's
+    /// considered answer instead of retrying.
     #[tokio::test]
-    async fn synthesises_start_frames_when_producer_skips_start() {
-        // A producer that drops without sending Start (poisoned
-        // model, immediate disconnect, …) should still produce a
-        // coherent stream — the projector synthesises the
-        // mandatory header frames before COMPLETED so the
-        // consumer never sees an output_text.done without a
-        // matching content_part.added.
+    async fn a_producer_that_dies_terminates_the_stream_as_failed() {
         let (tx, rx) = mpsc::channel::<InferenceEvent>(8);
         let out = project_responses_stream(rx, meta());
         drop(tx);
         let frames = collect(out).await;
         let names: Vec<&str> = frames.iter().map(|f| f.event_name).collect();
+        // The shell still opens, so the failure refers to a response.
         assert!(names.contains(&events::CREATED));
-        assert!(names.contains(&events::COMPLETED));
-        assert!(names.contains(&events::OUTPUT_TEXT_DONE));
+        assert!(
+            !names.contains(&events::COMPLETED),
+            "a crash must never be announced as a completed response"
+        );
+        let failed = frames
+            .iter()
+            .find(|f| f.event_name == events::FAILED)
+            .expect("the stream must terminate, or the client waits out its idle timeout");
+        assert_eq!(failed.data["response"]["status"], "failed");
+        assert_eq!(failed.data["response"]["error"]["code"], "server_error");
+        assert!(
+            failed.data["response"]["error"]["message"]
+                .as_str()
+                .is_some_and(|m| !m.is_empty()),
+            "pi-ai renders `code: message`, so an empty message helps nobody"
+        );
     }
 
     #[tokio::test]

crates/neuron/tests/activation.rs

@@ -30,6 +30,7 @@ async fn test_load_default_models_skips_unknown_harness() {
             tensor_parallel: None,
             devices: None,
             sampling: None,
+            preserve_thinking: None,
         },
         ModelSpec {
             model_id: "model-b".into(),
@@ -38,6 +39,7 @@ async fn test_load_default_models_skips_unknown_harness() {
             tensor_parallel: None,
             devices: None,
             sampling: None,
+            preserve_thinking: None,
         },
     ];
 
@@ -98,6 +100,7 @@ async fn test_load_default_models_skipped_on_driver_mismatch() {
         tensor_parallel: Some(2),
         devices: None,
         sampling: None,
+        preserve_thinking: None,
     }];
     let activation = ActivationTracker::new(&specs);
     let reason = "host NVIDIA driver/library mismatch (userspace NVML 580.159 vs loaded \
@@ -200,6 +203,7 @@ fn qwen_spec() -> ModelSpec {
         tensor_parallel: None,
         devices: None,
         sampling: None,
+        preserve_thinking: None,
     }
 }
 

crates/neuron/tests/candle_lifecycle.rs

@@ -54,6 +54,7 @@ async fn test_candle_qwen3_load_unload_lifecycle() {
         tensor_parallel: None,
         devices: Some(vec![0]),
         sampling: None,
+        preserve_thinking: None,
     };
 
     registry

crates/neuron/tests/data/qwen3.8-27b.chat_template.jinja

@@ -0,0 +1,170 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- set reasoning_instructions = '' %}
+{%- if enable_thinking is undefined or enable_thinking is true %}
+    {%- set resolved_reasoning_effort = reasoning_effort|default('xhigh') %}
+    {%- if resolved_reasoning_effort not in ('xhigh', 'medium', 'low') %}
+        {{- raise_exception('Unexpected reasoning effort ' ~ reasoning_effort ~ '. Supported types are xhigh (default), medium, and low.') }}
+    {%- endif %}
+    {%- if resolved_reasoning_effort == 'xhigh' %}
+        {%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}
+    {%- elif resolved_reasoning_effort == 'low' %}
+        {%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}
+    {%- endif %}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {%- if reasoning_instructions %}
+        {{- reasoning_instructions + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '')  + content + '<|im_end|>\n' }}
+        {%- elif reasoning_instructions %}
+            {{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
+        {%- endif %}
+    {%- elif reasoning_instructions %}
+        {{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if preserve_thinking is undefined or preserve_thinking is true or loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined and tool_call.arguments != '' %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | string if args_value is string else args_value | tojson | safe %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

crates/neuron/tests/preflight.rs

@@ -107,6 +107,7 @@ fn spec(model_id: &str, tp: Option<u32>, quant: Option<&str>) -> ModelSpec {
         tensor_parallel: tp,
         devices: None,
         sampling: None,
+        preserve_thinking: None,
     }
 }
 

script/check-config-consistency.py

@@ -15,6 +15,9 @@ Checked, per model id present in both files:
 
   * ``quant``      — the #252 field
   * ``sampling``   — the #283 field, compared field-by-field
+  * ``preserve_thinking`` — the H7 A/B gate: a model that replays
+    prior reasoning when resident and not when cold-loaded would
+    make the A/B unreadable, which is #252's shape again
 
 A model in only one file is not an error: plenty of catalogue entries are
 never resident anywhere, and a host may hold a model the catalogue does
@@ -46,7 +49,7 @@ REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 
 # Fields that must agree wherever both files mention the same model.
 # Extend this list whenever a serving-critical field lands in both.
-SCALAR_FIELDS = ["quant"]
+SCALAR_FIELDS = ["quant", "preserve_thinking"]
 TABLE_FIELDS = ["sampling"]
 
 

script/cuda-check.sh

@@ -0,0 +1,83 @@
+#!/usr/bin/env bash
+# Type-check neuron's `cuda` feature locally, in CI's own image.
+#
+# Why this exists: `#[cfg(feature = "cuda")]` code is invisible to a
+# default-feature build, so `cargo build`, `cargo clippy --all-targets`
+# and the whole test suite can pass on code that does not compile for
+# the target the fleet actually serves on. The only gate that catches it
+# is CI's `cuda-check` job — a ~20 minute round trip, and the failure
+# arrives after the push rather than before it.
+#
+# This runs the same command, with the same env, in the same image.
+#
+# Do NOT try to install the CUDA toolkit on a Fedora 44 host instead:
+# CUDA 13.0's nvcc rejects host compilers newer than gcc 15 and F44
+# ships gcc 16 with no compat package. The image is Fedora 43 for
+# exactly that reason, and additionally patches a glibc 2.41+ /
+# CUDA-header `noexcept` mismatch that nvcc hard-errors on. See
+# gongfoo's images/runner-cuda-13.0/Containerfile.
+#
+# No GPU is required — this is a borrow-check, not a run.
+#
+# First run pulls the image (~GBs) and compiles the candle/flash-attn
+# kernels, so it costs about what CI costs. Afterwards the cargo target
+# volume makes an incremental check a fraction of that, which is the
+# whole point.
+set -euo pipefail
+
+IMAGE="${CUDA_CHECK_IMAGE:-git.lair.cafe/gongfoo/runner-cuda-13.0:latest}"
+ROOT="$(git rev-parse --show-toplevel)"
+
+# A dedicated target volume. Sharing the host's `target/` would make the
+# default-feature and cuda-feature builds evict each other's artifacts on
+# every alternation, which would cost more than it saves.
+TARGET_VOL="${CUDA_CHECK_TARGET_VOL:-helexa-cuda-check-target}"
+# Mounted at the registry rather than over CARGO_HOME: the toolchain
+# binaries live in /usr/local/cargo/bin and must stay visible.
+REGISTRY_VOL="${CUDA_CHECK_REGISTRY_VOL:-helexa-cuda-check-registry}"
+
+if [ "${1:-}" = "--pull" ]; then
+    shift
+    podman pull "${IMAGE}"
+fi
+
+# Default to the exact command ci.yml's cuda-check job runs. Extra
+# arguments replace it, for narrowing to one crate while iterating.
+if [ "$#" -gt 0 ]; then
+    CARGO_CMD=("$@")
+else
+    CARGO_CMD=(cargo check -p neuron --features cuda,flash-attn --all-targets)
+fi
+
+# Allocate a TTY only when there is one to allocate. `-it` against a
+# pipe or a CI log wedges the container before cargo ever starts, which
+# looks exactly like a very slow compile.
+TTY_ARGS=()
+if [ -t 0 ] && [ -t 1 ]; then
+    TTY_ARGS=(-it)
+fi
+
+exec podman run --rm "${TTY_ARGS[@]}" \
+    -v "${ROOT}:/work:Z" \
+    -v "${TARGET_VOL}:/target" \
+    -v "${REGISTRY_VOL}:/usr/local/cargo/registry" \
+    -w /work \
+    -e CARGO_TARGET_DIR=/target \
+    -e CARGO_TERM_COLOR=always \
+    `# candle-kernels' build script falls back to nvidia-smi for` \
+    `# compute-cap detection when this is unset, and there is no GPU` \
+    `# here. Any valid cap borrow-checks; the real per-flavour caps` \
+    `# live in build-prerelease.yml's matrix.` \
+    -e CUDA_COMPUTE_CAP=86 \
+    `# The image ships sccache, but the local run has no shared cache` \
+    `# to talk to; an unset wrapper avoids a hard cargo failure.` \
+    -e RUSTC_WRAPPER= \
+    "${IMAGE}" \
+    bash -lc '
+        set -euo pipefail
+        # CUDA_HOME/PATH/LD_LIBRARY_PATH come from the image. LIBRARY_PATH
+        # does not, and cudarc needs it at link time — ci.yml exports all
+        # three explicitly, so keep this in step with that job.
+        export LIBRARY_PATH="${CUDA_HOME}/targets/x86_64-linux/lib:${CUDA_HOME}/lib64:${LIBRARY_PATH:-}"
+        exec "$@"
+    ' _ "${CARGO_CMD[@]}"