fix(toolcall): recover a call that tagged its function as a parameter

Reported from helexa.ai on 2026-08-29. Asked "what's the best hardware to
run qwen 3.8 flash next on?", the user's answer was a screenful of raw
markup:

    <tool_call> <parameter=web_search> <parameter=query>
    "Qwen3.8 Flash Next" hardware GPU </parameter> </function> </tool_call>

The model tagged the function itself as a parameter — there is no
`<function=` anywhere in that block. `parse_qwen_xml_tool_call` opens
with `body.split("<function=").nth(1)?`, so it returned None, and the
caller did what it does with an unparseable block: re-emitted it as
text. That text was the whole reply.

The journal shows the rest of the shape. Rounds one to four each finished
`ToolCalls` and ran real searches — 17 sources were collected — and the
fifth, tool-less round produced 35 tokens which were this malformed call.
So the turn spent five model passes and never produced a sentence.

Now a leading `<parameter=X>` is promoted to the function name when X is
a tool the request actually offered, and that pseudo-parameter is
excluded from the arguments.

The known-tool guard is the whole safety of it: without it any
`<parameter=x>` mints a function call, and a model rambling in angle
brackets starts invoking tools. `does_not_invent_a_function_from_an_unknown_parameter`
pins it, and was checked by deleting the guard — it fails, as it should.

Tested against the verbatim string from the report, plus a well-formed
call to prove the recovery does not disturb the normal path.

This makes the call parse, which is what lets the UI render it as a
search block rather than markup. It does not fix the deeper problem that
an unparseable block still reaches the user as prose — filed separately.

.gitea/workflows/build-prerelease.yml

@@ -262,7 +262,7 @@ jobs:
       # fast instead of triggering a slower uncached rebuild; only a real
       # sccache fault drops the cache. See script/ci-cargo-escalate.sh.
       - name: Clippy (sccache escalation)
-        run: script/ci-cargo-escalate.sh cargo clippy --workspace -- -D warnings
+        run: script/ci-cargo-escalate.sh cargo clippy --workspace --all-targets -- -D warnings
 
   test:
     name: Test

.gitea/workflows/ci.yml

@@ -101,14 +101,34 @@ jobs:
           echo "changed files:"
           printf '%s\n' "${files}" | sed 's/^/  /'
 
-          # Inert for Rust: the SPA, and documentation. Markdown is
-          # matched only at the repo root and under doc/ — a .md inside
-          # crates/ is left to fall through, since `include_str!` could
-          # make one load-bearing without this file knowing.
+          # Inert for Rust: the SPA, documentation, and fleet config.
+          # Markdown is matched only at the repo root and under doc/ — a
+          # .md inside crates/ is left to fall through, since
+          # `include_str!` could make one load-bearing without this file
+          # knowing.
+          #
+          # Fleet config (models.toml, asset/neuron/<host>.toml,
+          # asset/helexa-bench/<host>.toml) is read by a *running*
+          # daemon, never by a build script or an `include_str!`, so it
+          # cannot change a Rust result. Deploying one is a config rsync
+          # plus a service restart — neuron re-reads it and reloads the
+          # model — so spending ~25 minutes on fmt, clippy, test and a
+          # CUDA type-check to prove a one-line TOML edit compiles is
+          # pure latency (#291).
+          #
+          # The `config` job below stays ungated and still validates
+          # these files, so narrowing here removes the meritless work
+          # without removing the check that matters. build-prerelease
+          # already reaches the same conclusion on main: none of its
+          # per-package path regexes match these files, so a config-only
+          # push builds no binaries.
           rust_relevant="$(printf '%s\n' "${files}" \
             | grep -v '^helexa\.ai/' \
             | grep -v '^doc/' \
             | grep -vE '^[^/]+\.md$' \
+            | grep -vE '^asset/neuron/[^/]+\.toml$' \
+            | grep -vE '^asset/helexa-bench/[^/]+\.toml$' \
+            | grep -vE '^models\.toml$' \
             || true)"
           if [ -n "${rust_relevant}" ]; then
             echo "rust=true" >> "$GITHUB_OUTPUT"
@@ -157,6 +177,12 @@ jobs:
     steps:
       - uses: actions/checkout@v4
       - run: python3 script/check-config-consistency.py
+      # Two files describe one routing decision and nothing forced them
+      # to agree. On 2026-08-27 the router still aliased
+      # `helexa/balanced` to a model retired from the catalogue, so
+      # every web-chat turn evicted beast's resident flagship to
+      # cold-load it. Same shape as the check above (#252/#287).
+      - run: python3 script/check-alias-targets.py
 
   web:
     name: Web (lint + typecheck + i18n + build)
@@ -179,6 +205,12 @@ jobs:
       # reached main exactly that way.
       - run: npm run lint
       - run: npm run typecheck
+      # Behaviour, not just types. Gating from the first commit was the
+      # deliberate call (#304): with no existing suite there is nothing to
+      # stabilise, so a warn-first period would only create a window in
+      # which a vacuous test lands and gets normalised — and a suite
+      # nobody trusts is worse than none, because it gets cited.
+      - run: npm test
       # 42 locales with strict key parity is a promise that breaks
       # quietly. Adding an English string without translating it must fail
       # here, not in review, and certainly not in front of somebody
@@ -200,8 +232,13 @@ jobs:
       # in sync with build-prerelease.yml): a signal death (rustc SIGSEGV
       # / OOM-kill) keeps the cache and fails fast instead of an uncached
       # rebuild; only a real sccache fault drops the cache.
+      # --all-targets so `#[cfg(test)]` modules, tests/, benches/ and
+      # examples/ are linted too (#286). Without it clippy sees only lib
+      # and bin targets, so `-D warnings` never gated a single line of
+      # test code — which is how two tests lost their `#[test]` attribute
+      # and stopped running unnoticed.
       - name: Clippy (sccache escalation)
-        run: script/ci-cargo-escalate.sh cargo clippy --workspace -- -D warnings
+        run: script/ci-cargo-escalate.sh cargo clippy --workspace --all-targets -- -D warnings
 
   test:
     name: Test

.gitea/workflows/deploy.yml

@@ -59,6 +59,12 @@ jobs:
     steps:
       - uses: actions/checkout@v4
       - run: python3 script/check-config-consistency.py
+      # Two files describe one routing decision and nothing forced them
+      # to agree. On 2026-08-27 the router still aliased
+      # `helexa/balanced` to a model retired from the catalogue, so
+      # every web-chat turn evicted beast's resident flagship to
+      # cold-load it. Same shape as the check above (#252/#287).
+      - run: python3 script/check-alias-targets.py
 
   deploy-cortex:
     needs: [config-check]
@@ -526,6 +532,161 @@ jobs:
   # dependency on cortex or the neurons (the sweep loop is version-aware
   # and picks up whatever each neuron reports whenever). So it runs
   # alongside the cortex→neurons chain rather than after it.
+  # The vhost sync gets its own job, with NO `needs`, because it is not
+  # part of deploying the gateway binary and must not be able to stop
+  # one.
+  #
+  # It was briefly a step inside deploy-cortex, and that broke the fleet
+  # on its first run (2026-08-27, run 1284): the step hard-fails until an
+  # operator has run script/infra-setup.sh to install its sudoers grant,
+  # a failed step fails the job, and `deploy-neurons` has
+  # `needs: [deploy-cortex]` — so a missing nginx permission silently
+  # skipped the rollout of all three neuron flavours. The fleet stayed
+  # two commits behind while the run reported only "deploy-cortex:
+  # failure".
+  #
+  # #287 names this ordering trap. The lesson is narrower than "order the
+  # steps": a step whose prerequisite is installed by hand does not
+  # belong in a job that something else depends on, because its failure
+  # is not a statement about that job. Moving it later within
+  # deploy-cortex would not have helped — the job still ends red and the
+  # dependents still skip.
+  #
+  # Failing hard here is still right: drift between the repo and the live
+  # file is a real defect, and this job going red says that and nothing
+  # else.
+  deploy-nginx:
+    runs-on: fedora-43
+    if: >-
+      ${{
+        github.event_name == 'workflow_dispatch'
+        || github.event.workflow_run.conclusion == 'success'
+      }}
+    steps:
+      - uses: actions/checkout@v4
+      - name: SSH init
+        run: |
+          mkdir -p ~/.ssh
+          echo "${DEPLOY_KEY}" > ~/.ssh/id_ed25519
+          chmod 600 ~/.ssh/id_ed25519
+          ssh -o ConnectTimeout=5 -o StrictHostKeyChecking=accept-new \
+              gitea_ci@hanzalova.internal 'hostname -f'
+
+      # nginx vhost config for the internal site. Version-controlled in
+      # `asset/nginx/` since the beginning, but until now only ever
+      # installed by `script/infra-setup.sh` — a one-time, run-by-hand
+      # script. So the repo copy and the live file could diverge with
+      # nothing to detect it, which is the #252 shape: two sources of
+      # truth for one thing and nothing forcing agreement.
+      #
+      # Writes to sites-available and (re)points sites-enabled at it as
+      # a symlink. On this host sites-enabled was a *regular file*, not a
+      # symlink — two independent copies, of which nginx reads one. An
+      # operator editing "the config" would reasonably edit
+      # sites-available, watch `nginx -t` pass, reload, and see nothing
+      # change. Making the link canonical is part of the sync rather
+      # than a separate chore, so the layout cannot drift back.
+      #
+      # `nginx -t` gates the reload: a config that fails validation is
+      # left un-reloaded with the previous one still serving.
+      - name: Sync internal nginx vhost
+        run: |
+          src="asset/nginx/helexa.internal.conf"
+
+          # The sudoers drop-in that permits this is installed by
+          # script/infra-setup.sh, which is run by hand, so the live copy
+          # can lag the repo by however long it has been since anyone ran
+          # it. Check EVERY grant the step needs, not a representative
+          # one: the first version of this preflight tested `nginx -t`
+          # alone, which had existed for ages, so it passed and the job
+          # then died on the two helexa-ratelimit.conf grants added in the
+          # same commit as the check. A preflight that names a remedy
+          # while probing the wrong permission is worse than none — it
+          # tells you the sudoers are fine seconds before they are not.
+          #
+          # Each probe runs the command in a harmless form that still
+          # matches its sudoers pattern (sha256sum reads; nginx -t and
+          # `systemctl show` do not mutate). rsync and ln have no such
+          # form, so they are checked with `sudo -nl`, which reports
+          # whether a command is permitted without running it.
+          missing=""
+          probe() {  # probe <description> <command...>
+            local what="$1"; shift
+            ssh gitea_ci@hanzalova.internal "$@" >/dev/null 2>&1 \
+              || missing="${missing}
+                - ${what}"
+          }
+          probe "nginx -t" \
+            'sudo -n /usr/sbin/nginx -t'
+          probe "sha256sum /etc/nginx/sites-available/helexa.internal.conf" \
+            'sudo -n /usr/bin/sha256sum /etc/nginx/sites-available/helexa.internal.conf'
+          probe "sha256sum /etc/nginx/conf.d/helexa-ratelimit.conf" \
+            'sudo -n /usr/bin/sha256sum /etc/nginx/conf.d/helexa-ratelimit.conf'
+          probe "rsync -> /etc/nginx/sites-available/helexa.internal.conf" \
+            'sudo -nl /usr/bin/rsync -a x /etc/nginx/sites-available/helexa.internal.conf'
+          probe "rsync -> /etc/nginx/conf.d/helexa-ratelimit.conf" \
+            'sudo -nl /usr/bin/rsync -a x /etc/nginx/conf.d/helexa-ratelimit.conf'
+          probe "ln -sfn sites-available -> sites-enabled" \
+            'sudo -nl /usr/bin/ln -sfn /etc/nginx/sites-available/helexa.internal.conf /etc/nginx/sites-enabled/helexa.internal.conf'
+          probe "systemctl reload nginx" \
+            'sudo -nl /usr/bin/systemctl reload nginx'
+
+          if [ -n "${missing}" ]; then
+            echo "FAIL: gitea_ci is missing nginx sudoers grants on hanzalova:${missing}" >&2
+            echo "" >&2
+            echo "      Re-run 'script/infra-setup.sh' once from an operator shell to" >&2
+            echo "      install the current asset/sudoers.d/cortex-host.conf. Having run" >&2
+            echo "      it before is not enough — grants are added to that file as deploy" >&2
+            echo "      steps grow, and the host keeps whatever revision it last saw." >&2
+            exit 1
+          fi
+
+          rl="asset/nginx/helexa-ratelimit.conf"
+          local_sum=$(sha256sum "${src}" | cut -d' ' -f1)
+          local_rl=$(sha256sum "${rl}" | cut -d' ' -f1)
+          remote_sum=$(ssh gitea_ci@hanzalova.internal \
+            'sudo sha256sum /etc/nginx/sites-available/helexa.internal.conf 2>/dev/null | cut -d" " -f1' || true)
+          remote_rl=$(ssh gitea_ci@hanzalova.internal \
+            'sudo sha256sum /etc/nginx/conf.d/helexa-ratelimit.conf 2>/dev/null | cut -d" " -f1' || true)
+          link_ok=$(ssh gitea_ci@hanzalova.internal \
+            'test -L /etc/nginx/sites-enabled/helexa.internal.conf && echo yes || echo no')
+
+          if [ "${local_sum}" = "${remote_sum}" ] \
+             && [ "${local_rl}" = "${remote_rl}" ] \
+             && [ "${link_ok}" = "yes" ]; then
+            echo "nginx config unchanged, sites-enabled is a symlink — nothing to do"
+            exit 0
+          fi
+
+          # Order matters: the vhost references names defined at http
+          # context by helexa-ratelimit.conf (`limit_req_zone`s and the
+          # `helexa_noquery` log format). Ship that first, or a vhost
+          # referencing a not-yet-installed name fails `nginx -t` and the
+          # deploy stops with the two files out of step.
+          echo "syncing nginx rate-limit/log-format: ${remote_rl:-<absent>} -> ${local_rl}"
+          rsync --archive --compress \
+            --chown root:root --chmod 0644 \
+            --rsync-path 'sudo rsync' \
+            "${rl}" gitea_ci@hanzalova.internal:/etc/nginx/conf.d/helexa-ratelimit.conf
+
+          echo "syncing nginx vhost: ${remote_sum:-<absent>} -> ${local_sum} (symlink=${link_ok})"
+          rsync --archive --compress \
+            --chown root:root --chmod 0644 \
+            --rsync-path 'sudo rsync' \
+            "${src}" gitea_ci@hanzalova.internal:/etc/nginx/sites-available/helexa.internal.conf
+
+          ssh gitea_ci@hanzalova.internal 'bash -s' <<'NGINX'
+          set -eu
+          sudo /usr/bin/ln -sfn /etc/nginx/sites-available/helexa.internal.conf \
+                                /etc/nginx/sites-enabled/helexa.internal.conf
+          if ! sudo /usr/sbin/nginx -t; then
+            echo "FAIL: nginx -t rejected the synced config; not reloading" >&2
+            exit 1
+          fi
+          sudo /usr/bin/systemctl reload nginx
+          echo "nginx reloaded"
+          NGINX
+
   # Roll helexa-router + helexa-upstream onto the federation service host
   # (gallumbits). Same manifest-gated dnf pattern as deploy-cortex; the
   # two services are independent daemons on one host, so one job handles
@@ -539,6 +700,10 @@ jobs:
         || github.event.workflow_run.conclusion == 'success'
       }}
     steps:
+      # Needed since "Sync router config" reads helexa-router.toml out of
+      # the tree. This job managed without a checkout while it only ran
+      # package upgrades over ssh.
+      - uses: actions/checkout@v4
       - name: SSH init
         run: |
           mkdir -p ~/.ssh
@@ -606,6 +771,96 @@ jobs:
           done
           DEPLOY
 
+      # Router config (#287). Tracked because it holds no secrets, and
+      # synced here because being untracked is what let its
+      # `helexa/balanced` alias keep pointing at a retired model until
+      # beast fell over on 2026-08-27.
+      #
+      # After the package step, deliberately: this needs a sudoers grant
+      # installed by the hand-run script/infra-setup.sh, so on a host that
+      # has not seen the current grants it fails — and a step that fails
+      # before the package work would hold the service binaries hostage to
+      # an unrelated permission. deploy-gallumbits has no dependent jobs,
+      # so the blast radius stops here either way.
+      #
+      # Restarts only on a digest change: the router re-reads its aliases
+      # at startup, and a needless restart drops in-flight federation
+      # requests.
+      - name: Sync router config
+        run: |
+          missing=""
+          probe() {
+            local what="$1"; shift
+            ssh gitea_ci@gallumbits.kosherinata.internal "$@" >/dev/null 2>&1 \
+              || missing="${missing}
+                - ${what}"
+          }
+          probe "sha256sum /etc/helexa-router/helexa-router.toml" \
+            'sudo -n /usr/bin/sha256sum /etc/helexa-router/helexa-router.toml'
+          probe "rsync -> /etc/helexa-router/helexa-router.toml" \
+            'sudo -nl /usr/bin/rsync -a x /etc/helexa-router/helexa-router.toml'
+          probe "systemctl restart helexa-router.service" \
+            'sudo -nl /usr/bin/systemctl restart helexa-router.service'
+          if [ -n "${missing}" ]; then
+            echo "FAIL: gitea_ci is missing sudoers grants on gallumbits:${missing}" >&2
+            echo "" >&2
+            echo "      Re-run 'script/infra-setup.sh' once from an operator shell to" >&2
+            echo "      install the current asset/sudoers.d/gallumbits-host.conf." >&2
+            exit 1
+          fi
+
+          local_sum=$(sha256sum helexa-router.toml | cut -d' ' -f1)
+          remote_sum=$(ssh gitea_ci@gallumbits.kosherinata.internal \
+            'sudo -n /usr/bin/sha256sum /etc/helexa-router/helexa-router.toml 2>/dev/null | cut -d" " -f1' || true)
+
+          if [ "${local_sum}" = "${remote_sum}" ]; then
+            echo "router config unchanged (${local_sum})"
+            exit 0
+          fi
+
+          # Two questions, not one. The bytes decide whether to SYNC; the
+          # parsed settings decide whether to RESTART. The first run of
+          # this step proved why they differ: the only change was a
+          # comment header, and it bounced the router anyway, dropping
+          # whatever federation requests were in flight. The router reads
+          # its aliases at startup, so a comment is not a reason to
+          # interrupt anyone.
+          #
+          # Reading the remote copy into CI is safe here precisely because
+          # this file holds no secrets — the same property that lets it be
+          # tracked at all.
+          #
+          # The `cat` grant is deliberately NOT in the preflight above: if
+          # it is missing the read yields nothing, the comparison reports
+          # "changed", and the step restarts on any byte difference. That
+          # is the old behaviour, not a broken one, so a host mid-way
+          # through a sudoers rollout degrades instead of failing.
+          remote_copy="$(mktemp)"
+          trap 'rm -f "${remote_copy}"' EXIT
+          ssh gitea_ci@gallumbits.kosherinata.internal \
+            'sudo -n /usr/bin/cat /etc/helexa-router/helexa-router.toml' \
+            > "${remote_copy}" 2>/dev/null || true
+
+          settings_changed=yes
+          if python3 script/toml-settings-equal.py "${remote_copy}" helexa-router.toml; then
+            settings_changed=no
+          fi
+
+          echo "syncing helexa-router.toml: ${remote_sum:-<absent>} -> ${local_sum}"
+          rsync --archive --compress \
+            --chown root:root --chmod 0644 \
+            --rsync-path 'sudo rsync' \
+            helexa-router.toml \
+            gitea_ci@gallumbits.kosherinata.internal:/etc/helexa-router/helexa-router.toml
+
+          if [ "${settings_changed}" = yes ]; then
+            ssh gitea_ci@gallumbits.kosherinata.internal \
+              'sudo -n /usr/bin/systemctl restart helexa-router.service'
+            echo "helexa-router restarted: settings changed"
+          else
+            echo "helexa-router left running: only comments or formatting changed"
+          fi
+
       - name: Ensure firewalld allows helexa-router + helexa-upstream + helexa-tools + helexa-angels
         run: |
           ssh gitea_ci@gallumbits.kosherinata.internal '
@@ -870,6 +1125,12 @@ jobs:
           cd helexa.ai
           npm ci
           npm run i18n:check
+          # Also run here, not just in ci.yml, for the same reason
+          # i18n:check is: pushes to main skip ci.yml, so this is the only
+          # gate a merge itself passes through. Cheap, and it is the exact
+          # case the web CI comment describes — a defect reaching main and
+          # surfacing as a broken deploy afterwards.
+          npm test
           npm run build
 
       - name: SSH init

.gitignore

@@ -8,10 +8,23 @@ helexa.ai/.env.local
 *.swo
 .idea/
 .vscode/
+# Secret-bearing service configs. These carry API keys, a JWT secret, DB
+# passwords and SMTP credentials, so they are hand-synced by
+# script/infra-setup.sh rather than shipped by CI (#287 Gap 1).
+#
+# helexa-router.toml is deliberately NOT here: it holds no secrets, only
+# a listen address, the product-tier aliases and a mesh endpoint. It was
+# ignored by association, and that is precisely how its `helexa/balanced`
+# alias came to point at a retired model for long enough to take beast
+# down (2026-08-27). A file nothing can diff is a file nothing can catch.
+# Operator-owned credential layer, merged after the CI-deployed config
+# by cortex / helexa-upstream / helexa-angels. Must never be committed.
+secrets.toml
+**/secrets.toml
+
 cortex.toml
 doc/plan/*
 /target-cuda/
 .claude/
-helexa-router.toml
 helexa-upstream.toml
 helexa-angels.toml

CLAUDE.md

@@ -167,7 +167,7 @@ doc-comment; touch points (the `Job` enum, the dispatch handlers, the
 cargo build --release           # build all crates
 cargo run -p cortex-cli -- serve    # run the gateway
 cargo test                      # run all tests
-cargo clippy --workspace        # lint
+cargo clippy --workspace --all-targets   # lint (test code included)
 ```
 
 ## CI
@@ -177,7 +177,7 @@ pass before merging:
 
 ```sh
 cargo fmt --check --all                    # formatting
-cargo clippy --workspace -- -D warnings   # lint (warnings are errors)
+cargo clippy --workspace --all-targets -- -D warnings   # lint (warnings are errors)
 cargo test --workspace                     # tests
 ```
 
@@ -191,7 +191,8 @@ Work each change on its own branch; `main` stays releasable.
 
 1. Implement on a feature branch (`fix/<issue>-…`, `feat/<issue>-…`).
 2. Run the CI triad locally (`cargo fmt --check --all`,
-   `cargo clippy --workspace -- -D warnings`, `cargo test --workspace`).
+   `cargo clippy --workspace --all-targets -- -D warnings`,
+   `cargo test --workspace`).
    Local builds are **CPU-only** — the `#[cfg(feature = "cuda")]` neuron/TP
    paths do NOT compile locally. The branch CI's **CUDA type-check** job is
    the only thing that validates them, so for any neuron change the push to

asset/helexa-bench/bob.toml

@@ -22,6 +22,20 @@ iteration_pause_secs = 2
 request_timeout_secs = 600
 db_path = "/var/lib/helexa-bench/bench.sqlite"
 
+# Identity bench presents to neurons (#288). Without it bench is
+# anonymous, and since #262 anonymous callers are served only from
+# leftover capacity — capped below max_in_flight and parked at the class
+# gate. That moved beast's concurrency:8 from 168.9 tok/s / 0.53s
+# ttft_p95 to 104.6 / 11.77s the day #262 landed and held there, so every
+# number since measured the yield policy, not serving capacity.
+#
+# A dedicated pair, not a borrowed one: bench then holds its own
+# fair-share allocation (#54) and cannot starve interactive traffic —
+# the failure #262 existed to fix.
+[bench.principal]
+account_id = "helexa-bench"
+key_id = "fleet-benchmark"
+
 [scenarios]
 prompt_sizes = [128, 4096]
 max_tokens = 256
@@ -29,6 +43,10 @@ max_tokens = 256
 # carries p95-under-concurrency data before the F3 A/B gate (#94) needs
 # it for comparison. Levels mirror the real a0/hermes/opencode fan-out.
 concurrency_levels = [2, 4, 8]
+# Non-streaming bursts (#285/#288). Kept to the top level: it is where
+# serialization shows most starkly and each level costs a full burst per
+# sample per build.
+concurrency_nonstreaming_levels = [8]
 concurrency_prompt_tokens = 512
 
 # Capability probes (#91) — the reasoning/planning axis the speed

asset/neuron/beast.toml

@@ -7,10 +7,10 @@
 # ─────────────────────────────────────────────────────────────────────
 # neuron.toml for beast.hanzalova.internal
 #
-# 2x RTX 5090 (32 GB each) — TP-2 capable. Pre-warms Qwen3.6-27B with
-# q5k ISQ across both GPUs at activation, matching the validate-neuron
+# 2x RTX 5090 (32 GB each) — TP-2 capable. Pre-warms Qwen3.8-27B with
+# q6k ISQ across both GPUs at activation, matching the validate-neuron
 # invocation: `validate-neuron.sh beast.hanzalova.internal
-# Qwen/Qwen3.6-27B q5k 2`.
+# Qwen/Qwen3.8-27B q6k 2`.
 #
 # Synced to /etc/neuron/neuron.toml by script/infra-setup.sh. Edits
 # take effect after the next deploy workflow run restarts the service
@@ -29,14 +29,78 @@ name = "candle"
 # count. NEURON_BATCHING=0 (systemd drop-in) is the kill switch;
 # removing this section reverts to batch-1 serialization.
 [harness.candle.admission]
+# 8. max_in_flight bounds how many requests run concurrently. It does
+# NOT partition the KV budget: since #257 (`eca42510`, 2026-08-15) KV is
+# a shared pool and each request reserves what its own prompt needs via
+# `enter_with_kv(principal, kv_reservation_mb(prompt_len, ...))`. A
+# request too big for what is free waits for bytes to come back rather
+# than being handed a fixed 1/N slice.
+#
+# This value was briefly set to 2 (#291) on the stated grounds that "the
+# KV budget is divided by it at load, so 8 gives each slot 18,852
+# tokens". That was true before #257 and false after it, and #291 was
+# filed and actioned after that landed. Nothing in the tree divides the
+# budget by the slot count — `kv_budget` is a semaphore of MiB permits
+# and the engine's `max_slots` is only the batch width.
+#
+# The concurrency measurement in #291 stands as data (89.6% of busy time
+# at one request in flight, 4.9% utilisation over seven days) but the
+# conclusion drawn from it does not, and the traffic it described has
+# changed: while it was taken, the router's `helexa/balanced` alias
+# pointed web chat at a second 27B, so per-model concurrency was split
+# across two models that were evicting each other. With both tiers now
+# resolving to this model, chat concurrency lands here.
+#
+# What actually bounds a busy moment is the pool: 3,689 MiB at
+# 32 KiB/token/card is ~118k tokens of KV to share. Eight ordinary chat
+# turns fit easily; a few very long agentic sessions will queue on bytes,
+# which is the correct behaviour and is what #257 built.
 max_in_flight = 8
 
+# Snapshot budget for the cross-request prefix cache (#292).
+#
+# 3 GiB, not the 1 GiB default. At the measured 32 KiB/token/card, a
+# 1 GiB budget cannot hold a snapshot beyond ~32k tokens of K/V alone —
+# less once the GatedDeltaNet conv/recurrent state is deep-copied on top,
+# which on this arch is 48 of 64 layers in f32. Agentic sessions here run
+# well past that: `cached_tokens` was observed pinned at 2,016 for ten
+# consecutive turns while the prompt grew to 52,869, so every turn
+# re-prefilled ~50k tokens — 37 s each, against ~20 s of actual decode.
+# The cache was not missing; the snapshot would not fit.
+#
+# The snapshot budget is reserved BEFORE the KV budget, not from what is
+# left over: measured on this host, kv_budget_mb = 5737 - budget_mb,
+# exactly 1:1. So this number is a direct trade against how long a
+# single session can get, and both sides have a floor:
+#
+#   budget_mb   kv_budget   tokens/slot (2 slots)   holds a 52.9k snapshot
+#   1024        4713        75,400                  no  ← the #292 symptom
+#   3072        2665        42,600                  yes, but sessions
+#                                                   cannot reach 52.9k
+#   2048        3689        59,000                  yes
+#
+# 2 GiB is the value that satisfies both: a 52,869-token snapshot is
+# ~1,652 MB of K/V plus the GatedDeltaNet state (~1.73 GB total), and
+# per-slot KV still covers sessions comfortably past the deepest
+# observed. 3 GiB was tried first and starved KV — a session could no
+# longer reach the size whose snapshot the cache was being enlarged to
+# hold, which is self-defeating.
+#
+# max_entries stays at the default 8: the byte budget binds first here,
+# so the count cap is not what is forcing snapshots to be small.
+[harness.candle.prefix_cache]
+budget_mb = 2048
+
 [[default_models]]
-# Qwen3.8-27B is the resident flagship (#251). Architecturally identical
-# to the 3.6-27B it replaced — same model_type, layer counts and vision
-# tower — so the load spec is unchanged. The 3.6-27B stays catalogued at
-# the same residency class, so it can still be cold-loaded on demand and
-# the two displace each other in either direction.
+# Qwen3.8-27B is the resident flagship (#251). It replaced the
+# 3.6-27B — architecturally identical, same model_type, layer counts and
+# vision tower — so the load spec is unchanged.
+#
+# The 3.6-27B is no longer catalogued (retired 2026-08-27). While both
+# were listed at the same residency class they could displace each other
+# in either direction, which is what let a stale `helexa/balanced` alias
+# evict this model on every authenticated web-chat turn. See models.toml
+# for the incident that ended the arrangement.
 model_id = "Qwen/Qwen3.8-27B"
 harness = "candle"
 quant = "q6k"

asset/nginx/helexa-ratelimit.conf

@@ -24,3 +24,24 @@ limit_req_zone $binary_remote_addr zone=helexa_tools:10m rate=30r/m;
 limit_req_zone $binary_remote_addr zone=angels_invite:1m rate=20r/m;
 limit_req_zone $binary_remote_addr zone=angels_auth:1m rate=12r/m;
 limit_req_status 429;
+
+# Access-log format for the search locations, which must not record what
+# people searched for.
+#
+# The default `main` format logs `$request` — method, full URI *and
+# query string*. For a search proxy that means every query a user types
+# is written to /var/log/nginx/access.log next to their IP, for
+# authenticated and anonymous visitors alike. SearXNG's own web UI
+# defaults to POST partly to avoid exactly this.
+#
+# `$uri` is the path with the query stripped, so the operational signal
+# survives — who called, when, which endpoint, what status, how big, how
+# long — and the query itself does not. `$request_method` is kept
+# explicit since `$uri` drops it.
+#
+# Rate-limit debugging and abuse detection both work from this: they
+# need counts, statuses and client addresses, never the search terms.
+log_format helexa_noquery '$remote_addr - $remote_user [$time_local] '
+                          '"$request_method $uri $server_protocol" '
+                          '$status $body_bytes_sent "$http_referer" '
+                          '"$http_user_agent" $request_time';

asset/nginx/helexa.internal.conf

@@ -73,6 +73,9 @@ server {
     location = /tools/fetch {
         limit_req zone=helexa_tools burst=10 nodelay;
         limit_except GET { deny all; }
+        # The `url=` parameter is as revealing as a search query: it is
+        # the page a user is having read to them. Same treatment.
+        access_log /var/log/nginx/access.log helexa_noquery;
         proxy_pass http://gallumbits.kosherinata.internal:8889/fetch;
         proxy_set_header Host $host;
         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
@@ -83,7 +86,40 @@ server {
     # federation service host. GET only; the SPA requests format=json.
     location = /tools/web_search {
         limit_req zone=helexa_tools burst=10 nodelay;
-        limit_except GET { deny all; }
+        # GET and POST: SearXNG accepts both, and a client that POSTs
+        # keeps its query out of any intermediary's request line. Other
+        # methods have no meaning here.
+        limit_except GET POST { deny all; }
+        # A search query is small. Bounding the body stops this becoming
+        # a way to push arbitrary bytes at the upstream.
+        client_max_body_size 8k;
+        # Never log the query — see helexa_noquery in
+        # conf.d/helexa-ratelimit.conf.
+        access_log /var/log/nginx/access.log helexa_noquery;
+        proxy_pass http://gallumbits.kosherinata.internal:8888/search;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_read_timeout 30s;
+    }
+
+    # The same upstream under SearXNG's own path, for clients that speak
+    # the SearXNG API rather than our app's.
+    #
+    # `/tools/web_search` is our name for it, and a generic SearXNG
+    # client cannot guess it: they take a base URL and append `/search`,
+    # because that is what every SearXNG instance serves. Without this
+    # block such a request falls through to `location /` and is answered
+    # by the SPA — HTTP 200, `text/html`, and a JSON parse error at the
+    # client. A 404 would at least be honest; 200-with-HTML sends people
+    # looking for a bug in their own parser.
+    #
+    # Same upstream, same rate-limit zone, same GET-only restriction:
+    # this is an alias, not a second, laxer door.
+    location = /search {
+        limit_req zone=helexa_tools burst=10 nodelay;
+        limit_except GET POST { deny all; }
+        client_max_body_size 8k;
+        access_log /var/log/nginx/access.log helexa_noquery;
         proxy_pass http://gallumbits.kosherinata.internal:8888/search;
         proxy_set_header Host $host;
         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

asset/sudoers.d/cortex-host.conf

@@ -13,6 +13,18 @@ gitea_ci ALL=(root) NOPASSWD: /usr/bin/rsync * /etc/cortex/models.toml
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/rsync * /var/www/bench.helexa.ai/
 # deploy-website rsyncs the built helexa.ai SPA into the nginx webroot.
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/rsync * /var/www/helexa.ai/
+# deploy.yml's "Sync internal nginx vhost" step: write the vhost, make
+# sites-enabled a symlink to it, validate, reload. Scoped to the one
+# file and the one service — `nginx -t` is included because a reload
+# must never happen on a config nginx has rejected.
+gitea_ci ALL=(root) NOPASSWD: /usr/bin/rsync * /etc/nginx/sites-available/helexa.internal.conf
+gitea_ci ALL=(root) NOPASSWD: /usr/bin/rsync * /etc/nginx/conf.d/helexa-ratelimit.conf
+gitea_ci ALL=(root) NOPASSWD: /usr/bin/sha256sum /etc/nginx/conf.d/helexa-ratelimit.conf
+gitea_ci ALL=(root) NOPASSWD: /usr/bin/sha256sum /etc/nginx/sites-available/helexa.internal.conf
+gitea_ci ALL=(root) NOPASSWD: /usr/bin/ln -sfn /etc/nginx/sites-available/helexa.internal.conf /etc/nginx/sites-enabled/helexa.internal.conf
+gitea_ci ALL=(root) NOPASSWD: /usr/sbin/nginx -t
+gitea_ci ALL=(root) NOPASSWD: /usr/bin/systemctl reload nginx
+
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/systemctl start cortex.service
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/systemctl stop cortex.service
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/systemctl enable --now cortex.service

asset/sudoers.d/gallumbits-host.conf

@@ -4,9 +4,17 @@
 # rolls out helexa-router (data plane), helexa-upstream (account/budget
 # authority), helexa-tools (grounding fetcher) and helexa-angels (the
 # confidential investor portal) package upgrades.
-# Configs are NOT synced by CI — they carry secrets (DB password, JWT
-# secret, operator tokens) and ride in script/infra-setup.sh under the
-# operator's own sudo instead.
+# The SECRET-BEARING configs are not synced by CI — helexa-upstream.toml
+# and helexa-angels.toml carry a DB password, a JWT secret and SMTP
+# credentials, and ride in script/infra-setup.sh under the operator's own
+# sudo instead (#287 Gap 1, still open).
+#
+# helexa-router.toml is different and IS synced here: it holds no secrets,
+# only a listen address, the product-tier aliases and a mesh endpoint. It
+# was excluded by association with its neighbours, which is how its
+# `helexa/balanced` alias came to point at a model retired from the
+# catalogue and took beast down on 2026-08-27. A config CI cannot write is
+# a config no check can defend.
 
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/systemctl start helexa-router.service
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/systemctl stop helexa-router.service
@@ -21,6 +29,20 @@ gitea_ci ALL=(root) NOPASSWD: /usr/bin/systemctl start helexa-angels.service
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/systemctl stop helexa-angels.service
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/systemctl enable --now helexa-angels.service
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/systemctl daemon-reload
+
+# deploy-gallumbits' "Sync router config" step. Scoped to the one file:
+# the neighbouring service configs stay operator-only until #287 Gap 1
+# gives them a render-from-secrets path.
+gitea_ci ALL=(root) NOPASSWD: /usr/bin/rsync * /etc/helexa-router/helexa-router.toml
+gitea_ci ALL=(root) NOPASSWD: /usr/bin/sha256sum /etc/helexa-router/helexa-router.toml
+# Read-back, so the sync can tell a settings change from a comment
+# change and restart only for the former. Safe to grant and safe to read
+# into CI because this file holds no secrets — the same property that
+# lets it be tracked. If this grant is absent the step degrades to
+# restarting on any byte change, which is why it is not in the step's
+# fail-hard preflight.
+gitea_ci ALL=(root) NOPASSWD: /usr/bin/cat /etc/helexa-router/helexa-router.toml
+gitea_ci ALL=(root) NOPASSWD: /usr/bin/systemctl restart helexa-router.service
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/dnf install --refresh --allowerasing -y helexa-router
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/dnf upgrade --refresh --allowerasing -y helexa-router
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/dnf install --refresh --allowerasing -y helexa-upstream

bench/src/api.ts

@@ -1,4 +1,10 @@
-import type { Dimensions, ReportRow, RunRow, SeriesPoint } from "./types";
+import type {
+  Dimensions,
+  MeasurementRegime,
+  ReportRow,
+  RunRow,
+  SeriesPoint,
+} from "./types";
 
 // Empty default → `fetch('/api/...')` hits the dev proxy (vite.config.ts)
 // or the same origin. For a separately-hosted build, set VITE_API_BASE to
@@ -15,6 +21,7 @@ async function getJson<T>(path: string): Promise<T> {
 
 export const getDimensions = () => getJson<Dimensions>("/api/dimensions");
 export const getSummary = () => getJson<ReportRow[]>("/api/summary");
+export const getRegimes = () => getJson<MeasurementRegime[]>("/api/regimes");
 
 // host is resolved server-side (each model maps to one host today), so the
 // public UI selects by model + scenario alone.

bench/src/pages/Trends.tsx

@@ -11,8 +11,8 @@ import {
   XAxis,
   YAxis,
 } from "recharts";
-import { getDimensions, getSeries } from "../api";
-import type { Dimensions, SeriesPoint } from "../types";
+import { getDimensions, getRegimes, getSeries } from "../api";
+import type { Dimensions, MeasurementRegime, SeriesPoint } from "../types";
 import { BASELINE_SOURCE, baselineFor } from "../baseline";
 
 function Picker({
@@ -40,11 +40,95 @@ function Picker({
   );
 }
 
+type SeriesDef = {
+  key: string;
+  name: string;
+  stroke: string;
+  dashed?: boolean;
+};
+
+/** One titled chart over the shared build timeline.
+ *
+ * Every panel draws the same x-axis and the same regime divider, so a
+ * reader can line a change up across metrics — which is the whole point
+ * of having more than two of them. */
+/** A vertical rule at a build where the metric changed meaning. */
+type Rule = { at: string; label: string; detail?: string };
+
+function MetricChart({
+  title,
+  hint,
+  data,
+  lines,
+  rules,
+  unit,
+}: {
+  title: string;
+  hint?: string;
+  data: Record<string, unknown>[];
+  lines: SeriesDef[];
+  rules?: Rule[];
+  unit?: string;
+}) {
+  const hasAny = data.some((d) => lines.some((l) => d[l.key] != null));
+  if (!hasAny) return null;
+  return (
+    <>
+      <h5 className="mt-4">
+        {title}
+        {unit ? <span className="text-muted fw-normal"> ({unit})</span> : null}
+      </h5>
+      {hint && <p className="text-muted small mb-2">{hint}</p>}
+      <ResponsiveContainer width="100%" height={260}>
+        <LineChart data={data} margin={{ top: 8, right: 24, bottom: 8, left: 0 }}>
+          <CartesianGrid strokeDasharray="3 3" />
+          <XAxis dataKey="label" />
+          <YAxis />
+          <Tooltip />
+          <Legend />
+          {(rules ?? [])
+            // Only draw a rule whose build is actually on this x-axis;
+            // recharts would otherwise place it at the origin and imply
+            // the boundary sits before all the data.
+            .filter((r) => data.some((d) => d.label === r.at))
+            .map((r) => (
+              <ReferenceLine
+                key={`${r.at}-${r.label}`}
+                x={r.at}
+                stroke="#bbb"
+                strokeDasharray="3 3"
+                label={{
+                  value: r.label,
+                  position: "top",
+                  fill: "#999",
+                  fontSize: 11,
+                }}
+              />
+            ))}
+          {lines.map((l) => (
+            <Line
+              key={l.key}
+              type="monotone"
+              dataKey={l.key}
+              name={l.name}
+              stroke={l.stroke}
+              strokeDasharray={l.dashed ? "5 5" : undefined}
+              connectNulls
+              dot={false}
+            />
+          ))}
+        </LineChart>
+      </ResponsiveContainer>
+    </>
+  );
+}
+
 export default function Trends() {
   const [dims, setDims] = useState<Dimensions | null>(null);
   const [model, setModel] = useState("");
   const [scenario, setScenario] = useState("");
   const [series, setSeries] = useState<SeriesPoint[]>([]);
+  const [regimes, setRegimes] = useState<MeasurementRegime[]>([]);
   const [err, setErr] = useState<string | null>(null);
 
   useEffect(() => {
@@ -55,6 +139,10 @@ export default function Trends() {
         if (d.scenarios[0]) setScenario(d.scenarios[0]);
       })
       .catch((e) => setErr(String(e)));
+    // A missing/older API just means no rules — not a page failure.
+    getRegimes()
+      .then(setRegimes)
+      .catch(() => setRegimes([]));
   }, []);
 
   useEffect(() => {
@@ -68,10 +156,7 @@ export default function Trends() {
   // Prepend the pre-helexa-bench baseline (dashed, separate keys) so it
   // anchors the timeline without being merged into the live line. Different
   // measurement regime — see baseline.ts / doc/benchmarks.md.
-  const base = useMemo(
-    () => baselineFor(model, scenario),
-    [model, scenario],
-  );
+  const base = useMemo(() => baselineFor(model, scenario), [model, scenario]);
   const data = useMemo(
     () => [
       ...base.map((p) => ({
@@ -85,6 +170,14 @@ export default function Trends() {
         ttft: p.ttft_s_median,
         decode: p.decode_tps_median,
         total: p.total_s_median,
+        ttftP95: p.ttft_p95_s_median,
+        queueWait: p.queue_wait_ms_median,
+        rejected: p.rejected_median,
+        prefillTps: p.prefill_tps_median,
+        reasoning: p.reasoning_tokens_median,
+        cached: p.cached_tokens_median,
+        completion: p.completion_tokens_median,
+        tpot: p.tpot_p95_ms_median,
       })),
     ],
     [series, base],
@@ -95,6 +188,55 @@ export default function Trends() {
   const firstLive = series[0]?.git_sha;
   const showDivider = base.length > 0 && series.length > 0;
 
+  // The build where the measuring identity changed, derived from the
+  // data rather than declared — a constant would go stale the moment
+  // the principal is reconfigured (#288).
+  const identityShift = useMemo(() => {
+    for (let i = 1; i < series.length; i++) {
+      const prev = series[i - 1].principal ?? "anonymous";
+      const cur = series[i].principal ?? "anonymous";
+      if (prev !== cur) {
+        return {
+          at: series[i].git_sha,
+          label: `measured as ${cur === "anonymous" ? "anonymous" : "identified"}`,
+          detail:
+            "The identity bench measured under changed here. Anonymous " +
+            "samples are subject to the #262 yield policy and understate " +
+            "capacity, so points either side are not comparable (#288).",
+        } as Rule;
+      }
+    }
+    return null;
+  }, [series]);
+
+  /** Declared boundaries touching `metric`, plus the regime-independent
+   *  ones that apply to every panel. */
+  const rulesFor = (metric: string): Rule[] => {
+    const out: Rule[] = [];
+    if (showDivider && firstLive) {
+      out.push({ at: firstLive, label: "bench.py → helexa-bench" });
+    }
+    for (const r of regimes) {
+      if (r.affects.includes(metric)) {
+        out.push({ at: r.first_sha, label: r.label, detail: r.detail });
+      }
+    }
+    if (identityShift) out.push(identityShift);
+    return out;
+  };
+
+  // Anonymous and identified samples are not comparable once #262 is in
+  // the build: an anonymous caller is capped below max_in_flight and
+  // parked at the class gate, so it characterises the yield policy
+  // rather than serving capacity. Say so rather than letting someone
+  // read a step change as an engine regression (#288).
+  const identities = useMemo(
+    () => new Set(series.map((p) => p.principal ?? "anonymous")),
+    [series],
+  );
+  const mixedIdentity = identities.size > 1;
+  const anyAnonymous = identities.has("anonymous");
+
   if (err) return <Alert variant="danger">{err}</Alert>;
   if (!dims) return <Spinner animation="border" />;
 
@@ -122,6 +264,25 @@ export default function Trends() {
         </p>
       )}
 
+      {mixedIdentity && (
+        <Alert variant="warning" className="py-2">
+          <strong>Mixed measurement identity.</strong> Some builds were
+          sampled anonymously and some under a principal. Since{" "}
+          <code>#262</code> an anonymous caller is capped below{" "}
+          <code>max_in_flight</code> and yields to identified traffic, so
+          those points measure the admission policy rather than serving
+          capacity. A step change across that boundary is the instrument,
+          not the engine — see <code>#288</code>.
+        </Alert>
+      )}
+      {!mixedIdentity && anyAnonymous && (
+        <Alert variant="secondary" className="py-2 small">
+          Sampled anonymously. Since <code>#262</code> anonymous callers are
+          served from leftover capacity, so these numbers understate what an
+          authenticated caller gets (<code>#288</code>).
+        </Alert>
+      )}
+
       {data.length === 0 ? (
         <Alert variant="info">No data for this selection yet.</Alert>
       ) : (
@@ -133,87 +294,128 @@ export default function Trends() {
               see <code>doc/benchmarks.md</code>.
             </p>
           )}
-          <h5 className="mt-3">decode tok/s (higher is better)</h5>
-          <ResponsiveContainer width="100%" height={280}>
-            <LineChart data={data} margin={{ top: 8, right: 24, bottom: 8, left: 0 }}>
-              <CartesianGrid strokeDasharray="3 3" />
-              <XAxis dataKey="label" />
-              <YAxis />
-              <Tooltip />
-              <Legend />
-              {showDivider && firstLive && (
-                <ReferenceLine
-                  x={firstLive}
-                  stroke="#bbb"
-                  strokeDasharray="3 3"
-                  label={{
-                    value: "bench.py → helexa-bench",
-                    position: "top",
-                    fill: "#999",
-                    fontSize: 11,
-                  }}
-                />
-              )}
-              <Line
-                type="monotone"
-                dataKey="decode"
-                name="decode tok/s"
-                stroke="#0d6efd"
-                connectNulls
-              />
-              {base.length > 0 && (
-                <Line
-                  type="monotone"
-                  dataKey="baseDecode"
-                  name="baseline (bench.py · gateway)"
-                  stroke="#888"
-                  strokeDasharray="5 5"
-                  connectNulls
-                />
-              )}
-            </LineChart>
-          </ResponsiveContainer>
 
-          <h5 className="mt-4">TTFT seconds (lower is better)</h5>
-          <ResponsiveContainer width="100%" height={280}>
-            <LineChart data={data} margin={{ top: 8, right: 24, bottom: 8, left: 0 }}>
-              <CartesianGrid strokeDasharray="3 3" />
-              <XAxis dataKey="label" />
-              <YAxis />
-              <Tooltip />
-              <Legend />
-              {showDivider && firstLive && (
-                <ReferenceLine
-                  x={firstLive}
-                  stroke="#bbb"
-                  strokeDasharray="3 3"
-                  label={{
-                    value: "bench.py → helexa-bench",
-                    position: "top",
-                    fill: "#999",
-                    fontSize: 11,
-                  }}
-                />
-              )}
-              <Line
-                type="monotone"
-                dataKey="ttft"
-                name="TTFT (s)"
-                stroke="#dc3545"
-                connectNulls
-              />
-              {base.length > 0 && (
-                <Line
-                  type="monotone"
-                  dataKey="baseTtft"
-                  name="baseline (bench.py · gateway)"
-                  stroke="#888"
-                  strokeDasharray="5 5"
-                  connectNulls
-                />
-              )}
-            </LineChart>
-          </ResponsiveContainer>
+          <MetricChart
+            title="decode tok/s"
+            unit="higher is better"
+            data={data}
+            rules={rulesFor("decode")}
+            lines={[
+              { key: "decode", name: "decode tok/s", stroke: "#0d6efd" },
+              ...(base.length > 0
+                ? [
+                    {
+                      key: "baseDecode",
+                      name: "baseline (bench.py · gateway)",
+                      stroke: "#888",
+                      dashed: true,
+                    },
+                  ]
+                : []),
+            ]}
+          />
+
+          <MetricChart
+            title="prefill tok/s"
+            unit="higher is better"
+            hint="The other half of serving speed, derived from prefill_tokens / prefill_ms. A prefix-cache hit shortens prefill_ms while the token count stays whole, so a high rate here is itself the cache-hit signal."
+            data={data}
+            rules={rulesFor("prefillTps")}
+            lines={[
+              { key: "prefillTps", name: "prefill tok/s", stroke: "#20c997" },
+            ]}
+          />
+
+          <MetricChart
+            title="TTFT"
+            unit="seconds, lower is better"
+            hint="Median and p95 together on purpose. Under concurrency the median is dominated by whichever streams were admitted immediately; the p95 is the one that moves when a caller is made to wait. Charting only the median is how a 0.53 s → 11.77 s tail went unnoticed for a week."
+            data={data}
+            rules={rulesFor("ttft")}
+            lines={[
+              { key: "ttft", name: "TTFT median (s)", stroke: "#dc3545" },
+              { key: "ttftP95", name: "TTFT p95 (s)", stroke: "#fd7e14" },
+              ...(base.length > 0
+                ? [
+                    {
+                      key: "baseTtft",
+                      name: "baseline (bench.py · gateway)",
+                      stroke: "#888",
+                      dashed: true,
+                    },
+                  ]
+                : []),
+            ]}
+          />
+
+          <MetricChart
+            title="inter-token gap p95"
+            unit="ms, lower is better"
+            hint="Stream smoothness. decode tok/s is a mean over the whole window, so a stream that stalls and then catches up is indistinguishable from one that never stalled — this is the number a user feels."
+            data={data}
+            rules={rulesFor("tpot")}
+            lines={[
+              { key: "tpot", name: "inter-token p95 (ms)", stroke: "#6f42c1" },
+            ]}
+          />
+
+          <MetricChart
+            title="admission"
+            unit="queue wait ms · requests shed"
+            hint="Separates “the server is slow” from “you were queued behind someone”. Queue wait is TTFT minus server-measured prefill; rejected counts honest backpressure rather than silent failures."
+            data={data}
+            rules={rulesFor("queueWait")}
+            lines={[
+              { key: "queueWait", name: "queue wait (ms)", stroke: "#d63384" },
+              { key: "rejected", name: "rejected (count)", stroke: "#adb5bd" },
+            ]}
+          />
+
+          <MetricChart
+            title="tokens per sample"
+            unit="counts"
+            hint="Cost, not speed. Reasoning tokens are the dominant driver on a reasoning model and move independently of every rate above — a template or sampling change can double what the model thinks before answering while the speed charts stay flat. Cached tokens are why prefill timing varies between otherwise identical samples."
+            data={data}
+            rules={rulesFor("completion")}
+            lines={[
+              { key: "completion", name: "completion tokens", stroke: "#0dcaf0" },
+              { key: "reasoning", name: "reasoning tokens", stroke: "#ffc107" },
+              { key: "cached", name: "cached prompt tokens", stroke: "#198754" },
+            ]}
+          />
+
+          {(() => {
+            // Explain every rule actually drawn for this selection. The
+            // chart label is a name; without the reason, a reader still
+            // has to go and rediscover what it meant — which is the cost
+            // this whole feature exists to remove.
+            const shown = [
+              ...regimes
+                .filter((r) => series.some((p) => p.git_sha === r.first_sha))
+                .map((r) => ({ at: r.first_sha, label: r.label, detail: r.detail })),
+              ...(identityShift ? [identityShift] : []),
+            ];
+            if (shown.length === 0) return null;
+            return (
+              <div className="mt-4 pt-3 border-top">
+                <p className="text-muted small mb-2">
+                  <strong>Dashed rules mark measurement-regime changes</strong> —
+                  builds where a number changed meaning rather than value. A step
+                  across one is the instrument, not the engine.
+                </p>
+                <dl className="row small text-muted mb-0">
+                  {shown.map((r) => (
+                    <div key={`${r.at}-${r.label}`} className="mb-2">
+                      <dt className="fw-semibold">
+                        <code>{r.at}</code> — {r.label}
+                      </dt>
+                      <dd className="mb-0">{r.detail}</dd>
+                    </div>
+                  ))}
+                </dl>
+              </div>
+            );
+          })()}
         </>
       )}
     </>

bench/src/types.ts

@@ -41,6 +41,17 @@ export interface SeriesPoint {
   ttft_s_median: number | null;
   decode_tps_median: number | null;
   total_s_median: number | null;
+  ttft_p95_s_median: number | null;
+  queue_wait_ms_median: number | null;
+  rejected_median: number | null;
+  prefill_tps_median: number | null;
+  reasoning_tokens_median: number | null;
+  cached_tokens_median: number | null;
+  completion_tokens_median: number | null;
+  tpot_p95_ms_median: number | null;
+  /** Identity the samples were taken under, or null if anonymous.
+   *  Anonymous and identified points are not comparable (#288). */
+  principal: string | null;
   samples: number;
 }
 
@@ -67,3 +78,13 @@ export interface RunRow {
   ok: boolean;
   error: string | null;
 }
+
+/** A build at which a metric changed meaning rather than value (#288).
+ *  Served from the API so the UI and `report` cite one list. */
+export interface MeasurementRegime {
+  first_sha: string;
+  label: string;
+  detail: string;
+  /** Series keys this affects, as Trends.tsx names them. */
+  affects: string[];
+}

crates/cortex-core/src/catalogue.rs

@@ -67,6 +67,13 @@ pub struct ModelProfile {
     /// model, with nothing forcing them to agree.
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub sampling: Option<SamplingOverride>,
+    /// The model's own `preserve_thinking` template control, forwarded
+    /// on cold load so a model this gateway loads behaves the same as
+    /// one the host was configured to hold resident. Two config files
+    /// describing one model is how #252 happened; this rides along for
+    /// the same reason `sampling` does.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub preserve_thinking: Option<bool>,
 
     // ── Enrichment (issue #62) ────────────────────────────────
     /// Per-model token budget. When present, advertised in `/v1/models`
@@ -251,6 +258,7 @@ mod tests {
             residency_priority: None,
             source: None,
             sampling: None,
+            preserve_thinking: None,
             limit: None,
             cost: None,
             capabilities: vec![],

crates/cortex-core/src/config.rs

@@ -143,15 +143,48 @@ pub struct NeuronEndpoint {
 impl GatewayConfig {
     /// Load configuration from a TOML file, with environment variable overrides.
     /// Env vars are prefixed with `CORTEX_` and use `__` as a separator.
+    ///
+    /// A sibling `secrets.toml`, if present, is merged **after** the main
+    /// file and therefore wins. It exists so the two have different
+    /// owners: CI writes the main config, and only the operator writes
+    /// the secrets — see [`secrets_path`].
     pub fn load(path: impl AsRef<Path>) -> Result<Self, Box<figment::Error>> {
+        let path = path.as_ref();
         Figment::new()
             .merge(Toml::file(path))
+            .merge(Toml::file(secrets_path(path)))
             .merge(Env::prefixed("CORTEX_").split("__"))
             .extract()
             .map_err(Box::new)
     }
 }
 
+/// `secrets.toml` beside the main config.
+///
+/// The split exists to make a class of outage impossible rather than
+/// merely unlikely. Config that CI cannot write is config no check can
+/// defend: `helexa-router.toml` was excluded from git because it sat
+/// beside secret-bearing files, and its `helexa/balanced` alias then
+/// pointed at a retired model for long enough to take a node down
+/// (2026-08-27). The fix is not to hand CI the credentials — it is to
+/// stop mixing the two in one file.
+///
+/// So `cortex.toml` holds structure and is deployed by CI, while
+/// `secrets.toml` holds API keys and the upstream bearer, is written
+/// only by the operator, and is never read, written or diffed by the
+/// pipeline. Because it merges last it also wins, so a value present in
+/// both is the operator's.
+///
+/// A missing file is not an error — figment treats an absent
+/// `Toml::file` as an empty source — so a host that keeps everything in
+/// one file, and every existing deployment, keeps working unchanged.
+fn secrets_path(config: &Path) -> std::path::PathBuf {
+    config
+        .parent()
+        .unwrap_or_else(|| Path::new("."))
+        .join("secrets.toml")
+}
+
 impl Default for GatewayConfig {
     fn default() -> Self {
         Self {
@@ -170,3 +203,111 @@ impl Default for GatewayConfig {
         }
     }
 }
+
+#[cfg(test)]
+mod secrets_layer_tests {
+    use super::*;
+    use std::io::Write;
+
+    /// Write `main` (and optionally `secrets.toml`) into a fresh directory
+    /// under the target dir, and load through the real code path.
+    fn load_with(dir: &str, main: &str, secrets: Option<&str>) -> GatewayConfig {
+        // `CARGO_TARGET_TMPDIR` is only set for integration tests, not for
+        // unit tests inside src/, so derive a unique directory instead.
+        let base = std::env::temp_dir().join(format!("helexa-cortex-config-{dir}"));
+        let _ = std::fs::remove_dir_all(&base);
+        std::fs::create_dir_all(&base).expect("create test dir");
+        let cfg = base.join("cortex.toml");
+        write!(
+            std::fs::File::create(&cfg).expect("create config"),
+            "{main}"
+        )
+        .expect("write");
+        if let Some(s) = secrets {
+            let p = base.join("secrets.toml");
+            write!(std::fs::File::create(&p).expect("create secrets"), "{s}").expect("write");
+        }
+        GatewayConfig::load(&cfg).expect("config loads")
+    }
+
+    const MAIN: &str = r#"
+[gateway]
+listen = "0.0.0.0:31313"
+metrics_listen = "0.0.0.0:31314"
+[eviction]
+strategy = "lru"
+defrag_after_cycles = 50
+[entitlements]
+require_auth = true
+
+[[neurons]]
+name = "beast"
+endpoint = "http://beast.invalid:13131"
+"#;
+
+    /// The whole point: a host with no secrets.toml behaves exactly as
+    /// before. Every current deployment is in this state, so if this
+    /// regressed the change would take the fleet down on rollout rather
+    /// than on adoption.
+    #[test]
+    fn an_absent_secrets_file_is_not_an_error() {
+        let cfg = load_with("secrets_absent", MAIN, None);
+        assert!(cfg.entitlements.require_auth);
+        assert!(
+            cfg.entitlements.keys.is_empty(),
+            "no keys configured anywhere means no keys"
+        );
+    }
+
+    /// The array-of-tables case that ruled out env-var overrides:
+    /// `[[entitlements.keys]]` cannot be expressed as `CORTEX_*` env vars,
+    /// so the second TOML layer is what carries it.
+    #[test]
+    fn secrets_file_supplies_the_entitlement_keys() {
+        let cfg = load_with(
+            "secrets_keys",
+            MAIN,
+            Some(
+                r#"
+[[entitlements.keys]]
+key = "sk-test-aaa"
+account_id = "acct-a"
+key_id = "a"
+[[entitlements.keys]]
+key = "sk-test-bbb"
+account_id = "acct-b"
+key_id = "b"
+"#,
+            ),
+        );
+        assert_eq!(cfg.entitlements.keys.len(), 2);
+        assert_eq!(cfg.entitlements.keys[0].key, "sk-test-aaa");
+        assert_eq!(cfg.entitlements.keys[1].account_id, "acct-b");
+        assert!(
+            cfg.entitlements.require_auth,
+            "structure from the CI-owned file must survive the merge"
+        );
+    }
+
+    /// Ordering is the safety property. If CI ever ships a placeholder in
+    /// the main file, the operator's real value must still win — otherwise
+    /// a deploy silently swaps a live credential for a dummy.
+    #[test]
+    fn the_operators_value_wins_over_the_deployed_one() {
+        let cfg = load_with(
+            "secrets_precedence",
+            &format!(
+                "{MAIN}\n[upstream]\nenabled = true\nurl = \"https://example.invalid\"\nbearer = \"PLACEHOLDER\"\n"
+            ),
+            Some("[upstream]\nbearer = \"real-token\"\n"),
+        );
+        assert_eq!(
+            cfg.upstream.bearer, "real-token",
+            "secrets.toml merges last, so it wins"
+        );
+        assert_eq!(
+            cfg.upstream.url, "https://example.invalid",
+            "fields the operator does not override keep the deployed value"
+        );
+    }
+}

crates/cortex-core/src/harness.rs

@@ -55,13 +55,78 @@ pub struct SamplingOverride {
     pub top_p: Option<f64>,
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub top_k: Option<usize>,
+    /// Multiplier applied to recently-generated tokens before sampling.
+    /// `1.0` disables it; above that, recently-emitted tokens get less
+    /// likely.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub repeat_penalty: Option<f32>,
+    /// How many recently-generated tokens the penalty considers.
+    ///
+    /// The reason this is operator-settable at all: the built-in default
+    /// is 64, inherited from candle's chat example, and a reasoning
+    /// model's think block runs to tens of thousands of tokens. Anything
+    /// restated more than 64 tokens later is invisible to the penalty,
+    /// which is how a 28,672-token think block can contain the same
+    /// derivation five times over.
+    ///
+    /// Raising it is a real trade, not a free win: the penalty is
+    /// token-level, and source code legitimately repeats tokens
+    /// constantly (`const`, `function`, brace runs). A wide window at a
+    /// meaningful penalty distorts exactly the output this fleet is for.
+    /// Sequence-level repetition wants a sequence-level instrument
+    /// (frequency/presence penalties, or a DRY-style sampler); this knob
+    /// exists so the trade can be *measured* per model rather than
+    /// guessed once and frozen into a constant.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub repeat_last_n: Option<usize>,
+    /// Sequence-wide one-time penalty for already-seen tokens.
+    ///
+    /// This, not `repeat_last_n`, is the remedy Qwen prescribe for the
+    /// endless repetition their models fall into during long reasoning
+    /// (0..2, 1.5 when severe). It is scored over everything generated
+    /// so far, so a passage restated 5,000 tokens later is still
+    /// penalised — the case a trailing window cannot see.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub presence_penalty: Option<f32>,
+    /// Sequence-wide penalty proportional to a token's existing count.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub frequency_penalty: Option<f32>,
+    /// Pin the sampler's RNG for every request to this model.
+    ///
+    /// Exists because parameter search needs a control: comparing two
+    /// `presence_penalty` values across unseeded runs cannot separate
+    /// the parameter's effect from sampling variance, and the clients
+    /// that would otherwise pin it cannot — pi's request builder sends
+    /// no `seed` at all, so a caller-side seed is not available on the
+    /// path that matters.
+    ///
+    /// Note what this does and does not give. It removes sampling noise
+    /// as a confounder; it does not make two runs comparable
+    /// token-by-token, because changing a penalty changes the logits
+    /// and the sequences diverge at the first step where a token choice
+    /// flips.
+    ///
+    /// Leaving this set is a product decision, not just a testing one:
+    /// every identical prompt then yields an identical answer for every
+    /// caller. That is defensible — predictable and cacheable — but it
+    /// removes the variation a user may expect from retrying. A request
+    /// that names its own seed still wins.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub seed: Option<u64>,
 }
 
 impl SamplingOverride {
     /// True when the operator set nothing — an empty `[sampling]` table
     /// must behave exactly as no table at all.
     pub fn is_empty(&self) -> bool {
-        self.temperature.is_none() && self.top_p.is_none() && self.top_k.is_none()
+        self.temperature.is_none()
+            && self.top_p.is_none()
+            && self.top_k.is_none()
+            && self.repeat_penalty.is_none()
+            && self.repeat_last_n.is_none()
+            && self.presence_penalty.is_none()
+            && self.frequency_penalty.is_none()
+            && self.seed.is_none()
     }
 }
 
@@ -86,6 +151,27 @@ pub struct ModelSpec {
     /// still round-trips the spec.
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub sampling: Option<SamplingOverride>,
+    /// Whether prior assistant turns keep their reasoning when the chat
+    /// template re-renders the conversation.
+    ///
+    /// This is the model's own `preserve_thinking` template control, not
+    /// an invention of ours. `Qwen/Qwen3.8-27B` defaults it to *true* —
+    /// the whole transcript's think blocks are replayed — and `false`
+    /// keeps reasoning only for turns after the last user query, i.e.
+    /// the turn in progress.
+    ///
+    /// `None` leaves the kwarg unset so the template's own default
+    /// applies, which is the only safe default: the model's authors
+    /// chose it, and overriding that fleet-wide on a hunch is how you
+    /// ship a regression you cannot see.
+    ///
+    /// Exposed as operator config so the premise can be A/B'd on a real
+    /// workload — full replay is the larger prompt and pushes against
+    /// the prefix-cache budget and the throughput-derived context
+    /// ceiling, but whether it *helps* the model is a measurement, not
+    /// a belief. A request's own `chat_template_kwargs` still wins.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub preserve_thinking: Option<bool>,
 }
 
 /// Per-model token budget advertised by the catalogue or neuron.
@@ -141,9 +227,38 @@ fn is_zero(v: &usize) -> bool {
 #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
 pub struct ReasoningBudgetRung {
     /// The effort level as the caller spells it on the wire.
+    ///
+    /// Since #290 these are the rungs the **model's own chat template**
+    /// accepts, discovered by rendering it, rather than a ladder the
+    /// operator invented. `Qwen/Qwen3.8-27B` accepts exactly `low`,
+    /// `medium` and `xhigh` and raises on anything else; we previously
+    /// advertised `minimal`/`low`/`medium`/`high`, two of which the
+    /// template rejects and which omitted the model's own default.
+    ///
+    /// Advertise only rungs the template accepts: an invented name is a
+    /// name a caller can send and the model will reject. Note that a
+    /// client need not read this at all — pi-ai takes its levels from a
+    /// static `thinkingLevelMap` in the operator's config — so this is
+    /// a statement of truth, not a control surface we can rely on.
     pub effort: String,
-    /// Reasoning tokens that level is allowed to spend.
-    pub tokens: usize,
+    /// Backstop reasoning-token cap for this rung, when the deployment
+    /// sets one.
+    ///
+    /// `None` is the normal case and means "named rung, no cap": effort
+    /// is expressed to the model through its template, and the number of
+    /// tokens it then spends is the model's business. A cap is a safety
+    /// net against a runaway think block (#223), not the mechanism by
+    /// which effort is selected — enforcing effort by truncation is what
+    /// #290 fixed.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub tokens: Option<usize>,
+    /// Whether the model applies this rung when the caller names none.
+    ///
+    /// Taken from the template's own `|default(...)`. Without it a
+    /// caller that says nothing cannot tell what it is getting — on
+    /// Qwen3.8 that silently means `xhigh`.
+    #[serde(default, skip_serializing_if = "std::ops::Not::not")]
+    pub default: bool,
 }
 
 /// Operator-set pricing, **USD per 1,000,000 tokens, as JSON numbers**
@@ -218,6 +333,16 @@ pub struct ModelInfo {
     /// marker tokens (`<think>` / `<\/think>` or similar).
     #[serde(default)]
     pub reasoning: bool,
+    /// The operator's `preserve_thinking` for this loaded model, or
+    /// `None` when unset and the template's own default applies.
+    ///
+    /// Advertised so a run can be *stamped* with the value it ran
+    /// under. This is an A/B knob whose whole purpose is comparison
+    /// between runs, and a comparison whose arms cannot be told apart
+    /// afterwards is not a comparison. Recovering it from deploy
+    /// history is guesswork — a config sync can land mid-session.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub preserve_thinking: Option<bool>,
 
     /// Whether this node can actually serve a request for this model
     /// right now (#245).

crates/cortex-core/src/node.rs

@@ -194,10 +194,19 @@ pub struct CortexModelEntry {
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub max_model_len: Option<usize>,
     /// The same window under the two names generic OpenAI clients look
-    /// for (#278). pi-ai's provider discovery, for one, reads
-    /// `context_window` then `context_length` and nothing else — so a
-    /// client probing cortex found no window at all and fell back to a
-    /// number its operator had to invent by hand.
+    /// for (#278) — a client that probes for one of these rather than
+    /// `max_model_len` would otherwise find no window at all and fall
+    /// back to a number its operator had to invent by hand.
+    ///
+    /// Publishing them is cheap and harmless; do not assume any
+    /// *specific* client consumes them. An earlier version of this
+    /// comment credited pi-ai's provider discovery, which was wrong:
+    /// pi requires `contextWindow` as a static field on its model type
+    /// and reads none of these keys at runtime (checked against 0.82.1
+    /// and 0.84.3). The misattribution mattered — it made an operator's
+    /// hand-set 80k window look like our advertisement failing to land,
+    /// when in fact nothing was reading it and the number had to be
+    /// maintained by hand regardless.
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub context_window: Option<usize>,
     #[serde(default, skip_serializing_if = "Option::is_none")]
@@ -227,12 +236,16 @@ impl CortexModelEntry {
         self.context_window = self.limit.as_ref().map(|l| l.context);
         self.context_length = self.limit.as_ref().map(|l| l.context);
         self.max_input_tokens = self.limit.as_ref().and_then(|l| l.input);
-        // The ceiling, not the reserve (#278): this field is what
-        // OpenAI-ecosystem clients read to decide what to ask for, and
-        // pi-ai/DSH turns it straight into a per-request cap. Advertising
-        // the reserve here handed a reasoning model less budget than its
-        // own think block needs. Falls back to the reserve for a neuron
-        // too old to publish a ceiling.
+        // The ceiling, not the reserve (#278): a client that reads this
+        // field turns it into the cap it sends on each request, so
+        // advertising the reserve handed a reasoning model less budget
+        // than its own think block needs. Falls back to the reserve for
+        // a neuron too old to publish a ceiling.
+        //
+        // Observed with dsh. pi is *not* an example: its `maxTokens` is
+        // a required static field in models.json and it reads nothing
+        // from here — an operator has to copy this number across by
+        // hand, and gets no error if they copy it wrong.
         self.max_output_tokens = self.limit.as_ref().map(|l| {
             if l.output_ceiling > 0 {
                 l.output_ceiling

crates/cortex-core/src/openai.rs

@@ -43,6 +43,20 @@ pub struct ChatCompletionRequest {
     /// why it is a knob rather than a constant (#272).
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub repeat_last_n: Option<usize>,
+    /// OpenAI core, `-2.0..=2.0`. Subtracted once from any token that
+    /// has already appeared, regardless of how often.
+    ///
+    /// Unlike `repetition_penalty` this is scored over the **whole**
+    /// generated sequence, not a trailing window — which is what a
+    /// reasoning model needs, and what Qwen recommend (0..2, 1.5 for
+    /// severe cases) for the endless-repetition failure their own
+    /// models exhibit during long thinking.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub presence_penalty: Option<f32>,
+    /// OpenAI core, `-2.0..=2.0`. Subtracted in proportion to how many
+    /// times a token has already appeared.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub frequency_penalty: Option<f32>,
     /// The deprecated spelling of the output cap. Still what most
     /// clients send; see [`ChatCompletionRequest::effective_max_tokens`].
     #[serde(skip_serializing_if = "Option::is_none")]

crates/cortex-core/src/responses.rs

@@ -70,6 +70,11 @@ pub struct ResponsesRequest {
     /// Window `repetition_penalty` considers (#272).
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub repeat_last_n: Option<usize>,
+    /// OpenAI core; forwarded to the chat path unchanged.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub presence_penalty: Option<f32>,
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub frequency_penalty: Option<f32>,
     /// Chained-conversation identifier. We don't store responses
     /// server-side yet; if this is `Some`, the handler returns 400.
     #[serde(default, skip_serializing_if = "Option::is_none")]
@@ -215,11 +220,46 @@ pub struct ResponsesResponse {
     pub status: String,
     pub model: String,
     pub output: Vec<ResponsesOutputItem>,
+    /// Why a `status: "incomplete"` response stopped short. Required in
+    /// practice, not optional decoration: a client cannot tell an
+    /// honest truncation from a protocol fault without it, and at least
+    /// one treats the difference as fatal.
+    ///
+    /// pi-ai's `mapStopReason` maps
+    /// `incomplete` + `reason == "max_output_tokens"` to
+    /// `stopReason: "length"`, and `incomplete` with no reason to
+    /// `stopReason: "error"` ("Response incomplete without a provider
+    /// reason"). Its agent loop then either continues — failing any
+    /// truncated tool calls safely and letting the model recover on the
+    /// next turn — or halts outright. Omitting this field cost a live
+    /// agentic session its whole run.
+    ///
+    /// `None` for `completed` and `in_progress`.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub incomplete_details: Option<IncompleteDetails>,
     /// Populated on completion; `None` while streaming.
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub usage: Option<ResponsesUsage>,
 }
 
+/// Why a response is `incomplete`.
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
+pub struct IncompleteDetails {
+    /// OpenAI spells the output-cap case `"max_output_tokens"`; that is
+    /// the only reason we produce today. `content_filter` is the other
+    /// value the upstream API defines.
+    pub reason: String,
+}
+
+impl IncompleteDetails {
+    /// The response ran into the caller's output budget.
+    pub fn max_output_tokens() -> Self {
+        Self {
+            reason: "max_output_tokens".into(),
+        }
+    }
+}
+
 #[derive(Debug, Clone, Serialize, Deserialize)]
 #[serde(tag = "type", rename_all = "snake_case")]
 pub enum ResponsesOutputItem {
@@ -236,6 +276,15 @@ pub enum ResponsesOutputItem {
         #[serde(default = "default_item_status")]
         status: String,
     },
+    /// A think block, as its own output item ahead of the message —
+    /// the same shape the streaming projector emits, so a client sees
+    /// one contract whether or not it streams (#300).
+    Reasoning {
+        id: String,
+        summary: Vec<ResponsesSummaryPart>,
+        #[serde(default = "default_item_status")]
+        status: String,
+    },
     /// Reserved for the day tool-call extraction lands. The wire
     /// shape mirrors `ResponsesInputItem::FunctionCall`.
     FunctionCall {
@@ -248,6 +297,24 @@ pub enum ResponsesOutputItem {
     },
 }
 
+/// One `summary_text` part of a reasoning output item.
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct ResponsesSummaryPart {
+    /// Always `"summary_text"`.
+    #[serde(rename = "type")]
+    pub part_type: String,
+    pub text: String,
+}
+
+impl ResponsesSummaryPart {
+    pub fn summary_text(text: impl Into<String>) -> Self {
+        Self {
+            part_type: "summary_text".into(),
+            text: text.into(),
+        }
+    }
+}
+
 fn default_item_status() -> String {
     "completed".into()
 }
@@ -327,6 +394,24 @@ pub mod events {
     pub const FUNCTION_CALL_ARGUMENTS_DELTA: &str = "response.function_call_arguments.delta";
     pub const FUNCTION_CALL_ARGUMENTS_DONE: &str = "response.function_call_arguments.done";
     pub const COMPLETED: &str = "response.completed";
+    /// Terminal frame for a response that stopped short. Carries the
+    /// same `response` payload as [`COMPLETED`], with
+    /// `status: "incomplete"` and `incomplete_details.reason`.
+    ///
+    /// A client keyed on the event name — rather than on
+    /// `response.status` — sees no terminal event at all if a truncated
+    /// response is announced as `response.completed`, and waits out its
+    /// idle timeout.
+    pub const INCOMPLETE: &str = "response.incomplete";
+    /// Terminal frame for a response that did not finish because the
+    /// server failed — a poisoned device, an OOM, a producer that died
+    /// mid-stream. Carries `status: "failed"` and an `error` object.
+    ///
+    /// The alternative is worse than it looks: defaulting a missing
+    /// finish to a clean stop reports a crash as a complete answer, and
+    /// the caller believes a truncated or empty reply was the model's
+    /// considered response.
+    pub const FAILED: &str = "response.failed";
 }
 
 #[cfg(test)]
@@ -531,6 +616,7 @@ mod tests {
                 }],
                 status: "completed".into(),
             }],
+            incomplete_details: None,
             usage: Some(ResponsesUsage {
                 input_tokens: 5,
                 output_tokens: 3,
@@ -543,5 +629,36 @@ mod tests {
         let parsed: ResponsesResponse = serde_json::from_str(&json).unwrap();
         assert_eq!(parsed.id, "resp_1");
         assert_eq!(parsed.output.len(), 1);
+        assert!(
+            !json.contains("incomplete_details"),
+            "a completed response must not carry an incomplete reason"
+        );
+    }
+
+    /// The field pi-ai's `mapStopReason` keys on. Absent, it maps
+    /// `incomplete` to `stopReason: "error"` and its agent loop halts;
+    /// present with `max_output_tokens`, it maps to `"length"` and the
+    /// loop continues.
+    #[test]
+    fn an_incomplete_response_serialises_its_reason() {
+        let r = ResponsesResponse {
+            id: "resp_2".into(),
+            object: "response".into(),
+            created_at: 1700,
+            status: "incomplete".into(),
+            model: "m".into(),
+            output: vec![],
+            incomplete_details: Some(IncompleteDetails::max_output_tokens()),
+            usage: None,
+        };
+        let v: serde_json::Value = serde_json::to_value(&r).unwrap();
+        assert_eq!(v["status"], "incomplete");
+        assert_eq!(v["incomplete_details"]["reason"], "max_output_tokens");
+
+        let parsed: ResponsesResponse = serde_json::from_value(v).unwrap();
+        assert_eq!(
+            parsed.incomplete_details,
+            Some(IncompleteDetails::max_output_tokens())
+        );
     }
 }

crates/cortex-core/src/translate.rs

@@ -130,6 +130,8 @@ pub fn anthropic_to_openai(req: MessagesRequest) -> ChatCompletionRequest {
         seed: None,
         repetition_penalty: None,
         repeat_last_n: None,
+        presence_penalty: None,
+        frequency_penalty: None,
         max_tokens: Some(req.max_tokens),
         // Anthropic has one spelling; it maps onto the legacy field.
         max_completion_tokens: None,
@@ -413,17 +415,26 @@ fn anthropic_content_to_text(content: AnthropicContent) -> String {
 pub fn openai_to_anthropic(resp: ChatCompletionResponse) -> MessagesResponse {
     let choice = resp.choices.into_iter().next();
 
-    let (content_text, stop_reason) = match choice {
+    let (content_text, reasoning_text, stop_reason) = match choice {
         Some(c) => {
             let text = match c.message.content {
                 MessageContent::Text(t) => t,
                 MessageContent::Parts(parts) => serde_json::to_string(&parts).unwrap_or_default(),
                 MessageContent::Null => String::new(),
             };
+            // The model's think block, which the harness returns on
+            // `message.reasoning_content` (#300).
+            let reasoning = c
+                .message
+                .extra
+                .get("reasoning_content")
+                .and_then(|v| v.as_str())
+                .map(str::to_string)
+                .filter(|t| !t.is_empty());
             let stop = c.finish_reason.map(|r| map_stop_reason(&r));
-            (text, stop)
+            (text, reasoning, stop)
         }
-        None => (String::new(), None),
+        None => (String::new(), None, None),
     };
 
     let usage = resp.usage.unwrap_or(Usage {
@@ -439,10 +450,32 @@ pub fn openai_to_anthropic(resp: ChatCompletionResponse) -> MessagesResponse {
         id: resp.id,
         response_type: "message".into(),
         role: "assistant".into(),
-        content: vec![ContentBlock {
-            block_type: "text".into(),
-            data: json!({ "text": content_text }),
-        }],
+        // A `thinking` block ahead of the text, which is Anthropic's
+        // own ordering. This path accepted `thinking.budget_tokens` on
+        // the way in (see `anthropic_to_openai`) and returned nothing on
+        // the way out: the caller spent the tokens, could not see them,
+        // and could not replay the model's reasoning next turn (#300).
+        //
+        // Emitted without a `signature`. Anthropic signs its thinking
+        // blocks and a client that round-trips one may expect the field;
+        // we cannot mint a valid signature and inventing one would be
+        // worse than omitting it. If a real client rejects an unsigned
+        // block, the fix is to gate this behind a flag — not to go back
+        // to silently deleting the model's work.
+        content: {
+            let mut blocks = Vec::with_capacity(2);
+            if let Some(thinking) = reasoning_text {
+                blocks.push(ContentBlock {
+                    block_type: "thinking".into(),
+                    data: json!({ "thinking": thinking }),
+                });
+            }
+            blocks.push(ContentBlock {
+                block_type: "text".into(),
+                data: json!({ "text": content_text }),
+            });
+            blocks
+        },
         model: resp.model,
         stop_reason,
         usage: AnthropicUsage {
@@ -499,6 +532,7 @@ pub struct AnthropicStreamTranslator {
 enum OpenBlock {
     Text,
     ToolUse,
+    Thinking,
 }
 
 impl AnthropicStreamTranslator {
@@ -538,6 +572,32 @@ impl AnthropicStreamTranslator {
         }
 
         for choice in &chunk.choices {
+            // The model's think block. Dropped entirely until #300: this
+            // translator read `content` and `tool_calls` and nothing
+            // else, so an Anthropic client asked for extended thinking
+            // (`thinking.budget_tokens`, which `anthropic_to_openai`
+            // accepts), was billed for the tokens, and received a stream
+            // with no thinking in it — every turn, silently. Fixing only
+            // the non-streaming translation would have missed this,
+            // because the clients that ask for thinking stream.
+            if let Some(text) = choice
+                .delta
+                .get("reasoning_content")
+                .and_then(Value::as_str)
+                && !text.is_empty()
+            {
+                self.ensure_thinking_block(&mut out);
+                let index = self.open_block.as_ref().map(|(i, _)| *i).unwrap_or(0);
+                out.push((
+                    "content_block_delta".to_string(),
+                    json!({
+                        "type": "content_block_delta",
+                        "index": index,
+                        "delta": { "type": "thinking_delta", "thinking": text }
+                    }),
+                ));
+            }
+
             if let Some(text) = choice.delta.get("content").and_then(Value::as_str)
                 && !text.is_empty()
             {
@@ -651,6 +711,32 @@ impl AnthropicStreamTranslator {
         out
     }
 
+    /// Open a `thinking` block, closing whatever preceded it.
+    ///
+    /// Emitted without a `signature`. Anthropic signs its thinking
+    /// blocks; we cannot mint a valid one, and inventing a value would
+    /// be worse than omitting it. If a client rejects an unsigned block
+    /// the fix is a flag — not a return to deleting the model's work.
+    fn ensure_thinking_block(&mut self, out: &mut Vec<(String, Value)>) {
+        match &self.open_block {
+            Some((_, OpenBlock::Thinking)) => {}
+            _ => {
+                self.close_open_block(out);
+                let index = self.next_index;
+                self.next_index += 1;
+                self.open_block = Some((index, OpenBlock::Thinking));
+                out.push((
+                    "content_block_start".to_string(),
+                    json!({
+                        "type": "content_block_start",
+                        "index": index,
+                        "content_block": { "type": "thinking", "thinking": "" }
+                    }),
+                ));
+            }
+        }
+    }
+
     fn ensure_text_block(&mut self, out: &mut Vec<(String, Value)>) {
         match &self.open_block {
             Some((_, OpenBlock::Text)) => {}
@@ -1060,4 +1146,115 @@ mod request_tests {
         let tools = openai.extra.get("tools").and_then(Value::as_array).unwrap();
         assert_eq!(tools[0]["function"]["name"], "x");
     }
+
+    /// The Anthropic path must return the model's thinking (#300).
+    ///
+    /// It accepted `thinking.budget_tokens` on the way in and returned
+    /// a bare text block on the way out: the caller spent the tokens,
+    /// never saw them, and could not replay the model's reasoning on
+    /// the next turn. Taking a parameter and silently discarding what
+    /// it buys is the defect, not the missing field.
+    #[test]
+    fn anthropic_response_carries_a_thinking_block() {
+        let mut extra = serde_json::Map::new();
+        extra.insert("reasoning_content".into(), json!("weighing the options"));
+        let resp = ChatCompletionResponse {
+            id: "id".into(),
+            object: "chat.completion".into(),
+            created: 0,
+            model: "m".into(),
+            choices: vec![crate::openai::ChatCompletionChoice {
+                index: 0,
+                message: ChatMessage {
+                    role: "assistant".into(),
+                    content: MessageContent::Text("the answer".into()),
+                    extra: Value::Object(extra),
+                },
+                finish_reason: Some("stop".into()),
+                extra: Value::Null,
+            }],
+            usage: None,
+            extra: Value::Null,
+        };
+        let a = openai_to_anthropic(resp);
+        assert_eq!(a.content.len(), 2, "expected thinking + text");
+        assert_eq!(a.content[0].block_type, "thinking", "thinking comes first");
+        assert_eq!(a.content[0].data["thinking"], "weighing the options");
+        assert_eq!(a.content[1].block_type, "text");
+        assert_eq!(a.content[1].data["text"], "the answer");
+    }
+
+    /// No thinking means the unchanged single text block.
+    #[test]
+    fn anthropic_response_without_thinking_is_unchanged() {
+        let resp = ChatCompletionResponse {
+            id: "id".into(),
+            object: "chat.completion".into(),
+            created: 0,
+            model: "m".into(),
+            choices: vec![crate::openai::ChatCompletionChoice {
+                index: 0,
+                message: ChatMessage {
+                    role: "assistant".into(),
+                    content: MessageContent::Text("plain".into()),
+                    extra: Value::Object(Default::default()),
+                },
+                finish_reason: Some("stop".into()),
+                extra: Value::Null,
+            }],
+            usage: None,
+            extra: Value::Null,
+        };
+        let a = openai_to_anthropic(resp);
+        assert_eq!(a.content.len(), 1);
+        assert_eq!(a.content[0].block_type, "text");
+    }
+
+    /// Streaming Anthropic must carry thinking too (#300).
+    ///
+    /// Fixing only the non-streaming translation would have missed this
+    /// entirely — and missed it for the clients that matter, since the
+    /// ones asking for extended thinking stream.
+    #[test]
+    fn anthropic_stream_emits_thinking_deltas() {
+        use crate::openai::{ChatCompletionChunk, ChunkChoice};
+        let mut t = AnthropicStreamTranslator::new();
+        let chunk = |delta: Value| ChatCompletionChunk {
+            id: "c".into(),
+            object: "chat.completion.chunk".into(),
+            created: 0,
+            model: "m".into(),
+            choices: vec![ChunkChoice {
+                index: 0,
+                delta,
+                finish_reason: None,
+                extra: Value::Null,
+            }],
+            usage: None,
+            extra: Value::Null,
+        };
+
+        let ev = t.on_chunk(&chunk(json!({ "reasoning_content": "weighing it" })));
+        let start = ev
+            .iter()
+            .find(|(n, _)| n == "content_block_start")
+            .expect("a block must open");
+        assert_eq!(start.1["content_block"]["type"], "thinking");
+        let delta = ev
+            .iter()
+            .find(|(n, _)| n == "content_block_delta")
+            .expect("a delta must follow");
+        assert_eq!(delta.1["delta"]["type"], "thinking_delta");
+        assert_eq!(delta.1["delta"]["thinking"], "weighing it");
+
+        // Visible text opens a separate block, after thinking closes.
+        let ev = t.on_chunk(&chunk(json!({ "content": "the answer" })));
+        let names: Vec<&str> = ev.iter().map(|(n, _)| n.as_str()).collect();
+        assert!(
+            names.contains(&"content_block_stop"),
+            "the thinking block must close first: {names:?}"
+        );
+        let start = ev.iter().find(|(n, _)| n == "content_block_start").unwrap();
+        assert_eq!(start.1["content_block"]["type"], "text");
+    }
 }

crates/cortex-gateway/src/metering.rs

@@ -288,7 +288,7 @@ mod tests {
         // Requested output dominates; prompt estimate is small for a tiny body.
         let body = br#"{"model":"m","max_tokens":1000}"#;
         let est = reservation_estimate(body, Some(8192));
-        assert!(est >= 1000 && est < 1100, "est was {est}");
+        assert!((1000..1100).contains(&est), "est was {est}");
     }
 
     #[test]

crates/cortex-gateway/src/router.rs

@@ -539,6 +539,7 @@ async fn profile_to_spec(
         // (#283), so a cold-loaded model samples the same way as one
         // this host was configured to hold resident.
         sampling: profile.sampling.clone(),
+        preserve_thinking: profile.preserve_thinking,
     }
 }
 
@@ -647,6 +648,7 @@ mod tests {
             residency_priority: None,
             source: source.map(String::from),
             sampling: None,
+            preserve_thinking: None,
             limit: None,
             cost: None,
             capabilities: vec![],

crates/cortex-gateway/tests/budget_enforcement.rs

@@ -1,9 +1,9 @@
 //! Integration tests for budget enforcement (#52) — the A0 seatbelt.
 //!
 //! A reservation over the key's hard cap is refused *before* neuron is hit,
-//! with the #63 code matching the cap-window semantics (rate_limit_exceeded
-//! + Retry-After for a resetting window, insufficient_quota for a hard
-//! balance). Spend never exceeds the cap. No 402, ever.
+//! with the #63 code matching the cap-window semantics: `rate_limit_exceeded`
+//! plus `Retry-After` for a resetting window, `insufficient_quota` for a hard
+//! balance. Spend never exceeds the cap. No 402, ever.
 
 use axum::Json;
 use axum::extract::Path;

crates/cortex-gateway/tests/model_limits.rs

@@ -69,17 +69,20 @@ capabilities = ["text"]
         let mut nodes = fleet.nodes.write().await;
         let node = nodes.get_mut("mock-node").expect("node exists");
         node.healthy = true;
-        // The ladder the neuron reported at poll time (#223) — host
-        // configuration, held once per node.
-        node.reasoning_budget = ["minimal", "low", "medium", "high"]
+        // The rungs the neuron reported at poll time. Since #290 these
+        // come from the model's own chat template rather than operator
+        // configuration, so the fixture uses the shape a real reasoning
+        // model ships: Qwen3.8-27B accepts exactly low/medium/xhigh and
+        // defaults to xhigh.
+        node.reasoning_budget = ["low", "medium", "xhigh"]
             .into_iter()
-            .zip([1024usize, 4096, 12288, 32768])
-            .map(
-                |(effort, tokens)| cortex_core::harness::ReasoningBudgetRung {
-                    effort: effort.into(),
-                    tokens,
-                },
-            )
+            .map(|effort| cortex_core::harness::ReasoningBudgetRung {
+                default: effort == "xhigh",
+                effort: effort.into(),
+                // No cap: effort is expressed to the model through its
+                // template, not enforced by truncating its thinking.
+                tokens: None,
+            })
             .collect();
         node.models.insert(
             "test-model".into(),
@@ -162,14 +165,16 @@ capabilities = ["text"]
     // vLLM-convention probes (Hermes Agent) auto-detect the window.
     assert_eq!(entry["max_model_len"], 49152);
     assert_eq!(entry["max_input_tokens"], 40960);
-    // The ceiling, not the reserve (#278). pi-ai's discovery — and every
-    // client like it — turns this field into the cap it sends on each
-    // request; advertising the 8192 reserve told a reasoning model's
-    // harness to stop below the cost of a single think block.
+    // The ceiling, not the reserve (#278). A client that reads this
+    // field turns it into the cap it sends on each request; advertising
+    // the 8192 reserve told a reasoning model's harness to stop below
+    // the cost of a single think block. Observed with dsh — pi takes
+    // `maxTokens` from static config and reads nothing here.
     assert_eq!(entry["max_output_tokens"], 32768);
-    // What each effort level buys (#223). A client can only send the
-    // rung names — it has no way to express a token count — so a ladder
-    // it cannot see is a ladder it has to guess at.
+    // The rungs the model itself accepts (#290). A caller can only send
+    // a rung name, so a name we invent is one the template will reject.
+    // Publishing the true set is correct regardless of who reads it —
+    // pi-ai, for one, takes its levels from static config instead.
     let rungs = entry["reasoning_budget"]
         .as_array()
         .expect("reasoning_budget advertised for a reasoning model");
@@ -177,17 +182,34 @@ capabilities = ["text"]
         .iter()
         .map(|r| r["effort"].as_str().expect("effort"))
         .collect();
-    assert_eq!(names, ["minimal", "low", "medium", "high"]);
+    assert_eq!(names, ["low", "medium", "xhigh"]);
+    // Which rung applies when the caller names none. Without this a
+    // silent caller cannot tell what it is getting — on Qwen3.8 that is
+    // xhigh, the most expensive rung on the ladder.
+    let default_rung: Vec<&str> = rungs
+        .iter()
+        .filter(|r| r["default"].as_bool().unwrap_or(false))
+        .map(|r| r["effort"].as_str().expect("effort"))
+        .collect();
+    assert_eq!(
+        default_rung,
+        ["xhigh"],
+        "the model's own default must be visible"
+    );
+    // A named rung carries no token count: effort reaches the model
+    // through its template, and enforcing it by truncation is the defect
+    // #290 fixed.
     assert!(
-        rungs
-            .iter()
-            .all(|r| r["tokens"].as_u64().is_some_and(|t| t > 0)),
-        "every advertised rung must name a token count"
+        rungs.iter().all(|r| r.get("tokens").is_none()),
+        "a named rung must not masquerade as a token budget"
     );
 
-    // The window under the two names generic clients actually read.
-    // pi-ai looks for `context_window` then `context_length` and gives
-    // up if neither is present — `max_model_len` is invisible to it.
+    // The window under the two names a generic client may read, for one
+    // that probes these rather than `max_model_len`. Publish all three;
+    // do not assume a particular client consumes any of them. pi, in
+    // particular, reads none — `contextWindow` is a required static
+    // field on its model type (checked against 0.82.1 and 0.84.3), so
+    // its window is hand-maintained whatever we advertise.
     assert_eq!(entry["context_window"], 49152);
     assert_eq!(entry["context_length"], 49152);
 

crates/helexa-acp/README.md

@@ -80,7 +80,7 @@ The fastest path: env-var single-endpoint config.
 
 ```sh
 export HELEXA_ACP_BASE_URL=http://hanzalova.internal:31313/v1
-export HELEXA_ACP_MODEL=Qwen/Qwen3.6-27B
+export HELEXA_ACP_MODEL=Qwen/Qwen3.8-27B
 helexa-acp  # speaks ACP over stdin/stdout; not interactive
 ```
 
@@ -117,7 +117,7 @@ default_endpoint = "helexa"
 name = "helexa"
 base_url = "http://hanzalova.internal:31313/v1"
 wire_api = "openai-chat"
-default_model = "Qwen/Qwen3.6-27B"
+default_model = "Qwen/Qwen3.8-27B"
 max_tokens = 8192
 context_window = 32768
 
@@ -138,7 +138,7 @@ default_model = "claude-opus-4"
 
 Restart Zed. The model dropdown lists every model from every
 configured endpoint with the `endpoint:model` selector
-(`helexa:Qwen/Qwen3.6-27B`, `openrouter:anthropic/claude-opus-4`,
+(`helexa:Qwen/Qwen3.8-27B`, `openrouter:anthropic/claude-opus-4`,
 …). Switch mid-session; the next prompt routes to the new endpoint.
 
 When only one endpoint is configured the prefix is dropped (model
@@ -166,7 +166,7 @@ Copy-pasteable blocks. Mix and match.
 name = "helexa"
 base_url = "http://hanzalova.internal:31313/v1"
 wire_api = "openai-chat"
-default_model = "Qwen/Qwen3.6-27B"
+default_model = "Qwen/Qwen3.8-27B"
 max_tokens = 8192
 context_window = 32768
 ```
@@ -241,7 +241,7 @@ vLLM.
 name = "lan"
 base_url = "http://hanzalova.internal:31313/v1"
 wire_api = "openai-chat"
-default_model = "Qwen/Qwen3.6-27B"
+default_model = "Qwen/Qwen3.8-27B"
 
 [[endpoints]]
 name = "cloud"

crates/helexa-angels/src/config.rs

@@ -214,11 +214,39 @@ fn default_notify_to() -> String {
 }
 
 impl AngelsConfig {
+    /// A sibling `secrets.toml`, if present, merges after the main file
+    /// and wins — see [`secrets_path`].
     pub fn load(path: impl AsRef<Path>) -> anyhow::Result<Self> {
+        let path = path.as_ref();
         let cfg: Self = Figment::new()
-            .merge(Toml::file(path.as_ref()))
+            .merge(Toml::file(path))
+            .merge(Toml::file(secrets_path(path)))
             .merge(Env::prefixed("ANGELS_").split("__"))
             .extract()?;
         Ok(cfg)
     }
 }
+
+/// `secrets.toml` beside the main config.
+///
+/// The main file holds structure and is deployed by CI; this one holds
+/// credentials, is written only by the operator, and is never read,
+/// written or diffed by the pipeline. Merging it last means the
+/// operator's value wins, so a deploy cannot swap a live credential for
+/// a placeholder.
+///
+/// The split exists because config CI cannot write is config no check
+/// can defend — `helexa-router.toml` was kept out of git for sitting
+/// beside secret-bearing files, and its stale alias then took a node
+/// down (2026-08-27). The answer is to separate secrets from structure,
+/// not to hand the pipeline the credentials.
+///
+/// A missing file is not an error: figment treats an absent
+/// `Toml::file` as an empty source, so existing single-file deployments
+/// keep working untouched.
+fn secrets_path(config: &Path) -> std::path::PathBuf {
+    config
+        .parent()
+        .unwrap_or_else(|| Path::new("."))
+        .join("secrets.toml")
+}

crates/helexa-bench/src/api.rs

@@ -34,6 +34,7 @@ pub fn api_routes(state: ApiState) -> Router {
     Router::new()
         .route("/api/health", get(health))
         .route("/api/dimensions", get(dimensions))
+        .route("/api/regimes", get(regimes))
         .route("/api/summary", get(summary))
         .route("/api/scaling", get(scaling))
         .route("/api/concurrency", get(concurrency))
@@ -65,6 +66,14 @@ async fn health(State(s): State<ApiState>) -> Result<Json<serde_json::Value>, Ap
     Ok(Json(json!({ "status": "ok", "run_count": count })))
 }
 
+/// Declared measurement-regime boundaries (#288).
+///
+/// Static — no store access — but served from the API rather than
+/// hardcoded in the frontend so `report` and the UI cite one list.
+async fn regimes() -> Json<&'static [crate::regime::MeasurementRegime]> {
+    Json(crate::regime::REGIMES)
+}
+
 async fn dimensions(State(s): State<ApiState>) -> Result<Json<crate::store::Dimensions>, ApiError> {
     let store = s.lock().await;
     store.dimensions().map(Json).map_err(err500)

crates/helexa-bench/src/client.rs

@@ -6,6 +6,7 @@ use crate::config::{TargetConfig, TargetKind};
 use anyhow::{Context, Result, anyhow};
 use cortex_core::build_info::BuildInfo;
 use cortex_core::discovery::{DiscoveryResponse, HealthResponse};
+use cortex_core::entitlements::{HEADER_ACCOUNT_ID, HEADER_KEY_ID};
 use cortex_core::harness::{ModelInfo, ModelSpec};
 use cortex_core::openai::ModelsResponse;
 use std::time::Duration;
@@ -19,10 +20,49 @@ pub struct TargetClient {
 
 impl TargetClient {
     pub fn new(request_timeout: Duration) -> Result<Self> {
-        let http = reqwest::Client::builder()
-            .timeout(request_timeout)
-            .build()
-            .context("building HTTP client")?;
+        Self::with_principal(request_timeout, None)
+    }
+
+    /// Build a client that stamps `principal`'s identity headers on every
+    /// request (#288).
+    ///
+    /// Set on the client rather than at each call site so no scenario can
+    /// be added later that silently measures as anonymous — which is the
+    /// failure this fixes, and it was invisible for a week.
+    ///
+    /// `None` keeps the historical anonymous behaviour, which is correct
+    /// for `openai` targets (the headers mean nothing to a foreign
+    /// engine) and for a fresh install with no principal configured.
+    pub fn with_principal(
+        request_timeout: Duration,
+        principal: Option<&crate::config::PrincipalSettings>,
+    ) -> Result<Self> {
+        let mut builder = reqwest::Client::builder().timeout(request_timeout);
+        if let Some(p) = principal {
+            let mut headers = reqwest::header::HeaderMap::new();
+            headers.insert(
+                HEADER_ACCOUNT_ID,
+                reqwest::header::HeaderValue::from_str(&p.account_id)
+                    .context("principal.account_id is not a valid header value")?,
+            );
+            headers.insert(
+                HEADER_KEY_ID,
+                reqwest::header::HeaderValue::from_str(&p.key_id)
+                    .context("principal.key_id is not a valid header value")?,
+            );
+            tracing::info!(
+                account_id = %p.account_id,
+                key_id = %p.key_id,
+                "bench authenticates: measurements reflect identified-caller capacity (#288)"
+            );
+            builder = builder.default_headers(headers);
+        } else {
+            tracing::warn!(
+                "bench has no [bench.principal]: requests are anonymous, so since #262 \
+                 they measure the anonymous-yield policy rather than serving capacity (#288)"
+            );
+        }
+        let http = builder.build().context("building HTTP client")?;
         Ok(TargetClient { http })
     }
 
@@ -177,6 +217,7 @@ impl TargetClient {
             // entry supplies the override; it would matter if this were
             // ever used to load a model the host has no config for.
             sampling: None,
+            preserve_thinking: None,
         })
     }
 
@@ -241,6 +282,7 @@ impl TargetClient {
                         cost: None,
                         tool_call: false,
                         reasoning: false,
+                        preserve_thinking: None,
                         // A generic OpenAI endpoint says nothing about
                         // serviceability, and absent means "no opinion".
                         servable: None,

crates/helexa-bench/src/config.rs

@@ -67,6 +67,44 @@ pub struct BenchSettings {
     /// SQLite system-of-record path.
     #[serde(default = "default_db_path")]
     pub db_path: String,
+    /// Identity bench presents to a neuron (#288).
+    ///
+    /// Without this bench is *anonymous*, and since #262 anonymous
+    /// callers are served only from capacity left over once identified
+    /// traffic is satisfied — they are capped below `max_in_flight` and
+    /// park at the class gate ahead of every other gate. Measured on
+    /// beast, that moved `concurrency:8` from 168.9 tok/s / 0.53 s
+    /// ttft_p95 to 104.6 / 11.77 s the day #262 landed, and held there.
+    ///
+    /// So an unauthenticated bench does not measure serving capacity; it
+    /// measures the yield policy. Real traffic arrives through cortex
+    /// with a principal and is subject to none of it.
+    ///
+    /// Deliberately not an admission *exemption*: bench should be
+    /// subject to the same rules as any caller, or it measures something
+    /// no user experiences — the same class of error in the other
+    /// direction. Give it a principal and let it queue like everyone
+    /// else.
+    #[serde(default)]
+    pub principal: Option<PrincipalSettings>,
+}
+
+/// The account/key pair bench stamps on inference requests (#288).
+///
+/// These are the headers cortex asserts after a bearer resolves (#49).
+/// Bench talks to neurons **directly** over the WireGuard mesh, so it
+/// stamps them itself — the same trust model the link already relies on,
+/// since a neuron accepts them from cortex on exactly that basis.
+///
+/// Give bench its *own* pair rather than borrowing a real caller's: it
+/// then holds its own fair-share allocation (#54) and cannot starve
+/// interactive traffic, which is the failure #262 was fixing.
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct PrincipalSettings {
+    /// Stamped as `x-helexa-account-id`.
+    pub account_id: String,
+    /// Stamped as `x-helexa-key-id`.
+    pub key_id: String,
 }
 
 impl Default for BenchSettings {
@@ -77,6 +115,7 @@ impl Default for BenchSettings {
             iteration_pause_secs: default_iter_pause(),
             request_timeout_secs: default_timeout(),
             db_path: default_db_path(),
+            principal: None,
         }
     }
 }
@@ -110,6 +149,18 @@ pub struct ScenarioConfig {
     /// deliberately, e.g. `concurrency_levels = [2, 4, 8]`.
     #[serde(default)]
     pub concurrency_levels: Vec<u32>,
+    /// Concurrency levels to run **non-streaming** (#285). Separate from
+    /// `concurrency_levels` because the two shapes are not
+    /// interchangeable: streaming multiplexes through the batch engine,
+    /// non-streaming did not until #285 and serialized completely — a
+    /// measured 1.00x aggregate throughput at every level against 3.98x
+    /// streamed.
+    ///
+    /// Defaults to the top level only. The signal is starkest there and
+    /// each extra level costs a full burst per sample per build; an
+    /// operator who wants the whole curve can list it.
+    #[serde(default = "default_concurrency_nonstreaming_levels")]
+    pub concurrency_nonstreaming_levels: Vec<u32>,
     /// Square image sizes (px per side) — one `image:<px>` scenario per
     /// entry (#203), run only against models advertising the `image`
     /// capability. Defaults to `[1024]`.
@@ -148,6 +199,7 @@ impl Default for ScenarioConfig {
             prompt_sizes: default_prompt_sizes(),
             max_tokens: default_max_tokens(),
             concurrency_levels: Vec::new(),
+            concurrency_nonstreaming_levels: Vec::new(),
             concurrency_prompt_tokens: default_concurrency_prompt_tokens(),
             image_sizes: default_image_sizes(),
             capability_probes: Vec::new(),
@@ -215,6 +267,10 @@ fn default_timeout() -> u64 {
 fn default_db_path() -> String {
     "/var/lib/helexa-bench/bench.sqlite".to_string()
 }
+fn default_concurrency_nonstreaming_levels() -> Vec<u32> {
+    vec![8]
+}
+
 fn default_api_enabled() -> bool {
     true
 }
@@ -243,6 +299,44 @@ fn default_capability_max_tokens() -> u64 {
 // figment's, not ours, so suppress the lint here.
 #[allow(clippy::result_large_err)]
 mod tests {
+
+    /// #288: a config with no `[bench.principal]` must still parse — the
+    /// anonymous behaviour is the historical one and correct for
+    /// `openai` targets — but a configured one must round-trip, because
+    /// a silently-dropped principal reverts the measurement to the
+    /// anonymous-yield path without changing a visible number.
+    #[test]
+    fn principal_is_optional_and_round_trips() {
+        let parse = |t: &str| -> BenchConfig {
+            figment::Figment::new()
+                .merge(figment::providers::Toml::string(t))
+                .extract()
+                .expect("parse")
+        };
+        let none = parse("[bench]\nsamples_per_version = 3\n");
+        assert!(none.bench.principal.is_none());
+
+        let some = parse(
+            "[bench]\nsamples_per_version = 3\n\n\
+             [bench.principal]\naccount_id = \"acct\"\nkey_id = \"k1\"\n",
+        );
+        let p = some.bench.principal.expect("principal");
+        assert_eq!((p.account_id.as_str(), p.key_id.as_str()), ("acct", "k1"));
+    }
+
+    /// The non-streaming burst defaults on (#285). If it defaulted off,
+    /// the path that serialized completely would stay unmeasured on every
+    /// fleet that had not opted in — which is how it went unnoticed.
+    #[test]
+    fn nonstreaming_concurrency_defaults_to_the_top_level() {
+        let cfg: BenchConfig = figment::Figment::new()
+            .merge(figment::providers::Toml::string(
+                "[scenarios]\nprompt_sizes = [128]\n",
+            ))
+            .extract()
+            .expect("parse");
+        assert_eq!(cfg.scenarios.concurrency_nonstreaming_levels, vec![8]);
+    }
     use super::*;
     use figment::Jail;
 

crates/helexa-bench/src/lib.rs

@@ -7,6 +7,7 @@
 pub mod api;
 pub mod client;
 pub mod config;
+pub mod regime;
 pub mod report;
 pub mod scenario;
 pub mod store;

crates/helexa-bench/src/regime.rs

@@ -0,0 +1,148 @@
+//! Measurement-regime boundaries — builds where a number changed
+//! meaning rather than value.
+//!
+//! A trend line silently spans them, and every reader re-derives the
+//! same thing: that a cliff was the instrument, not the engine. Three
+//! have already cost real investigation time:
+//!
+//! - beast's `concurrency:8` fell 168.9 → 104.6 tok/s with `ttft_p95`
+//!   rising 0.53 → 11.77 s, and it read as a 38% throughput regression
+//!   until the one commit in the window turned out to be an admission
+//!   *policy* change that bench, being anonymous, was subject to (#288).
+//! - quadbrat's `capability:debug-reason` TTFT fell 88.8 → 0.1 s, which
+//!   read as a latency win and was a redefinition of when the clock
+//!   stops.
+//! - the same redefinition put decode rates at ~250 tok/s on a 3060
+//!   beforehand, by dividing reasoning-inclusive token counts by a
+//!   visible-content-only window.
+//!
+//! So the boundaries are declared, not remembered. Declared here rather
+//! than in the frontend because `report` renders the same series into
+//! `benchmarks.md` and needs the same caveats.
+//!
+//! ## What belongs here
+//!
+//! Only changes to **what a metric means**. A change that made the
+//! server genuinely faster, slower, or differently behaved does *not*
+//! belong — that is the signal these lines exist to keep readable. #280
+//! (fetch the model's own chat template) moved quadbrat's
+//! `completion_tokens` from 4096 to 1053 by ending a truncation loop;
+//! that is a real behavioural change and stays unmarked.
+//!
+//! The identified/anonymous split is deliberately absent too: it is
+//! recorded per-row in `runs.principal`, so the UI derives it from the
+//! data instead of trusting a constant to stay true.
+
+use serde::Serialize;
+
+/// A build at which one or more metrics changed meaning.
+#[derive(Debug, Clone, Serialize)]
+pub struct MeasurementRegime {
+    /// Short SHA of the first build measured under the new semantics.
+    /// Matched against `SeriesPoint::git_sha`, so a boundary whose build
+    /// never appears in a selection simply does not draw.
+    pub first_sha: &'static str,
+    /// Terse label for the chart rule.
+    pub label: &'static str,
+    /// What changed, and why the step is not a performance result.
+    pub detail: &'static str,
+    /// Series keys the change affects, as the API/UI name them. A
+    /// boundary only draws on panels carrying one of these — marking
+    /// every panel would train people to ignore the rules.
+    pub affects: &'static [&'static str],
+}
+
+/// The declared boundaries, oldest first.
+pub const REGIMES: &[MeasurementRegime] = &[
+    MeasurementRegime {
+        first_sha: "92dddc0",
+        label: "#262 anonymous callers yield",
+        detail: "Anonymous traffic became servable only from capacity left over \
+                 once identified traffic is satisfied — capped below max_in_flight \
+                 and parked at the class gate. An anonymous bench therefore \
+                 characterises the yield policy rather than serving capacity. \
+                 Steps here are admission policy, not the engine (#288).",
+        affects: &["decode", "ttft", "ttftP95", "queueWait", "rejected"],
+    },
+    MeasurementRegime {
+        first_sha: "0fe7aa3",
+        label: "#117 reasoning counts as liveness",
+        detail: "TTFT previously waited for the first *visible content* chunk, so \
+                 on a thinking model it measured the whole reasoning span (88.8 s \
+                 on quadbrat). It now stops at the first delta of any kind. The \
+                 same change fixed decode rates computed by dividing \
+                 reasoning-inclusive token counts by a content-only window, which \
+                 had produced ~250 tok/s on a 3060.",
+        affects: &["decode", "ttft"],
+    },
+];
+
+/// Boundaries affecting `metric`, in declaration order.
+pub fn regimes_for(metric: &str) -> Vec<&'static MeasurementRegime> {
+    REGIMES
+        .iter()
+        .filter(|r| r.affects.contains(&metric))
+        .collect()
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    /// A boundary that names no metric would draw nowhere and mislead by
+    /// omission; one naming an unknown key would draw nowhere and look
+    /// fine. Both are typos this catches.
+    #[test]
+    fn every_regime_affects_a_known_series_key() {
+        // The keys `Trends.tsx` maps `SeriesPoint` onto.
+        const KNOWN: &[&str] = &[
+            "decode",
+            "ttft",
+            "ttftP95",
+            "queueWait",
+            "rejected",
+            "prefillTps",
+            "reasoning",
+            "cached",
+            "completion",
+            "tpot",
+        ];
+        for r in REGIMES {
+            assert!(
+                !r.affects.is_empty(),
+                "{} affects nothing, so it would never draw",
+                r.first_sha
+            );
+            for key in r.affects {
+                assert!(
+                    KNOWN.contains(key),
+                    "{} names unknown series key {key:?}",
+                    r.first_sha
+                );
+            }
+        }
+    }
+
+    #[test]
+    fn regimes_for_selects_by_metric() {
+        assert!(regimes_for("ttft").iter().any(|r| r.first_sha == "0fe7aa3"));
+        assert!(
+            regimes_for("completion").is_empty(),
+            "#280 changed behaviour, not meaning — it must not be marked"
+        );
+    }
+
+    /// Short SHAs are matched against `SeriesPoint::git_sha`, which the
+    /// store records at the width `/version` reports.
+    #[test]
+    fn declared_shas_are_short_form() {
+        for r in REGIMES {
+            assert_eq!(
+                r.first_sha.len(),
+                7,
+                "{} is not a 7-char short sha; it would never match a series point",
+                r.first_sha
+            );
+        }
+    }
+}

crates/helexa-bench/src/scenario.rs

@@ -9,6 +9,7 @@
 use crate::config::ScenarioConfig;
 use anyhow::{Context, Result, anyhow};
 use async_trait::async_trait;
+use cortex_core::entitlements::HEADER_KEY_ID;
 use cortex_core::harness::ModelInfo;
 use cortex_core::openai::ChatCompletionChunk;
 use eventsource_stream::Eventsource;
@@ -46,6 +47,17 @@ pub struct RunCtx<'a> {
     pub model_id: String,
     pub max_tokens: u64,
     pub timeout: Duration,
+    /// Base `x-helexa-key-id` bench presents (#288), when configured.
+    ///
+    /// The client already sends this as a default header, so every
+    /// scenario is identified without having to remember. A concurrency
+    /// burst overrides it per stream — see
+    /// [`ConcurrencyScenario`] — because fair-share (#54) bounds one
+    /// principal to `max_per_principal` in-flight-or-queued requests,
+    /// which defaults to 2. Eight streams under one identity would be
+    /// capped at two and the rest rejected, measuring the fair-share cap
+    /// instead of the server.
+    pub principal_key_id: Option<String>,
 }
 
 /// Operator-felt metrics for a single measured request.
@@ -72,6 +84,34 @@ pub struct ScenarioMetrics {
     pub decode_ms: Option<u64>,
     /// Tokens submitted to prefill — the denominator for prefill tok/s.
     pub prefill_tokens: Option<u64>,
+    /// Tokens spent inside the reasoning span, from
+    /// `usage.completion_tokens_details.reasoning_tokens` (#223). A
+    /// sub-count of `completion_tokens`.
+    ///
+    /// Worth a trend of its own: on a reasoning model this is the
+    /// dominant cost driver and it moves independently of speed. A
+    /// template change, a sampling change (#283) or a reasoning-budget
+    /// change can double what the model thinks before answering while
+    /// every tok/s number stays flat — the bill doubles and no chart
+    /// moves.
+    pub reasoning_tokens: Option<u64>,
+    /// p95 inter-token arrival gap in milliseconds — the tail of the
+    /// stream's smoothness, client-observed.
+    ///
+    /// `decode_tps` is a mean over the whole decode window, so a stream
+    /// that stalls for a second and then catches up is indistinguishable
+    /// from one that never stalled. This is the number a user actually
+    /// feels, and the one a batching stall or a mid-stream rebatch shows
+    /// up in. `None` for non-streaming, which has no inter-token gaps by
+    /// construction, and for streams too short to have a tail.
+    pub tpot_p95_ms: Option<f64>,
+    /// Prompt tokens served from neuron's prefix KV cache (#269), from
+    /// `usage.prompt_tokens_details.cached_tokens`.
+    ///
+    /// The reason prefill timing varies between otherwise identical
+    /// samples. #269 exists because the saving was invisible and every
+    /// client reported a 0% hit rate; bench then threw the number away.
+    pub cached_tokens: Option<u64>,
     // ── Concurrency / agentic-load fields (#89) ──────────────────────────
     // Set only by the concurrency scenario, which fans out N simultaneous
     // streams to characterize the real a0/hermes/opencode workload that
@@ -203,6 +243,9 @@ impl Scenario for ImageLatencyScenario {
                 .as_u64()
                 .map(|d| d + timing["decode_ms"].as_u64().unwrap_or(0)),
             prefill_tokens: None,
+            reasoning_tokens: None,
+            cached_tokens: None,
+            tpot_p95_ms: None,
             concurrency: None,
             ttft_p95_s: None,
             queue_wait_ms_median: None,
@@ -233,6 +276,18 @@ pub fn build_scenarios(cfg: &ScenarioConfig) -> Vec<Box<dyn Scenario>> {
             id: format!("concurrency:{n}"),
             concurrency: n,
             approx_prompt_tokens: cfg.concurrency_prompt_tokens,
+            streaming: true,
+        }) as Box<dyn Scenario>);
+    }
+    // Non-streaming bursts (#285). A separate cell id so the two shapes
+    // never average together — they measured 1.00x against 3.98x before
+    // #285, and a combined number would have hidden both.
+    for &n in &cfg.concurrency_nonstreaming_levels {
+        scenarios.push(Box::new(ConcurrencyScenario {
+            id: format!("concurrency:{n}:nostream"),
+            concurrency: n,
+            approx_prompt_tokens: cfg.concurrency_prompt_tokens,
+            streaming: false,
         }) as Box<dyn Scenario>);
     }
     for &side in &cfg.image_sizes {
@@ -257,7 +312,7 @@ pub fn build_scenarios(cfg: &ScenarioConfig) -> Vec<Box<dyn Scenario>> {
 pub async fn cold_probe(ctx: &RunCtx<'_>) -> Result<ScenarioMetrics> {
     let prompt = build_prompt(128);
     let payload = chat_payload(ctx, &prompt);
-    tokio::time::timeout(ctx.timeout, stream_and_measure(ctx, &payload))
+    tokio::time::timeout(ctx.timeout, stream_and_measure(ctx, &payload, None))
         .await
         .map_err(|_| anyhow!("cold probe timed out after {:?}", ctx.timeout))?
 }
@@ -295,7 +350,7 @@ impl Scenario for ChatLatencyScenario {
     async fn run(&self, ctx: &RunCtx) -> Result<ScenarioMetrics> {
         let prompt = build_prompt(self.approx_prompt_tokens);
         let payload = chat_payload(ctx, &prompt);
-        let fut = stream_and_measure(ctx, &payload);
+        let fut = stream_and_measure(ctx, &payload, None);
         tokio::time::timeout(ctx.timeout, fut)
             .await
             .map_err(|_| anyhow!("request timed out after {:?}", ctx.timeout))?
@@ -313,6 +368,16 @@ pub struct ConcurrencyScenario {
     id: String,
     concurrency: u32,
     approx_prompt_tokens: u32,
+    /// Whether the burst streams (#285).
+    ///
+    /// Both shapes matter and they are not interchangeable. Streaming
+    /// multiplexes through the batch engine; non-streaming did not until
+    /// #285, and serialized completely — 1.00x aggregate throughput at
+    /// every concurrency level against 3.98x streamed. A bench that only
+    /// streams cannot see that, which is why it went unmeasured: the
+    /// neuron reports `in_flight: 8`, admission accepts, `/health` looks
+    /// healthy, and throughput is single-stream.
+    streaming: bool,
 }
 
 #[async_trait]
@@ -327,13 +392,26 @@ impl Scenario for ConcurrencyScenario {
 
     async fn run(&self, ctx: &RunCtx) -> Result<ScenarioMetrics> {
         let prompt = build_prompt(self.approx_prompt_tokens);
-        let payload = chat_payload(ctx, &prompt);
+        let payload = if self.streaming {
+            chat_payload(ctx, &prompt)
+        } else {
+            nonstreaming_payload(ctx, &prompt)
+        };
 
-        // Fire all streams at once; each is independently timed and capped by
-        // the per-request timeout so one hung stream can't stall the burst.
+        // Fire all requests at once; each is independently timed and capped
+        // by the per-request timeout so one hung request can't stall the
+        // burst.
         let burst_start = Instant::now();
-        let futs = (0..self.concurrency).map(|_| async {
-            tokio::time::timeout(ctx.timeout, stream_and_measure(ctx, &payload)).await
+        let streaming = self.streaming;
+        let payload = &payload;
+        let futs = (0..self.concurrency).map(move |i| async move {
+            // Each stream presents its own identity — see
+            // `with_stream_identity`.
+            if streaming {
+                tokio::time::timeout(ctx.timeout, stream_and_measure(ctx, payload, Some(i))).await
+            } else {
+                tokio::time::timeout(ctx.timeout, request_and_measure(ctx, payload, Some(i))).await
+            }
         });
         let results = futures::future::join_all(futs).await;
         let burst_window = burst_start.elapsed().as_secs_f64();
@@ -351,7 +429,7 @@ impl Scenario for ConcurrencyScenario {
         }
         if streams.is_empty() {
             return Err(anyhow!(
-                "all {} concurrent streams failed ({rejected} shed by admission)",
+                "all {} concurrent requests failed ({rejected} shed by admission)",
                 self.concurrency
             ));
         }
@@ -380,9 +458,50 @@ impl Scenario for ConcurrencyScenario {
             total_s: burst_window,
             prompt_tokens: streams.iter().find_map(|m| m.prompt_tokens),
             completion_tokens: total_tokens,
-            prefill_ms: None,
-            decode_ms: None,
-            prefill_tokens: None,
+            // Median server-side phase timing across the burst. These
+            // were `None`, which left the prefill-tok/s panel blank for
+            // exactly the cells where queue behaviour matters most, and
+            // made `queue_wait_ms` (TTFT minus server prefill, #85)
+            // uncomputable for a burst — the one number that separates
+            // "the server is slow" from "you were queued".
+            //
+            // Median rather than sum: every stream prefills its own copy
+            // of the same prompt, so the representative per-stream cost
+            // is the meaningful figure and it composes with `ttft_s`,
+            // which is also a median.
+            prefill_ms: median(
+                &streams
+                    .iter()
+                    .filter_map(|m| m.prefill_ms.map(|v| v as f64))
+                    .collect::<Vec<_>>(),
+            )
+            .map(|v| v as u64),
+            decode_ms: median(
+                &streams
+                    .iter()
+                    .filter_map(|m| m.decode_ms.map(|v| v as f64))
+                    .collect::<Vec<_>>(),
+            )
+            .map(|v| v as u64),
+            prefill_tokens: median(
+                &streams
+                    .iter()
+                    .filter_map(|m| m.prefill_tokens.map(|v| v as f64))
+                    .collect::<Vec<_>>(),
+            )
+            .map(|v| v as u64),
+            // Summed across the burst, like `completion_tokens`: the cell
+            // describes the whole burst, so a per-stream figure here
+            // would not compose with it.
+            reasoning_tokens: sum_opt(&streams, |m| m.reasoning_tokens),
+            cached_tokens: sum_opt(&streams, |m| m.cached_tokens),
+            // The worst stream in the burst, not a median of medians:
+            // under load the question is how bad it got for somebody,
+            // and averaging tails is how a stall stays invisible.
+            tpot_p95_ms: streams
+                .iter()
+                .filter_map(|m| m.tpot_p95_ms)
+                .max_by(|a, b| a.total_cmp(b)),
             concurrency: Some(self.concurrency),
             ttft_p95_s: percentile(&ttfts, 95.0),
             queue_wait_ms_median: median(&queue_waits),
@@ -393,6 +512,146 @@ impl Scenario for ConcurrencyScenario {
     }
 }
 
+/// Sum an optional per-stream count across a burst, `None` when no
+/// stream reported it — so "nobody measured it" stays distinguishable
+/// from "measured as zero", the same distinction #269 exists to
+/// preserve.
+fn sum_opt(
+    streams: &[ScenarioMetrics],
+    f: impl Fn(&ScenarioMetrics) -> Option<u64>,
+) -> Option<u64> {
+    let vals: Vec<u64> = streams.iter().filter_map(f).collect();
+    (!vals.is_empty()).then(|| vals.iter().sum())
+}
+
+/// Per-stream identity for a concurrency burst (#288/#54).
+///
+/// Fair-share bounds *one principal* to `max_per_principal`
+/// in-flight-or-queued requests, defaulting to 2. A burst of 8 under a
+/// single identity would therefore have 6 rejected with `PrincipalCap`,
+/// and the scenario would measure the fair-share cap rather than the
+/// server.
+///
+/// Giving each stream its own key is also the more faithful model: a
+/// concurrency-8 burst is meant to stand in for eight simultaneous
+/// *callers*, not one caller issuing eight requests. Fair-share is still
+/// exercised — each identity is subject to it — just not self-inflicted.
+fn with_stream_identity(
+    rb: reqwest::RequestBuilder,
+    ctx: &RunCtx<'_>,
+    stream: Option<u32>,
+) -> reqwest::RequestBuilder {
+    match (ctx.principal_key_id.as_deref(), stream) {
+        (Some(base), Some(i)) => rb.header(HEADER_KEY_ID, format!("{base}-{i}")),
+        // No principal configured, or a single-request scenario: the
+        // client's default headers already carry the base identity.
+        _ => rb,
+    }
+}
+
+/// The non-streaming counterpart to [`chat_payload`] (#285).
+///
+/// `stream_options` is deliberately absent: usage is part of the body on
+/// this shape, and sending the streaming-only option to a strict server
+/// is a needless compatibility risk.
+fn nonstreaming_payload(ctx: &RunCtx, prompt: &str) -> serde_json::Value {
+    json!({
+        "model": ctx.model_id,
+        "messages": [{"role": "user", "content": prompt}],
+        "max_tokens": ctx.max_tokens,
+        "temperature": 0,
+        "stream": false,
+    })
+}
+
+/// Time one non-streaming chat completion (#285).
+///
+/// There is no first-chunk moment on this shape — the whole body arrives
+/// at once — so `ttft_s` is the full request wall-clock rather than a
+/// separate measurement. That is the honest reading: for a non-streaming
+/// caller, time-to-first-token *is* time-to-everything, and it is exactly
+/// what makes the serialization this scenario exists to detect so
+/// expensive.
+///
+/// `decode_tps` is per-request; the burst aggregate is computed by the
+/// caller over the whole window, the same way the streaming path does it.
+async fn request_and_measure(
+    ctx: &RunCtx<'_>,
+    payload: &serde_json::Value,
+    stream: Option<u32>,
+) -> Result<ScenarioMetrics> {
+    let start = Instant::now();
+    let resp = with_stream_identity(ctx.client.post(&ctx.chat_url), ctx, stream)
+        .json(payload)
+        .send()
+        .await
+        .context("sending non-streaming chat request")?;
+    if !resp.status().is_success() {
+        let status = resp.status();
+        let body = resp.text().await.unwrap_or_default();
+        return Err(anyhow!("upstream returned {status}: {}", body.trim()));
+    }
+    let v: serde_json::Value = resp
+        .json()
+        .await
+        .context("non-streaming chat response was not JSON")?;
+    let total_s = start.elapsed().as_secs_f64();
+
+    let usage = v.get("usage");
+    let completion_tokens = usage
+        .and_then(|u| u.get("completion_tokens"))
+        .and_then(|t| t.as_u64())
+        .unwrap_or(0);
+    let prompt_tokens = usage
+        .and_then(|u| u.get("prompt_tokens"))
+        .and_then(|t| t.as_u64());
+    let reasoning_tokens = usage
+        .and_then(|u| u.get("completion_tokens_details"))
+        .and_then(|d| d.get("reasoning_tokens"))
+        .and_then(|t| t.as_u64());
+    let cached_tokens = usage
+        .and_then(|u| u.get("prompt_tokens_details"))
+        .and_then(|d| d.get("cached_tokens"))
+        .and_then(|t| t.as_u64());
+    let timing = usage.and_then(|u| u.get("helexa_timing"));
+    let field = |name: &str| timing.and_then(|t| t.get(name)).and_then(|x| x.as_u64());
+    let (prefill_ms, decode_ms, prefill_tokens) = (
+        field("prefill_ms"),
+        field("decode_ms"),
+        field("prefill_tokens"),
+    );
+
+    // Prefer the server's own decode window when it reports one (#85);
+    // it excludes prefill, so it is comparable with the streaming path's
+    // decode-window rate rather than being diluted by it.
+    let decode_tps = match decode_ms {
+        Some(ms) if ms > 200 => Some(completion_tokens as f64 / (ms as f64 / 1000.0)),
+        _ if total_s > 0.2 => Some(completion_tokens as f64 / total_s),
+        _ => None,
+    };
+
+    Ok(ScenarioMetrics {
+        ttft_s: total_s,
+        decode_tps,
+        total_s,
+        prompt_tokens,
+        completion_tokens,
+        prefill_ms,
+        decode_ms,
+        prefill_tokens,
+        reasoning_tokens,
+        cached_tokens,
+        // Non-streaming has no inter-token gaps: the body arrives whole.
+        tpot_p95_ms: None,
+        concurrency: None,
+        ttft_p95_s: None,
+        queue_wait_ms_median: None,
+        rejected: None,
+        artifact: None,
+        image_units: None,
+    })
+}
+
 /// Quality probe (#91): runs a fixed prompt and stores the full generated
 /// text as an artifact for later scoring (manual now, LLM-judge later). The
 /// point is to compare reasoning/planning quality across models — the axis
@@ -425,7 +684,7 @@ impl Scenario for CapabilityScenario {
             "stream": true,
             "stream_options": {"include_usage": true},
         });
-        let fut = stream_and_measure_inner(ctx, &payload, true);
+        let fut = stream_and_measure_inner(ctx, &payload, true, None);
         tokio::time::timeout(ctx.timeout, fut)
             .await
             .map_err(|_| anyhow!("capability probe timed out after {:?}", ctx.timeout))?
@@ -468,8 +727,9 @@ fn percentile(values: &[f64], p: f64) -> Option<f64> {
 async fn stream_and_measure(
     ctx: &RunCtx<'_>,
     payload: &serde_json::Value,
+    stream: Option<u32>,
 ) -> Result<ScenarioMetrics> {
-    stream_and_measure_inner(ctx, payload, false).await
+    stream_and_measure_inner(ctx, payload, false, stream).await
 }
 
 /// As [`stream_and_measure`] but accumulates the full visible text when
@@ -479,11 +739,10 @@ async fn stream_and_measure_inner(
     ctx: &RunCtx<'_>,
     payload: &serde_json::Value,
     capture_text: bool,
+    stream_ix: Option<u32>,
 ) -> Result<ScenarioMetrics> {
     let start = Instant::now();
-    let resp = ctx
-        .client
-        .post(&ctx.chat_url)
+    let resp = with_stream_identity(ctx.client.post(&ctx.chat_url), ctx, stream_ix)
         .json(payload)
         .send()
         .await
@@ -503,6 +762,10 @@ async fn stream_and_measure_inner(
     let mut prefill_ms: Option<u64> = None;
     let mut decode_ms: Option<u64> = None;
     let mut prefill_tokens: Option<u64> = None;
+    let mut reasoning_tokens: Option<u64> = None;
+    let mut cached_tokens: Option<u64> = None;
+    // Inter-token arrival gaps, for the p95 tail.
+    let mut gaps_ms: Vec<f64> = Vec::new();
     let mut captured = String::new();
 
     while let Some(event) = stream.next().await {
@@ -540,6 +803,11 @@ async fn stream_and_measure_inner(
             if content.is_some() || reasoning.is_some() {
                 if first.is_none() {
                     first = Some(now);
+                } else if let Some(prev) = last {
+                    // Gaps only *between* generated deltas: the wait for
+                    // the first one is TTFT and belongs to prefill, not
+                    // to stream smoothness.
+                    gaps_ms.push(now.duration_since(prev).as_secs_f64() * 1000.0);
                 }
                 last = Some(now);
                 chunk_count += 1;
@@ -551,6 +819,14 @@ async fn stream_and_measure_inner(
         if let Some(usage) = chunk.usage {
             prompt_tokens = Some(usage.prompt_tokens);
             completion_tokens = Some(usage.completion_tokens);
+            reasoning_tokens = usage
+                .completion_tokens_details
+                .as_ref()
+                .map(|d| d.reasoning_tokens);
+            cached_tokens = usage
+                .prompt_tokens_details
+                .as_ref()
+                .map(|d| d.cached_tokens);
             if let Some(t) = usage.helexa_timing {
                 prefill_ms = Some(t.prefill_ms);
                 decode_ms = Some(t.decode_ms);
@@ -589,6 +865,9 @@ async fn stream_and_measure_inner(
         prefill_ms,
         decode_ms,
         prefill_tokens,
+        reasoning_tokens,
+        cached_tokens,
+        tpot_p95_ms: percentile(&gaps_ms, 95.0),
         // Concurrency fields unset on the single-request path; the
         // concurrency scenario builds its own aggregate (#89).
         concurrency: None,
@@ -604,6 +883,113 @@ async fn stream_and_measure_inner(
 mod tests {
     use super::*;
 
+    /// A burst must report server-side phase timing, or `queue_wait_ms`
+    /// (TTFT minus server prefill, #85) cannot be computed for the very
+    /// cells it was designed for, and the prefill-tok/s panel stays
+    /// blank under concurrency.
+    #[test]
+    fn a_burst_reports_median_phase_timing_not_none() {
+        let s = |prefill_ms, decode_ms, prefill_tokens| ScenarioMetrics {
+            ttft_s: 0.1,
+            decode_tps: Some(10.0),
+            total_s: 1.0,
+            prompt_tokens: Some(100),
+            completion_tokens: 10,
+            prefill_ms,
+            decode_ms,
+            prefill_tokens,
+            reasoning_tokens: None,
+            cached_tokens: None,
+            tpot_p95_ms: Some(5.0),
+            concurrency: None,
+            ttft_p95_s: None,
+            queue_wait_ms_median: None,
+            rejected: None,
+            artifact: None,
+            image_units: None,
+        };
+        let streams = [
+            s(Some(10), Some(100), Some(512)),
+            s(Some(20), Some(200), Some(512)),
+            s(Some(30), Some(300), Some(512)),
+        ];
+        let med = |f: fn(&ScenarioMetrics) -> Option<u64>| {
+            median(
+                &streams
+                    .iter()
+                    .filter_map(|m| f(m).map(|v| v as f64))
+                    .collect::<Vec<_>>(),
+            )
+            .map(|v| v as u64)
+        };
+        assert_eq!(med(|m| m.prefill_ms), Some(20));
+        assert_eq!(med(|m| m.decode_ms), Some(200));
+        // Every stream prefills the same prompt, so the median is the
+        // prompt size rather than a sum across the burst.
+        assert_eq!(med(|m| m.prefill_tokens), Some(512));
+    }
+
+    /// #288/#54: a burst must present one identity *per stream*.
+    ///
+    /// Fair-share bounds a single principal to `max_per_principal`
+    /// in-flight-or-queued requests, which defaults to 2. Eight streams
+    /// under one key would have six rejected with `PrincipalCap`, and
+    /// `concurrency:8` would silently measure the fair-share cap instead
+    /// of the server — a worse failure than the anonymity it replaced,
+    /// because it looks like backpressure.
+    #[test]
+    fn a_burst_presents_one_identity_per_stream() {
+        let client = reqwest::Client::new();
+        let ctx = RunCtx {
+            client: &client,
+            chat_url: "http://x/v1/chat/completions".into(),
+            model_id: "m".into(),
+            max_tokens: 16,
+            timeout: Duration::from_secs(1),
+            principal_key_id: Some("fleet-benchmark".into()),
+        };
+        let key_of = |stream: Option<u32>| -> Option<String> {
+            with_stream_identity(client.post(&ctx.chat_url), &ctx, stream)
+                .build()
+                .expect("build")
+                .headers()
+                .get(HEADER_KEY_ID)
+                .map(|v| v.to_str().expect("utf8").to_string())
+        };
+        let keys: Vec<Option<String>> = (0..3).map(|i| key_of(Some(i))).collect();
+        assert_eq!(
+            keys,
+            vec![
+                Some("fleet-benchmark-0".to_string()),
+                Some("fleet-benchmark-1".to_string()),
+                Some("fleet-benchmark-2".to_string()),
+            ]
+        );
+        // Distinct, or the cap bites anyway.
+        let uniq: std::collections::HashSet<_> = keys.iter().collect();
+        assert_eq!(uniq.len(), 3);
+    }
+
+    /// A single-request scenario adds no override: the client's default
+    /// headers already carry the base identity, and a per-request copy
+    /// would be a second place for it to drift.
+    #[test]
+    fn a_single_request_scenario_keeps_the_client_identity() {
+        let client = reqwest::Client::new();
+        let ctx = RunCtx {
+            client: &client,
+            chat_url: "http://x/v1/chat/completions".into(),
+            model_id: "m".into(),
+            max_tokens: 16,
+            timeout: Duration::from_secs(1),
+            principal_key_id: Some("fleet-benchmark".into()),
+        };
+        let req = with_stream_identity(client.post(&ctx.chat_url), &ctx, None)
+            .build()
+            .expect("build");
+        assert!(req.headers().get(HEADER_KEY_ID).is_none());
+    }
+
     #[test]
     fn prompt_grows_with_token_target() {
         let small = build_prompt(128);
@@ -646,6 +1032,7 @@ mod tests {
             prompt_sizes: vec![128],
             max_tokens: 64,
             concurrency_levels: vec![2, 8],
+            concurrency_nonstreaming_levels: vec![8],
             concurrency_prompt_tokens: 512,
             capability_probes: vec![CapabilityProbe {
                 name: "plan".into(),

crates/helexa-bench/src/store.rs

@@ -14,6 +14,22 @@ use std::path::Path;
 #[derive(Debug, Clone)]
 pub struct RunRecord {
     pub ts: String, // RFC3339
+    /// Identity this sample was taken under (#288), `account/key`, or
+    /// `None` for an anonymous run. Anonymous samples measure the
+    /// admission yield policy rather than serving capacity once #262 is
+    /// in the build, so this is the column that says whether two rows
+    /// are comparable at all.
+    pub principal: Option<String>,
+    /// Tokens spent reasoning (#223) — the dominant cost driver on a
+    /// reasoning model, and one that moves independently of every tok/s
+    /// number on the dashboard.
+    pub reasoning_tokens: Option<u64>,
+    /// Prompt tokens served from the prefix cache (#269) — the reason
+    /// prefill timing varies between otherwise identical samples.
+    pub cached_tokens: Option<u64>,
+    /// p95 inter-token arrival gap (ms) — stream smoothness, which a
+    /// mean tok/s cannot express.
+    pub tpot_p95_ms: Option<f64>,
     // target
     pub target_name: String,
     pub target_kind: String,
@@ -204,6 +220,18 @@ impl Store {
                 ("artifact", "TEXT"),
                 ("quality_score", "REAL"),
                 ("scorer", "TEXT"),
+                // #288: which identity the sample was taken under. Rows
+                // predating this are NULL, which is honest — they were
+                // anonymous, and since #262 that measures a different
+                // thing. Without the column the 2026-08-18 discontinuity
+                // is folklore rather than data.
+                ("principal", "TEXT"),
+                // #223 / #269: both are on the wire and were being discarded.
+                ("reasoning_tokens", "INTEGER"),
+                ("cached_tokens", "INTEGER"),
+                // #285: inter-token tail. `decode_tps` is a mean and hides
+                // a stall that later catches up.
+                ("tpot_p95_ms", "REAL"),
             ],
         )?;
         Ok(())
@@ -264,6 +292,7 @@ impl Store {
                 swap_unload_ms, swap_load_ms,
                 artifact, quality_score, scorer,
                 image_units,
+                principal, reasoning_tokens, cached_tokens, tpot_p95_ms,
                 ok, error
             ) VALUES (
                 ?1, ?2, ?3, ?4,
@@ -280,7 +309,8 @@ impl Store {
                 ?42, ?43,
                 ?44, ?45, ?46,
                 ?47,
-                ?48, ?49
+                ?48, ?49, ?50, ?51,
+                ?52, ?53
             )",
             params![
                 r.ts,
@@ -330,6 +360,10 @@ impl Store {
                 r.quality_score,
                 r.scorer,
                 r.image_units,
+                r.principal,
+                r.reasoning_tokens,
+                r.cached_tokens,
+                r.tpot_p95_ms,
                 r.ok as i64,
                 r.error,
             ],
@@ -703,7 +737,15 @@ impl Store {
             }
         };
         let mut stmt = self.conn.prepare(
-            "SELECT git_sha, build_timestamp, package_version, ttft_s, decode_tps, total_s, ts
+            "SELECT git_sha, build_timestamp, package_version, ttft_s, decode_tps, total_s,
+                    ttft_p95_s, queue_wait_ms, rejected,
+                    -- Prefill rate derived here rather than stored: the
+                    -- inputs are already recorded and a stored rate
+                    -- would be a third thing to keep consistent.
+                    CASE WHEN prefill_ms > 0 AND prefill_tokens IS NOT NULL
+                         THEN (prefill_tokens * 1000.0) / prefill_ms END AS prefill_tps,
+                    reasoning_tokens, cached_tokens, completion_tokens,
+                    tpot_p95_ms, principal, ts
              FROM runs
              WHERE ok=1 AND target_name=?1 AND model_id=?2 AND scenario_id=?3
              ORDER BY id",
@@ -717,7 +759,16 @@ impl Store {
                     ttft_s: r.get(3)?,
                     decode_tps: r.get(4)?,
                     total_s: r.get(5)?,
-                    ts: r.get(6)?,
+                    ttft_p95_s: r.get(6)?,
+                    queue_wait_ms: r.get(7)?,
+                    rejected: r.get(8)?,
+                    prefill_tps: r.get(9)?,
+                    reasoning_tokens: r.get(10)?,
+                    cached_tokens: r.get(11)?,
+                    completion_tokens: r.get(12)?,
+                    tpot_p95_ms: r.get(13)?,
+                    principal: r.get(14)?,
+                    ts: r.get(15)?,
                 })
             })?
             .collect::<rusqlite::Result<_>>()?;
@@ -838,6 +889,33 @@ pub struct SeriesPoint {
     pub ttft_s_median: Option<f64>,
     pub decode_tps_median: Option<f64>,
     pub total_s_median: Option<f64>,
+    // ── Added because the two above cannot show what changed ──────
+    /// Tail TTFT. The median above stayed flat at ~0.3 s across the
+    /// build where p95 went 0.53 s → 11.77 s (#288); charting only the
+    /// median is why that went unnoticed for a week.
+    pub ttft_p95_s_median: Option<f64>,
+    /// Admission queue wait — separates "the server is slow" from
+    /// "you were queued behind someone".
+    pub queue_wait_ms_median: Option<f64>,
+    /// Requests shed by admission during the burst.
+    pub rejected_median: Option<f64>,
+    /// Prefill throughput, derived from `prefill_tokens / prefill_ms`.
+    /// The other half of serving speed; only decode was ever charted.
+    pub prefill_tps_median: Option<f64>,
+    /// Tokens spent reasoning (#223) — the dominant cost driver on a
+    /// reasoning model, and independent of every speed metric here.
+    pub reasoning_tokens_median: Option<f64>,
+    /// Prompt tokens served from the prefix cache (#269).
+    pub cached_tokens_median: Option<f64>,
+    /// Total generated tokens — catches a model that starts truncating
+    /// or rambling, which no rate metric shows.
+    pub completion_tokens_median: Option<f64>,
+    /// p95 inter-token gap (ms) — stream smoothness.
+    pub tpot_p95_ms_median: Option<f64>,
+    /// Identity the samples were taken under (#288), or `None` if
+    /// anonymous. Anonymous and identified points are **not
+    /// comparable** once #262 is in the build.
+    pub principal: Option<String>,
     pub samples: usize,
 }
 
@@ -848,6 +926,15 @@ struct SeriesRaw {
     ttft_s: Option<f64>,
     decode_tps: Option<f64>,
     total_s: Option<f64>,
+    ttft_p95_s: Option<f64>,
+    queue_wait_ms: Option<f64>,
+    rejected: Option<f64>,
+    prefill_tps: Option<f64>,
+    reasoning_tokens: Option<f64>,
+    cached_tokens: Option<f64>,
+    completion_tokens: Option<f64>,
+    tpot_p95_ms: Option<f64>,
+    principal: Option<String>,
     ts: String,
 }
 
@@ -880,6 +967,19 @@ fn aggregate_series(raws: Vec<SeriesRaw>) -> Vec<SeriesPoint> {
                 ttft_s_median: median(rows.iter().filter_map(|r| r.ttft_s)),
                 decode_tps_median: median(rows.iter().filter_map(|r| r.decode_tps)),
                 total_s_median: median(rows.iter().filter_map(|r| r.total_s)),
+                ttft_p95_s_median: median(rows.iter().filter_map(|r| r.ttft_p95_s)),
+                queue_wait_ms_median: median(rows.iter().filter_map(|r| r.queue_wait_ms)),
+                rejected_median: median(rows.iter().filter_map(|r| r.rejected)),
+                prefill_tps_median: median(rows.iter().filter_map(|r| r.prefill_tps)),
+                reasoning_tokens_median: median(rows.iter().filter_map(|r| r.reasoning_tokens)),
+                cached_tokens_median: median(rows.iter().filter_map(|r| r.cached_tokens)),
+                completion_tokens_median: median(rows.iter().filter_map(|r| r.completion_tokens)),
+                tpot_p95_ms_median: median(rows.iter().filter_map(|r| r.tpot_p95_ms)),
+                // Any row's principal: a build's samples are all taken
+                // under one identity, and a mixture would mean the
+                // config changed mid-build, which the chart should show
+                // as a discontinuity rather than average away.
+                principal: rows.iter().find_map(|r| r.principal.clone()),
                 samples: rows.len(),
             };
             (sort_key, point)
@@ -1281,6 +1381,10 @@ mod tests {
     fn rec(target: &str, sha: &str, model: &str, scenario: &str, ok: bool) -> RunRecord {
         RunRecord {
             ts: "2026-06-13T00:00:00Z".into(),
+            principal: None,
+            reasoning_tokens: None,
+            cached_tokens: None,
+            tpot_p95_ms: None,
             target_name: target.into(),
             target_kind: "neuron".into(),
             endpoint: "http://x:13131".into(),

crates/helexa-bench/src/sweep.rs

@@ -78,7 +78,10 @@ pub struct Sweeper {
 
 impl Sweeper {
     pub fn new(cfg: BenchConfig) -> Result<Self> {
-        let client = TargetClient::new(cfg.bench.request_timeout())?;
+        let client = TargetClient::with_principal(
+            cfg.bench.request_timeout(),
+            cfg.bench.principal.as_ref(),
+        )?;
         let store = Store::open(&cfg.bench.db_path)?;
         Ok(Sweeper { cfg, client, store })
     }
@@ -169,6 +172,7 @@ impl Sweeper {
             model_id: model.id.clone(),
             max_tokens: self.cfg.scenarios.max_tokens,
             timeout: self.cfg.bench.request_timeout(),
+            principal_key_id: self.cfg.bench.principal.as_ref().map(|p| p.key_id.clone()),
         };
         let cold = crate::scenario::cold_probe(&ctx).await;
         let swap = SwapTiming { unload_ms, load_ms };
@@ -252,6 +256,7 @@ impl Sweeper {
                     model_id: model.id.clone(),
                     max_tokens: self.cfg.scenarios.max_tokens,
                     timeout: self.cfg.bench.request_timeout(),
+                    principal_key_id: self.cfg.bench.principal.as_ref().map(|p| p.key_id.clone()),
                 };
 
                 // One unmeasured warmup when the cell is empty (matches
@@ -346,6 +351,15 @@ impl Sweeper {
 
         RunRecord {
             ts: chrono::Utc::now().to_rfc3339(),
+            // #288: stamp the identity this sample was taken under, so a
+            // later reader can tell an anonymous row from an identified
+            // one instead of inferring it from the date.
+            principal: self
+                .cfg
+                .bench
+                .principal
+                .as_ref()
+                .map(|p| format!("{}/{}", p.account_id, p.key_id)),
             target_name: target.name.clone(),
             target_kind: kind_str(target.kind).to_string(),
             endpoint: target.endpoint.clone(),
@@ -383,6 +397,9 @@ impl Sweeper {
             prefill_ms: m.and_then(|m| m.prefill_ms),
             decode_ms: m.and_then(|m| m.decode_ms),
             prefill_tokens: m.and_then(|m| m.prefill_tokens),
+            reasoning_tokens: m.and_then(|x| x.reasoning_tokens),
+            cached_tokens: m.and_then(|x| x.cached_tokens),
+            tpot_p95_ms: m.and_then(|x| x.tpot_p95_ms),
             vram_used_mb: health.map(|h| h.vram_used_mb),
             gpu_util_pct: health.map(|h| h.gpu_util_pct),
             gpu_temp_c: health.map(|h| h.gpu_temp_c),

crates/helexa-bench/tests/api.rs

@@ -15,6 +15,10 @@ fn rec(
     ok: bool,
 ) -> RunRecord {
     RunRecord {
+        principal: None,
+        reasoning_tokens: None,
+        cached_tokens: None,
+        tpot_p95_ms: None,
         ts: "2026-06-13T00:00:00Z".into(),
         target_name: host.into(),
         target_kind: "neuron".into(),

crates/helexa-bench/tests/sweep_integration.rs

@@ -80,6 +80,7 @@ async fn spawn_mock(sha: &str) -> (String, Arc<Mutex<String>>) {
 fn config_for(endpoint: String, db_path: String) -> BenchConfig {
     BenchConfig {
         bench: BenchSettings {
+            principal: None,
             sweep_interval_secs: 1,
             samples_per_version: 2,
             iteration_pause_secs: 0,
@@ -91,6 +92,7 @@ fn config_for(endpoint: String, db_path: String) -> BenchConfig {
             prompt_sizes: vec![128], // single scenario keeps assertions simple
             max_tokens: 16,
             concurrency_levels: Vec::new(),
+            concurrency_nonstreaming_levels: vec![],
             concurrency_prompt_tokens: 512,
             capability_probes: Vec::new(),
         },

crates/helexa-tools/src/config.rs

@@ -58,6 +58,11 @@ mod tests {
         assert!(cfg.max_redirects > 0);
     }
 
+    // The closure's error type is `figment::Error`, which is 208 bytes and
+    // not ours to shrink; `Jail::expect_with` dictates the signature. The
+    // usual remedy (box the Err) is not available through a third-party
+    // callback contract, and this is a test closure run once.
+    #[allow(clippy::result_large_err)]
     #[test]
     fn loads_toml_with_env_override() {
         figment::Jail::expect_with(|jail| {

crates/helexa-upstream/src/config.rs

@@ -287,15 +287,44 @@ fn default_from_addr() -> String {
 impl UpstreamConfig {
     /// Load from a TOML file with `UPSTREAM_`-prefixed env overrides
     /// (`__` nesting separator).
+    ///
+    /// A sibling `secrets.toml`, if present, merges after the main file
+    /// and wins — see [`secrets_path`].
     pub fn load(path: impl AsRef<Path>) -> Result<Self, Box<figment::Error>> {
+        let path = path.as_ref();
         Figment::new()
             .merge(Toml::file(path))
+            .merge(Toml::file(secrets_path(path)))
             .merge(Env::prefixed("UPSTREAM_").split("__"))
             .extract()
             .map_err(Box::new)
     }
 }
 
+/// `secrets.toml` beside the main config.
+///
+/// The main file holds structure and is deployed by CI; this one holds
+/// credentials, is written only by the operator, and is never read,
+/// written or diffed by the pipeline. Merging it last means the
+/// operator's value wins, so a deploy cannot swap a live credential for
+/// a placeholder.
+///
+/// The split exists because config CI cannot write is config no check
+/// can defend — `helexa-router.toml` was kept out of git for sitting
+/// beside secret-bearing files, and its stale alias then took a node
+/// down (2026-08-27). The answer is to separate secrets from structure,
+/// not to hand the pipeline the credentials.
+///
+/// A missing file is not an error: figment treats an absent
+/// `Toml::file` as an empty source, so existing single-file deployments
+/// keep working untouched.
+fn secrets_path(config: &Path) -> std::path::PathBuf {
+    config
+        .parent()
+        .unwrap_or_else(|| Path::new("."))
+        .join("secrets.toml")
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;

crates/neuron/src/api.rs

@@ -2,6 +2,7 @@
 
 use crate::activation::ActivationTracker;
 use crate::harness::HarnessRegistry;
+use crate::harness::candle::WireSurface;
 use crate::harness::candle::{CandleHarness, InferenceError};
 use crate::harness::preflight::PreflightError;
 use crate::health::HealthCache;
@@ -362,7 +363,7 @@ async fn chat_completions(
 
     if req.stream.unwrap_or(false) {
         match candle
-            .chat_completion_stream_with(req, chat_config, principal)
+            .chat_completion_stream_with(req, chat_config, principal, WireSurface::Chat)
             .await
         {
             Ok(rx) => {
@@ -381,7 +382,10 @@ async fn chat_completions(
             Err(e) => inference_error_response(e),
         }
     } else {
-        match candle.chat_completion(req, principal).await {
+        match candle
+            .chat_completion(req, principal, WireSurface::Chat)
+            .await
+        {
             Ok(resp) => Json(resp).into_response(),
             Err(e) => inference_error_response(e),
         }
@@ -434,7 +438,13 @@ async fn responses(
 
     if stream_requested {
         match candle
-            .responses_stream(chat_req, response_id, message_item_id, principal)
+            .responses_stream(
+                chat_req,
+                response_id,
+                message_item_id,
+                principal,
+                WireSurface::Responses,
+            )
             .await
         {
             Ok(rx) => {
@@ -458,7 +468,10 @@ async fn responses(
         // and translate the result. We don't currently re-tokenise
         // to compute usage; the harness returns it via the chat
         // response and we pass it through.
-        match candle.chat_completion(chat_req, principal).await {
+        match candle
+            .chat_completion(chat_req, principal, WireSurface::Responses)
+            .await
+        {
             Ok(chat_resp) => {
                 // Extract the assistant text (chat completions
                 // always emits one choice on the candle path).
@@ -486,9 +499,9 @@ async fn responses(
                     input_tokens: u.prompt_tokens,
                     output_tokens: u.completion_tokens,
                     total_tokens: u.prompt_tokens + u.completion_tokens,
-                    // Carry the reasoning sub-count through from the chat
-                    // usage — the non-streaming path now splits off the
-                    // `<think>` span and counts it (see `split_off_reasoning`).
+                    // Carry the reasoning sub-count through from the
+                    // chat usage: the collector counts the `<think>`
+                    // span and returns its text separately.
                     output_tokens_details: u.completion_tokens_details.as_ref().map(|d| {
                         OutputTokensDetails {
                             reasoning_tokens: d.reasoning_tokens,
@@ -502,8 +515,18 @@ async fn responses(
                     model_id,
                     message_item_id: mint_message_item_id(),
                 };
-                let _ = chat_resp; // make the borrow-checker happy if `text` consumed it
-                let resp = openai_responses::build_response(&meta, text, finish, usage);
+                // The model's think block, which the chat collector now
+                // returns on `message.reasoning_content` (#300). Without
+                // this the caller is billed for `reasoning_tokens` it
+                // never sees, and cannot replay the model's own
+                // reasoning on the next turn.
+                let reasoning = chat_resp
+                    .choices
+                    .first()
+                    .and_then(|c| c.message.extra.get("reasoning_content"))
+                    .and_then(|v| v.as_str())
+                    .map(str::to_string);
+                let resp = openai_responses::build_response(&meta, text, finish, usage, reasoning);
                 Json(resp).into_response()
             }
             Err(e) => inference_error_response(e),

crates/neuron/src/harness/admission.rs

@@ -1051,7 +1051,7 @@ mod tests {
 
         // Fill the single queue slot with a waiter that parks on the semaphore.
         let ctrl2 = Arc::clone(&ctrl);
-        let waiter = tokio::spawn(async move { ctrl2.enter(None).await.map(|p| drop(p)) });
+        let waiter = tokio::spawn(async move { ctrl2.enter(None).await.map(drop) });
         // Give the waiter a moment to occupy the queue slot.
         tokio::time::sleep(Duration::from_millis(50)).await;
         assert_eq!(ctrl.queue_depth(), 1);

crates/neuron/src/harness/arch/qwen3_5/linear_attn.rs

@@ -380,7 +380,23 @@ impl GatedDeltaNet {
             self.head_v_dim,
         )?;
         // Stash the updated recurrent state for the next call.
-        self.state.recurrent_state = Some(new_state.to_dtype(dtype)?);
+        //
+        // The delta rule ran in f32 and `state_init` upcasts back to f32
+        // on entry, so storing at the model dtype is a round trip that
+        // buys only memory (#284). Decode takes it once per token.
+        //
+        // Storing f32 keeps `new_state` as-is, which on the CUDA decode
+        // path is the same buffer the kernel updated in place — so the
+        // stored state aliases it, where the `to_dtype` branch always
+        // allocated afresh. That is sound: the pre-update value is dead
+        // by this point, and prefix snapshots deep-copy
+        // (`snapshot_state`) precisely so they never share storage with
+        // a buffer a later forward will overwrite.
+        self.state.recurrent_state = Some(if gdn_state_f32() {
+            new_state
+        } else {
+            new_state.to_dtype(dtype)?
+        });
 
         // core_attn_out: (B, H, L, D_v) → (B, L, H, D_v) → (B*L*H, D_v).
         let core_attn_out = core_attn_out.transpose(1, 2)?.contiguous()?; // (B, L, H, D_v)
@@ -464,6 +480,33 @@ fn chunked_prefill_enabled() -> bool {
     })
 }
 
+/// `NEURON_GDN_STATE_F32=1` keeps the recurrent state in f32 between
+/// forwards instead of storing it at the model's activation dtype (#284).
+///
+/// The delta rule computes in f32 throughout, so storing the result at
+/// BF16 and upcasting it again next call is a pure round-trip loss.
+/// Measured on the chunked path: bit-exact carrying f32 at every chunk
+/// size, against 1.3e-3 relative error after 16 prefill chunks and
+/// **1.0e-2 after 4096 decode steps** — decode round-trips once per
+/// token, across 48 of this arch's 64 layers.
+///
+/// Off by default on purpose. Every #283 measurement was taken against
+/// the bf16 baseline, and flipping this silently would move the ground
+/// underneath the sampling A/B and the decode-rate work (#282, #285)
+/// at the same time. Turn it on deliberately, measure, then decide the
+/// default.
+///
+/// Costs ~2× the recurrent slab: +37.7 MB per GPU per sequence on the
+/// deployed 27B under TP=2, so +302 MB/GPU at `max_in_flight = 8`.
+fn gdn_state_f32() -> bool {
+    static ENABLED: std::sync::OnceLock<bool> = std::sync::OnceLock::new();
+    *ENABLED.get_or_init(|| {
+        std::env::var("NEURON_GDN_STATE_F32")
+            .map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
+            .unwrap_or(false)
+    })
+}
+
 /// Chunk-parallel gated delta rule — a faithful port of the HF
 /// reference `torch_chunk_gated_delta_rule` (chunk_size = 64) in
 /// `transformers/models/qwen3_5/modeling_qwen3_5.py`, minus the steps
@@ -1097,6 +1140,94 @@ mod tests {
         assert!(d_state < 2e-4, "final state diverged: {d_state}");
     }
 
+    /// #284: the state carried *between* forwards must not lose
+    /// precision. Every other parity test here is single-call and
+    /// all-f32, so none of them crosses the boundary where
+    /// `GatedDeltaNet::forward` stores the state at the model's
+    /// activation dtype — which is why the defect lived.
+    ///
+    /// Two properties, one test:
+    ///
+    /// 1. carrying f32, an N-chunk run equals a single call **exactly**,
+    ///    so the chunk-boundary math itself is sound (this kills the
+    ///    "chunked prefill corrupts state" hypothesis, #283 H1);
+    /// 2. carrying the model dtype loses a measurable, chunk-count-
+    ///    dependent amount — the defect the `NEURON_GDN_STATE_F32` flag
+    ///    exists to switch off.
+    ///
+    /// Asserting (2) is deliberate: if a future change makes the bf16
+    /// carry lossless this test fails loudly and the flag can go, rather
+    /// than the flag quietly outliving its reason.
+    #[test]
+    fn state_carried_across_chunks_is_exact_in_f32_and_lossy_in_bf16() {
+        let (b, h, dk, dv) = (1, 2, 32, 32);
+        let dev = Device::Cpu;
+        let l = 1024;
+        // Long-memory decay: g near zero keeps the state alive across
+        // every boundary, which is both the realistic regime for
+        // long-context retrieval and the one that accumulates most.
+        let scale = 1.0 / (dk as f64).sqrt();
+        let q = Tensor::randn(0f32, 1.0, (b, h, l, dk), &dev).unwrap();
+        let q = (l2norm(&q, 1e-6).unwrap() * scale).unwrap();
+        let k = Tensor::randn(0f32, 1.0, (b, h, l, dk), &dev).unwrap();
+        let k = l2norm(&k, 1e-6).unwrap();
+        let v = (Tensor::randn(0f32, 1.0, (b, h, l, dv), &dev).unwrap() * 0.5).unwrap();
+        let g = (Tensor::rand(0f32, 1f32, (b, h, l), &dev).unwrap() * -0.02).unwrap();
+        let beta = Tensor::rand(0.05f32, 0.95f32, (b, h, l), &dev).unwrap();
+        let zeros = || Tensor::zeros((b, h, dk, dv), DType::F32, &dev).unwrap();
+
+        // Ground truth: the whole sequence in one call, f32 throughout.
+        let (out_ref, _) = run_chunk_gated_delta_rule(&q, &k, &v, &g, &beta, zeros()).unwrap();
+
+        // Replay it in `chunk`-sized pieces, carrying the state the way
+        // `forward` does.
+        let replay = |chunk: usize, carry: Option<DType>| {
+            let mut state = zeros();
+            let mut outs = Vec::new();
+            let mut pos = 0;
+            while pos < l {
+                let n = chunk.min(l - pos);
+                let take = |t: &Tensor| t.narrow(2, pos, n).unwrap().contiguous().unwrap();
+                let (o, s) = run_chunk_gated_delta_rule(
+                    &take(&q),
+                    &take(&k),
+                    &take(&v),
+                    &take(&g),
+                    &take(&beta),
+                    state,
+                )
+                .unwrap();
+                state = match carry {
+                    Some(dt) => s.to_dtype(dt).unwrap().to_dtype(DType::F32).unwrap(),
+                    None => s,
+                };
+                outs.push(o);
+                pos += n;
+            }
+            Tensor::cat(&outs, 2).unwrap()
+        };
+
+        for chunk in [128usize, 256, 512] {
+            let exact = max_abs_diff(&out_ref, &replay(chunk, None));
+            assert_eq!(
+                exact, 0.0,
+                "chunk={chunk}: an f32 state carry must reproduce the \
+                 single-call result bit for bit, got {exact:e}"
+            );
+        }
+
+        // And the dtype round trip is what costs us. Compared on the
+        // final chunk, where the accumulated state has done the most work.
+        let tail = |t: &Tensor| t.narrow(2, l - 128, 128).unwrap().contiguous().unwrap();
+        let lossy = max_abs_diff(&tail(&out_ref), &tail(&replay(128, Some(DType::BF16))));
+        assert!(
+            lossy > 1e-5,
+            "storing the state at BF16 between chunks is expected to lose \
+             precision (#284); if this now passes losslessly the flag can \
+             be retired, got {lossy:e}"
+        );
+    }
+
     /// Adversarially correlated inputs: near-identical keys with
     /// beta ≈ 1 and negligible decay make the UT-transform matrix T
     /// maximally coherent — raw powers of T grow combinatorially

crates/neuron/src/harness/arch/qwen3_5/snapshot.rs

@@ -562,9 +562,9 @@ mod tests {
                 .forward_batch_decode(&input, &positions, mask.as_ref())
                 .expect("batched step");
             assert_eq!(h.dims()[0], 3);
-            for row in 0..3 {
+            for (row, exp) in expected.iter().enumerate() {
                 let got: Vec<f32> = h.i((row, 0, ..)).unwrap().to_vec1().unwrap();
-                let diff = max_abs_diff(&expected[row][t], &got);
+                let diff = max_abs_diff(&exp[t], &got);
                 assert!(diff < 1e-4, "row {row} step {t} diverged: {diff}");
             }
         }
@@ -610,9 +610,9 @@ mod tests {
         let h = model
             .forward_batch_decode(&input, &[3, 3], None)
             .expect("step");
-        for row in 0..2 {
+        for (row, exp) in expected.iter().enumerate() {
             let got: Vec<f32> = h.i((row, 0, ..)).unwrap().to_vec1().unwrap();
-            let diff = max_abs_diff(&expected[row], &got);
+            let diff = max_abs_diff(exp, &got);
             assert!(diff < 1e-4, "row {row} diverged: {diff}");
         }
     }
@@ -674,9 +674,9 @@ mod tests {
             let h = model
                 .forward_batch_decode(&input, &positions, mask.as_ref())
                 .expect("batched step");
-            for row in 0..2 {
+            for (row, exp) in expected.iter().enumerate() {
                 let got: Vec<f32> = h.i((row, 0, ..)).unwrap().to_vec1().unwrap();
-                let diff = max_abs_diff(&expected[row][t], &got);
+                let diff = max_abs_diff(&exp[t], &got);
                 assert!(diff < 1e-4, "batched row {row} step {t}: {diff}");
             }
         }

crates/neuron/src/harness/chat_template.rs

@@ -438,6 +438,139 @@ mod tests {
         assert!(out.contains("<|vision_start|>") && out.contains("<|vision_end|>"));
     }
 
+    /// A file read back through a tool result must reach the model
+    /// byte-for-byte.
+    ///
+    /// The reported symptom: the model writes a file correctly, reads it
+    /// back, and sees errors that are not in the file — consistently,
+    /// after every write. Anything that mangles `<`, `&`, `"` or
+    /// newlines on this path would produce exactly that, and would make
+    /// the model burn its budget reconciling a file against a corrupted
+    /// view of itself.
+    #[test]
+    fn a_file_read_back_through_a_tool_result_is_byte_exact() {
+        let template = include_str!("../../tests/data/qwen3.8-27b.chat_template.jinja");
+        // The characters most likely to be mangled, in the shapes a real
+        // web project contains them.
+        let file = "<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n  <meta charset=\"utf-8\">\n</head>\n<body>\n  <script>\n    if (a && b || c < d) { g.fillStyle = '#b8905a'; }\n    const s = `rgba(${r},${g},0.5)`;\n\n    // a comment & an ampersand\n  </script>\n</body>\n</html>\n";
+
+        let mut tool_extras = serde_json::Map::new();
+        tool_extras.insert("tool_call_id".into(), json!("call_0"));
+        let mut assistant_extras = serde_json::Map::new();
+        assistant_extras.insert(
+            "tool_calls".into(),
+            json!([{"id":"call_0","type":"function",
+                    "function":{"name":"read","arguments":"{\"path\":\"/tmp/index.html\"}"}}]),
+        );
+
+        let messages = vec![
+            user_msg("read the file back"),
+            ChatMessage {
+                role: "assistant".into(),
+                content: MessageContent::Null,
+                extra: Value::Object(assistant_extras),
+            },
+            ChatMessage {
+                role: "tool".into(),
+                content: MessageContent::Text(file.to_string()),
+                extra: Value::Object(tool_extras),
+            },
+            user_msg("is it correct?"),
+        ];
+        let out =
+            render_chat_template(template, &messages, &Value::Null, &Value::Null).expect("render");
+
+        for probe in [
+            "<!DOCTYPE html>",
+            "<html lang=\"en\">",
+            "if (a && b || c < d)",
+            "g.fillStyle = '#b8905a';",
+            "`rgba(${r},${g},0.5)`",
+            "// a comment & an ampersand",
+        ] {
+            assert!(
+                out.contains(probe),
+                "the model would not see {probe:?} verbatim — rendered prompt mangles it"
+            );
+        }
+        // And nothing HTML-escaped anywhere.
+        for bad in ["&lt;", "&gt;", "&amp;", "&quot;", "&#x27;", "&#39;"] {
+            assert!(
+                !out.contains(bad),
+                "found {bad:?} in the rendered prompt: content is being HTML-escaped"
+            );
+        }
+    }
+
+    /// Blank lines inside a replayed tool-call argument must survive
+    /// rendering intact.
+    ///
+    /// A model that writes a source file and then re-reads it from its
+    /// own replayed tool call is comparing what it remembers writing
+    /// against what the prompt now shows. Any whitespace the pipeline
+    /// eats makes those disagree, and the model has no way to tell a
+    /// rendering artifact from a mistake of its own — which is a
+    /// plausible route into the announce-then-stop stall (#296).
+    ///
+    /// The JSON string→object normalisation above is the step that
+    /// could plausibly lose them, since it round-trips the arguments
+    /// through a parser.
+    #[test]
+    fn a_replayed_tool_call_keeps_the_blank_lines_in_its_arguments() {
+        let template = include_str!("../../tests/data/qwen3.8-27b.chat_template.jinja");
+        // Blank lines between sections, exactly as a written source file
+        // carries them.
+        let file_body = "'use strict';\n\n// ---- constants ----\nconst A = 1;\n\n\nfunction f() {\n  return A;\n}\n";
+        let args = json!({ "path": "/tmp/game.js", "content": file_body });
+        let mut assistant_extras = serde_json::Map::new();
+        assistant_extras.insert(
+            "tool_calls".into(),
+            json!([{
+                "id": "call_0", "type": "function",
+                "function": {
+                    "name": "write",
+                    // The OpenAI wire form: arguments as a JSON *string*.
+                    "arguments": serde_json::to_string(&args).unwrap()
+                }
+            }]),
+        );
+        let mut tool_extras = serde_json::Map::new();
+        tool_extras.insert("tool_call_id".into(), json!("call_0"));
+
+        let messages = vec![
+            user_msg("write the file"),
+            ChatMessage {
+                role: "assistant".into(),
+                content: MessageContent::Null,
+                extra: Value::Object(assistant_extras),
+            },
+            ChatMessage {
+                role: "tool".into(),
+                content: MessageContent::Text("Successfully wrote 84 bytes".into()),
+                extra: Value::Object(tool_extras),
+            },
+            user_msg("now check it parses"),
+        ];
+        let out =
+            render_chat_template(template, &messages, &Value::Null, &Value::Null).expect("render");
+
+        assert!(
+            out.contains(file_body),
+            "the file body must appear verbatim in the rendered prompt"
+        );
+        let blanks_in = file_body.lines().filter(|l| l.is_empty()).count();
+        assert_eq!(blanks_in, 3, "fixture sanity");
+        // And specifically the double newline the model would notice.
+        assert!(
+            out.contains("'use strict';\n\n// ---- constants ----"),
+            "blank line after the prologue was eaten"
+        );
+        assert!(
+            out.contains("const A = 1;\n\n\nfunction f()"),
+            "consecutive blank lines were collapsed"
+        );
+    }
+
     /// The second turn of an OpenAI-native agentic loop: the client
     /// replays its own assistant turn, which carries `tool_calls` and
     /// `"content": null`. That must render as the tool-call block —

crates/neuron/src/harness/context_limit.rs

@@ -36,6 +36,28 @@ use crate::config::ContextLimitConfig;
 /// few turns.
 const RATE_EMA_ALPHA: f64 = 0.3;
 
+/// Smallest prompt whose prefill time says anything about throughput.
+///
+/// A prefill has a fixed per-request cost — tokenize, launch, sync —
+/// that dominates a short prompt completely. The deploy's own smoke
+/// probe prefills 19 tokens in ~130 ms and reads as **145 tok/s** on a
+/// host that measures ~1,400. Folding that in does not make the
+/// estimate slightly worse; it replaces it.
+///
+/// That mattered because the fallback it displaces is the whole
+/// cold-start guard: with no sample at all, `derive_limit` uses
+/// `bootstrap_prefill_tok_per_sec` (800, ≈96k context at the default
+/// 120 s target). One 19-token probe replaced it with 145 tok/s, and
+/// the host advertised **17,408** — a sixth of what it can serve —
+/// until real traffic happened to warm it (#295).
+///
+/// 512 excludes the smoke probe and the short auxiliary calls agent
+/// harnesses make (observed at 19, 143 and 469 tokens) while admitting
+/// ordinary prompts. At ~1,400 tok/s a 512-token prefill runs ~370 ms,
+/// so fixed overhead is roughly a tenth of it — informative rather than
+/// dominant.
+const MIN_PREFILL_SAMPLE_TOKENS: usize = 512;
+
 /// Fold one throughput sample (`tokens` in `elapsed`) into the EMA held in
 /// `bits`. No-op for degenerate inputs so a probe request or a clock blip
 /// can't poison the average.
@@ -99,7 +121,15 @@ impl ThroughputEma {
 
     /// Fold one prefill measurement (`prompt_tokens` processed in
     /// `elapsed`) into the prefill EMA.
+    ///
+    /// Samples below [`MIN_PREFILL_SAMPLE_TOKENS`] are ignored: they
+    /// measure fixed per-request cost rather than throughput, and the
+    /// value they displace — the bootstrap estimate — is better than
+    /// they are.
     pub fn record(&self, prompt_tokens: usize, elapsed: Duration) {
+        if prompt_tokens < MIN_PREFILL_SAMPLE_TOKENS {
+            return;
+        }
         fold_rate(&self.prefill_bits, prompt_tokens, elapsed);
     }
 
@@ -356,6 +386,60 @@ pub fn derive_limit(
 
 #[cfg(test)]
 mod tests {
+
+    /// The deploy's smoke probe must not set the advertised context.
+    ///
+    /// 19 tokens in 130 ms reads as ~145 tok/s on a host that measures
+    /// ~1,400. Before this guard that one sample displaced the
+    /// bootstrap estimate and the host advertised 17,408 instead of
+    /// ~96,000 — until real traffic happened along (#295).
+    #[test]
+    fn a_smoke_probe_does_not_poison_the_prefill_rate() {
+        let ema = ThroughputEma::new();
+        ema.record(19, Duration::from_millis(130));
+        assert_eq!(
+            ema.get(),
+            None,
+            "a 19-token prefill must leave the EMA unset so the bootstrap value stands"
+        );
+    }
+
+    /// The short auxiliary calls agent harnesses make are the same
+    /// shape as the probe and equally uninformative — observed at 143
+    /// and 469 tokens against prompts three orders of magnitude larger.
+    #[test]
+    fn short_auxiliary_prompts_are_ignored_too() {
+        let ema = ThroughputEma::new();
+        for tokens in [143usize, 469] {
+            ema.record(tokens, Duration::from_millis(200));
+        }
+        assert_eq!(ema.get(), None);
+    }
+
+    /// A real prompt still records, and still tracks a shift.
+    #[test]
+    fn representative_prefills_still_measure() {
+        let ema = ThroughputEma::new();
+        ema.record(1400, Duration::from_secs(1));
+        let first = ema.get().expect("a 1400-token sample must record");
+        assert!((first - 1400.0).abs() < 1.0, "got {first}");
+
+        // A slower host is tracked, not ignored.
+        for _ in 0..20 {
+            ema.record(700, Duration::from_secs(1));
+        }
+        let after = ema.get().unwrap();
+        assert!(after < first && after > 600.0, "got {after}");
+    }
+
+    /// Exactly at the threshold counts — the boundary is inclusive so
+    /// the constant means what it says.
+    #[test]
+    fn the_threshold_itself_is_a_valid_sample() {
+        let ema = ThroughputEma::new();
+        ema.record(MIN_PREFILL_SAMPLE_TOKENS, Duration::from_secs(1));
+        assert!(ema.get().is_some());
+    }
     use super::*;
 
     /// The ceiling a client is told to plan against must not be the

crates/neuron/src/harness/mod.rs

@@ -13,6 +13,7 @@ pub mod prefix_cache;
 pub mod preflight;
 pub mod preprocess;
 pub mod reasoning_budget;
+pub mod reasoning_effort;
 pub mod sampling;
 pub mod speculative;
 pub mod tp;

crates/neuron/src/harness/preflight.rs

@@ -402,6 +402,7 @@ mod tests {
             tensor_parallel: tp,
             devices: None,
             sampling: None,
+            preserve_thinking: None,
         }
     }
 

crates/neuron/src/harness/reasoning_budget.rs

@@ -42,6 +42,31 @@ pub struct ReasoningBudgetLadder {
     pub xhigh: Option<usize>,
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub max: Option<usize>,
+    /// Tokens of the caller's **own** output budget held back for the
+    /// answer, when the caller names no explicit `reasoning.max_tokens`.
+    ///
+    /// This is what an OpenAI-shaped client gets instead of the rungs
+    /// above, and it is deliberately not derived from effort. Effort is
+    /// a prompt instruction the model is trained to follow (#290);
+    /// making it *also* set a guillotine is what told Qwen3.8-27B to
+    /// reason at `xhigh` and then cut it off at `medium`'s 12,288.
+    ///
+    /// A reserve has no such double meaning. The model thinks as long
+    /// as it likes within the budget the caller declared, minus enough
+    /// to answer — so the caller always gets an answer, and the size of
+    /// the think block stays the model's decision rather than ours.
+    ///
+    /// Set to `0` to disable, restoring fully unbounded reasoning.
+    #[serde(default = "default_answer_reserve")]
+    pub answer_reserve_tokens: usize,
+}
+
+/// Enough for a substantive answer with tool calls, not so much that it
+/// meaningfully shortens a think block: at the 32,768 budget pi sends,
+/// this leaves ~28.7 k for reasoning — above the ~27.2 k a full agentic
+/// design turn was measured to use.
+fn default_answer_reserve() -> usize {
+    4_096
 }
 
 impl Default for ReasoningBudgetLadder {
@@ -58,6 +83,7 @@ impl Default for ReasoningBudgetLadder {
             high: 32_768,
             xhigh: None,
             max: None,
+            answer_reserve_tokens: default_answer_reserve(),
         }
     }
 }

crates/neuron/src/harness/reasoning_effort.rs

@@ -0,0 +1,302 @@
+//! Which reasoning-effort rungs a model actually supports, discovered
+//! from the model itself (#290).
+//!
+//! ## Why this is discovered rather than configured
+//!
+//! `Qwen/Qwen3.8-27B` accepts exactly three efforts and hard-fails on
+//! anything else:
+//!
+//! ```jinja
+//! {%- set resolved_reasoning_effort = reasoning_effort|default('xhigh') %}
+//! {%- if resolved_reasoning_effort not in ('xhigh', 'medium', 'low') %}
+//!     {{- raise_exception('Unexpected reasoning effort ...') }}
+//! ```
+//!
+//! We previously advertised an invented ladder — `minimal`, `low`,
+//! `medium`, `high` — of which two values the template rejects outright
+//! and which omits `xhigh`, the model's own default and strongest rung.
+//!
+//! A caution about *why* that mattered, because the obvious story is
+//! wrong. It is tempting to say clients read our advertisement and were
+//! misled by it. pi-ai does not: its `getSupportedThinkingLevels` reads
+//! `model.thinkingLevelMap`, a static field in the operator's
+//! `models.json`, and treats `xhigh`/`max` as unavailable unless that
+//! map names them explicitly. So a pi user saw exactly
+//! `minimal, low, medium, high` — the invented ladder, to the value —
+//! for a reason that had nothing to do with what we published.
+//!
+//! The coincidence is the trap: our wrong advertisement and pi's
+//! default produced the same symptom, which made the advertisement look
+//! causal. Publishing rungs the model actually accepts is still correct
+//! — it is the truth, other clients may read it, and it is what makes
+//! `apply_effort_kwarg` right — but do not assume a given client
+//! consumes it. For pi the operator must also set `thinkingLevelMap`,
+//! by hand, and nothing validates that they match.
+//!
+//! A hardcoded per-model table would drift the moment a model ships a
+//! new template, which is exactly the class of defect #280 fixed by
+//! fetching the template instead of assuming it. So the rungs come from
+//! the artifact.
+//!
+//! ## How the probe works
+//!
+//! `raise_exception` is already wired to a render error, so the template
+//! answers the question directly: render it once per candidate effort
+//! and see which ones survive.
+//!
+//! Two ambiguities the probe has to resolve, or it would report
+//! confident nonsense:
+//!
+//! 1. **A template that ignores `reasoning_effort` entirely** renders
+//!    successfully for every candidate. Supporting "all six" would be a
+//!    lie. So a candidate only counts when the rendered prompt actually
+//!    *differs* across efforts — if every render is identical the model
+//!    has no effort control and we advertise none.
+//! 2. **The default** is whichever supported candidate renders
+//!    identically to a render with no effort supplied. Qwen3.8 defaults
+//!    to `xhigh`; without detecting that, a caller who says nothing gets
+//!    the strongest rung while we believe it got nothing.
+
+use anyhow::Result;
+use cortex_core::openai::{ChatMessage, MessageContent};
+use serde_json::{Value, json};
+
+/// Effort names in ascending order of thinking, as clients spell them.
+///
+/// Deliberately a superset of what any one model supports: this is the
+/// *candidate* set the probe tries, and the conventional vocabulary
+/// OpenAI-compatible clients use (pi-ai's `EXTENDED_THINKING_LEVELS`
+/// minus `off`, which is not an effort but a separate switch handled by
+/// `enable_thinking`).
+pub const CANDIDATE_EFFORTS: &[&str] = &["minimal", "low", "medium", "high", "xhigh", "max"];
+
+/// What one model's template accepts.
+#[derive(Debug, Clone, Default, PartialEq, Eq)]
+pub struct SupportedEfforts {
+    /// Accepted efforts, ascending, as the template spells them.
+    pub levels: Vec<String>,
+    /// The effort applied when the caller names none — the template's
+    /// own `|default(...)`. `None` when it cannot be determined.
+    pub default: Option<String>,
+}
+
+impl SupportedEfforts {
+    pub fn is_empty(&self) -> bool {
+        self.levels.is_empty()
+    }
+
+    pub fn supports(&self, effort: &str) -> bool {
+        self.levels.iter().any(|l| l == effort)
+    }
+
+    /// Map a caller's effort onto the nearest rung this model has.
+    ///
+    /// Exact match wins. Otherwise walk **down** from the requested
+    /// level first, then up — a caller asking for more thinking than the
+    /// model offers should get the most it has, and a caller asking for
+    /// less should not be silently upgraded into a slower, costlier
+    /// response than they asked for.
+    ///
+    /// Descending-first is the opposite of pi-ai's `clampThinkingLevel`,
+    /// which searches upward first. That is deliberate: pi is choosing
+    /// among levels it believes exist, whereas we are the ones who know
+    /// what exists, and over-delivering thinking is the failure that
+    /// costs the caller money and latency.
+    ///
+    /// `None` when the model has no rungs at all.
+    pub fn nearest(&self, requested: &str) -> Option<&str> {
+        if self.levels.is_empty() {
+            return None;
+        }
+        if let Some(exact) = self.levels.iter().find(|l| *l == requested) {
+            return Some(exact.as_str());
+        }
+        let idx = CANDIDATE_EFFORTS.iter().position(|c| *c == requested)?;
+        for c in CANDIDATE_EFFORTS[..idx].iter().rev() {
+            if let Some(hit) = self.levels.iter().find(|l| l.as_str() == *c) {
+                return Some(hit.as_str());
+            }
+        }
+        for c in &CANDIDATE_EFFORTS[idx + 1..] {
+            if let Some(hit) = self.levels.iter().find(|l| l.as_str() == *c) {
+                return Some(hit.as_str());
+            }
+        }
+        None
+    }
+}
+
+/// Render a probe prompt at `effort`, or `None` if the template rejects
+/// it.
+fn probe_render(template: &str, effort: Option<&str>) -> Option<String> {
+    let messages = [ChatMessage {
+        role: "user".into(),
+        content: MessageContent::Text("probe".into()),
+        extra: Value::Null,
+    }];
+    let kwargs = match effort {
+        Some(e) => json!({ "reasoning_effort": e }),
+        None => Value::Null,
+    };
+    super::chat_template::render_chat_template(template, &messages, &Value::Null, &kwargs).ok()
+}
+
+/// Discover which efforts `template` accepts (#290).
+///
+/// Errors are not propagated: a template that cannot render even the
+/// baseline probe yields no rungs, and the caller behaves as it did
+/// before this existed.
+pub fn probe(template: &str, model_id: &str) -> Result<SupportedEfforts> {
+    let Some(baseline) = probe_render(template, None) else {
+        tracing::debug!(
+            model = %model_id,
+            "reasoning effort: template does not render a baseline probe; advertising no rungs"
+        );
+        return Ok(SupportedEfforts::default());
+    };
+
+    let mut rendered: Vec<(&str, String)> = Vec::new();
+    for candidate in CANDIDATE_EFFORTS {
+        if let Some(out) = probe_render(template, Some(candidate)) {
+            rendered.push((candidate, out));
+        }
+    }
+
+    // A template that never reads `reasoning_effort` renders the same
+    // bytes whatever we pass. Advertising every candidate in that case
+    // would be the invented-ladder mistake again, in a new place.
+    let effort_aware = rendered.windows(2).any(|w| w[0].1 != w[1].1)
+        || rendered.iter().any(|(_, out)| *out != baseline);
+    if !effort_aware {
+        tracing::debug!(
+            model = %model_id,
+            "reasoning effort: template ignores reasoning_effort; advertising no rungs"
+        );
+        return Ok(SupportedEfforts::default());
+    }
+
+    let default = rendered
+        .iter()
+        .find(|(_, out)| *out == baseline)
+        .map(|(name, _)| (*name).to_string());
+    let levels: Vec<String> = rendered
+        .iter()
+        .map(|(name, _)| (*name).to_string())
+        .collect();
+
+    tracing::info!(
+        model = %model_id,
+        levels = %levels.join(","),
+        default = ?default,
+        "reasoning effort: rungs discovered from the model's chat template (#290)"
+    );
+    Ok(SupportedEfforts { levels, default })
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    /// The probe against the **real** deployed Qwen3.8-27B template,
+    /// not a hand-written approximation of it. A synthetic template
+    /// proves the probe logic; only the shipped artifact proves the
+    /// answer, and the artifact is what #290 is about.
+    #[test]
+    fn the_real_qwen38_template_yields_low_medium_xhigh_defaulting_to_xhigh() {
+        let path = concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/data/qwen3.8-27b.chat_template.jinja"
+        );
+        let Ok(template) = std::fs::read_to_string(path) else {
+            // The fixture is checked in; if it is ever removed this test
+            // should fail loudly rather than silently pass.
+            panic!("missing fixture {path}");
+        };
+        let s = probe(&template, "Qwen/Qwen3.8-27B").expect("probe");
+        assert_eq!(
+            s.levels,
+            vec!["low", "medium", "xhigh"],
+            "the shipped template accepts exactly these"
+        );
+        assert_eq!(s.default.as_deref(), Some("xhigh"));
+        // The two rungs we used to advertise are rejected outright.
+        assert!(!s.supports("minimal"));
+        assert!(!s.supports("high"));
+    }
+
+    /// The shape Qwen3.8-27B actually ships: three rungs, `xhigh`
+    /// default, `raise_exception` on anything else.
+    const QWEN38: &str = r#"
+{%- set resolved = reasoning_effort|default('xhigh') %}
+{%- if resolved not in ('xhigh', 'medium', 'low') %}
+{{- raise_exception('Unexpected reasoning effort ' ~ resolved) }}
+{%- endif %}
+{%- if resolved == 'xhigh' %}EFFORT-XHIGH{% elif resolved == 'low' %}EFFORT-LOW{% else %}EFFORT-MEDIUM{% endif %}
+{%- for m in messages %}<|im_start|>{{ m.role }}
+{{ m.content }}<|im_end|>
+{%- endfor %}
+"#;
+
+    /// A template with no effort control at all.
+    const PLAIN: &str = r#"
+{%- for m in messages %}<|im_start|>{{ m.role }}
+{{ m.content }}<|im_end|>
+{%- endfor %}
+"#;
+
+    #[test]
+    fn discovers_exactly_the_rungs_the_template_accepts() {
+        let s = probe(QWEN38, "test").expect("probe");
+        assert_eq!(s.levels, vec!["low", "medium", "xhigh"]);
+        assert!(!s.supports("minimal"), "template raises on minimal");
+        assert!(!s.supports("high"), "template raises on high");
+    }
+
+    /// Without this, a caller who names no effort silently receives the
+    /// strongest rung while we believe it received nothing — which is
+    /// how the xhigh instruction ended up on every request in #290.
+    #[test]
+    fn detects_the_templates_own_default() {
+        let s = probe(QWEN38, "test").expect("probe");
+        assert_eq!(s.default.as_deref(), Some("xhigh"));
+    }
+
+    /// A template that ignores the kwarg must advertise nothing rather
+    /// than appearing to support all six.
+    #[test]
+    fn a_template_without_effort_control_advertises_no_rungs() {
+        let s = probe(PLAIN, "test").expect("probe");
+        assert!(s.is_empty(), "got {:?}", s.levels);
+        assert_eq!(s.nearest("medium"), None);
+    }
+
+    #[test]
+    fn an_exact_match_is_returned_unchanged() {
+        let s = probe(QWEN38, "test").expect("probe");
+        assert_eq!(s.nearest("medium"), Some("medium"));
+        assert_eq!(s.nearest("xhigh"), Some("xhigh"));
+    }
+
+    /// `high` does not exist on this model. Descending first means it
+    /// resolves to `medium`, not `xhigh`: a caller is never given more
+    /// thinking (slower, dearer) than it asked for.
+    #[test]
+    fn an_unsupported_level_resolves_downward_not_upward() {
+        let s = probe(QWEN38, "test").expect("probe");
+        assert_eq!(s.nearest("high"), Some("medium"));
+    }
+
+    /// `minimal` is below every rung the model has, so there is nothing
+    /// to descend to and the nearest is the weakest available.
+    #[test]
+    fn a_level_below_every_rung_resolves_up_to_the_weakest() {
+        let s = probe(QWEN38, "test").expect("probe");
+        assert_eq!(s.nearest("minimal"), Some("low"));
+    }
+
+    #[test]
+    fn an_unknown_level_name_resolves_to_nothing_rather_than_a_guess() {
+        let s = probe(QWEN38, "test").expect("probe");
+        assert_eq!(s.nearest("turbo"), None);
+    }
+}

crates/neuron/src/harness/sampling.rs

@@ -75,6 +75,15 @@ pub struct ModelGenerationDefaults {
     pub temperature: Option<f64>,
     pub top_p: Option<f64>,
     pub top_k: Option<usize>,
+    /// Not published in `generation_config.json` — this struct is the
+    /// resolved per-model default layer, and the operator override
+    /// (#283) lands here so the repetition knobs sit in the same
+    /// precedence chain as the rest.
+    pub repeat_penalty: Option<f32>,
+    pub repeat_last_n: Option<usize>,
+    pub presence_penalty: Option<f32>,
+    pub frequency_penalty: Option<f32>,
+    pub seed: Option<u64>,
 }
 
 impl ModelGenerationDefaults {
@@ -155,6 +164,21 @@ impl ModelGenerationDefaults {
         if let Some(k) = op.top_k {
             self.top_k = Some(k);
         }
+        if let Some(r) = op.repeat_penalty {
+            self.repeat_penalty = Some(r);
+        }
+        if let Some(n) = op.repeat_last_n {
+            self.repeat_last_n = Some(n);
+        }
+        if let Some(v) = op.presence_penalty {
+            self.presence_penalty = Some(v);
+        }
+        if let Some(v) = op.frequency_penalty {
+            self.frequency_penalty = Some(v);
+        }
+        if let Some(v) = op.seed {
+            self.seed = Some(v);
+        }
         tracing::info!(
             model = %model_id,
             model_temperature = ?before.temperature,
@@ -163,6 +187,11 @@ impl ModelGenerationDefaults {
             temperature = ?self.temperature,
             top_p = ?self.top_p,
             top_k = ?self.top_k,
+            repeat_penalty = ?self.repeat_penalty,
+            repeat_last_n = ?self.repeat_last_n,
+            presence_penalty = ?self.presence_penalty,
+            frequency_penalty = ?self.frequency_penalty,
+            seed = ?self.seed,
             "sampling: operator override applied over the model's published defaults (#283)"
         );
         self
@@ -179,6 +208,8 @@ pub struct RequestedSampling {
     pub seed: Option<u64>,
     pub repeat_penalty: Option<f32>,
     pub repeat_last_n: Option<usize>,
+    pub presence_penalty: Option<f32>,
+    pub frequency_penalty: Option<f32>,
 }
 
 /// Fully-resolved sampling for one request.
@@ -190,6 +221,9 @@ pub struct SamplingParams {
     pub seed: u64,
     pub repeat_penalty: f32,
     pub repeat_last_n: usize,
+    /// Sequence-wide penalties, OpenAI semantics. `0.0` disables.
+    pub presence_penalty: f32,
+    pub frequency_penalty: f32,
 }
 
 impl SamplingParams {
@@ -215,9 +249,23 @@ impl SamplingParams {
             // whenever the caller stayed quiet.
             top_p: requested.top_p.or(model.top_p),
             top_k: requested.top_k.or(model.top_k),
-            seed: requested.seed.unwrap_or(random_seed),
-            repeat_penalty: requested.repeat_penalty.unwrap_or(DEFAULT_REPEAT_PENALTY),
-            repeat_last_n: requested.repeat_last_n.unwrap_or(DEFAULT_REPEAT_LAST_N),
+            seed: requested.seed.or(model.seed).unwrap_or(random_seed),
+            repeat_penalty: requested
+                .repeat_penalty
+                .or(model.repeat_penalty)
+                .unwrap_or(DEFAULT_REPEAT_PENALTY),
+            repeat_last_n: requested
+                .repeat_last_n
+                .or(model.repeat_last_n)
+                .unwrap_or(DEFAULT_REPEAT_LAST_N),
+            presence_penalty: requested
+                .presence_penalty
+                .or(model.presence_penalty)
+                .unwrap_or(0.0),
+            frequency_penalty: requested
+                .frequency_penalty
+                .or(model.frequency_penalty)
+                .unwrap_or(0.0),
         }
     }
 
@@ -252,6 +300,7 @@ mod tests {
             temperature: Some(1.0),
             top_p: Some(0.95),
             top_k: Some(20),
+            ..Default::default()
         }
     }
 
@@ -298,6 +347,7 @@ mod tests {
             temperature,
             top_p,
             top_k,
+            ..Default::default()
         }
     }
 
@@ -394,6 +444,103 @@ mod tests {
     /// A model with no `generation_config.json` must behave exactly as
     /// it did before this landed, so nothing regresses on the models
     /// that do not ship one.
+    /// An operator can pin the seed, because the client that needs it
+    /// pinned cannot send one: pi's request builder emits no `seed`, so
+    /// without this a parameter search runs unseeded and sampling
+    /// variance is indistinguishable from the effect being measured.
+    #[test]
+    fn an_operator_can_pin_the_seed() {
+        let op = cortex_core::harness::SamplingOverride {
+            seed: Some(424_242),
+            ..Default::default()
+        };
+        let defaults = ModelGenerationDefaults::default().with_operator_override("m", Some(&op));
+        // The random seed argument must be ignored once an operator pinned one.
+        let p = SamplingParams::resolve(&RequestedSampling::default(), &defaults, 999);
+        assert_eq!(p.seed, 424_242);
+    }
+
+    /// Precedence holds: a caller that names its own seed keeps it.
+    #[test]
+    fn a_requested_seed_outranks_the_operator() {
+        let op = cortex_core::harness::SamplingOverride {
+            seed: Some(424_242),
+            ..Default::default()
+        };
+        let defaults = ModelGenerationDefaults::default().with_operator_override("m", Some(&op));
+        let req = RequestedSampling {
+            seed: Some(7),
+            ..Default::default()
+        };
+        assert_eq!(SamplingParams::resolve(&req, &defaults, 999).seed, 7);
+    }
+
+    /// With neither, the per-request random seed still applies — an
+    /// unset override must not accidentally freeze the fleet's RNG.
+    #[test]
+    fn no_pinned_seed_leaves_generation_random() {
+        let d = ModelGenerationDefaults::default().with_operator_override("m", None);
+        assert_eq!(
+            SamplingParams::resolve(&RequestedSampling::default(), &d, 999).seed,
+            999
+        );
+    }
+
+    /// The gap this closes: `repeat_last_n` defaults to 64 — candle's
+    /// chat example value — while a reasoning model's think block runs
+    /// to tens of thousands of tokens. Anything restated more than 64
+    /// tokens later is invisible to the penalty, and before this there
+    /// was no operator route to the knob at all: `SamplingOverride`
+    /// carried only temperature/top_p/top_k, and OpenAI-shaped clients
+    /// do not send repetition parameters. The 64 was effectively
+    /// hardcoded in production.
+    #[test]
+    fn an_operator_can_widen_the_repetition_window() {
+        let op = cortex_core::harness::SamplingOverride {
+            repeat_penalty: Some(1.05),
+            repeat_last_n: Some(2048),
+            presence_penalty: None,
+            frequency_penalty: None,
+            ..Default::default()
+        };
+        let defaults = ModelGenerationDefaults::default().with_operator_override("m", Some(&op));
+        let p = SamplingParams::resolve(&RequestedSampling::default(), &defaults, 7);
+        assert_eq!(p.repeat_last_n, 2048);
+        assert!((p.repeat_penalty - 1.05).abs() < f32::EPSILON);
+    }
+
+    /// Precedence is request > operator > built-in, same as every other
+    /// sampling field — a caller that named a window keeps it.
+    #[test]
+    fn a_requested_repetition_window_outranks_the_operator() {
+        let op = cortex_core::harness::SamplingOverride {
+            repeat_last_n: Some(2048),
+            presence_penalty: None,
+            frequency_penalty: None,
+            ..Default::default()
+        };
+        let defaults = ModelGenerationDefaults::default().with_operator_override("m", Some(&op));
+        let req = RequestedSampling {
+            repeat_last_n: Some(128),
+            presence_penalty: None,
+            frequency_penalty: None,
+            ..Default::default()
+        };
+        assert_eq!(
+            SamplingParams::resolve(&req, &defaults, 7).repeat_last_n,
+            128
+        );
+    }
+
+    /// An operator who sets nothing must not change the constants.
+    #[test]
+    fn no_operator_setting_leaves_the_repetition_defaults_alone() {
+        let d = ModelGenerationDefaults::default().with_operator_override("m", None);
+        let p = SamplingParams::resolve(&RequestedSampling::default(), &d, 7);
+        assert_eq!(p.repeat_last_n, DEFAULT_REPEAT_LAST_N);
+        assert!((p.repeat_penalty - DEFAULT_REPEAT_PENALTY).abs() < f32::EPSILON);
+    }
+
     #[test]
     fn without_model_defaults_the_old_behaviour_is_preserved() {
         let p = SamplingParams::resolve(
@@ -495,6 +642,8 @@ mod tests {
             &RequestedSampling {
                 repeat_penalty: Some(1.05),
                 repeat_last_n: Some(512),
+                presence_penalty: None,
+                frequency_penalty: None,
                 ..Default::default()
             },
             &ModelGenerationDefaults::default(),

crates/neuron/src/harness/tp/tp_qwen3_5.rs

@@ -2445,7 +2445,7 @@ mod tests {
     }
 
     #[cfg(not(feature = "cuda"))]
-    fn vb_over(path: &std::path::Path) -> ShardedVarBuilder {
+    fn vb_over(path: &std::path::Path) -> ShardedVarBuilder<'_> {
         // SAFETY: mmap of a file the test just wrote; nothing mutates it.
         unsafe {
             candle_nn::var_builder::ShardedSafeTensors::var_builder(

crates/neuron/src/wire/openai_chat.rs

@@ -27,8 +27,8 @@
 //! propagate without us writing any logic.
 
 use cortex_core::openai::{
-    ChatCompletionChunk, ChunkChoice, CompletionTokensDetails, HelexaTiming, PromptTokensDetails,
-    Usage,
+    ChatCompletionChoice, ChatCompletionChunk, ChatCompletionResponse, ChatMessage, ChunkChoice,
+    CompletionTokensDetails, HelexaTiming, MessageContent, PromptTokensDetails, Usage,
 };
 use serde_json::json;
 use tokio::sync::mpsc;
@@ -450,10 +450,379 @@ fn usage_chunk(
     }
 }
 
+/// Drain an [`InferenceEvent`] stream into a single non-streaming
+/// [`ChatCompletionResponse`] (#285).
+///
+/// Non-streaming used to be a *separate inference path* that took the
+/// per-model `inference_lock` for the whole request. That made it
+/// invisible to the batch engine (#98), so concurrent non-streaming
+/// callers serialized completely — measured on beast at 1.00x aggregate
+/// throughput across 1/2/4/8 concurrency, against 3.98x for the same
+/// work streamed. `max_in_flight` bought those requests admission and
+/// nothing else.
+///
+/// Expressing non-streaming as "run the stream, buffer the result"
+/// collapses the two paths onto one producer, so batching, prefix-cache
+/// reuse, tool-call extraction, the reasoning governor and server-side
+/// timing all apply identically to both. It also removes a second place
+/// where the same wire object could be assembled differently — the
+/// shape of defect #252 and #272 both were.
+///
+/// Reasoning is kept out of `content` — it is not the assistant's
+/// reply — but it is **returned** on `message.reasoning_content`, the
+/// same field the streaming path emits as
+/// `choice.delta.reasoning_content`.
+///
+/// It used to be discarded here (#300). That was inherited from the
+/// path this replaced and was wrong on both counts: the tokens are
+/// metered in `usage.completion_tokens_details.reasoning_tokens`, so a
+/// caller paid for text it could never see; and a client cannot replay
+/// what it never received, so on the next turn the model saw its own
+/// prior turn as a bare assertion with the reasoning that produced it
+/// deleted.
+///
+/// `finish_reason` becomes `"tool_calls"` whenever any tool call was
+/// parsed.
+///
+/// Two fields that the old path hardcoded to `None` now populate,
+/// because the events carry them and the struct already had the slots:
+/// `prompt_tokens_details.cached_tokens` and `helexa_timing`.
+///
+/// A stream that ends without a `Finish` is an error, not an empty
+/// reply: that is how a dropped engine slot or a fatal worker fault
+/// presents, and returning a 200 with partial text would hide it.
+pub async fn collect_chat_completion(
+    mut rx: mpsc::Receiver<InferenceEvent>,
+    id: String,
+    created: u64,
+    model_id: String,
+) -> Result<ChatCompletionResponse, String> {
+    let mut content = String::new();
+    let mut reasoning = String::new();
+    let mut tool_calls: Vec<serde_json::Value> = Vec::new();
+    let mut finish: Option<(FinishReason, u32, u32, u32, u32, Option<FinishTiming>)> = None;
+
+    while let Some(event) = rx.recv().await {
+        match event {
+            InferenceEvent::Start => {}
+            InferenceEvent::TextDelta(t) => content.push_str(&t),
+            // Kept out of `content`, returned on `reasoning_content`.
+            InferenceEvent::ReasoningDelta(t) => reasoning.push_str(&t),
+            InferenceEvent::ToolCall {
+                index,
+                id: call_id,
+                name,
+                arguments,
+            } => tool_calls.push(json!({
+                "index": index,
+                "id": call_id,
+                "type": "function",
+                "function": { "name": name, "arguments": arguments },
+            })),
+            InferenceEvent::Finish {
+                reason,
+                prompt_tokens,
+                completion_tokens,
+                reasoning_tokens,
+                cached_tokens,
+                timing,
+            } => {
+                finish = Some((
+                    reason,
+                    prompt_tokens,
+                    completion_tokens,
+                    reasoning_tokens,
+                    cached_tokens,
+                    timing,
+                ));
+                break;
+            }
+        }
+    }
+
+    let Some((reason, prompt_tokens, completion_tokens, reasoning_tokens, cached_tokens, timing)) =
+        finish
+    else {
+        return Err(
+            "inference stream ended without a finish event (engine slot dropped?)".to_string(),
+        );
+    };
+
+    // A parsed tool call outranks whatever the sampler stopped on, so a
+    // model that emits a call and then hits EOS still reports
+    // `tool_calls` and the client dispatches instead of rendering.
+    let finish_reason = if tool_calls.is_empty() {
+        reason.as_openai_str().to_string()
+    } else {
+        "tool_calls".to_string()
+    };
+    let mut message_extra = serde_json::Map::new();
+    if !tool_calls.is_empty() {
+        message_extra.insert("tool_calls".into(), json!(tool_calls));
+    }
+    // Absent rather than empty when the model did not think: a
+    // non-reasoning model must keep the unchanged message shape.
+    if !reasoning.is_empty() {
+        message_extra.insert("reasoning_content".into(), json!(reasoning));
+    }
+    let message_extra = serde_json::Value::Object(message_extra);
+
+    Ok(ChatCompletionResponse {
+        id,
+        object: "chat.completion".into(),
+        created,
+        model: model_id,
+        choices: vec![ChatCompletionChoice {
+            index: 0,
+            message: ChatMessage {
+                role: "assistant".into(),
+                content: MessageContent::Text(content),
+                extra: message_extra,
+            },
+            finish_reason: Some(finish_reason),
+            extra: serde_json::Value::Object(Default::default()),
+        }],
+        usage: Some(Usage {
+            prompt_tokens: prompt_tokens as u64,
+            completion_tokens: completion_tokens as u64,
+            total_tokens: (prompt_tokens + completion_tokens) as u64,
+            completion_tokens_details: (reasoning_tokens > 0).then_some(CompletionTokensDetails {
+                reasoning_tokens: reasoning_tokens as u64,
+            }),
+            prompt_tokens_details: (cached_tokens > 0).then_some(PromptTokensDetails {
+                cached_tokens: cached_tokens as u64,
+            }),
+            helexa_timing: timing.map(|t| HelexaTiming {
+                prefill_ms: t.prefill_ms as u64,
+                decode_ms: t.decode_ms as u64,
+                prefill_tokens: t.prefill_tokens as u64,
+            }),
+        }),
+        extra: serde_json::Value::Object(Default::default()),
+    })
+}
+
 #[cfg(test)]
 mod tests {
+
+    /// The model's think block must come back on the non-streaming
+    /// path too (#300).
+    ///
+    /// It used to be dropped here while the streaming path emitted it
+    /// as `choice.delta.reasoning_content`. The tokens were metered
+    /// either way, so a non-streaming caller paid for text it could
+    /// never see — and, worse, could not replay the model's reasoning
+    /// on the next turn, leaving the model to face its own prior turn
+    /// as a bare assertion.
+    #[tokio::test]
+    async fn non_streaming_returns_the_models_reasoning() {
+        let (tx, rx) = mpsc::channel::<InferenceEvent>(8);
+        tx.send(InferenceEvent::Start).await.unwrap();
+        tx.send(InferenceEvent::ReasoningDelta(
+            "weighing the options".into(),
+        ))
+        .await
+        .unwrap();
+        tx.send(InferenceEvent::TextDelta("the answer".into()))
+            .await
+            .unwrap();
+        tx.send(InferenceEvent::Finish {
+            reason: FinishReason::Stop,
+            prompt_tokens: 5,
+            completion_tokens: 7,
+            reasoning_tokens: 3,
+            cached_tokens: 0,
+            timing: None,
+        })
+        .await
+        .unwrap();
+        drop(tx);
+
+        let r = collect_chat_completion(rx, "id".into(), 0, "m".into())
+            .await
+            .expect("collect");
+        let msg = &r.choices[0].message;
+        assert_eq!(
+            msg.extra["reasoning_content"], "weighing the options",
+            "reasoning must be returned, not discarded"
+        );
+        // And it must NOT be folded into the visible reply.
+        assert!(
+            matches!(&msg.content, MessageContent::Text(t) if t == "the answer"),
+            "reasoning leaked into content"
+        );
+    }
+
+    /// A non-reasoning model must keep the unchanged message shape —
+    /// absent, not an empty string.
+    #[tokio::test]
+    async fn no_reasoning_means_no_reasoning_field() {
+        let (tx, rx) = mpsc::channel::<InferenceEvent>(8);
+        tx.send(InferenceEvent::Start).await.unwrap();
+        tx.send(InferenceEvent::TextDelta("plain".into()))
+            .await
+            .unwrap();
+        tx.send(InferenceEvent::Finish {
+            reason: FinishReason::Stop,
+            prompt_tokens: 1,
+            completion_tokens: 1,
+            reasoning_tokens: 0,
+            cached_tokens: 0,
+            timing: None,
+        })
+        .await
+        .unwrap();
+        drop(tx);
+        let r = collect_chat_completion(rx, "id".into(), 0, "m".into())
+            .await
+            .expect("collect");
+        assert!(
+            r.choices[0]
+                .message
+                .extra
+                .get("reasoning_content")
+                .is_none()
+        );
+    }
     use super::*;
 
+    // ── Non-streaming collector (#285) ────────────────────────────
+
+    async fn collect_from(events: Vec<InferenceEvent>) -> Result<ChatCompletionResponse, String> {
+        let (tx, rx) = mpsc::channel(64);
+        for e in events {
+            tx.send(e).await.expect("send");
+        }
+        drop(tx);
+        collect_chat_completion(rx, "chatcmpl-test".into(), 1_700_000_000, "m".into()).await
+    }
+
+    fn finish(reason: FinishReason) -> InferenceEvent {
+        InferenceEvent::Finish {
+            reason,
+            prompt_tokens: 10,
+            completion_tokens: 4,
+            reasoning_tokens: 0,
+            cached_tokens: 0,
+            timing: None,
+        }
+    }
+
+    #[tokio::test]
+    async fn collector_concatenates_text_deltas() {
+        let r = collect_from(vec![
+            InferenceEvent::Start,
+            InferenceEvent::TextDelta("hel".into()),
+            InferenceEvent::TextDelta("lo".into()),
+            finish(FinishReason::Stop),
+        ])
+        .await
+        .expect("collected");
+        let c = &r.choices[0];
+        assert!(matches!(&c.message.content, MessageContent::Text(t) if t == "hello"));
+        assert_eq!(c.finish_reason.as_deref(), Some("stop"));
+        assert_eq!(r.object, "chat.completion");
+    }
+
+    /// Reasoning is counted but never rendered — the behaviour of the
+    /// path this replaced. A regression here would leak scratchpad into
+    /// `content` for every non-streaming caller.
+    #[tokio::test]
+    async fn collector_drops_reasoning_text_but_keeps_the_count() {
+        let (tx, rx) = mpsc::channel(64);
+        tx.send(InferenceEvent::ReasoningDelta("secret".into()))
+            .await
+            .expect("send");
+        tx.send(InferenceEvent::TextDelta("answer".into()))
+            .await
+            .expect("send");
+        tx.send(InferenceEvent::Finish {
+            reason: FinishReason::Stop,
+            prompt_tokens: 10,
+            completion_tokens: 9,
+            reasoning_tokens: 5,
+            cached_tokens: 0,
+            timing: None,
+        })
+        .await
+        .expect("send");
+        drop(tx);
+        let r = collect_chat_completion(rx, "id".into(), 1, "m".into())
+            .await
+            .expect("collected");
+        let c = &r.choices[0];
+        assert!(matches!(&c.message.content, MessageContent::Text(t) if t == "answer"));
+        let u = r.usage.expect("usage");
+        assert_eq!(
+            u.completion_tokens_details
+                .expect("details")
+                .reasoning_tokens,
+            5
+        );
+    }
+
+    /// A parsed tool call outranks the sampler's stop reason, matching
+    /// the streaming projector.
+    #[tokio::test]
+    async fn collector_promotes_finish_reason_to_tool_calls() {
+        let r = collect_from(vec![
+            InferenceEvent::ToolCall {
+                index: 0,
+                id: "call_1".into(),
+                name: "read_file".into(),
+                arguments: r#"{"path":"a.txt"}"#.into(),
+            },
+            finish(FinishReason::Stop),
+        ])
+        .await
+        .expect("collected");
+        let c = &r.choices[0];
+        assert_eq!(c.finish_reason.as_deref(), Some("tool_calls"));
+        let calls = c.message.extra.get("tool_calls").expect("tool_calls");
+        assert_eq!(calls[0]["function"]["name"], "read_file");
+        assert_eq!(calls[0]["type"], "function");
+        assert_eq!(calls[0]["id"], "call_1");
+    }
+
+    /// cached_tokens and helexa_timing were hardcoded to `None` on the
+    /// old non-streaming path; the events carried them all along.
+    #[tokio::test]
+    async fn collector_surfaces_cached_tokens_and_timing() {
+        let (tx, rx) = mpsc::channel(8);
+        tx.send(InferenceEvent::Finish {
+            reason: FinishReason::Stop,
+            prompt_tokens: 100,
+            completion_tokens: 5,
+            reasoning_tokens: 0,
+            cached_tokens: 64,
+            timing: Some(FinishTiming {
+                prefill_ms: 12,
+                decode_ms: 34,
+                prefill_tokens: 100,
+            }),
+        })
+        .await
+        .expect("send");
+        drop(tx);
+        let r = collect_chat_completion(rx, "id".into(), 1, "m".into())
+            .await
+            .expect("collected");
+        let u = r.usage.expect("usage");
+        assert_eq!(u.prompt_tokens_details.expect("details").cached_tokens, 64);
+        let t = u.helexa_timing.expect("timing");
+        assert_eq!((t.prefill_ms, t.decode_ms, t.prefill_tokens), (12, 34, 100));
+    }
+
+    /// A dropped engine slot ends the channel with no Finish. Returning
+    /// a 200 with partial text would hide a fatal worker fault.
+    #[tokio::test]
+    async fn collector_errors_when_the_stream_ends_without_finish() {
+        let err = collect_from(vec![InferenceEvent::TextDelta("partial".into())])
+            .await
+            .expect_err("must not succeed");
+        assert!(err.contains("without a finish"), "unexpected: {err}");
+    }
+
     /// Drain the projection's output into a Vec for assertion.
     async fn collect(mut rx: mpsc::Receiver<ChatCompletionChunk>) -> Vec<ChatCompletionChunk> {
         let mut out = Vec::new();

crates/neuron/src/wire/openai_responses.rs

@@ -176,6 +176,29 @@ pub fn request_to_chat(req: ResponsesRequest) -> Result<ChatCompletionRequest, T
                     {
                         attach_reasoning(&mut msg, text);
                     }
+                    // A `function_call` item belongs to the assistant
+                    // turn that precedes it, not to a turn of its own.
+                    //
+                    // Responses splits one turn across several items —
+                    // `reasoning`, then `message`, then `function_call`
+                    // — while chat carries all three on a single
+                    // assistant message. Pushing each item separately
+                    // produced *consecutive assistant turns*: one that
+                    // deliberates and then ends without acting, and one
+                    // that acts with an empty `<think></think>`.
+                    //
+                    // Neither shape exists in training, and the model
+                    // imitates what it is shown: after four such
+                    // examples in its own replayed context it produced
+                    // a think block and a preamble, then ended the turn
+                    // without the tool call it had just announced
+                    // (#296). Observed on a real session; the model was
+                    // copying our transcript, not misbehaving.
+                    if let Some(prev) = messages.last_mut()
+                        && merge_tool_call_into(prev, &mut msg)
+                    {
+                        continue;
+                    }
                     messages.push(msg);
                 }
             }
@@ -239,6 +262,8 @@ pub fn request_to_chat(req: ResponsesRequest) -> Result<ChatCompletionRequest, T
         seed: req.seed,
         repetition_penalty: req.repetition_penalty,
         repeat_last_n: req.repeat_last_n,
+        presence_penalty: req.presence_penalty,
+        frequency_penalty: req.frequency_penalty,
         max_tokens: req.max_output_tokens,
         // Responses spells the cap `max_output_tokens`, already
         // resolved onto the legacy field above.
@@ -376,6 +401,57 @@ fn apply_reasoning_effort(extra: &mut serde_json::Map<String, Value>) {
     }
 }
 
+/// Fold a `function_call`-derived assistant message into the assistant
+/// turn it belongs to.
+///
+/// Returns `true` when the call was merged and the caller should drop
+/// its message. Only ever merges a *pure* tool-call message (empty
+/// content, no reasoning of its own) into a preceding assistant turn,
+/// so a genuine two-turn sequence is never collapsed and a reasoning
+/// item that arrived with the call keeps its own turn.
+///
+/// Consecutive calls accumulate, which is what parallel tool calling
+/// looks like on the wire: one assistant turn, several `tool_calls`.
+fn merge_tool_call_into(prev: &mut ChatMessage, msg: &mut ChatMessage) -> bool {
+    if prev.role != "assistant" || msg.role != "assistant" {
+        return false;
+    }
+    // Only a bare tool-call carrier folds in.
+    if !matches!(&msg.content, MessageContent::Text(t) if t.is_empty()) {
+        return false;
+    }
+    if msg.extra.get("reasoning_content").is_some() {
+        return false;
+    }
+    let Some(incoming) = msg
+        .extra
+        .get_mut("tool_calls")
+        .and_then(|v| v.as_array_mut())
+        .map(std::mem::take)
+    else {
+        return false;
+    };
+    // Everything else the item carried must survive the fold, or this
+    // trades one silent drop for another.
+    if msg
+        .extra
+        .as_object()
+        .is_some_and(|o| o.keys().any(|k| k != "tool_calls"))
+    {
+        return false;
+    }
+    let Some(obj) = prev.extra.as_object_mut() else {
+        return false;
+    };
+    match obj.get_mut("tool_calls").and_then(|v| v.as_array_mut()) {
+        Some(existing) => existing.extend(incoming),
+        None => {
+            obj.insert("tool_calls".into(), Value::Array(incoming));
+        }
+    }
+    true
+}
+
 /// The text of a replayed reasoning item, under either spelling.
 ///
 /// `summary` is what neuron emits (`summary_text` parts) and `content`
@@ -747,6 +823,31 @@ async fn run_projection(
         return;
     }
 
+    // No `Finish` means the producer died rather than finished — the
+    // poisoned-model path named above, an OOM, a dropped worker. This
+    // used to default to `FinishReason::Stop`, which told the caller a
+    // crash was a complete answer: pi-ai maps `status: "completed"` to
+    // `stopReason: "stop"` and treats whatever partial text arrived as
+    // the model's considered reply.
+    //
+    // Decided here, before the message item is opened, so a failed
+    // stream does not leave an item open that nothing will ever close.
+    let Some(reason) = finish else {
+        let _ = tx
+            .send(ResponseStreamFrame {
+                event_name: events::FAILED,
+                data: json!({
+                    "response": failed_response_shell(
+                        &meta,
+                        "server_error",
+                        "inference ended before the model finished",
+                    )
+                }),
+            })
+            .await;
+        return;
+    };
+
     // A stream can end while still thinking — a reasoning model that
     // exhausts max_output_tokens mid-block never emits visible text.
     // Close the item so the client sees a finished response rather than
@@ -761,7 +862,6 @@ async fn run_projection(
         return;
     }
 
-    let reason = finish.unwrap_or(FinishReason::Stop);
     let _ = emit_finish_frames(
         &tx,
         &meta,
@@ -777,6 +877,23 @@ async fn run_projection(
     .await;
 }
 
+/// The `response` payload for a [`events::FAILED`] frame.
+///
+/// pi-ai renders this as `` `${error.code}: ${error.message}` `` and
+/// throws, which is the correct outcome — the request genuinely failed
+/// — and is strictly better than a silent success.
+fn failed_response_shell(meta: &ResponseMeta, code: &str, message: &str) -> Value {
+    json!({
+        "id": meta.response_id,
+        "object": "response",
+        "created_at": meta.created_at,
+        "status": "failed",
+        "model": meta.model_id,
+        "output": [],
+        "error": { "code": code, "message": message },
+    })
+}
+
 /// Streaming state for a reasoning model's think block (#267).
 ///
 /// The block is its own output item, ahead of the message, so the
@@ -1043,7 +1160,14 @@ async fn emit_finish_frames(
             }),
         },
         ResponseStreamFrame {
-            event_name: events::COMPLETED,
+            // `response.incomplete` when we stopped short, so a client
+            // keyed on the event name rather than on `response.status`
+            // still sees a terminal frame instead of timing out.
+            event_name: if status == "incomplete" {
+                events::INCOMPLETE
+            } else {
+                events::COMPLETED
+            },
             data: json!({
                 "response": response_shell(meta, status, &output_items, usage)
             }),
@@ -1068,6 +1192,15 @@ fn response_shell(
     obj.insert("object".into(), Value::String("response".into()));
     obj.insert("created_at".into(), json!(meta.created_at));
     obj.insert("status".into(), Value::String(status.into()));
+    // Why we stopped short. A client cannot distinguish an honest
+    // truncation from a protocol fault without this, and pi-ai treats
+    // the difference as continue-vs-halt.
+    if status == "incomplete" {
+        obj.insert(
+            "incomplete_details".into(),
+            json!({ "reason": "max_output_tokens" }),
+        );
+    }
     obj.insert("model".into(), Value::String(meta.model_id.clone()));
     obj.insert("output".into(), Value::Array(output.to_vec()));
     if let Some(u) = usage {
@@ -1116,28 +1249,53 @@ fn finish_to_status(reason: FinishReason) -> &'static str {
 /// Collect a chat-completions response into a non-streaming
 /// [`ResponsesResponse`]. Used by the `/v1/responses` handler when
 /// the request doesn't set `stream: true`.
+///
+/// `reasoning` is the model's think block, when it produced one. It
+/// becomes its own output item ahead of the message — the same shape
+/// and ordering the streaming projector emits. Omitting it (as this did
+/// until #300) meant a non-streaming caller was billed for
+/// `reasoning_tokens` it could never see, and could not replay the
+/// model's own thinking on the next turn.
 pub fn build_response(
     meta: &ResponseMeta,
     full_text: String,
     reason: FinishReason,
     usage: Option<ResponsesUsage>,
+    reasoning: Option<String>,
 ) -> ResponsesResponse {
     let status = finish_to_status(reason).to_string();
+    let mut output: Vec<ResponsesOutputItem> = Vec::with_capacity(2);
+    if let Some(text) = reasoning.filter(|t| !t.is_empty()) {
+        output.push(ResponsesOutputItem::Reasoning {
+            id: reasoning_item_id(meta),
+            summary: vec![cortex_core::responses::ResponsesSummaryPart::summary_text(
+                text,
+            )],
+            status: "completed".into(),
+        });
+    }
     ResponsesResponse {
         id: meta.response_id.clone(),
         object: "response".into(),
         created_at: meta.created_at,
         status: status.clone(),
         model: meta.model_id.clone(),
-        output: vec![ResponsesOutputItem::Message {
-            id: meta.message_item_id.clone(),
-            role: "assistant".into(),
-            content: vec![ResponsesOutputContent::OutputText {
-                text: full_text,
-                annotations: vec![],
-            }],
-            status,
-        }],
+        output: {
+            output.push(ResponsesOutputItem::Message {
+                id: meta.message_item_id.clone(),
+                role: "assistant".into(),
+                content: vec![ResponsesOutputContent::OutputText {
+                    text: full_text,
+                    annotations: vec![],
+                }],
+                status,
+            });
+            output
+        },
+        // Same contract as the streaming shell: a truncated response
+        // says so, or the caller cannot tell it from a fault.
+        incomplete_details: matches!(reason, FinishReason::Length)
+            .then(cortex_core::responses::IncompleteDetails::max_output_tokens),
         usage,
     }
 }
@@ -1147,6 +1305,173 @@ mod tests {
     use super::*;
     use cortex_core::openai::MessageContent;
 
+    /// The non-streaming Responses body must carry a reasoning item
+    /// (#300), matching the item the streaming projector emits.
+    ///
+    /// Observed before the fix: a response reporting
+    /// `output_tokens_details.reasoning_tokens: 4000` with an empty
+    /// `output` — 4,000 tokens generated, metered, and invisible.
+    #[test]
+    fn non_streaming_response_carries_a_reasoning_item() {
+        let r = build_response(
+            &meta(),
+            "the answer".into(),
+            FinishReason::Stop,
+            None,
+            Some("weighing the options".into()),
+        );
+        let v = serde_json::to_value(&r).unwrap();
+        let items = v["output"].as_array().expect("output array");
+        assert_eq!(items.len(), 2, "expected a reasoning item and a message");
+        assert_eq!(items[0]["type"], "reasoning", "reasoning comes first");
+        assert_eq!(items[0]["summary"][0]["type"], "summary_text");
+        assert_eq!(items[0]["summary"][0]["text"], "weighing the options");
+        assert_eq!(items[1]["type"], "message");
+        assert_eq!(items[1]["content"][0]["text"], "the answer");
+    }
+
+    /// A model that did not think produces no reasoning item — the
+    /// unchanged single-item shape.
+    #[test]
+    fn no_reasoning_means_no_reasoning_item() {
+        let r = build_response(&meta(), "plain".into(), FinishReason::Stop, None, None);
+        let v = serde_json::to_value(&r).unwrap();
+        let items = v["output"].as_array().unwrap();
+        assert_eq!(items.len(), 1);
+        assert_eq!(items[0]["type"], "message");
+    }
+
+    /// A file read back must survive the *whole* chain: Responses
+    /// `function_call_output` -> chat message -> rendered prompt.
+    ///
+    /// The reported symptom is a model that writes a file correctly,
+    /// reads it back, and consistently sees errors that are not there.
+    /// This walks the real path rather than a hand-built chat message,
+    /// because the translation is the part in between.
+    #[test]
+    fn a_file_read_back_survives_the_whole_translation() {
+        let file = "<!DOCTYPE html>\n<html lang=\"en\">\n\n  <script>\n    if (a && b || c < d) { s = '#b8905a'; }\n  </script>\n</html>\n";
+        let req: ResponsesRequest = serde_json::from_value(serde_json::json!({
+            "model": "m",
+            "input": [
+                {"type": "message", "role": "user",
+                 "content": [{"type": "input_text", "text": "read it back"}]},
+                {"type": "function_call", "call_id": "c1", "name": "read",
+                 "arguments": "{\"path\":\"/tmp/index.html\"}"},
+                // pi sends the tool result as a plain string (its
+                // `convertToolResultOutput` only produces an array when
+                // images are involved).
+                {"type": "function_call_output", "call_id": "c1", "output": file},
+                {"type": "message", "role": "user",
+                 "content": [{"type": "input_text", "text": "any errors?"}]}
+            ]
+        }))
+        .unwrap();
+        let chat = request_to_chat(req).expect("translate");
+
+        // The tool message must carry the file verbatim.
+        let tool = chat
+            .messages
+            .iter()
+            .find(|m| m.role == "tool")
+            .expect("a tool message");
+        let got = match &tool.content {
+            MessageContent::Text(t) => t.clone(),
+            other => panic!("tool content is not text: {other:?}"),
+        };
+        assert_eq!(got, file, "translation altered the file content");
+
+        // And it must survive the render too.
+        let template = include_str!("../../tests/data/qwen3.8-27b.chat_template.jinja");
+        let out = crate::harness::chat_template::render_chat_template(
+            template,
+            &chat.messages,
+            &Value::Null,
+            &Value::Null,
+        )
+        .expect("render");
+        for probe in ["<!DOCTYPE html>", "if (a && b || c < d)", "s = '#b8905a';"] {
+            assert!(out.contains(probe), "rendered prompt lost {probe:?}");
+        }
+        for bad in ["&lt;", "&amp;", "&quot;"] {
+            assert!(!out.contains(bad), "content HTML-escaped: {bad:?}");
+        }
+    }
+
+    /// One agentic step must render as ONE assistant turn.
+    ///
+    /// Responses splits a turn into `reasoning` + `message` +
+    /// `function_call` items. Emitting a message per item produced
+    /// *consecutive assistant turns* — one that deliberates and then
+    /// ends without acting, one that acts with an empty think block.
+    /// Shown four of those in its own replayed context, the model
+    /// copied the pattern: it produced a think block and a preamble,
+    /// then ended the turn without the tool call it had just announced
+    /// (#296).
+    #[test]
+    fn one_agentic_step_becomes_one_assistant_turn() {
+        let req: ResponsesRequest = serde_json::from_value(serde_json::json!({
+            "model": "m",
+            "input": [
+                {"type": "message", "role": "user",
+                 "content": [{"type": "input_text", "text": "build it"}]},
+                {"type": "reasoning",
+                 "content": [{"type": "reasoning_text", "text": "planning the file"}]},
+                {"type": "message", "role": "assistant",
+                 "content": [{"type": "output_text", "text": "Writing index.html:"}]},
+                {"type": "function_call", "call_id": "c1",
+                 "name": "write", "arguments": "{\"path\":\"a.html\"}"},
+                {"type": "function_call_output", "call_id": "c1", "output": "wrote 10 bytes"},
+                {"type": "message", "role": "user",
+                 "content": [{"type": "input_text", "text": "now check it"}]}
+            ]
+        }))
+        .unwrap();
+        let chat = request_to_chat(req).expect("translate");
+        let roles: Vec<&str> = chat.messages.iter().map(|m| m.role.as_str()).collect();
+        assert_eq!(
+            roles,
+            vec!["user", "assistant", "tool", "user"],
+            "no two assistant turns may be adjacent: got {roles:?}"
+        );
+
+        let a = &chat.messages[1];
+        assert!(
+            a.extra.get("reasoning_content").is_some(),
+            "reasoning must ride the turn it describes"
+        );
+        assert!(
+            matches!(&a.content, MessageContent::Text(t) if t.contains("Writing index.html")),
+            "the assistant's own text must survive the fold"
+        );
+        let calls = a.extra["tool_calls"].as_array().expect("tool_calls");
+        assert_eq!(calls.len(), 1);
+        assert_eq!(calls[0]["function"]["name"], "write");
+    }
+
+    /// Parallel tool calls are several calls on one turn, not several
+    /// turns.
+    #[test]
+    fn consecutive_function_calls_accumulate_on_one_turn() {
+        let req: ResponsesRequest = serde_json::from_value(serde_json::json!({
+            "model": "m",
+            "input": [
+                {"type": "message", "role": "user",
+                 "content": [{"type": "input_text", "text": "go"}]},
+                {"type": "message", "role": "assistant",
+                 "content": [{"type": "output_text", "text": "doing both:"}]},
+                {"type": "function_call", "call_id": "c1", "name": "a", "arguments": "{}"},
+                {"type": "function_call", "call_id": "c2", "name": "b", "arguments": "{}"}
+            ]
+        }))
+        .unwrap();
+        let chat = request_to_chat(req).expect("translate");
+        let roles: Vec<&str> = chat.messages.iter().map(|m| m.role.as_str()).collect();
+        assert_eq!(roles, vec!["user", "assistant"]);
+        let calls = chat.messages[1].extra["tool_calls"].as_array().unwrap();
+        assert_eq!(calls.len(), 2, "both calls belong to the same turn");
+    }
+
     /// Wrap typed items as `input` elements. Most translator tests
     /// exercise the typed path; the bare easy-message and unknown-item
     /// paths have dedicated tests below.
@@ -1184,6 +1509,8 @@ mod tests {
             seed: None,
             repetition_penalty: None,
             repeat_last_n: None,
+            presence_penalty: None,
+            frequency_penalty: None,
             previous_response_id: None,
             extra: Value::Object(Default::default()),
         };
@@ -1210,6 +1537,8 @@ mod tests {
             seed: None,
             repetition_penalty: None,
             repeat_last_n: None,
+            presence_penalty: None,
+            frequency_penalty: None,
             previous_response_id: None,
             extra: Value::Object(Default::default()),
         };
@@ -1237,6 +1566,8 @@ mod tests {
             seed: None,
             repetition_penalty: None,
             repeat_last_n: None,
+            presence_penalty: None,
+            frequency_penalty: None,
             previous_response_id: Some("resp_prev".into()),
             extra: Value::Object(Default::default()),
         };
@@ -1273,6 +1604,8 @@ mod tests {
             seed: None,
             repetition_penalty: None,
             repeat_last_n: None,
+            presence_penalty: None,
+            frequency_penalty: None,
             previous_response_id: None,
             extra: Value::Object(Default::default()),
         };
@@ -1307,6 +1640,8 @@ mod tests {
             seed: None,
             repetition_penalty: None,
             repeat_last_n: None,
+            presence_penalty: None,
+            frequency_penalty: None,
             previous_response_id: None,
             extra: Value::Object(Default::default()),
         };
@@ -1356,6 +1691,8 @@ mod tests {
             seed: None,
             repetition_penalty: None,
             repeat_last_n: None,
+            presence_penalty: None,
+            frequency_penalty: None,
             previous_response_id: None,
             extra: Value::Object(Default::default()),
         };
@@ -1397,6 +1734,8 @@ mod tests {
             seed: None,
             repetition_penalty: None,
             repeat_last_n: None,
+            presence_penalty: None,
+            frequency_penalty: None,
             previous_response_id: None,
             extra: Value::Object(Default::default()),
         };
@@ -1433,6 +1772,8 @@ mod tests {
             seed: None,
             repetition_penalty: None,
             repeat_last_n: None,
+            presence_penalty: None,
+            frequency_penalty: None,
             previous_response_id: None,
             extra: Value::Object(Default::default()),
         };
@@ -2003,9 +2344,19 @@ mod tests {
             names.contains(&events::CONTENT_PART_ADDED),
             "message item must still be opened so the finish frames refer to something"
         );
-        assert_eq!(names.last(), Some(&events::COMPLETED));
+        // This is the live failure in miniature: the model spent its
+        // whole budget thinking and produced almost no visible text.
+        // The stream must still close, and must say *why* it stopped —
+        // this frame used to be named `response.completed` while
+        // carrying `status: "incomplete"`, an inconsistency that cost a
+        // real agentic session its run.
+        assert_eq!(names.last(), Some(&events::INCOMPLETE));
         let last = frames.last().unwrap();
         assert_eq!(last.data["response"]["status"], "incomplete");
+        assert_eq!(
+            last.data["response"]["incomplete_details"]["reason"],
+            "max_output_tokens"
+        );
     }
 
     /// Tool calls sit after the message, which sits after any reasoning
@@ -2299,29 +2650,93 @@ mod tests {
         .unwrap();
         drop(tx);
         let frames = collect(out).await;
+        // The terminal frame is `response.incomplete`, not
+        // `response.completed` — a client keyed on the event name must
+        // still see an end to the stream.
+        assert!(
+            !frames.iter().any(|f| f.event_name == events::COMPLETED),
+            "a truncated response must not announce itself as completed"
+        );
+        let terminal = frames
+            .iter()
+            .find(|f| f.event_name == events::INCOMPLETE)
+            .expect("a terminal frame is required");
+        assert_eq!(terminal.data["response"]["status"], "incomplete");
+        // The field that decides whether a client recovers or halts.
+        assert_eq!(
+            terminal.data["response"]["incomplete_details"]["reason"], "max_output_tokens",
+            "pi-ai maps `incomplete` with no reason to stopReason:error \
+             and ends the agent loop; with max_output_tokens it maps to \
+             stopReason:length, fails truncated tool calls safely, and \
+             takes another turn"
+        );
+    }
+
+    /// The converse: a normal finish must stay `response.completed`
+    /// and carry no reason, or every client sees phantom truncation.
+    #[tokio::test]
+    async fn a_normal_finish_carries_no_incomplete_reason() {
+        let (tx, rx) = mpsc::channel::<InferenceEvent>(8);
+        let out = project_responses_stream(rx, meta());
+        tx.send(InferenceEvent::Start).await.unwrap();
+        tx.send(InferenceEvent::Finish {
+            reason: FinishReason::Stop,
+            prompt_tokens: 0,
+            completion_tokens: 0,
+            reasoning_tokens: 0,
+            cached_tokens: 0,
+            timing: None,
+        })
+        .await
+        .unwrap();
+        drop(tx);
+        let frames = collect(out).await;
         let completed = frames
             .iter()
             .find(|f| f.event_name == events::COMPLETED)
-            .unwrap();
-        assert_eq!(completed.data["response"]["status"], "incomplete");
+            .expect("a normal finish stays `response.completed`");
+        assert_eq!(completed.data["response"]["status"], "completed");
+        assert!(
+            completed.data["response"]
+                .get("incomplete_details")
+                .is_none(),
+            "a completed response must not carry an incomplete reason"
+        );
     }
 
+    /// A producer that drops without ever sending `Finish` did not
+    /// finish — it died (poisoned model, OOM, dropped worker). The
+    /// stream must still terminate, and must terminate as a *failure*.
+    ///
+    /// This previously asserted `COMPLETED`, encoding the defect: a
+    /// crashed inference was announced as a complete response, so a
+    /// client took whatever partial text had arrived to be the model's
+    /// considered answer instead of retrying.
     #[tokio::test]
-    async fn synthesises_start_frames_when_producer_skips_start() {
-        // A producer that drops without sending Start (poisoned
-        // model, immediate disconnect, …) should still produce a
-        // coherent stream — the projector synthesises the
-        // mandatory header frames before COMPLETED so the
-        // consumer never sees an output_text.done without a
-        // matching content_part.added.
+    async fn a_producer_that_dies_terminates_the_stream_as_failed() {
         let (tx, rx) = mpsc::channel::<InferenceEvent>(8);
         let out = project_responses_stream(rx, meta());
         drop(tx);
         let frames = collect(out).await;
         let names: Vec<&str> = frames.iter().map(|f| f.event_name).collect();
+        // The shell still opens, so the failure refers to a response.
         assert!(names.contains(&events::CREATED));
-        assert!(names.contains(&events::COMPLETED));
-        assert!(names.contains(&events::OUTPUT_TEXT_DONE));
+        assert!(
+            !names.contains(&events::COMPLETED),
+            "a crash must never be announced as a completed response"
+        );
+        let failed = frames
+            .iter()
+            .find(|f| f.event_name == events::FAILED)
+            .expect("the stream must terminate, or the client waits out its idle timeout");
+        assert_eq!(failed.data["response"]["status"], "failed");
+        assert_eq!(failed.data["response"]["error"]["code"], "server_error");
+        assert!(
+            failed.data["response"]["error"]["message"]
+                .as_str()
+                .is_some_and(|m| !m.is_empty()),
+            "pi-ai renders `code: message`, so an empty message helps nobody"
+        );
     }
 
     #[tokio::test]
@@ -2369,6 +2784,7 @@ mod tests {
                 output_tokens_details: None,
                 input_tokens_details: None,
             }),
+            None,
         );
         assert_eq!(r.status, "completed");
         match &r.output[0] {
@@ -2394,7 +2810,7 @@ mod tests {
 
     #[test]
     fn build_response_length_yields_incomplete_status() {
-        let r = build_response(&meta(), "trunc".into(), FinishReason::Length, None);
+        let r = build_response(&meta(), "trunc".into(), FinishReason::Length, None, None);
         assert_eq!(r.status, "incomplete");
     }
 }

crates/neuron/tests/activation.rs

@@ -30,6 +30,7 @@ async fn test_load_default_models_skips_unknown_harness() {
             tensor_parallel: None,
             devices: None,
             sampling: None,
+            preserve_thinking: None,
         },
         ModelSpec {
             model_id: "model-b".into(),
@@ -38,6 +39,7 @@ async fn test_load_default_models_skips_unknown_harness() {
             tensor_parallel: None,
             devices: None,
             sampling: None,
+            preserve_thinking: None,
         },
     ];
 
@@ -98,6 +100,7 @@ async fn test_load_default_models_skipped_on_driver_mismatch() {
         tensor_parallel: Some(2),
         devices: None,
         sampling: None,
+        preserve_thinking: None,
     }];
     let activation = ActivationTracker::new(&specs);
     let reason = "host NVIDIA driver/library mismatch (userspace NVML 580.159 vs loaded \
@@ -200,6 +203,7 @@ fn qwen_spec() -> ModelSpec {
         tensor_parallel: None,
         devices: None,
         sampling: None,
+        preserve_thinking: None,
     }
 }
 

crates/neuron/tests/candle_lifecycle.rs

@@ -54,6 +54,7 @@ async fn test_candle_qwen3_load_unload_lifecycle() {
         tensor_parallel: None,
         devices: Some(vec![0]),
         sampling: None,
+        preserve_thinking: None,
     };
 
     registry

crates/neuron/tests/data/qwen3.8-27b.chat_template.jinja

@@ -0,0 +1,170 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- set reasoning_instructions = '' %}
+{%- if enable_thinking is undefined or enable_thinking is true %}
+    {%- set resolved_reasoning_effort = reasoning_effort|default('xhigh') %}
+    {%- if resolved_reasoning_effort not in ('xhigh', 'medium', 'low') %}
+        {{- raise_exception('Unexpected reasoning effort ' ~ reasoning_effort ~ '. Supported types are xhigh (default), medium, and low.') }}
+    {%- endif %}
+    {%- if resolved_reasoning_effort == 'xhigh' %}
+        {%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}
+    {%- elif resolved_reasoning_effort == 'low' %}
+        {%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}
+    {%- endif %}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {%- if reasoning_instructions %}
+        {{- reasoning_instructions + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '')  + content + '<|im_end|>\n' }}
+        {%- elif reasoning_instructions %}
+            {{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
+        {%- endif %}
+    {%- elif reasoning_instructions %}
+        {{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if preserve_thinking is undefined or preserve_thinking is true or loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined and tool_call.arguments != '' %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | string if args_value is string else args_value | tojson | safe %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

crates/neuron/tests/numerical_reference.rs

@@ -60,7 +60,9 @@ fn read_f32(path: &Path) -> Vec<f32> {
     let bytes = std::fs::read(path).unwrap_or_else(|e| panic!("read {path:?}: {e}"));
     assert!(bytes.len().is_multiple_of(4), "truncated f32 file {path:?}");
     bytes
-        .chunks_exact(4)
+        .as_chunks::<4>()
+        .0
+        .iter()
         .map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]]))
         .collect()
 }

crates/neuron/tests/preflight.rs

@@ -107,6 +107,7 @@ fn spec(model_id: &str, tp: Option<u32>, quant: Option<&str>) -> ModelSpec {
         tensor_parallel: tp,
         devices: None,
         sampling: None,
+        preserve_thinking: None,
     }
 }
 

crates/neuron/tests/qwen3_next_parity.rs

@@ -37,7 +37,9 @@ fn fixture_dir() -> PathBuf {
 fn read_f32(path: &Path) -> Vec<f32> {
     let bytes = std::fs::read(path).unwrap_or_else(|e| panic!("read {path:?}: {e}"));
     bytes
-        .chunks_exact(4)
+        .as_chunks::<4>()
+        .0
+        .iter()
         .map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]]))
         .collect()
 }

helexa-acp.example.toml

@@ -35,7 +35,7 @@ default_endpoint = "helexa"
 name = "helexa"
 base_url = "http://hanzalova.internal:31313/v1"
 wire_api = "openai-chat"
-default_model = "Qwen/Qwen3.6-27B"
+default_model = "Qwen/Qwen3.8-27B"
 max_tokens = 8192
 # Compaction kicks in when the rolling history grows past this token
 # budget. Set to your model's context window. Disable by removing

helexa-router.example.toml

@@ -23,7 +23,7 @@ listen = "0.0.0.0:8088"
 #
 # [aliases]
 # "helexa/small" = "Qwen/Qwen3-8B"
-# "helexa/balanced" = "Qwen/Qwen3.6-27B"
+# "helexa/balanced" = "Qwen/Qwen3.8-27B"
 
 # -- Downstream cortexes -------------------------------------------------
 # Each [[cortexes]] entry is an operator-run cortex the router may dispatch

helexa-router.toml

@@ -0,0 +1,37 @@
+# ─────────────────────────────────────────────────────────────────────
+# MANAGED BY CI — .gitea/workflows/deploy.yml is the only writer.
+#
+# Edits made directly to the deployed copy on gallumbits are reverted by
+# the next deploy, and until then the fleet runs something no commit
+# describes. Change it here, commit, let the pipeline ship it.
+# ─────────────────────────────────────────────────────────────────────
+# helexa-router — federation data plane on gallumbits.
+#
+# Tracked despite living beside the secret-bearing service configs,
+# because it holds no secrets: a listen address, the product-tier
+# aliases, and a mesh endpoint. Being untracked cost an outage on
+# 2026-08-27 — `helexa/balanced` still resolved to Qwen/Qwen3.6-27B
+# after that model had been retired from the catalogue, so every
+# authenticated web-chat turn evicted beast's resident flagship and
+# cold-loaded a model nobody had asked for. Nothing could diff the two
+# files, so nothing did.
+#
+# `script/check-alias-targets.py` now fails CI if any alias below names
+# a model that models.toml does not define.
+
+[router]
+listen = "0.0.0.0:8088"
+poll_interval_secs = 10
+region = "eu-central"
+
+# Product tiers → real model ids (#166). The SPA sends the tier names.
+[aliases]
+"helexa/small" = "Qwen/Qwen3-8B"
+"helexa/balanced" = "Qwen/Qwen3.8-27B"
+
+# hanzalova operator cortex, plaintext over the WireGuard mesh (the mesh
+# is the trust boundary; no TLS pin needed for http://).
+[[cortexes]]
+name = "hanzalova"
+endpoint = "http://hanzalova.internal:31313"
+region = "eu-central"

helexa.ai/content/docs/2-operating/02-cortex.md

@@ -44,6 +44,70 @@ change.
 VRAM fragmentation has had time to accumulate. It is a signal to restart
 that neuron, not an automatic action.
 
+## secrets.toml
+
+Credentials do not belong in the file CI deploys. Put them in a
+`secrets.toml` beside the main config and the service will merge it
+last, so its values win:
+
+```
+/etc/cortex/cortex.toml     structure — deployed by CI, safe to diff
+/etc/cortex/secrets.toml    credentials — yours, root-only, never in git
+```
+
+```toml
+# /etc/cortex/secrets.toml   (root:cortex 0640)
+[[entitlements.keys]]
+key = "sk-…"
+account_id = "acct-a"
+key_id = "a"
+
+[upstream]
+bearer = "…"
+```
+
+`helexa-upstream` and `helexa-angels` read a `secrets.toml` beside their
+own configs the same way — the DB URL, JWT secret and SMTP URL belong
+there.
+
+**An absent `secrets.toml` is not an error.** A single-file deployment
+keeps working exactly as before, so this is opt-in per host and can be
+adopted one service at a time.
+
+### Why bother
+
+Because a config CI cannot write is a config no check can defend, and
+the cost of that is not hypothetical. `helexa-router.toml` was kept out
+of git for sitting beside secret-bearing files. Nothing could diff it
+against the catalogue, so its `helexa/balanced` alias went on pointing
+at a model that had been retired — and since both models were
+deliberately given equal residency priority so they could displace each
+other, every authenticated chat turn evicted the resident flagship to
+cold-load the retired one. Four of those overlapped and the node ended
+up holding 32 GB of GPU memory with no model loaded and nothing
+serving.
+
+Splitting secrets from structure is what lets the structure be
+version-controlled, reviewed and checked. The ordering matters too: the
+operator's file merges last, so a deploy can never replace a live
+credential with a placeholder.
+
+### Moving existing secrets
+
+One-time, per host, and reversible — the service reads both files, so
+you can move a value and restart before moving the next:
+
+1. Create `secrets.toml` beside the config, owned `root:<service>` mode
+   `0640` (`0600` where the service runs as root).
+2. Move the credential blocks across. Leave everything else in place.
+3. Restart the service and confirm it still authenticates — for cortex,
+   a request with a real key should be billed rather than treated as
+   anonymous.
+4. Delete the moved lines from the main file only once step 3 passes.
+
+Until step 4 the value exists in both files, which is safe: the
+operator's copy wins.
+
 ## models.toml
 
 The catalogue says what each model needs and where it may run:
@@ -78,7 +142,7 @@ Clients should not hardcode model ids. Aliases let them ask for a role:
 [aliases]
 "helexa/small" = "Qwen/Qwen3-1.7B"
 "helexa/balanced" = "Qwen/Qwen3-8B"
-"helexa/large" = "Qwen/Qwen3.6-27B"
+"helexa/large" = "Qwen/Qwen3.8-27B"
 "helexa/image" = "Tongyi-MAI/Z-Image-Turbo"
 ```
 

helexa.ai/content/docs/2-operating/03-neurons.md

@@ -58,6 +58,32 @@ Two things that bite:
 - **Cold loads are disk-bound.** A 50 GB model off spinning rust is a
   very different experience from the same model on NVMe.
 
+### Where weights come from
+
+By default a bare `org/name` id is fetched from HuggingFace. Operators
+running a mirror, or serving models from more than one place, can
+declare sources explicitly:
+
+```toml
+[harness.candle]
+default_source = "huggingface"
+
+[harness.candle.sources.huggingface]
+endpoint = "https://huggingface.co"
+auth_env = "HF_TOKEN"          # env var name, never the token itself
+cache_dir = "/archive/llm"     # optional; per-source cache
+```
+
+`auth_env` names an environment variable rather than holding a token, so
+credentials stay out of the config file — worth keeping to even when the
+file is not obviously shared, because config is exactly what gets copied
+between hosts.
+
+Give a source its own `cache_dir` when two sources can serve the same
+`org/name`. The cache tree is keyed on the model id alone, so sharing a
+directory between sources that disagree about what `org/name` means will
+collide on disk.
+
 ## Default models
 
 Models listed as defaults load when the daemon activates, so the host
@@ -66,7 +92,7 @@ load cost:
 
 ```toml
 [[default_models]]
-model_id = "Qwen/Qwen3.6-27B"
+model_id = "Qwen/Qwen3.8-27B"
 harness = "candle"
 quant = "q6k"
 tensor_parallel = 2
@@ -91,10 +117,17 @@ Each loaded model has bounded admission control:
 [harness.candle.admission]
 max_in_flight = 8        # concurrent requests actually running
 max_queue_depth = 8      # waiting before rejection
-max_wait_secs = 30       # how long a queued request waits
+max_wait_secs = 30       # how long a queued request waits for a slot
+kv_max_wait_secs = 300   # how long it waits for enough KV budget
 max_per_principal = 2    # fair share, so one caller cannot monopolise
 ```
 
+`kv_max_wait_secs` is deliberately much longer than `max_wait_secs`.
+The first waits for a slot to turn over; the second waits for a whole
+long-context sequence to finish, which on a 27B can be minutes. A
+request that needs KV budget nobody has yet is not a request to reject
+in thirty seconds.
+
 Over-capacity requests are rejected promptly with `429`/`503` and a
 `Retry-After` rather than being allowed to queue invisibly. A fast
 rejection a client can retry is far better than a request that hangs
@@ -102,8 +135,34 @@ until something times out.
 
 `max_in_flight` above 1 enables batched decode on architectures that
 support it, so concurrent requests share a decode step instead of
-serialising. Raise it on big cards, leave it at 1 on small ones — the
-KV cache for several concurrent long contexts is itself substantial.
+serialising.
+
+**`max_in_flight` does not partition the KV budget.** KV is a shared
+pool, and each request reserves what its own prompt actually needs when
+it is admitted — which is why `kv_max_wait_secs` above exists at all. A
+request larger than what is currently free waits for bytes to come back
+rather than being handed a fixed 1/N slice, and a single long session
+can legitimately hold most of the pool.
+
+So the slot count and the context window are separate questions. What
+bounds a busy moment is the pool: divide `kv_budget_mb` by the model's
+KV cost per token (`GET /health` reports both) to see how many tokens
+are available to share. Many ordinary chat turns fit at once; a few very
+long agentic sessions will queue on bytes.
+
+Size `max_in_flight` against the concurrency you actually want to serve
+concurrently, not against context. Anything above it queues on admission
+with a `Retry-After` rather than being refused, so setting it too low
+converts capacity you have into latency your callers feel. Setting it
+very high mainly costs batch-step efficiency, since the engine pads to
+the widest active sequence.
+
+Measure before assuming you need a high number — but measure the traffic
+you will have, not the traffic you had. On one production host, seven
+days of spans showed 89.6% of busy time at a single request in flight;
+that host's chat tier was at the time being routed to a *different*
+model, and the figure stopped describing it the moment that was
+corrected.
 
 `max_per_principal` is what stops one busy client consuming the whole
 model.
@@ -150,7 +209,8 @@ reservation is doing its job, not that the model is overloaded.
 neuron refuses work it cannot finish rather than crashing partway:
 
 - a **static floor** of free VRAM below which prefill is refused
-  (`NEURON_MIN_FREE_VRAM_MB`, default 1500)
+  (`NEURON_MIN_FREE_VRAM_MB`, or `min_free_floor_mb` under
+  `[harness.candle.context_limit]`; default 1500)
 - a **length-aware check** that estimates the KV cache the request will
   actually need and refuses if it will not fit
 
@@ -159,15 +219,207 @@ can pass the static floor and still be impossible, because KV grows with
 prompt *and* generation length. See [context
 limits](/docs/operating/context-limits) for how the numbers relate.
 
+`activation_headroom_mb` reserves VRAM per card for prefill activations
+on top of the weights and the KV cache, before the context ceiling is
+computed. Raise it if prefill OOMs on a host whose steady-state
+occupancy looks fine — activations are transient and do not show up in
+the resident figure.
+
 A host that cannot currently serve a model reports that on `/health`,
 and cortex stops routing to it — an unservable replica is not a routing
 target.
 
+## Image models
+
+```toml
+[harness.candle.image]
+te_device = "cpu"     # where the text encoder runs
+te_resident = true    # keep it loaded between generations
+max_dim = 2048        # resolution ceiling per side
+```
+
+The text encoder is the placement decision. On CPU the ~8 GB encoder
+never touches the GPU — prompt encoding is one short forward and the
+features are tiny — which is what lets a 24 GB card serve a BF16 DiT at
+all. Move it to `cuda` only on a card with room to spare.
+
+`te_resident` trades memory for latency: resident costs ~16 GB of system
+RAM on CPU (or ~8 GB of VRAM on CUDA) and saves a few seconds per
+request. CPU defaults to resident, CUDA to rebuilding per request, which
+are the right defaults for where each is scarce.
+
+`max_dim` rejects oversized requests **before admission**, so an
+impossible resolution costs nothing. That matters more than it sounds:
+an OOM on the device poisons the worker context, so refusing early is
+the difference between one rejected request and a model that needs
+restarting.
+
 ## Prompt caps
 
 `NEURON_MAX_PROMPT_TOKENS` sets a hard ceiling; larger prompts are
 rejected before any GPU work, so a runaway client costs nothing.
 
+## Sampling
+
+Every model publishes its own sampling defaults in
+`generation_config.json`, and a caller can override them per request.
+Between those two sits an operator override, for the cases where the
+model's published defaults are wrong for how you serve it:
+
+```toml
+[[default_models]]
+model_id = "Qwen/Qwen3.8-27B"
+
+[default_models.sampling]
+temperature = 0.6
+presence_penalty = 1.0
+```
+
+Precedence is **request > operator > model > built-in**. Anything you
+leave unset changes nothing, so an empty table behaves exactly as no
+table at all.
+
+Set the same values in the gateway's `models.toml` under
+`[models.sampling]`. A model that is cold-loaded on demand gets its
+sampling from the catalogue, not from this file, so the two must agree
+or the same model samples differently depending on how it arrived —
+`script/check-config-consistency.py` fails the deploy if they diverge.
+
+### Which knob for which symptom
+
+| symptom | knob |
+|---|---|
+| Structurally malformed output — truncated JSON, unbalanced braces | `temperature` |
+| The think block restates the same passage over and over | `presence_penalty` |
+| Common tokens over-represented across a long answer | `frequency_penalty` |
+| Short immediate loops — "no, no, no…" | `repeat_penalty` |
+| You need two runs to be comparable | `seed` |
+
+**`temperature`** is the blunt one. Qwen3.8-27B publishes `1.0`, which
+measured a 20% structural-defect rate on long agentic turns against 0/60
+at `0.6`; the fleet therefore overrides it. Note that `0.6` is also what
+Qwen recommend for their thinking variants, so this is not fighting the
+model.
+
+**`presence_penalty`** and **`frequency_penalty`** are scored over the
+*whole* generated sequence. They are the ones that matter for reasoning
+models, and Qwen name `presence_penalty` (0–2, 1.5 when severe) as the
+remedy for the endless repetition their models fall into during long
+thinking. Both default to `0.0`, which is off.
+
+**`repeat_penalty`** (default `1.1`) and **`repeat_last_n`** (default
+`64`) are the older, *windowed* form: the penalty only sees the last
+`repeat_last_n` tokens. That is fine for chat and close to useless for a
+think block — a 20,000-token deliberation restating a passage 5,000
+tokens later is entirely outside a 64-token window. Measured across
+recorded sessions, **86% of think blocks over 60k characters contained
+verbatim repeated sentences**, one of them repeating 76% of its own
+content.
+
+Widening the window is the obvious move and the wrong one: the penalty
+is token-level, and source code legitimately repeats tokens constantly
+(`const`, `function`, brace runs), so a wide window at a meaningful
+penalty degrades exactly the output you care about. Reach for
+`presence_penalty` first — what recurs is passages, not tokens.
+
+**`seed`** pins the sampler's RNG for every request to this model. It
+exists because parameter search needs a control and the clients that
+would otherwise pin it often cannot. Leaving it set is a product
+decision, not just a testing one: every identical prompt then returns an
+identical answer to every caller, which is predictable and cacheable but
+removes the variation a user may expect from retrying.
+
+## Reasoning
+
+```toml
+[default_models.sampling]
+# nothing here — reasoning settings live in their own places, below
+```
+
+**`preserve_thinking`** controls whether prior turns keep their think
+blocks when the conversation is re-rendered. It is the model's own chat
+template control, not ours:
+
+```toml
+[[default_models]]
+preserve_thinking = false   # keep reasoning only for the turn in progress
+```
+
+Unset is the default and means the template decides. On Qwen3.8-27B that
+is `true` — every prior turn's reasoning is replayed. Full replay is
+what makes prompt growth track *total* completion rather than visible
+output: one measured session produced 28,672 reasoning tokens on turn 1
+and carried all of them into turn 2's 31,269-token prompt, at ~22 s of
+prefill on every turn thereafter.
+
+Setting it to `false` is a legitimate experiment, not a recommendation.
+Whether narrowing replay *helps* the model is unmeasured. Set it in both
+`models.toml` and this file, like sampling, and the value is reported on
+`/v1/models` and logged at load so a session can be attributed to the
+arm it ran under.
+
+**`reasoning_budget.answer_reserve_tokens`** (default `4096`) holds back
+part of the caller's output budget for the answer:
+
+```toml
+[harness.candle.reasoning_budget]
+answer_reserve_tokens = 4096
+```
+
+A reasoning model with no stopping criterion fills whatever budget it is
+given. Measured on Qwen3.8-27B: given 16,384 tokens it used 16,384;
+given 32,768 it used 32,768 — and both times returned no answer at all.
+The reserve bounds the think block at `max_tokens - answer_reserve` so
+the rest survives for the reply.
+
+Deliberately *not* derived from reasoning effort. Effort is a prompt
+instruction the model is trained to follow; making it also set a
+guillotine is how a model came to be told to reason at `xhigh` and then
+cut off at `medium`'s budget. Set `0` to disable, restoring unbounded
+reasoning.
+
+The trade to know: when the reserve fires, the model is interrupted
+mid-deliberation and its answer phase tends to state an *intention*
+rather than a conclusion. That is better than returning nothing, but it
+is not free.
+
+## The prefix cache
+
+Snapshots of previously-served prefixes, so a follow-up turn re-uses the
+KV cache instead of re-prefilling the whole conversation:
+
+```toml
+[harness.candle.prefix_cache]
+enabled = true
+budget_mb = 2048    # snapshot bytes per loaded model
+max_entries = 8     # live snapshots per model, regardless of budget
+```
+
+**Reach for `budget_mb` when `cached_tokens` stops growing mid-session.**
+That is the symptom: the prompt keeps growing, the cached prefix does
+not, and every turn re-prefills the difference. Observed on a 1 GiB
+default — `cached_tokens` pinned at 2,016 for ten consecutive turns
+while the prompt reached 52,869, so ~50k tokens were re-prefilled every turn
+at roughly 37 s each, against ~20 s of actual decode.
+
+The number is not free, and the direction of the trade is the opposite
+of what it looks like. **The snapshot budget is reserved before the KV
+budget, not from what is left over.** On a 2×32 GB host serving a 27B,
+measured 1:1 — the token column is the pool all concurrent requests
+share, not a per-slot allowance:
+
+| `budget_mb` | KV budget | tokens in the pool | holds a 53k snapshot |
+|---|---|---|---|
+| 1024 | 4713 MB | 150,800 | no |
+| 2048 | 3689 MB | 118,000 | yes |
+| 3072 | 2665 MB | 85,300 | yes, but a session cannot reach 53k |
+
+So raising it too far starves the KV cache and shortens how long a
+single session can run — the model can no longer reach the context whose
+snapshot the larger cache was meant to hold. Size it against the deepest
+session you actually serve, and check `kv_budget_mb` in the journal
+after a reload.
+
 ## Checking on it
 
 ```sh

helexa.ai/content/docs/2-operating/04-placement.md

@@ -24,7 +24,7 @@ confined anywhere.
 
 ```toml
 [[models]]
-id = "Qwen/Qwen3.6-27B"
+id = "Qwen/Qwen3.8-27B"
 harness = "candle"
 min_devices = 2
 min_device_vram_mb = 24000

helexa.ai/content/docs/2-operating/05-context-limits.md

@@ -32,6 +32,43 @@ daemon now does automatically.
 | `limit.input` | same | compaction trigger — opencode compacts to keep the prompt at/under this |
 | `limit.output` | same | generation reserve opencode leaves below the wall |
 
+### Tuning the derived limit
+
+Since #67 the neuron computes `limit` itself (see below), and three
+`[harness.candle.context_limit]` settings shape that computation:
+
+```toml
+[harness.candle.context_limit]
+target_prefill_latency_secs = 120     # longest acceptable prefill per turn
+bootstrap_prefill_tok_per_sec = 800   # cold-start estimate until measured
+output_reserve_tokens = 8192          # what a request gets when it names no cap
+max_output_tokens = 32768             # the largest output a request may name
+```
+
+**`target_prefill_latency_secs`** is the one that usually binds. The
+throughput ceiling is `target_prefill_latency_secs × measured
+prefill tok/s`, so on a host measuring ~1,400 tok/s a 120-second target
+advertises about 168k — above most hard ceilings, in which case it stops
+mattering. On a slower host, or one whose recent traffic has been small
+prompts, it is what caps the window.
+
+**`bootstrap_prefill_tok_per_sec`** applies until the model has measured
+its own rate, and stops the advertised window collapsing on a
+freshly-restarted host. Prefill samples below 512 tokens are ignored
+when computing that rate: a short prompt is dominated by fixed
+per-request cost and says nothing about throughput — the deploy's own
+19-token smoke probe reads as ~145 tok/s on a host that measures ~1,400.
+So a host that has served only small requests keeps the bootstrap
+estimate rather than adopting a misleading one, and no warm-up is
+needed after a restart.
+
+**`output_reserve_tokens`** is what a request that names no cap
+receives. **`max_output_tokens`** is the ceiling advertised to clients
+as `max_output_tokens`, and it is the *ceiling*, not the reserve —
+publishing the reserve tells every client that reads the advertisement
+to cap itself there, which on a reasoning model can be less than a
+single think block.
+
 ## How they must relate
 
 For a single model on a single neuron, all of these must hold:
@@ -60,9 +97,9 @@ Notes:
   lands, treat the VRAM-safe ceiling in rule 5 as a hard limit you set
   `NEURON_MAX_PROMPT_TOKENS` below, not something the daemon enforces.
 
-## VRAM cost of context (Qwen3.6-27B on beast)
+## VRAM cost of context (Qwen3.8-27B on beast)
 
-Qwen3.6-27B is a hybrid linear-attention model: of its 64 layers only
+Qwen3.8-27B is a hybrid linear-attention model: of its 64 layers only
 every 4th is full-attention (`full_attention_interval = 4` → **16**
 full-attn layers); the rest are `linear_attention` with constant-size
 recurrent state. KV cache grows **only** on the 16 full-attn layers
@@ -131,7 +168,7 @@ value wins over a hand override of the same variable. Use `local.conf`
 only for genuinely host-local, transient experiments — and remember a
 later deploy will re-assert `model.conf`.
 
-**opencode** — `opencode.json`, `provider.models."Qwen/Qwen3.6-27B".limit`:
+**opencode** — `opencode.json`, `provider.models."Qwen/Qwen3.8-27B".limit`:
 
 ```json
 { "context": 131072, "input": 122880, "output": 8192 }

helexa.ai/package-lock.json

@@ -36,7 +36,8 @@
         "globals": "^16.5.0",
         "typescript": "~5.9.3",
         "typescript-eslint": "^8.46.4",
-        "vite": "^7.2.0"
+        "vite": "^7.2.0",
+        "vitest": "^3.2.7"
       }
     },
     "node_modules/@babel/code-frame": {
@@ -1814,6 +1815,17 @@
         "@swc/counter": "^0.1.3"
       }
     },
+    "node_modules/@types/chai": {
+      "version": "5.2.3",
+      "resolved": "https://registry.npmjs.org/@types/chai/-/chai-5.2.3.tgz",
+      "integrity": "sha512-Mw558oeA9fFbv65/y4mHtXDs9bPnFMZAL/jxdPFUpOHHIXX91mcgEHbS5Lahr+pwZFR8A7GQleRWeI6cGFC2UA==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@types/deep-eql": "*",
+        "assertion-error": "^2.0.1"
+      }
+    },
     "node_modules/@types/debug": {
       "version": "4.1.13",
       "resolved": "https://registry.npmjs.org/@types/debug/-/debug-4.1.13.tgz",
@@ -1823,6 +1835,13 @@
         "@types/ms": "*"
       }
     },
+    "node_modules/@types/deep-eql": {
+      "version": "4.0.2",
+      "resolved": "https://registry.npmjs.org/@types/deep-eql/-/deep-eql-4.0.2.tgz",
+      "integrity": "sha512-c9h9dVVMigMPc4bwTvC5dxqtqJZwQPePsWjPlpSOnojbor6pGqdk541lfA7AqFQr5pB1BRdq0juY9db81BwyFw==",
+      "dev": true,
+      "license": "MIT"
+    },
     "node_modules/@types/estree": {
       "version": "1.0.9",
       "resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.9.tgz",
@@ -2243,6 +2262,121 @@
         "vite": "^4 || ^5 || ^6 || ^7 || ^8"
       }
     },
+    "node_modules/@vitest/expect": {
+      "version": "3.2.7",
+      "resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-3.2.7.tgz",
+      "integrity": "sha512-E8eBXaKibuvH2pSZErOjdVb5vF4PbKYcrnluBTYxEk1l/VhhwZg1kZQsdtjq+CsF5CFydf2Rdkz7jDHKSisi3w==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@types/chai": "^5.2.2",
+        "@vitest/spy": "3.2.7",
+        "@vitest/utils": "3.2.7",
+        "chai": "^5.2.0",
+        "tinyrainbow": "^2.0.0"
+      },
+      "funding": {
+        "url": "https://opencollective.com/vitest"
+      }
+    },
+    "node_modules/@vitest/mocker": {
+      "version": "3.2.7",
+      "resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-3.2.7.tgz",
+      "integrity": "sha512-Trr0hYO9CM3Wj6ksWHRhK9IZpIY6wTMO5u/MqXurMxT57sWBaOPEtP3Oq60ihZuh5JsiagKfz95OcxdEP6dBrA==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@vitest/spy": "3.2.7",
+        "estree-walker": "^3.0.3",
+        "magic-string": "^0.30.17"
+      },
+      "funding": {
+        "url": "https://opencollective.com/vitest"
+      },
+      "peerDependencies": {
+        "msw": "^2.4.9",
+        "vite": "^5.0.0 || ^6.0.0 || ^7.0.0-0"
+      },
+      "peerDependenciesMeta": {
+        "msw": {
+          "optional": true
+        },
+        "vite": {
+          "optional": true
+        }
+      }
+    },
+    "node_modules/@vitest/pretty-format": {
+      "version": "3.2.7",
+      "resolved": "https://registry.npmjs.org/@vitest/pretty-format/-/pretty-format-3.2.7.tgz",
+      "integrity": "sha512-KUHlwqVu0sRlhCdyPdQ/wBoTfRahjUky1MubOmYw9fWfIZy1gNoHpuaaQBPAaMaVYdQYHJLurzj8ECCj5OwTqA==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "tinyrainbow": "^2.0.0"
+      },
+      "funding": {
+        "url": "https://opencollective.com/vitest"
+      }
+    },
+    "node_modules/@vitest/runner": {
+      "version": "3.2.7",
+      "resolved": "https://registry.npmjs.org/@vitest/runner/-/runner-3.2.7.tgz",
+      "integrity": "sha512-sB9y4ovltoQP+WaUPwmSxO9WIg9Ig694Di5PalVPsYHklAdE027mehpWF2SQSVq+k6sFgaivbTjTJwZLSHbedA==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@vitest/utils": "3.2.7",
+        "pathe": "^2.0.3",
+        "strip-literal": "^3.0.0"
+      },
+      "funding": {
+        "url": "https://opencollective.com/vitest"
+      }
+    },
+    "node_modules/@vitest/snapshot": {
+      "version": "3.2.7",
+      "resolved": "https://registry.npmjs.org/@vitest/snapshot/-/snapshot-3.2.7.tgz",
+      "integrity": "sha512-7C+MwShwtBSI5Buwoyg3s/iY1eHL9PKAf+O1wVh/TdnjXUtkoL/9YQtre90i4MtNXM6edP1wJ2zOBpfCyhIS7g==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@vitest/pretty-format": "3.2.7",
+        "magic-string": "^0.30.17",
+        "pathe": "^2.0.3"
+      },
+      "funding": {
+        "url": "https://opencollective.com/vitest"
+      }
+    },
+    "node_modules/@vitest/spy": {
+      "version": "3.2.7",
+      "resolved": "https://registry.npmjs.org/@vitest/spy/-/spy-3.2.7.tgz",
+      "integrity": "sha512-Q2eQGI6d2L/hBtZ0qNuKcAGid68XK6cv1xsoaIma6PaJhHPoqcEJhYpXZ/5myCMqkNgtP6UKuBhbc0nHKnrkuQ==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "tinyspy": "^4.0.3"
+      },
+      "funding": {
+        "url": "https://opencollective.com/vitest"
+      }
+    },
+    "node_modules/@vitest/utils": {
+      "version": "3.2.7",
+      "resolved": "https://registry.npmjs.org/@vitest/utils/-/utils-3.2.7.tgz",
+      "integrity": "sha512-x6BDOd7dyo3PFLY3I9/HJ25X/6OurhGXk2/B9gOZNPF7XDVjeBK4k01lQE5uvDpbuheErh91qYuE1E2OEjK3Rw==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@vitest/pretty-format": "3.2.7",
+        "loupe": "^3.1.4",
+        "tinyrainbow": "^2.0.0"
+      },
+      "funding": {
+        "url": "https://opencollective.com/vitest"
+      }
+    },
     "node_modules/acorn": {
       "version": "8.17.0",
       "resolved": "https://registry.npmjs.org/acorn/-/acorn-8.17.0.tgz",
@@ -2318,6 +2452,16 @@
         "node": ">=10"
       }
     },
+    "node_modules/assertion-error": {
+      "version": "2.0.1",
+      "resolved": "https://registry.npmjs.org/assertion-error/-/assertion-error-2.0.1.tgz",
+      "integrity": "sha512-Izi8RQcffqCeNVgFigKli1ssklIbpHnCYc6AknXGYoB6grJqyeby7jv12JUQgmTAnIDnbck1uxksT4dzN3PWBA==",
+      "dev": true,
+      "license": "MIT",
+      "engines": {
+        "node": ">=12"
+      }
+    },
     "node_modules/bail": {
       "version": "2.0.2",
       "resolved": "https://registry.npmjs.org/bail/-/bail-2.0.2.tgz",
@@ -2412,6 +2556,16 @@
         "node": "^6 || ^7 || ^8 || ^9 || ^10 || ^11 || ^12 || >=13.7"
       }
     },
+    "node_modules/cac": {
+      "version": "6.7.14",
+      "resolved": "https://registry.npmjs.org/cac/-/cac-6.7.14.tgz",
+      "integrity": "sha512-b6Ilus+c3RrdDk+JhLKUAQfzzgLEPy6wcXqS7f/xe1EETvsDP6GORG7SFuOs6cID5YkqchW/LXZbX5bc8j7ZcQ==",
+      "dev": true,
+      "license": "MIT",
+      "engines": {
+        "node": ">=8"
+      }
+    },
     "node_modules/callsites": {
       "version": "3.1.0",
       "resolved": "https://registry.npmjs.org/callsites/-/callsites-3.1.0.tgz",
@@ -2453,6 +2607,23 @@
         "url": "https://github.com/sponsors/wooorm"
       }
     },
+    "node_modules/chai": {
+      "version": "5.3.3",
+      "resolved": "https://registry.npmjs.org/chai/-/chai-5.3.3.tgz",
+      "integrity": "sha512-4zNhdJD/iOjSH0A05ea+Ke6MU5mmpQcbQsSOkgdaUMJ9zTlDTD/GYlwohmIE2u0gaxHYiVHEn1Fw9mZ/ktJWgw==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "assertion-error": "^2.0.1",
+        "check-error": "^2.1.1",
+        "deep-eql": "^5.0.1",
+        "loupe": "^3.1.0",
+        "pathval": "^2.0.0"
+      },
+      "engines": {
+        "node": ">=18"
+      }
+    },
     "node_modules/chalk": {
       "version": "4.1.2",
       "resolved": "https://registry.npmjs.org/chalk/-/chalk-4.1.2.tgz",
@@ -2510,6 +2681,16 @@
         "url": "https://github.com/sponsors/wooorm"
       }
     },
+    "node_modules/check-error": {
+      "version": "2.1.3",
+      "resolved": "https://registry.npmjs.org/check-error/-/check-error-2.1.3.tgz",
+      "integrity": "sha512-PAJdDJusoxnwm1VwW07VWwUN1sl7smmC3OKggvndJFadxxDRyFJBX/ggnu/KE4kQAB7a3Dp8f/YXC1FlUprWmA==",
+      "dev": true,
+      "license": "MIT",
+      "engines": {
+        "node": ">= 16"
+      }
+    },
     "node_modules/classnames": {
       "version": "2.5.1",
       "resolved": "https://registry.npmjs.org/classnames/-/classnames-2.5.1.tgz",
@@ -2633,6 +2814,16 @@
         "url": "https://github.com/sponsors/wooorm"
       }
     },
+    "node_modules/deep-eql": {
+      "version": "5.0.2",
+      "resolved": "https://registry.npmjs.org/deep-eql/-/deep-eql-5.0.2.tgz",
+      "integrity": "sha512-h5k/5U50IJJFpzfL6nO9jaaumfjO/f2NjK/oYB2Djzm4p9L+3T9qWpZqZ2hAbLPuuYq9wrU08WQyBTL5GbPk5Q==",
+      "dev": true,
+      "license": "MIT",
+      "engines": {
+        "node": ">=6"
+      }
+    },
     "node_modules/deep-is": {
       "version": "0.1.4",
       "resolved": "https://registry.npmjs.org/deep-is/-/deep-is-0.1.4.tgz",
@@ -2695,6 +2886,13 @@
       "dev": true,
       "license": "ISC"
     },
+    "node_modules/es-module-lexer": {
+      "version": "1.7.0",
+      "resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-1.7.0.tgz",
+      "integrity": "sha512-jEQoCwk8hyb2AZziIOLhDqpm5+2ww5uIE6lkO/6jcOCusfk6LhMHpXXfBLXTZ7Ydyt0j4VoUQv6uGNYbdW+kBA==",
+      "dev": true,
+      "license": "MIT"
+    },
     "node_modules/esbuild": {
       "version": "0.27.7",
       "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.27.7.tgz",
@@ -2944,6 +3142,16 @@
         "url": "https://opencollective.com/unified"
       }
     },
+    "node_modules/estree-walker": {
+      "version": "3.0.3",
+      "resolved": "https://registry.npmjs.org/estree-walker/-/estree-walker-3.0.3.tgz",
+      "integrity": "sha512-7RUKfXgSMMkzt6ZuXmqapOurLGPPfgj6l9uRZ7lRGolvk0y2yocc35LdcxKC5PQZdn2DMqioAQ2NoWcrTKmm6g==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@types/estree": "^1.0.0"
+      }
+    },
     "node_modules/esutils": {
       "version": "2.0.3",
       "resolved": "https://registry.npmjs.org/esutils/-/esutils-2.0.3.tgz",
@@ -2954,6 +3162,16 @@
         "node": ">=0.10.0"
       }
     },
+    "node_modules/expect-type": {
+      "version": "1.4.0",
+      "resolved": "https://registry.npmjs.org/expect-type/-/expect-type-1.4.0.tgz",
+      "integrity": "sha512-KfYbmpRm0VbLjEvVa9yGwCi9GI34xvi7A/HXYWQO65CSD2u3MczUJSuwXKFIxlGsgBQizV9q5J9NHj4VG0n+pA==",
+      "dev": true,
+      "license": "Apache-2.0",
+      "engines": {
+        "node": ">=12.0.0"
+      }
+    },
     "node_modules/extend": {
       "version": "3.0.2",
       "resolved": "https://registry.npmjs.org/extend/-/extend-3.0.2.tgz",
@@ -3539,6 +3757,13 @@
         "loose-envify": "cli.js"
       }
     },
+    "node_modules/loupe": {
+      "version": "3.2.1",
+      "resolved": "https://registry.npmjs.org/loupe/-/loupe-3.2.1.tgz",
+      "integrity": "sha512-CdzqowRJCeLU72bHvWqwRBBlLcMEtIvGrlvef74kMnV2AolS9Y8xUv1I0U/MNAWMhBlKIoyuEgoJ0t/bbwHbLQ==",
+      "dev": true,
+      "license": "MIT"
+    },
     "node_modules/lowlight": {
       "version": "3.3.0",
       "resolved": "https://registry.npmjs.org/lowlight/-/lowlight-3.3.0.tgz",
@@ -3564,6 +3789,16 @@
         "yallist": "^3.0.2"
       }
     },
+    "node_modules/magic-string": {
+      "version": "0.30.21",
+      "resolved": "https://registry.npmjs.org/magic-string/-/magic-string-0.30.21.tgz",
+      "integrity": "sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@jridgewell/sourcemap-codec": "^1.5.5"
+      }
+    },
     "node_modules/markdown-table": {
       "version": "3.0.4",
       "resolved": "https://registry.npmjs.org/markdown-table/-/markdown-table-3.0.4.tgz",
@@ -4591,6 +4826,23 @@
         "node": ">=8"
       }
     },
+    "node_modules/pathe": {
+      "version": "2.0.3",
+      "resolved": "https://registry.npmjs.org/pathe/-/pathe-2.0.3.tgz",
+      "integrity": "sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w==",
+      "dev": true,
+      "license": "MIT"
+    },
+    "node_modules/pathval": {
+      "version": "2.0.1",
+      "resolved": "https://registry.npmjs.org/pathval/-/pathval-2.0.1.tgz",
+      "integrity": "sha512-//nshmD55c46FuFw26xV/xFAaB5HF9Xdap7HJBBnrKdAd6/GxDBaNA1870O79+9ueg61cZLSVc+OaFlfmObYVQ==",
+      "dev": true,
+      "license": "MIT",
+      "engines": {
+        "node": ">= 14.16"
+      }
+    },
     "node_modules/picocolors": {
       "version": "1.1.1",
       "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz",
@@ -5096,6 +5348,13 @@
         "node": ">=8"
       }
     },
+    "node_modules/siginfo": {
+      "version": "2.0.0",
+      "resolved": "https://registry.npmjs.org/siginfo/-/siginfo-2.0.0.tgz",
+      "integrity": "sha512-ybx0WO1/8bSBLEWXZvEd7gMW3Sn3JFlW3TvX1nREbDLRNQNaeNN8WK0meBwPdAaOI7TtRRRJn/Es1zhrrCHu7g==",
+      "dev": true,
+      "license": "ISC"
+    },
     "node_modules/source-map-js": {
       "version": "1.2.1",
       "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.1.tgz",
@@ -5116,6 +5375,20 @@
         "url": "https://github.com/sponsors/wooorm"
       }
     },
+    "node_modules/stackback": {
+      "version": "0.0.2",
+      "resolved": "https://registry.npmjs.org/stackback/-/stackback-0.0.2.tgz",
+      "integrity": "sha512-1XMJE5fQo1jGH6Y/7ebnwPOBEkIEnT4QF32d5R1+VXdXveM0IBMJt8zfaxX1P3QhVwrYe+576+jkANtSS2mBbw==",
+      "dev": true,
+      "license": "MIT"
+    },
+    "node_modules/std-env": {
+      "version": "3.10.0",
+      "resolved": "https://registry.npmjs.org/std-env/-/std-env-3.10.0.tgz",
+      "integrity": "sha512-5GS12FdOZNliM5mAOxFRg7Ir0pWz8MdpYm6AY6VPkGpbA7ZzmbzNcBJQ0GPvvyWgcY7QAhCgf9Uy89I03faLkg==",
+      "dev": true,
+      "license": "MIT"
+    },
     "node_modules/stringify-entities": {
       "version": "4.0.4",
       "resolved": "https://registry.npmjs.org/stringify-entities/-/stringify-entities-4.0.4.tgz",
@@ -5143,6 +5416,26 @@
         "url": "https://github.com/sponsors/sindresorhus"
       }
     },
+    "node_modules/strip-literal": {
+      "version": "3.1.0",
+      "resolved": "https://registry.npmjs.org/strip-literal/-/strip-literal-3.1.0.tgz",
+      "integrity": "sha512-8r3mkIM/2+PpjHoOtiAW8Rg3jJLHaV7xPwG+YRGrv6FP0wwk/toTpATxWYOW0BKdWwl82VT2tFYi5DlROa0Mxg==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "js-tokens": "^9.0.1"
+      },
+      "funding": {
+        "url": "https://github.com/sponsors/antfu"
+      }
+    },
+    "node_modules/strip-literal/node_modules/js-tokens": {
+      "version": "9.0.1",
+      "resolved": "https://registry.npmjs.org/js-tokens/-/js-tokens-9.0.1.tgz",
+      "integrity": "sha512-mxa9E9ITFOt0ban3j6L5MpjwegGz6lBQmM1IJkWeBZGcMxto50+eWdjC/52xDbS2vy0k7vIMK0Fe2wfL9OQSpQ==",
+      "dev": true,
+      "license": "MIT"
+    },
     "node_modules/style-to-js": {
       "version": "1.1.21",
       "resolved": "https://registry.npmjs.org/style-to-js/-/style-to-js-1.1.21.tgz",
@@ -5174,6 +5467,20 @@
         "node": ">=8"
       }
     },
+    "node_modules/tinybench": {
+      "version": "2.9.0",
+      "resolved": "https://registry.npmjs.org/tinybench/-/tinybench-2.9.0.tgz",
+      "integrity": "sha512-0+DUvqWMValLmha6lr4kD8iAMK1HzV0/aKnCtWb9v9641TnP/MFb7Pc2bxoxQjTXAErryXVgUOfv2YqNllqGeg==",
+      "dev": true,
+      "license": "MIT"
+    },
+    "node_modules/tinyexec": {
+      "version": "0.3.2",
+      "resolved": "https://registry.npmjs.org/tinyexec/-/tinyexec-0.3.2.tgz",
+      "integrity": "sha512-KQQR9yN7R5+OSwaK0XQoj22pwHoTlgYqmUscPYoknOoWCWfj/5/ABTMRi69FrKU5ffPVh5QcFikpWJI/P1ocHA==",
+      "dev": true,
+      "license": "MIT"
+    },
     "node_modules/tinyglobby": {
       "version": "0.2.17",
       "resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.17.tgz",
@@ -5191,6 +5498,36 @@
         "url": "https://github.com/sponsors/SuperchupuDev"
       }
     },
+    "node_modules/tinypool": {
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/tinypool/-/tinypool-1.1.1.tgz",
+      "integrity": "sha512-Zba82s87IFq9A9XmjiX5uZA/ARWDrB03OHlq+Vw1fSdt0I+4/Kutwy8BP4Y/y/aORMo61FQ0vIb5j44vSo5Pkg==",
+      "dev": true,
+      "license": "MIT",
+      "engines": {
+        "node": "^18.0.0 || >=20.0.0"
+      }
+    },
+    "node_modules/tinyrainbow": {
+      "version": "2.0.0",
+      "resolved": "https://registry.npmjs.org/tinyrainbow/-/tinyrainbow-2.0.0.tgz",
+      "integrity": "sha512-op4nsTR47R6p0vMUUoYl/a+ljLFVtlfaXkLQmqfLR1qHma1h/ysYk4hEXZ880bf2CYgTskvTa/e196Vd5dDQXw==",
+      "dev": true,
+      "license": "MIT",
+      "engines": {
+        "node": ">=14.0.0"
+      }
+    },
+    "node_modules/tinyspy": {
+      "version": "4.0.4",
+      "resolved": "https://registry.npmjs.org/tinyspy/-/tinyspy-4.0.4.tgz",
+      "integrity": "sha512-azl+t0z7pw/z958Gy9svOTuzqIk6xq+NSheJzn5MMWtWTFywIacg2wUlzKFGtt3cthx0r2SxMK0yzJOR0IES7Q==",
+      "dev": true,
+      "license": "MIT",
+      "engines": {
+        "node": ">=14.0.0"
+      }
+    },
     "node_modules/trim-lines": {
       "version": "3.0.1",
       "resolved": "https://registry.npmjs.org/trim-lines/-/trim-lines-3.0.1.tgz",
@@ -5557,6 +5894,102 @@
         }
       }
     },
+    "node_modules/vite-node": {
+      "version": "3.2.4",
+      "resolved": "https://registry.npmjs.org/vite-node/-/vite-node-3.2.4.tgz",
+      "integrity": "sha512-EbKSKh+bh1E1IFxeO0pg1n4dvoOTt0UDiXMd/qn++r98+jPO1xtJilvXldeuQ8giIB5IkpjCgMleHMNEsGH6pg==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "cac": "^6.7.14",
+        "debug": "^4.4.1",
+        "es-module-lexer": "^1.7.0",
+        "pathe": "^2.0.3",
+        "vite": "^5.0.0 || ^6.0.0 || ^7.0.0-0"
+      },
+      "bin": {
+        "vite-node": "vite-node.mjs"
+      },
+      "engines": {
+        "node": "^18.0.0 || ^20.0.0 || >=22.0.0"
+      },
+      "funding": {
+        "url": "https://opencollective.com/vitest"
+      }
+    },
+    "node_modules/vitest": {
+      "version": "3.2.7",
+      "resolved": "https://registry.npmjs.org/vitest/-/vitest-3.2.7.tgz",
+      "integrity": "sha512-KrxIJ62Fd89gfysR4WotlgZABiz2dqFPgqGzX7s+CwsqLFomRH7777ZcrOD6+WVAh7khPQP41A+BKbpcJFrdEg==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@types/chai": "^5.2.2",
+        "@vitest/expect": "3.2.7",
+        "@vitest/mocker": "3.2.7",
+        "@vitest/pretty-format": "^3.2.7",
+        "@vitest/runner": "3.2.7",
+        "@vitest/snapshot": "3.2.7",
+        "@vitest/spy": "3.2.7",
+        "@vitest/utils": "3.2.7",
+        "chai": "^5.2.0",
+        "debug": "^4.4.1",
+        "expect-type": "^1.2.1",
+        "magic-string": "^0.30.17",
+        "pathe": "^2.0.3",
+        "picomatch": "^4.0.2",
+        "std-env": "^3.9.0",
+        "tinybench": "^2.9.0",
+        "tinyexec": "^0.3.2",
+        "tinyglobby": "^0.2.14",
+        "tinypool": "^1.1.1",
+        "tinyrainbow": "^2.0.0",
+        "vite": "^5.0.0 || ^6.0.0 || ^7.0.0-0",
+        "vite-node": "3.2.4",
+        "why-is-node-running": "^2.3.0"
+      },
+      "bin": {
+        "vitest": "vitest.mjs"
+      },
+      "engines": {
+        "node": "^18.0.0 || ^20.0.0 || >=22.0.0"
+      },
+      "funding": {
+        "url": "https://opencollective.com/vitest"
+      },
+      "peerDependencies": {
+        "@edge-runtime/vm": "*",
+        "@types/debug": "^4.1.12",
+        "@types/node": "^18.0.0 || ^20.0.0 || >=22.0.0",
+        "@vitest/browser": "3.2.7",
+        "@vitest/ui": "3.2.7",
+        "happy-dom": "*",
+        "jsdom": "*"
+      },
+      "peerDependenciesMeta": {
+        "@edge-runtime/vm": {
+          "optional": true
+        },
+        "@types/debug": {
+          "optional": true
+        },
+        "@types/node": {
+          "optional": true
+        },
+        "@vitest/browser": {
+          "optional": true
+        },
+        "@vitest/ui": {
+          "optional": true
+        },
+        "happy-dom": {
+          "optional": true
+        },
+        "jsdom": {
+          "optional": true
+        }
+      }
+    },
     "node_modules/void-elements": {
       "version": "3.1.0",
       "resolved": "https://registry.npmjs.org/void-elements/-/void-elements-3.1.0.tgz",
@@ -5591,6 +6024,23 @@
         "node": ">= 8"
       }
     },
+    "node_modules/why-is-node-running": {
+      "version": "2.3.0",
+      "resolved": "https://registry.npmjs.org/why-is-node-running/-/why-is-node-running-2.3.0.tgz",
+      "integrity": "sha512-hUrmaWBdVDcxvYqnyh09zunKzROWjbZTiNy8dBEjkS7ehEDQibXJ7XvlmtbwuTclUiIyN+CyXQD4Vmko8fNm8w==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "siginfo": "^2.0.0",
+        "stackback": "0.0.2"
+      },
+      "bin": {
+        "why-is-node-running": "cli.js"
+      },
+      "engines": {
+        "node": ">=8"
+      }
+    },
     "node_modules/word-wrap": {
       "version": "1.2.5",
       "resolved": "https://registry.npmjs.org/word-wrap/-/word-wrap-1.2.5.tgz",

helexa.ai/package.json

@@ -9,6 +9,8 @@
     "preview": "vite preview",
     "lint": "eslint .",
     "typecheck": "tsc -b",
+    "test": "vitest run",
+    "test:watch": "vitest",
     "i18n:check": "node ./scripts/check-i18n-keys.mjs",
     "i18n:meta": "node ./scripts/check-i18n-metadata.mjs",
     "i18n:lang-labels": "node ./scripts/check-i18n-lang-labels.mjs",
@@ -43,6 +45,7 @@
     "globals": "^16.5.0",
     "typescript": "~5.9.3",
     "typescript-eslint": "^8.46.4",
-    "vite": "^7.2.0"
+    "vite": "^7.2.0",
+    "vitest": "^3.2.7"
   }
 }

helexa.ai/src/App.css

@@ -2063,3 +2063,108 @@ html[dir="rtl"] .hx-landing-composer textarea {
         display: none;
     }
 }
+
+/* Assistant working — reasoning and tool calls (#TurnBlocks).
+ *
+ * Deliberately quieter than the answer: muted text, no bubble of its
+ * own, hairline separators. The whole point is that it can be present
+ * without competing, so it reads as an annotation on the reply rather
+ * than as more reply. */
+.hx-blocks {
+    display: flex;
+    flex-direction: column;
+    gap: 0.25rem;
+    margin-bottom: 0.5rem;
+}
+.hx-block {
+    border-left: 2px solid var(--color-border-subtle);
+    padding-left: 0.5rem;
+}
+.hx-block-open {
+    border-left-color: var(--color-accent-soft);
+}
+.hx-block-head {
+    display: flex;
+    align-items: center;
+    gap: 0.35rem;
+    width: 100%;
+    padding: 0.15rem 0;
+    background: none;
+    border: 0;
+    color: var(--color-text-muted);
+    font-size: 0.78rem;
+    text-align: start;
+    cursor: pointer;
+}
+.hx-block-head:hover {
+    color: var(--color-text);
+}
+.hx-block-caret {
+    flex: none;
+    transition: transform 0.12s ease;
+}
+.hx-block-caret-open {
+    transform: rotate(90deg);
+}
+.hx-block-label {
+    flex: none;
+    font-weight: 600;
+}
+.hx-block-running {
+    flex: none;
+    opacity: 0.7;
+    font-style: italic;
+}
+/* The peek is the first thing to give up space — it is a hint, and a
+ * wrapped one would cost the header its single-line shape. */
+.hx-block-peek {
+    flex: 1 1 auto;
+    min-width: 0;
+    overflow: hidden;
+    text-overflow: ellipsis;
+    white-space: nowrap;
+    opacity: 0.6;
+}
+.hx-block-body {
+    /* Capped and scrollable: a long think block would otherwise push the
+     * answer off-screen, which defeats the point of showing it. */
+    max-height: 16rem;
+    overflow-y: auto;
+    margin: 0.15rem 0 0.35rem;
+    padding-inline-end: 0.4rem;
+    color: var(--color-text-muted);
+    font-size: 0.82rem;
+    line-height: 1.45;
+    white-space: pre-wrap;
+    overflow-wrap: anywhere;
+}
+
+/* Reasoning toggle in the composer. Sits with the send button rather
+ * than in a settings panel: it changes what the next message does, so it
+ * belongs where the message is written. */
+.hx-thinking-toggle {
+    display: inline-flex;
+    align-items: center;
+    gap: 0.35rem;
+    margin: 0;
+    padding: 0 0.5rem;
+    color: var(--color-text-muted);
+    font-size: 0.78rem;
+    white-space: nowrap;
+    cursor: pointer;
+    user-select: none;
+}
+.hx-thinking-toggle:hover {
+    color: var(--color-text);
+}
+.hx-thinking-toggle input {
+    margin: 0;
+    cursor: pointer;
+}
+/* Narrow screens: the composer is already tight, and the label is the
+ * expendable half — the checkbox still reads as a control. */
+@media (max-width: 30rem) {
+    .hx-thinking-toggle span {
+        display: none;
+    }
+}

helexa.ai/src/components/TurnBlocks.tsx

@@ -0,0 +1,129 @@
+import { useState } from "react";
+import { useTranslation } from "react-i18next";
+import { FaChevronRight, FaBrain, FaWrench } from "react-icons/fa6";
+import type { TurnBlock } from "../data/db";
+import { isOpen } from "../lib/turnBlockState";
+
+/**
+ * The assistant's working — reasoning and tool calls — shown above its
+ * answer.
+ *
+ * Two things are in tension. All of this should be *available*, because a
+ * model that reasons for two minutes and then answers has done work worth
+ * inspecting, and a tool call the user cannot see is a claim they cannot
+ * check. But none of it is what they asked for, so it must not bury the
+ * answer.
+ *
+ * The resolution is that visibility follows activity:
+ *
+ *   * While a block is the live one — still streaming, no answer text yet
+ *     — it is open. That span is otherwise dead air; a spinner would say
+ *     "working", this says what it is working *on*.
+ *   * The moment answer text starts, every block collapses to a one-line
+ *     header. The thing the user asked for takes the space.
+ *   * Completed turns load collapsed, so scrollback stays readable.
+ *
+ * A block the user opens or closes by hand stays that way. Explicit
+ * intent outranks the automatic rule, including for the live block —
+ * someone who closes a noisy reasoning stream should not have it reopen
+ * itself on the next token.
+ */
+export function TurnBlocks({
+  blocks,
+  streaming,
+  answerStarted,
+}: {
+  blocks: TurnBlock[];
+  /** This turn is still being generated. */
+  streaming: boolean;
+  /** Answer text has begun arriving, so the working is now history. */
+  answerStarted: boolean;
+}) {
+  const { t } = useTranslation();
+  // Only blocks the user has touched appear here; everything else follows
+  // the automatic rule. Keyed by index, which is stable because blocks
+  // are only ever appended within a turn.
+  const [overrides, setOverrides] = useState<Record<number, boolean>>({});
+
+  if (blocks.length === 0) return null;
+
+  return (
+    <div className="hx-blocks">
+      {blocks.map((b, i) => {
+        const open = isOpen(i, blocks.length, streaming, answerStarted, overrides[i]);
+        const body = b.kind === "reasoning" ? b.text : blockToolBody(b);
+        return (
+          <div key={i} className={`hx-block${open ? " hx-block-open" : ""}`}>
+            <button
+              type="button"
+              className="hx-block-head"
+              aria-expanded={open}
+              onClick={() => setOverrides((o) => ({ ...o, [i]: !open }))}
+            >
+              <FaChevronRight
+                className={`hx-block-caret${open ? " hx-block-caret-open" : ""}`}
+                size={10}
+                aria-hidden="true"
+              />
+              {b.kind === "reasoning" ? (
+                <FaBrain size={11} aria-hidden="true" />
+              ) : (
+                <FaWrench size={11} aria-hidden="true" />
+              )}
+              <span className="hx-block-label">
+                {b.kind === "reasoning"
+                  ? t("chat:blockReasoning")
+                  : t("chat:blockTool", { name: b.name })}
+              </span>
+              {/* A pending tool call is the one case where the header
+                * alone should say something is happening: its body is
+                * empty until the result lands, so an open block would
+                * otherwise look like a stall. */}
+              {b.kind === "tool" && b.result === undefined && streaming && (
+                <span className="hx-block-running">{t("chat:blockRunning")}</span>
+              )}
+              {!open && (
+                <span className="hx-block-peek">{peek(body)}</span>
+              )}
+            </button>
+            {open && <div className="hx-block-body">{body}</div>}
+          </div>
+        );
+      })}
+    </div>
+  );
+}
+
+/** Arguments, then the result once it arrives. */
+function blockToolBody(b: Extract<TurnBlock, { kind: "tool" }>): string {
+  const args = prettyArgs(b.args);
+  return b.result === undefined ? args : `${args}\n\n${b.result}`;
+}
+
+/**
+ * Tool arguments as sent, pretty-printed when they are JSON.
+ *
+ * Models emit malformed arguments often enough that this must never
+ * throw — a UI that blanks because the model produced bad JSON hides the
+ * very thing worth seeing.
+ */
+function prettyArgs(raw: string): string {
+  try {
+    return JSON.stringify(JSON.parse(raw), null, 2);
+  } catch {
+    return raw;
+  }
+}
+
+/**
+ * The tail of a collapsed block, so a closed row still carries a hint of
+ * its content.
+ *
+ * The tail rather than the head: while a block streams, the end is where
+ * the model currently is. A head-anchored preview freezes on the first
+ * few words and stops telling you anything.
+ */
+function peek(text: string): string {
+  const flat = text.replace(/\s+/g, " ").trim();
+  return flat.length <= 80 ? flat : `…${flat.slice(-79)}`;
+}

helexa.ai/src/data/db.ts

@@ -39,6 +39,23 @@ export interface MessageSource {
   url: string;
 }
 
+/**
+ * Work the assistant did on the way to an answer: a stretch of reasoning,
+ * or one tool call and its result.
+ *
+ * Kept as an ordered list rather than two separate fields because the
+ * order is the story — a turn can reason, call a tool, reason about what
+ * came back, and call another. Flattening that into "the reasoning" and
+ * "the tool calls" would lose which thought preceded which call.
+ *
+ * `text` and `result` grow while streaming, so a block is also its own
+ * progress indicator: a reasoning block with text and no following block
+ * is what the model is doing *right now*.
+ */
+export type TurnBlock =
+  | { kind: "reasoning"; text: string }
+  | { kind: "tool"; name: string; args: string; result?: string };
+
 export interface Message {
   id: string;
   conversationId: string;
@@ -50,6 +67,16 @@ export interface Message {
   promptTokens?: number;
   completionTokens?: number;
   sources?: MessageSource[];
+  /**
+   * Reasoning and tool calls, in the order they happened. Absent on user
+   * turns and on assistant turns that answered directly.
+   *
+   * Not indexed, so no schema version bump: Dexie stores the whole
+   * object and only the declared indexes constrain it. Messages written
+   * before this field existed simply have none, which renders as an
+   * answer with no working shown — correct, since none was captured.
+   */
+  blocks?: TurnBlock[];
 }
 
 /**

helexa.ai/src/data/repositories.ts

@@ -10,6 +10,7 @@ import {
   type Message,
   type MessageRole,
   type Project,
+  type TurnBlock,
 } from "./db";
 
 function uuid(): string {
@@ -138,10 +139,22 @@ export async function setMessageContent(id: string, content: string): Promise<vo
   await db.messages.update(id, { content });
 }
 
+/**
+ * Write the whole block list as an absolute snapshot.
+ *
+ * Same discipline as `setMessageContent`: the caller owns the array and
+ * writes it entire, because a read-modify-write append races itself when
+ * deltas arrive faster than the IndexedDB round-trip — which is how the
+ * content path dropped tokens before it was written this way.
+ */
+export async function setMessageBlocks(id: string, blocks: TurnBlock[]): Promise<void> {
+  await db.messages.update(id, { blocks });
+}
+
 export async function finalizeMessage(
   id: string,
   patch: Partial<
-    Pick<Message, "status" | "errorCode" | "promptTokens" | "completionTokens" | "sources">
+    Pick<Message, "status" | "errorCode" | "promptTokens" | "completionTokens" | "sources" | "blocks">
   >,
 ): Promise<void> {
   await db.messages.update(id, patch);

helexa.ai/src/i18n/resources/am/chat.json

@@ -29,5 +29,10 @@
   "reading": "{{host}} እየተነበበ ነው…",
   "copyCode": "ኮድ ቅዳ",
   "copiedCode": "ተቀድቷል",
-  "anonNearLimit": "ወደ የስም-አልባ ተጠቃሚ ገደብ እየተቃረቡ ነው። ነፃ ድልድል ለማግኘት ይመዝገቡ።"
+  "anonNearLimit": "ወደ የስም-አልባ ተጠቃሚ ገደብ እየተቃረቡ ነው። ነፃ ድልድል ለማግኘት ይመዝገቡ።",
+  "blockReasoning": "በማሰብ ላይ",
+  "blockTool": "{{name}}",
+  "blockRunning": "በማስኬድ ላይ…",
+  "showThinking": "ማመዛዘን",
+  "showThinkingHint": "ሞዴሉ ከመመለሱ በፊት እንዲያስብ ጠይቅ፣ እና ማመዛዘኑን አሳይ"
 }

helexa.ai/src/i18n/resources/ar/chat.json

@@ -29,5 +29,10 @@
   "reading": "تجري قراءة {{host}}…",
   "copyCode": "نسخ الكود",
   "copiedCode": "تم النسخ",
-  "anonNearLimit": "أوشكت على بلوغ حد الاستخدام المجهول. أنشئ حسابًا للحصول على مخصّص مجاني."
+  "anonNearLimit": "أوشكت على بلوغ حد الاستخدام المجهول. أنشئ حسابًا للحصول على مخصّص مجاني.",
+  "blockReasoning": "جارٍ التفكير",
+  "blockTool": "{{name}}",
+  "blockRunning": "قيد التشغيل…",
+  "showThinking": "التفكير",
+  "showThinkingHint": "اطلب من النموذج التفكير قبل الإجابة، واعرض تفكيره"
 }

helexa.ai/src/i18n/resources/bg/chat.json

@@ -29,5 +29,10 @@
   "reading": "четене на {{host}}…",
   "copyCode": "Копиране на кода",
   "copiedCode": "Копирано",
-  "anonNearLimit": "Наближавате анонимния лимит. Регистрирайте се за безплатна квота."
+  "anonNearLimit": "Наближавате анонимния лимит. Регистрирайте се за безплатна квота.",
+  "blockReasoning": "Разсъждаване",
+  "blockTool": "{{name}}",
+  "blockRunning": "изпълнява се…",
+  "showThinking": "Разсъждение",
+  "showThinkingHint": "Моделът да мисли преди да отговори и да показва разсъжденията си"
 }

helexa.ai/src/i18n/resources/ca/chat.json

@@ -29,5 +29,10 @@
   "reading": "s'està llegint {{host}}…",
   "copyCode": "Copia el codi",
   "copiedCode": "Copiat",
-  "anonNearLimit": "Us acosteu al límit anònim. Registreu-vos per obtenir una assignació gratuïta."
+  "anonNearLimit": "Us acosteu al límit anònim. Registreu-vos per obtenir una assignació gratuïta.",
+  "blockReasoning": "Pensant",
+  "blockTool": "{{name}}",
+  "blockRunning": "en execució…",
+  "showThinking": "Raonament",
+  "showThinkingHint": "Demana al model que pensi abans de respondre i mostra'n el raonament"
 }

helexa.ai/src/i18n/resources/cs/chat.json

@@ -29,5 +29,10 @@
   "reading": "čtení {{host}}…",
   "copyCode": "Kopírovat kód",
   "copiedCode": "Zkopírováno",
-  "anonNearLimit": "Blížíte se k anonymnímu limitu. Zaregistrujte se a získejte bezplatný příděl."
+  "anonNearLimit": "Blížíte se k anonymnímu limitu. Zaregistrujte se a získejte bezplatný příděl.",
+  "blockReasoning": "Přemýšlení",
+  "blockTool": "{{name}}",
+  "blockRunning": "probíhá…",
+  "showThinking": "Uvažování",
+  "showThinkingHint": "Ať model před odpovědí přemýšlí a ukáže své uvažování"
 }

helexa.ai/src/i18n/resources/da/chat.json

@@ -29,5 +29,10 @@
   "reading": "læser {{host}}…",
   "copyCode": "Kopiér kode",
   "copiedCode": "Kopieret",
-  "anonNearLimit": "Du nærmer dig den anonyme grænse. Opret en konto og få en gratis tildeling."
+  "anonNearLimit": "Du nærmer dig den anonyme grænse. Opret en konto og få en gratis tildeling.",
+  "blockReasoning": "Tænker",
+  "blockTool": "{{name}}",
+  "blockRunning": "kører…",
+  "showThinking": "Ræsonnement",
+  "showThinkingHint": "Bed modellen tænke før den svarer, og vis dens ræsonnement"
 }

helexa.ai/src/i18n/resources/de/chat.json

@@ -29,5 +29,10 @@
   "reading": "{{host}} wird gelesen…",
   "copyCode": "Code kopieren",
   "copiedCode": "Kopiert",
-  "anonNearLimit": "Sie nähern sich dem anonymen Limit. Registrieren Sie sich für ein kostenloses Kontingent."
+  "anonNearLimit": "Sie nähern sich dem anonymen Limit. Registrieren Sie sich für ein kostenloses Kontingent.",
+  "blockReasoning": "Denkt nach",
+  "blockTool": "{{name}}",
+  "blockRunning": "läuft…",
+  "showThinking": "Denkprozess",
+  "showThinkingHint": "Das Modell vor der Antwort nachdenken lassen und den Denkprozess anzeigen"
 }

helexa.ai/src/i18n/resources/el/chat.json

@@ -29,5 +29,10 @@
   "reading": "ανάγνωση του {{host}}…",
   "copyCode": "Αντιγραφή κώδικα",
   "copiedCode": "Αντιγράφηκε",
-  "anonNearLimit": "Πλησιάζετε το όριο ανώνυμης χρήσης. Εγγραφείτε για μια δωρεάν κατανομή."
+  "anonNearLimit": "Πλησιάζετε το όριο ανώνυμης χρήσης. Εγγραφείτε για μια δωρεάν κατανομή.",
+  "blockReasoning": "Σκέφτεται",
+  "blockTool": "{{name}}",
+  "blockRunning": "εκτελείται…",
+  "showThinking": "Συλλογισμός",
+  "showThinkingHint": "Ζήτα από το μοντέλο να σκεφτεί πριν απαντήσει και δείξε τον συλλογισμό του"
 }

helexa.ai/src/i18n/resources/en/chat.json

@@ -29,5 +29,10 @@
   "reading": "reading {{host}}…",
   "copyCode": "Copy code",
   "copiedCode": "Copied",
-  "anonNearLimit": "You're close to the anonymous limit. Sign up for a free allocation."
+  "anonNearLimit": "You're close to the anonymous limit. Sign up for a free allocation.",
+  "blockReasoning": "Thinking",
+  "blockTool": "{{name}}",
+  "blockRunning": "running…",
+  "showThinking": "Reasoning",
+  "showThinkingHint": "Ask the model to think before answering, and show its reasoning"
 }

helexa.ai/src/i18n/resources/es/chat.json

@@ -29,5 +29,10 @@
   "reading": "leyendo {{host}}…",
   "copyCode": "Copiar código",
   "copiedCode": "Copiado",
-  "anonNearLimit": "Estás cerca del límite anónimo. Regístrate para obtener una asignación gratuita."
+  "anonNearLimit": "Estás cerca del límite anónimo. Regístrate para obtener una asignación gratuita.",
+  "blockReasoning": "Pensando",
+  "blockTool": "{{name}}",
+  "blockRunning": "ejecutando…",
+  "showThinking": "Razonamiento",
+  "showThinkingHint": "Pide al modelo que piense antes de responder y muestra su razonamiento"
 }

helexa.ai/src/i18n/resources/et/chat.json

@@ -29,5 +29,10 @@
   "reading": "loetakse {{host}}…",
   "copyCode": "Kopeeri kood",
   "copiedCode": "Kopeeritud",
-  "anonNearLimit": "Oled lähenemas anonüümse kasutuse piirile. Registreeru tasuta eraldise saamiseks."
+  "anonNearLimit": "Oled lähenemas anonüümse kasutuse piirile. Registreeru tasuta eraldise saamiseks.",
+  "blockReasoning": "Mõtleb",
+  "blockTool": "{{name}}",
+  "blockRunning": "töötab…",
+  "showThinking": "Arutlus",
+  "showThinkingHint": "Palu mudelil enne vastamist mõelda ja näita tema arutlust"
 }

helexa.ai/src/i18n/resources/fa/chat.json

@@ -29,5 +29,10 @@
   "reading": "در حال خواندن {{host}}…",
   "copyCode": "کپی کد",
   "copiedCode": "کپی شد",
-  "anonNearLimit": "به محدودیت کاربران ناشناس نزدیک می‌شوید. برای دریافت سهمیه رایگان ثبت‌نام کنید."
+  "anonNearLimit": "به محدودیت کاربران ناشناس نزدیک می‌شوید. برای دریافت سهمیه رایگان ثبت‌نام کنید.",
+  "blockReasoning": "در حال تفکر",
+  "blockTool": "{{name}}",
+  "blockRunning": "در حال اجرا…",
+  "showThinking": "استدلال",
+  "showThinkingHint": "از مدل بخواهید پیش از پاسخ فکر کند و استدلالش را نشان دهد"
 }

helexa.ai/src/i18n/resources/fi/chat.json

@@ -29,5 +29,10 @@
   "reading": "luetaan {{host}}…",
   "copyCode": "Kopioi koodi",
   "copiedCode": "Kopioitu",
-  "anonNearLimit": "Lähestyt anonyymin käytön rajaa. Rekisteröidy saadaksesi ilmaisen kiintiön."
+  "anonNearLimit": "Lähestyt anonyymin käytön rajaa. Rekisteröidy saadaksesi ilmaisen kiintiön.",
+  "blockReasoning": "Ajattelee",
+  "blockTool": "{{name}}",
+  "blockRunning": "suoritetaan…",
+  "showThinking": "Päättely",
+  "showThinkingHint": "Pyydä mallia ajattelemaan ennen vastaamista ja näytä sen päättely"
 }

helexa.ai/src/i18n/resources/fr/chat.json

@@ -29,5 +29,10 @@
   "reading": "lecture de {{host}}…",
   "copyCode": "Copier le code",
   "copiedCode": "Copié",
-  "anonNearLimit": "Vous approchez de la limite anonyme. Inscrivez-vous pour obtenir une allocation gratuite."
+  "anonNearLimit": "Vous approchez de la limite anonyme. Inscrivez-vous pour obtenir une allocation gratuite.",
+  "blockReasoning": "Réflexion",
+  "blockTool": "{{name}}",
+  "blockRunning": "en cours…",
+  "showThinking": "Raisonnement",
+  "showThinkingHint": "Demander au modèle de réfléchir avant de répondre et afficher son raisonnement"
 }

helexa.ai/src/i18n/resources/ga/chat.json

@@ -29,5 +29,10 @@
   "reading": "ag léamh {{host}}…",
   "copyCode": "Cóipeáil an cód",
   "copiedCode": "Cóipeáilte",
-  "anonNearLimit": "Tá tú ag druidim leis an teorainn gan ainm. Cláraigh le haghaidh leithdháileadh saor in aisce."
+  "anonNearLimit": "Tá tú ag druidim leis an teorainn gan ainm. Cláraigh le haghaidh leithdháileadh saor in aisce.",
+  "blockReasoning": "Ag machnamh",
+  "blockTool": "{{name}}",
+  "blockRunning": "ar siúl…",
+  "showThinking": "Réasúnaíocht",
+  "showThinkingHint": "Iarr ar an tsamhail machnamh a dhéanamh sula bhfreagraíonn sí, agus taispeáin é"
 }

helexa.ai/src/i18n/resources/ha/chat.json

@@ -29,5 +29,10 @@
   "reading": "ana karanta {{host}}…",
   "copyCode": "Kwafi lambar",
   "copiedCode": "An kwafa",
-  "anonNearLimit": "Ka kusa kai iyakar amfani ba tare da shiga ba. Yi rajista don samun rabo kyauta."
+  "anonNearLimit": "Ka kusa kai iyakar amfani ba tare da shiga ba. Yi rajista don samun rabo kyauta.",
+  "blockReasoning": "Yana tunani",
+  "blockTool": "{{name}}",
+  "blockRunning": "yana gudana…",
+  "showThinking": "Tunani",
+  "showThinkingHint": "Ka nemi samfurin ya yi tunani kafin amsa, kuma a nuna tunaninsa"
 }

helexa.ai/src/i18n/resources/he/chat.json

@@ -29,5 +29,10 @@
   "reading": "מתבצעת קריאה של {{host}}…",
   "copyCode": "העתקת הקוד",
   "copiedCode": "הועתק",
-  "anonNearLimit": "אתם מתקרבים למגבלת השימוש האנונימי. הירשמו לקבלת הקצאה חינם."
+  "anonNearLimit": "אתם מתקרבים למגבלת השימוש האנונימי. הירשמו לקבלת הקצאה חינם.",
+  "blockReasoning": "חושב",
+  "blockTool": "{{name}}",
+  "blockRunning": "פועל…",
+  "showThinking": "חשיבה",
+  "showThinkingHint": "בקש מהמודל לחשוב לפני שהוא עונה, והצג את החשיבה שלו"
 }

helexa.ai/src/i18n/resources/hr/chat.json

@@ -29,5 +29,10 @@
   "reading": "čitanje {{host}}…",
   "copyCode": "Kopiraj kôd",
   "copiedCode": "Kopirano",
-  "anonNearLimit": "Približavate se anonimnom limitu. Registrirajte se za besplatnu kvotu."
+  "anonNearLimit": "Približavate se anonimnom limitu. Registrirajte se za besplatnu kvotu.",
+  "blockReasoning": "Razmišljanje",
+  "blockTool": "{{name}}",
+  "blockRunning": "izvodi se…",
+  "showThinking": "Zaključivanje",
+  "showThinkingHint": "Zatraži da model razmisli prije odgovora i prikaži njegovo zaključivanje"
 }

helexa.ai/src/i18n/resources/hu/chat.json

@@ -29,5 +29,10 @@
   "reading": "{{host}} olvasása…",
   "copyCode": "Kód másolása",
   "copiedCode": "Másolva",
-  "anonNearLimit": "Közeledsz az anonim limithez. Regisztrálj ingyenes keretért."
+  "anonNearLimit": "Közeledsz az anonim limithez. Regisztrálj ingyenes keretért.",
+  "blockReasoning": "Gondolkodik",
+  "blockTool": "{{name}}",
+  "blockRunning": "fut…",
+  "showThinking": "Gondolatmenet",
+  "showThinkingHint": "Kérd meg a modellt, hogy gondolkodjon válasz előtt, és mutasd a gondolatmenetét"
 }

helexa.ai/src/i18n/resources/it/chat.json

@@ -29,5 +29,10 @@
   "reading": "lettura di {{host}} in corso…",
   "copyCode": "Copia il codice",
   "copiedCode": "Copiato",
-  "anonNearLimit": "Ti stai avvicinando al limite anonimo. Registrati per un’allocazione gratuita."
+  "anonNearLimit": "Ti stai avvicinando al limite anonimo. Registrati per un’allocazione gratuita.",
+  "blockReasoning": "Sto pensando",
+  "blockTool": "{{name}}",
+  "blockRunning": "in esecuzione…",
+  "showThinking": "Ragionamento",
+  "showThinkingHint": "Chiedi al modello di ragionare prima di rispondere e mostra il ragionamento"
 }

helexa.ai/src/i18n/resources/ka/chat.json

@@ -29,5 +29,10 @@
   "reading": "მიმდინარეობს {{host}}-ის კითხვა…",
   "copyCode": "კოდის კოპირება",
   "copiedCode": "დაკოპირდა",
-  "anonNearLimit": "უახლოვდები ანონიმურ ლიმიტს. დარეგისტრირდი უფასო კვოტისთვის."
+  "anonNearLimit": "უახლოვდები ანონიმურ ლიმიტს. დარეგისტრირდი უფასო კვოტისთვის.",
+  "blockReasoning": "ფიქრობს",
+  "blockTool": "{{name}}",
+  "blockRunning": "მიმდინარეობს…",
+  "showThinking": "მსჯელობა",
+  "showThinkingHint": "სთხოვე მოდელს იფიქროს პასუხამდე და აჩვენე მისი მსჯელობა"
 }

helexa.ai/src/i18n/resources/kk/chat.json

@@ -29,5 +29,10 @@
   "reading": "{{host}} оқылуда…",
   "copyCode": "Кодты көшіру",
   "copiedCode": "Көшірілді",
-  "anonNearLimit": "Анонимді шекке жақындап қалдыңыз. Тегін квота алу үшін тіркеліңіз."
+  "anonNearLimit": "Анонимді шекке жақындап қалдыңыз. Тегін квота алу үшін тіркеліңіз.",
+  "blockReasoning": "Ойлануда",
+  "blockTool": "{{name}}",
+  "blockRunning": "орындалуда…",
+  "showThinking": "Пайымдау",
+  "showThinkingHint": "Жауап бермес бұрын модель ойлансын және пайымдауын көрсетсін"
 }

helexa.ai/src/i18n/resources/lt/chat.json

@@ -29,5 +29,10 @@
   "reading": "skaitoma {{host}}…",
   "copyCode": "Kopijuoti kodą",
   "copiedCode": "Nukopijuota",
-  "anonNearLimit": "Artėjate prie anoniminio naudojimo ribos. Užsiregistruokite ir gaukite nemokamą kvotą."
+  "anonNearLimit": "Artėjate prie anoniminio naudojimo ribos. Užsiregistruokite ir gaukite nemokamą kvotą.",
+  "blockReasoning": "Mąsto",
+  "blockTool": "{{name}}",
+  "blockRunning": "vykdoma…",
+  "showThinking": "Samprotavimas",
+  "showThinkingHint": "Paprašyk modelio pamąstyti prieš atsakant ir rodyk jo samprotavimą"
 }

helexa.ai/src/i18n/resources/lv/chat.json

@@ -29,5 +29,10 @@
   "reading": "lasa {{host}}…",
   "copyCode": "Kopēt kodu",
   "copiedCode": "Nokopēts",
-  "anonNearLimit": "Tuvojaties anonīmās lietošanas limitam. Reģistrējieties bezmaksas piešķīrumam."
+  "anonNearLimit": "Tuvojaties anonīmās lietošanas limitam. Reģistrējieties bezmaksas piešķīrumam.",
+  "blockReasoning": "Domā",
+  "blockTool": "{{name}}",
+  "blockRunning": "notiek…",
+  "showThinking": "Spriešana",
+  "showThinkingHint": "Lūdz modelim padomāt pirms atbildes un rādi tā spriešanu"
 }

helexa.ai/src/i18n/resources/ma/chat.json

@@ -29,5 +29,10 @@
   "reading": "كنقراو {{host}}…",
   "copyCode": "نسخ الكود",
   "copiedCode": "تنسخ",
-  "anonNearLimit": "قربتي توصل للحد ديال الاستعمال بلا حساب. سجّل باش تاخد حصة بلاش."
+  "anonNearLimit": "قربتي توصل للحد ديال الاستعمال بلا حساب. سجّل باش تاخد حصة بلاش.",
+  "blockReasoning": "كيفكر",
+  "blockTool": "{{name}}",
+  "blockRunning": "كيخدم…",
+  "showThinking": "التفكير",
+  "showThinkingHint": "طلب من الموديل يفكر قبل ما يجاوب، وبيّن التفكير ديالو"
 }

helexa.ai/src/i18n/resources/mt/chat.json

@@ -29,5 +29,10 @@
   "reading": "qed naqra {{host}}…",
   "copyCode": "Ikkopja l-kodiċi",
   "copiedCode": "Ikkupjat",
-  "anonNearLimit": "Qed toqrob lejn il-limitu anonimu. Irreġistra għal allokazzjoni b'xejn."
+  "anonNearLimit": "Qed toqrob lejn il-limitu anonimu. Irreġistra għal allokazzjoni b'xejn.",
+  "blockReasoning": "Qed jaħseb",
+  "blockTool": "{{name}}",
+  "blockRunning": "għaddej…",
+  "showThinking": "Raġunament",
+  "showThinkingHint": "Itlob lill-mudell jaħseb qabel iwieġeb, u uri r-raġunament tiegħu"
 }

helexa.ai/src/i18n/resources/nl/chat.json

@@ -29,5 +29,10 @@
   "reading": "lezen van {{host}}…",
   "copyCode": "Code kopiëren",
   "copiedCode": "Gekopieerd",
-  "anonNearLimit": "Je nadert de anonieme limiet. Registreer je voor een gratis tegoed."
+  "anonNearLimit": "Je nadert de anonieme limiet. Registreer je voor een gratis tegoed.",
+  "blockReasoning": "Denkt na",
+  "blockTool": "{{name}}",
+  "blockRunning": "bezig…",
+  "showThinking": "Redenering",
+  "showThinkingHint": "Laat het model nadenken vóór het antwoordt en toon de redenering"
 }