fix(neuron): truncated-mid-think non-streaming responses are all reasoning (#112) #113

Merged

grenade merged 1 commits from fix/112-prompt-opened-reasoning into main 2026-07-02 09:04:51 +00:00

Conversation 0 Commits 1 Files Changed 1 +51 -18

grenade commented 2026-07-02 09:04:40 +00:00

Owner

Copy Link

Closes #112

When the chat template force-opens the think block in the generation prompt (Qwen3-Next-80B-A3B-Thinking ends with <|im_start|>assistant\n<think>\n) and generation exhausts max_tokens before emitting </think>, the non-streaming split_off_reasoning had no close-marker anchor and returned the entire chain-of-thought as content. The streaming path was already correct (prompt_opens_reasoning initializes its state machine); this threads the same signal into the non-streaming split for both the single-GPU and TP paths: no close marker + prompt-opened block ⇒ content empty, every generated token counted as reasoning_tokens.

Six unit tests cover the split matrix, including the new prompt-opened truncation case.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TczcGF7JSjJs8r15RSSGpx

Closes #112 When the chat template force-opens the think block in the generation prompt (`Qwen3-Next-80B-A3B-Thinking` ends with `<|im_start|>assistant\n<think>\n`) and generation exhausts `max_tokens` before emitting `</think>`, the non-streaming `split_off_reasoning` had no close-marker anchor and returned the entire chain-of-thought as `content`. The streaming path was already correct (`prompt_opens_reasoning` initializes its state machine); this threads the same signal into the non-streaming split for both the single-GPU and TP paths: no close marker + prompt-opened block ⇒ `content` empty, every generated token counted as `reasoning_tokens`. Six unit tests cover the split matrix, including the new prompt-opened truncation case. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01TczcGF7JSjJs8r15RSSGpx

grenade added 1 commit 2026-07-02 09:04:40 +00:00

fix(neuron): truncated-mid-think non-streaming responses are all reasoning (#112) ... 41ec05e40c

When the chat template force-opens the think block in the generation
prompt (Qwen3-Next-80B-A3B-Thinking ends with
'<|im_start|>assistant\n<think>\n') and the generation exhausts
max_tokens before emitting </think>, the non-streaming
split_off_reasoning had no close-marker anchor and returned the whole
chain-of-thought as content. Thread the existing
prompt_opens_reasoning result (already used by the streaming state
machine) into the split: no close marker + prompt-opened block means
content is empty and every generated token counts as reasoning.

Closes #112

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TczcGF7JSjJs8r15RSSGpx

grenade merged commit f611e3479b into main 2026-07-02 09:04:51 +00:00

grenade deleted branch fix/112-prompt-opened-reasoning 2026-07-02 09:04:51 +00:00

grenade referenced this issue from a commit 2026-07-02 09:04:53 +00:00

Merge pull request 'fix(neuron): truncated-mid-think non-streaming responses are all reasoning (#112)' (#113) from fix/112-prompt-opened-reasoning into main

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

out-of-scope

Outside helexa's constraining principles: near-frontier models on consumer hardware, served transparently and predictably

p1-now

Active pain or unblocking work — pick up first

p2-next

Engine levers — start once P1 baselines exist

p3-later

Dependent on earlier work, or demand-driven

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

grenade

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: helexa/helexa#113