rob thijssen fc9a8c42a3 feat(neuron): extract <tool_call> blocks to structured tool_calls deltas ...

Closes #6.

Same model-agnostic seam as #8 but for tool-call markers
(`<tool_call>` / `</tool_call>` on Qwen3-Coder, Hermes-format,
DeepSeek-Coder, gpt-oss, …). Lets Zed's tool-use feature and any
other vanilla OpenAI chat client get structured `tool_calls` deltas
out of cortex without having to parse markers themselves.

## Implementation

1. **Tokenizer probe at load time** (`detect_tool_call_token_pair`
   in `wire::event`) — same shape as the reasoning-marker probe
   from #8. Both open AND close must resolve to single token ids;
   non-tool-use models get `None` and pass through unchanged.
   Stored on `LoadedModel.tool_call_tokens` and the TP analogue.

2. **New `InferenceEvent::ToolCall` variant** — carries `index`
   (call slot, per-turn counter), generated `id` (`call_<hex>_<idx>`),
   `name`, and the complete `arguments` JSON string. One event per
   parsed call.

3. **Token-level state machine** in all three streaming paths
   (CPU `run_inference_streaming`, CUDA single-GPU
   `stream_inference_via_worker`, CUDA TP `chat_completion_tp_stream`)
   layered on top of #8's reasoning routing:
   - `<tool_call>` token → enter buffering state, clear buffer.
   - Tokens while buffering → accumulate into `tool_call_buf`
     via the decoder (so multi-byte UTF-8 still buffers correctly)
     without emitting anything visible.
   - `</tool_call>` token → take the buffer, parse with
     `parse_tool_call_body` (extract `name` + `arguments`),
     emit a structured `ToolCall` event with a fresh `call_<hex>`
     id and the parsed fields.
   - On parse failure → fall back to re-emitting the original
     `<tool_call>{buf}</tool_call>` block as plain text content
     so helexa-acp's existing `ToolCallParser` repair passes still
     have a chance to recover the call.

4. **OpenAI chat projector** emits the OpenAI streaming
   `tool_calls` delta shape on `InferenceEvent::ToolCall` —
   `{tool_calls: [{index, id, type:"function",
   function:{name, arguments}}]}`. One chunk per call slot.

5. **OpenAI Responses projector** drops `ToolCall` events for
   now (Responses-side function_call event family routing tracked
   under #7); the chat path is what unblocks Zed's tool use today.

## Acceptance

- Vanilla OpenAI chat clients (Zed's tool-use feature, any other
  OpenAI-compatible tool-call consumer) get structured tool_calls
  deltas against cortex+neuron without having to parse `<tool_call>`
  markers in content.
- helexa-acp continues to work — when neuron parses cleanly, it
  consumes the structured deltas through its existing decoder.
  When the model emits malformed JSON, neuron falls back to text
  pass-through and helexa-acp's `ToolCallParser` recovers via the
  same path it always did.
- Models without tool-call markers in their tokenizer pass through
  unchanged.
- No hardcoded model knowledge — entirely driven by tokenizer
  metadata.

## Tests

2 new detection tests in `wire::event` (Qwen3-style marker
detection, no-marker case). The streaming paths themselves stay
covered by the existing chat-completions integration tests; full
end-to-end exercise of the new path requires GPU-loaded models
and lives outside the CI test surface.

215 workspace tests pass; clippy + fmt clean across the
workspace.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-05-31 23:26:31 +03:00

..

cortex-cli

feat(neuron): OpenAI-compatible non-streaming chat completion

2026-05-18 16:47:58 +03:00

cortex-core

feat(neuron): OpenAI Responses API + ci cuda-check runner label

2026-05-31 11:13:44 +03:00

cortex-gateway

feat(cortex-gateway): proxy /v1/responses to neuron

2026-05-31 11:21:43 +03:00

helexa-acp

feat(neuron): strip reasoning from chat completions by default

2026-05-31 17:55:04 +03:00

neuron

feat(neuron): extract <tool_call> blocks to structured tool_calls deltas

2026-05-31 23:26:31 +03:00