rob thijssen
cabec1d08a
All checks were successful
CI / Format (push) Successful in 40s
CI / CUDA type-check (push) Successful in 1m38s
CI / Clippy (push) Successful in 2m15s
CI / Test (push) Successful in 6m28s
CI / Build cortex SRPM (push) Has been skipped
CI / Publish cortex to COPR (push) Has been skipped
CI / Build neuron SRPM (push) Has been skipped
CI / Publish neuron to COPR (push) Has been skipped
CI / Bump version in source (push) Has been skipped
The true-streaming SSE passthrough (Body::from_stream, no full-response buffering, with chunk-observation hooks) was cortex-only. helexa-router (#69) needs the same mechanism to proxy a chat-completions/messages stream verbatim to a selected cortex. Extract it once. New `crates/helexa-stream` owns the *mechanism* (kept HTTP-free cortex-core untouched — it would have forced axum/reqwest/futures onto every cortex-core consumer): - `forward_streaming(client, url, headers, body, observer)` — POST and stream the response back chunk-for-chunk; status-agnostic, so a non-2xx (e.g. cortex 429) is passed through with status+headers intact (the #69 backpressure-passthrough requirement). - `ChunkObserver` trait + `ObservedStream` wrapper — feeds each chunk to the observer, calls `finish` exactly once on clean end or on drop (client disconnect). - `BodyTail` (bounded tail accumulator) + `last_count_for` (trailing OpenAI `usage` extraction) — the reusable pieces an observer uses. cortex keeps its *policy*: `proxy.rs` now supplies a `CortexMetrics` observer (per-request token metrics + per-principal reservation settle), its logging contract, and the error envelope, driving the shared mechanism. `proxy::last_count_for` is re-exported so `handlers`/ `anthropic_sse` call sites are unchanged. No behaviour change — the existing cortex `streaming.rs` tests pass as-is. helexa-stream tests prove chunk-for-chunk incremental delivery, observer finish-once, usage extraction, and non-2xx passthrough. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F6o3ddqmYNh9kzdwq6eowh
22 lines
431 B
TOML
22 lines
431 B
TOML
[package]
|
|
name = "helexa-stream"
|
|
version.workspace = true
|
|
edition.workspace = true
|
|
license.workspace = true
|
|
repository.workspace = true
|
|
|
|
[lib]
|
|
name = "helexa_stream"
|
|
path = "src/lib.rs"
|
|
|
|
[dependencies]
|
|
axum = { workspace = true }
|
|
reqwest = { workspace = true }
|
|
futures = { workspace = true }
|
|
thiserror = { workspace = true }
|
|
|
|
[dev-dependencies]
|
|
tokio = { workspace = true }
|
|
tokio-stream = { workspace = true }
|
|
async-stream = "0.3"
|