rob thijssen
7984d27553
All checks were successful
CI / Format (push) Successful in 40s
CI / CUDA type-check (push) Successful in 1m37s
CI / Clippy (push) Successful in 2m16s
CI / Test (push) Successful in 4m53s
CI / Build cortex SRPM (push) Has been skipped
CI / Build neuron SRPM (push) Has been skipped
CI / Publish cortex to COPR (push) Has been skipped
CI / Publish neuron to COPR (push) Has been skipped
CI / Bump version in source (push) Has been skipped
The router's data path. Wires the topology poller (#72) and the shared streaming proxy (#71) into real request routing. - `dispatch.rs`: `select_cortexes(model)` ranks reachable cortexes that can serve the model, best-first — loaded/warm before cold-loadable, region match before not, more healthy nodes before fewer, name for determinism. `dispatch()` extracts `model`, picks candidates, and forwards via `helexa_stream::forward_streaming` (bearer + bytes verbatim, SSE streamed back). Cortex's #63 rejections (429/400/…) pass through untouched; transport failures fail over to the next candidate; a genuine HTTP response — any status — is returned as-is, never retried away. - Router-originated rejections use the #63 envelope: 404 model_not_found (no operator serves it), 503 service_unavailable + Retry-After (known but all unreachable / all candidates failed to connect), 400 missing_model_field. `error.rs` is the router's envelope→axum adapter (mirrors cortex-gateway's). - `handlers.rs`: `/v1/chat/completions`, `/v1/completions`, `/v1/responses`, `/v1/messages` dispatch to the same path on a chosen cortex. The router holds zero entitlement logic — routes on capacity, not budget. - Config: optional `region` on the router and per-cortex for geo affinity. Tests (`dispatch.rs`): routes to a serving cortex + forwards the bearer; cortex 429 passes through and is NOT retried; transport failure fails over to a live cortex; unknown→404, known-but-unreachable→503, missing-model→400; ranking order (warm/region/headroom). 7 new, existing skeleton/topology suites unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F6o3ddqmYNh9kzdwq6eowh
36 lines
832 B
TOML
36 lines
832 B
TOML
[package]
|
|
name = "helexa-router"
|
|
version.workspace = true
|
|
edition.workspace = true
|
|
license.workspace = true
|
|
repository.workspace = true
|
|
|
|
[[bin]]
|
|
name = "helexa-router"
|
|
path = "src/main.rs"
|
|
|
|
[lib]
|
|
name = "helexa_router"
|
|
path = "src/lib.rs"
|
|
|
|
[dependencies]
|
|
cortex-core = { workspace = true }
|
|
helexa-stream = { path = "../helexa-stream" }
|
|
|
|
tokio = { workspace = true }
|
|
axum = { workspace = true }
|
|
tower-http = { workspace = true }
|
|
reqwest = { workspace = true }
|
|
serde = { workspace = true }
|
|
serde_json = { workspace = true }
|
|
figment = { workspace = true }
|
|
anyhow = { workspace = true }
|
|
clap = { workspace = true }
|
|
tracing = { workspace = true }
|
|
tracing-subscriber = { workspace = true }
|
|
chrono = { workspace = true }
|
|
|
|
[dev-dependencies]
|
|
# Jail (isolated cwd + env) for config tests.
|
|
figment = { workspace = true, features = ["test"] }
|