cortex can now validate locally-unrecognised bearer keys against the
helexa-upstream authority and reserve/settle their budget there — mesh
accounts work for real inference. The EntitlementProvider trait is the
seam, so cortex's enforcement (auth.rs, metering.rs) is otherwise
unchanged.
- entitlements_upstream.rs: UpstreamEntitlementProvider over reqwest →
B2's /authz/v1 (resolve/reserve/settle/release/snapshot), presenting the
operator client bearer. Maps the wire contract back to the trait: granted
→ Reservation, rejected → BudgetError, 401 → InvalidKey. Fail-closed —
unreachable resolve → AuthError::Unavailable (503, never 401);
unreachable reserve → retryable BudgetError::RateLimited (refuse, never
serve un-authorized). settle/release are best-effort (the upstream
sweeper reaps a lost one).
- entitlements_chain.rs: ChainedEntitlementProvider tries local first
(operator + infra keys, no network), falls through to upstream for
unknown keys, and dispatches reserve/settle/release/snapshot to whichever
backend resolved each account (local treats unknown principals as
uncapped, so it can't be the blind default).
- cortex-core: AuthError::Unavailable{retry_after_secs}; [upstream] config
(enabled/url/bearer/timeout). auth.rs maps Unavailable → 503 +
Retry-After distinctly from InvalidKey → 401, regardless of require_auth.
- state.rs wires the chain when [upstream].enabled, else stays purely local.
Tests (upstream_chain.rs, 4): local key resolves without touching upstream;
unknown key falls through to a mock upstream; unknown-everywhere → 401
InvalidKey; upstream-unreachable → Unavailable (503-mapped), with local keys
still resolving. Existing gateway suites updated for the new config field.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F6o3ddqmYNh9kzdwq6eowh
Stage 2 completes: when a model is loaded on more than one healthy neuron,
the router picks the least-busy replica instead of always taking the first,
and neuron backpressure propagates to the client intact.
- NodeState.model_load: per-model admission load (in_flight + queue_depth),
stashed by the poller from neuron's /health (#53/#2b).
- router::resolve collects all loaded replicas and picks the one with the
lowest in_flight+queue_depth (ties break by node name for determinism),
replacing the previous first-match-wins.
- Backpressure passthrough: the existing streaming proxy already forwards
the upstream status + all headers verbatim, so a neuron 503/429 +
Retry-After + #60 envelope reaches the client unmodified — now covered by
a regression test so a future change can't silently unwrap it.
Tests (tests/load_routing.rs): routes to the idle replica and follows the
lighter load when it flips; ties break by name; a saturated neuron's 503 +
Retry-After + envelope propagates through the gateway intact. All
cortex-side (no CUDA); local fmt/clippy/test green.
Retry-route-to-another-replica-on-backpressure (the issue's stretch goal)
is deferred — least-busy spread + honest passthrough is the substantive win.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
