CI / Classify changes (push) Successful in 13s
CI / Web (lint + typecheck + i18n + build) (push) Successful in 1m43s
CI / Format (push) Successful in 9s
CI / Clippy (push) Successful in 2m31s
CI / Test (push) Successful in 6m54s
CI / CUDA type-check (push) Successful in 19m6s
CI / Build cortex SRPM (push) Has been skipped
CI / Publish cortex to COPR (push) Has been skipped
CI / Build neuron SRPM (push) Has been skipped
CI / Publish neuron to COPR (push) Has been skipped
CI / Bump version in source (push) Has been skipped
Three incidents today were each diagnosable only by reading source and reproducing arithmetic by hand, because every failure path logged nothing useful and returned nothing parseable: - a 422 that killed a harness session (#253) — axum's `Json` extractor rejects before any handler runs, with a bare string body and no log; - a 503 that discarded 30 minutes of agentic work (#257) — the envelope reached the client, but neither neuron nor cortex recorded the reason; - a device fault that poisoned a model — cause present in the error but formatted away before anyone saw it. Four fixes, in rough order of what they cost us. **Error chains survive (56 sites).** `anyhow!("ctx: {e}")` keeps only the outermost context, so wrapping an error discarded everything beneath it. `{e:#}` keeps the chain. This is not just cosmetic: `is_device_fault` is *default-true* — it poisons unless it sees a known non-device marker like "shape mismatch", and those markers live in the inner error. A flattened chain therefore turns a benign pre-kernel failure into a device fault, poisoning the model and forcing an unload+reload for nothing. Pinned by a test that shows the same error classified both ways depending on formatting. It also means today's `AssembleKvBatch` fault was never established as a device fault at all — it was classified as one by default, with its cause invisible. **`ApiJson` replaces `Json` on every body-taking route.** Same #63 envelope as every other error, plus a WARN naming endpoint and detail. 422 for a shape we don't accept, 400 for malformed JSON, 415 for a bad content-type — distinguished because they need different fixes from the caller. **Every rejection logs once, at the boundary.** `inference_error_response` now emits status, type, code, message and the `extra` numbers that make it actionable (free_mb/required_mb, prompt_len/max, retry_after), so the line stands alone instead of pointing at source to go and read. **cortex logs upstream error bodies.** Previously only `status=503`, on the grounds that the body was a stream. Streaming is what a *success* body needs; an error body is a few hundred bytes and streaming it only guarantees nobody sees the reason. Non-2xx is now buffered (64 KiB cap), logged truncated, and handed on unchanged. Over the cap it says the body was dropped rather than passing an empty one off as the upstream's answer. Refs #253, #257.
780 lines
26 KiB
Rust
780 lines
26 KiB
Rust
use cortex_core::discovery::{DeviceInfo, DiscoveryResponse};
|
|
use neuron::activation::ActivationTracker;
|
|
use neuron::api::{self, NeuronState};
|
|
use neuron::harness::HarnessRegistry;
|
|
use neuron::health::HealthCache;
|
|
use serde_json::json;
|
|
use std::sync::Arc;
|
|
use tokio::sync::RwLock;
|
|
|
|
async fn spawn_neuron(discovery: DiscoveryResponse) -> String {
|
|
let health_cache = Arc::new(HealthCache::new());
|
|
let registry = HarnessRegistry::new();
|
|
|
|
let state = Arc::new(NeuronState {
|
|
discovery,
|
|
health_cache,
|
|
registry: RwLock::new(registry),
|
|
candle: None,
|
|
activation: Arc::new(ActivationTracker::new(&[])),
|
|
});
|
|
|
|
let app = api::neuron_routes().with_state(state);
|
|
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
|
|
let addr = listener.local_addr().unwrap();
|
|
tokio::spawn(async move {
|
|
axum::serve(listener, app).await.unwrap();
|
|
});
|
|
format!("http://{addr}")
|
|
}
|
|
|
|
fn fake_discovery() -> DiscoveryResponse {
|
|
DiscoveryResponse {
|
|
hostname: "test-node".into(),
|
|
os: "Linux".into(),
|
|
kernel: "6.19.0".into(),
|
|
cuda_version: Some("12.8".into()),
|
|
driver_version: Some("570.86.16".into()),
|
|
devices: vec![
|
|
DeviceInfo {
|
|
index: 0,
|
|
name: "NVIDIA GeForce RTX 5090".into(),
|
|
vram_total_mb: 32614,
|
|
compute_capability: "12.0".into(),
|
|
},
|
|
DeviceInfo {
|
|
index: 1,
|
|
name: "NVIDIA GeForce RTX 5090".into(),
|
|
vram_total_mb: 32614,
|
|
compute_capability: "12.0".into(),
|
|
},
|
|
],
|
|
harnesses: vec![],
|
|
cuda_unavailable_reason: None,
|
|
max_prompt_tokens: 16384,
|
|
}
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_discovery_endpoint() {
|
|
let url = spawn_neuron(fake_discovery()).await;
|
|
|
|
let client = reqwest::Client::new();
|
|
let resp = client
|
|
.get(format!("{url}/discovery"))
|
|
.send()
|
|
.await
|
|
.expect("request should succeed");
|
|
|
|
assert_eq!(resp.status(), 200);
|
|
|
|
let body: serde_json::Value = resp.json().await.unwrap();
|
|
assert_eq!(body["hostname"], "test-node");
|
|
assert_eq!(body["cuda_version"], "12.8");
|
|
|
|
let devices = body["devices"].as_array().unwrap();
|
|
assert_eq!(devices.len(), 2);
|
|
assert_eq!(devices[0]["name"], "NVIDIA GeForce RTX 5090");
|
|
assert_eq!(devices[0]["vram_total_mb"], 32614);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_version_endpoint() {
|
|
let url = spawn_neuron(fake_discovery()).await;
|
|
|
|
let client = reqwest::Client::new();
|
|
let resp = client
|
|
.get(format!("{url}/version"))
|
|
.send()
|
|
.await
|
|
.expect("request should succeed");
|
|
|
|
assert_eq!(resp.status(), 200);
|
|
|
|
// Deserialize into the shared type to lock the wire contract.
|
|
let body: cortex_core::build_info::BuildInfo = resp.json().await.unwrap();
|
|
assert_eq!(body.package_version, env!("CARGO_PKG_VERSION"));
|
|
// git_sha is always present — a real short SHA in a git checkout, or
|
|
// the literal "unknown" in a tarball build. Either way, non-empty.
|
|
assert!(!body.git_sha.is_empty());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_health_endpoint() {
|
|
let url = spawn_neuron(fake_discovery()).await;
|
|
|
|
let client = reqwest::Client::new();
|
|
let resp = client
|
|
.get(format!("{url}/health"))
|
|
.send()
|
|
.await
|
|
.expect("request should succeed");
|
|
|
|
assert_eq!(resp.status(), 200);
|
|
|
|
let body: serde_json::Value = resp.json().await.unwrap();
|
|
assert_eq!(body["uptime_secs"], 0);
|
|
// Per-model admission load (#53) is always present, even with no models
|
|
// loaded (empty array) — cortex's load-aware router (#55) relies on it.
|
|
assert!(
|
|
body["models"].is_array(),
|
|
"/health must expose a models load array"
|
|
);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_discovery_no_gpus() {
|
|
let disc = DiscoveryResponse {
|
|
hostname: "cpu-only".into(),
|
|
os: "Linux".into(),
|
|
kernel: "6.19.0".into(),
|
|
cuda_version: None,
|
|
driver_version: None,
|
|
devices: vec![],
|
|
harnesses: vec![],
|
|
cuda_unavailable_reason: None,
|
|
max_prompt_tokens: 16384,
|
|
};
|
|
let url = spawn_neuron(disc).await;
|
|
|
|
let client = reqwest::Client::new();
|
|
let resp = client
|
|
.get(format!("{url}/discovery"))
|
|
.send()
|
|
.await
|
|
.expect("request should succeed");
|
|
|
|
assert_eq!(resp.status(), 200);
|
|
|
|
let body: serde_json::Value = resp.json().await.unwrap();
|
|
assert_eq!(body["hostname"], "cpu-only");
|
|
assert!(body["cuda_version"].is_null());
|
|
assert!(body["devices"].as_array().unwrap().is_empty());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_models_empty_registry() {
|
|
let url = spawn_neuron(fake_discovery()).await;
|
|
|
|
let client = reqwest::Client::new();
|
|
let resp = client
|
|
.get(format!("{url}/models"))
|
|
.send()
|
|
.await
|
|
.expect("request should succeed");
|
|
|
|
assert_eq!(resp.status(), 200);
|
|
|
|
let body: serde_json::Value = resp.json().await.unwrap();
|
|
assert!(body.as_array().unwrap().is_empty());
|
|
}
|
|
|
|
/// Verify the candle harness registers, list is empty by default, and a
|
|
/// load attempt for an obviously-bogus model id returns a 4xx error
|
|
/// without crashing the daemon. Real load/unload exercising actual GGUF
|
|
/// download is covered by `tests/candle_lifecycle.rs` (cuda-integration).
|
|
#[tokio::test]
|
|
async fn test_candle_harness_registers_and_rejects_bogus_model() {
|
|
use cortex_core::harness::HarnessConfig;
|
|
use neuron::config::HarnessSettings;
|
|
|
|
let registry = HarnessRegistry::from_configs(
|
|
&[HarnessConfig {
|
|
name: "candle".into(),
|
|
}],
|
|
"http://localhost:13131",
|
|
&HarnessSettings::default(),
|
|
);
|
|
|
|
let candle = registry.candle();
|
|
let health_cache = Arc::new(HealthCache::new());
|
|
let state = Arc::new(NeuronState {
|
|
discovery: fake_discovery(),
|
|
health_cache,
|
|
registry: RwLock::new(registry),
|
|
candle,
|
|
activation: Arc::new(ActivationTracker::new(&[])),
|
|
});
|
|
|
|
let app = api::neuron_routes().with_state(state);
|
|
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
|
|
let neuron_addr = listener.local_addr().unwrap();
|
|
tokio::spawn(async move {
|
|
axum::serve(listener, app).await.unwrap();
|
|
});
|
|
let neuron_url = format!("http://{neuron_addr}");
|
|
|
|
let client = reqwest::Client::new();
|
|
|
|
let resp = client
|
|
.get(format!("{neuron_url}/models"))
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), 200);
|
|
let models: Vec<serde_json::Value> = resp.json().await.unwrap();
|
|
assert!(models.is_empty());
|
|
|
|
// Sending a wrong-harness spec should be rejected synchronously
|
|
// without touching the network or the model registry.
|
|
let resp = client
|
|
.post(format!("{neuron_url}/models/load"))
|
|
.json(&json!({"model_id": "definitely/not-real", "harness": "not-candle"}))
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), 400);
|
|
|
|
// Registry still empty.
|
|
let resp = client
|
|
.get(format!("{neuron_url}/models"))
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
let models: Vec<serde_json::Value> = resp.json().await.unwrap();
|
|
assert!(models.is_empty());
|
|
}
|
|
|
|
/// `/v1/chat/completions` returns 503 when no candle harness is registered.
|
|
#[tokio::test]
|
|
async fn test_chat_completions_no_candle_harness() {
|
|
let registry = HarnessRegistry::new();
|
|
let health_cache = Arc::new(HealthCache::new());
|
|
let state = Arc::new(NeuronState {
|
|
discovery: fake_discovery(),
|
|
health_cache,
|
|
registry: RwLock::new(registry),
|
|
candle: None,
|
|
activation: Arc::new(ActivationTracker::new(&[])),
|
|
});
|
|
let app = api::neuron_routes().with_state(state);
|
|
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
|
|
let addr = listener.local_addr().unwrap();
|
|
tokio::spawn(async move {
|
|
axum::serve(listener, app).await.unwrap();
|
|
});
|
|
let url = format!("http://{addr}");
|
|
|
|
let resp = reqwest::Client::new()
|
|
.post(format!("{url}/v1/chat/completions"))
|
|
.json(&json!({
|
|
"model": "anything",
|
|
"messages": [{"role": "user", "content": "hi"}]
|
|
}))
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), 503);
|
|
}
|
|
|
|
/// `/v1/chat/completions` returns 404 when the requested model isn't loaded.
|
|
#[tokio::test]
|
|
async fn test_chat_completions_model_not_loaded() {
|
|
use cortex_core::harness::HarnessConfig;
|
|
use neuron::config::HarnessSettings;
|
|
|
|
let registry = HarnessRegistry::from_configs(
|
|
&[HarnessConfig {
|
|
name: "candle".into(),
|
|
}],
|
|
"http://localhost:0",
|
|
&HarnessSettings::default(),
|
|
);
|
|
let candle = registry.candle();
|
|
let health_cache = Arc::new(HealthCache::new());
|
|
let state = Arc::new(NeuronState {
|
|
discovery: fake_discovery(),
|
|
health_cache,
|
|
registry: RwLock::new(registry),
|
|
candle,
|
|
activation: Arc::new(ActivationTracker::new(&[])),
|
|
});
|
|
let app = api::neuron_routes().with_state(state);
|
|
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
|
|
let addr = listener.local_addr().unwrap();
|
|
tokio::spawn(async move {
|
|
axum::serve(listener, app).await.unwrap();
|
|
});
|
|
let url = format!("http://{addr}");
|
|
|
|
let resp = reqwest::Client::new()
|
|
.post(format!("{url}/v1/chat/completions"))
|
|
.json(&json!({
|
|
"model": "definitely/not-loaded",
|
|
"messages": [{"role": "user", "content": "hi"}]
|
|
}))
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), 404);
|
|
}
|
|
|
|
/// `/v1/chat/completions` with `stream: true` returns 404 when the
|
|
/// model isn't loaded — same surface as the non-streaming path. The
|
|
/// streaming code only kicks in once the model lookup succeeds.
|
|
#[tokio::test]
|
|
async fn test_chat_completions_streaming_model_not_loaded() {
|
|
use cortex_core::harness::HarnessConfig;
|
|
use neuron::config::HarnessSettings;
|
|
|
|
let registry = HarnessRegistry::from_configs(
|
|
&[HarnessConfig {
|
|
name: "candle".into(),
|
|
}],
|
|
"http://localhost:0",
|
|
&HarnessSettings::default(),
|
|
);
|
|
let candle = registry.candle();
|
|
let health_cache = Arc::new(HealthCache::new());
|
|
let state = Arc::new(NeuronState {
|
|
discovery: fake_discovery(),
|
|
health_cache,
|
|
registry: RwLock::new(registry),
|
|
candle,
|
|
activation: Arc::new(ActivationTracker::new(&[])),
|
|
});
|
|
let app = api::neuron_routes().with_state(state);
|
|
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
|
|
let addr = listener.local_addr().unwrap();
|
|
tokio::spawn(async move {
|
|
axum::serve(listener, app).await.unwrap();
|
|
});
|
|
let url = format!("http://{addr}");
|
|
|
|
let resp = reqwest::Client::new()
|
|
.post(format!("{url}/v1/chat/completions"))
|
|
.json(&json!({
|
|
"model": "definitely/not-loaded",
|
|
"messages": [{"role": "user", "content": "hi"}],
|
|
"stream": true
|
|
}))
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), 404);
|
|
}
|
|
|
|
// ── /v1/responses ────────────────────────────────────────────────────
|
|
|
|
/// `/v1/responses` returns 503 when no candle harness is registered —
|
|
/// matches the chat-completions error shape so a client can swap
|
|
/// endpoints without re-handling 503s.
|
|
#[tokio::test]
|
|
async fn test_responses_no_candle_harness() {
|
|
let registry = HarnessRegistry::new();
|
|
let health_cache = Arc::new(HealthCache::new());
|
|
let state = Arc::new(NeuronState {
|
|
discovery: fake_discovery(),
|
|
health_cache,
|
|
registry: RwLock::new(registry),
|
|
candle: None,
|
|
activation: Arc::new(ActivationTracker::new(&[])),
|
|
});
|
|
let app = api::neuron_routes().with_state(state);
|
|
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
|
|
let addr = listener.local_addr().unwrap();
|
|
tokio::spawn(async move {
|
|
axum::serve(listener, app).await.unwrap();
|
|
});
|
|
let url = format!("http://{addr}");
|
|
|
|
let resp = reqwest::Client::new()
|
|
.post(format!("{url}/v1/responses"))
|
|
.json(&json!({"model": "anything", "input": "hi"}))
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), 503);
|
|
}
|
|
|
|
/// `previous_response_id` is rejected at translate time with 400 —
|
|
/// we don't store responses server-side yet, so chained
|
|
/// conversations can't be honoured.
|
|
#[tokio::test]
|
|
async fn test_responses_rejects_previous_response_id() {
|
|
use cortex_core::harness::HarnessConfig;
|
|
use neuron::config::HarnessSettings;
|
|
|
|
let registry = HarnessRegistry::from_configs(
|
|
&[HarnessConfig {
|
|
name: "candle".into(),
|
|
}],
|
|
"http://localhost:0",
|
|
&HarnessSettings::default(),
|
|
);
|
|
let candle = registry.candle();
|
|
let health_cache = Arc::new(HealthCache::new());
|
|
let state = Arc::new(NeuronState {
|
|
discovery: fake_discovery(),
|
|
health_cache,
|
|
registry: RwLock::new(registry),
|
|
candle,
|
|
activation: Arc::new(ActivationTracker::new(&[])),
|
|
});
|
|
let app = api::neuron_routes().with_state(state);
|
|
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
|
|
let addr = listener.local_addr().unwrap();
|
|
tokio::spawn(async move {
|
|
axum::serve(listener, app).await.unwrap();
|
|
});
|
|
let url = format!("http://{addr}");
|
|
|
|
let resp = reqwest::Client::new()
|
|
.post(format!("{url}/v1/responses"))
|
|
.json(&json!({
|
|
"model": "anything",
|
|
"input": "hi",
|
|
"previous_response_id": "resp_prev_42"
|
|
}))
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), 400);
|
|
let body: serde_json::Value = resp.json().await.unwrap();
|
|
assert_eq!(body["code"], "chained_conversation_not_supported");
|
|
}
|
|
|
|
/// `/v1/responses` returns 404 when the model isn't loaded — same
|
|
/// surface as chat completions.
|
|
#[tokio::test]
|
|
async fn test_responses_model_not_loaded() {
|
|
use cortex_core::harness::HarnessConfig;
|
|
use neuron::config::HarnessSettings;
|
|
|
|
let registry = HarnessRegistry::from_configs(
|
|
&[HarnessConfig {
|
|
name: "candle".into(),
|
|
}],
|
|
"http://localhost:0",
|
|
&HarnessSettings::default(),
|
|
);
|
|
let candle = registry.candle();
|
|
let health_cache = Arc::new(HealthCache::new());
|
|
let state = Arc::new(NeuronState {
|
|
discovery: fake_discovery(),
|
|
health_cache,
|
|
registry: RwLock::new(registry),
|
|
candle,
|
|
activation: Arc::new(ActivationTracker::new(&[])),
|
|
});
|
|
let app = api::neuron_routes().with_state(state);
|
|
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
|
|
let addr = listener.local_addr().unwrap();
|
|
tokio::spawn(async move {
|
|
axum::serve(listener, app).await.unwrap();
|
|
});
|
|
let url = format!("http://{addr}");
|
|
|
|
let resp = reqwest::Client::new()
|
|
.post(format!("{url}/v1/responses"))
|
|
.json(&json!({"model": "not-loaded", "input": "hi"}))
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), 404);
|
|
}
|
|
|
|
/// Same model-not-loaded surface on the streaming path. The
|
|
/// stream is opened only after model lookup succeeds, so a
|
|
/// missing model fails fast with a non-SSE 404 response.
|
|
#[tokio::test]
|
|
async fn test_responses_streaming_model_not_loaded() {
|
|
use cortex_core::harness::HarnessConfig;
|
|
use neuron::config::HarnessSettings;
|
|
|
|
let registry = HarnessRegistry::from_configs(
|
|
&[HarnessConfig {
|
|
name: "candle".into(),
|
|
}],
|
|
"http://localhost:0",
|
|
&HarnessSettings::default(),
|
|
);
|
|
let candle = registry.candle();
|
|
let health_cache = Arc::new(HealthCache::new());
|
|
let state = Arc::new(NeuronState {
|
|
discovery: fake_discovery(),
|
|
health_cache,
|
|
registry: RwLock::new(registry),
|
|
candle,
|
|
activation: Arc::new(ActivationTracker::new(&[])),
|
|
});
|
|
let app = api::neuron_routes().with_state(state);
|
|
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
|
|
let addr = listener.local_addr().unwrap();
|
|
tokio::spawn(async move {
|
|
axum::serve(listener, app).await.unwrap();
|
|
});
|
|
let url = format!("http://{addr}");
|
|
|
|
let resp = reqwest::Client::new()
|
|
.post(format!("{url}/v1/responses"))
|
|
.json(&json!({
|
|
"model": "not-loaded",
|
|
"input": "hi",
|
|
"stream": true
|
|
}))
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), 404);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_driver_mismatch_rejects_load_and_rides_discovery() {
|
|
// #19: a host with the driver/library mismatch advertises the
|
|
// reason on /discovery (so cortex routes around it) and fast-
|
|
// rejects /models/load with 503 + the actionable message instead
|
|
// of dying minutes later inside cuInit/NCCL.
|
|
let reason = "host NVIDIA driver/library mismatch (userspace NVML 580.159 vs loaded \
|
|
kernel module 580.159.03) — reboot the host to reload the kernel module; \
|
|
all CUDA inference is unavailable until then";
|
|
let disc = DiscoveryResponse {
|
|
hostname: "mismatched".into(),
|
|
os: "Linux".into(),
|
|
kernel: "6.19.0".into(),
|
|
cuda_version: Some("13.0".into()),
|
|
driver_version: None,
|
|
devices: vec![],
|
|
harnesses: vec!["candle".into()],
|
|
cuda_unavailable_reason: Some(reason.into()),
|
|
max_prompt_tokens: 16384,
|
|
};
|
|
let url = spawn_neuron(disc).await;
|
|
let client = reqwest::Client::new();
|
|
|
|
let body: serde_json::Value = client
|
|
.get(format!("{url}/discovery"))
|
|
.send()
|
|
.await
|
|
.expect("discovery request")
|
|
.json()
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(body["cuda_unavailable_reason"], reason);
|
|
|
|
let resp = client
|
|
.post(format!("{url}/models/load"))
|
|
.json(&serde_json::json!({
|
|
"model_id": "Qwen/Qwen3.6-27B",
|
|
"harness": "candle",
|
|
"quant": "q6k",
|
|
"tensor_parallel": 2
|
|
}))
|
|
.send()
|
|
.await
|
|
.expect("load request");
|
|
assert_eq!(resp.status(), 503);
|
|
let body: serde_json::Value = resp.json().await.unwrap();
|
|
assert_eq!(body["code"], "cuda_unavailable");
|
|
assert!(
|
|
body["error"].as_str().unwrap().contains("reboot the host"),
|
|
"error must be operator-actionable: {body}"
|
|
);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_healthy_discovery_omits_cuda_unavailable_reason() {
|
|
// No false positives: the field must be absent (not null) from the
|
|
// wire format on healthy hosts.
|
|
let url = spawn_neuron(fake_discovery()).await;
|
|
let body: serde_json::Value = reqwest::Client::new()
|
|
.get(format!("{url}/discovery"))
|
|
.send()
|
|
.await
|
|
.expect("discovery request")
|
|
.json()
|
|
.await
|
|
.unwrap();
|
|
assert!(
|
|
body.get("cuda_unavailable_reason").is_none(),
|
|
"healthy host must omit the field entirely: {body}"
|
|
);
|
|
}
|
|
|
|
// ── /v1/images/generations envelope tests (#200) ─────────────────
|
|
//
|
|
// These exercise the HTTP surface: parameter validation, envelope
|
|
// shape, and modality/not-loaded errors. Real generation needs CUDA +
|
|
// weights and is validated on the fleet (see #198/#200 acceptance).
|
|
|
|
async fn spawn_neuron_with_candle(discovery: DiscoveryResponse) -> String {
|
|
use neuron::harness::candle::CandleHarness;
|
|
|
|
let health_cache = Arc::new(HealthCache::new());
|
|
let registry = HarnessRegistry::new();
|
|
let candle = CandleHarness::new(
|
|
"http://localhost:13131".into(),
|
|
&neuron::config::CandleHarnessConfig::default(),
|
|
);
|
|
|
|
let state = Arc::new(NeuronState {
|
|
discovery,
|
|
health_cache,
|
|
registry: RwLock::new(registry),
|
|
candle: Some(candle),
|
|
activation: Arc::new(ActivationTracker::new(&[])),
|
|
});
|
|
|
|
let app = api::neuron_routes().with_state(state);
|
|
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
|
|
let addr = listener.local_addr().unwrap();
|
|
tokio::spawn(async move {
|
|
axum::serve(listener, app).await.unwrap();
|
|
});
|
|
format!("http://{addr}")
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_images_no_candle_harness_503() {
|
|
let url = spawn_neuron(fake_discovery()).await;
|
|
let resp = reqwest::Client::new()
|
|
.post(format!("{url}/v1/images/generations"))
|
|
.json(&json!({"model": "m", "prompt": "a cat"}))
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), 503);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_images_model_not_loaded_404() {
|
|
let url = spawn_neuron_with_candle(fake_discovery()).await;
|
|
let resp = reqwest::Client::new()
|
|
.post(format!("{url}/v1/images/generations"))
|
|
.json(&json!({"model": "Tongyi-MAI/Z-Image-Turbo", "prompt": "a cat"}))
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), 404);
|
|
let body: serde_json::Value = resp.json().await.unwrap();
|
|
assert_eq!(body["error"]["code"], "model_not_found");
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_images_n_rejected() {
|
|
let url = spawn_neuron_with_candle(fake_discovery()).await;
|
|
let resp = reqwest::Client::new()
|
|
.post(format!("{url}/v1/images/generations"))
|
|
.json(&json!({"model": "m", "prompt": "a cat", "n": 4}))
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), 400);
|
|
let body: serde_json::Value = resp.json().await.unwrap();
|
|
assert_eq!(body["error"]["code"], "unsupported_parameter");
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_images_bad_size_rejected() {
|
|
let url = spawn_neuron_with_candle(fake_discovery()).await;
|
|
let resp = reqwest::Client::new()
|
|
.post(format!("{url}/v1/images/generations"))
|
|
.json(&json!({"model": "m", "prompt": "a cat", "size": "huge"}))
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), 400);
|
|
let body: serde_json::Value = resp.json().await.unwrap();
|
|
assert_eq!(body["error"]["code"], "invalid_size");
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_images_url_response_format_rejected() {
|
|
let url = spawn_neuron_with_candle(fake_discovery()).await;
|
|
let resp = reqwest::Client::new()
|
|
.post(format!("{url}/v1/images/generations"))
|
|
.json(&json!({"model": "m", "prompt": "a cat", "response_format": "url"}))
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), 400);
|
|
let body: serde_json::Value = resp.json().await.unwrap();
|
|
assert_eq!(
|
|
body["error"]["message"].as_str().unwrap(),
|
|
"response_format 'url' is not supported; use b64_json"
|
|
);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_images_chat_at_image_endpoint_is_not_found_for_text_model() {
|
|
// A text model id that isn't loaded still 404s (not 422) — the
|
|
// WrongModality 422 needs a *loaded* text model, which requires
|
|
// weights; that path is covered by fleet validation.
|
|
let url = spawn_neuron_with_candle(fake_discovery()).await;
|
|
let resp = reqwest::Client::new()
|
|
.post(format!("{url}/v1/images/generations"))
|
|
.json(&json!({"model": "Qwen/Qwen3-8B", "prompt": "a cat"}))
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), 404);
|
|
}
|
|
|
|
/// A malformed body must come back as the #63 envelope, not axum's bare
|
|
/// string (#253). An OpenAI client reads `error.code` / `error.type`; a
|
|
/// plain string leaves it with nothing to act on, which is exactly how a
|
|
/// harness ended up quitting silently mid-session.
|
|
#[tokio::test]
|
|
async fn malformed_body_returns_a_parseable_error_envelope() {
|
|
let base = spawn_neuron(fake_discovery()).await;
|
|
let client = reqwest::Client::new();
|
|
|
|
// Valid JSON, wrong shape for the request type → 422.
|
|
let resp = client
|
|
.post(format!("{base}/v1/chat/completions"))
|
|
.json(&json!({"model": "m", "messages": "not-an-array"}))
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), 422);
|
|
let body: serde_json::Value = resp.json().await.expect("body must be JSON");
|
|
assert_eq!(body["error"]["type"], "invalid_request_error");
|
|
assert_eq!(body["error"]["code"], "invalid_request_body");
|
|
assert!(
|
|
body["error"]["message"]
|
|
.as_str()
|
|
.is_some_and(|m| !m.is_empty()),
|
|
"message must say what was wrong: {body}"
|
|
);
|
|
|
|
// Not JSON at all → 400, distinguished from the above so the caller
|
|
// knows whether to fix its serializer or its schema.
|
|
let resp = client
|
|
.post(format!("{base}/v1/chat/completions"))
|
|
.header("content-type", "application/json")
|
|
.body("{not json")
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), 400);
|
|
let body: serde_json::Value = resp.json().await.expect("body must be JSON");
|
|
assert_eq!(body["error"]["code"], "malformed_json");
|
|
}
|
|
|
|
/// The shape from #253 itself: an assistant turn carrying only
|
|
/// `tool_calls`. It must be *accepted* now — this guards the envelope
|
|
/// work from quietly re-breaking what #253 fixed.
|
|
#[tokio::test]
|
|
async fn null_assistant_content_is_not_a_body_rejection() {
|
|
let base = spawn_neuron(fake_discovery()).await;
|
|
let resp = reqwest::Client::new()
|
|
.post(format!("{base}/v1/chat/completions"))
|
|
.json(&json!({
|
|
"model": "m",
|
|
"messages": [
|
|
{"role": "user", "content": "hi"},
|
|
{"role": "assistant", "content": null, "tool_calls": [
|
|
{"id": "c1", "type": "function",
|
|
"function": {"name": "b", "arguments": "{}"}}]},
|
|
{"role": "tool", "content": "ok", "tool_call_id": "c1"}
|
|
]
|
|
}))
|
|
.send()
|
|
.await
|
|
.unwrap();
|
|
// 503 (no candle harness in this fixture) — the point is that it got
|
|
// past body extraction rather than being rejected as malformed.
|
|
assert_ne!(resp.status(), 422, "null content must not be a body error");
|
|
assert_ne!(resp.status(), 400, "null content must not be a body error");
|
|
}
|