refactor: cortex talks to neurons instead of mistral.rs directly
All checks were successful
CI / Format, lint, build, test (push) Successful in 2m46s
CI / Build SRPM (push) Has been skipped
CI / Publish to COPR (push) Has been skipped

Replace NodeConfig (static vram_mb, pinned) with NeuronEndpoint.
Hardware discovery and model pinning now come from neuron API and
models.toml catalogue respectively.

- config.rs: nodes -> neurons, add models_config path
- catalogue.rs: ModelProfile with pinned_on, ModelCatalogue
- poller.rs: poll neuron GET /models (ModelInfo format)
- router.rs: resolve inference endpoint via neuron GET /models/{id}/endpoint
- evictor.rs: call neuron POST /models/unload
- node.rs: remove vram_mb, pinned fields (come from discovery/catalogue)
- All 22 gateway tests updated to mock neuron API
- Remove MistralModelsResponse, ModelLifecycleRequest (no longer needed)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-15 14:42:52 +03:00
parent 26e5e7ead8
commit e42e8ee81f
19 changed files with 385 additions and 437 deletions

View File

@@ -4,21 +4,17 @@ use serde_json::json;
#[tokio::test]
async fn test_metrics_emitted_after_proxy() {
// Install a test recorder (no HTTP listener, renders to string).
// This sets the global recorder, so only one test can do this.
let handle = cortex_gateway::metrics::install_test_recorder().expect("recorder should install");
let mock_url = common::spawn_mock_backend().await;
let mock_url = common::spawn_mock_neuron().await;
let gw_url = common::spawn_gateway(&mock_url).await;
// Verify no request metrics yet.
let before = handle.render();
assert!(
!before.contains("cortex_requests_total"),
"no request metrics before any requests"
);
// Make a successful request.
let client = reqwest::Client::new();
let resp = client
.post(format!("{gw_url}/v1/chat/completions"))
@@ -31,10 +27,8 @@ async fn test_metrics_emitted_after_proxy() {
.await
.expect("request should succeed");
assert_eq!(resp.status(), 200);
// Consume the response body to ensure the proxy completes.
let _body: serde_json::Value = resp.json().await.unwrap();
// Check metrics were emitted.
let after = handle.render();
assert!(
@@ -45,7 +39,6 @@ async fn test_metrics_emitted_after_proxy() {
after.contains("cortex_request_duration_seconds"),
"cortex_request_duration_seconds should be present.\nMetrics:\n{after}"
);
// Should NOT have error or cold start counters for this request.
assert!(
!after.contains("cortex_request_errors_total"),
"no errors expected for a successful request"