rob thijssen
b9e7a76a7a
The poller now fetches /health alongside /models on each neuron and stashes the activation snapshot on NodeState. The /v1/models handler gains a Pass 3 that synthesises Loading locations from each neuron's activation.in_progress and activation.pending lists, so a catalogued model that's mid-prewarm surfaces as `status: "loading"` rather than appearing absent (loaded=false, locations=[]). Without this, a client polling /v1/models during a beast restart sees Qwen3.6-27B disappear for the ~5 minutes the q5k load takes, then reappear. Now it stays visible the whole time with a clear status. Adds ModelStatus::Loading to cortex-core. The router's per-node priority loop gets an explicit (no-op) arm: Loading models aren't routable yet, and falling through to the catalogue cold-load path is the existing race — no worse than before, but tagged as a known follow-up needing neuron-side in-flight tracking on /models/load. New test_poller_captures_activation_from_health exercises the full round-trip: mock neuron with empty /models but a pre_warming /health → poller writes node.activation. Common test helpers gain spawn_mock_neuron_with_models_and_health and default_health_response. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
49 lines
1.6 KiB
Rust
49 lines
1.6 KiB
Rust
use cortex_core::catalogue::ModelCatalogue;
|
|
use cortex_core::config::{EvictionSettings, GatewayConfig, NeuronEndpoint};
|
|
use cortex_core::node::NodeState;
|
|
use std::collections::HashMap;
|
|
use tokio::sync::RwLock;
|
|
|
|
/// Shared fleet state, protected by a RwLock for concurrent reader access.
|
|
pub struct CortexState {
|
|
pub nodes: RwLock<HashMap<String, NodeState>>,
|
|
pub neuron_configs: Vec<NeuronEndpoint>,
|
|
pub eviction: EvictionSettings,
|
|
pub catalogue: ModelCatalogue,
|
|
pub http_client: reqwest::Client,
|
|
}
|
|
|
|
impl CortexState {
|
|
pub fn from_config(config: &GatewayConfig) -> Self {
|
|
let mut nodes = HashMap::new();
|
|
for nc in &config.neurons {
|
|
nodes.insert(
|
|
nc.name.clone(),
|
|
NodeState {
|
|
name: nc.name.clone(),
|
|
endpoint: nc.endpoint.clone(),
|
|
healthy: false,
|
|
models: HashMap::new(),
|
|
lifecycle_cycles: 0,
|
|
last_poll: None,
|
|
discovery: None,
|
|
activation: None,
|
|
},
|
|
);
|
|
}
|
|
|
|
let catalogue = ModelCatalogue::load(&config.models_config);
|
|
|
|
Self {
|
|
nodes: RwLock::new(nodes),
|
|
neuron_configs: config.neurons.clone(),
|
|
eviction: config.eviction.clone(),
|
|
catalogue,
|
|
http_client: reqwest::Client::builder()
|
|
.timeout(std::time::Duration::from_secs(300))
|
|
.build()
|
|
.expect("failed to build HTTP client"),
|
|
}
|
|
}
|
|
}
|