Files
helexa/crates/helexa-router/tests/tls.rs
rob thijssen 1115bb0942
All checks were successful
CI / Format (push) Successful in 41s
CI / CUDA type-check (push) Successful in 1m34s
CI / Clippy (push) Successful in 2m23s
CI / Test (push) Successful in 5m19s
CI / Build cortex SRPM (push) Has been skipped
CI / Build neuron SRPM (push) Has been skipped
CI / Publish cortex to COPR (push) Has been skipped
CI / Publish neuron to COPR (push) Has been skipped
CI / Bump version in source (push) Has been skipped
feat(#74): verify downstream cortex TLS certs (outbound pinning)
The router is a TLS client to cortexes; the router->cortex hop crosses
the helexa->operator boundary carrying the client's bearer. This pins
that hop to an enrolled cert.

Trust mechanism (the open question): per-cortex enrolled trust anchor.
Each [[cortexes]] entry gets an optional `tls_ca` — a PEM CA (or
self-signed cert) the cortex's TLS cert must chain to. When set, the
router builds a client that trusts ONLY that anchor (platform roots
disabled), so the cortex must present the expected cert and a rogue
endpoint with any other (even publicly-valid) cert is rejected at the
handshake. Enrolment = the operator hands helexa the cortex's cert,
referenced by path in router config. This is the natural model for
self-hosted operators behind their own nginx/private CA, and reuses the
reqwest public API (no custom rustls verifier, no new TLS backend).

- `RouterState` now holds a per-cortex `reqwest::Client` map
  (`client_for`), replacing the single shared client; poller and dispatch
  use the per-cortex client. `build_client(tls_ca)` is the builder.
- Fail closed: a `tls_ca` that can't load omits the cortex from the
  client map — it's never polled or routed to, rather than silently
  degrading to unpinned TLS. The poller treats a missing client (and a
  rejected handshake) as a failed poll, so #72's existing reachability
  debounce excludes it.

Tests (`tls.rs`, 4): a live tokio-rustls HTTPS server proves a client
enrolled with the server's cert is accepted (200) while clients pinned to
a different cert — or using default roots — are rejected; the poller
marks a wrong-cert cortex unreachable while a correctly-enrolled one is
reachable; a missing pin file disables the cortex (fail closed); garbage
PEM is rejected at build. Existing suites updated for the per-cortex
client + new config field.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F6o3ddqmYNh9kzdwq6eowh
2026-06-21 21:23:20 +03:00

211 lines
7.3 KiB
Rust

//! Outbound TLS pinning tests for #74.
//!
//! Proves the router, as a TLS client to cortexes, reaches a cortex
//! presenting its **enrolled** cert and rejects one presenting an
//! unexpected (or untrusted) cert — and that a rejected handshake flows
//! through the existing reachability path (#72) to exclude the cortex.
//!
//! A minimal `tokio-rustls` HTTPS server presents a self-signed cert; the
//! router's `reqwest` client (native-tls) validates against the PEM anchor
//! enrolled in config. Server (rustls) and client (native-tls) interoperate
//! at the protocol level — what matters is the trust decision.
use helexa_router::config::{CortexEndpoint, RouterConfig};
use helexa_router::poller::poll_once;
use helexa_router::state::{RouterState, build_client};
use std::io::Write;
use std::sync::Arc;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpListener;
use tokio_rustls::TlsAcceptor;
/// A self-signed cert: PEM (for the reqwest pin file) + DER cert/key (for
/// the rustls server).
struct TestCert {
cert_pem: String,
cert_der: rustls::pki_types::CertificateDer<'static>,
key_der: Vec<u8>,
}
fn make_cert() -> TestCert {
let key = rcgen::generate_simple_self_signed(vec!["127.0.0.1".to_string()]).unwrap();
TestCert {
cert_pem: key.cert.pem(),
cert_der: key.cert.der().clone(),
key_der: key.key_pair.serialize_der(),
}
}
/// Write a cert PEM to a unique temp file (named by `tag`) and return the
/// path. `tag` is caller-unique (we use the bound port), so no randomness.
fn write_pem(tag: &str, pem: &str) -> String {
let path = std::env::temp_dir().join(format!("helexa-router-tls-{tag}.pem"));
let mut f = std::fs::File::create(&path).unwrap();
f.write_all(pem.as_bytes()).unwrap();
path.to_string_lossy().into_owned()
}
/// Spawn a minimal HTTPS server presenting `cert`, answering every request
/// with a canned `/v1/models`-shaped 200. Returns its `https://` base URL.
async fn spawn_https(cert: &TestCert) -> String {
let _ = rustls::crypto::aws_lc_rs::default_provider().install_default();
let key = rustls::pki_types::PrivateKeyDer::Pkcs8(rustls::pki_types::PrivatePkcs8KeyDer::from(
cert.key_der.clone(),
));
let config = rustls::ServerConfig::builder()
.with_no_client_auth()
.with_single_cert(vec![cert.cert_der.clone()], key)
.unwrap();
let acceptor = TlsAcceptor::from(Arc::new(config));
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap();
tokio::spawn(async move {
loop {
let Ok((stream, _)) = listener.accept().await else {
continue;
};
let acceptor = acceptor.clone();
tokio::spawn(async move {
if let Ok(mut tls) = acceptor.accept(stream).await {
let mut buf = [0u8; 2048];
let _ = tls.read(&mut buf).await; // consume request line/headers
let body = "{\"object\":\"list\",\"data\":[]}";
let resp = format!(
"HTTP/1.1 200 OK\r\ncontent-type: application/json\r\ncontent-length: {}\r\nconnection: close\r\n\r\n{}",
body.len(),
body
);
let _ = tls.write_all(resp.as_bytes()).await;
let _ = tls.shutdown().await;
}
});
}
});
format!("https://{addr}")
}
fn tag_for(url: &str) -> String {
url.rsplit(':').next().unwrap_or("0").to_string()
}
#[tokio::test]
async fn pinned_client_accepts_enrolled_cert_and_rejects_others() {
let server_cert = make_cert();
let other_cert = make_cert();
let url = spawn_https(&server_cert).await;
let tag = tag_for(&url);
let good_pin = write_pem(&format!("{tag}-good"), &server_cert.cert_pem);
let bad_pin = write_pem(&format!("{tag}-bad"), &other_cert.cert_pem);
// Enrolled with the server's own cert → handshake trusted → 200.
let good = build_client(Some(&good_pin)).unwrap();
let resp = good.get(format!("{url}/v1/models")).send().await;
assert!(resp.is_ok(), "enrolled cert must be accepted: {resp:?}");
assert_eq!(resp.unwrap().status(), 200);
// Enrolled with a different cert → server's cert is unexpected → reject.
let bad = build_client(Some(&bad_pin)).unwrap();
assert!(
bad.get(format!("{url}/v1/models")).send().await.is_err(),
"unexpected cert must be rejected"
);
// No enrollment (default platform roots) → self-signed cert untrusted.
let default = build_client(None).unwrap();
assert!(
default
.get(format!("{url}/v1/models"))
.send()
.await
.is_err(),
"un-enrolled self-signed cert must be rejected by default roots"
);
}
#[tokio::test]
async fn poller_excludes_cortex_with_unexpected_cert() {
let server_cert = make_cert();
let other_cert = make_cert();
let url = spawn_https(&server_cert).await;
let tag = tag_for(&url);
let good_pin = write_pem(&format!("{tag}-pgood"), &server_cert.cert_pem);
let bad_pin = write_pem(&format!("{tag}-pbad"), &other_cert.cert_pem);
// Cortex A enrolled correctly → reachable. Cortex B enrolled with the
// wrong cert → TLS handshake fails → excluded.
let cfg = RouterConfig {
cortexes: vec![
CortexEndpoint {
name: "good".into(),
endpoint: url.clone(),
region: None,
tls_ca: Some(good_pin),
},
CortexEndpoint {
name: "bad".into(),
endpoint: url.clone(),
region: None,
tls_ca: Some(bad_pin),
},
],
..Default::default()
};
let state = RouterState::from_config(&cfg);
poll_once(&state).await;
let topo = state.topology.read().await;
assert!(
topo["good"].reachable,
"correctly-enrolled cortex reachable"
);
assert!(
!topo["bad"].reachable,
"cortex presenting an unexpected cert is excluded"
);
}
#[tokio::test]
async fn misconfigured_pin_disables_cortex_fail_closed() {
// A `tls_ca` pointing at a nonexistent file must NOT fall back to an
// unpinned client — the cortex is disabled entirely.
let cfg = RouterConfig {
cortexes: vec![
CortexEndpoint {
name: "broken".into(),
endpoint: "https://127.0.0.1:1".into(),
region: None,
tls_ca: Some("/no/such/anchor.pem".into()),
},
CortexEndpoint {
name: "plain".into(),
endpoint: "http://127.0.0.1:1".into(),
region: None,
tls_ca: None,
},
],
..Default::default()
};
let state = RouterState::from_config(&cfg);
assert!(
state.client_for("broken").is_none(),
"a cortex with an unloadable pin is disabled (fail closed)"
);
assert!(
state.client_for("plain").is_some(),
"an un-pinned cortex still gets a client"
);
}
#[test]
fn build_client_rejects_garbage_pem() {
let path = write_pem(
"garbage",
"-----BEGIN CERTIFICATE-----\nnope\n-----END CERTIFICATE-----",
);
assert!(build_client(Some(&path)).is_err());
}