mirror of
https://github.com/openai/codex.git
synced 2026-10-30 19:18:49 +00:00
Alexi Christakis
ddabe594e6
## Why A stalled rendezvous WebSocket upgrade can prevent reconnects, while short-lived connections reset reconnect backoff too early. Blocked relay writes also prevent received pongs from clearing their heartbeat deadline. ## What changed - Bound rendezvous WebSocket connection attempts with `DEFAULT_REMOTE_EXEC_SERVER_CONNECT_TIMEOUT`. - Retain exponential reconnect backoff across short-lived connections, resetting it only after a connection lasts at least 120 seconds. - Process pongs during blocked data writes while preserving the in-flight send and its independent write timeout. Prefer queued pongs over an expired heartbeat deadline. - Abort the relay writer when its owning connection task is dropped. ## Testing Add coverage for pongs arriving during blocked writes, missing and queued pongs, independent write timeouts, and writer cancellation. Extend reconnect tests to cover stalled WebSocket upgrades without re-registration and backoff reset after a stable session. GitOrigin-RevId: 0efa350c2b4ada9b89e23df8909063fbb69f33dc
43 lines
1.2 KiB
Rust
43 lines
1.2 KiB
Rust
//! Spread executor reconnects and retain backoff until a connection has been stable.
|
|
|
|
use std::time::Duration;
|
|
|
|
const INITIAL_BACKOFF: Duration = Duration::from_secs(1);
|
|
const MAX_BACKOFF: Duration = Duration::from_secs(30);
|
|
// Longer than the first keepalive interval plus its pong timeout (30s + 60s).
|
|
const STABLE_CONNECTION_DURATION: Duration = Duration::from_secs(120);
|
|
|
|
pub(super) struct ReconnectBackoff {
|
|
delay: Duration,
|
|
}
|
|
|
|
impl ReconnectBackoff {
|
|
pub(super) fn new() -> Self {
|
|
Self {
|
|
delay: INITIAL_BACKOFF,
|
|
}
|
|
}
|
|
|
|
pub(super) fn connection_closed(&mut self, connected_for: Duration) {
|
|
if connected_for >= STABLE_CONNECTION_DURATION {
|
|
self.delay = INITIAL_BACKOFF;
|
|
}
|
|
}
|
|
|
|
pub(super) fn next_delay(&mut self, random_sample: u64) -> Duration {
|
|
let delay = reconnect_delay(self.delay, random_sample);
|
|
self.delay = (self.delay * 2).min(MAX_BACKOFF);
|
|
delay
|
|
}
|
|
}
|
|
|
|
fn reconnect_delay(backoff: Duration, random_sample: u64) -> Duration {
|
|
let upper_ms = backoff.as_millis() as u64;
|
|
let lower_ms = upper_ms / 2;
|
|
Duration::from_millis(lower_ms + random_sample % (upper_ms - lower_ms + 1))
|
|
}
|
|
|
|
#[cfg(test)]
|
|
#[path = "reconnect_backoff_tests.rs"]
|
|
mod tests;
|