Files
codex/codex-rs/exec-server/src/remote/registration_retry.rs
T
tgrue-openai 582a8812e0 Reload remote executor credentials safely across token rotation (#58251)
## Why

ChatGPT credentials can rotate while a remote executor is idle. Registry requests need to follow stored rotations without continuing after logout or switching the executor's identity. Concurrent refreshes must also avoid overwriting newer credentials.

## What changed

- Reload and validate ChatGPT credentials before registry requests, pinning the credential mode, account, user, and workspace policy. Reuse cached metadata for unchanged personal access tokens.
- Retry registration when temporary credential resolution failures prevent a request from being sent. Bound credential resolution and block reuse of a token generation whose refresh outcome is unknown.
- Restrict supported authentication modes before executor startup loads credentials or cloud configuration.
- Serialize persistent credential writes across cooperating processes and compare stored credentials before committing refreshed tokens.
- Replace `auth.json` atomically so overlapping readers see a complete document, preserving symlink targets and protecting file permissions, including Windows access controls.

## Testing

Add coverage for credential rotation, logout, identity changes, cancelled refresh callers, temporary metadata failures, registration retries, concurrent credential writes, and atomic file replacement.

GitOrigin-RevId: 6b210944bf8eba2f45bc27113f901d2c3cb06b48
2026-10-30 05:30:20 +00:00

78 lines
3.3 KiB
Rust

//! Retry explicit registration conflicts and authentication outages before registration writes.
//! Ambiguous failures must not be replayed: a timed-out request can still replace a newer registration.
//! The enclosing remote-transport future owns cancellation; retries spawn no background work.
use http::StatusCode;
use tokio::time::sleep;
use tracing::warn;
use super::EnvironmentRegistryClient;
use crate::EnvironmentRegistryRegistrationResponse;
use crate::ExecServerError;
use crate::NoiseChannelPublicKey;
use crate::client::registry_recovery_retry_delay;
impl EnvironmentRegistryClient {
pub(super) async fn register_environment_with_retry(
&self,
environment_id: &str,
executor_public_key: &NoiseChannelPublicKey,
) -> Result<EnvironmentRegistryRegistrationResponse, ExecServerError> {
// Competing executors for the same environment must not retry in lockstep.
let retry_key = uuid::Uuid::new_v4().to_string();
let mut attempt = 0_u32;
loop {
match self
.register_environment(environment_id, executor_public_key)
.await
{
Ok(response) => return Ok(response),
// Credential resolution failed before any registration was sent.
// Keep the executor alive without replaying an ambiguous write.
Err(ExecServerError::RequestNotSent(_)) => {
let delay = registry_recovery_retry_delay(&retry_key, attempt);
attempt = attempt.saturating_add(1);
warn!(
noise_event = "registration",
noise_outcome = "retry",
retry_attempt = attempt,
retry_delay_ms = delay.as_millis() as u64,
"Noise executor retrying unavailable credential resolution"
);
sleep(delay).await;
}
Err(ExecServerError::EnvironmentRegistryHttp { status, code, .. })
if matches!(
(status, code.as_deref()),
(
StatusCode::SERVICE_UNAVAILABLE,
Some("registration_conflict")
) | (
StatusCode::BAD_GATEWAY,
Some("authentication_service_unavailable")
)
) =>
{
let delay = registry_recovery_retry_delay(&retry_key, attempt);
attempt = attempt.saturating_add(1);
// Do not log response bodies or transport errors: they can contain credentials.
warn!(
noise_event = "registration",
noise_outcome = "retry",
retry_attempt = attempt,
retry_delay_ms = delay.as_millis() as u64,
status = status.as_u16(),
"Noise executor retrying uncommitted registry registration"
);
sleep(delay).await;
}
Err(error) => return Err(error),
}
}
}
}
#[cfg(test)]
#[path = "registration_retry_tests.rs"]
mod tests;