## What changed
- Retry transient environment-registry failures while opening the initial Noise rendezvous connection, including timeouts, interrupted response bodies, retryable HTTP statuses, and temporarily offline environments.
- Bound registry attempts with per-request and overall timeouts, exponential backoff, and a retry limit while returning permanent registry errors immediately.
- Preserve the existing one-time bundle refresh after an unauthorized WebSocket handshake, with a fresh registry retry window.
- Treat malformed complete JSON responses as terminal rather than retryable transport failures.
## Testing
- Cover transient and permanent registry responses, stalled requests, retry limits and deadlines, interrupted bodies, malformed JSON, unauthorized bundle refresh, and session-resume behavior.
GitOrigin-RevId: 5929274103702ad477e7a638ab489319a43a93fa
## Why
Transient executor disconnects could leave capability discovery and skill catalogs
stuck on a cached failure for the rest of a thread, even after the executor
reconnected.
## What changed
- Replay capability discovery after executor recovery and retry transient failures
on later requests while continuing to cache permanent failures.
- Avoid caching skill catalogs produced from failed discovery so a later step can
load the recovered catalog.
- Mark the MCP runtime dirty when recovered manifests change the projected MCP
servers, and allow discovery to be cancelled with the turn.
## Testing
- Cover same-request recovery after a disconnect and recovery on a later request.
- Cover retry classification through connection-attempt errors and skill catalog
caching after discovery recovers.
GitOrigin-RevId: a57f90844351e73ea831931f72a9ddc4e4f3335c
## What changed
- Rename `ReqwestHttpClient` to `RouteAwareHttpClient` to reflect that delegated HTTP uses Codex's shared route-aware transport.
- Use `codex_http_client` response and error types plus transport-neutral `http` and `url` types, removing the exec server's direct `reqwest` dependency.
## Testing
- Cover fragment stripping and Unicode hostname normalization for delegated HTTP requests.
GitOrigin-RevId: 8b0fc60a76004feb57198bfb4afb1371c9ceb1bd
## Why
A malicious exec-server can send out-of-order process notifications
faster than missing events arrive. The client retained every future
event in a per-session reorder map, so many tiny events or a few large
output chunks could grow orchestrator memory without bound.
## What changed
- cap each session's pending process-event reorder state at 256 events
and 1 MiB
- reject individual process events larger than 1 MiB
- release byte accounting as events publish or a session fails
- fail and detach only the affected process session when a limit is
exceeded
- let the next expected event drain a full buffer, so the limits apply
to retained future state
- apply the same bounded insertion path during reconnect recovery while
safely handling dense tail output, missing exit events, newer live
notifications, and conflicting `Closed` sequences
Sequence distance is intentionally not capped because it does not affect
allocation; the event-count and byte limits are the resource-exhaustion
boundary.
## Tests
Focused exec-server tests cover oversized output, full count/byte
buffers, gap-closing delivery, dense recovery interleaved with newer
notifications, missing exit reconstruction, and conflicting terminal
sequences.
## Scope
This is limited to per-session process-event reorder state. It does not
introduce a scheduler, quota framework, or new public configuration.
## Summary
- complete unified-exec processes from the ordered event stream instead
of issuing a final zero-wait `process/read`
- add optional executor sandbox-denial state to `process/exited`
- retain `process/read` as a retained-output and compatibility fallback
for receiver lag, sequence gaps, and legacy servers
- recover sandbox-denial state across transport reconnection
- cover the real `TestCodex` remote-exec path without adding a public
test-only event constructor
## Why
A successful one-shot tool call currently receives its output and
terminal notifications, then pays another wide-area `process/read` round
trip before returning. Staging traces showed that remote response wait
accounted for more than 99.8% of RPC time; local serialization,
queueing, and deserialization were below 0.6 ms.
## Measured impact
A direct staging A/B used the same build and route and changed only
completion mode. Each arm ran three times with 30 one-shot
`/usr/bin/true` calls per run. The table reports the median of the three
per-run percentiles.
| Metric | Final `process/read` | Pushed events | Change |
| --- | ---: | ---: | ---: |
| End-to-end completion p50 | 159.5 ms | 118.7 ms | -40.8 ms (-25.6%) |
| End-to-end completion p95 | 182.4 ms | 131.7 ms | -50.6 ms (-27.8%) |
| Completion-wait p50 | 80.1 ms | 41.5 ms | -38.5 ms (-48.1%) |
| Final `process/read` RPC p50 | 79.9 ms | eliminated | -79.9 ms |
TCP_NODELAY was enabled in both A/B arms, so its effect cancels out. The
successful, complete, in-order event path issued zero final
`process/read` calls.
## Compatibility and recovery
- new servers send `sandboxDenied` on `process/exited`
- legacy servers omit it, which triggers one compatibility
`process/read`
- broadcast lag or a sequence gap triggers a retained-output read
- recovery remains bounded by the server's existing 1 MiB
retained-output window
- complete, in-order event streams issue no completion read
- sandbox denial is attached to the exit event before consumers can
observe process completion
- server-first and client-first rollouts remain wire-compatible;
server-first realizes the latency win immediately
## Integration coverage
The `TestCodex` suite exercises four distinct remote-exec contracts:
- complete pushed output/exit/close with zero reads
- direct pushed sandbox denial with zero reads
- legacy missing denial metadata with exactly one compatibility read
- count-bounded replay eviction recovered from retained output without
duplication
## Validation
- `just test -p codex-core
exec_command_consumes_pushed_remote_process_events`: 4 passed
- `just test -p codex-core unified_exec::process_tests::`: 4 passed
- `just test -p codex-exec-server`: 294 passed, 2 skipped
- `just test -p codex-exec-server-protocol`: 5 passed
- `just test -p codex-rmcp-client`: 89 passed, 2 skipped
- focused Bazel `//codex-rs/core:core-all-test`: passed across 16 shards
- scoped `just fix` passed for core and exec-server
- `just fmt` passed
The complete workspace suite was not rerun; focused Cargo and Bazel
coverage passed for the changed behavior.
## Summary
- retry ERS `409 environment_offline` responses inside the existing
exec-server recovery loop
- keep all other registry conflicts terminal
- add focused coverage for both cases
## Root cause
When an exec server disconnects and reconnects, the client already
starts recovery and calls ERS `/connect`. During the transient executor
presence gap, ERS can return `409 environment_offline`. The retry
classifier treated every 409 as terminal, so the first response aborted
the existing 25-second recovery window before the executor came back
online. That then caused active processes to be marked lost.
This change classifies only the structured `environment_offline`
conflict as retryable. Recovery continues with the existing bounded
deadline, exponential backoff, and jitter.
## Validation
- `just test -p codex-exec-server client::recovery::tests` — 4 passed
- `just fix -p codex-exec-server` — passed
- `just fmt` — passed
- Full `just test -p codex-exec-server` reached unrelated macOS
filesystem-sandbox integration failures because nested
`/usr/bin/sandbox-exec` is denied in this environment (`sandbox_apply:
Operation not permitted`).
## Why
PR #28512 retries a failed session recovery every 100 ms. Every Noise
recovery attempt first asks the environment registry for a fresh
connection bundle, even when the eventual failure comes from the
WebSocket or initialize handshake. During an outage, that could make
each disconnected client call the registry about 250 times during the
25-second recovery window.
## What changes
All retryable Noise recovery failures now use a separate backoff
schedule:
```text
base: 500 ms -> 1 s -> 2 s -> 4 s -> 5 s maximum
actual: 500-750 ms, 1-1.5 s, 2-3 s, 4-6 s, 5-7.5 s
```
The extra 0-50% is deterministic per-session jitter so disconnected
clients do not retry together. Direct WebSocket recovery keeps the
existing 100 ms retry because it does not re-enter the registry.
richardopenai