## Why
A WebSocket close with code `1009` before any text arrives indicates that the request exceeds the peer's message size limit. Retrying the request over another WebSocket does not address that limit.
## What changed
- Classify these early size rejections as requiring HTTP fallback and switch transports without exhausting WebSocket retries, including during prewarming.
- Preserve ordinary stream retry behavior for size closes after any text frame and for other close codes.
## Testing
Add integration coverage for early size rejections during generation and prewarming, size closes after metadata, rate limits, or response creation, and ordinary closes.
GitOrigin-RevId: 4550d289bb414fec1faae2a195bcc635d394117b
## What changed
Once Responses requests fall back to HTTP after exhausting WebSocket retries or receiving HTTP 426, keep subsequent requests and turns on HTTP for the remainder of the session. Clear cached WebSocket state and remove background recovery probes, recovery trials, and their acknowledgement timeout.
## Testing
Update fallback tests to verify that a second turn stays on HTTP without additional WebSocket attempts. Adjust retry and `Retry-After` request-count assertions to exclude background probes.
GitOrigin-RevId: a4f6376604527c14db8d60f79bd21c3aaf289f70
## Why
A transient WebSocket failure previously kept the session on HTTP for its remaining lifetime, even if WebSocket connectivity recovered.
## What changed
- Probe WebSocket connectivity in the background during HTTP fallback, allowing a subsequent request to use a ready connection without delaying HTTP requests.
- Verify recovery with a real request and bound its first-response wait to five seconds. Keep HTTP 426 responses and recovery timeouts on HTTP for the rest of the session.
- Keep retries of an HTTP request on HTTP. If a recovered connection fails before output, replay the request over HTTP without consuming its ordinary retry budget; failures after output use normal retry accounting.
- Rate-limit recovery probes and honor `Retry-After` deadlines.
## Testing
Extend integration coverage for successful recovery within a turn, disconnects before and after output, full-context replay followed by incremental WebSocket requests, persistent HTTP 426 fallback, and retry deadlines.
GitOrigin-RevId: 1c139a4bbce1a67442db5622fd70ff0cc8056c9c
## What changed
- Record time spent waiting for each stream event in `codex.response.receive_wait_ms` on the existing `handle_responses` span.
- Add trace spans for startup prewarm consumption, turn-start lifecycle callbacks, assistant text flushing, and response stream error handling.
- Include the turn ID and lifecycle phase on the turn-start span and the request type on the error-handling span.
GitOrigin-RevId: 7792508c830ef05b03aed01eb840ca7e6577ad0b
## Why
Responses overload and exhausted-request errors could terminate despite server retry advice, and WebSocket-to-HTTP fallback could issue a request before the advised deadline.
## What changed
- Allow `ServerOverloaded` and `RetryLimit` errors to retry when server advice is present, within the configured retry budgets.
- Preserve `Retry-After` headers from rejected WebSocket upgrades and wait for the server deadline before falling back to HTTP.
- Keep quota, usage-limit, and policy failures terminal even when they include retry advice.
## Testing
Add coverage for HTTP overload and rate-limit recovery, WebSocket upgrade rejection and fallback timing, retry deadline preservation through error mapping, and retry limits for sampling and manual and automatic compaction.
GitOrigin-RevId: 72d30a9f078652664754dfa971fd7dc7a78f20e8
## What changed
Move content-filter guidance recording from the sampling loop into
`handle_response_stream_error`. Pass `StepContext` so the handler can resolve
model-specific guidance and record it before retry decisions for sampling
requests. Keep remote compaction excluded from guidance recording and update
callers and the existing retry-deadline test to use the new signature.
GitOrigin-RevId: 030bde1d4f1597fc4f3971ad837478e5304c5aa9
## Why
Cold Rosetta translation and cache invalidation from hard-linked executables can interfere with timed CLI tests. Sampling the clock twice also makes local retry telemetry report a delay shorter than the selected backoff.
## What changed
- Warm CLI fixtures with `--version` on macOS x86_64 before timed checks, and use copies or Unix symlinks to avoid invalidating the shared executable's Rosetta cache.
- Use one clock sample to calculate the response retry deadline and recorded delay.
- Include the retry telemetry event in delay-range assertion failures for the headerless rate-limit stream test.
GitOrigin-RevId: fc573bfff912e999cf74b94a1c6e7ae1be07974b
## Why
HTTP retries used local backoff even when the server supplied `Retry-After`. Relative retry delays also failed to account for time spent propagating errors or reporting retries.
## What changed
- Parse `Retry-After` delay seconds and HTTP dates into a monotonic deadline captured when response headers arrive.
- Honor that deadline for HTTP retries, falling back to local backoff when valid advice is absent.
- Preserve deadlines through API and Bedrock error mapping, stream retries, and exhausted retry state. Expired advice remains a zero delay instead of triggering local backoff.
## Testing
Add coverage for header parsing, deadline countdown and expiry, error mapping after exhausted HTTP retries, and delayed stream retry notifications. Update Responses and remote compaction tests to verify that HTTP retries honor `Retry-After`.
GitOrigin-RevId: a2eff02946e5524f29e06f7ba1431e3d339261a4
## What changed
Replace `is_retryable()` with `retry_delay(retry_count)`, which returns `None` for terminal errors and a delay for retryable errors. Prefer server advice when present; otherwise use exponential backoff with jitter. Keep `server_retry_delay()` for callers that need only the server-provided delay.
Route sampling and remote compaction errors through the shared response stream handler, which now checks retryability and obtains the delay together. Preserve server-only retry timing when the retry budget is exhausted.
## Testing
Add coverage for attempt-dependent backoff, zero-duration server advice, and terminal errors that remain non-retryable despite server advice. Update API mapping and provider tests to use the combined retry API.
GitOrigin-RevId: 903f92935e71dea043afe1fad8ccb9389471aa75
## Why
Transient rate limits can end automatic approval reviews prematurely, and review failures currently report high risk even when no assessment completed.
## What changed
- Retry rate limits and recoverable exhausted-stream errors, while excluding non-transient HTTP failures.
- Preserve server retry delays after stream retries are exhausted and honor them within the review deadline. Scope retry advice to the current turn so reused sessions cannot apply stale delays.
- Keep failed reviews denied, but leave risk and authorization unset and explain that the review could not complete without declaring the action unsafe.
## Testing
Add an integration test covering rate-limit recovery through approval and tool execution, asserting that the action executes exactly once after approval. Update failure assertions to check absent assessment fields and the review-failure explanation.
GitOrigin-RevId: 1163cfde35c6b8eb23b6f24f1f24461b86ded838
## Why
Thread settings can change while a turn is running, and delegated review or
compaction steps can select a different model. Each request needs a consistent
set of settings resolved against the model that will execute it.
## What changed
- Capture immutable resolved settings for each model step, including model
metadata, reasoning options, service tier, approvals, and personality.
- Keep in-flight steps on their captured settings while applying thread updates
to subsequent work.
- Resolve inherited reasoning summaries and filter service tiers against the
selected model and feature configuration for review and compaction steps.
## Testing
Added coverage for settings updates during paused turns, model changes,
previous-model compaction, review-model defaults, and service-tier filtering.
GitOrigin-RevId: fca4b3b305379f1bf26a5cac75ee0dd1747ac1f6
## What changed
- Add the stable, default-enabled `unbounded_connection_retries` feature.
- Require the feature for unbounded sampling retries after connection failures.
When disabled, connection failures use the existing bounded retry and
transport fallback path.
- Expose the feature in the generated configuration schema.
GitOrigin-RevId: bd80d02ef065ba924f805c29b639752fc817ed64
## What changed
- Classify HTTP connection failures separately from other network errors without exposing request URLs.
- For sampling requests, retry connection failures with exponential delays from 5 to 60 seconds and show a `Reconnecting... waiting for network` stream error.
- Preserve the normal stream retry budget while waiting for the provider to become reachable. Keep the existing bounded retry behavior for other retryable errors.
## Testing
- Verify connection errors are classified without leaking URL contents.
- Verify a turn recovers after its provider becomes reachable and still applies the configured retry limit to a subsequent incomplete stream.
GitOrigin-RevId: 646553290c865a1332abd30c4a64ed9266bbfc6f
## What changed
- Wrap `CodexErrorDetails` and an optional retry delay in `CodexErr`, allowing any mapped error to preserve server-provided retry timing.
- Generate the payload-free `CodexErrKind` classification alongside the error details and reuse it for analytics.
- Update error handling sites to inspect `CodexErr::details()` while preserving existing display, debug, protocol mapping, and retryability behavior.
## Testing
- Add coverage for legacy debug formatting, error-specific retryability, and retry-delay propagation through API error mapping.
GitOrigin-RevId: d3ab8a305f2a2ee21d0c0a8c8c388b06dda9c59a
## What changed
- Add structured `turn_id`, `retries`, `max_retries`, and `sampling_error` fields to sampling-stream retry warnings.
- Add a focused test that captures the warning and verifies its retry and error context.
GitOrigin-RevId: dea868875d162b3f97b3c8a256359278f7267c1c
## Why
#23951 added remote compaction v2 retries, but it left the retry and WS
-> HTTPS fallback behavior duplicated between normal Responses turns and
compaction. This follow-up centralizes the common retry handling so
future changes to fallback, retry delay, retry notifications, and retry
sleep do not have to be kept in sync across both callsites.
## What changed
- Added `core/src/responses_retry.rs` with a shared handler for
retryable Responses stream errors.
- Reused that handler from normal turn sampling and remote compaction
v2.
- Kept each callsite responsible for its retry budget: normal turns
still use `stream_max_retries`, while compaction v2 still uses
`min(stream_max_retries, 2)`.
- Preserved caller-specific behavior around non-retryable errors,
context-window errors, usage-limit errors, and compact-specific final
failure logging.
The shared handler now owns:
- WS -> HTTPS fallback warning emission
- retry delay selection, including server-requested stream retry delay
- retry logging
- first-WebSocket-retry notification suppression
- `Reconnecting... n/max` stream-error notification
- sleeping before the next retry attempt
## Verification
- `cargo test -p codex-core remote_compact_v2`
- `cargo test -p codex-core websocket_fallback`
- `just fix -p codex-core`
Did not run the full workspace test suite.
---------
Co-authored-by: jif-oai <jif@openai.com>
Charlie Marsh