Commit Graph
84 Commits
Author SHA1 Message Date
richardopenai 431185b78b Defer environment disconnect notifications until recovery fails (#55401)
## Why

A temporary exec-server transport loss could emit a thread environment disconnect notification while the session was still recovering.

## What changed

- Add `EnvironmentConnectionState::Recovering` for interrupted transports and pending session recovery.
- Suppress thread lifecycle notifications during recovery, emit a connected notification after successful recovery, and report disconnection when recovery fails.
- Ignore recovery callbacks from retired clients so they cannot overwrite the connection state during refresh.

## Testing

Add coverage for fast and near-deadline recovery, recovery timeout, rejected session resume, and retired-client callbacks. Extend app-server coverage to verify one lifecycle notification per selected thread and usable `environment/info` after recovery.

GitOrigin-RevId: d6dce5cc5264af29b6f993daa209b93102122e40
2026-10-19 22:55:42 +00:00
Adam Perry @ OpenAI a45b8ccb01 Add streaming file replacement to filesystem abstractions (#54505)
## What changed

Add `write_file_stream` to `ExecutorFileSystem` and `EnvironmentAccess`, preserving the accessor's captured sandbox permissions. Local and streaming-capable remote filesystems replace files one bounded chunk at a time and return the byte count. Older executors buffer the input before sending a single `writeFile` request.

Close remote handles on completion or failure, perform best-effort cleanup after cancellation, and invalidate pending metadata lookups. Document that failure or cancellation can leave a truncated or partial file and that in-flight I/O may outlive cancellation.

## Testing

Add coverage for sequential writes, binary and empty files across local and remote filesystems, source failures, handle cleanup, metadata invalidation after cancellation, and compatibility with older executors.

GitOrigin-RevId: d3eb44e0ac2f0f336e9e157d686464f4d45a3f26
2026-10-16 04:42:32 +00:00
Anthony Tafoya 703d253ed6 Wire V2 capability discovery through exec-server RPC (#53723)
## What changed

- Register `capabilities/discoverV2` for initialized sessions and expose it through `ExecServerClient` and remote `Environment` APIs, checking executor support before calling it.
- Reuse startup discovery locations and retry failed prewarming on demand. Replay discovery once after executor recovery while preserving the request parameters.
- Track whether all required filesystem watches registered successfully and reload locations and metadata on each request when registration fails.

## Testing

Add coverage for RPC metadata refresh, omission of skill instruction bodies, sandbox isolation, startup scan recovery, disconnect replay, and successful or failed watch registration.

GitOrigin-RevId: 726e8f7e8b4d835b877f31d1d8a9f8eea1ad9c8b
2026-10-13 17:22:34 +00:00
aaronl-openai 7efd184c24 Add observers for initial exec-server connection attempts (#52724)
## What changed

Expose `Environment::observe_connection_attempts` and `ConnectionAttemptOutcome` to report elapsed time and success, failure, or cancellation for initial connections. Measure from the end of provisioning through client installation, including connection retries.

The first registered observer wins. Past attempts and reconnections are omitted.

## Testing

Extend tests to assert failure followed by success when startup is retried, and a single cancellation notification when an environment is replaced during startup.

GitOrigin-RevId: 88300f38dcf9428b824c5ff36aa1b40d7d466b6f
2026-10-09 23:28:38 +00:00
Henry Levy 6e093c6081 Add timeout-aware exec-server configuration reads (#52679)
## Why

Remote configuration reads need a bounded wait without disrupting valid in-flight work on connections that cannot reconnect.

## What changed

- Add `read_environment_config_with_timeout` to `ExecServerClient` and `Environment`, bounding RPC sending and receiving after connection establishment. Local reads retain their existing behavior.
- Return a typed `ExecServerError::RpcTimedOut` and classify it as retryable for read-only preparation.
- Share timeout handling with environment-info requests. Close the timed-out transport and request recovery only when a reconnect strategy exists; otherwise, keep the connection open.

## Testing

Add a regression test verifying that a timed-out configuration read preserves an in-flight stdio read and allows subsequent reads on the same connection, including after a late response.

GitOrigin-RevId: 6785d5846f4c3b07a6e7c505e359238609c3a6b6
2026-10-09 19:22:21 +00:00
Nick Steele ecc78e4cf5 Add renewable EMA HTTP authentication and credential versioning (#49987)
## What changed

- Add `EmaAuthenticatedHttpClient` to cache and renew resource bearers, serialize concurrent renewals, and include authentication time in the request deadline. Confine bearers to the configured MCP endpoint and reject redirects.
- Return authentication-required errors for `401` responses without replaying MCP operations. Preserve a newer cached bearer when a delayed response rejects an older one.
- Add revocable request authority that cancels pending requests and ends response streams.
- Pin enterprise credentials to a persistent version advanced on save or deletion. Use short credential locks and invalidate rejected grants only when their record and version are still current, preserving newer logins.
- Hide keyring account identifiers from enterprise credential errors and traces while preserving authentication failure classification.

## Testing

Add tests for token expiry and renewal, endpoint confinement, redirects, request deadlines, revocation during renewal and streaming, delayed `401` responses, credential replacement and logout races, and credential error privacy.

GitOrigin-RevId: ee3e5cf7ecbba3d09e1ab29ed4fbb2a35eacc60d
2026-10-01 13:57:48 +00:00
Adam Perry @ OpenAI e7798c9944 Add capability-gated writable file streams to the exec-server client (#49805)
## Why

Older executors can ignore the file open mode and silently return a read-only handle for a writable open.

## What changed

Add `ExecServerClient::fs_write_block` and require the `file_write_streaming` capability for both block writes and `FsOpenMode::Replace` opens. Return a protocol error before sending either request when the executor lacks support, while allowing read-only opens.

## Testing

Add a WebSocket regression test verifying that unsupported writable opens and block writes are rejected locally and read-only opens still reach the executor.

GitOrigin-RevId: f206e1da80c12af76ead14c75f8ffe71f82cce89
2026-10-01 01:36:03 +00:00
Adam Perry @ OpenAI c538fbabe5 Share cached exec-server environment info with Arc (#49798)
## What changed

Return `Arc<EnvironmentInfo>` from `ExecServerClient::environment_info()` so repeated reads and cloned clients share cached metadata without cloning its contents. Wrap metadata from initialization and lazy RPC fetches in `Arc`, while preserving the owned return value in `LazyRemoteExecServerClient`.

## Testing

Extend the cache test to assert pointer identity across cloned clients for both initialization-provided metadata and lazy fetching after connecting to a legacy server.

GitOrigin-RevId: 3f56f4d51a4aebc0c4821c0b128a70efb23933d4
2026-10-01 01:22:28 +00:00
Vivian Fang 67352b5b4a Recover exec-server sessions after environment info timeouts (#49407)
## Why

A stuck transport can fill the outbound queue, leaving an `environment/info` request blocked while sending. The timeout must cover sending as well as waiting for a response.

## What changed

Wrap the live metadata RPC in a 30-second timeout. On timeout, close the probed connection and request session recovery, returning the timeout error without retrying the failed request. Document the experimental `environment/info` endpoint and its timeout behavior.

## Testing

Extend the WebSocket session recovery test to cover an unanswered `environment/info` request, checking that it times out, closes the connection, and resumes the same session while retaining the client instance.

GitOrigin-RevId: 9035d2edab3626e780d8bd40ebcdf0ca327bfddb
2026-09-30 00:22:43 +00:00
richardopenai 985cf47a4e Allow provisioned executors more time to come online (#48575)
## Why

A provisioned executor can still be resuming after readiness is reported. Initial connection attempts can exhaust the ordinary registry retry limits before the executor comes online.

## What changed

Retry `environment_offline` registry responses during initial Noise rendezvous connections for provisioned environments until a fixed five-minute deadline, starting after provisioning succeeds. Keep the existing retry limits for other registry errors and stop on permanent failures.

## Testing

Add tests covering the five-minute deadline, ordinary limits for other errors and stalled requests, and termination on a later permanent error. Verify that readiness and info requests remain pending through an extended offline period and succeed using the same environment handle once the executor comes online.

GitOrigin-RevId: e22211499d9ec2f3590e75224eb39e2e4bf3538d
2026-09-27 00:01:17 +00:00
Adam Perry @ OpenAI 6fa3bcaae8 Add exec-server RPC timing and process startup tracing (#47680)
## Why

Detached work can keep request spans open beyond response enqueueing. Record explicit phase timings to distinguish request handling from the full span lifetime.

## What changed

- Record server dispatch and response enqueue offsets from request receipt, plus a response enqueue event for HTTP responses that bypass the dispatcher.
- Emit client request enqueue and response receipt events, including success or server-error outcomes and reader-to-caller delay. Carry completion timestamps to the awaiting task so receipt events retain its tracing context.
- Add spans for process startup, sandbox preparation, shell snapshot preparation and capture, and process spawning. Record process identifiers only when nonempty and at most 64 bytes long.

GitOrigin-RevId: 237db898b3284fe8a6a36cb0055b3c7991d8e8a4
2026-09-23 22:10:28 +00:00
acrognale-oai db3cb33eca Honor network policy in remote control and recover remote execution (#47410)
## What changed

- Route remote-control enrollment, pairing, client management, and WebSocket connections through the authenticated HTTP client factory. Keep policy denials distinct from authentication failures and wait for policy changes before reconnecting.
- Preserve policy denials through exec-server connection failures so later connection attempts and capability discovery can recover. Keep direct executors running through temporary policy outages while rejecting permanent policy failures.
- Build cached HTTP transports on blocking workers, serialize construction, and retain successful builds after callers time out so request deadlines cover queued construction.

## Testing

Add regression coverage for policy recovery during remote startup, direct registration, accepted WebSocket reconnection, and capability discovery; queued transport construction timeouts; and remote-control WebSocket fallback with invalid custom CA files.

GitOrigin-RevId: c81db18e83de957175bb213d165281ac701089dd
2026-09-23 00:48:53 +00:00
viyatb-oai 328feb0c29 Track executor registrations across connection refresh and recovery (#46555)
## Why

An executor can renew its registration while retaining its Noise identity. Connection refresh must distinguish registrations even when their keys match, and a stale registry lookup must not retire a session that recovery has already renewed.

## What changed

- Track the installed session's registration and expose it through `Environment::cached_executor_registration_id()` without connecting. Publish registration changes only when the new connection is installed.
- Require both registration and executor key to match before refresh reuses a session. Retry stale lookups and prevent recovery from installing a connection after retirement.
- Pin session recovery to the original Noise key while allowing registration renewal to preserve the session and running processes.
- Expose `CodexThread::active_turn_environment_selections()` so hosts can authorize steering against the active turn's selections, including starting and failed environments, independently of later settings updates.

## Testing

Add regression coverage for registration replacement, renewal with a running process, stale refresh lookups, missing sessions, and rejection of changed Noise keys. Extend active-turn tests to check selection snapshots across settings updates, deferred startup, and turn completion.

GitOrigin-RevId: 4395bc6c57f137691887e6585f52dcc5b958101c
2026-09-19 01:27:02 +00:00
Adam Perry @ OpenAI 841b5490b2 Preserve filesystem sandbox policy context when the cwd disappears (#46112)
## Why

Removing the selected working directory can prevent filesystem sandbox helpers from launching, even when the requested absolute paths remain accessible. Permission rules must stay anchored to the selected directory while those operations continue.

## What changed

- Require a policy `cwd` in `FileSystemSandboxContext` and launch filesystem helpers from the filesystem root while preserving the policy directory and workspace roots.
- Carry explicit `policyContext` in filesystem RPCs, preserving legacy wire fields and resolving omitted directories from older clients at executor ingress.
- Keep permission paths as executor file URIs and validate host compatibility where they are enforced.
- Bind Windows relative denial globs to the policy directory before changing the helper's launch directory, preserving home-relative patterns.

## Testing

Add regression coverage for `apply_patch` after working-directory removal, legacy RPC directory fallbacks, cross-platform permission URI transport, and Windows relative read denials. The patch regression verifies that an allowed file is updated while an explicitly denied file remains unreadable and unchanged.

GitOrigin-RevId: b0f4db722b27cb72ec129fc297c85732afac11f7
2026-09-17 04:37:54 +00:00
felixxia-oai dbe2f6d528 Expose a stable executor build identity in environment metadata (#43513)
## Why

Executor compatibility checks need a build identity that distinguishes commits
and compiler targets independently of the package release version.

## What changed

- Add optional `providerId` to exec-server environment metadata, cached at startup
  and returned by initialization and `environment/info`.
- Derive the ID as SHA-256 of `git:<lowercase commit>:<target>`. It identifies a
  standard build configuration, not exact executable bytes, and is omitted when
  the commit stamp or target is unavailable or invalid.
- Embed the compiler target in `BuildInfo`, preserve compatibility with historical
  metadata without a target, and stamp Cargo release builds with `STABLE_GIT_COMMIT`.
- Group Bazel build-script argument files under one `--arg-file` flag to reduce
  Windows command-line length while preserving file order.

## Testing

Add deterministic build-ID vectors across targets, commit normalization and invalid
input coverage, and historical metadata compatibility tests. Extend exec-server
coverage for metadata caching and ensure runtime environment overrides cannot
replace the executor's build identity.

GitOrigin-RevId: 125a18c23de7ad006571940ba305836376c983f4
2026-09-07 16:55:06 +00:00
jif 8e85265c39 Handle pending network reviews after process completion (#42746)
## Why

A remote process can finish while a network policy review is still pending. Normal process cleanup should withdraw that review without turning the completed command into a review failure or losing its output.

## What changed

- Record whether a network policy request was withdrawn because the process finished, was cancelled, lost its executor connection, or timed out.
- Treat normal process completion as cleanup while retaining fail-closed behavior for other cancellation causes.
- Preserve explicit network denials before policy persistence so cleanup cannot replace the reported call outcome.

## Testing

Add an integration test that completes a remote process during a pending network review and verifies that the command reports its successful exit and output without approving the withdrawn request.

GitOrigin-RevId: 7f42d75631ee29eba43bf04cc953eea490f553fc
2026-09-04 12:08:02 +00:00
bkotsopoulos 781c183c3b Bound Noise handshakes by the exec server initialization timeout (#42623)
## What changed

- Wait for the authenticated Noise handshake before sending the JSON-RPC
  `initialize` request.
- Share the configured initialization timeout across the handshake and RPC,
  while preserving the configured duration in timeout diagnostics.
- Record the handshake and initialization RPC as child spans of the existing
  initialization span, and terminate transport tasks when handshake readiness
  fails.

## Testing

- Cover stalled handshake timeouts and cleanup of the underlying WebSocket.
- Cover timeout reporting after the handshake has consumed part of the budget.

GitOrigin-RevId: c3eb5f7cfedd8d155fde5b7614fc2eb399ce337c
2026-09-03 22:16:19 +00:00
ostepanian 801ca0d0d1 Support trusted headers for remote exec WebSockets (#42606)
## What changed

- Add `RemoteEnvironmentOptions` and `upsert_environment_with_options` so embedding hosts can attach trusted HTTP headers to remote exec-server WebSocket handshakes.
- Preserve the headers across session reconnects while redacting them from debug output.
- Reject invalid, duplicate, and WebSocket-controlled headers, and require `wss://` for non-loopback destinations.
- Connect header-bearing loopback WebSockets directly, with DNS results restricted to loopback addresses.

## Testing

- Cover header validation and redaction, delivery on initial connections and reconnects, and loopback destination enforcement.

GitOrigin-RevId: f2e2b0456c8e33c13fd8ab4ee040a8b5c15e885d
2026-09-03 20:01:07 +00:00
Anton Panasenko e6249b5296 Recover deferred environments after provisioning failure (#42388)
## What changed

- Allow a valid Ready report to replace a deferred environment's provisioning failure while preserving the same environment instance.
- Treat provisioning failures as retryable and reconnect when a completed or concurrently publishing startup attempt contains a stale failure.
- Report provisioning failures from `status()` without starting a connection, and retain Ready state when a late failure arrives.

## Testing

- Cover recovery after failed and invalid provisioning reports, capability-root resolution after recovery, and the race between Ready and publication of an earlier startup failure.

GitOrigin-RevId: cce419403248168da81bc5a6ab20f919aed81142
2026-09-02 23:49:29 +00:00
viyatb-oai f53c91be2c Add attributed exec process lifecycle telemetry (#42373)
## Why

Exec-server process and network-policy events need stable attribution to the
launching tool call and executor without recording process payloads.

## What changed

- Add optional `ExecMetadata` to `ExecParams` and propagate the thread and tool
  call IDs from unified exec.
- Emit bounded OpenTelemetry events for process start, spawn failure, sandbox
  denial, and exit. Correlate them with the launch trace and registry-issued
  executor identity while excluding arguments, paths, environment values,
  output, and error text.
- Preserve launch attribution across long-running processes and reconnects, and
  attach the same metadata to network-policy audit events.
- Keep the protocol backward compatible when metadata is omitted, and prevent
  invalid trace headers from inheriting an unrelated active span.

## Testing

Add coverage for metadata serialization and propagation, lifecycle event
fields, trace relationships, reconnect behavior, spawn failures, sandbox
denials, and network-policy attribution.

GitOrigin-RevId: 7aa480a7289c73cb95e2c124c35500bb6f0d5084
2026-09-02 21:58:58 +00:00
felixxia-oai 8d32abcd01 Report the exec-server release version in environment info (#42270)
## Why

Clients need the executor's release version to make version-based compatibility decisions.

## What changed

- Add `executorVersion` to `EnvironmentInfo` and return it from both `initialize` and `environment/info`.
- Resolve the package release version at process startup so the advertised value remains stable for the server's lifetime.
- Use `0.0.0` when the version is unknown or omitted by a legacy executor.

## Testing

- Cover packaged and source-build initialization, startup caching, protocol serialization, and client-side metadata caching.

GitOrigin-RevId: a0c7f0b701a8e7e50345cd5fb1b5c5d2d38ff3b1
2026-09-02 12:07:32 +00:00
Rasmus Rygaard eb49f491c6 Add explicit remote executor connection refresh (#40710)
## Why

Planned executor replacement needs a fresh session without waiting for the old
session's transient-disconnect recovery to finish.

## What changed

- Add `Environment::refresh_connection` for remote Noise registry-backed
  environments. It performs a fresh registry lookup, reuses a healthy session
  when the executor identity is unchanged, and connects to a replacement when
  it has changed.
- Retire superseded sessions and connection attempts so they cannot publish
  stale state, accept late RPC results, or replay outstanding work.
- Preserve the existing environment and filesystem handles while replacing the
  underlying client, and require a live status probe before refresh succeeds.

## Testing

Add coverage for replacement and session reuse, recovery and connection races,
lookup and handshake failures, handle preservation, and late RPC responses.

GitOrigin-RevId: f1d11208cbfe8af8feb25f6b6b8100da82169a99
2026-08-25 21:23:17 +00:00
cgst-oai 2df6705423 Cache executor metadata from initialization (#40343)
## What changed

- Include `environmentInfo` in the exec-server `initialize` response and seed the Rust client's metadata cache from it.
- Fall back to one lazy `environment/info` request when connecting to older servers that omit initialization metadata, then share the cached result across client clones and session recovery.
- Keep app-server `environment/info` requests uncached so they continue to probe the executor.

## Testing

- Cover initialization metadata, the legacy-server fallback, shared caching, accepted WebSocket environments, and repeated uncached app-server probes.

GitOrigin-RevId: 380132d9db183998d73e8c6b0954e4d70c7e8406
2026-08-24 06:43:07 +00:00
ostepanian 9e680a52e7 Support host-accepted exec-server WebSockets (#39786)
## What changed

- Add `EnvironmentManager::from_accepted_websocket` so embedding hosts can
  construct a remote environment from an already accepted and authenticated
  Axum WebSocket.
- Add `replace_accepted_websocket` to retire the current transport and resume
  the same exec-server session on a host-supplied replacement connection.
- Serialize replacement handoffs, reject overlapping replacements, and release
  the handoff claim when a replacement attempt is cancelled or fails.

## Testing

- Cover initial connection validation and immediate environment readiness.
- Verify replacement retry behavior and recovery of a running process and its
  output after reconnecting.

GitOrigin-RevId: 1f2ab7bcf7b5abbbece5c101801432dc84a8058d
2026-08-20 19:33:48 +00:00
richardopenai 5663754f62 Retry transient registry failures during initial exec connection (#39777)
## What changed

- Retry transient environment-registry failures while opening the initial Noise rendezvous connection, including timeouts, interrupted response bodies, retryable HTTP statuses, and temporarily offline environments.
- Bound registry attempts with per-request and overall timeouts, exponential backoff, and a retry limit while returning permanent registry errors immediately.
- Preserve the existing one-time bundle refresh after an unauthorized WebSocket handshake, with a fresh registry retry window.
- Treat malformed complete JSON responses as terminal rather than retryable transport failures.

## Testing

- Cover transient and permanent registry responses, stalled requests, retry limits and deadlines, interrupted bodies, malformed JSON, unauthorized bundle refresh, and session-resume behavior.

GitOrigin-RevId: 5929274103702ad477e7a638ab489319a43a93fa
2026-08-20 18:48:17 +00:00
jif d0cc662b8c Cache shell snapshots in the exec server (#39756)
## What changed

- Add the `shellSnapshotV2` executor capability and an optional shell snapshot request to `ExecParams`.
- Capture and restore Unix shell state and profile exports from an in-memory, attachment-scoped cache for `bash`, `zsh`, and `sh`.
- Apply environment policies, runtime `PATH` entries, sandbox context, and live managed-proxy settings when preparing restored commands.
- Bound snapshot size, capture time, scope length, and cache capacity, and fall back to the original command when capture fails.

## Testing

- Cover local, remote, TTY, sandboxed, and supported-shell execution, plus environment filtering, proxy handling, in-memory reuse, and capture failure fallback.

GitOrigin-RevId: 624f747972c249c88c6f10f42cf0af97b75b5541
2026-08-20 17:39:06 +00:00
viyatb-oai 15fde8c1f2 Forward executor network policy decisions for auditing (#38670)
## What changed

- Add a best-effort `network/policyDecision` notification for final domain and non-domain policy decisions made by executor-local proxies.
- Validate notifications against the active process on the controller and emit audit events with controller-trusted session and execution metadata.
- Reserve outbound RPC capacity so audit notifications cannot block control messages, and expose valid `chatgpt-account-id` header values for audit attribution.

## Testing

- Cover notification serialization, proxy decision capture, executor-to-controller delivery, trusted metadata handling, and reserved RPC capacity.

GitOrigin-RevId: a39f96a6b3d9401c03d54eaef5b9a6d3fe0da78b
2026-08-14 23:42:40 +00:00
hesham-oai 588e18aae5 Recover capability discovery after executor disconnects (#38420)
## Why

Transient executor disconnects could leave capability discovery and skill catalogs
stuck on a cached failure for the rest of a thread, even after the executor
reconnected.

## What changed

- Replay capability discovery after executor recovery and retry transient failures
  on later requests while continuing to cache permanent failures.
- Avoid caching skill catalogs produced from failed discovery so a later step can
  load the recovered catalog.
- Mark the MCP runtime dirty when recovered manifests change the projected MCP
  servers, and allow discovery to be cancelled with the turn.

## Testing

- Cover same-request recovery after a disconnect and recovery on a later request.
- Cover retry classification through connection-attempt errors and skill catalog
  caching after discovery recovers.

GitOrigin-RevId: a57f90844351e73ea831931f72a9ddc4e4f3335c
2026-08-13 19:14:30 +00:00
richardopenai 4c5fc230a9 Retry transient exec-server startup failures (#38020)
## Why

A retryable failure during the initial remote exec-server connection left the
environment disconnected on subsequent readiness checks.

## What changed

Route retryable startup errors through the existing reconnect path. This lets
later `get()` and `wait_until_ready()` calls establish a replacement client,
while concurrent callers share the same reconnect attempt.

## Testing

Add a WebSocket test that fails the initial handshake, then verifies that later
readiness and client requests share a successful replacement connection.

GitOrigin-RevId: 82d425af0a73331884439b4350a7a1052d4ae865
2026-08-11 15:33:52 +00:00
sayan-oai 95c7265e84 Add executor-local config reads to the exec server (#37408)
## What changed

- Add the `environmentConfig/read` RPC for selecting literal TOML paths from executor-local config and requirements layers.
- Return layer precedence, cloud insertion points, source and base-directory metadata, and executor home and hostname context without normalizing path-bearing values.
- Expose the operation through both remote clients and local `Environment` instances, with invalid selectors reported as invalid parameters.

## Testing

- Cover projected remote config reads and rejection of empty selectors.

GitOrigin-RevId: fee15bf833de6bcb2058d405ed5b1d5d928d7218
2026-08-07 08:32:30 +00:00
TAFOYA-OAI f5345f1ee8 Track provisioned environment state across registration (#37147)
## What changed

- Add pending, ready, and failed provisioning states for Noise environments.
- Preserve the same environment instance whether provisioning is reported before or after materialization, and reject conflicts with ordinary environments.
- Make readiness and failure reports idempotent while rejecting contradictory terminal transitions.
- Delay connection attempts until a provisioned environment is selected and provisioning succeeds.

## Testing

- Cover status reports before and after materialization, terminal failures, repeated and contradictory reports, and replacement between ordinary and deferred environments.

GitOrigin-RevId: 4360a8f2a80c1a99a1dc9257e5d77c07b72b8eb3
2026-08-05 20:08:41 +00:00
viyatb-oai 0042b00986 Record normalized sandbox violation events (#36207)
## Why

Filesystem denials and managed-network blocks did not share a structured event
shape, requiring downstream consumers to rediscover enforcement paths and parse
backend-specific output. See https://github.com/openai/codex/pull/17573.

## What changed

- Add normalized filesystem and network violation types in `codex-sandboxing`
  and emit them through a shared tracing seam.
- Classify filesystem denials by backend and reason, retaining an optional path
  and bounded output snippet, and preserve managed-network block context.
- Report the sandbox type through exec-server responses so unified exec can
  classify remote denials without guessing; omitted values remain compatible
  with older peers.
- Record violations from exec, apply-patch, shell-escalation, unified-exec, and
  managed-network enforcement paths without changing denial behavior.

## Testing

- Cover filesystem classification, path extraction, `SIGSYS`, network event
  conversion, protocol compatibility, and remote sandbox-type propagation.

GitOrigin-RevId: d673173b4fa6bdf0a24421194a8a61c81fab9c96
2026-07-30 19:23:33 +00:00
viyatb-oai 462ed19a05 Enable network policy callbacks for remote exec (#35652)
## What changed

- Forward remote managed-network policy requests to the controller-side decider when Guardian review is enabled.
- Bound callbacks using the configured permission-hook and Guardian review timeouts, restore trusted execution attribution, and recheck live baseline policy before requesting a decision.
- Cancel pending decisions when the process or execution scope ends, and propagate EOF across the sandbox proxy bridge with half-closes.

## Testing

- Cover callback enablement, strict allowlist behavior, live policy updates, trusted attribution, and execution-scope cancellation.

GitOrigin-RevId: 83b47657c8c4546dfe98eec8b272833e96a46b7e
2026-07-27 18:07:31 +00:00
viyatb-oai 3a08af44b2 Handle exec-server network policy requests in the client (#35359)
## What changed

- Add client-side handling for exec-server network policy requests, including request validation, per-process decision routing, and allow, deny, or ask responses.
- Bound concurrent callbacks and fail closed when requests are invalid, exceed capacity, time out, or outlive their process session.
- Replace the callback opt-in flag with a launch-level `policyDecisionTimeoutMs` value and include transport overhead in the executor timeout.
- Preserve callback admission limits across connection recovery and clean up policy state when process startup is abandoned or a session ends.

## Testing

- Cover decision forwarding, invalid and excess requests, cancellation during process cleanup, abandoned starts, timeout propagation, and launch-config serialization.

GitOrigin-RevId: 6ee2a0139e0f8170cf1a5cdd646bd811b7d3c1f6
2026-07-25 16:03:00 +00:00
rphilizaire-openai c3e926e61c Trace remote exec-server connection setup (#35275)
## What changed

- Preserve the current tracing span and subscriber when lazy remote environment
  startup runs in a background task.
- Add spans for the remote connection, environment registry, Noise, and
  rendezvous WebSocket stages, including relevant environment and registration
  identifiers.
- Propagate the current trace context to environment registry connection and
  harness-key validation requests.

GitOrigin-RevId: 0044c3eb94d59c6a6b92752d3d372195be4adb81
2026-07-25 01:39:17 +00:00
Celia Chen 94ebae725e Route exec-server WebSockets through configured proxies (#35056)
## Why

Remote environment connections need to honor Codex's effective outbound proxy policy, including when a rendezvous connection reconnects.

## What changed

- Pass the configured `HttpClientFactory` into remote environment transports and use `WebSocketConnector` for exec-server and rendezvous WebSockets.
- Resolve proxy routes asynchronously so these connections can use the configured system proxy.
- Add connector options that preserve Tungstenite's default TLS behavior and enable `TCP_NODELAY` for latency-sensitive rendezvous traffic.

## Testing

- Verify prepared remote environments connect through a configured system proxy.
- Verify initial and reconnected encrypted relay peers use the system proxy.
- Cover default TLS selection and opt-in `TCP_NODELAY` behavior in the WebSocket client.

GitOrigin-RevId: 8a8da2116e37cb3a891269d0c0b037986fecdd3c
2026-07-24 00:16:42 +00:00
Celia Chen d45055ae58 Route environment registry requests through the shared HTTP client (#35034)
## Why

Noise environment registry requests need to follow the exec server's effective
outbound proxy policy without exposing registry URLs or response headers in HTTP
diagnostics.

## What changed

- Build the registry client from the supplied `HttpClientFactory` and use a
  route-aware API client with redirects and request logging disabled.
- Defer construction of the Noise connection provider until the outbound HTTP
  policy is available.
- Map route-aware request failures into registry errors while retaining timeout
  detection across response body reads.

## Testing

Add coverage for system-proxy routing, sensitive registry metadata redaction,
stalled response-body timeouts, and prepared Noise configuration validation.

GitOrigin-RevId: d312dfe037f72732085bf38109af44df76ed0b53
2026-07-23 23:25:53 +00:00
Charlie Marsh 221a341029 Remove unused Rust helpers (#34345)
## What changed

- Remove unused APIs and test helpers across the Rust workspace.
- Route Linux sandbox test setup through the shared exec request path.
- Drop dependencies that became unnecessary after the cleanup.

GitOrigin-RevId: 48fd9b2b1575bad7add0819786b4d8ec88d8486f
2026-07-20 13:45:29 +00:00
viyatb-oai 9f6c29e281 Launch managed network proxies on remote executors (#33906)
## Why

Remote executions need managed-network proxy listeners in the executor so their
loopback proxy addresses are reachable by the launched process.

## What changed

- Add a capability-gated exec-server protocol field for executor-local proxy
  launch configuration, including network policy, audit metadata, and execution
  attribution.
- Start the proxy while preparing a remote process, replace inherited proxy
  environment variables with its local addresses, and derive the sandbox
  context from its listeners.
- Keep the proxy alive until inherited output streams close, then shut it down.
- Reject unsupported remote settings such as MITM and credential injection.

## Testing

- Cover configuration round trips and rejection of unsupported settings.
- Verify executor-local startup, blocked-domain enforcement, protocol
  compatibility, and proxy lifetime through process closure.

GitOrigin-RevId: c984f54e3e600aa9ebcbf8cf4574046e2c199d11
2026-07-17 21:20:14 +00:00
Bryan Ashley 08e30a2e4e Add batched executor capability discovery (#33852)
## Why

Selected capability roots can contribute plugins, MCP servers, connectors, and
skills. Discovering each contribution separately requires repeated access to the
executor filesystem.

## What changed

- Add the `capabilityRoots/discoverV1` exec-server RPC to scan selected roots and
  materialize recognized plugin manifests, configuration files, skill
  instructions, and skill metadata in one bounded request.
- Add the opt-in `executor_capability_discovery` feature, with a thread-scoped
  cache and per-step snapshot shared by MCP and skill discovery.
- Parse MCP, connector, and skill contributions from the materialized snapshot,
  including serving cached skill instructions without another filesystem read.

## Testing

- Cover discovery limits, manifest precedence, root-local failures, cache reuse,
  plugin contributions, and parity with the existing environment skill loader.

GitOrigin-RevId: f98fd2321cafb58c596db02da1f83c09d8eb375d
2026-07-17 15:45:48 +00:00
sayan-oai 3afbd8dd45 Report selected environment connection transitions (#33251)
## What changed

- Track connected and disconnected states across initial remote exec-server connections and reconnection attempts.
- Emit experimental `thread/environment/connected` and `thread/environment/disconnected` app-server notifications for each thread selecting the environment. Each payload identifies the thread and environment; current state is not replayed when a thread starts.
- Stop forwarding connection events when an environment selection is removed or replaced.

## Testing

- Cover connection, disconnection, reconnection, shared-environment notifications, and replacement of a selected environment.

GitOrigin-RevId: 5dd767372363c4a2a8319fc16164be117d5bd20c
2026-07-15 05:21:13 +00:00
TAFOYA-OAI 32cd5d4eab Defer Noise environment connections until registration (#33166)
## What changed

- Replace pending WebSocket URL registration with deferred Noise environment
  registration that gates connection attempts on an explicit readiness signal.
- Reuse the Noise rendezvous transport after readiness so reconnects request a
  fresh connection bundle.
- Preserve terminal errors for failed or dropped registrations and keep late
  completion isolated from replacement environments.

## Testing

- Add coverage for readiness gating, registration failure and replacement,
  eager Noise connections, and reconnection through a fresh rendezvous bundle.

GitOrigin-RevId: 83e23fa03a02e3b2bdf1a83fe26d7ac461f55cf3
2026-07-14 21:07:27 +00:00
Adam Perry @ OpenAI 75470c3e2f Add exec-server environment status checks (#32899)
## What changed

- Add the initialized `environment/status` RPC, which reports `ready` when the exec server can handle requests.
- Expose environment IDs and `ready`, `pending`, or `disconnected` status through `EnvironmentManager` and `Environment`.
- Keep status checks non-mutating: they do not start or recover lazy remote environments, and probe only an existing connection.

## Testing

- Cover the status RPC over WebSocket and the in-process request processor.
- Verify that checking an unstarted stdio environment leaves it pending and that failed connections report as disconnected.

GitOrigin-RevId: 22febeb6a3457849292128a8991c6400c22b3fd8
2026-07-13 23:43:15 +00:00
pakrym-oai 4472698728 Support pending remote environment registration (#32231)
## Why

Remote environment provisioning can finish after a thread starts, before an
exec-server WebSocket URL is available.

## What changed

- Add `EnvironmentManager::register_pending_environment` and a one-shot
  `PendingEnvironmentRegistration` handle that resolves to either a validated
  WebSocket URL or a terminal provisioning error.
- Let lazy remote exec-server clients wait for that result, while preserving
  reconnection behavior after a successful registration.
- Keep replacement registrations isolated so completing an older handle does
  not resolve the current environment with the same ID.

## Testing

Add coverage for successful connection and reconnection, provisioning and
dropped-registration failures, invalid URLs, replacement isolation, and the
deferred-executor startup flow.

GitOrigin-RevId: 5c05be2b72291b77a1f71176d7075b1ad63332a5
2026-07-10 18:08:35 +00:00
jif 54c44b9ed4 Propagate tracing subscribers to exec start tasks (#32135)
## Why

The process-start background task can move to a Tokio worker thread, where the
caller's thread-local tracing subscriber is otherwise unavailable. This can
break propagation of the caller's trace context to the exec-server request.

## What changed

- Attach both the current span and current tracing subscriber to spawned
  process-start tasks.
- Run exec-server Bazel unit tests serially because their tracing setup uses
  process-global state.

## Testing

Run the trace-context regression test on a multi-thread Tokio runtime so it
exercises propagation across the background task.

GitOrigin-RevId: a586eefd6983916d670fc5a90f0decd0468d273b
2026-07-10 11:00:12 +00:00
jif 4ba2815014 Bound streamed exec-server HTTP response bodies (#32112)
## Why

Frame-count backpressure does not bound the amount of executor-controlled data
retained in streamed HTTP response queues, and a single body delta could exceed
the intended wire size.

## What changed

- Limit each decoded `http/request/bodyDelta` payload to 1 MiB, split locally
  produced response chunks at that boundary, and reject oversized incoming
  deltas.
- Apply a shared 16 MiB byte budget across queued HTTP response streams. Release
  capacity as deltas are consumed and fail a stream when the budget is
  exhausted.

## Testing

Added coverage for rejecting an oversized delta and for failing a stream after
its queued deltas exhaust the shared byte budget.

GitOrigin-RevId: be9205fef44b0ec96b84b31a8e059b1cb9cd3f3b
2026-07-10 09:56:03 +00:00
jif 13ba8058f2 Resolve selected capability roots without starting executors (#31581)
## Why

A thread can select skill roots that live in an executor environment.
`skills/list` needs a passive snapshot of the roots that are usable now:
it must not start an executor, wait for recovery, or reconnect a failed
environment.

The initial implementation checked the immutable first startup result.
After a successful connection later entered recovery or failed, that
result still looked successful. A read-only catalog request could then
wait for recovery or trigger a new connection while reading the
filesystem.

## What

- inspect readiness from the current exec-server connection state
- return roots only while their environment can serve a request
immediately
- omit environments that have not started, are connecting, or are
recovering
- return warnings for missing environments and terminal connection
failures
- add a fail-fast filesystem view that never starts, waits for, or
reconnects an environment
- expose the passive selected-root snapshot through `CodexThread`

## Behavior

- Local and currently connected environments are ready.
- Starting and recovering environments are omitted without a warning so
callers can retry later.
- Missing and terminally failed environments are omitted with a warning.
- A disconnect between readiness inspection and filesystem access fails
promptly instead of crossing into the normal recovery path.
- Normal model-turn and execution paths keep their existing reconnect
behavior.

## Design

The recovery policy is private to the exec-server client. Callers choose
the explicit fail-fast filesystem method; the existing client and
filesystem APIs remain reconnecting. This keeps the passive contract at
the transport boundary instead of plumbing timeout or retry flags
through the skills stack.

## Coverage

- a lazy stdio environment stays unstarted during passive inspection
- missing and terminally failed environments surface warnings
- a real websocket disconnect proves current readiness drops, a
previously acquired fail-fast filesystem handle returns promptly, and
readiness returns after recovery

## Scope

This PR only provides passive readiness and fail-fast filesystem
primitives. It does not add app-server API fields or notifications.

## Stack

- #31582 uses these primitives for experimental thread-scoped
`skills/list`.
- #30228 adds targeted invalidation notifications.
2026-07-09 11:17:05 +01:00
jif e398a99edf Bound exec-server process event reordering (#31576)
## Why

A malicious exec-server can send out-of-order process notifications
faster than missing events arrive. The client retained every future
event in a per-session reorder map, so many tiny events or a few large
output chunks could grow orchestrator memory without bound.

## What changed

- cap each session's pending process-event reorder state at 256 events
and 1 MiB
- reject individual process events larger than 1 MiB
- release byte accounting as events publish or a session fails
- fail and detach only the affected process session when a limit is
exceeded
- let the next expected event drain a full buffer, so the limits apply
to retained future state
- apply the same bounded insertion path during reconnect recovery while
safely handling dense tail output, missing exit events, newer live
notifications, and conflicting `Closed` sequences

Sequence distance is intentionally not capped because it does not affect
allocation; the event-count and byte limits are the resource-exhaustion
boundary.

## Tests

Focused exec-server tests cover oversized output, full count/byte
buffers, gap-closing delivery, dense recovery interleaved with newer
notifications, missing exit reconstruction, and conflicting terminal
sequences.

## Scope

This is limited to per-session process-event reorder state. It does not
introduce a scheduler, quota framework, or new public configuration.
2026-07-09 09:56:14 +01:00
jif a14b4c2d7f Bound exec-server pending RPCs (#31578)
## Why

An untrusted exec-server can stop reading requests, never answer them,
or send guessed responses before queued requests are written. Without
client-side admission, the orchestrator can retain unbounded RPC call
futures and request payloads.

We need a hard bound without adding a blanket timeout, because
individual operations already own their timeout and cleanup semantics.

## What changed

- hold one of 64 shared admission permits for the full lifetime of each
regular RPC call, so an early response cannot free capacity while the
call remains queued
- allow `process/terminate` and `fs/close` to use one additional cleanup
permit
- close the transport and fail pending calls if the cleanup permit is
also stuck, allowing teardown or recovery to release remote resources
- leave the wire format and existing timeout behavior unchanged

## Tests

- `rpc_client_call_has_no_implicit_deadline` verifies that ordinary
calls remain untimed
- `rpc_client_bounds_in_flight_calls_and_preserves_cleanup` covers the
regular-call cap, guessed early responses, cleanup admission, and the
cleanup circuit breaker
2026-07-08 19:46:18 +01:00
Adam Perry @ OpenAI a86d525e4d core: trace executor skill discovery (#30318)
## Why

Make it easier to measure the performance of different parts of skill
loading.

## What

- Add spans for step-context capture, world-state construction, executor
catalog snapshot/root loading, and environment skill loading.
- Record the discovered environment skill count.
- Trace outbound exec-server requests with client kind and RPC method
fields.
- Update trace propagation tests to assert that requests keep the parent
trace id while creating their own child span.
2026-07-06 18:44:08 +00:00