Commit Graph
89 Commits
Author SHA1 Message Date
Anthony Tafoya cf0ed4c600 Acquire and cache opt-in executor capability inventories via V2 (#53736)
## What changed

- Add the disabled-by-default `executor_capability_discovery_v2` feature to acquire `capabilities/discoverV2` inventories from supporting remote executors using each selected environment's working directory and sandbox.
- Share snapshots with MCP and world-state contributors and skill providers without activating discovered plugins or publishing discovered skills to the model.
- Cache inventories by environment instance, executor session, working directory, and sandbox. Coalesce concurrent requests, refresh the cache at turn start, retry transient failures, and refresh MCP projections after recovery.
- Resolve the system skills root in the executor and pass it explicitly to capability location discovery.

## Testing

Add coverage for feature gating, unsupported executors, inert inventories, cache reuse and eviction, turn refreshes, failure recovery, executor replacement, and system skill locations.

GitOrigin-RevId: 1bdca28fa7f17622b27e2713ab3dcc0c13426889
2026-10-13 18:15:50 +00:00
Anthony Tafoya 703d253ed6 Wire V2 capability discovery through exec-server RPC (#53723)
## What changed

- Register `capabilities/discoverV2` for initialized sessions and expose it through `ExecServerClient` and remote `Environment` APIs, checking executor support before calling it.
- Reuse startup discovery locations and retry failed prewarming on demand. Replay discovery once after executor recovery while preserving the request parameters.
- Track whether all required filesystem watches registered successfully and reload locations and metadata on each request when registration fails.

## Testing

Add coverage for RPC metadata refresh, omission of skill instruction bodies, sandbox isolation, startup scan recovery, disconnect replay, and successful or failed watch registration.

GitOrigin-RevId: 726e8f7e8b4d835b877f31d1d8a9f8eea1ad9c8b
2026-10-13 17:22:34 +00:00
mattj-openai 54685110a7 Support request IDs in exec-server WebSocket handshakes (#52234)
## What changed

- Add optional `websocketRequestId` to `environment/add` and send it as `X-Request-ID` on direct WebSocket upgrades, including reconnects. It requires a secure transport or a loopback destination.
- Echo a supplied `X-Request-ID` in the exec-server WebSocket upgrade response.

## Testing

Add coverage for `websocketRequestId` serialization and for handshake header forwarding and echoing, both with and without a request ID.

GitOrigin-RevId: 235303ff7e1150405b7e1158d6024223144e0cc3
2026-10-08 19:13:52 +00:00
zm-oai ed59a6c1cd Preserve the CLI MXC preference in executor config reads (#51525)
## Why

Executor config reads omitted the startup `features.prefer_mxc` override, so clients could not see the CLI sandbox preference.

## What changed

- Include the explicit CLI `features.prefer_mxc` value in `environmentConfig/read` as a `session-flags` layer, above project config and below legacy managed file and MDM config.
- Retain only this preference from startup overrides.
- Expose `windows_mxc_allowed_by_config` to share MXC eligibility checks after callers resolve the active permission profile.

## Testing

Add CLI and remote executor coverage for preference propagation and filtering, plus a unit test verifying that both legacy managed config sources retain precedence.

GitOrigin-RevId: 5ac6f5257f1e807c2f197dcf5dcb2385cedca0ea
2026-10-07 00:29:04 +00:00
Adam Perry @ OpenAI c39bfa4c8f Enable writable file streaming in exec-server (#50177)
## What changed

Advertise `fileWriteStreaming` and support `fs/open` with `mode: "replace"` to create or truncate a file for writing. Implement `fs/writeBlock` with explicit offsets and nonempty chunks of up to 1 MiB, rejecting ranges beyond the signed 64-bit file offset limit.

Apply write sandbox permissions when opening writable handles, including when full-disk reads are allowed. Keep `read` as the default open mode.

## Testing

Add integration coverage for positional writes, chunk and offset bounds, replacement truncation, read/write handle permissions, and sandbox write restrictions. Verify that duplicate handle IDs and exhausted handle capacity reject replacement opens without creating or truncating files.

GitOrigin-RevId: 696ee6437dd5f5b69d5a7667dddea0ff44472094
2026-10-02 04:25:52 +00:00
Adam Perry @ OpenAI d25c114d49 Bound in-flight file opens in exec-server (#50162)
## Why

The per-connection limit of 128 file read handles was checked only after opening the file, leaving pending opens outside the limit.

## What changed

Reserve a semaphore permit before polling the file-open future, counting pending opens and registered handles against the same limit. Release capacity when an open fails or is cancelled, or when a handle closes, even if a read still retains the file.

Use synchronous handle-table locks that never cross an await, and recheck duplicate IDs after opening to preserve the winning handle during concurrent opens.

## Testing

Add tests for stalled-open progress, pending-open admission limits, permit cleanup after failure or cancellation, capacity reuse while a file remains in use, and concurrent duplicate IDs.

GitOrigin-RevId: caabf38c74dda5951f2fc75234a7149daaa01263
2026-10-02 03:31:52 +00:00
Adam Perry @ OpenAI c51f5bfb82 Handle unsupported fs/writeBlock requests in exec-server (#49811)
## What changed

Register `fs/writeBlock` and require filesystem initialization before handling it. Validate the file handle ID, then return an invalid-request error stating that writable file streams are unsupported.

GitOrigin-RevId: 13440e497a88ca6505ee60d95bbb771d3cae06b8
2026-10-01 01:47:35 +00:00
Adam Perry @ OpenAI 875bf9209b Define exec-server protocol types for streamed file writes (#49778)
## What changed

- Add `read` and `replace` modes to `fs/open`, defaulting omitted modes to `read` for existing callers.
- Define `fs/writeBlock` request and response types for offset-based writes, with a maximum decoded chunk size of 1 MiB.
- Add the `fileWriteStreaming` capability, defaulting to `false`. Keep writable streams unsupported in the server and explicitly reject replacement opens.

## Testing

Add a protocol test verifying that legacy `fs/open` requests without a mode deserialize as read-only opens.

GitOrigin-RevId: 4d3997717e821cd1f62df12a03c8eec36332a66d
2026-10-01 00:43:55 +00:00
Adam Perry @ OpenAI 3b16b5a5b0 Rename exec-server file handle management identifiers (#49702)
## What changed

Rename `file_read` to `file_handle` and `FileReadHandleManager` to `FileHandleManager`. Update the filesystem handler field, handle ID validation helper, and related constants to use file handle terminology. File operations, limits, and error messages remain unchanged.

GitOrigin-RevId: 25ad81a917386da23fd42ecff79bd47a482ce430
2026-09-30 18:37:18 +00:00
jif 65c3f40bef Model exec-server session attachment state as an enum (#49286)
Replace the three optional fields in `AttachmentState` with `Attached` and
`Detached` variants, making inconsistent field combinations unrepresentable.
Keep the connection ID in both variants and the expiration deadline only in
`Detached`, and update attachment and expiration checks to match on the state.

GitOrigin-RevId: 6a5f05f0ce2f064acc8f5cacd00bc374c4f47986
2026-09-29 15:41:22 +00:00
open-matt b8d5e3f12e Allow exec-server to proxy permitted private IPs upstream (#48568)
## Why

Private IP destinations always bypassed inherited upstream proxies, preventing their use for private networks reachable through an upstream VPN proxy.

## What changed

- Add `codex exec-server --proxy-private-ips-via-upstream`, also configurable with `CODEX_EXEC_SERVER_PROXY_PRIVATE_IPS_VIA_UPSTREAM=true`. The setting defaults to disabled.
- Allow permitted RFC 1918, carrier-grade NAT, and IPv6 unique-local destinations to use an applicable upstream proxy. Loopback and link-local destinations retain direct routing, and destination access policy still applies.
- Keep connections direct when no valid upstream proxy applies or `allow_upstream_proxy=false`. Errors after selecting an upstream proxy do not trigger a direct retry.
- Rename `ExecServerRuntimePaths` to `ExecServerRuntimeOptions` and carry the routing setting from executor startup into the managed network proxy.

## Testing

Add routing coverage for private address ranges, special-use addresses, and public targets with the option enabled and disabled. Verify that HTTP and CONNECT requests still enforce destination allowlists and denylists, and update the CLI help snapshot.

GitOrigin-RevId: b7c9cc7da0e0f545694a6521b74c9b36b7b92769
2026-09-26 23:17:41 +00:00
Anthony Tafoya ab7439231c Add executor capability discovery V2 infrastructure (#47683)
## What changed

- Define `capabilities/discoverV2` request and inventory types and advertise `capabilityDiscoveryV2` in executor metadata.
- Prewarm installed plugin and global skill locations at server startup, with nonfatal scan failures.
- Add sandbox-scoped discovery caching with file watcher invalidation and request-time reloads when watching is unavailable.
- Batch metadata reads through the filesystem sandbox helper and bound discovery responses to 4 MiB and 2,048 capabilities.
- Share plugin identity and installed-version selection in `codex-core-plugin-common`, and fall back to `.app.json` when a plugin does not declare an apps configuration.

The V2 discovery manager is not yet wired into RPC request dispatch.

## Testing

Add coverage for inventory metadata, sandbox cache isolation, watcher invalidation, startup scan failures, response limits, and compatibility with older executor metadata.

GitOrigin-RevId: a223a5bc15fdf251023c0c71f468fd92c3b2d847
2026-09-23 22:20:34 +00:00
Adam Perry @ OpenAI 6fa3bcaae8 Add exec-server RPC timing and process startup tracing (#47680)
## Why

Detached work can keep request spans open beyond response enqueueing. Record explicit phase timings to distinguish request handling from the full span lifetime.

## What changed

- Record server dispatch and response enqueue offsets from request receipt, plus a response enqueue event for HTTP responses that bypass the dispatcher.
- Emit client request enqueue and response receipt events, including success or server-error outcomes and reader-to-caller delay. Carry completion timestamps to the awaiting task so receipt events retain its tracing context.
- Add spans for process startup, sandbox preparation, shell snapshot preparation and capture, and process spawning. Record process identifiers only when nonempty and at most 64 bytes long.

GitOrigin-RevId: 237db898b3284fe8a6a36cb0055b3c7991d8e8a4
2026-09-23 22:10:28 +00:00
Ruslan Nigmatullin 2210190435 Add opt-in WebSocket authentication to exec-server (#47601)
## What changed

Expose the shared `--ws-auth` options on `codex exec-server` for direct WebSocket listeners. Support capability tokens configured by token file or SHA-256 digest, and signed JWT bearer tokens. Validate `Authorization: Bearer TOKEN` before each WebSocket upgrade, rejecting missing or invalid credentials with HTTP 401 when authentication is enabled.

Reject listener authentication with stdio, `--remote`, or `forward`. Document the options and connection-time authentication behavior.

## Testing

Add CLI integration tests for all three credential configurations, unauthorized upgrade rejection, authenticated RPC initialization and reconnects, invalid configuration, and incompatible transports.

GitOrigin-RevId: 941fbd3d43e732c910d29b6f8137077b7c332835
2026-09-23 17:56:46 +00:00
acrognale-oai 8f8ace78cb Enforce application network policy for AWS auth and telemetry (#47408)
## Why

AWS credential discovery, analytics, and telemetry used clients that did not share the application's network policy. These requests need to honor destination restrictions and permission revocation.

## What changed

- Route AWS credential and region HTTP requests through the shared HTTP client, and check the signing destination before loading credentials. Skip discovery for static access keys with an explicit region.
- Apply account-scoped policy to Bedrock authentication. Require unrestricted policy for credential exporters and reauthentication commands, and cancel active work when permission is revoked. Document that AWS profile `credential_process` network traffic remains outside the application's HTTP policy.
- Send analytics through the authenticated account's HTTP client factory.
- Guard OTLP log, trace, and metric exports with revocable permits, disable them under destination restrictions, and make managed HTTP exports cancellable. Suppress global Statsig settings while managed policy is active.

## Testing

Add coverage for metadata credential request cancellation, allowed and denied SigV4 destinations, AWS credential precedence and endpoint configuration, and blocked credential exporter recovery. Update telemetry tests for account transitions and managed-policy Statsig suppression.

GitOrigin-RevId: d8a2018bfbbe971ec430699b0d3f86a7a9e1e072
2026-09-23 00:47:34 +00:00
Adam Perry @ OpenAI a4ee536f01 Route filesystem reads and writes by their own sandbox permissions (#46122)
## Why

Filesystem reads previously required a sandbox whenever writes were restricted, even with full-disk read permission. This made permitted reads depend on sandbox availability.

## What changed

- Select sandboxing independently for reads and writes, allowing full-disk reads directly while keeping restricted operations sandboxed.
- Use the executor's path convention when evaluating full-disk access, including `:slash_tmp` denials.
- Apply read-specific checks to capability discovery and skill resource reads, allowing unrestricted reads on executors without sandboxed discovery support.

## Testing

Add regression tests for direct read APIs, restricted writes and reads, executor-specific permission rules, and capability discovery without sandbox support. Update Windows tests to verify sandbox enforcement through writes.

GitOrigin-RevId: a03844bd1f0ea583bb54326683cddbfdd050119f
2026-09-17 05:09:03 +00:00
Adam Perry @ OpenAI 841b5490b2 Preserve filesystem sandbox policy context when the cwd disappears (#46112)
## Why

Removing the selected working directory can prevent filesystem sandbox helpers from launching, even when the requested absolute paths remain accessible. Permission rules must stay anchored to the selected directory while those operations continue.

## What changed

- Require a policy `cwd` in `FileSystemSandboxContext` and launch filesystem helpers from the filesystem root while preserving the policy directory and workspace roots.
- Carry explicit `policyContext` in filesystem RPCs, preserving legacy wire fields and resolving omitted directories from older clients at executor ingress.
- Keep permission paths as executor file URIs and validate host compatibility where they are enforced.
- Bind Windows relative denial globs to the policy directory before changing the helper's launch directory, preserving home-relative patterns.

## Testing

Add regression coverage for `apply_patch` after working-directory removal, legacy RPC directory fallbacks, cross-platform permission URI transport, and Windows relative read denials. The patch regression verifies that an allowed file is updated while an explicitly denied file remains unreadable and unchanged.

GitOrigin-RevId: b0f4db722b27cb72ec129fc297c85732afac11f7
2026-09-17 04:37:54 +00:00
iceweasel-oai d4e11a9b97 Separate executor sandbox selection from Windows sandbox levels (#45730)
## Why

MXC is a sandbox implementation, not a restricted-token sandbox level. Executor requests need to represent that choice separately from `WindowsSandboxLevel`.

## What changed

- Introduce `WindowsSandboxSelection` for executor sandbox contexts and remove `Mxc` from `WindowsSandboxLevel`.
- Preserve the `windowsSandboxLevel` wire field and its serialized values for compatibility.
- Share sandbox selection between executor process launches and filesystem helpers, and use the new selection in capability discovery and skill reads.
- Disable Windows sandbox selection for executor paths that do not use Windows path conventions.

## Testing

Extend coverage for MXC wire serialization, Windows skill-read sandbox checks, and capability discovery with distinct permissions. Exercise remote filesystem write restrictions with both restricted-token and MXC sandboxes, including rejection when native MXC is unavailable.

GitOrigin-RevId: 266211377bcb138a0dc75861e9ff2225fa37a53d
2026-09-15 16:40:06 +00:00
bkotsopoulos 4f2449b4b2 Measure total exec-server request duration including queueing (#44207)
## Why

The existing request duration metric starts at dispatch, leaving out time spent waiting before dispatch.

## What changed

Add `exec_server_request_total_duration_seconds` to measure time from decoded receipt until response enqueue or disconnection, including queueing. Record it alongside the existing duration metric with the same `method` and `result` labels for success, error, and disconnection paths. Preserve existing dispatch and queue timing semantics.

## Testing

Add coverage for successful requests, route errors, unknown methods, response delivery failures, and disconnection during execution. Extend admission-wait coverage to verify total timing includes queueing and route setup without double-counting, and assert each completion and duration is recorded once.

GitOrigin-RevId: ed0f719751b821b8f9f9556914b0f67dc6c5ad8f
2026-09-09 17:03:57 +00:00
felixxia-oai dbe2f6d528 Expose a stable executor build identity in environment metadata (#43513)
## Why

Executor compatibility checks need a build identity that distinguishes commits
and compiler targets independently of the package release version.

## What changed

- Add optional `providerId` to exec-server environment metadata, cached at startup
  and returned by initialization and `environment/info`.
- Derive the ID as SHA-256 of `git:<lowercase commit>:<target>`. It identifies a
  standard build configuration, not exact executable bytes, and is omitted when
  the commit stamp or target is unavailable or invalid.
- Embed the compiler target in `BuildInfo`, preserve compatibility with historical
  metadata without a target, and stamp Cargo release builds with `STABLE_GIT_COMMIT`.
- Group Bazel build-script argument files under one `--arg-file` flag to reduce
  Windows command-line length while preserving file order.

## Testing

Add deterministic build-ID vectors across targets, commit normalization and invalid
input coverage, and historical metadata compatibility tests. Extend exec-server
coverage for metadata caching and ensure runtime environment overrides cannot
replace the executor's build identity.

GitOrigin-RevId: 125a18c23de7ad006571940ba305836376c983f4
2026-09-07 16:55:06 +00:00
viyatb-oai f53c91be2c Add attributed exec process lifecycle telemetry (#42373)
## Why

Exec-server process and network-policy events need stable attribution to the
launching tool call and executor without recording process payloads.

## What changed

- Add optional `ExecMetadata` to `ExecParams` and propagate the thread and tool
  call IDs from unified exec.
- Emit bounded OpenTelemetry events for process start, spawn failure, sandbox
  denial, and exit. Correlate them with the launch trace and registry-issued
  executor identity while excluding arguments, paths, environment values,
  output, and error text.
- Preserve launch attribution across long-running processes and reconnects, and
  attach the same metadata to network-policy audit events.
- Keep the protocol backward compatible when metadata is omitted, and prevent
  invalid trace headers from inheriting an unrelated active span.

## Testing

Add coverage for metadata serialization and propagation, lifecycle event
fields, trace relationships, reconnect behavior, spawn failures, sandbox
denials, and network-policy attribution.

GitOrigin-RevId: 7aa480a7289c73cb95e2c124c35500bb6f0d5084
2026-09-02 21:58:58 +00:00
felixxia-oai 8d32abcd01 Report the exec-server release version in environment info (#42270)
## Why

Clients need the executor's release version to make version-based compatibility decisions.

## What changed

- Add `executorVersion` to `EnvironmentInfo` and return it from both `initialize` and `environment/info`.
- Resolve the package release version at process startup so the advertised value remains stable for the server's lifetime.
- Use `0.0.0` when the version is unknown or omitted by a legacy executor.

## Testing

- Cover packaged and source-build initialization, startup caching, protocol serialization, and client-side metadata caching.

GitOrigin-RevId: a0c7f0b701a8e7e50345cd5fb1b5c5d2d38ff3b1
2026-09-02 12:07:32 +00:00
jif eb15245d82 Batch sandboxed capability root discovery (#40443)
## What changed

- Run capability discovery for roots with the same sandbox context in one sandbox helper invocation.
- Keep roots with different sandbox permissions isolated, and fall back to per-root discovery if the batched invocation fails.

## Testing

- Add coverage for single-launch batching, fallback after a sandbox launch failure, and roots with different permission profiles.

GitOrigin-RevId: a6e185a8edfd028fc3639b7c6c3cc7da342b6fe3
2026-08-24 17:39:16 +00:00
cgst-oai 2df6705423 Cache executor metadata from initialization (#40343)
## What changed

- Include `environmentInfo` in the exec-server `initialize` response and seed the Rust client's metadata cache from it.
- Fall back to one lazy `environment/info` request when connecting to older servers that omit initialization metadata, then share the cached result across client clones and session recovery.
- Keep app-server `environment/info` requests uncached so they continue to probe the executor.

## Testing

- Cover initialization metadata, the legacy-server fallback, shared caching, accepted WebSocket environments, and repeated uncached app-server probes.

GitOrigin-RevId: 380132d9db183998d73e8c6b0954e4d70c7e8406
2026-08-24 06:43:07 +00:00
jif d0cc662b8c Cache shell snapshots in the exec server (#39756)
## What changed

- Add the `shellSnapshotV2` executor capability and an optional shell snapshot request to `ExecParams`.
- Capture and restore Unix shell state and profile exports from an in-memory, attachment-scoped cache for `bash`, `zsh`, and `sh`.
- Apply environment policies, runtime `PATH` entries, sandbox context, and live managed-proxy settings when preparing restored commands.
- Bound snapshot size, capture time, scope length, and cache capacity, and fall back to the original command when capture fails.

## Testing

- Cover local, remote, TTY, sandboxed, and supported-shell execution, plus environment filtering, proxy handling, in-memory reuse, and capture failure fallback.

GitOrigin-RevId: 624f747972c249c88c6f10f42cf0af97b75b5541
2026-08-20 17:39:06 +00:00
rhan-oai 8c828b18d6 Remove private executor directory creation (#39736)
## What changed

- Create remote plugin metrics directories through the standard executor filesystem API.
- Remove the `private` directory-creation protocol option and its platform-specific handling.
- Update the executor temporary-directory documentation to describe child-visible sidecars without an owner-private guarantee.

GitOrigin-RevId: 9a8532403a3ad2bf998281735be0b668893918c9
2026-08-20 16:22:45 +00:00
pakrym-oai e3e5ad2847 Harden unsandboxed patch filesystem access (#39659)
## Why

An `apply_patch` path can be replaced with a symlink after verification, allowing an unsandboxed patch operation to reach a different file than the one that was approved.

## What changed

- Add `follow_symlinks` options to executor filesystem reads, writes, metadata lookups, directory creation, and removal, including the corresponding `followSymlinks` protocol fields.
- Implement no-follow filesystem operations on Unix and Windows that reject links in any path component and restrict file access to regular files.
- Run `apply_patch` with symlink traversal disabled when an otherwise-required sandbox is bypassed, while retaining the existing follow-symlink default for standalone callers.

## Testing

- Cover leaf and ancestor symlinks across patch add, update, delete, and move operations, including a path swap after verification.
- Exercise local and remote no-follow filesystem behavior, concurrent directory creation, special-file rejection, and Windows reparse points.

GitOrigin-RevId: 43fd479084891493ce13564fbd894b98f329c6dd
2026-08-20 08:10:08 +00:00
Ben Romano fd34ad7297 Trace exec-server requests from receipt through completion (#39098)
## What changed

- Start inbound exec-server request spans when messages enter the connection queue and carry them through dispatch and response handling.
- Record request outcomes for client-handled network policy callbacks, including errors and disconnections.
- Add the `exec_server_request_queue_duration_seconds` histogram, labeled by bounded route name, while excluding synchronous route setup time.

## Testing

- Cover span lifetime and trace-parent propagation across server and client queues.
- Verify queue-duration telemetry and outcome recording for completed, rejected, and cancelled requests.

GitOrigin-RevId: ed67fe5305048bdf283a26ec874337d549e3324f
2026-08-17 22:53:55 +00:00
Kyle Brown 9579479d28 Collect plugin metrics from remote executors (#38283)
## What changed

- Resolve manifest-declared metric operations against the executor filesystem for remote plugin commands.
- Create the measurement sidecar in an executor-native, owner-private temporary directory, stream its bounded output back for validation, and clean up the directory afterward.
- Extend the exec-server protocol with the executor temporary directory and atomic private-directory creation.

## Testing

- Cover remote unified-exec measurements for foreground and background commands.
- Verify private directories use owner-only permissions on Unix and fail closed on unsupported platforms.

GitOrigin-RevId: dbbd0a84717b91237fc5728e510e18994eb46dd4
2026-08-13 00:39:04 +00:00
sayan-oai 95c7265e84 Add executor-local config reads to the exec server (#37408)
## What changed

- Add the `environmentConfig/read` RPC for selecting literal TOML paths from executor-local config and requirements layers.
- Return layer precedence, cloud insertion points, source and base-directory metadata, and executor home and hostname context without normalizing path-bearing values.
- Expose the operation through both remote clients and local `Environment` instances, with invalid selectors reported as invalid parameters.

## Testing

- Cover projected remote config reads and rejection of empty selectors.

GitOrigin-RevId: fee15bf833de6bcb2058d405ed5b1d5d928d7218
2026-08-07 08:32:30 +00:00
Adam Perry @ OpenAI eeae88d8a6 Add opt-in concurrent exec-server request dispatch (#36987)
## Why

Sequential dispatch lets a long-running request block unrelated health checks
and cleanup on the same connection.

## What changed

- Add `--concurrent-requests <COUNT>` for local and remote exec-server
  connections, while retaining sequential dispatch when the option is omitted
  or set to `1`.
- Preserve handshake ordering before enabling concurrent dispatch.
- Reserve separate capacity for status, signal, terminate, and close requests so
  they remain responsive when ordinary request capacity is saturated.
- Drain queued client responses during disconnect and cancel outstanding
  request tasks during connection shutdown.

## Testing

- Cover CLI parsing and concurrency-limit validation.
- Verify default sequential behavior, pipelined handshake ordering, concurrent
  request progress, control-request responsiveness, and disconnect handling.

GitOrigin-RevId: 48e4b092e318204ed635543f01f9ee0e7df095fc
2026-08-04 22:28:15 +00:00
Adam Perry @ OpenAI ee0247f95a Extract exec-server request dispatching (#36440)
## What changed

- Move JSON-RPC request, notification, response, error, and malformed-message handling into a dedicated `RequestDispatcher`.
- Keep the connection loop responsible for receiving events and closing the connection when dispatch reports a terminal condition.

## Testing

- Add an integration test confirming that ordinary requests are processed serially by default, including when a blocking `process/read` queues later requests.

GitOrigin-RevId: 29d1358d4524edd492ff3855b29f23c42c8b3390
2026-08-01 06:02:48 +00:00
Celia Chen 09241ae4db Decouple exec-server HTTP from reqwest types (#35059)
## What changed

- Rename `ReqwestHttpClient` to `RouteAwareHttpClient` to reflect that delegated HTTP uses Codex's shared route-aware transport.
- Use `codex_http_client` response and error types plus transport-neutral `http` and `url` types, removing the exec server's direct `reqwest` dependency.

## Testing

- Cover fragment stripping and Unicode hostname normalization for delegated HTTP requests.

GitOrigin-RevId: 8b0fc60a76004feb57198bfb4afb1371c9ceb1bd
2026-07-24 00:53:05 +00:00
Celia Chen 94ebae725e Route exec-server WebSockets through configured proxies (#35056)
## Why

Remote environment connections need to honor Codex's effective outbound proxy policy, including when a rendezvous connection reconnects.

## What changed

- Pass the configured `HttpClientFactory` into remote environment transports and use `WebSocketConnector` for exec-server and rendezvous WebSockets.
- Resolve proxy routes asynchronously so these connections can use the configured system proxy.
- Add connector options that preserve Tungstenite's default TLS behavior and enable `TCP_NODELAY` for latency-sensitive rendezvous traffic.

## Testing

- Verify prepared remote environments connect through a configured system proxy.
- Verify initial and reconnected encrypted relay peers use the system proxy.
- Cover default TLS selection and opt-in `TCP_NODELAY` behavior in the WebSocket client.

GitOrigin-RevId: 8a8da2116e37cb3a891269d0c0b037986fecdd3c
2026-07-24 00:16:42 +00:00
Celia Chen 1ee8f49175 Route exec-server HTTP through configured proxy policy (#35023)
## Why

Delegated HTTP requests need to honor the same outbound proxy policy as the
Codex process that starts the exec server.

## What changed

- Pass the configured `HttpClientFactory` through local and remote exec-server
  startup and use route-aware client pools for delegated HTTP and local MCP
  requests.
- Preserve per-request timeouts and follow-or-stop redirect behavior while
  keeping request URLs and sensitive response headers out of diagnostics.

## Testing

- Cover configured system-proxy routing across the exec-server transport.
- Cover both redirect policies and verify that success and failure logs do not
  expose request or response secrets.

GitOrigin-RevId: 4af6aec1d265c4db62dfcb6e1fb076fb31736137
2026-07-23 22:39:28 +00:00
viyatb-oai 32f4687b8c Enable exec-server network policy callbacks (#34770)
## What changed

- Let `exec-server` issue JSON-RPC requests to its client and correlate responses, with bounded concurrency, timeouts, and connection cleanup.
- When `request_policy_decisions` is enabled, forward proxy policy requests with the process ID and network destination, then apply the client's allow, deny, or ask decision.
- Fail closed for invalid inputs, unavailable or malformed responses, timeouts, process shutdown, and disconnects. Keep callbacks active while inherited process streams keep the proxy alive.

## Testing

- Cover response correlation, request limits, timeouts, disconnect cleanup, validation boundaries, decision forwarding, and process lifecycle behavior.

GitOrigin-RevId: 60e658102c50036589ca9de844760a8bffaa53b0
2026-07-22 15:22:21 +00:00
viyatb-oai 9f6c29e281 Launch managed network proxies on remote executors (#33906)
## Why

Remote executions need managed-network proxy listeners in the executor so their
loopback proxy addresses are reachable by the launched process.

## What changed

- Add a capability-gated exec-server protocol field for executor-local proxy
  launch configuration, including network policy, audit metadata, and execution
  attribution.
- Start the proxy while preparing a remote process, replace inherited proxy
  environment variables with its local addresses, and derive the sandbox
  context from its listeners.
- Keep the proxy alive until inherited output streams close, then shut it down.
- Reject unsupported remote settings such as MITM and credential injection.

## Testing

- Cover configuration round trips and rejection of unsupported settings.
- Verify executor-local startup, blocked-domain enforcement, protocol
  compatibility, and proxy lifetime through process closure.

GitOrigin-RevId: c984f54e3e600aa9ebcbf8cf4574046e2c199d11
2026-07-17 21:20:14 +00:00
Bryan Ashley 08e30a2e4e Add batched executor capability discovery (#33852)
## Why

Selected capability roots can contribute plugins, MCP servers, connectors, and
skills. Discovering each contribution separately requires repeated access to the
executor filesystem.

## What changed

- Add the `capabilityRoots/discoverV1` exec-server RPC to scan selected roots and
  materialize recognized plugin manifests, configuration files, skill
  instructions, and skill metadata in one bounded request.
- Add the opt-in `executor_capability_discovery` feature, with a thread-scoped
  cache and per-step snapshot shared by MCP and skill discovery.
- Parse MCP, connector, and skill contributions from the materialized snapshot,
  including serving cached skill instructions without another filesystem read.

## Testing

- Cover discovery limits, manifest precedence, root-local failures, cache reuse,
  plugin contributions, and parity with the existing environment skill loader.

GitOrigin-RevId: f98fd2321cafb58c596db02da1f83c09d8eb375d
2026-07-17 15:45:48 +00:00
jif 325cf16194 Bound exec-server JSON-RPC decoding complexity (#33013)
## Why

Compact JSON arrays can expand into millions of heap values during decoding, and duplicate object keys make a message ambiguous.

## What changed

- Limit exec-server JSON-RPC messages to 256K JSON values and reject duplicate object keys.
- Cap `fs/read_directory` results and retained `process/read` output at 50,000 entries or chunks so locally produced responses remain within the decoder budget.

## Testing

Add coverage for all JSON-RPC variants, large scalar payloads, duplicate keys, compact array amplification, and retained process output at the chunk limit.

GitOrigin-RevId: e31d1f25ab0a7e2272015c98174fd2b7cdd669d7
2026-07-14 09:15:27 +00:00
Adam Perry @ OpenAI 75470c3e2f Add exec-server environment status checks (#32899)
## What changed

- Add the initialized `environment/status` RPC, which reports `ready` when the exec server can handle requests.
- Expose environment IDs and `ready`, `pending`, or `disconnected` status through `EnvironmentManager` and `Environment`.
- Keep status checks non-mutating: they do not start or recover lazy remote environments, and probe only an existing connection.

## Testing

- Cover the status RPC over WebSocket and the in-process request processor.
- Verify that checking an unstarted stdio environment leaves it pending and that failed connections report as disconnected.

GitOrigin-RevId: 22febeb6a3457849292128a8991c6400c22b3fd8
2026-07-13 23:43:15 +00:00
richardopenai 2dec46e30a [codex] Record exec-server lifecycle metrics (#27467)
## Summary

- Record bounded connection, request, and process lifecycle metrics.
- Report active gauges from callbacks on every collection, including
delta exports.
- Serialize active-count updates so concurrent starts and finishes
cannot publish stale values.
- Serialize process exit, explicit termination, and shutdown through the
process registry so exactly one completion result wins.
- Keep the implementation small with single-owner RAII guards and one
real OTLP/HTTP integration test using the existing `wiremock`
dependency.

## Root cause

Process exit and session shutdown previously used cloned completion
state. That avoided duplicate emission, but it duplicated lifecycle
ownership and made the ordering harder to reason about. The process
registry mutex already defines the lifecycle ordering, so the final
implementation stores the metric guard and termination flag directly on
the process entry. Whichever path claims the entry first owns the
completion result.

Production metric export uses delta temporality. Event-only synchronous
gauge recordings disappear after the next collection when no count
changes, so active counts now use observable callbacks that report
current state on every collection.

The cleanup also removes the constant `result="accepted"` connection
tag, redundant route and response assertions, a custom HTTP collector,
and fallback initialization machinery that did not add behavior.

## Stack

Review and land this stack in order:

1. #27466 — trace exec-server JSON-RPC requests
2. #27467 — record bounded connection, request, and process lifecycle
metrics **(this PR)**
3. #27470 — observe remote registration and Noise rendezvous lifecycle

## Validation

- `just test -p codex-exec-server --lib` (158 passed)
- `just test -p codex-cli --test exec_server` (3 passed)
- `just test -p codex-otel
observable_gauge_is_collected_on_every_delta_snapshot` (1 passed)
- `CARGO_BUILD_JOBS=1 just fix -p codex-otel -p codex-exec-server`
- `just fmt`
- `git diff --check`
2026-06-25 11:02:11 -07:00
jif b215961a56 Support OAuth for HTTP MCP servers from selected executor plugins (#28529)
## Why

#28522 routes selected-plugin HTTP MCP traffic through the owning
executor, but OAuth bootstrap and refresh still used host-local clients.
Executor-only servers therefore cannot complete discovery or login
through the same network boundary as the MCP connection.

## What changed

- adapt `codex_exec_server::HttpClient` to RMCP 1.8's `OAuthHttpClient`
contract
- let RMCP own discovery, dynamic registration, PKCE, token exchange,
and refresh
- route auth status, persisted-token startup, and app-server login
through the server runtime while preserving the existing local discovery
path
- add optional `threadId` to `mcpServer/oauth/login` and echo it in the
completion notification
- implement RMCP's redirect policy and 1 MiB OAuth response limit over
executor HTTP
- cover selected-thread OAuth discovery and login through an
executor-only route

Depends on #28522.
2026-06-25 10:31:17 +01:00
richardopenai 74dcce594d [codex] Trace exec-server JSON-RPC requests (#27466)
## Why

Exec-server JSON-RPC calls can cross local and remote transports, but
trace context stopped at the RPC boundary. That made client and server
work difficult to correlate when diagnosing latency or failures.

## What changed

- Propagate the current W3C trace context on outbound JSON-RPC requests.
- Parent inbound request spans from received trace context.
- Record the received JSON-RPC method on server spans and keep each span
open through response enqueue.
- Add only the OTEL dependencies required by the exec-server crate.

## Stack

Review and land this stack in order:

1. #27466 — trace exec-server JSON-RPC requests **(this PR)**
2. #27467 — record bounded connection, request, and process lifecycle
metrics
3. #27470 — observe remote registration and Noise rendezvous lifecycle

## Validation

- `just test -p codex-exec-server --lib` (153 passed)
- `just bazel-lock-check`
- `just fix -p codex-exec-server`
2026-06-24 12:50:18 -07:00
jif c14623d04c Add a bounded filesystem walk RPC (#29841)
Stack 1 of 3. Follow-ups: #29842 and #29844.

## What changes

Adds a general bounded `fs/walk` operation to the exec server.

The operation returns file and directory entries plus recoverable
per-path errors. It skips symlinks, preserves the existing filesystem
sandbox routing, and enforces depth, directory, entry, and response-size
limits.

This PR only defines and wires the filesystem operation. It does not
change any callers yet.
2026-06-24 16:05:43 +01:00
Adam Perry @ OpenAI 829f5b6b59 protocol: separate app and exec RPC ownership (#29714)
## Why

The app-server and exec-server expose separate JSON-RPC APIs, but
exec-server currently sources its serialized protocol and envelope types
through app-server-oriented code. Giving each API an explicit owner
makes the crate boundary legible without introducing shared generic
envelopes.

## What changed

- Added `codex-exec-server-protocol` to own exec DTOs, process IDs, and
JSON-RPC envelopes.
- Updated exec-server clients, transports, handlers, and tests to use
the new crate.
- Exposed app-server's existing JSON-RPC types through a public `rpc`
module while retaining root re-exports.
- Preserved existing wire shapes, including exec `PathUri` behavior.

## Stack

This is PR 1 of 6. Next: [PR
#29721](https://github.com/openai/codex/pull/29721), which moves auth
mode below the app wire boundary.

## Validation

- Exec-server protocol and server coverage passed in the focused
protocol test runs.
- App-server protocol schema fixtures passed.
2026-06-23 22:37:31 +00:00
jif e476fc16ce Prepare managed network sandbox context (#29456)
## Why

Managed network configures commands to use local HTTP and SOCKS proxies.
For commands delegated to the exec server, the proxy environment and the
sandbox policy were prepared separately. On macOS, that meant a command
could receive `HTTPS_PROXY=http://127.0.0.1:43123` while Seatbelt still
denied access to port `43123`.

## What changed

`NetworkProxy` now prepares the command environment and sandbox context
together from the same runtime snapshot:

```text
Prepared managed network
├── command environment: HTTPS_PROXY=http://127.0.0.1:43123
└── sandbox context: allow outbound to 127.0.0.1:43123
```

That context travels with remote exec requests. The exec server
preserves the managed proxy and CA environment, and macOS Seatbelt
allows only the prepared loopback proxy ports without enabling broad
network access or local binding.

The protocol field is optional and the existing enforcement flag remains
in place, preserving compatibility with callers that do not send the new
context.
2026-06-23 20:07:09 +01:00
Adam Perry @ OpenAI 11fab432be path-uri: clarify host-native path conversion (#29501)
## Why

Downstream refactors are producing confusing code with this
functionality having a very generic name. Encoding the specific
conversion approach in the method name makes it clearer.

## What

Rename `PathUri::from_path` to `PathUri::from_host_native_path` and
update its Rust call sites.
2026-06-23 00:02:33 +00:00
jif 9c3b10e5d4 Apply sandbox intent inside remote exec servers (#29113)
## Why

PR #29108 lets the orchestrator send sandbox intent with `process/start`
without wrapping the command for its own operating system.

This PR completes that boundary by making the executor interpret and
enforce the intent using its own filesystem paths and sandbox
implementation.

For example, a macOS TUI targeting a Linux devbox sends `/bin/bash -lc
pwd`. The Linux executor turns that into its own `codex-linux-sandbox
... /bin/bash -lc pwd` launch.

## What changes

- Keep `process/start` unchanged when no sandbox intent is present.
- Convert sandbox `PathUri` values into native paths on the executor.
- Bind symbolic `:workspace_roots` permissions to the executor's native
sandbox cwd.
- Select the sandbox implementation on the executor and wrap the
original command immediately before spawning it.
- Reject sandbox-required execution before spawning when the executor
cannot enforce the intent.
- Pass exec-server runtime paths into process creation so Linux can
locate `codex-linux-sandbox`.

The boundary is therefore:

```text
orchestrator                         executor
original argv + sandbox intent  ->  select and enforce local sandbox
```

This PR intentionally treats a denied remote command as an ordinary
command failure. Draft follow-up #29424 carries a semantic
`sandboxDenied` result back to unified exec for the existing approval
and retry flow.

## Platform scope

Linux and macOS use their existing direct-spawn sandbox transforms.

Windows sandboxed remote process launch is intentionally unsupported in
this PR. The current Windows direct-spawn wrapper does not correctly
preserve arbitrary argv, TTY behavior, or pass the full child
environment out of band. The executor rejects the request instead of
running it incorrectly or unsandboxed.

## Known follow-ups

- The transported permission profile can still contain
orchestrator-materialized helper or explicit paths. A `TODO(jif)` marks
where the executor boundary should receive pre-host-materialization
permission intent.
- The sandbox wrapper currently replaces a requested custom inner
`arg0`. A `TODO(jif)` marks where this must be preserved or rejected
explicitly.
- Draft PR #29424 contains the deferred sandbox-denial classification
and approval/retry behavior.

## Rollout assumption

This executor-sandbox stack is unreleased and its client and executor
are expected to move together. This PR does not add mixed-version
negotiation with older exec servers.
2026-06-22 12:45:37 +02:00
jif 1088b30fda Test pipelined scalar exec-server requests (#29325)
## Summary

This adds focused coverage for the simpler same-connection scalar
request path.

The exec-server connection already supports multiple in-flight JSON-RPC
scalar requests on one connection. This test locks in that behavior by
sending two normal requests before reading either response, without
adding a batch frame or any new API surface.

## What changed

- Added a processor-level test that initializes an exec-server
connection.
- Sends two scalar `environment/info` requests back-to-back on the same
connection.
- Verifies both responses come back on the same connection by request
id.

Checked locally with:

- `just test -p codex-exec-server
connection_accepts_pipelined_scalar_requests`
2026-06-21 13:40:51 +02:00
jif bd2968a4db Carry sandbox intent to remote exec servers (#29108)
## What changed

PR #29099 stopped sending the orchestrator's concrete sandbox wrapper to
a remote exec-server. Remote commands now arrive as plain native argv.

This PR adds the next piece: Codex also sends portable sandbox intent
next to that plain argv.

For a remote unified-exec command, the request can now include:

- the canonical permission profile before local workspace-root
materialization
- the sandbox cwd and workspace roots as `PathUri` values
- Windows sandbox settings
- the legacy Landlock setting
- whether managed networking must be enforced

The important part is that symbolic entries such as `:workspace_roots`
stay symbolic while crossing the boundary. The executor can then bind
them to its own workspace-root paths instead of receiving
orchestrator-local absolute paths.

The data travels through `ExecRequest` into `ExecParams`. Older
exec-servers can still deserialize requests because the new fields have
defaults.

## Why

The orchestrator should not decide how another machine implements
sandboxing.

For example:

- a local macOS Codex would normally build a Seatbelt command
- a remote Linux executor needs a Linux sandbox command instead

The orchestrator now sends the plain command plus the policy it intended
to enforce. A later PR can let the exec-server choose and build the
correct sandbox for its own operating system.

## Important detail

This keeps the portable intent separate from the local `SandboxType`.

`SandboxType::None` is ambiguous:

- it can mean the command was explicitly approved to run without a
sandbox
- it can also mean the orchestrator host has no concrete sandbox
implementation available

Those cases are different for remote execution. This PR adds
`sandbox_requested` so an executor can still receive sandbox intent when
the orchestrator cannot build a local wrapper. Explicit unsandboxed
retries still send no sandbox context.

## Behavior today

This PR only transports the intent. The exec-server accepts the new
fields but does not apply them yet.

Remote commands therefore remain unsandboxed after this PR, just as they
are after PR #29099.

## Follow-up

The next PR will make exec-server read this portable intent, bind
symbolic workspace permissions to executor-native roots, choose the
sandbox for its own operating system, build the wrapper locally, and
then spawn the command.
2026-06-21 12:33:21 +02:00