Commit Graph

137 Commits

Author SHA1 Message Date
iceweasel-oai
2013e04354 Preserve filesystem permission path conventions (#39084)
## Why

Filesystem permission paths can use a convention that differs from the host
running Codex. Converting them immediately to native absolute paths can change
the meaning of ambiguous paths such as `/C:/secret` or Windows UNC paths.

## What changed

- Store literal filesystem permission paths as `PathUri` values through the
  runtime policy and execution protocol.
- Keep legacy string-based serialization at explicit protocol boundaries and
  reject conversions that cannot be represented losslessly.
- Encode native paths as opaque URIs when a normal file URI would imply the
  wrong path convention.

## Testing

Added coverage for cross-platform and ambiguous path round trips, UNC path
variants, permission-profile serialization, and deny-policy enforcement.

GitOrigin-RevId: 5247713796d1f2bb4e02f94eb9fc82d4698060f0
2026-08-17 21:49:30 +00:00
jif
3134ab6572 Restrict filesystem helper sandbox access (#38561)
## Why

Filesystem helpers only need access to their own executables. Granting access to
the containing directories also makes unrelated sibling files readable.

## What changed

- Allow filesystem helpers to read the Codex and Linux sandbox executables
  directly instead of their parent directories.
- Give macOS filesystem helpers a narrower Seatbelt profile that omits the
  normal process sandbox's `/Applications` read access.
- Preserve the existing platform defaults for normal sandboxed processes.

## Testing

Add unit and integration coverage for direct and symlinked sibling-file access,
sandboxed copies, allowed workspace reads, and the macOS-specific Seatbelt
profiles.

GitOrigin-RevId: 9380c6868a695bdf2275baa74bd5e8a30a64ba1b
2026-08-14 11:47:12 +00:00
hesham-oai
588e18aae5 Recover capability discovery after executor disconnects (#38420)
## Why

Transient executor disconnects could leave capability discovery and skill catalogs
stuck on a cached failure for the rest of a thread, even after the executor
reconnected.

## What changed

- Replay capability discovery after executor recovery and retry transient failures
  on later requests while continuing to cache permanent failures.
- Avoid caching skill catalogs produced from failed discovery so a later step can
  load the recovered catalog.
- Mark the MCP runtime dirty when recovered manifests change the projected MCP
  servers, and allow discovery to be cancelled with the turn.

## Testing

- Cover same-request recovery after a disconnect and recovery on a later request.
- Cover retry classification through connection-attempt errors and skill catalog
  caching after discovery recovers.

GitOrigin-RevId: a57f90844351e73ea831931f72a9ddc4e4f3335c
2026-08-13 19:14:30 +00:00
viyatb-oai
779e9114ae Reap orphaned processes in Linux sandboxes (#38396)
## Why

Sandboxed descendants can outlive their immediate parent and must be collected by
PID 1 in the Bubblewrap namespace.

## What changed

- Launch `codex-linux-sandbox` with Bubblewrap's `--as-pid-1` option, and fall
  back to the bundled Bubblewrap when the system version does not support it.
- Run the sandboxed command as a child, forward signals to it, reap other exited
  descendants, and preserve the command's exit status.
- Verify proxy bridge parent identity when arming its parent-death signal.

## Testing

Added Linux sandbox coverage for the filtered namespace reaper, orphan
collection, and fallback from an incompatible system Bubblewrap.

GitOrigin-RevId: 379f08d6c2732ea0a4caeb61f93ae302e16d2458
2026-08-13 15:45:51 +00:00
jif
9ed0047a61 Stabilize exec-server byte-budget tests (#38362)
## What changed

- Send the scripted HTTP response before queuing body deltas in the
  single-stream byte-budget test.
- Allow 30 seconds for barrier requests in both byte-budget tests while
  retaining the default timeout for other operations.

GitOrigin-RevId: ac65580b479d93fcbcca787f15e04c084f0aa56b
2026-08-13 12:00:19 +00:00
jif
c30a3e49c9 Support sandboxed file streaming in exec-server (#38356)
## Why

Streaming reads previously rejected requests that used a platform filesystem
sandbox.

## What changed

- Open streamed files in the sandbox helper and return the open file to the
  exec-server by passing a file descriptor on Unix or duplicating a file handle
  on Windows.
- Advertise support through the `sandboxedFileStreaming` environment capability.
- Preserve close-on-exec behavior for transferred descriptors, including the
  required inherited-descriptor cleanup on macOS.

## Testing

- Cover bounded sandboxed streams, continued reads after path replacement, and
  rejection of symlink escapes outside readable roots.

GitOrigin-RevId: 677b2444b74e834b78b87a8554bc119c1c6e08b2
2026-08-13 11:11:26 +00:00
Kyle Brown
9579479d28 Collect plugin metrics from remote executors (#38283)
## What changed

- Resolve manifest-declared metric operations against the executor filesystem for remote plugin commands.
- Create the measurement sidecar in an executor-native, owner-private temporary directory, stream its bounded output back for validation, and clean up the directory afterward.
- Extend the exec-server protocol with the executor temporary directory and atomic private-directory creation.

## Testing

- Cover remote unified-exec measurements for foreground and background commands.
- Verify private directories use owner-only permissions on Unix and fail closed on unsupported platforms.

GitOrigin-RevId: dbbd0a84717b91237fc5728e510e18994eb46dd4
2026-08-13 00:39:04 +00:00
iceweasel-oai
34db7e5563 Sandbox remote apply_patch operations (#38043)
## Why

Cross-platform remote `apply_patch` calls were rejected when filesystem writes
were restricted because patch verification and writes could not be safely
performed against executor files.

## What changed

- Route intercepted and direct remote patches through the executor-managed
  filesystem sandbox, including the configured workspace roots.
- Select the restricted-token sandbox for Windows executor paths when no
  Windows sandbox level was configured.
- Fail closed when an executor cannot enforce the requested sandbox, and treat
  executor-managed access failures as sandbox denials so approval can retry the
  patch without sandboxing.

## Testing

- Cover sandboxed remote patches, denied writes, approval retries, Windows
  sandbox selection, and executor filesystem enforcement.

GitOrigin-RevId: caddeed0b266c456a689080a14a3a58e2bd7887c
2026-08-11 17:47:22 +00:00
sayan-oai
646f7c0a91 Advertise environment config read support (#37654)
## What changed

- Add `environmentConfigRead` to exec-server environment capabilities and advertise it for local executors.
- Default the capability to `false` when deserializing responses from older executors.

## Testing

- Verify legacy capability responses remain compatible and the environment config integration reports support.

GitOrigin-RevId: c2110fea9e0e6e756eaeaaad096c50a943668c99
2026-08-09 03:05:10 +00:00
iceweasel-oai
92fb33b758 Report temporary directories in exec-server environment info (#37479)
## What changed

- Add optional `temporaryDirectories` file URIs to `EnvironmentInfo` so clients can resolve `:tmpdir` against executor-local defaults.
- Populate local environment info from `TMPDIR` on Unix and `TEMP`/`TMP` on Windows, resolving relative Unix paths against the working directory and removing duplicates.
- Cover protocol round trips, platform environment discovery, relative Unix paths, and the exec-server response.

GitOrigin-RevId: a41580f783b004011c064c7a4f9fdc7adf4fba10
2026-08-07 17:54:41 +00:00
sayan-oai
95c7265e84 Add executor-local config reads to the exec server (#37408)
## What changed

- Add the `environmentConfig/read` RPC for selecting literal TOML paths from executor-local config and requirements layers.
- Return layer precedence, cloud insertion points, source and base-directory metadata, and executor home and hostname context without normalizing path-bearing values.
- Expose the operation through both remote clients and local `Environment` instances, with invalid selectors reported as invalid parameters.

## Testing

- Cover projected remote config reads and rejection of empty selectors.

GitOrigin-RevId: fee15bf833de6bcb2058d405ed5b1d5d928d7218
2026-08-07 08:32:30 +00:00
TAFOYA-OAI
f8ac8fa6c6 Consolidate deferred environment provisioning APIs (#37340)
## What changed

- Remove the separate deferred registration handle, direct readiness publisher,
  and Noise environment upsert API from `EnvironmentManager`.
- Use `report_environment_provisioning_status` and
  `materialize_pending_noise_environment` as the provisioning flow while keeping
  ordinary environments isolated from provisioning reports.
- Cover readiness updates, invalid reports, duplicate materialization, and
  conflicts with ordinary environments in the deferred environment tests.

GitOrigin-RevId: d14207c0abb0636d331be4f875fec6e53f3d2fe6
2026-08-07 00:32:59 +00:00
TAFOYA-OAI
b6cddbf6d5 Test remote environments reported ready before selection (#37156)
## What changed

- Add end-to-end coverage for a remote environment whose provisioning status is
  reported ready before it is selected for a turn.
- Verify that `wait_for_environment` makes remote execution tools and selected
  capability-root context available once the transport is ready.
- Update pending-environment and relay tests to cover separate materialization
  and readiness reporting, including deferred connection and reconnection.

GitOrigin-RevId: d86d66e8f722e9bafac88b93d93e7c5244a7156e
2026-08-05 20:39:11 +00:00
TAFOYA-OAI
f5345f1ee8 Track provisioned environment state across registration (#37147)
## What changed

- Add pending, ready, and failed provisioning states for Noise environments.
- Preserve the same environment instance whether provisioning is reported before or after materialization, and reject conflicts with ordinary environments.
- Make readiness and failure reports idempotent while rejecting contradictory terminal transitions.
- Delay connection attempts until a provisioned environment is selected and provisioning succeeds.

## Testing

- Cover status reports before and after materialization, terminal failures, repeated and contradictory reports, and replacement between ordinary and deferred environments.

GitOrigin-RevId: 4360a8f2a80c1a99a1dc9257e5d77c07b72b8eb3
2026-08-05 20:08:41 +00:00
Adam Perry @ OpenAI
eeae88d8a6 Add opt-in concurrent exec-server request dispatch (#36987)
## Why

Sequential dispatch lets a long-running request block unrelated health checks
and cleanup on the same connection.

## What changed

- Add `--concurrent-requests <COUNT>` for local and remote exec-server
  connections, while retaining sequential dispatch when the option is omitted
  or set to `1`.
- Preserve handshake ordering before enabling concurrent dispatch.
- Reserve separate capacity for status, signal, terminate, and close requests so
  they remain responsive when ordinary request capacity is saturated.
- Drain queued client responses during disconnect and cancel outstanding
  request tasks during connection shutdown.

## Testing

- Cover CLI parsing and concurrency-limit validation.
- Verify default sequential behavior, pipelined handshake ordering, concurrent
  request progress, control-request responsiveness, and disconnect handling.

GitOrigin-RevId: 48e4b092e318204ed635543f01f9ee0e7df095fc
2026-08-04 22:28:15 +00:00
Charlie Marsh
40e5de94e9 Avoid redundant filesystem metadata probes (#36898)
## What changed

- Reuse directory-entry file types in local memory listing while continuing to
  exclude symlinks, and reuse rollout metadata when reading modification times.
- Avoid following non-symlinks twice in direct filesystem metadata and directory
  listing operations while preserving target classification for valid symlinks.

## Testing

- Cover symlink handling in local memory listing and search.
- Extend Unix filesystem tests for followed file and directory symlinks and
  dangling metadata links.

GitOrigin-RevId: e4e24576e2e9db704f9da54727928e121f81dc86
2026-08-04 13:56:52 +00:00
Adam Perry @ OpenAI
ee0247f95a Extract exec-server request dispatching (#36440)
## What changed

- Move JSON-RPC request, notification, response, error, and malformed-message handling into a dedicated `RequestDispatcher`.
- Keep the connection loop responsible for receiving events and closing the connection when dispatch reports a terminal condition.

## Testing

- Add an integration test confirming that ordinary requests are processed serially by default, including when a blocking `process/read` queues later requests.

GitOrigin-RevId: 29d1358d4524edd492ff3855b29f23c42c8b3390
2026-08-01 06:02:48 +00:00
viyatb-oai
0042b00986 Record normalized sandbox violation events (#36207)
## Why

Filesystem denials and managed-network blocks did not share a structured event
shape, requiring downstream consumers to rediscover enforcement paths and parse
backend-specific output. See https://github.com/openai/codex/pull/17573.

## What changed

- Add normalized filesystem and network violation types in `codex-sandboxing`
  and emit them through a shared tracing seam.
- Classify filesystem denials by backend and reason, retaining an optional path
  and bounded output snippet, and preserve managed-network block context.
- Report the sandbox type through exec-server responses so unified exec can
  classify remote denials without guessing; omitted values remain compatible
  with older peers.
- Record violations from exec, apply-patch, shell-escalation, unified-exec, and
  managed-network enforcement paths without changing denial behavior.

## Testing

- Cover filesystem classification, path extraction, `SIGSYS`, network event
  conversion, protocol compatibility, and remote sandbox-type propagation.

GitOrigin-RevId: d673173b4fa6bdf0a24421194a8a61c81fab9c96
2026-07-30 19:23:33 +00:00
jif
5decb399ae Respect filesystem permissions during capability discovery (#36124)
## Why

Executor capability discovery can traverse plugin and skill roots, including
symlinks. Under restricted filesystem permissions, discovery must not expose
files outside the permitted paths.

## What changed

- Pass each environment's filesystem sandbox context through capability root
  discovery and apply it to metadata, directory walks, and file reads.
- Enable discovery for restricted sessions so permitted executor skills remain
  available while inaccessible roots and symlink targets are omitted.
- Key discovery caches by sandbox context and reject sandboxed discovery on
  executors that do not advertise support for it.
- Split requests with more than 128 roots into supported-size batches.

## Testing

- Cover permitted and denied external symlink targets.
- Verify restricted skill listing excludes inaccessible skills.
- Verify cache separation across permission contexts and discovery of 129 roots.

GitOrigin-RevId: 44d16468ca003403bdb8b71a04ae8c9ff94ed494
2026-07-30 10:01:06 +00:00
jif
1da9f846b3 Test exec-server compatibility across Codex versions (#35990)
## Why

The executor protocol supports Codex releases back to `0.145.0`, so compatibility needs to hold when either side of the app-server/exec-server connection is upgraded first.

## What changed

- Define `MINIMUM_SUPPORTED_CODEX_VERSION` in `codex-exec-server-protocol`.
- Add a Unix test harness that runs current-to-released and released-to-current command execution over authenticated Noise connections.
- Test the current binary against itself, the latest release, and the minimum supported release by default, while allowing explicit release versions.
- Verify that the remote command runs successfully and relay payloads remain encrypted, and increase the relay test timeout to accommodate the end-to-end scenarios.

GitOrigin-RevId: faea8d44fce161f40ed15170876a1282a6de4c22
2026-07-29 16:30:23 +00:00
TAFOYA-OAI
6c13b113a3 Allow environment readiness updates in place (#35875)
## What changed

- Add `EnvironmentManager::publish_ready_info` to update the selected capability
  roots for an existing environment without replacing it.
- Store readiness as an atomically replaceable snapshot so repeated publications
  expose the latest roots while deferred completion still controls connection
  readiness.
- Validate published roots and reject updates for missing environments.

## Testing

- Cover publication, replacement, repeated and invalid updates, and interaction
  with deferred environment completion.

GitOrigin-RevId: 1477f60e1319f840e6b33703ef2af052713779fd
2026-07-29 03:07:13 +00:00
iceweasel-oai
6b23635a7e Terminate Windows non-TTY processes on interrupt (#35655)
## Why

Windows non-TTY exec sessions reported interrupts as unsupported, so sending
Ctrl-C through `write_stdin` did not stop the running process.

## What changed

- Route interrupt requests for Windows non-TTY processes through their existing
  termination callback, including pipe-backed processes.
- Track whether driver-backed Windows sessions use a TTY so PTY interrupts keep
  their existing behavior.
- Consume the terminator after a successful interrupt to avoid invoking it again
  when the process handle is dropped.

## Testing

Added coverage for local and remote exec-server sessions, unified exec, pipe
fallbacks, and legacy Windows sandbox processes.

GitOrigin-RevId: 34504d01f091ef57bb961e98ad5a8d9f1acee4ee
2026-07-27 18:37:11 +00:00
Celia Chen
09241ae4db Decouple exec-server HTTP from reqwest types (#35059)
## What changed

- Rename `ReqwestHttpClient` to `RouteAwareHttpClient` to reflect that delegated HTTP uses Codex's shared route-aware transport.
- Use `codex_http_client` response and error types plus transport-neutral `http` and `url` types, removing the exec server's direct `reqwest` dependency.

## Testing

- Cover fragment stripping and Unicode hostname normalization for delegated HTTP requests.

GitOrigin-RevId: 8b0fc60a76004feb57198bfb4afb1371c9ceb1bd
2026-07-24 00:53:05 +00:00
Celia Chen
94ebae725e Route exec-server WebSockets through configured proxies (#35056)
## Why

Remote environment connections need to honor Codex's effective outbound proxy policy, including when a rendezvous connection reconnects.

## What changed

- Pass the configured `HttpClientFactory` into remote environment transports and use `WebSocketConnector` for exec-server and rendezvous WebSockets.
- Resolve proxy routes asynchronously so these connections can use the configured system proxy.
- Add connector options that preserve Tungstenite's default TLS behavior and enable `TCP_NODELAY` for latency-sensitive rendezvous traffic.

## Testing

- Verify prepared remote environments connect through a configured system proxy.
- Verify initial and reconnected encrypted relay peers use the system proxy.
- Cover default TLS selection and opt-in `TCP_NODELAY` behavior in the WebSocket client.

GitOrigin-RevId: 8a8da2116e37cb3a891269d0c0b037986fecdd3c
2026-07-24 00:16:42 +00:00
Celia Chen
1ee8f49175 Route exec-server HTTP through configured proxy policy (#35023)
## Why

Delegated HTTP requests need to honor the same outbound proxy policy as the
Codex process that starts the exec server.

## What changed

- Pass the configured `HttpClientFactory` through local and remote exec-server
  startup and use route-aware client pools for delegated HTTP and local MCP
  requests.
- Preserve per-request timeouts and follow-or-stop redirect behavior while
  keeping request URLs and sensitive response headers out of diagnostics.

## Testing

- Cover configured system-proxy routing across the exec-server transport.
- Cover both redirect policies and verify that success and failure logs do not
  expose request or response secrets.

GitOrigin-RevId: 4af6aec1d265c4db62dfcb6e1fb076fb31736137
2026-07-23 22:39:28 +00:00
Celia Chen
265cd2e100 Initialize execution environments with the final HTTP policy (#34995)
## Why

The TUI must inspect the default execution environment before loading its final
configuration. Initializing the environment manager at that point can give
startup services the bootstrap HTTP policy instead of the effective policy after
managed requirements are applied.

## What changed

- Split environment discovery from manager construction so callers can inspect
  the default environment without starting remote connections.
- Build the environment manager after final configuration loading and pass its
  resolved `HttpClientFactory` through all construction paths.
- Add shared test support for managers that use the legacy default HTTP policy.

## Testing

- Cover connection-free environment discovery and explicit HTTP policy
  propagation.
- Verify TUI startup services use the final managed `respect_system_proxy` value.

GitOrigin-RevId: 928fa31e6b4bcfbe1a121cade2f351427fdfa0f4
2026-07-23 19:24:41 +00:00
Adam Perry @ OpenAI
ad020f29ae Initialize missing-path behavior in exec-server sandbox test (#34615)
## What changed

Set `missing_path_behavior` to `None` for the minimal and project-root
filesystem entries in the custom-`arg0` exec-server test.

GitOrigin-RevId: c1d8df7b5ca8dcf6c89c314b92747bfeae80286e
2026-07-21 21:10:14 +00:00
iceweasel-oai
87f71e35b8 Skip missing paths in filesystem sandbox entries (#34598)
## Why

Default read-only protections for project metadata should apply when paths such
as `.git`, `.agents`, and `.codex` exist, without causing sandbox setup to
materialize missing paths as ACL targets.

## What changed

- Add an optional `missing_path_behavior` to filesystem sandbox entries and
  mark default project-metadata protections with `skip`.
- Preserve the behavior through permission transforms and exec/MCP protocol
  serialization while keeping existing path wire variants stable.
- Ignore skip-missing entries when projecting configuration or Windows sandbox
  overrides, while retaining explicit metadata carveouts.

## Testing

- Cover protocol round trips for path and special-path entries.
- Verify default metadata protections and Windows explicit carveout handling.

GitOrigin-RevId: 6df13dadacdd131c44aab9f15a967c81051355c1
2026-07-21 19:17:18 +00:00
jif
4f1992732c Preserve custom arg0 for sandboxed exec-server processes (#34497)
## Why

Sandbox wrappers replaced the process launch command and did not carry an
`ExecParams.arg0` override through to the inner process.

## What changed

- Route sandboxed Unix launches with a custom `arg0` through a helper mode that
  re-execs the requested program with the override.
- Expose the helper executable to the filesystem sandbox and dispatch its mode
  from Codex and exec-server test binaries.

## Testing

Add coverage for the prepared sandbox command and an end-to-end remote process
that verifies both the custom `arg0` and filesystem restrictions.

GitOrigin-RevId: c9f8eef3906d184e670184c2eeed250d5895a9ca
2026-07-21 08:59:41 +00:00
iceweasel-oai
35c2278dd5 Support Windows sandboxing in the exec server (#34423)
## Why

Sandboxed process launch through the exec server was unsupported on Windows.

## What changed

- Add a shared native process launcher that selects the Windows sandbox session backend when required and otherwise launches PTY or pipe-based processes.
- Pass Windows permission, workspace-root, proxy, filesystem-override, and desktop settings through exec-server process preparation.
- Preserve piped stdin and inherited file descriptors across the shared launch path.

## Testing

- Verify that a remote Windows sandbox process accepts stdin while blocking a write under a read-only policy.
- Enable workspace-root sandbox tests on native Windows while continuing to skip environments that cannot emulate Windows sandbox semantics.

GitOrigin-RevId: 6719ccbe87ad0d5a57af56503e3f23f894c60a71
2026-07-20 22:35:14 +00:00
viyatb-oai
9f6c29e281 Launch managed network proxies on remote executors (#33906)
## Why

Remote executions need managed-network proxy listeners in the executor so their
loopback proxy addresses are reachable by the launched process.

## What changed

- Add a capability-gated exec-server protocol field for executor-local proxy
  launch configuration, including network policy, audit metadata, and execution
  attribution.
- Start the proxy while preparing a remote process, replace inherited proxy
  environment variables with its local addresses, and derive the sandbox
  context from its listeners.
- Keep the proxy alive until inherited output streams close, then shut it down.
- Reject unsupported remote settings such as MITM and credential injection.

## Testing

- Cover configuration round trips and rejection of unsupported settings.
- Verify executor-local startup, blocked-domain enforcement, protocol
  compatibility, and proxy lifetime through process closure.

GitOrigin-RevId: c984f54e3e600aa9ebcbf8cf4574046e2c199d11
2026-07-17 21:20:14 +00:00
Bryan Ashley
08e30a2e4e Add batched executor capability discovery (#33852)
## Why

Selected capability roots can contribute plugins, MCP servers, connectors, and
skills. Discovering each contribution separately requires repeated access to the
executor filesystem.

## What changed

- Add the `capabilityRoots/discoverV1` exec-server RPC to scan selected roots and
  materialize recognized plugin manifests, configuration files, skill
  instructions, and skill metadata in one bounded request.
- Add the opt-in `executor_capability_discovery` feature, with a thread-scoped
  cache and per-step snapshot shared by MCP and skill discovery.
- Parse MCP, connector, and skill contributions from the materialized snapshot,
  including serving cached skill instructions without another filesystem read.

## Testing

- Cover discovery limits, manifest precedence, root-local failures, cache reuse,
  plugin contributions, and parity with the existing environment skill loader.

GitOrigin-RevId: f98fd2321cafb58c596db02da1f83c09d8eb375d
2026-07-17 15:45:48 +00:00
jif
79177c3e20 Propagate deferred environment capability roots to MCP (#33427)
## What changed

- Let deferred environments provide selected capability roots with their ready signal.
- Validate that those roots have unique, non-empty IDs, belong to the registering environment, and stay within the root limit.
- Include roots from ready turn environments when resolving MCP contributions, and refresh the MCP runtime when the selected root set changes.
- Expose the exact ready root set to MCP contributors so executor plugins become available with their environment.

## Testing

- Cover ready-root propagation, validation failures, replacement isolation, reconnection, and MCP plugin availability refresh.

GitOrigin-RevId: ec3498aab1164824025094e96a9b1063b7b731ad
2026-07-15 21:07:43 +00:00
sayan-oai
3afbd8dd45 Report selected environment connection transitions (#33251)
## What changed

- Track connected and disconnected states across initial remote exec-server connections and reconnection attempts.
- Emit experimental `thread/environment/connected` and `thread/environment/disconnected` app-server notifications for each thread selecting the environment. Each payload identifies the thread and environment; current state is not replayed when a thread starts.
- Stop forwarding connection events when an environment selection is removed or replaced.

## Testing

- Cover connection, disconnection, reconnection, shared-environment notifications, and replacement of a selected environment.

GitOrigin-RevId: 5dd767372363c4a2a8319fc16164be117d5bd20c
2026-07-15 05:21:13 +00:00
TAFOYA-OAI
32cd5d4eab Defer Noise environment connections until registration (#33166)
## What changed

- Replace pending WebSocket URL registration with deferred Noise environment
  registration that gates connection attempts on an explicit readiness signal.
- Reuse the Noise rendezvous transport after readiness so reconnects request a
  fresh connection bundle.
- Preserve terminal errors for failed or dropped registrations and keep late
  completion isolated from replacement environments.

## Testing

- Add coverage for readiness gating, registration failure and replacement,
  eager Noise connections, and reconnection through a fresh rendezvous bundle.

GitOrigin-RevId: 83e23fa03a02e3b2bdf1a83fe26d7ac461f55cf3
2026-07-14 21:07:27 +00:00
Adam Perry @ OpenAI
75470c3e2f Add exec-server environment status checks (#32899)
## What changed

- Add the initialized `environment/status` RPC, which reports `ready` when the exec server can handle requests.
- Expose environment IDs and `ready`, `pending`, or `disconnected` status through `EnvironmentManager` and `Environment`.
- Keep status checks non-mutating: they do not start or recover lazy remote environments, and probe only an existing connection.

## Testing

- Cover the status RPC over WebSocket and the in-process request processor.
- Verify that checking an unstarted stdio environment leaves it pending and that failed connections report as disconnected.

GitOrigin-RevId: 22febeb6a3457849292128a8991c6400c22b3fd8
2026-07-13 23:43:15 +00:00
pakrym-oai
4472698728 Support pending remote environment registration (#32231)
## Why

Remote environment provisioning can finish after a thread starts, before an
exec-server WebSocket URL is available.

## What changed

- Add `EnvironmentManager::register_pending_environment` and a one-shot
  `PendingEnvironmentRegistration` handle that resolves to either a validated
  WebSocket URL or a terminal provisioning error.
- Let lazy remote exec-server clients wait for that result, while preserving
  reconnection behavior after a successful registration.
- Keep replacement registrations isolated so completing an older handle does
  not resolve the current environment with the same ID.

## Testing

Add coverage for successful connection and reconnection, provisioning and
dropped-registration failures, invalid URLs, replacement isolation, and the
deferred-executor startup flow.

GitOrigin-RevId: 5c05be2b72291b77a1f71176d7075b1ad63332a5
2026-07-10 18:08:35 +00:00
pakrym-oai
c8dc8e5fd5 Propagate workspace roots to exec-server sandboxes (#32214)
## What changed

- Pass configured workspace roots from core to the exec server so filesystem and process sandbox permissions are materialized against the intended roots.
- Preserve an explicitly empty workspace-root list instead of treating the sandbox working directory as an implicit root.
- Initialize filesystem sandbox contexts with their working directory as the default workspace root.

## Testing

- Add end-to-end coverage for patch and command writes inside and outside workspace roots.
- Verify remote filesystem and process sandboxes do not grant access through an empty workspace-root list.

GitOrigin-RevId: 6684c8f7de50970b45a29e2a6323df954c96f9f6
2026-07-10 16:55:01 +00:00
jif
c094a58c9f Test the shared exec-server HTTP response byte budget (#32122)
## Testing

Add a concurrent-stream regression test that fills the connection-wide queued
body budget across two HTTP responses. Verify that the overflowing stream
reports the byte-budget error while the other stream still drains successfully.

GitOrigin-RevId: 65efacb15f1368b1de3b98e7ef65ebca66a3840b
2026-07-10 10:16:47 +00:00
jif
4ba2815014 Bound streamed exec-server HTTP response bodies (#32112)
## Why

Frame-count backpressure does not bound the amount of executor-controlled data
retained in streamed HTTP response queues, and a single body delta could exceed
the intended wire size.

## What changed

- Limit each decoded `http/request/bodyDelta` payload to 1 MiB, split locally
  produced response chunks at that boundary, and reject oversized incoming
  deltas.
- Apply a shared 16 MiB byte budget across queued HTTP response streams. Release
  capacity as deltas are consumed and fail a stream when the budget is
  exhausted.

## Testing

Added coverage for rejecting an oversized delta and for failing a stream after
its queued deltas exhaust the shared byte budget.

GitOrigin-RevId: be9205fef44b0ec96b84b31a8e059b1cb9cd3f3b
2026-07-10 09:56:03 +00:00
pakrym-oai
1f0566d3f5 exec-server: expose process helper to outer sandbox (#31937)
## Why

Sandboxed exec-server process requests can use a restricted filesystem
profile that does not expose the exec-server binary. On Linux, the outer
bubblewrap stage re-enters that binary with the `codex-linux-sandbox`
argv0 to install seccomp, so hiding the binary prevents the requested
process from starting.

## What changed

- add the configured `codex_self_exe` to the process permission profile
before constructing the outer platform sandbox
- add a Linux exec-server integration test that starts a real remote
process with restricted reads and verifies it can read an allowed
workspace file

## Test plan

- `just test -p codex-exec-server process_sandbox`
- `just test -p codex-exec-server --test exec_process
remote_process_keeps_sandbox_helper_visible_with_restricted_reads`
2026-07-09 18:28:55 -07:00
pakrym-oai
ac3da4fb1a exec-server: materialize filesystem workspace roots (#31892)
Filesystem helper requests currently turn symbolic `:workspace_roots`
permissions into a sandbox policy before applying the workspace roots
from the filesystem sandbox context. This can accidentally broaden
filesystem access to the cwd instead of limiting it to the selected
workspace roots.

Materialize project-root permissions using the context workspace roots
before deriving the filesystem sandbox policy. The same converted roots
are then reused when constructing the sandbox command, keeping policy
enforcement and process setup aligned.

Adds a remote filesystem integration test that verifies a file inside
the selected workspace root is readable while a sibling under the cwd is
denied.

## Validation

- `just test -p codex-exec-server
remote_read_file_materializes_environment_workspace_roots` (macOS,
outside the outer Seatbelt sandbox)
2026-07-09 16:20:06 -07:00
jif
13ba8058f2 Resolve selected capability roots without starting executors (#31581)
## Why

A thread can select skill roots that live in an executor environment.
`skills/list` needs a passive snapshot of the roots that are usable now:
it must not start an executor, wait for recovery, or reconnect a failed
environment.

The initial implementation checked the immutable first startup result.
After a successful connection later entered recovery or failed, that
result still looked successful. A read-only catalog request could then
wait for recovery or trigger a new connection while reading the
filesystem.

## What

- inspect readiness from the current exec-server connection state
- return roots only while their environment can serve a request
immediately
- omit environments that have not started, are connecting, or are
recovering
- return warnings for missing environments and terminal connection
failures
- add a fail-fast filesystem view that never starts, waits for, or
reconnects an environment
- expose the passive selected-root snapshot through `CodexThread`

## Behavior

- Local and currently connected environments are ready.
- Starting and recovering environments are omitted without a warning so
callers can retry later.
- Missing and terminally failed environments are omitted with a warning.
- A disconnect between readiness inspection and filesystem access fails
promptly instead of crossing into the normal recovery path.
- Normal model-turn and execution paths keep their existing reconnect
behavior.

## Design

The recovery policy is private to the exec-server client. Callers choose
the explicit fail-fast filesystem method; the existing client and
filesystem APIs remain reconnecting. This keeps the passive contract at
the transport boundary instead of plumbing timeout or retry flags
through the skills stack.

## Coverage

- a lazy stdio environment stays unstarted during passive inspection
- missing and terminally failed environments surface warnings
- a real websocket disconnect proves current readiness drops, a
previously acquired fail-fast filesystem handle returns promptly, and
readiness returns after recovery

## Scope

This PR only provides passive readiness and fail-fast filesystem
primitives. It does not add app-server API fields or notifications.

## Stack

- #31582 uses these primitives for experimental thread-scoped
`skills/list`.
- #30228 adds targeted invalidation notifications.
2026-07-09 11:17:05 +01:00
jif
a52b35fcf6 fs: support pruning hidden directories during walks (#31570)
Why

Filtering hidden directories after a walk is too late: their descendants
consume traversal limits, and canonical directory deduplication can let
a hidden path claim a target before a visible symlink reaches it.

What this changes

- Add an optional pruneHiddenDirectories walk option. It defaults to
false and is omitted from the wire when disabled.
- Return hidden directory entries, but do not traverse them or add their
canonical identities to the visited set.
- Cover the visible-symlink-to-hidden-directory case through both local
and remote filesystem implementations.

This is the small filesystem prerequisite for #31566. Skill-specific
behavior remains in that PR.
2026-07-08 13:49:05 +01:00
richardopenai
d4ec08b8f0 [codex] consume pushed exec-server process events (#30273)
## Summary

- complete unified-exec processes from the ordered event stream instead
of issuing a final zero-wait `process/read`
- add optional executor sandbox-denial state to `process/exited`
- retain `process/read` as a retained-output and compatibility fallback
for receiver lag, sequence gaps, and legacy servers
- recover sandbox-denial state across transport reconnection
- cover the real `TestCodex` remote-exec path without adding a public
test-only event constructor

## Why

A successful one-shot tool call currently receives its output and
terminal notifications, then pays another wide-area `process/read` round
trip before returning. Staging traces showed that remote response wait
accounted for more than 99.8% of RPC time; local serialization,
queueing, and deserialization were below 0.6 ms.

## Measured impact

A direct staging A/B used the same build and route and changed only
completion mode. Each arm ran three times with 30 one-shot
`/usr/bin/true` calls per run. The table reports the median of the three
per-run percentiles.

| Metric | Final `process/read` | Pushed events | Change |
| --- | ---: | ---: | ---: |
| End-to-end completion p50 | 159.5 ms | 118.7 ms | -40.8 ms (-25.6%) |
| End-to-end completion p95 | 182.4 ms | 131.7 ms | -50.6 ms (-27.8%) |
| Completion-wait p50 | 80.1 ms | 41.5 ms | -38.5 ms (-48.1%) |
| Final `process/read` RPC p50 | 79.9 ms | eliminated | -79.9 ms |

TCP_NODELAY was enabled in both A/B arms, so its effect cancels out. The
successful, complete, in-order event path issued zero final
`process/read` calls.

## Compatibility and recovery

- new servers send `sandboxDenied` on `process/exited`
- legacy servers omit it, which triggers one compatibility
`process/read`
- broadcast lag or a sequence gap triggers a retained-output read
- recovery remains bounded by the server's existing 1 MiB
retained-output window
- complete, in-order event streams issue no completion read
- sandbox denial is attached to the exit event before consumers can
observe process completion
- server-first and client-first rollouts remain wire-compatible;
server-first realizes the latency win immediately

## Integration coverage

The `TestCodex` suite exercises four distinct remote-exec contracts:

- complete pushed output/exit/close with zero reads
- direct pushed sandbox denial with zero reads
- legacy missing denial metadata with exactly one compatibility read
- count-bounded replay eviction recovered from retained output without
duplication

## Validation

- `just test -p codex-core
exec_command_consumes_pushed_remote_process_events`: 4 passed
- `just test -p codex-core unified_exec::process_tests::`: 4 passed
- `just test -p codex-exec-server`: 294 passed, 2 skipped
- `just test -p codex-exec-server-protocol`: 5 passed
- `just test -p codex-rmcp-client`: 89 passed, 2 skipped
- focused Bazel `//codex-rs/core:core-all-test`: passed across 16 shards
- scoped `just fix` passed for core and exec-server
- `just fmt` passed

The complete workspace suite was not rerun; focused Cargo and Bazel
coverage passed for the changed behavior.
2026-06-26 18:05:52 -07:00
stevenlee-oai
b5866eebd6 Persist Cloudflare affinity cookies for MCP HTTP (#29516)
[Codex Thread
019ef1f9-36e2-7e91-9337-504f097b9dc1](https://codex-thread-link.openai.chatgpt-team.site/thread/019ef1f9-36e2-7e91-9337-504f097b9dc1)

## Why

Hosted plugin-service Streamable HTTP MCP traffic uses
`https://chatgpt.com/backend-api/ps/mcp` and depends on Cloudflare's
`__cflb` cookie for load-balancer affinity. The local and exec-server
`http/request` path built a fresh reqwest client for each request
without installing Codex's existing shared ChatGPT Cloudflare cookie
store, so affinity could be lost between calls.

This is an affinity-hardening change motivated by an incident
investigation. It does not establish the broader connector-cache
incident RCA or claim to fix that incident in full.

## What changed

- Install the existing process-local, strictly allowlisted ChatGPT
Cloudflare cookie store on the reqwest client used by
`ReqwestHttpClient`.
- Fresh clients now share allowed Cloudflare infrastructure cookies
within the process that originates the local or exec-server network
request.
- Keep the existing HTTPS ChatGPT-host and Cloudflare-cookie-name
restrictions. This does not introduce a general cookie jar or send
ChatGPT Cloudflare cookies to unrelated hosts.

## Test coverage

- `codex-client` unit coverage verifies that the existing strict store
accepts and returns `__cflb` for HTTPS ChatGPT URLs.
- The exec-server HTTPS integration test sends four independent
`http/request` calls through a local TLS-intercepting proxy and verifies
that:
- `Set-Cookie: __cflb=west` is sent on the next plugin-service request;
  - a later `Set-Cookie: __cflb=central` replaces the stored value;
  - non-Cloudflare session cookies are discarded;
  - no stored ChatGPT Cloudflare cookie is sent to a non-ChatGPT host.
- `just test -p codex-client` — 38 passed.
- `just test -p codex-exec-server --test chatgpt_cloudflare_affinity` —
1 passed.
- `just bazel-lock-check` — passed.

## Non-goals

- No persistence of ChatGPT auth, account, session, residency, or
arbitrary cookies.
- No cookie persistence for third-party MCP servers.
- No special composition of caller-provided `Cookie` headers.
- No plugin-service, connector-cache, Habitat/habicache, routing,
redirect, or API-contract changes.
- No broader incident RCA conclusions.
2026-06-26 02:23:24 -04:00
jif
8f02973d25 Persist selected capability roots and resolve availability per model step (#29856)
## Why

`selectedCapabilityRoots` is durable thread intent: “use this capability
root from environment `worker`.”

The important product assumption is:

> One environment ID always names the same logical executor and stable
contents.

`worker` does not silently change from executor A to an unrelated
executor B. The process-local connection handle for `worker` can still
be replaced while Codex is running, though, for example when
`environment/add` registers a fresh handle for the same logical
environment.

The thread should persist only the stable selection. Each model step
should pair that selection with the exact ready handle captured for that
step.

## The boundary

```text
persisted thread intent
  plugin@1 -> environment "worker"
                |
                | capture the current step
                v
model-step view
  unavailable, or
  plugin@1 + worker's exact captured ready handle
```

The environment ID is the stable identity and cache key. The
`Arc<Environment>` is only a process-local handle retained so consumers
of one model step use the same captured environment. It is never
persisted and it does not imply different environment contents.

## What changes

### Persist the stable selection

Selected roots are written into `SessionMeta` and restored with the
thread. Forked subagents inherit the same selections, including
bounded-history forks.

Only stable data is persisted: root ID, environment ID, and root path.

### Capture readiness together with the exact handle

The environment snapshot records:

```rust
environment_id -> Some(Arc<Environment>) // ready in this step
environment_id -> None                   // still starting in this step
```

This prevents readiness and execution from coming from different
registry snapshots.

For example:

```text
step snapshot: worker -> handle A, ready
environment/add: worker -> fresh handle B for the same logical environment
current step: plugin@1 still uses captured handle A
```

Without carrying handle A in the snapshot, the resolver could combine “A
was ready” with handle B and treat B as ready before it had finished
starting.

This does not change cache invalidation. Stable capability metadata
remains identified by environment ID and capability root. Replacing a
process-local handle under the same stable environment ID does not
invalidate or rediscover that metadata.

### Resolve availability per model step

- A ready captured environment produces resolved roots using its
captured handle.
- A starting, missing, or failed environment is omitted from that step.
- A selected lazy environment that is outside the turn's captured
environment set is asked to start, and a later step can observe it as
ready.
- No capability files are scanned here.

Transient transport disconnects remain the remote client's reconnect
concern. This PR models initial attachment/readiness; it does not add
live socket-connectivity state.

## Example

```text
thread selection: plugin@1 -> environment "worker"

step 1: worker is starting -> plugin@1 unavailable
step 2: worker is ready    -> plugin@1 resolves through worker's captured handle
step 3: fresh local handle -> current step remains pinned; a later step captures its own view
```

Temporary unavailability does not discard the durable selection. Later
PRs can retain stable metadata caches while projecting only currently
available capabilities into model-visible World State.

## Compatibility

The app-server request shape does not change. Older rollouts without
`selected_capability_roots` deserialize to an empty list.

## Stack

1. **This PR:** persist stable selected roots and resolve them through
an exact model-step handle.
2. #29960: cache stable skill metadata and project available skills into
World State.
3. #29946: cache stable plugin declarations and manage the separate live
MCP runtime.
2026-06-25 17:49:43 +00:00
jif
b215961a56 Support OAuth for HTTP MCP servers from selected executor plugins (#28529)
## Why

#28522 routes selected-plugin HTTP MCP traffic through the owning
executor, but OAuth bootstrap and refresh still used host-local clients.
Executor-only servers therefore cannot complete discovery or login
through the same network boundary as the MCP connection.

## What changed

- adapt `codex_exec_server::HttpClient` to RMCP 1.8's `OAuthHttpClient`
contract
- let RMCP own discovery, dynamic registration, PKCE, token exchange,
and refresh
- route auth status, persisted-token startup, and app-server login
through the server runtime while preserving the existing local discovery
path
- add optional `threadId` to `mcpServer/oauth/login` and echo it in the
completion notification
- implement RMCP's redirect policy and 1 MiB OAuth response limit over
executor HTTP
- cover selected-thread OAuth discovery and login through an
executor-only route

Depends on #28522.
2026-06-25 10:31:17 +01:00
jif
96d8e34712 Follow directory symlinks in filesystem walks (#29844)
Stack 3 of 3. Stacked on #29842.

## What changes

Adds an opt-in `followDirectorySymlinks` setting to `fs/walk`.

When enabled, the walk follows directory symlinks but continues to
ignore symlinked files. Canonical directory identities prevent symlink
cycles, while normal paths keep their existing spelling.

Environment skill discovery enables the setting so symlinked skill
directories continue to work with the new single-RPC scan.
2026-06-24 20:52:36 +01:00
richardopenai
74dcce594d [codex] Trace exec-server JSON-RPC requests (#27466)
## Why

Exec-server JSON-RPC calls can cross local and remote transports, but
trace context stopped at the RPC boundary. That made client and server
work difficult to correlate when diagnosing latency or failures.

## What changed

- Propagate the current W3C trace context on outbound JSON-RPC requests.
- Parent inbound request spans from received trace context.
- Record the received JSON-RPC method on server spans and keep each span
open through response enqueue.
- Add only the OTEL dependencies required by the exec-server crate.

## Stack

Review and land this stack in order:

1. #27466 — trace exec-server JSON-RPC requests **(this PR)**
2. #27467 — record bounded connection, request, and process lifecycle
metrics
3. #27470 — observe remote registration and Noise rendezvous lifecycle

## Validation

- `just test -p codex-exec-server --lib` (153 passed)
- `just bazel-lock-check`
- `just fix -p codex-exec-server`
2026-06-24 12:50:18 -07:00