Commit Graph

247 Commits

Author SHA1 Message Date
Alexi Christakis
166658a34a Tie remote exec servers to their parent stdin (#35843)
## What changed

- Add `--exit-on-stdin-close` and the `CODEX_EXEC_SERVER_EXIT_ON_STDIN_CLOSE` environment variable as opt-in controls for remote exec servers.
- Gracefully drain active sessions and processes when the parent closes stdin, then flush telemetry before exiting.
- Remove the parent-lifetime environment variable from child process environments.

## Testing

- Cover parent disconnects after signal-listener failures.
- Exercise remote shutdown end to end, including child termination and final telemetry metrics.
- Verify that explicitly disabling the environment variable preserves local exec-server behavior.

GitOrigin-RevId: 63063bc097b54684c370bd545cd32d17c4e55d90
2026-07-28 23:27:46 +00:00
iceweasel-oai
6b23635a7e Terminate Windows non-TTY processes on interrupt (#35655)
## Why

Windows non-TTY exec sessions reported interrupts as unsupported, so sending
Ctrl-C through `write_stdin` did not stop the running process.

## What changed

- Route interrupt requests for Windows non-TTY processes through their existing
  termination callback, including pipe-backed processes.
- Track whether driver-backed Windows sessions use a TTY so PTY interrupts keep
  their existing behavior.
- Consume the terminator after a successful interrupt to avoid invoking it again
  when the process handle is dropped.

## Testing

Added coverage for local and remote exec-server sessions, unified exec, pipe
fallbacks, and legacy Windows sandbox processes.

GitOrigin-RevId: 34504d01f091ef57bb961e98ad5a8d9f1acee4ee
2026-07-27 18:37:11 +00:00
viyatb-oai
462ed19a05 Enable network policy callbacks for remote exec (#35652)
## What changed

- Forward remote managed-network policy requests to the controller-side decider when Guardian review is enabled.
- Bound callbacks using the configured permission-hook and Guardian review timeouts, restore trusted execution attribution, and recheck live baseline policy before requesting a decision.
- Cancel pending decisions when the process or execution scope ends, and propagate EOF across the sandbox proxy bridge with half-closes.

## Testing

- Cover callback enablement, strict allowlist behavior, live policy updates, trusted attribution, and execution-scope cancellation.

GitOrigin-RevId: 83b47657c8c4546dfe98eec8b272833e96a46b7e
2026-07-27 18:07:31 +00:00
viyatb-oai
3a08af44b2 Handle exec-server network policy requests in the client (#35359)
## What changed

- Add client-side handling for exec-server network policy requests, including request validation, per-process decision routing, and allow, deny, or ask responses.
- Bound concurrent callbacks and fail closed when requests are invalid, exceed capacity, time out, or outlive their process session.
- Replace the callback opt-in flag with a launch-level `policyDecisionTimeoutMs` value and include transport overhead in the executor timeout.
- Preserve callback admission limits across connection recovery and clean up policy state when process startup is abandoned or a session ends.

## Testing

- Cover decision forwarding, invalid and excess requests, cancellation during process cleanup, abandoned starts, timeout propagation, and launch-config serialization.

GitOrigin-RevId: 6ee2a0139e0f8170cf1a5cdd646bd811b7d3c1f6
2026-07-25 16:03:00 +00:00
rphilizaire-openai
c3e926e61c Trace remote exec-server connection setup (#35275)
## What changed

- Preserve the current tracing span and subscriber when lazy remote environment
  startup runs in a background task.
- Add spans for the remote connection, environment registry, Noise, and
  rendezvous WebSocket stages, including relevant environment and registration
  identifiers.
- Propagate the current trace context to environment registry connection and
  harness-key validation requests.

GitOrigin-RevId: 0044c3eb94d59c6a6b92752d3d372195be4adb81
2026-07-25 01:39:17 +00:00
Celia Chen
09241ae4db Decouple exec-server HTTP from reqwest types (#35059)
## What changed

- Rename `ReqwestHttpClient` to `RouteAwareHttpClient` to reflect that delegated HTTP uses Codex's shared route-aware transport.
- Use `codex_http_client` response and error types plus transport-neutral `http` and `url` types, removing the exec server's direct `reqwest` dependency.

## Testing

- Cover fragment stripping and Unicode hostname normalization for delegated HTTP requests.

GitOrigin-RevId: 8b0fc60a76004feb57198bfb4afb1371c9ceb1bd
2026-07-24 00:53:05 +00:00
Celia Chen
94ebae725e Route exec-server WebSockets through configured proxies (#35056)
## Why

Remote environment connections need to honor Codex's effective outbound proxy policy, including when a rendezvous connection reconnects.

## What changed

- Pass the configured `HttpClientFactory` into remote environment transports and use `WebSocketConnector` for exec-server and rendezvous WebSockets.
- Resolve proxy routes asynchronously so these connections can use the configured system proxy.
- Add connector options that preserve Tungstenite's default TLS behavior and enable `TCP_NODELAY` for latency-sensitive rendezvous traffic.

## Testing

- Verify prepared remote environments connect through a configured system proxy.
- Verify initial and reconnected encrypted relay peers use the system proxy.
- Cover default TLS selection and opt-in `TCP_NODELAY` behavior in the WebSocket client.

GitOrigin-RevId: 8a8da2116e37cb3a891269d0c0b037986fecdd3c
2026-07-24 00:16:42 +00:00
iceweasel-oai
b115de97d7 Preserve Windows sandbox proxy settings in guardian sessions (#35036)
## Why

Guardian review commands can run without the proxy-port environment used to
configure the parent session. Reconciling persistent Windows sandbox settings
for those commands can discard the parent's proxy configuration.

## What changed

- Add a session-level Windows sandbox proxy-settings mode and use `Preserve`
  for guardian review sessions while keeping `Reconcile` as the default.
- Carry the mode through unified exec and the exec-server sandbox context to
  Windows process launches.
- In preserve mode, reuse the proxy settings recorded by the existing sandbox
  setup marker.

## Testing

Added coverage for guardian session configuration, exec-server transport, and
preserving an existing setup marker when proxy-port environment variables are
absent.

GitOrigin-RevId: 214655c6e6d97360906431773dc0de0fdda2db6e
2026-07-23 23:35:29 +00:00
Celia Chen
d45055ae58 Route environment registry requests through the shared HTTP client (#35034)
## Why

Noise environment registry requests need to follow the exec server's effective
outbound proxy policy without exposing registry URLs or response headers in HTTP
diagnostics.

## What changed

- Build the registry client from the supplied `HttpClientFactory` and use a
  route-aware API client with redirects and request logging disabled.
- Defer construction of the Noise connection provider until the outbound HTTP
  policy is available.
- Map route-aware request failures into registry errors while retaining timeout
  detection across response body reads.

## Testing

Add coverage for system-proxy routing, sensitive registry metadata redaction,
stalled response-body timeouts, and prepared Noise configuration validation.

GitOrigin-RevId: d312dfe037f72732085bf38109af44df76ed0b53
2026-07-23 23:25:53 +00:00
Celia Chen
1ee8f49175 Route exec-server HTTP through configured proxy policy (#35023)
## Why

Delegated HTTP requests need to honor the same outbound proxy policy as the
Codex process that starts the exec server.

## What changed

- Pass the configured `HttpClientFactory` through local and remote exec-server
  startup and use route-aware client pools for delegated HTTP and local MCP
  requests.
- Preserve per-request timeouts and follow-or-stop redirect behavior while
  keeping request URLs and sensitive response headers out of diagnostics.

## Testing

- Cover configured system-proxy routing across the exec-server transport.
- Cover both redirect policies and verify that success and failure logs do not
  expose request or response secrets.

GitOrigin-RevId: 4af6aec1d265c4db62dfcb6e1fb076fb31736137
2026-07-23 22:39:28 +00:00
Celia Chen
265cd2e100 Initialize execution environments with the final HTTP policy (#34995)
## Why

The TUI must inspect the default execution environment before loading its final
configuration. Initializing the environment manager at that point can give
startup services the bootstrap HTTP policy instead of the effective policy after
managed requirements are applied.

## What changed

- Split environment discovery from manager construction so callers can inspect
  the default environment without starting remote connections.
- Build the environment manager after final configuration loading and pass its
  resolved `HttpClientFactory` through all construction paths.
- Add shared test support for managers that use the legacy default HTTP policy.

## Testing

- Cover connection-free environment discovery and explicit HTTP policy
  propagation.
- Verify TUI startup services use the final managed `respect_system_proxy` value.

GitOrigin-RevId: 928fa31e6b4bcfbe1a121cade2f351427fdfa0f4
2026-07-23 19:24:41 +00:00
jif
44436fd075 Reject dynamic environments named local (#34784)
## Why

`local` identifies the environment owned by `EnvironmentManager`, but dynamic
remote environment registration previously allowed that ID to be replaced.

## What changed

Reject `local` in the shared environment ID validator used by dynamic remote
environment registration.

GitOrigin-RevId: 2622551f78989c16c432e6dd3d7da9bd8b0527c5
2026-07-22 16:22:34 +00:00
viyatb-oai
32f4687b8c Enable exec-server network policy callbacks (#34770)
## What changed

- Let `exec-server` issue JSON-RPC requests to its client and correlate responses, with bounded concurrency, timeouts, and connection cleanup.
- When `request_policy_decisions` is enabled, forward proxy policy requests with the process ID and network destination, then apply the client's allow, deny, or ask decision.
- Fail closed for invalid inputs, unavailable or malformed responses, timeouts, process shutdown, and disconnects. Keep callbacks active while inherited process streams keep the proxy alive.

## Testing

- Cover response correlation, request limits, timeouts, disconnect cleanup, validation boundaries, decision forwarding, and process lifecycle behavior.

GitOrigin-RevId: 60e658102c50036589ca9de844760a8bffaa53b0
2026-07-22 15:22:21 +00:00
viyatb-oai
c5eb33aed1 Harden managed proxy setup for sandboxed executions (#34641)
## What changed

- Make the generated Linux proxy socket directory readable inside restricted
  `bubblewrap` sandboxes so the proxy bridge can connect.
- Route `WS_PROXY` and `WSS_PROXY` through the Linux managed proxy bridge.
- Remove inherited proxy attribution tokens from unscoped executions while
  continuing to replace them with the current token for scoped executions.

## Testing

- Exercise `WSS_PROXY` routing with a minimal filesystem policy.
- Cover attribution-token removal and replacement during environment setup.

GitOrigin-RevId: 2f14ddb26baab0147a786bc354d93eb5aad059f3
2026-07-22 00:31:30 +00:00
viyatb-oai
d838ea0f64 Add exec-server network policy callback types (#34620)
## What changed

- Define the `network/policyRequest` RPC payloads for associating a network request with a process and returning an `allow`, `deny`, or `ask` decision.
- Cover HTTP, HTTPS CONNECT, and SOCKS5 TCP/UDP requests, with shared size limits for callback fields.
- Add the backward-compatible `requestPolicyDecisions` remote proxy option so executors can opt in to sending policy decisions to clients.
- Re-export the new protocol types from `codex-exec-server`.

## Testing

- Verify the request and decision JSON shapes.
- Verify that the proxy opt-in is omitted and defaults to `false` for legacy configurations, while an enabled value round-trips.

GitOrigin-RevId: 65b0a0723d1abb6e4f504a41d3a0dd41f5f21bf7
2026-07-21 21:54:31 +00:00
Adam Perry @ OpenAI
ad020f29ae Initialize missing-path behavior in exec-server sandbox test (#34615)
## What changed

Set `missing_path_behavior` to `None` for the minimal and project-root
filesystem entries in the custom-`arg0` exec-server test.

GitOrigin-RevId: c1d8df7b5ca8dcf6c89c314b92747bfeae80286e
2026-07-21 21:10:14 +00:00
iceweasel-oai
999a715089 Route Windows sandbox proxy traffic by restricting SID (#34613)
## Why

Elevated Windows sandboxes need stable managed-proxy ports while preserving the network policy and environment attribution of each sandboxed process.

## What changed

- Keep shared HTTP and SOCKS5 loopback ingress listeners alive across managed-proxy instances.
- Add a per-route restricting SID to elevated sandbox tokens and dispatch incoming connections to the matching proxy policy after attributing the client process.
- Reject connections without exactly one registered route, remove routes when their proxy handle is dropped, and keep unsandboxed Windows launches off the managed ingress.
- Provision the elevated sandbox with the configured proxy ports and local-binding setting, honoring the selected profile and CLI overrides.

## Testing

- Add Windows unit tests for TCP ownership attribution, route selection, restricting-token propagation, and setup settings.
- Add an end-to-end Windows test covering stable ports, isolated environment policies, HTTP and SOCKS5 routing, and route teardown.

GitOrigin-RevId: 783fac6e0f904dc9bb1955b75d4a5895e8bb9690
2026-07-21 21:06:04 +00:00
iceweasel-oai
87f71e35b8 Skip missing paths in filesystem sandbox entries (#34598)
## Why

Default read-only protections for project metadata should apply when paths such
as `.git`, `.agents`, and `.codex` exist, without causing sandbox setup to
materialize missing paths as ACL targets.

## What changed

- Add an optional `missing_path_behavior` to filesystem sandbox entries and
  mark default project-metadata protections with `skip`.
- Preserve the behavior through permission transforms and exec/MCP protocol
  serialization while keeping existing path wire variants stable.
- Ignore skip-missing entries when projecting configuration or Windows sandbox
  overrides, while retaining explicit metadata carveouts.

## Testing

- Cover protocol round trips for path and special-path entries.
- Verify default metadata protections and Windows explicit carveout handling.

GitOrigin-RevId: 6df13dadacdd131c44aab9f15a967c81051355c1
2026-07-21 19:17:18 +00:00
jif
2d5c259202 Size Noise handshake buffers to their messages (#34544)
## What changed

Allocate Noise handshake input and output buffers from the actual message size
and the handshake-reported overhead instead of placing maximum-size arrays on
the stack. Truncate variable-length buffers to the bytes produced or read.

GitOrigin-RevId: d4776079ad8116b3cfe1ceaa37c04509d31d1436
2026-07-21 13:39:33 +00:00
jif
4f1992732c Preserve custom arg0 for sandboxed exec-server processes (#34497)
## Why

Sandbox wrappers replaced the process launch command and did not carry an
`ExecParams.arg0` override through to the inner process.

## What changed

- Route sandboxed Unix launches with a custom `arg0` through a helper mode that
  re-execs the requested program with the override.
- Expose the helper executable to the filesystem sandbox and dispatch its mode
  from Codex and exec-server test binaries.

## Testing

Add coverage for the prepared sandbox command and an end-to-end remote process
that verifies both the custom `arg0` and filesystem restrictions.

GitOrigin-RevId: c9f8eef3906d184e670184c2eeed250d5895a9ca
2026-07-21 08:59:41 +00:00
iceweasel-oai
35c2278dd5 Support Windows sandboxing in the exec server (#34423)
## Why

Sandboxed process launch through the exec server was unsupported on Windows.

## What changed

- Add a shared native process launcher that selects the Windows sandbox session backend when required and otherwise launches PTY or pipe-based processes.
- Pass Windows permission, workspace-root, proxy, filesystem-override, and desktop settings through exec-server process preparation.
- Preserve piped stdin and inherited file descriptors across the shared launch path.

## Testing

- Verify that a remote Windows sandbox process accepts stdin while blocking a write under a read-only policy.
- Enable workspace-root sandbox tests on native Windows while continuing to skip environments that cannot emulate Windows sandbox semantics.

GitOrigin-RevId: 6719ccbe87ad0d5a57af56503e3f23f894c60a71
2026-07-20 22:35:14 +00:00
Charlie Marsh
221a341029 Remove unused Rust helpers (#34345)
## What changed

- Remove unused APIs and test helpers across the Rust workspace.
- Route Linux sandbox test setup through the shared exec request path.
- Drop dependencies that became unnecessary after the cleanup.

GitOrigin-RevId: 48fd9b2b1575bad7add0819786b4d8ec88d8486f
2026-07-20 13:45:29 +00:00
viyatb-oai
9f6c29e281 Launch managed network proxies on remote executors (#33906)
## Why

Remote executions need managed-network proxy listeners in the executor so their
loopback proxy addresses are reachable by the launched process.

## What changed

- Add a capability-gated exec-server protocol field for executor-local proxy
  launch configuration, including network policy, audit metadata, and execution
  attribution.
- Start the proxy while preparing a remote process, replace inherited proxy
  environment variables with its local addresses, and derive the sandbox
  context from its listeners.
- Keep the proxy alive until inherited output streams close, then shut it down.
- Reject unsupported remote settings such as MITM and credential injection.

## Testing

- Cover configuration round trips and rejection of unsupported settings.
- Verify executor-local startup, blocked-domain enforcement, protocol
  compatibility, and proxy lifetime through process closure.

GitOrigin-RevId: c984f54e3e600aa9ebcbf8cf4574046e2c199d11
2026-07-17 21:20:14 +00:00
Bryan Ashley
08e30a2e4e Add batched executor capability discovery (#33852)
## Why

Selected capability roots can contribute plugins, MCP servers, connectors, and
skills. Discovering each contribution separately requires repeated access to the
executor filesystem.

## What changed

- Add the `capabilityRoots/discoverV1` exec-server RPC to scan selected roots and
  materialize recognized plugin manifests, configuration files, skill
  instructions, and skill metadata in one bounded request.
- Add the opt-in `executor_capability_discovery` feature, with a thread-scoped
  cache and per-step snapshot shared by MCP and skill discovery.
- Parse MCP, connector, and skill contributions from the materialized snapshot,
  including serving cached skill instructions without another filesystem read.

## Testing

- Cover discovery limits, manifest precedence, root-local failures, cache reuse,
  plugin contributions, and parity with the existing environment skill loader.

GitOrigin-RevId: f98fd2321cafb58c596db02da1f83c09d8eb375d
2026-07-17 15:45:48 +00:00
jif
79177c3e20 Propagate deferred environment capability roots to MCP (#33427)
## What changed

- Let deferred environments provide selected capability roots with their ready signal.
- Validate that those roots have unique, non-empty IDs, belong to the registering environment, and stay within the root limit.
- Include roots from ready turn environments when resolving MCP contributions, and refresh the MCP runtime when the selected root set changes.
- Expose the exact ready root set to MCP contributors so executor plugins become available with their environment.

## Testing

- Cover ready-root propagation, validation failures, replacement isolation, reconnection, and MCP plugin availability refresh.

GitOrigin-RevId: ec3498aab1164824025094e96a9b1063b7b731ad
2026-07-15 21:07:43 +00:00
sayan-oai
3afbd8dd45 Report selected environment connection transitions (#33251)
## What changed

- Track connected and disconnected states across initial remote exec-server connections and reconnection attempts.
- Emit experimental `thread/environment/connected` and `thread/environment/disconnected` app-server notifications for each thread selecting the environment. Each payload identifies the thread and environment; current state is not replayed when a thread starts.
- Stop forwarding connection events when an environment selection is removed or replaced.

## Testing

- Cover connection, disconnection, reconnection, shared-environment notifications, and replacement of a selected environment.

GitOrigin-RevId: 5dd767372363c4a2a8319fc16164be117d5bd20c
2026-07-15 05:21:13 +00:00
rphilizaire-openai
35b33e4304 Instrument environment and plugin resolution paths (#33223)
## What changed

Add named tracing spans around step environment snapshots, capability and
executor plugin resolution, `AGENTS.md` refreshes, and MCP runtime projection
and refresh operations. Skip recording function arguments to keep these spans
focused on timing and execution flow.

GitOrigin-RevId: 73a452ef1b5da6f7f2b00d24421a815c5d514eaf
2026-07-15 03:27:16 +00:00
TAFOYA-OAI
32cd5d4eab Defer Noise environment connections until registration (#33166)
## What changed

- Replace pending WebSocket URL registration with deferred Noise environment
  registration that gates connection attempts on an explicit readiness signal.
- Reuse the Noise rendezvous transport after readiness so reconnects request a
  fresh connection bundle.
- Preserve terminal errors for failed or dropped registrations and keep late
  completion isolated from replacement environments.

## Testing

- Add coverage for readiness gating, registration failure and replacement,
  eager Noise connections, and reconnection through a fresh rendezvous bundle.

GitOrigin-RevId: 83e23fa03a02e3b2bdf1a83fe26d7ac461f55cf3
2026-07-14 21:07:27 +00:00
jif
325cf16194 Bound exec-server JSON-RPC decoding complexity (#33013)
## Why

Compact JSON arrays can expand into millions of heap values during decoding, and duplicate object keys make a message ambiguous.

## What changed

- Limit exec-server JSON-RPC messages to 256K JSON values and reject duplicate object keys.
- Cap `fs/read_directory` results and retained `process/read` output at 50,000 entries or chunks so locally produced responses remain within the decoder budget.

## Testing

Add coverage for all JSON-RPC variants, large scalar payloads, duplicate keys, compact array amplification, and retained process output at the chunk limit.

GitOrigin-RevId: e31d1f25ab0a7e2272015c98174fd2b7cdd669d7
2026-07-14 09:15:27 +00:00
Adam Perry @ OpenAI
f96cf4d1c3 Expose environment status through app-server (#32920)
## What changed

- Add the experimental `environment/status` request for inspecting a configured environment without starting or reconnecting it.
- Report `ready`, `pending`, `disconnected`, or `unknown`, including error details for disconnected and unknown environments.
- Probe ready remote environments over their existing exec-server connection.

## Testing

- Add an app-server integration test covering local and remote ready, pending, disconnected, and unknown environments.

GitOrigin-RevId: 397bba603aa9e0b59008ee6cb7cbde46e357652c
2026-07-14 01:49:16 +00:00
Adam Perry @ OpenAI
75470c3e2f Add exec-server environment status checks (#32899)
## What changed

- Add the initialized `environment/status` RPC, which reports `ready` when the exec server can handle requests.
- Expose environment IDs and `ready`, `pending`, or `disconnected` status through `EnvironmentManager` and `Environment`.
- Keep status checks non-mutating: they do not start or recover lazy remote environments, and probe only an existing connection.

## Testing

- Cover the status RPC over WebSocket and the in-process request processor.
- Verify that checking an unstarted stdio environment leaves it pending and that failed connections report as disconnected.

GitOrigin-RevId: 22febeb6a3457849292128a8991c6400c22b3fd8
2026-07-13 23:43:15 +00:00
pakrym-oai
4472698728 Support pending remote environment registration (#32231)
## Why

Remote environment provisioning can finish after a thread starts, before an
exec-server WebSocket URL is available.

## What changed

- Add `EnvironmentManager::register_pending_environment` and a one-shot
  `PendingEnvironmentRegistration` handle that resolves to either a validated
  WebSocket URL or a terminal provisioning error.
- Let lazy remote exec-server clients wait for that result, while preserving
  reconnection behavior after a successful registration.
- Keep replacement registrations isolated so completing an older handle does
  not resolve the current environment with the same ID.

## Testing

Add coverage for successful connection and reconnection, provisioning and
dropped-registration failures, invalid URLs, replacement isolation, and the
deferred-executor startup flow.

GitOrigin-RevId: 5c05be2b72291b77a1f71176d7075b1ad63332a5
2026-07-10 18:08:35 +00:00
pakrym-oai
c8dc8e5fd5 Propagate workspace roots to exec-server sandboxes (#32214)
## What changed

- Pass configured workspace roots from core to the exec server so filesystem and process sandbox permissions are materialized against the intended roots.
- Preserve an explicitly empty workspace-root list instead of treating the sandbox working directory as an implicit root.
- Initialize filesystem sandbox contexts with their working directory as the default workspace root.

## Testing

- Add end-to-end coverage for patch and command writes inside and outside workspace roots.
- Verify remote filesystem and process sandboxes do not grant access through an empty workspace-root list.

GitOrigin-RevId: 6684c8f7de50970b45a29e2a6323df954c96f9f6
2026-07-10 16:55:01 +00:00
jif
54c44b9ed4 Propagate tracing subscribers to exec start tasks (#32135)
## Why

The process-start background task can move to a Tokio worker thread, where the
caller's thread-local tracing subscriber is otherwise unavailable. This can
break propagation of the caller's trace context to the exec-server request.

## What changed

- Attach both the current span and current tracing subscriber to spawned
  process-start tasks.
- Run exec-server Bazel unit tests serially because their tracing setup uses
  process-global state.

## Testing

Run the trace-context regression test on a multi-thread Tokio runtime so it
exercises propagation across the background task.

GitOrigin-RevId: a586eefd6983916d670fc5a90f0decd0468d273b
2026-07-10 11:00:12 +00:00
jif
707bd3cc18 Test stdio JSON-RPC size limits with LF and CRLF (#32134)
## What changed

Update the exact-size-limit stdio connection test to exercise both `\n` and
`\r\n` line endings, sizing the duplex buffer for each delimiter.

GitOrigin-RevId: 949e28bef3f1a34d379f3623f6aa7d99c0ecea80
2026-07-10 10:52:13 +00:00
jif
c4c21b68a8 Bound exec-server stdio JSON-RPC messages (#32123)
## Why

Newline-delimited stdio input could buffer an unterminated JSON-RPC message without a per-message limit. Apply the same 64 MiB ceiling used by the other exec-server transports.

## What changed

- Read stdio messages with bounded lookahead and disconnect when a message exceeds the limit.
- Preserve LF and CRLF framing, including messages whose payload is exactly at the limit.

## Testing

Added tests for accepting a limit-sized CRLF message and rejecting an unterminated overlong message.

GitOrigin-RevId: 73eaf883324a7777960725037cc5b9c34720084f
2026-07-10 10:21:26 +00:00
jif
c094a58c9f Test the shared exec-server HTTP response byte budget (#32122)
## Testing

Add a concurrent-stream regression test that fills the connection-wide queued
body budget across two HTTP responses. Verify that the overflowing stream
reports the byte-budget error while the other stream still drains successfully.

GitOrigin-RevId: 65efacb15f1368b1de3b98e7ef65ebca66a3840b
2026-07-10 10:16:47 +00:00
jif
4ba2815014 Bound streamed exec-server HTTP response bodies (#32112)
## Why

Frame-count backpressure does not bound the amount of executor-controlled data
retained in streamed HTTP response queues, and a single body delta could exceed
the intended wire size.

## What changed

- Limit each decoded `http/request/bodyDelta` payload to 1 MiB, split locally
  produced response chunks at that boundary, and reject oversized incoming
  deltas.
- Apply a shared 16 MiB byte budget across queued HTTP response streams. Release
  capacity as deltas are consumed and fail a stream when the budget is
  exhausted.

## Testing

Added coverage for rejecting an oversized delta and for failing a stream after
its queued deltas exhaust the shared byte budget.

GitOrigin-RevId: be9205fef44b0ec96b84b31a8e059b1cb9cd3f3b
2026-07-10 09:56:03 +00:00
pakrym-oai
1f0566d3f5 exec-server: expose process helper to outer sandbox (#31937)
## Why

Sandboxed exec-server process requests can use a restricted filesystem
profile that does not expose the exec-server binary. On Linux, the outer
bubblewrap stage re-enters that binary with the `codex-linux-sandbox`
argv0 to install seccomp, so hiding the binary prevents the requested
process from starting.

## What changed

- add the configured `codex_self_exe` to the process permission profile
before constructing the outer platform sandbox
- add a Linux exec-server integration test that starts a real remote
process with restricted reads and verifies it can read an allowed
workspace file

## Test plan

- `just test -p codex-exec-server process_sandbox`
- `just test -p codex-exec-server --test exec_process
remote_process_keeps_sandbox_helper_visible_with_restricted_reads`
2026-07-09 18:28:55 -07:00
pakrym-oai
ac3da4fb1a exec-server: materialize filesystem workspace roots (#31892)
Filesystem helper requests currently turn symbolic `:workspace_roots`
permissions into a sandbox policy before applying the workspace roots
from the filesystem sandbox context. This can accidentally broaden
filesystem access to the cwd instead of limiting it to the selected
workspace roots.

Materialize project-root permissions using the context workspace roots
before deriving the filesystem sandbox policy. The same converted roots
are then reused when constructing the sandbox command, keeping policy
enforcement and process setup aligned.

Adds a remote filesystem integration test that verifies a file inside
the selected workspace root is readable while a sibling under the cwd is
denied.

## Validation

- `just test -p codex-exec-server
remote_read_file_materializes_environment_workspace_roots` (macOS,
outside the outer Seatbelt sandbox)
2026-07-09 16:20:06 -07:00
jif
13ba8058f2 Resolve selected capability roots without starting executors (#31581)
## Why

A thread can select skill roots that live in an executor environment.
`skills/list` needs a passive snapshot of the roots that are usable now:
it must not start an executor, wait for recovery, or reconnect a failed
environment.

The initial implementation checked the immutable first startup result.
After a successful connection later entered recovery or failed, that
result still looked successful. A read-only catalog request could then
wait for recovery or trigger a new connection while reading the
filesystem.

## What

- inspect readiness from the current exec-server connection state
- return roots only while their environment can serve a request
immediately
- omit environments that have not started, are connecting, or are
recovering
- return warnings for missing environments and terminal connection
failures
- add a fail-fast filesystem view that never starts, waits for, or
reconnects an environment
- expose the passive selected-root snapshot through `CodexThread`

## Behavior

- Local and currently connected environments are ready.
- Starting and recovering environments are omitted without a warning so
callers can retry later.
- Missing and terminally failed environments are omitted with a warning.
- A disconnect between readiness inspection and filesystem access fails
promptly instead of crossing into the normal recovery path.
- Normal model-turn and execution paths keep their existing reconnect
behavior.

## Design

The recovery policy is private to the exec-server client. Callers choose
the explicit fail-fast filesystem method; the existing client and
filesystem APIs remain reconnecting. This keeps the passive contract at
the transport boundary instead of plumbing timeout or retry flags
through the skills stack.

## Coverage

- a lazy stdio environment stays unstarted during passive inspection
- missing and terminally failed environments surface warnings
- a real websocket disconnect proves current readiness drops, a
previously acquired fail-fast filesystem handle returns promptly, and
readiness returns after recovery

## Scope

This PR only provides passive readiness and fail-fast filesystem
primitives. It does not add app-server API fields or notifications.

## Stack

- #31582 uses these primitives for experimental thread-scoped
`skills/list`.
- #30228 adds targeted invalidation notifications.
2026-07-09 11:17:05 +01:00
jif
e398a99edf Bound exec-server process event reordering (#31576)
## Why

A malicious exec-server can send out-of-order process notifications
faster than missing events arrive. The client retained every future
event in a per-session reorder map, so many tiny events or a few large
output chunks could grow orchestrator memory without bound.

## What changed

- cap each session's pending process-event reorder state at 256 events
and 1 MiB
- reject individual process events larger than 1 MiB
- release byte accounting as events publish or a session fails
- fail and detach only the affected process session when a limit is
exceeded
- let the next expected event drain a full buffer, so the limits apply
to retained future state
- apply the same bounded insertion path during reconnect recovery while
safely handling dense tail output, missing exit events, newer live
notifications, and conflicting `Closed` sequences

Sequence distance is intentionally not capped because it does not affect
allocation; the event-count and byte limits are the resource-exhaustion
boundary.

## Tests

Focused exec-server tests cover oversized output, full count/byte
buffers, gap-closing delivery, dense recovery interleaved with newer
notifications, missing exit reconstruction, and conflicting terminal
sequences.

## Scope

This is limited to per-session process-event reorder state. It does not
introduce a scheduler, quota framework, or new public configuration.
2026-07-09 09:56:14 +01:00
jif
a14b4c2d7f Bound exec-server pending RPCs (#31578)
## Why

An untrusted exec-server can stop reading requests, never answer them,
or send guessed responses before queued requests are written. Without
client-side admission, the orchestrator can retain unbounded RPC call
futures and request payloads.

We need a hard bound without adding a blanket timeout, because
individual operations already own their timeout and cleanup semantics.

## What changed

- hold one of 64 shared admission permits for the full lifetime of each
regular RPC call, so an early response cannot free capacity while the
call remains queued
- allow `process/terminate` and `fs/close` to use one additional cleanup
permit
- close the transport and fail pending calls if the cleanup permit is
also stuck, allowing teardown or recovery to release remote resources
- leave the wire format and existing timeout behavior unchanged

## Tests

- `rpc_client_call_has_no_implicit_deadline` verifies that ordinary
calls remain untimed
- `rpc_client_bounds_in_flight_calls_and_preserves_cleanup` covers the
regular-call cap, guessed early responses, cleanup admission, and the
cleanup circuit breaker
2026-07-08 19:46:18 +01:00
jif
a52b35fcf6 fs: support pruning hidden directories during walks (#31570)
Why

Filtering hidden directories after a walk is too late: their descendants
consume traversal limits, and canonical directory deduplication can let
a hidden path claim a target before a visible symlink reaches it.

What this changes

- Add an optional pruneHiddenDirectories walk option. It defaults to
false and is omitted from the wire when disabled.
- Return hidden directory entries, but do not traverse them or add their
canonical identities to the visited set.
- Cover the visible-symlink-to-hidden-directory case through both local
and remote filesystem implementations.

This is the small filesystem prerequisite for #31566. Skill-specific
behavior remains in that PR.
2026-07-08 13:49:05 +01:00
Adam Perry @ OpenAI
f158b31db5 test: generalize exec-server fixture (#31422)
## Why

Remote-executor integration tests need one host-agnostic exec-server
fixture target instead of a Windows-only wrapper.

## What

- rename the testing binary target to exec-server
- make the fixture source and target host-agnostic
- update Windows remote-executor test wiring to use the shared target

## Validation

- bazel build //codex-rs/exec-server/testing:exec-server
- bazel cquery --config=ci-windows-cross
'set(//codex-rs/exec-server/testing:exec-server
//codex-rs/core/tests/remote_env_windows:smoke-test)'

## Stack

1. [#31422 test: generalize exec-server
fixture](https://github.com/openai/codex/pull/31422)
2. [#31425 test: add TestAppServer
builder](https://github.com/openai/codex/pull/31425)
3. [#31427 test: add delayed exec-server
transport](https://github.com/openai/codex/pull/31427)
4. [#31295 bench: add cold skill load
macrobenchmark](https://github.com/openai/codex/pull/31295)
5. [#31428 bench: add e2e benchmark
entrypoints](https://github.com/openai/codex/pull/31428)
6. [#31429 ci: smoke Bazel e2e
benchmarks](https://github.com/openai/codex/pull/31429)
2026-07-07 11:21:56 -07:00
Michael Bolin
9365b08467 exec-server: use virtual time in Noise relay test (#31344)
## Why

`fragmented_writes_yield_to_keepalive_and_queued_pong` deliberately
blocks WebSocket writes while exercising keepalive and queued-Pong
scheduling. It previously advanced those states with wall-clock sleeps.
Under a sufficiently delayed CI worker, those sleeps and scheduling gaps
could consume the test-only 100 ms Pong-watchdog budget, causing the
relay to exit and the next write-permit send to fail with
`TrySendError::Disconnected`.

The failure was therefore a timing flake in the harness test, not
evidence that the production relay mishandled a Pong.

## What changed

- Run this test with Tokio time paused.
- Advance the virtual clock through its two keepalive transitions
instead of sleeping in wall-clock time.
- Enable Tokio's `test-util` feature only for `codex-exec-server` dev
dependencies.

No production code or timeout values change.

## Review guide

The behavioral change is confined to `noise_relay/harness_tests.rs`; the
`Cargo.toml` change only exposes Tokio's paused-clock test APIs.

## Validation

- `just test -p codex-exec-server
fragmented_writes_yield_to_keepalive_and_queued_pong`
- `just fix -p codex-exec-server`
- `just bazel-lock-update` (no lockfile changes)
2026-07-06 20:25:33 -07:00
Michael Bolin
641aa1b619 Migrate direct HTTP consumers to codex-http-client (#31331)
## Why

#31323 introduces `codex-http-client` and leaves compatibility
re-exports in `codex-client`. Low-level HTTP consumers should depend on
the crate that now owns those APIs rather than continuing through the
transitional compatibility layer.

This stacked follow-up makes that ownership explicit and moves the
repository toward enforcing the abstraction without mixing call-site
churn into the extraction itself.

## What changed

- Switched `codex-backend-client`, `codex-cloud-tasks`,
`codex-exec-server`, `codex-login`, and `codex-model-provider` from
`codex-client` to `codex-http-client` where they only use low-level HTTP
APIs.
- Added the direct dependency to `codex-api` for its custom-CA request
and websocket paths while retaining `codex-client` for higher-level
retry and transport policy.
- Updated imports and normalized login's internal client type name from
`CodexHttpClient` to `HttpClient`, while preserving its existing
`CodexRequestBuilder` re-export.
- Updated `Cargo.lock` to reflect the new direct dependency edges.

## Review guide

This PR is intentionally mechanical: 20 files and 92 changed lines, with
no runtime logic changes. The largest diff is
`codex-rs/login/src/auth/default_client.rs`, where the only
semantic-looking changes are type and import renames. The remaining
source changes replace `codex_client` import paths with
`codex_http_client`; the manifest and lockfile changes mirror those
imports.

## Test plan

- Compile-checked `codex-api`, `codex-backend-client`,
`codex-cloud-tasks`, `codex-exec-server`, `codex-login`, and
`codex-model-provider` together.
2026-07-07 01:34:36 +00:00
Adam Perry @ OpenAI
a86d525e4d core: trace executor skill discovery (#30318)
## Why

Make it easier to measure the performance of different parts of skill
loading.

## What

- Add spans for step-context capture, world-state construction, executor
catalog snapshot/root loading, and environment skill loading.
- Record the discovered environment skill count.
- Trace outbound exec-server requests with client kind and RPC method
fields.
- Update trace propagation tests to assert that requests keep the parent
trace id while creating their own child span.
2026-07-06 18:44:08 +00:00
richardopenai
042e61726d [codex] bound Rendezvous WebSocket liveness (#30643)
## Summary

- require a Pong within 60 seconds for established Noise Rendezvous
WebSockets on both the harness and executor
- bound steady-state WebSocket writes and harness event delivery so
backpressure cannot mask the deadline
- classify executor disconnects with bounded reasons and feed them into
the existing reconnect metric and structured log
- cover silent peers, responsive peers, continuous non-Pong traffic, and
local application backpressure

## Why

The existing periodic Pings did not track Pongs, so a half-open or
blackholed connection could remain stuck until the operating system's
TCP timeout. This adds the smallest explicit liveness contract without
new spans, RTT histograms, feature flags, or TCP diagnostics.

## Testing

- `just test -p codex-exec-server` on devbox `richard-6` — 300 passed, 2
skipped
- `just fix -p codex-exec-server`
- `just fmt`
- independent correctness, performance/security, and YAGNI reviews — no
findings
2026-07-01 14:15:34 -07:00
richardopenai
cfead68e5d [codex] disable Nagle on Rendezvous WebSockets (#30269)
## Summary

Disable Nagle unconditionally for both exec-server Rendezvous WebSocket
connections.

- pass `disable_nagle=true` at the executor and harness connection call
sites
- keep the existing signed URL, protocol, and connection flow unchanged
- add no feature flag, rollout schema, path variant, or
experiment-specific telemetry

The companion internal PR enables `TCP_NODELAY` on accepted Rendezvous
sockets: https://github.com/openai/openai/pull/1082463

## Why

Rendezvous carries small, latency-sensitive relay and JSON-RPC frames.
Three staging runs of 30 steady-state `process/read` calls per
configuration measured p50 improving from 139.1 ms to 81.5 ms and p95
from 162.0 ms to 95.8 ms with Nagle disabled.

The expected packet overhead is small at the current connection scale.
We will use existing latency, error, packet, and CPU monitoring and
revert normally if production regresses.

## Rollout and rollback

The client and accepted-socket changes can deploy independently. New
connections receive the setting as each side deploys. Rollback is a
normal code revert; there is no persisted assignment or gate state to
unwind.

## Validation

- `just test -p codex-exec-server --lib`: 164 passed
- `just fix -p codex-exec-server`: passed
- `just fmt`: passed
- independent final review found no actionable issue
2026-06-29 19:14:47 -05:00