Commit Graph
201 Commits
Author SHA1 Message Date
jif 3882ced09c Add in-memory shell snapshots to unified exec (#39957)
## What changed

- Add the under-development `shell_snapshot_v2` feature and advertise executor support through environment capabilities.
- Use executor-managed, in-memory snapshots for eligible direct `bash`, `zsh`, and `sh` login commands, while applying the configured shell environment policy and avoiding duplicate automatic startup-file effects.
- Keep file-backed snapshots available for user-shell commands and fall back to the existing execution path when in-memory snapshots are unsupported or inapplicable.

## Testing

- Cover policy filtering, snapshot reuse without snapshot files, automatic startup files, local and remote execution, and legacy user-shell snapshots.

GitOrigin-RevId: 4ad6cdf13824913ac8c393ba38b9844230677579
2026-08-21 15:24:56 +00:00
Tamir Duberstein 748d8ac834 Bound unified exec output delta frames (#39937)
## What changed

- Cap each `ExecCommandOutputDelta` frame at 8 KiB and stop building frames once the per-call delta quota is exhausted.
- Preserve complete UTF-8 scalars across producer chunks while passing malformed or incomplete trailing bytes through without losing them.
- Append all process output to the transcript independently of delta emission.

## Testing

- Cover split multibyte characters, malformed and trailing bytes, frame and event limits, transcript preservation, and interactive PTY output.

GitOrigin-RevId: ef4100e353df01569a8fbae650c70f439920a2fd
2026-08-21 14:04:33 +00:00
jif d0cc662b8c Cache shell snapshots in the exec server (#39756)
## What changed

- Add the `shellSnapshotV2` executor capability and an optional shell snapshot request to `ExecParams`.
- Capture and restore Unix shell state and profile exports from an in-memory, attachment-scoped cache for `bash`, `zsh`, and `sh`.
- Apply environment policies, runtime `PATH` entries, sandbox context, and live managed-proxy settings when preparing restored commands.
- Bound snapshot size, capture time, scope length, and cache capacity, and fall back to the original command when capture fails.

## Testing

- Cover local, remote, TTY, sandboxed, and supported-shell execution, plus environment filtering, proxy handling, in-memory reuse, and capture failure fallback.

GitOrigin-RevId: 624f747972c249c88c6f10f42cf0af97b75b5541
2026-08-20 17:39:06 +00:00
Tamir Duberstein 4a942885c8 Reduce unified exec output buffer allocations (#39712)
## What changed

- Accept borrowed byte slices when appending output chunks, avoiding clones before
  forwarding those chunks to streaming consumers.
- Merge drained head-tail buffers by reusing their owned storage while preserving
  the retained prefix, latest suffix, and omitted-byte count.
- Make unified exec output handles capacity-generic so bounded-output tests can
  exercise repeated drains with small buffers.

## Testing

- Update head-tail buffer and process manager tests to cover chunk retention,
  omission accounting, and repeated output drains with the new APIs.

GitOrigin-RevId: caa9813d3c9d9921cff28888263a212def381640
2026-08-20 13:17:20 +00:00
Tamir Duberstein 18937b2265 Use mem::take to drain unified exec output buffers (#39515)
## What changed

Replace the custom `HeadTailBuffer::drain` helper with `std::mem::take` at
the output collection call site. This moves the buffered output out while
resetting the shared buffer to its default empty state.

GitOrigin-RevId: 7a93d1312f2cea0d7913a5264eeb01726b45e0ea
2026-08-19 16:30:10 +00:00
Tamir Duberstein fcdf2b5014 Make head-tail buffer capacity const generic (#39493)
## What changed

- Parameterize `HeadTailBuffer` by a const `MAX_BYTES` value and derive its
  head and tail budgets from that value.
- Keep `UNIFIED_EXEC_OUTPUT_MAX_BYTES` as the default capacity for production
  call sites.
- Update focused buffer tests to select their smaller capacities through the
  type parameter.

GitOrigin-RevId: 106e81a2c2db935f7f77a4f301ef591017958287
2026-08-19 14:49:58 +00:00
jif 7d9990fa30 Bind unified exec approvals to shell executables (#39311)
## Why

An unfamiliar executable can ignore its arguments, so trust in an apparent
inner command must not implicitly trust the executable that runs it.

## What changed

- Evaluate unfamiliar shell executables alongside their parsed commands when
  determining unified exec approval requirements. Inner commands can add
  restrictions, but cannot grant trust to the executable.
- Include the executable in reusable approval keys so approval for one custom
  shell does not apply to another.
- Parse literal PowerShell scripts without launching the requested executable,
  preserving command policy checks before approval.

## Testing

Add cross-platform coverage for spoofed shell paths, allowed and forbidden
inner commands, explicit custom-shell approval, and session approval isolation.

GitOrigin-RevId: 0dd2c7e9a2ac30965ef5fe5de1a8d2968bb5f9d0
2026-08-18 23:00:53 +00:00
Tamir Duberstein 3006151ad4 Simplify unified exec output snapshots (#39259)
## What changed

- Return retained unified-exec output as one contiguous byte buffer.
- Use the buffer directly for sandbox-denial checks, removing intermediate
  chunk aggregation and an unnecessary owned string conversion.
- Update the output-retention test to assert the flattened snapshot.

GitOrigin-RevId: 84eacee0166b86d3cb2d6a37e96125977fad0267
2026-08-18 18:46:44 +00:00
sayan-oai e38290846c Enforce environment-specific command policies (#38942)
## What changed

- Add an optional restrictive execution policy to `EnvironmentConfig` and merge it over the active command policy for shell execution and approval decisions.
- Reject environment policies that contain allow rules, so environment configuration can only tighten command access.
- Include the environment policy fingerprint in cached approval keys so policy changes require fresh session approvals.

## Testing

- Cover restrictions overriding saved prefix approvals, policy changes invalidating session approvals, and environment overlays remaining effective when model-specific prefix filtering is active.

GitOrigin-RevId: e94f877549ed5e0f577b8cb52f3c1872c58f029d
2026-08-17 04:37:50 +00:00
sayan-oai 6c108912ee Honor per-environment shell variable policies (#38902)
## What changed

- Carry `ShellEnvironmentPolicy` in each resolved `EnvironmentConfig` and use
  the selected turn environment's policy for shell commands, user shell tasks,
  and unified exec.
- Infer the policy from the thread configuration when an environment does not
  provide its own resolved configuration.
- Redact the policy from `EnvironmentConfig` debug output because it can contain
  explicit environment variable values.

## Testing

- Verify shell handlers and unified exec filter inherited variables according
  to the selected environment while preserving its explicit overrides.

GitOrigin-RevId: 9f6a52aa4af60e5f17251a0e1b11e1926b779055
2026-08-16 20:35:05 +00:00
Kyle Brown 9579479d28 Collect plugin metrics from remote executors (#38283)
## What changed

- Resolve manifest-declared metric operations against the executor filesystem for remote plugin commands.
- Create the measurement sidecar in an executor-native, owner-private temporary directory, stream its bounded output back for validation, and clean up the directory afterward.
- Extend the exec-server protocol with the executor temporary directory and atomic private-directory creation.

## Testing

- Cover remote unified-exec measurements for foreground and background commands.
- Verify private directories use owner-only permissions on Unix and fail closed on unsupported platforms.

GitOrigin-RevId: dbbd0a84717b91237fc5728e510e18994eb46dd4
2026-08-13 00:39:04 +00:00
Kyle Brown 96e8afbfb8 Track plugin metrics for background unified exec commands (#38276)
## Why

Unified exec can yield while a command is still running. Plugin measurement
collection must remain active until that background command exits, including
when its item completion arrives after the turn has completed.

## What changed

- Keep the plugin metrics sidecar with the stored process and let either the
  exit watcher or a poll that observes completion finalize it exactly once.
- Retain completed turn analytics state while tool items are pending so late
  command completion events can be emitted without duplicating the turn event.

## Testing

- Verify that a background command completed after its turn emits a command
  execution event and does not emit a second turn event.

GitOrigin-RevId: ecf715b3e047aa29ca9a417d12d955257fed8557
2026-08-13 00:04:57 +00:00
Kyle Brown 6e7daed1e9 Collect plugin metrics from unified exec commands (#38253)
## What changed

- Create a metrics sidecar for attributed local plugin commands launched through unified exec, and grant the sandbox access to its output file.
- Publish valid measurements when the initial command exits, while discarding the sidecar when the process remains live for later polling.
- Strip inherited or configured `CODEX_PLUGIN_METRICS_OUTPUT` values so each command can only use its own sidecar.

GitOrigin-RevId: 43e669e4c77171aec8a798de0c12ca13bfb8adee
2026-08-12 21:27:30 +00:00
edwardysun3 44d992c14e Track artifact operations from trusted plugin markers (#38057)
## What changed

- Recognize validated create and edit marker commands from the trusted presentations, documents, spreadsheets, and PDF plugins.
- Emit a `codex_artifact_operation` analytics event plus started-count and expected-output-count metrics when a recognized marker starts through unified exec.
- Attribute remote executor commands only when their plugin cache path and script contents match a trusted local plugin, including primary-runtime plugins.

## Testing

- Cover supported marker combinations and reject mismatched plugins, paths, arguments, counts, and output formats.
- Cover local, relocated remote, and Windows-style plugin attribution, including rejection of modified remote scripts.

GitOrigin-RevId: 475cc1689aa7f371a24dec0a8bdb6f7fd5a2e922
2026-08-11 19:26:41 +00:00
jif 97729885d4 Expose the session ID to shell commands (#37848)
## What changed

- Set `CODEX_SESSION_ID` to the shared root-session ID for shell tool calls,
  unified exec processes, and user shell commands.
- Preserve `CODEX_SESSION_ID` when restoring a shell snapshot, alongside the
  existing runtime-owned environment variables.

GitOrigin-RevId: 5228867d25507105c7edf11607a8298958c3b917
2026-08-10 17:02:19 +00:00
Charlie Marsh c9c6c0daa9 Add a feature flag to preserve apply_patch line endings (#37758)
## What changed

- Add the `apply_patch_preserve_line_endings` feature, disabled by default, to
  preserve CRLF, CR, and mixed line endings when `apply_patch` updates files.
- Apply the feature consistently to built-in patch handling and patches invoked
  through shell, user-shell, unified-exec, and app-server command execution.
- Keep the active feature configuration authoritative over inherited, shell
  snapshot, and client-provided environment values.

## Testing

- Cover line-ending behavior with the feature enabled and disabled for custom
  tool calls, shell heredocs, command execution, and the `apply_patch` CLI.

GitOrigin-RevId: 531a7c66761959c650270559f57941929f03e6c4
2026-08-10 01:44:30 +00:00
sayan-oai 420accf199 Use the step context for command approval prefix rules (#37641)
## What changed

Read `allow_prefix_rules` from the turn attached to the active step context
when selecting the exec policy and constructing unified exec approval requests.

GitOrigin-RevId: d2d26e88b9cefa28bb2df5b9f71aa7b30b7ea331
2026-08-09 00:29:14 +00:00
sayan-oai dd43a9967f Use step environments for Guardian approval reviews (#37618)
## Why

Deferred environments can become ready after a turn starts. Guardian approval
reviews must use the environment selected for the current step so their working
directory and permission context are not taken from the stale turn snapshot.

## What changed

- Carry `StepContext` through tool execution and approval handling.
- Add a Guardian review context that pairs the parent turn with the current
  step's environment snapshot.
- Use that snapshot for Guardian prompts, reviewer sessions, and delegated
  reviewer threads while retaining turn-level fallbacks for callers without a
  step context.

## Testing

Added an integration test that makes a deferred remote environment ready before
an escalated command and verifies Guardian receives that environment's working
directory and denied-read policy instead of the initial local environment's.

GitOrigin-RevId: 779096bc7d0e0c7453a009c02141029714b9b407
2026-08-08 20:07:27 +00:00
cooper-oai c4513cb982 Prevent launch context from reaching child processes (#37607)
## Why

Model-reachable child processes should not inherit Codex launch context.

## What changed

- Treat `OPENAI_FEDERATION_RULE_ID` and `OPENAI_IDENTITY_TOKEN_FILE` as non-inheritable environment variables, with case-insensitive matching.
- Remove them after shell environment policy overrides and before spawning commands across execution, MCP, hooks, Git helpers, and remote helper processes.

## Testing

- Cover inherited and explicitly configured variants, including mixed-case names.
- Verify the variables are absent from real child environments and app-server command and process execution.

GitOrigin-RevId: 2535527893985fef0995617f4c5b2462bea7c136
2026-08-08 16:58:26 +00:00
andrewgu-oai e734a1a5c1 Ignore reusable command approvals for cyber models (#37516)
## What changed

- Filter saved `allow` prefix rules from the execution policy for cyber-specialized models and models listed in `auto_review.ignore_rules`, while preserving prompt, forbidden, network, and host-executable policy entries.
- Apply the filtered policy consistently to permission instructions, shell commands, unified exec, and the zsh-fork backend.
- Limit cyber-model command approvals to one-time decisions without proposing reusable policy amendments. General models continue to honor saved prefixes, including after switching models within a thread.

## Testing

- Add policy-level coverage for filtering allow-prefix rules while retaining restrictive and network rules.
- Add end-to-end coverage across shell and unified exec paths, zsh-fork execution, user and automatic review, and model switching.

GitOrigin-RevId: f3bd3872424291b12354ec415f33986ec369a368
2026-08-08 00:59:39 +00:00
jif 5c44f11064 Consolidate unified exec output state (#37083)
## What changed

Store the output buffer, notifications, closed state, and cancellation token
together in `OutputHandles`. Pass that shared bundle through output collection
and local output tasks instead of cloning and forwarding each field separately.

GitOrigin-RevId: 2f0f66dd419778e0b17ff47323c795f5dc88fe33
2026-08-05 11:21:31 +00:00
sayan-oai bac3ef1d8e Use turn environment permissions for tool execution (#37038)
## What changed

- Derive sandboxing, approval, and permission-profile environment variables from the selected turn environment for shell, unified exec, `apply_patch`, image viewing, and network approval flows.
- Materialize project-root permissions against every workspace root in that environment, allowing `apply_patch` to write to secondary workspace roots while preserving protected paths.
- Carry the selected environment's permission profile into spawned or resumed agents and Guardian permission context.

## Testing

- Add an integration test covering `apply_patch` writes in a secondary workspace root.
- Extend tool, Guardian, and multi-agent tests to distinguish environment permissions from thread-level permissions.

GitOrigin-RevId: e22da8974bcd3de1e724978b1a7790307a51e4bc
2026-08-05 06:38:57 +00:00
jif 6a828ca26f Read approval policy from the current turn configuration (#36912)
## Why

Thread settings can update the approval policy after a turn context is created.
Keeping a separate copy on `TurnContext` could leave tool approval checks using
the previous policy.

## What changed

- Remove the duplicated approval-policy field from `TurnContext`.
- Resolve the policy through the turn's current configuration everywhere it is
  needed, including tool execution, Guardian routing, MCP handling, and
  permission requests.

## Testing

- Update the shell escalation test to apply a thread-level policy override and
  verify that the next turn rejects an escalation using the updated policy.

GitOrigin-RevId: e5966ba08f179d53fd76871ad904762958c0f5ea
2026-08-04 15:32:31 +00:00
viyatb-oai 0042b00986 Record normalized sandbox violation events (#36207)
## Why

Filesystem denials and managed-network blocks did not share a structured event
shape, requiring downstream consumers to rediscover enforcement paths and parse
backend-specific output. See https://github.com/openai/codex/pull/17573.

## What changed

- Add normalized filesystem and network violation types in `codex-sandboxing`
  and emit them through a shared tracing seam.
- Classify filesystem denials by backend and reason, retaining an optional path
  and bounded output snippet, and preserve managed-network block context.
- Report the sandbox type through exec-server responses so unified exec can
  classify remote denials without guessing; omitted values remain compatible
  with older peers.
- Record violations from exec, apply-patch, shell-escalation, unified-exec, and
  managed-network enforcement paths without changing denial behavior.

## Testing

- Cover filesystem classification, path extraction, `SIGSYS`, network event
  conversion, protocol compatibility, and remote sandbox-type propagation.

GitOrigin-RevId: d673173b4fa6bdf0a24421194a8a61c81fab9c96
2026-07-30 19:23:33 +00:00
jif 789c72dcf6 Avoid shifting bytes in streaming output buffers (#36194)
## Why

Removing each decoded prefix from a `Vec` shifts all remaining bytes. Repeating
that work is costly for streams containing many invalid UTF-8 bytes or many
framed messages in one relay record.

## What changed

- Buffer unified exec output in a `VecDeque` and use UTF-8 error offsets to
  consume the largest valid prefix while continuing to make progress on invalid
  input.
- Buffer Noise relay messages in `BytesMut` and advance past decoded frames
  without moving the remaining bytes.

## Testing

Added regression coverage for valid data surrounding invalid UTF-8, front
consumption without shifting, and reassembling many JSON-RPC messages from one
record.

GitOrigin-RevId: 661ccfffc68960ac6c9d37c738224da8ea83cbf0
2026-07-30 18:04:45 +00:00
viyatb-oai 0dcad0c972 Use permission profiles throughout sandbox execution (#36183)
## What changed

- Pass the canonical `PermissionProfile` through execution requests, sandbox
  selection, escalation, and the exec server instead of carrying separate
  derived filesystem and network policies.
- Derive runtime sandbox policies from the effective profile at their points of
  use, including after applying additional permissions.
- Update sandbox and escalation tests to inspect policies through the profile.

GitOrigin-RevId: 695b82ea94f5289516b0933bb23074cb2dcce8ac
2026-07-30 16:30:32 +00:00
iceweasel-oai fd41e813cb Raise the Windows exec yield floor to 10 seconds (#35670)
## What changed

- Clamp the initial `exec_command` yield time to at least 10 seconds on Windows.
- Update the tool description and unit coverage for the new effective range.
- Allow the Windows Ctrl-C integration test up to 20 seconds to complete.

GitOrigin-RevId: 65d4d1c9a89dc9979e10d3e565af61c9f632c058
2026-07-27 21:12:00 +00:00
Adam Perry @ OpenAI c21c8a5e21 Fix the async watcher test harness on Windows (#35665)
Initialize the Windows-only `ProcessDriver::tty` field to `false` when
constructing the streaming output test harness.

GitOrigin-RevId: 95683779072cba31664d059750b81ee0d9c17664
2026-07-27 20:00:11 +00:00
viyatb-oai 462ed19a05 Enable network policy callbacks for remote exec (#35652)
## What changed

- Forward remote managed-network policy requests to the controller-side decider when Guardian review is enabled.
- Bound callbacks using the configured permission-hook and Guardian review timeouts, restore trusted execution attribution, and recheck live baseline policy before requesting a decision.
- Cancel pending decisions when the process or execution scope ends, and propagate EOF across the sandbox proxy bridge with half-closes.

## Testing

- Cover callback enablement, strict allowlist behavior, live policy updates, trusted attribution, and execution-scope cancellation.

GitOrigin-RevId: 83b47657c8c4546dfe98eec8b272833e96a46b7e
2026-07-27 18:07:31 +00:00
iceweasel-oai b115de97d7 Preserve Windows sandbox proxy settings in guardian sessions (#35036)
## Why

Guardian review commands can run without the proxy-port environment used to
configure the parent session. Reconciling persistent Windows sandbox settings
for those commands can discard the parent's proxy configuration.

## What changed

- Add a session-level Windows sandbox proxy-settings mode and use `Preserve`
  for guardian review sessions while keeping `Reconcile` as the default.
- Carry the mode through unified exec and the exec-server sandbox context to
  Windows process launches.
- In preserve mode, reuse the proxy settings recorded by the existing sandbox
  setup marker.

## Testing

Added coverage for guardian session configuration, exec-server transport, and
preserving an existing setup marker when proxy-port environment variables are
absent.

GitOrigin-RevId: 214655c6e6d97360906431773dc0de0fdda2db6e
2026-07-23 23:35:29 +00:00
Kyle Brown 84fa68b429 Attribute command executions to trusted plugin scripts (#35020)
## What changed

- Resolve shell and unified-exec commands against the trusted plugin roots loaded for each turn.
- Add optional `pluginId` and safe plugin-relative `scriptPath` fields to command execution items and legacy execution events, and propagate them through app-server notifications.
- Include the attribution in command execution analytics while rejecting absolute, unsafe, and unattributed script paths.

## Testing

- Cover attribution for cached curated and remote plugin scripts from command execution through core and app-server events.
- Verify analytics serialization and unsafe-path filtering.

GitOrigin-RevId: 02fac3a233284ccfc6642fa502a95f1881dba83d
2026-07-23 22:31:53 +00:00
Adam Perry @ OpenAI 7bafdada8b Separate Codex error details from retry metadata (#34996)
## What changed

- Wrap `CodexErrorDetails` and an optional retry delay in `CodexErr`, allowing any mapped error to preserve server-provided retry timing.
- Generate the payload-free `CodexErrKind` classification alongside the error details and reuse it for analytics.
- Update error handling sites to inspect `CodexErr::details()` while preserving existing display, debug, protocol mapping, and retryability behavior.

## Testing

- Add coverage for legacy debug formatting, error-specific retryability, and retry-delay propagation through API error mapping.

GitOrigin-RevId: d3ab8a305f2a2ee21d0c0a8c8c388b06dda9c59a
2026-07-23 19:32:26 +00:00
jif 9fc715c086 Order unified exec lifecycle events reliably (#34713)
## What changed

- Treat output-task closure as the signal that trailing output is complete, drain any remaining chunks before publishing command completion, and retain the grace period as a fallback.
- Wait for deferred network-denial classification before emitting the final command result.
- Serialize `write_stdin` interaction and completion events so an interaction that exits a process is published first, and avoid pruning processes while their terminal events are being finalized.

## Testing

- Add coverage for late output, missing output-close signals, late network denials, interaction/completion ordering, cross-platform aggregated output, and pruning during finalization.

GitOrigin-RevId: 46a118552bb1a4658b78aa5a5ca76eb8c5fba571
2026-07-22 09:29:09 +00:00
iceweasel-oai 999a715089 Route Windows sandbox proxy traffic by restricting SID (#34613)
## Why

Elevated Windows sandboxes need stable managed-proxy ports while preserving the network policy and environment attribution of each sandboxed process.

## What changed

- Keep shared HTTP and SOCKS5 loopback ingress listeners alive across managed-proxy instances.
- Add a per-route restricting SID to elevated sandbox tokens and dispatch incoming connections to the matching proxy policy after attributing the client process.
- Reject connections without exactly one registered route, remove routes when their proxy handle is dropped, and keep unsandboxed Windows launches off the managed ingress.
- Provision the elevated sandbox with the configured proxy ports and local-binding setting, honoring the selected profile and CLI overrides.

## Testing

- Add Windows unit tests for TCP ownership attribution, route selection, restricting-token propagation, and setup settings.
- Add an end-to-end Windows test covering stable ports, isolated environment policies, HTTP and SOCKS5 routing, and route teardown.

GitOrigin-RevId: 783fac6e0f904dc9bb1955b75d4a5895e8bb9690
2026-07-21 21:06:04 +00:00
iceweasel-oai 35c2278dd5 Support Windows sandboxing in the exec server (#34423)
## Why

Sandboxed process launch through the exec server was unsupported on Windows.

## What changed

- Add a shared native process launcher that selects the Windows sandbox session backend when required and otherwise launches PTY or pipe-based processes.
- Pass Windows permission, workspace-root, proxy, filesystem-override, and desktop settings through exec-server process preparation.
- Preserve piped stdin and inherited file descriptors across the shared launch path.

## Testing

- Verify that a remote Windows sandbox process accepts stdin while blocking a write under a read-only policy.
- Enable workspace-root sandbox tests on native Windows while continuing to skip environments that cannot emulate Windows sandbox semantics.

GitOrigin-RevId: 6719ccbe87ad0d5a57af56503e3f23f894c60a71
2026-07-20 22:35:14 +00:00
viyatb-oai 9f6c29e281 Launch managed network proxies on remote executors (#33906)
## Why

Remote executions need managed-network proxy listeners in the executor so their
loopback proxy addresses are reachable by the launched process.

## What changed

- Add a capability-gated exec-server protocol field for executor-local proxy
  launch configuration, including network policy, audit metadata, and execution
  attribution.
- Start the proxy while preparing a remote process, replace inherited proxy
  environment variables with its local addresses, and derive the sandbox
  context from its listeners.
- Keep the proxy alive until inherited output streams close, then shut it down.
- Reject unsupported remote settings such as MITM and credential injection.

## Testing

- Cover configuration round trips and rejection of unsupported settings.
- Verify executor-local startup, blocked-domain enforcement, protocol
  compatibility, and proxy lifetime through process closure.

GitOrigin-RevId: c984f54e3e600aa9ebcbf8cf4574046e2c199d11
2026-07-17 21:20:14 +00:00
jif 771d9d7d26 Remove stale ignored core tests (#33868)
## What changed

- Remove the ignored explorer-role test for role-specific model and reasoning defaults.
- Remove two ignored unified exec tests for timeout capping and completed-session cleanup.

GitOrigin-RevId: 39386546c7628c2e1cf713139cd09fd0bf172700
2026-07-17 16:35:01 +00:00
sayan-oai c4ce0493dc Keep active-turn environments stable across settings updates (#33658)
## Why

With deferred execution enabled, rebuilding a step context from the latest
thread environment snapshot could apply a settings update to a turn that was
already in progress.

## What changed

- Preserve each turn's captured environment selections when creating later
  step contexts.
- Refresh only the readiness of environments that were already starting, while
  retaining their original selection order.
- Apply updated environment selections to the next turn.

## Testing

Add coverage for changing the selected workspace while a turn is paused,
verifying that the active turn keeps its original working directory and the
next turn uses the new one.

GitOrigin-RevId: c6f4388c843384ef045b7abdbc303dcfca11d230
2026-07-16 18:23:43 +00:00
pakrym-oai f64233d142 Run write_stdin concurrently across terminal sessions (#33645)
## What changed

- Allow parallel `write_stdin` tool calls to interact with independent terminal sessions.
- Serialize reads and writes within each session so its output buffer and process lifecycle remain consistent.
- Prevent active sessions from being pruned while a `write_stdin` interaction is in progress.

## Testing

- Added unified exec and code mode coverage that uses two terminal sessions concurrently and verifies output from both.

GitOrigin-RevId: 618f5503f4c4670421f680c0f0c27414bc6b2ddd
2026-07-16 16:58:41 +00:00
iceweasel-oai 4bc2c723ef Select the elevated Windows sandbox for network proxies (#33445)
## Why

Windows firewall enforcement is tied to the logon-user sandbox identities. Proxy-enforced commands therefore need the elevated backend even when the configured sandbox mode is the default restricted-token mode.

## What changed

- Select the elevated Windows sandbox backend whenever a command uses the managed network proxy.
- Remove configuration and startup checks that required users or managed requirements to select the elevated sandbox explicitly.
- Start and expose managed network proxy state consistently on Windows.

## Testing

- Verify that proxy enforcement selects the elevated backend for the restricted-token sandbox mode.

GitOrigin-RevId: e01ac052587721c468a8fbcf4e2c7c5a185a3e05
2026-07-15 23:19:00 +00:00
iceweasel-oai abbb8c569c Require the elevated Windows sandbox for network proxies (#32857)
## Why

Network proxy enforcement on Windows requires the elevated sandbox backend. Silently selecting that backend for an unelevated configuration makes the effective sandbox differ from the configured mode.

## What changed

- Reject enabled network proxy configurations on Windows unless managed requirements allow only the elevated sandbox and `windows.sandbox` resolves to `"elevated"`.
- Preserve the configured sandbox backend during execution and validate proxy compatibility at config loading, sandbox setup, and process launch boundaries.
- Defer this validation during cloud-config bootstrap so authoritative managed requirements can be loaded first.
- Do not start or expose disabled Windows proxies, and render transitions to disabled networking as `<network enabled="false"></network>`.

## Testing

- Cover Windows requirement and sandbox compatibility matrices, bootstrap behavior, backend selection, disabled proxy handling, and environment-context updates.

GitOrigin-RevId: bfa1dda98594e0db61883f8e7f65bd560e3453e6
2026-07-13 19:04:22 +00:00
jif 6138909d6e Keep unified exec output collection bounded (#32150)
## Why

Unified exec can drain process output multiple times while waiting for a command. Collecting those drains into an uncapped buffer allows large commands to exceed the output collection limit.

## What changed

- Accumulate drained output through the capped head/tail buffer so collection remains bounded while preserving the beginning and end of the stream.
- Include a `... N bytes omitted ...` marker when the collection cap drops output.
- Calculate the original token estimate from all observed bytes and preserve omission metadata through truncation and sandbox-denial responses.

## Testing

- Cover repeated drains, previously omitted output, omission markers, and large-output summaries.

GitOrigin-RevId: c6e2efeb875ed8f35914365eab7431f25fd3484c
2026-07-10 12:22:04 +00:00
Owen Lin cca16a1087 feat(core): emit canonical command execution items (#31297)
This PR depends on [#31296](https://github.com/openai/codex/pull/31296)
for the canonical-to-legacy event mappings.

## Description

This PR makes command execution emit canonical
`TurnItem::CommandExecution` lifecycle from both the shell tool path and
user `/shell` commands.

App-server v2 consumes the canonical command items directly and ignores
the mapped `ExecCommandBegin` / `ExecCommandEnd` compatibility events,
so clients still receive one command item lifecycle.

`UnifiedExecInteraction` stays on the legacy path because
`TerminalInteraction` is still the v2 surface for stdin and poll events.
Emitting a command item there would render the same wait twice.

## Why

This is the first live producer migration after the compatibility
mappings in #31296. Keeping command execution separate makes the unified
exec exception reviewable without mixing in dynamic tools or multi-agent
behavior.

## What changed

- Emit canonical command execution items from shell tool events and user
shell commands.
- Preserve the existing unified exec interaction carveout.
- Move app-server command deduplication and completion bookkeeping onto
canonical item events.
- Update unified exec coverage to assert the completed command item.
2026-07-06 21:22:51 -07:00
Channing Conger 84fe70c30e elicitations: Move to shared ElicitationService (#30627)
## Why

Code-mode tool results could return to the model while an MCP
elicitation was still waiting for user input. This differed from
parallel tool calling and could let the model continue before the user
resolved the request.

We need one session-level view of outstanding elicitations so tool
runtimes can consistently hold results until every pending elicitation
is resolved.

  ## What changed

- Added a counted, session-owned ElicitationService with RAII
registrations.
- Registered both core-originated and server-originated MCP elicitations
with the service.
- Migrated out-of-band elicitation tracking and unified exec timeout
pausing to the shared service.
- Made code-mode functions.exec and functions.wait capture their runtime
result normally, then hold it before returning while an elicitation is
outstanding.
  - Kept terminate: true immediate; only its result is held.
  - Preserved model-visible wall time across the elicitation hold.
- Kept the behavior session-scoped, with concurrent elicitations holding
the pause until all registrations are released.
2026-07-06 11:20:32 -07:00
richardopenai d4ec08b8f0 [codex] consume pushed exec-server process events (#30273)
## Summary

- complete unified-exec processes from the ordered event stream instead
of issuing a final zero-wait `process/read`
- add optional executor sandbox-denial state to `process/exited`
- retain `process/read` as a retained-output and compatibility fallback
for receiver lag, sequence gaps, and legacy servers
- recover sandbox-denial state across transport reconnection
- cover the real `TestCodex` remote-exec path without adding a public
test-only event constructor

## Why

A successful one-shot tool call currently receives its output and
terminal notifications, then pays another wide-area `process/read` round
trip before returning. Staging traces showed that remote response wait
accounted for more than 99.8% of RPC time; local serialization,
queueing, and deserialization were below 0.6 ms.

## Measured impact

A direct staging A/B used the same build and route and changed only
completion mode. Each arm ran three times with 30 one-shot
`/usr/bin/true` calls per run. The table reports the median of the three
per-run percentiles.

| Metric | Final `process/read` | Pushed events | Change |
| --- | ---: | ---: | ---: |
| End-to-end completion p50 | 159.5 ms | 118.7 ms | -40.8 ms (-25.6%) |
| End-to-end completion p95 | 182.4 ms | 131.7 ms | -50.6 ms (-27.8%) |
| Completion-wait p50 | 80.1 ms | 41.5 ms | -38.5 ms (-48.1%) |
| Final `process/read` RPC p50 | 79.9 ms | eliminated | -79.9 ms |

TCP_NODELAY was enabled in both A/B arms, so its effect cancels out. The
successful, complete, in-order event path issued zero final
`process/read` calls.

## Compatibility and recovery

- new servers send `sandboxDenied` on `process/exited`
- legacy servers omit it, which triggers one compatibility
`process/read`
- broadcast lag or a sequence gap triggers a retained-output read
- recovery remains bounded by the server's existing 1 MiB
retained-output window
- complete, in-order event streams issue no completion read
- sandbox denial is attached to the exit event before consumers can
observe process completion
- server-first and client-first rollouts remain wire-compatible;
server-first realizes the latency win immediately

## Integration coverage

The `TestCodex` suite exercises four distinct remote-exec contracts:

- complete pushed output/exit/close with zero reads
- direct pushed sandbox denial with zero reads
- legacy missing denial metadata with exactly one compatibility read
- count-bounded replay eviction recovered from retained output without
duplication

## Validation

- `just test -p codex-core
exec_command_consumes_pushed_remote_process_events`: 4 passed
- `just test -p codex-core unified_exec::process_tests::`: 4 passed
- `just test -p codex-exec-server`: 294 passed, 2 skipped
- `just test -p codex-exec-server-protocol`: 5 passed
- `just test -p codex-rmcp-client`: 89 passed, 2 skipped
- focused Bazel `//codex-rs/core:core-all-test`: passed across 16 shards
- scoped `just fix` passed for core and exec-server
- `just fmt` passed

The complete workspace suite was not rerun; focused Cargo and Bazel
coverage passed for the changed behavior.
2026-06-26 18:05:52 -07:00
Michael Bolin c65cfeab14 core: expose permission profile to shell tools (#29941)
## tl;dr

Inject a `CODEX_PERMISSION_PROFILE` environment variable with the name
of the current permission profile when invoking a shell tool.

## Why

Shell tool owners may need to launch nested commands under the same
named permission profile, including through `codex sandbox -P PROFILE
--include-managed-config`. Until now, child processes could observe
sandbox and network metadata but could not identify the active named
permission profile.

The `--include-managed-config` flag is essential when a helper
reconstructs the sandbox from a profile name: it ensures the nested
sandbox also loads managed enterprise requirements. Without it, using
the inherited profile could unintentionally create a sandbox that does
not enforce the organization's managed restrictions.

The new environment value is intentionally informational and **must not
be treated as trusted input**. Any process in the ancestry can overwrite
an environment variable, so a consumer that passes this value to `codex
sandbox -P` must first validate it against the profiles that helper is
authorized to use.

## Example Use Case

Suppose an organization provides a trusted `remote-bash` wrapper that
lets Codex run a command on an approved build host. The local shell
command uses the named `:workspace` permission profile:

```toml
default_permissions = ":workspace"
```

The command exposed to the model is a small zsh wrapper. It deliberately
delegates with `exec`, preserving the original arguments and process
environment:

```zsh
#!/usr/bin/env zsh
exec /opt/codex-tools/remote_bash.py "$@"
```

The model invokes the public wrapper, not its Python implementation:

```sh
/opt/codex-tools/remote-bash \
  --host builder.example.com \
  -- printf '%s' 'hello world'
```

Only the inner implementation is authorized to escape the local sandbox:

```starlark
prefix_rule(
    pattern=["/opt/codex-tools/remote_bash.py"],
    decision="allow",
)
```

With zsh-fork, execution begins with `remote-bash` inside the
`:workspace` sandbox. When the wrapper calls `exec`, the exact prefix
rule matches `remote_bash.py`, so that inner script is restarted
unsandboxed. The escalated process inherits:

```text
CODEX_PERMISSION_PROFILE=:workspace
```

Inheritance does not make the value trustworthy. `remote_bash.py`
independently allowlists both the remote host and the permission profile
before using either value. In particular, a forged value such as
`:danger-full-access` is rejected before it can reach `codex sandbox
-P`:

```python
import argparse
import os
import shlex
import sys

ALLOWED_HOSTS = {"builder.example.com"}
ALLOWED_PROFILES = {":workspace"}

parser = argparse.ArgumentParser()
parser.add_argument("--host", required=True)
separator = sys.argv.index("--")
args = parser.parse_args(sys.argv[1:separator])
command = sys.argv[separator + 1:]

if args.host not in ALLOWED_HOSTS:
    parser.error("host is not allowlisted")
if not command:
    parser.error("the remote command must not be empty")

profile = os.environ.get("CODEX_PERMISSION_PROFILE")
if not profile:
    raise SystemExit("CODEX_PERMISSION_PROFILE must not be empty")
if profile not in ALLOWED_PROFILES:
    raise SystemExit("CODEX_PERMISSION_PROFILE is not allowlisted")

remote_command = shlex.join(command)
sandbox_command = shlex.join([
    "codex", "sandbox", "-P", profile,
    "--include-managed-config", "--",
    "bash", "-lc", remote_command,
])
print(shlex.join(["ssh", args.host, sandbox_command]))
```

This builds each command layer as an argument vector and uses
`shlex.join()` at the boundary, rather than interpolating untrusted
shell text. After validation and parsing, the nested command has this
structure:

```text
ssh argv:
  ["ssh", "builder.example.com", SANDBOX_COMMAND]

SANDBOX_COMMAND argv:
  ["codex", "sandbox", "-P", ":workspace",
   "--include-managed-config", "--",
   "bash", "-lc", "printf %s 'hello world'"]

bash -lc payload argv:
  ["printf", "%s", "hello world"]
```

A production implementation could execute that SSH command. The
integration fixture prints it and parses the result back into arguments,
verifying the complete flow:

```text
model invokes outer wrapper
  -> zsh-fork starts wrapper under :workspace
  -> wrapper execs allowlisted Python script
  -> prefix rule restarts Python script unsandboxed
  -> Python script inherits CODEX_PERMISSION_PROFILE=:workspace
  -> Python script verifies :workspace is allowlisted
  -> remote command runs codex sandbox -P :workspace
     with --include-managed-config
  -> nested sandbox honors managed enterprise requirements
```

This gives the trusted helper access to resources outside the local
sandbox—such as SSH credentials—while ensuring that it can select only
an explicitly authorized profile and that work on the remote host
remains subject to the organization's managed requirements.

## What changed

- Inject `CODEX_PERMISSION_PROFILE` after shell environment policy
evaluation so the active profile wins over inherited or configured stale
values.
- Apply the variable to both `shell_command` and unified `exec_command`,
including local, zsh-fork, and remote exec-server paths.
- Remove stale values when the session has no active named profile.
- Preserve the current profile value when loading a shell snapshot so a
parent snapshot cannot restore an older profile.

## Testing

- Added classic-shell integration coverage proving an exact prefix rule
can run a `require_escalated` script outside the `:workspace` sandbox
while preserving `CODEX_PERMISSION_PROFILE=:workspace`.
- Added zsh-fork integration coverage in which the model invokes an
outer zsh wrapper, an inner allowlisted `remote_bash.py` runs
unsandboxed, and its printed SSH command reconstructs the inherited
`:workspace` sandbox with `--include-managed-config` while preserving
every argument after `--`.
- The example helper treats `CODEX_PERMISSION_PROFILE` as untrusted and
validates it against `ALLOWED_PROFILES` before constructing the nested
command.
- Assert that the reconstructed sandbox command includes
`--include-managed-config` so nested use of the inherited profile cannot
bypass managed enterprise requirements.
- Added coverage for overriding and removing stale profile values.
- Verified `shell_command` receives the selected active profile.
- Added shell snapshot coverage using `printenv
CODEX_PERMISSION_PROFILE`.
2026-06-25 19:00:23 +00:00
Adam Perry @ OpenAI 9a79536e6b test: branch on target OS instead of runner flavor (#29712)
## Why

Core tests should branch on the executor's operating system, not on
runner details such as Docker or Wine. This keeps platform behavior
stable as new test backends are added and reserves Wine-specific skips
for actual runner debt.

## What

- Add `TestTargetOs` and target/host-aware skip helpers while keeping
`TestEnvironment` internal.
- Replace topology enum access with remote predicates and a narrow
Docker accessor.
- Migrate OS-semantic Wine skips, preserve runner-specific gaps, and
document the skip taxonomy.

## Validation

- `just test -p core_test_support`
- `just test -p codex-core
remote_test_env_can_connect_and_use_filesystem`
- `bazel test //codex-rs/core:core-all-wine-exec-test
--test_output=errors` reached test execution; unrelated existing
view-image, path, and timing failures remain.
- `just test -p codex-core` and `just test` reached broad test
execution; this checkout has unrelated helper, sandbox, and timing
failures.
2026-06-23 14:27:13 -07:00
jif e476fc16ce Prepare managed network sandbox context (#29456)
## Why

Managed network configures commands to use local HTTP and SOCKS proxies.
For commands delegated to the exec server, the proxy environment and the
sandbox policy were prepared separately. On macOS, that meant a command
could receive `HTTPS_PROXY=http://127.0.0.1:43123` while Seatbelt still
denied access to port `43123`.

## What changed

`NetworkProxy` now prepares the command environment and sandbox context
together from the same runtime snapshot:

```text
Prepared managed network
├── command environment: HTTPS_PROXY=http://127.0.0.1:43123
└── sandbox context: allow outbound to 127.0.0.1:43123
```

That context travels with remote exec requests. The exec server
preserves the managed proxy and CA environment, and macOS Seatbelt
allows only the prepared loopback proxy ports without enabling broad
network access or local binding.

The protocol field is optional and the existing enforcement flag remains
in place, preserving compatibility with callers that do not send the new
context.
2026-06-23 20:07:09 +01:00
jif 9f06cf1a09 Report remote sandbox denials semantically (#29424)
## Why

#29113 moved remote sandbox setup and enforcement to the exec server.
That gives the executor ownership of the platform-specific work: a Linux
executor chooses and runs a Linux sandbox even when the Codex
orchestrator is running on macOS or Windows.

It also means the orchestrator no longer knows which concrete sandbox
the executor selected. When that sandbox blocks a remote command, the
orchestrator currently sees only a failed process and can treat the
denial as an ordinary command failure. The existing sandbox approval and
retry path is then skipped.

This PR lets the executor report one portable fact:

> This command probably failed because the executor sandbox blocked it.

The executor keeps its concrete sandbox type private. The protocol sends
only the semantic result.

## Example

Suppose a local macOS Codex session asks a Linux devbox to write outside
the allowed workspace.

Before this PR:

```text
Linux sandbox blocks the write
    -> remote process exits with "Permission denied"
    -> local orchestrator sees an ordinary command failure
    -> the normal sandbox approval and retry path can be skipped
```

With this PR:

```text
Linux sandbox blocks the write
    -> executor reports sandboxDenied: true
    -> unified exec returns UnifiedExecError::SandboxDenied
    -> the existing approval prompt is shown
    -> an approved retry runs through the existing unsandboxed retry path
```

## What changes

### The executor remembers its selected sandbox

The prepared remote process now retains the executor-selected
`SandboxType`. This value never crosses the executor boundary.

Commands started without a sandbox retain `SandboxType::None` and are
never reported as sandbox denials.

### The executor uses the existing denial heuristic

The existing local denial heuristic moves from `codex-core` into the
shared `codex-sandboxing` crate.

When a sandboxed remote process exits, the executor:

1. waits the same short output grace period used by local unified exec;
2. reads the output currently available in the existing retained output
buffer;
3. runs the existing heuristic using the exit code and common denial
messages;
4. stores the yes/no result before publishing the process exit.

This deliberately matches the old local unified-exec behavior. It does
not add a new streaming classifier, another output buffer, or stronger
output-retention guarantees.

### The protocol reports a portable boolean

`process/read` gains `sandboxDenied`:

```json
{
  "exited": true,
  "exitCode": 1,
  "closed": false,
  "sandboxDenied": true
}
```

The field defaults to `false` when an older executor omits it. The
response does not expose the executor sandbox implementation or
executor-native paths.

### Unified exec uses the existing error path

The exec-server client carries `sandboxDenied` into the unified process
state. If it is true, unified exec returns the existing `SandboxDenied`
error instead of trying to classify remote output using an
orchestrator-side sandbox type.

Remote process exit remains visible as soon as the process exits. This
PR does not wait for stdout or stderr to close and does not change the
existing process lifecycle.

## Scope

This PR is intentionally limited to matching the existing local
unified-exec behavior for the initial command execution path.

It does not add:

- incremental denial tracking across the full output stream;
- new denial handling for commands completed later through
`write_stdin`;
- new guarantees for preserving the semantic flag during the narrow
reconnect-recovery race.

Those can be considered separately if the same behavior is added for
local execution.

## Test coverage

One remote end-to-end integration test covers the complete intended
flow:

```text
remote read-only sandbox
    -> denied write
    -> executor reports the denial
    -> Codex requests approval
    -> user approves
    -> retry succeeds on the remote executor
```

Existing lifecycle coverage continues to verify that remote process exit
is reported before late output streams close.
2026-06-22 19:33:28 +02:00
jif bd2968a4db Carry sandbox intent to remote exec servers (#29108)
## What changed

PR #29099 stopped sending the orchestrator's concrete sandbox wrapper to
a remote exec-server. Remote commands now arrive as plain native argv.

This PR adds the next piece: Codex also sends portable sandbox intent
next to that plain argv.

For a remote unified-exec command, the request can now include:

- the canonical permission profile before local workspace-root
materialization
- the sandbox cwd and workspace roots as `PathUri` values
- Windows sandbox settings
- the legacy Landlock setting
- whether managed networking must be enforced

The important part is that symbolic entries such as `:workspace_roots`
stay symbolic while crossing the boundary. The executor can then bind
them to its own workspace-root paths instead of receiving
orchestrator-local absolute paths.

The data travels through `ExecRequest` into `ExecParams`. Older
exec-servers can still deserialize requests because the new fields have
defaults.

## Why

The orchestrator should not decide how another machine implements
sandboxing.

For example:

- a local macOS Codex would normally build a Seatbelt command
- a remote Linux executor needs a Linux sandbox command instead

The orchestrator now sends the plain command plus the policy it intended
to enforce. A later PR can let the exec-server choose and build the
correct sandbox for its own operating system.

## Important detail

This keeps the portable intent separate from the local `SandboxType`.

`SandboxType::None` is ambiguous:

- it can mean the command was explicitly approved to run without a
sandbox
- it can also mean the orchestrator host has no concrete sandbox
implementation available

Those cases are different for remote execution. This PR adds
`sandbox_requested` so an executor can still receive sandbox intent when
the orchestrator cannot build a local wrapper. Explicit unsandboxed
retries still send no sandbox context.

## Behavior today

This PR only transports the intent. The exec-server accepts the new
fields but does not apply them yet.

Remote commands therefore remain unsandboxed after this PR, just as they
are after PR #29099.

## Follow-up

The next PR will make exec-server read this portable intent, bind
symbolic workspace permissions to executor-native roots, choose the
sandbox for its own operating system, build the wrapper locally, and
then spawn the command.
2026-06-21 12:33:21 +02:00