Commit Graph

271 Commits

Author SHA1 Message Date
rka-oai
b9ba969f39 Enable remote compaction for Amazon Bedrock (#36981)
## What changed

- Add provider-owned remote compaction capabilities for unsupported, v1, and v2 protocols.
- Mark Amazon Bedrock as v1-only so manual and automatic compaction use `/v1/responses/compact`, even when the v2 feature is enabled.
- Preserve v2 support for OpenAI and Azure Responses providers and local compaction for unsupported providers.

## Testing

- Cover Amazon Bedrock manual and automatic remote compaction, including endpoint selection, authentication headers, model selection, and reuse of the compacted summary.

GitOrigin-RevId: d58093178605d1f8155b93d9a161408177f6ac72
2026-08-04 22:07:56 +00:00
rka-oai
bb5054fe47 Capture rollout budget units from response usage (#36641)
## What changed

- Parse `codex_rollout_budget_units` from completed Responses API usage into
  `TokenUsage`.
- Keep the provider-only value out of serialized protocol, JSON schema, and
  TypeScript representations.

## Testing

- Cover the value in unit and end-to-end SSE response parsing tests.

GitOrigin-RevId: a53d4202beb9c8985d25894cb10e7c01f20b2a44
2026-08-03 00:30:57 +00:00
jif
35eab50501 Pass sessions directly to session tasks (#36354)
## What changed

- Replace `SessionTaskContext` with `Arc<Session>` in the session task run and abort interfaces.
- Read extension data from `TurnContext` during regular turns instead of storing and forwarding a duplicate reference through the running task.

GitOrigin-RevId: 04bc0d324e5ba781f35ac9c61e811888208b74ae
2026-07-31 16:05:52 +00:00
rka-oai
9eeac78b3f Refresh environment dates from the configured clock (#36187)
## Why

The `<current_date>` in environment context could diverge from the clock used by current-time features, especially when a host provides an external clock.

## What changed

- Read the current time from the session's time provider whenever world state is built, and render its local date in environment context.
- Use each session's own thread ID for clock reads, including guardian sessions.
- Propagate clock-read failures through turn error events and stop before model inference.

## Testing

- Cover date updates across turns and app-server requests.
- Verify guardian clock reads use the corresponding root or guardian thread ID and do not run during prewarm.
- Verify clock failures emit an error without sending an inference request.

GitOrigin-RevId: bb7f32ccc0229b7bc6f14d7356db00bcce78066d
2026-07-30 16:44:18 +00:00
viyatb-oai
0dcad0c972 Use permission profiles throughout sandbox execution (#36183)
## What changed

- Pass the canonical `PermissionProfile` through execution requests, sandbox
  selection, escalation, and the exec server instead of carrying separate
  derived filesystem and network policies.
- Derive runtime sandbox policies from the effective profile at their points of
  use, including after applying additional permissions.
- Update sandbox and escalation tests to inspect policies through the profile.

GitOrigin-RevId: 695b82ea94f5289516b0933bb23074cb2dcce8ac
2026-07-30 16:30:32 +00:00
Eddie Chen
1def0a8925 Track parent turns for nested Codex requests (#35835)
## What changed

- Propagate the initiating turn ID through agent spawns, follow-up tasks, reviews, and delegated Codex sessions.
- Add `parent_turn_id` to Responses client and turn metadata while keeping it out of external MCP metadata.
- Preserve parent-turn provenance across queued agent messages when their triggering parent is unambiguous.

## Testing

- Cover spawned, resumed, nested, reviewed, delegated, and WebSocket request metadata.
- Verify queued messages do not claim ambiguous or queue-only parent turns.

GitOrigin-RevId: 481fdebbe7df2031880fe259509273cce50b20a8
2026-07-28 21:58:33 +00:00
Kyle Brown
84fa68b429 Attribute command executions to trusted plugin scripts (#35020)
## What changed

- Resolve shell and unified-exec commands against the trusted plugin roots loaded for each turn.
- Add optional `pluginId` and safe plugin-relative `scriptPath` fields to command execution items and legacy execution events, and propagate them through app-server notifications.
- Include the attribution in command execution analytics while rejecting absolute, unsafe, and unattributed script paths.

## Testing

- Cover attribution for cached curated and remote plugin scripts from command execution through core and app-server events.
- Verify analytics serialization and unsafe-path filtering.

GitOrigin-RevId: 02fac3a233284ccfc6642fa502a95f1881dba83d
2026-07-23 22:31:53 +00:00
Adam Perry @ OpenAI
7bafdada8b Separate Codex error details from retry metadata (#34996)
## What changed

- Wrap `CodexErrorDetails` and an optional retry delay in `CodexErr`, allowing any mapped error to preserve server-provided retry timing.
- Generate the payload-free `CodexErrKind` classification alongside the error details and reuse it for analytics.
- Update error handling sites to inspect `CodexErr::details()` while preserving existing display, debug, protocol mapping, and retryability behavior.

## Testing

- Add coverage for legacy debug formatting, error-specific retryability, and retry-delay propagation through API error mapping.

GitOrigin-RevId: d3ab8a305f2a2ee21d0c0a8c8c388b06dda9c59a
2026-07-23 19:32:26 +00:00
Boyang Niu
44d76c6a6d Wake sleeping threads for queued agent mail (#34852)
## Why

An idle thread with an outstanding durable sleep must resume when agent work
arrives, even when that message would normally remain queued without triggering a
turn.

## What changed

- Treat any pending mailbox message as wake-up work while a durable sleep is
  attached to the thread.
- Keep requiring `trigger_turn` for idle threads that are not durably asleep.
- Cover waking a sleeping root thread from queue-only agent mail and persisting
  the message in thread history.

GitOrigin-RevId: e361ac1b104436d4d63aebfe5e2422691a1cc139
2026-07-23 01:04:01 +00:00
Ahmed Ibrahim
d7e8f4c3dc Preserve user input when MCP startup is interrupted (#34839)
## Why

Interrupting a turn while MCP tools were still starting could abort before the
submitted user input was recorded in conversation history.

## What changed

- Build and retain the MCP tool list and tool router as part of each step
  snapshot, using the turn cancellation token during tool discovery.
- Record turn hooks and submitted input when cancellation occurs during startup
  prewarming, pre-sampling compaction, or first-step capture.
- Reuse the captured router for sampling, compaction, and prompt debugging so
  advertised and executable tools share the same request-scoped view.

## Testing

Add coverage that interrupts each affected MCP startup phase and verifies that
the user prompt remains in history before the turn-aborted marker.

GitOrigin-RevId: 6e96c6814357a55545d69b580e28f35933cfadf7
2026-07-22 22:50:13 +00:00
Ben Romano
bd5b55e403 Track compaction time in turn profiles (#34835)
## What changed

- Add `compaction_ms` to turn profile facts and turn analytics events.
- Measure manual and automatic compaction as an exclusive profile phase instead of including it in surrounding idle time.
- Complete the profile and total turn duration from the same instant so their elapsed times remain consistent.

## Testing

- Cover compaction phase accounting and profile-duration consistency.
- Update turn event serialization coverage for `compaction_ms`.

GitOrigin-RevId: 50f0e1cf19789a807f9f56c2ef763c620a5d641c
2026-07-22 22:13:02 +00:00
jif
687f05cb94 Remove CSV-backed agent jobs (#34413)
## What changed

- Remove the `spawn_agents_on_csv` and `report_agent_job_result` tools and their agent-job runtime and state models.
- Drop the legacy `agent_jobs` and `agent_job_items` tables during state database migration.
- Keep `features.enable_fanout` and `agents.job_max_runtime_seconds` accepted as no-op compatibility settings while omitting them from the generated configuration schema.

## Testing

- Verify upgrades remove both legacy agent-job tables.
- Verify the removed feature and configuration keys still parse without taking effect.

GitOrigin-RevId: 8cc3337da78c67162229f02f40a747f503542646
2026-07-20 21:00:17 +00:00
viyatb-oai
9f6c29e281 Launch managed network proxies on remote executors (#33906)
## Why

Remote executions need managed-network proxy listeners in the executor so their
loopback proxy addresses are reachable by the launched process.

## What changed

- Add a capability-gated exec-server protocol field for executor-local proxy
  launch configuration, including network policy, audit metadata, and execution
  attribution.
- Start the proxy while preparing a remote process, replace inherited proxy
  environment variables with its local addresses, and derive the sandbox
  context from its listeners.
- Keep the proxy alive until inherited output streams close, then shut it down.
- Reject unsupported remote settings such as MITM and credential injection.

## Testing

- Cover configuration round trips and rejection of unsupported settings.
- Verify executor-local startup, blocked-domain enforcement, protocol
  compatibility, and proxy lifetime through process closure.

GitOrigin-RevId: c984f54e3e600aa9ebcbf8cf4574046e2c199d11
2026-07-17 21:20:14 +00:00
Eric Traut
2edad72de3 Track prompt cache write token usage (#33454)
## What changed

- Parse `cache_write_tokens` from response input token details and carry it through token usage aggregation.
- Expose `cache_write_input_tokens` in protocol, app-server, exec, and TypeScript SDK usage events, defaulting it to zero for compatibility with older payloads.
- Report cache-write usage in analytics, metrics, traces, and rollout data alongside existing cache-read usage.

## Testing

- Add coverage for response parsing, usage event output, aggregation, analytics, metrics, and telemetry fields.

GitOrigin-RevId: 634e4fbe3086a8923c0ee13b622573e7d9d221bd
2026-07-16 00:22:53 +00:00
jif
c28770a42f Respect final-answer boundaries for queued agent mail (#33367)
## Why

Queue-only child-agent updates that arrive around a final answer should not
restart sampling, but they must remain available to the next turn. Explicitly
injected response items still need to reopen the current turn.

## What changed

- Defer queue-only inter-agent mail after a final answer without treating it as
  same-turn pending input.
- Re-enable current-turn mailbox delivery when the model requests a follow-up,
  a stop hook continues the turn, or a response item is explicitly injected.
- Check for turn-triggering mailbox work after clearing the completed active
  turn so pending work can start safely.

## Testing

Added coverage that queued child mail waits for the next user turn and that an
injected response item after a final answer triggers a follow-up request.

GitOrigin-RevId: aa81d00707943284b7ab069b60970e547908c895
2026-07-15 15:48:12 +00:00
pakrym-oai
e4711f2a3b Separate session state from session I/O (#33209)
## What changed

- Replace the combined `Codex` wrapper with an `Arc<Session>` and a dedicated
  `SessionIo` handle for submissions, events, status, and loop termination.
- Make `Session::spawn` return the session and I/O handle separately, and have
  `CodexThread` own both.
- Update thread management, delegated sessions, guardian reviews, and agent
  control to use the appropriate session or I/O interface.

GitOrigin-RevId: 8523bad7e7366e2969fd0cc3849b4247b141427a
2026-07-15 02:16:53 +00:00
Ahmed Ibrahim
4657ab06c6 Derive collaboration settings from turn context (#32900)
## Why

`TurnContext` stored model and reasoning settings both as resolved turn fields and inside `CollaborationMode`, requiring the copies to stay synchronized when the model changed.

## What changed

- Store the mode and collaboration-mode developer instructions directly on `TurnContext`.
- Reconstruct `CollaborationMode` from those values plus the turn's current model and reasoning effort when callers need it.
- Read the mode directly for plan-mode checks, lifecycle events, and telemetry.

GitOrigin-RevId: b8fccafdd5d5cbccc84bc70d542588087ddf0093
2026-07-13 23:48:42 +00:00
pakrym-oai
c9d52de5ca Require prefixes for outbound response item IDs (#32312)
## What changed

- Add a `ResponseItemId` type that generates item-specific prefixes with UUIDv7 suffixes and use it across response items.
- Keep deserialization permissive for legacy histories, but omit empty or unprefixed item IDs from HTTP and WebSocket requests.
- Export the new type in the generated TypeScript protocol schema.

## Testing

- Cover prefixed ID generation, legacy deserialization, prefix recognition, and outbound request filtering.

GitOrigin-RevId: 0209fe430c826d0ae88bc4648652c4ebf390c4a4
2026-07-11 02:29:14 +00:00
Owen Lin
c4318c386d Include terminal errors in turn completion events (#32280)
## What changed

- Add an optional `error` payload to `TurnCompleteEvent` and omit it when a turn completes without an error.
- Preserve the full terminal `ErrorEvent` through the turn lifecycle so the completion event includes the same error details emitted separately by `EventMsg::Error`.

## Testing

- Extend the stream-error integration test to verify that `TurnCompleteEvent` contains the emitted error and that the next turn can still proceed.

GitOrigin-RevId: 8d32942a44132763076130102d8b89ab356d59d0
2026-07-10 22:04:03 +00:00
Owen Lin
bca577d69a Include start times in terminal turn events (#32263)
## What changed

- Add an optional `started_at` Unix timestamp to `TurnCompleteEvent` and
  `TurnAbortedEvent`.
- Populate the timestamp from turn timing state when a turn completes or is
  aborted.
- Preserve the timestamp when synthesizing interrupted fork history and when
  importing external sessions.

## Testing

- Verify that interrupting a turn emits a `TurnAbortedEvent` with a start
  timestamp.

GitOrigin-RevId: 75b3911b95c060e5fc1843a5e293d8fe47377186
2026-07-10 20:25:03 +00:00
Owen Lin
6463ede7db Generate unique IDs for review rollout messages (#32213)
## What changed

Assign a fresh UUIDv7 to each synthesized user and assistant message recorded
when exiting review mode, instead of reusing fixed IDs across review runs.

GitOrigin-RevId: 8524176895e2d8c750760554f515f397728a8554
2026-07-10 16:51:23 +00:00
jif
cf6d8ad921 Remove the network proxy config wrapper (#31767)
## Why

`NetworkProxyConfig` only wrapped `NetworkProxySettings` in a single
`network` field. That extra level made runtime callers repeat `.network`
everywhere without representing a real boundary.

## What changed

- move the managed-network fields directly onto `NetworkProxyConfig`
- collapse the matching partial-config wrapper
- update runtime callers and tests to use the direct fields
- keep the user-facing permissions/profile TOML layout unchanged

The internal serialized shape now matches the runtime type itself. This
does not change managed-network behavior or the `config.toml` shape.
2026-07-09 17:33:37 +01:00
Owen Lin
23aac925e7 feat(core): emit canonical review mode items (#31473)
## Description

This PR moves review-mode markers onto canonical `TurnItem` lifecycle:

- `TurnItem::EnteredReviewMode`
- `TurnItem::ExitedReviewMode`

Core now emits `ItemStarted` / `ItemCompleted` for both. The completed
items map back into the existing `EnteredReviewMode` /
`ExitedReviewMode` events, so raw core event consumers and legacy
rollout persistence keep seeing the old events.

This is the compatibility layer needed before paginated rollouts persist
review markers as `ItemCompleted(TurnItem)`.

## Why

Review markers were one of the remaining app-server thread items created
directly from legacy events. Giving them canonical items lets paginated
history persist stable turn/item IDs without changing legacy rollouts.

## What changed

- Added canonical review-mode `TurnItem`s and switched review flow to
emit their lifecycle.
- Added completed-item → legacy review event mappings with stable
turn/item IDs.
- Switched app-server live notifications to the generic canonical item
path and kept legacy replay compatible with old payloads.
- Updated `ThreadHistoryBuilder` to replay canonical review items even
though review turns still do not emit `TurnStarted`.
2026-07-08 09:59:50 -07:00
Owen Lin
cca16a1087 feat(core): emit canonical command execution items (#31297)
This PR depends on [#31296](https://github.com/openai/codex/pull/31296)
for the canonical-to-legacy event mappings.

## Description

This PR makes command execution emit canonical
`TurnItem::CommandExecution` lifecycle from both the shell tool path and
user `/shell` commands.

App-server v2 consumes the canonical command items directly and ignores
the mapped `ExecCommandBegin` / `ExecCommandEnd` compatibility events,
so clients still receive one command item lifecycle.

`UnifiedExecInteraction` stays on the legacy path because
`TerminalInteraction` is still the v2 surface for stdin and poll events.
Emitting a command item there would render the same wait twice.

## Why

This is the first live producer migration after the compatibility
mappings in #31296. Keeping command execution separate makes the unified
exec exception reviewable without mixing in dynamic tools or multi-agent
behavior.

## What changed

- Emit canonical command execution items from shell tool events and user
shell commands.
- Preserve the existing unified exec interaction carveout.
- Move app-server command deduplication and completion bookkeeping onto
canonical item events.
- Update unified exec coverage to assert the completed command item.
2026-07-06 21:22:51 -07:00
Tom
f5f812389e [codex] fix terminal rollout event durability (#30144)
Currently session code does not flush the thread store after appending
the `TurnComplete` / `TurnAborted` events.

This isn't a problem in practice for local storage because append_items
itself effectively blocks, but any thread stores that buffer in
append_items and only commit on flush effectively never get these events
persisted.

The fix adds explicit rollout flushes at the terminal emitters after
normal completion and interruption.

Added test cases that assert the number of flushes when completing or
aborting turns. These are admittedly a little brittle and I'm open to
better ideas on how to add automated testing.
2026-06-25 21:01:11 -07:00
Michael Bolin
32b65bbf7a core: reset context for token budget compaction (#29743)
## Why

When `Feature::TokenBudget` is enabled, compaction should behave like
`new_context`: start a fresh context window with the standard injected
context, without asking the server to summarize old history and without
carrying prior user or assistant messages into the next model request.

This is still a compaction operation from the client lifecycle
perspective. Manual `/compact` and auto-compaction should keep the same
observable side effects that clients and hooks expect, including compact
hooks and `TurnItem::ContextCompaction`.

## What changed

- Added `compact_token_budget` to run token-budget manual and inline
auto-compaction through a shared compaction lifecycle.
- Split pending `new_context` requests from forced context-window
startup: `take_new_context_window_request()` consumes pending requests,
and `start_new_context_window()` installs a fresh context window.
- Routed token-budget manual `/compact` and inline auto-compaction to
install a fresh context window locally instead of calling server/local
summarization.
- Preserved compact lifecycle side effects for token-budget compaction
by running pre/post compact hooks and emitting `ContextCompaction` item
start/completion events.
- Updated token-budget tests to assert fresh window IDs, absence of
server-side compaction calls, dropped prior transcript messages/tool
output after reset, and compact hook/item lifecycle behavior.

## Testing

- `just test -p codex-core
token_budget_context_uses_new_window_after_compaction`
- `just test -p codex-core token_budget_compaction_runs_compact_hooks`
- `just test -p codex-core
token_budget_mid_turn_auto_compaction_resets_before_active_follow_up`

---------

Co-authored-by: pakrym-oai <pakrym@openai.com>
2026-06-23 16:59:04 -07:00
jif
e476fc16ce Prepare managed network sandbox context (#29456)
## Why

Managed network configures commands to use local HTTP and SOCKS proxies.
For commands delegated to the exec server, the proxy environment and the
sandbox policy were prepared separately. On macOS, that meant a command
could receive `HTTPS_PROXY=http://127.0.0.1:43123` while Seatbelt still
denied access to port `43123`.

## What changed

`NetworkProxy` now prepares the command environment and sandbox context
together from the same runtime snapshot:

```text
Prepared managed network
├── command environment: HTTPS_PROXY=http://127.0.0.1:43123
└── sandbox context: allow outbound to 127.0.0.1:43123
```

That context travels with remote exec requests. The exec server
preserves the managed proxy and CA environment, and macOS Seatbelt
allows only the prepared loopback proxy ports without enabling broad
network access or local binding.

The protocol field is optional and the existing enforcement flag remains
in place, preserving compatibility with callers that do not send the new
context.
2026-06-23 20:07:09 +01:00
Celia Chen
3310fc8ae5 chore: warn when Code Mode lacks model metadata (#29490) 2026-06-23 01:15:11 +00:00
Owen Lin
5b95745eae core: rename metadata -> internal_chat_message_metadata_passthrough (#28968)
## Description
This PR cuts Codex over from generic `ResponseItem.metadata` (introduced
here: https://github.com/openai/codex/pull/28355) to
`ResponseItem.internal_chat_message_metadata_passthrough`, which is the
blessed path and has strongly-typed keys.

For now we have to drop this MAv2 usage of `metadata`:
https://github.com/openai/codex/pull/28561 until we figure out where
that should live.
2026-06-22 11:11:25 -07:00
jif
bd2968a4db Carry sandbox intent to remote exec servers (#29108)
## What changed

PR #29099 stopped sending the orchestrator's concrete sandbox wrapper to
a remote exec-server. Remote commands now arrive as plain native argv.

This PR adds the next piece: Codex also sends portable sandbox intent
next to that plain argv.

For a remote unified-exec command, the request can now include:

- the canonical permission profile before local workspace-root
materialization
- the sandbox cwd and workspace roots as `PathUri` values
- Windows sandbox settings
- the legacy Landlock setting
- whether managed networking must be enforced

The important part is that symbolic entries such as `:workspace_roots`
stay symbolic while crossing the boundary. The executor can then bind
them to its own workspace-root paths instead of receiving
orchestrator-local absolute paths.

The data travels through `ExecRequest` into `ExecParams`. Older
exec-servers can still deserialize requests because the new fields have
defaults.

## Why

The orchestrator should not decide how another machine implements
sandboxing.

For example:

- a local macOS Codex would normally build a Seatbelt command
- a remote Linux executor needs a Linux sandbox command instead

The orchestrator now sends the plain command plus the policy it intended
to enforce. A later PR can let the exec-server choose and build the
correct sandbox for its own operating system.

## Important detail

This keeps the portable intent separate from the local `SandboxType`.

`SandboxType::None` is ambiguous:

- it can mean the command was explicitly approved to run without a
sandbox
- it can also mean the orchestrator host has no concrete sandbox
implementation available

Those cases are different for remote execution. This PR adds
`sandbox_requested` so an executor can still receive sandbox intent when
the orchestrator cannot build a local wrapper. Explicit unsandboxed
retries still send no sandbox context.

## Behavior today

This PR only transports the intent. The exec-server accepts the new
fields but does not apply them yet.

Remote commands therefore remain unsandboxed after this PR, just as they
are after PR #29099.

## Follow-up

The next PR will make exec-server read this portable intent, bind
symbolic workspace permissions to executor-native roots, choose the
sandbox for its own operating system, build the wrapper locally, and
then spawn the command.
2026-06-21 12:33:21 +02:00
rka-oai
dac588f413 [codex] abort turns when rollout budgets expire (token budget 3/3) (#28707)
## Stack

Depends on #28494.

## Description

This PR propagates shared rollout-budget exhaustion through the existing
`CodexErr::TurnAborted` task result.

Each thread records its model usage against the same ledger. Once the
ledger is exhausted, that usage update and all later usage updates
return `TurnAborted`. The task wrapper emits the normal aborted-turn
event and lifecycle instead of completing the turn.

This is intentionally a soft boundary: there is no cross-thread
`Op::Interrupt` fanout. An in-flight thread can finish its current
response before it observes the exhausted ledger, but every thread
aborts at its next usage-accounting boundary.

## Tests

The integration coverage verifies that:

- the response that exhausts the budget aborts its turn;
- a later response also aborts because the shared ledger remains
exhausted; and
- sub-agent usage draws from the same shared ledger; and
- local and remote-v2 compaction abort without retrying or emitting a
generic error.

Local checks:

- `just test -p codex-core
exhausted_budget_aborts_current_and_later_turns`
- `just test -p codex-core subagent_usage_draws_from_the_shared_budget`
- `just test -p codex-core
abort_regular_task_emits_marker_before_turn_aborted`
- `just test -p codex-core
compaction_budget_exhaustion_aborts_without_error_or_retry`
- `just fix -p codex-core`
- `just fmt`
- `git diff --check`

The full workspace test suite was not run locally.
2026-06-19 02:00:01 -07:00
jif
0369b24d54 Add network environment ID plumbing (#28766)
## Why

Prepare network approval scoping to distinguish execution environments
without changing behavior yet.

## What changed

- Add optional environment IDs to network policy requests.
- Add optional network environment IDs to exec and sandbox request
structs.
- Thread default None values through existing construction points.
- Fix stale constructor call sites that caused the CI compile failures.

## Not included

- Per-environment proxy listeners.
- Network approval cache or prompt behavior changes.
- Ambiguous request attribution handling.

Those behavior changes moved to stacked follow-up #28899.

## Validation

- just fmt
- CI will run tests and clippy
2026-06-18 14:09:38 +02:00
Adam Perry @ OpenAI
3931bc2bde unified-exec: retain PathUri in command events (#28780)
## Why

App-server must report command events containing foreign-platform paths
without changing existing client or rollout path-string formats.

## What changed

- retain `PathUri` through exec command begin/end events
- convert cwd values to `LegacyAppPathString` at the app-server
compatibility boundary
- drop command actions with foreign paths and log them
- serialize rollout-trace cwd values using their inferred native path
representation
- restore Wine coverage for retained Windows cwd values and successful
completion
2026-06-18 05:00:04 +00:00
Adam Perry @ OpenAI
5867b529ae unified-exec: preserve PathUri through exec-server (#28681)
## Why

It should be possible for app-server to handle "foreign" OS paths in
unified_exec working directories, allowing e.g. a Linux app-server to
run processes on e.g. a Windows exec-server.

## What

Convert the core unified_exec cwd values to use `PathUri`.

Adds fallible path conversion in several places to try to minimize the
scope of this change. The only time this change suppresses errors from
converting `PathUri` to an `AbsolutePathBuf` is when the turn is
configured with no sandboxing at all to allow us to make progress
testing without sandboxing.

Future changes to apply_patch and sandboxing will clean up these error
paths.

A tool's cwd is resolved from joining a model-provided workdir to the
environment's cwd. When using `AbsolutePathBuf::join()`, an
absolute-path workdir would overwrite the environment's cwd and we would
resolve permissions/sandboxing against the model-provided path. This
change extends `PathUri::join()` to also treat an absolute rhs as an
override of the base/lhs.

This also removes some coverage from the remove_env_windows tests until
a follow-up converts foreign paths in command exec events correctly.

## Breaking Changes

When using `AbsolutePathBuf::join()` for workdir resolution, we ended up
resolving tilde-prefixed paths against the app-server's `$HOME`, e.g.
`~/foo/bar` becomes `/home/anp/foo/bar`. It's difficult to do this with
`PathUri` joining, so after offline discussion this PR no longer
implements it.

A quick check of some power users' rollouts suggests that models don't
actually generate home-prefixed absolute working directories for their
spawns, so this shouldn't have any real blast radius.
2026-06-17 19:36:16 +00:00
pakrym-oai
172b2218a5 core: remove redundant TurnContext and Prompt fields (#28638)
## Why

`TurnContext` had accumulated dead fields and cached projections of
values already owned by its per-turn `Config` or `ModelInfo`. Keeping
both copies made ownership unclear and allowed artificial split-brain
states, such as a compatibility hash differing from the model metadata
it came from.

`Prompt` similarly carried a write-only personality after personality
selection had already been materialized into its base instructions.

This makes the canonical owner explicit: configuration-backed values
come from `config`, model-derived values come from `model_info`, and
prompts contain only data consumed by request construction.

## What changed

- Remove the unused `ghost_snapshot`, `codex_self_exe`, and
`thread_source` fields.
- Remove duplicate `comp_hash`, `truncation_policy`, `features`,
`shell_environment_policy`, `codex_linux_sandbox_exe`, `compact_prompt`,
and `tool_mode` fields.
- Read those values directly from `TurnContext::config` or
`TurnContext::model_info` at their consumers.
- Remove the write-only `Prompt::personality` field and its constructor
assignments.
- Preserve review-turn inheritance of the parent turn's shell policy,
Linux sandbox executable, and compact prompt through the review config.

## Testing

- `cargo check -p codex-core --tests`
2026-06-16 22:17:24 -07:00
Adam Perry @ OpenAI
f8850cab1d app-server: preserve target-native environment cwd (#28146)
## Why

app-server may run on a different OS from the selected exec-server
environment. Parsing that environment’s cwd with the Codex host’s path
rules prevents thread startup.

## What

Carry environment cwd values as `LegacyAppPathString` at the app-server
boundary and `PathUri` internally. Existing tool-call schemas and
relative-path behavior stay host-native; remaining local-only consumers
convert explicitly and leave follow-up TODOs.

The Wine integration test verifies app-server can start a thread and
complete an ordinary turn with a Windows environment cwd from Linux.

## Validation

- `bazel test //codex-rs/core/tests/remote_env_windows:smoke-test
--test_output=errors`
- focused app-server environment-selection and protocol schema tests
- scoped Clippy for `codex-core` and `codex-app-server-protocol`
2026-06-16 21:42:28 +00:00
pakrym-oai
1e015884c5 [codex] Use local environment for user shell commands (#28163)
## Why

User shell commands still read the legacy turn cwd and session shell
even though execution context is now owned by selected turn
environments. App-server also defines `thread/shellCommand` as a
local-host escape hatch, so it must use an available local environment
even when a remote environment is primary.

## What changed

- Add `ResolvedTurnEnvironments::local()` to find the selected local
environment.
- Resolve the user shell command cwd and shell from that local
`TurnEnvironment`.
- Emit the standard `shell is unavailable in this session` error when no
selected local environment or resolved local shell is available.
- Add an integration test covering `/shell` without a local environment.

## Test plan

- `just test -p codex-core
user_shell_command_without_local_environment_emits_error`
2026-06-16 04:55:20 +00:00
pakrym-oai
022f1221e8 [codex] Bind shell snapshots to retained thread environments (#28421)
## Why

Shell snapshots are currently session-scoped even though shell and cwd
are properties of a selected turn environment. That makes snapshot
refresh depend on separate session-cwd plumbing, prevents retained
environments from retaining their snapshot work, and can make snapshot
construction use a different shell than command execution.

This follows #27955 by making the retained thread-environment service
own environment snapshot lifecycles. Session configuration remains the
requested selection state, while `ThreadEnvironments` remains the source
of successfully resolved environments.

## What changed

- Configure the shell-snapshot builder before initial environment
resolution.
- Start each local environment snapshot task when its `TurnEnvironment`
is built and retain that shared task while environment ID and cwd still
match.
- Inherit retained environment snapshots into spawned child threads.
- Carry the selected `TurnEnvironment` through shell runtimes so
snapshot construction and command execution use the same
environment-specific shell and cwd.
- Load project instructions and warm plugins/skills after initial
environment resolution.
- Continue decoding invalid UTF-8 instruction files lossily without
emitting a startup warning.
- Keep requested selections in `SessionConfiguration`; failed or
duplicate resolutions only affect the resolved environment snapshot.

## Validation

- `cargo check -p codex-core --tests`
- `just test -p codex-home instructions` (6 passed)
- Focused environment, instruction, shell-snapshot, and user-shell tests
(84 passed)
- Focused shell-snapshot, user-shell, and unified-exec tests (126
passed; two event-timing tests passed on retry)
2026-06-15 20:10:53 -07:00
Owen Lin
040dafa32d feat(core): add metadata field to ResponseItem (#28355)
## Description

This PR adds an optional `metadata` field to `ResponseItem` for
Responses API calls. Only mechanical plumbing, no actual values
populated and sent yet. Turns out just adding a new field to
`ResponseItem` has quite a large blast radius already.

This change is backwards compatible because `metadata` is optional and
omitted when absent, so existing response items and rollout history
without it still deserialize and requests that do not set it keep the
same wire shape. For provider compatibility, we strip out `metadata`
before non-OpenAI Responses requests so Azure and AWS Bedrock never see
this field.

My followup PR here will actually make use of it to start storing and
passing along `turn_id`: https://github.com/openai/codex/pull/28360

## What changed

- Added `ResponseItemMetadata` with optional `turn_id`, plus optional
`metadata` on Responses API item variants and inter-agent communication.
- Preserved item metadata through response-item rewrites such as
truncation, missing tool-output synthesis, compaction history
rebuilding, visible-history conversion, rollout/resume, and generated
app-server schemas/types.
- Strip item metadata from non-OpenAI Responses requests while
preserving it for OpenAI-shaped requests.
- Updated the mechanical fixture/test construction churn required by the
new optional field.
2026-06-15 15:05:28 -07:00
pakrym-oai
1aa3e432cc [codex] simplify shell snapshot ownership (#27756)
## Why

Shell snapshot lifecycle state was split between `Shell` and
`SessionServices`: `Shell` carried the receiver while session code
exposed and forwarded the raw sender. That coupled shell identity to
mutable snapshot state and made refresh, inheritance, and file lifetime
harder to reason about.

## What changed

- make each `Arc<ShellSnapshot>` represent one cwd-specific snapshot
generation
- store the active generation in `SessionServices` with `ArcSwapOption`
- have construction start the background build and expose only a
cwd-validated snapshot path
- use `ShellSnapshotFile` ownership to delete snapshot files
automatically
- pass snapshot paths explicitly to shell runtimes instead of storing
snapshot state on `Shell`
- preserve inherited and in-flight generations by pinning their `Arc`
while they are in use

## Test plan

- `cargo check -p codex-core --lib`
- `just test -p codex-core 'shell_snapshot::tests'`
- `just test -p codex-core
shell_command_snapshot_still_intercepts_apply_patch`
- `just test -p codex-core
shell_snapshot_deleted_after_shutdown_with_skills`
2026-06-15 08:18:13 -07:00
jif
8f2d6416ce Support plaintext agent messages (#27830)
## Why

Multi-agent v2 `send_message` deliveries already reach the receiving
model as typed `agent_message` items with encrypted content.
Child-completion notifications are generated by Codex itself, so their
content is plaintext and previously fell back to a serialized JSON
envelope inside an assistant message.

With plaintext `input_text` supported for `agent_message`, both delivery
paths can use the same model-visible type while preserving explicit
author and recipient metadata.

## What changed

- add plaintext `input_text` support to `AgentMessageInputContent` and
regenerate the affected app-server schemas
- preserve `InterAgentCommunication` as structured mailbox input instead
of converting it to assistant text
- record delivered communications as typed `agent_message` history items
- persist a dedicated rollout item so local delivery metadata such as
`trigger_turn` remains available without leaking into the Responses
request
- reconstruct typed agent messages on resume and preserve fork-turn
truncation behavior
- remove request-time assistant-content parsing
- preserve plaintext and encrypted inter-agent deliveries in stage-one
memory inputs
- normalize and link plaintext and encrypted agent messages in rollout
traces without treating inbound messages as child results
- cover the real MultiAgent V2 child-completion path end to end with
deterministic mailbox synchronization

## Verification

- `just test -p codex-core
plaintext_multi_agent_v2_completion_sends_agent_message`
- `just test -p codex-core input_queue_drains_mailbox_in_delivery_order
record_initial_history_reconstructs_typed_inter_agent_message
fork_turn_positions_use_inter_agent_delivery_metadata`
- `just test -p codex-memories-write
serializes_inter_agent_communications_for_memory`
- `just test -p codex-rollout-trace
agent_messages_preserve_routing_and_content
sub_agent_started_activity_creates_spawn_edge`
- `just test -p codex-rollout-trace
agent_result_edge_falls_back_to_child_thread_without_result_message`
- `just test -p codex-protocol -p codex-rollout -p
codex-app-server-protocol`
2026-06-12 13:50:04 -07:00
mchen-oai
92d9036910 Add spans to turn lifecycle gaps (#27623)
## Why
Codex app-server latency traces do not granularly cover turn task
startup and inter-request handoffs. These spans help attribute time
across task execution, startup prewarm, in-flight tool completion, and
rollout persistence.

## What changed
- Add `session_task.run` spans around task execution and
`session_task.flush_rollout` around flushing pending conversation
transcript writes to durable storage
- Add `regular_task.prepare_run_turn` around regular-turn startup (Send
the `TurnStarted` event, reset turn-specific reasoning state, and
resolve any startup prewarm)
- Add `startup_prewarm.resolve` around waiting for background session
prewarming to finish, fail, time out, or be cancelled
- Add a function-level trace span around draining in-flight tool calls
(Wait for tool calls to complete, record tool result in conversation
history, and other bookkeeping)

## Verification
Trigger Codex rollout and observe new spans are included
2026-06-11 16:55:01 -07:00
Eric Traut
a1a8807e9d Add app-server background terminal process APIs (#26041)
## Summary

Codex Apps needs app-server as the source of truth for chat-started
background terminals instead of guessing from local process trees.

This PR adds experimental v2 APIs to list and terminate background
terminals for a loaded thread using app-server process ids, so clients
can manage background terminals without local PID discovery.

## Changes

- `thread/backgroundTerminals/list` returns paginated background
terminal records with `itemId`, app-server `processId`, `command`,
`cwd`, nullable `osPid`, nullable `cpuPercent`, and nullable `rssKb`.
- `thread/backgroundTerminals/terminate` terminates one running
background terminal by app-server `processId` and returns whether a
process was terminated.
- Background terminal list and terminate operations use unified-exec
process manager state as their source of truth.
2026-06-10 11:18:09 -07:00
jif
743f5aad38 feat: count V2 concurrency by active execution (#26969)
## Why

Multi-Agent V2 concurrency should count active non-root turns, not
resident or durable agent threads. The limit is intentionally best
effort: admission checks are synchronous, but concurrent successful
checks may overshoot slightly.

## What changed

- Keep one root-derived execution limit on the shared `AgentControl`.
- Count active V2 subagent turns with an RAII guard owned by
`RunningTask`.
- Check capacity before spawning or starting an idle agent, including
direct app-server `turn/start` submissions.
- Preserve queued delivery for agents that are already running.
- Exempt automatic idle continuations so `/goal` work is not dropped
when capacity is temporarily full.
- Keep root and V1 turns outside this limiter.

## Test coverage

- `execution_guards_count_active_v2_subagent_turns`
- `execution_guards_ignore_root_and_v1_turns`
- `v2_nested_spawn_checks_shared_active_execution_capacity`
2026-06-08 14:21:28 +02:00
Eric Traut
479a14cf59 [2 of 2] Finish moving goal runtime to extension (#26548)
## Stack

1. [#26547](https://github.com/openai/codex/pull/26547) - [1 of 2] Align
goal extension with core behavior
2. [#26548](https://github.com/openai/codex/pull/26548) - [2 of 2] Move
goal runtime to extension

## Why

This PR completes the switch of the goal behavior to the
extension-backed runtime and removes the old core goal implementation.

## What Changed

- Installs the goal extension for app-server `ThreadManager` sessions.
- Routes app-server thread goal `get`, `set`, and `clear` through
`GoalService`.
- Uses thread-idle lifecycle emission after goal resume and snapshot
ordering so the extension can decide whether to continue the goal.
- Forwards extension goal updates through a FIFO async app-server
notification path so backpressure does not drop them or reorder updates.
- Keeps review turns from enabling goal runtime behavior.
- Plans extension tools before dynamic tools so built-in goal tool names
keep their old precedence when goals are enabled.
- Removes the old core goal runtime, core goal tool handlers, and core
goal tool specs.
- Updates tests that were coupled to the core-owned goal runtime while
leaving the legacy `<goal_context>` compatibility path in core for old
threads.
- Removes the stale cargo-shear ignore now that `codex-goal-extension`
is used by the workspace.
- Keeps realtime event matching exhaustive after removing the old
goal-specific realtime text path.


## Validation

- Ran manual `/goal` runs in TUI. Validated time accounting matched
wall-clock time and goal lifecycle state transitions.
2026-06-05 14:17:30 -07:00
Ahmed Ibrahim
8d72fb6de9 [codex] Add turn profiling analytics (#26484)
## Summary

Add flat profiling fields to `codex_turn_event` so analytics can explain
where turn wall-clock time is spent without changing tool execution
behavior.

The profile reports:
- time before the first sampling request
- sampling time across all attempts and follow-ups
- overhead between sampling requests
- time blocked in the post-sampling tool drain
- time after the final sampling request
- sampling request and retry counts

## Implementation

- Extend the existing turn timing state with constant-memory phase
accounting and one RAII phase guard.
- Observe sampling and the existing post-sampling drain only at turn
orchestration boundaries.
- Keep tool runtime, tool futures, response item handling, and turn
lifecycle values unchanged.
- Add the profiling fields directly to the existing analytics turn event
without changing app-server protocol or rollout persistence.
- Use the existing turn `status` to distinguish completed, failed, and
interrupted profiles.

Exact sampling/tool overlap is intentionally omitted because measuring
tool completion accurately would require hooks in the tool execution
path.

## Validation

- Add app-server end-to-end coverage for a single-sampling turn with no
blocking tool work.
- Add app-server end-to-end coverage for `request_user_input` blocking
followed by a second sampling request.
- CI is running on the PR; tests were not executed locally per
repository guidance.
2026-06-05 11:27:10 -07:00
Michael Bolin
6bcccb0ee6 cli: add package path from install context (#26189)
## Why

Codex package installs include helper binaries in `codex-path`, such as
the bundled `rg`. Package-layout launches should add that directory
before user commands run, but standalone launches were missing it while
npm launches only worked because `codex.js` had its own legacy `PATH`
rewrite. That made npm and standalone package behavior diverge.

Shell snapshot restoration can also reset `PATH` after runtime setup.
Any package-owned `PATH` prepend has to be recorded as an explicit
runtime override so shells, unified exec, and user-shell commands keep
access to `codex-path` after a snapshot is sourced.

## Repro

Before this change, a curl-installed package could contain `rg` under
`codex-path` but still fail to put it on `PATH`:

```shell
mkdir /tmp/test-codex-curl
curl -fsSL https://chatgpt.com/codex/install.sh \
  | CODEX_HOME=/tmp/test-codex-curl CODEX_NON_INTERACTIVE=1 sh
/tmp/test-codex-curl/packages/standalone/current/bin/codex exec \
  --skip-git-repo-check 'print `which -a rg`'
find /tmp/test-codex-curl -name rg
```

The `which -a rg` output omitted the packaged helper even though `find`
showed it under
`/tmp/test-codex-curl/packages/standalone/releases/.../codex-path/rg`.

The npm install path behaved differently only because
`codex-cli/bin/codex.js` had legacy `PATH` rewriting:

```shell
mkdir /tmp/test-codex-npm
cd /tmp/test-codex-npm
npm install @openai/codex
./node_modules/.bin/codex exec --skip-git-repo-check 'print `which -a rg`'
```

That printed the npm package's `vendor/<target>/codex-path/rg` first.
This PR moves that behavior into Rust-side package launch setup so
curl/standalone and npm/bun launches agree without JS rewriting `PATH`.

## What Changed

- `codex-rs/arg0` now uses
`InstallContext::current().package_layout.path_dir` to prepend the
package helper directory before any threads are created.
- Package helper `PATH` setup is independent from the temporary arg0
alias setup, so `codex-path` is still added even if CODEX_HOME tempdir,
lock, or symlink setup fails.
- `codex-rs/install-context` detects the canonical package layout we
ship: `bin/`, `codex-resources/`, and `codex-path/` next to
`codex-package.json`.
- Shell, local unified exec, and user-shell runtimes now record package
`codex-path` prepends in `explicit_env_overrides`, matching the existing
zsh-fork behavior so shell snapshots cannot restore over the package
helper path.
- Remote unified exec requests do not receive the local app-server
package path overlay.
- `codex-cli/bin/codex.js` no longer computes or overrides `PATH`; it
only locates the native binary in the canonical package layout and
passes npm/bun management metadata.
- Added regression tests for `PATH` ordering, package layout detection,
and shell snapshot preservation of package path prepends.

## Verification

- `node --check codex-cli/bin/codex.js`
- `just test -p codex-install-context -p codex-arg0`
- `just test -p codex-core
user_shell_snapshot_preserves_package_path_prepend`
- `just test -p codex-core tools::runtimes::tests`
- `just bazel-lock-update`
- `just bazel-lock-check`
- `just fix -p codex-install-context -p codex-arg0 -p codex-core`
2026-06-03 19:08:19 -07:00
jif
c9ae0f48a1 chore: mechanical rename (#26156)
Rename `Session::conversation_id` to `Session::thread_id` with an auto
refactor in RustRover
2026-06-03 15:38:30 +02:00
jif-oai
bf9fd885b2 Resolve per-thread multi-agent runtime (#25722)
Stack split from #25708. Original PR intentionally left open. This third
PR resolves the effective per-thread multi-agent runtime from persisted
metadata, inherited runtime, and current model selection.
2026-06-02 14:31:00 +02:00
Winston Howes
bca18cba40 Wire managed MITM CA trust into child env (#22668)
## Stack
1. Parent PR: #18240 uses named MITM permissions config.
2. This PR wires managed MITM CA trust into spawned child processes.

## Why
When Codex terminates HTTPS for limited mode or MITM hooks, child HTTPS
clients need to trust Codex's managed MITM CA. Exporting proxy URLs
alone is not enough, but blindly replacing user CA settings would be
wrong: it can break custom enterprise/test roots, leak unreadable CA
files into generated bundles, or make the child env disagree with its
sandbox policy.

## Summary
1. Build immutable managed CA bundles under `$CODEX_HOME/proxy` that
include native roots, the managed MITM CA, and only inherited or
command-scoped CA bundles the child is allowed to read.
2. Export curated CA env vars alongside managed proxy env vars while
preserving user CA override semantics, including nested Codex
`SSL_CERT_FILE` precedence.
3. Thread generated CA bundle paths into child sandbox readable roots,
including debug sandbox execution, so the exported env vars work inside
sandboxed commands.
4. Remove only Codex-generated MITM CA bundle env when a child
intentionally drops managed proxying for escalation or no-proxy retry.
5. Document the managed CA bundle behavior and cover env injection,
per-child bundle generation, sandbox readable roots, and no-proxy
cleanup in tests.

## Validation
1. Ran `just test -p codex-network-proxy`.
2. Ran `just test -p codex-protocol`.
3. Ran `just fix -p codex-network-proxy -p codex-protocol`.
4. Tried focused `codex-core` validation, but the crate currently fails
to compile in `core/tests/suite/guardian_review.rs` because an existing
`Op::UserInput` initializer is missing `additional_context`.

---------

Co-authored-by: Eva Wong <evawong@openai.com>
2026-06-01 23:23:59 +00:00