mirror of
https://github.com/openai/codex.git
synced 2026-10-20 17:38:45 +00:00
dev/stevenlee/tool-search-app-server-api
3933
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Steven Lee
|
caef31349a | Add tool search test argument comments | ||
|
Steven Lee
|
01b752dc25 | Keep tool search debug results internally consistent | ||
|
Steven Lee
|
f1825e8741 | Add app-server toolSearch debug RPC | ||
|
Adam Perry @ OpenAI
|
a86d525e4d |
core: trace executor skill discovery (#30318)
## Why Make it easier to measure the performance of different parts of skill loading. ## What - Add spans for step-context capture, world-state construction, executor catalog snapshot/root loading, and environment skill loading. - Record the discovered environment skill count. - Trace outbound exec-server requests with client kind and RPC method fields. - Update trace propagation tests to assert that requests keep the parent trace id while creating their own child span. |
||
|
Channing Conger
|
84fe70c30e |
elicitations: Move to shared ElicitationService (#30627)
## Why Code-mode tool results could return to the model while an MCP elicitation was still waiting for user input. This differed from parallel tool calling and could let the model continue before the user resolved the request. We need one session-level view of outstanding elicitations so tool runtimes can consistently hold results until every pending elicitation is resolved. ## What changed - Added a counted, session-owned ElicitationService with RAII registrations. - Registered both core-originated and server-originated MCP elicitations with the service. - Migrated out-of-band elicitation tracking and unified exec timeout pausing to the shared service. - Made code-mode functions.exec and functions.wait capture their runtime result normally, then hold it before returning while an elicitation is outstanding. - Kept terminate: true immediate; only its result is held. - Preserved model-visible wall time across the elicitation hold. - Kept the behavior session-scoped, with concurrent elicitations holding the pause until all registrations are released. |
||
|
Alexi Christakis
|
7b4e70d567 |
Revert "[core] Support interleaved response items" (#31261)
Reverts openai/codex#30876 |
||
|
Francis Chalissery
|
7094fa467e |
[codex] Read retry model from buffering events (#31262)
## Summary - deserialize `retry_model` from streamed `safety_buffering` payloads - preserve the existing downstream faster-model API and legacy header fallback - update SSE, WebSocket, and end-to-end safety-buffering coverage ## Root cause Follow-up to #31064. The Responses API emits the retry target as `retry_model`, but the client was looking for `faster_model`, so the payload value was ignored in favor of the compatibility fallback. ## Behavior A non-null `retry_model` from the buffering payload takes precedence. An explicit null leaves the retry target unset, while an omitted field continues to fall back to the existing response header. ## Validation - `just test -p codex-api` (135 tests) - `just test -p codex-core safety_buffering` (2 tests) - `just fix -p codex-api` - `just fmt` - `git diff --check` |
||
|
Alexi Christakis
|
8917244f7d |
[core] Support interleaved response items (#30876)
## Summary - preserve reasoning item IDs on summary part and text delta events - track streamed response items by ID so reasoning summaries can continue after later items begin - keep TUI output complete and deduplicated when reasoning and final-answer events interleave ## Stack - **1/2: this PR — interleaved item support** - 2/2: #30752 — wire reasoning summary delivery configuration through the CLI and app-server ## Validation - just test -p codex-api preserves_reasoning_summary_item_ids - just test -p codex-core interleaved_reasoning_summary_events_keep_reasoning_item_metadata - just test -p codex-tui live_reasoning_summary_is_not_rendered_twice_when_item_completes - just fix -p codex-api -p codex-core -p codex-tui - just fmt |
||
|
Charlie Marsh
|
bce481fdcb |
Fix cancelled review leaving MCP startup busy (#31189)
## Summary Cancelling an inline review could leave the TUI stuck in MCP startup state, so subsequent `/review` commands were rejected as though another task were still running. Specifically, the child’s “Starting MCP servers” event was incorrectly forwarded to the parent TUI, so the parent was marking itself as busy. This keeps delegate-session MCP startup events inside the delegate instead of exposing them as parent-session state. ## Reproduction Here's the before -- notice that after I cancel the first review, I'm prevented from running `/review` again: https://github.com/user-attachments/assets/571a0793-3253-4bcc-8f10-3782d176162f And the after -- notice that after I cancel, I can immediately start a new `/review`: https://github.com/user-attachments/assets/dd98b1d7-6a71-4a1f-a4b2-3295bc4848e5 |
||
|
sayan-oai
|
9c5be7e1d5 |
Make plugin guidance react to environment readiness (#30223)
## Why Generic plugin guidance is currently emitted only with initial context from host plugin state. An executor-selected plugin can become available later in the same turn, making its skills and tools usable without ever telling the model how plugin capabilities should be used. ## What - project every ready selected plugin package, including skill-only plugins - carry plugin availability with the exact MCP runtime projection while preserving MCP manager reuse when servers and connectors are unchanged - move generic plugin guidance from the static initial-context path into persisted World State - recognize legacy and retained plugin fragments so resume and compaction do not duplicate guidance ## Testing - `just test -p codex-mcp-extension` - `just test -p codex-core plugins_instructions` - `just test -p codex-core plugin_availability_change_reuses_the_mcp_manager` - `just test -p codex-app-server --test all selected_capabilit` |
||
|
Francis Chalissery
|
be33f80bc6 |
[codex] Read buffering metadata from response events (#31064)
## Summary - read optional faster-model metadata from streamed buffering payloads - use the buffering payload itself to determine whether buffering UI should be shown - retain the existing header value as a compatibility fallback when the payload omits the field ## Behavior An object-valued buffering signal now enables the buffering UI. The response event's faster-model field takes precedence when present, while omitted fields fall back to existing response metadata. An explicit null leaves the retry target unset. ## Validation - `just test -p codex-api` - `just fix -p codex-api` - `cargo fmt --all -- --check` - `git diff --check` |
||
|
Shijie Rao
|
da4c8ca57d |
[codex] Add configurable multi-agent mode hint text (#30493)
## Why
Multi-agent V2 normally derives its mode instructions from reasoning
effort: Ultra enables proactive delegation, while other efforts require
an explicit request. Some deployments need to provide one configured
delegation policy that replaces those built-ins and remains stable when
reasoning effort changes.
## What changed
- Add `features.multi_agent_v2.multi_agent_mode_hint_text` alongside the
existing root and subagent hint settings.
- Treat any configured value, including an empty string, as
`MultiAgentMode::Custom(hint_text)`, so the configured text replaces the
built-in explicit-only and proactive policies.
- Persist the full custom variant and hint text in the turn-context
snapshot, so the durable comparison baseline detects both
reasoning-effort changes and configured policy-text changes.
- Preserve the existing explicit-only/proactive behavior when the
setting is absent.
- Replace the ambiguous `MultiAgentMode::None` variant with
`MultiAgentMode::Custom(String)` in new rollouts and API schemas. A
compatibility wire type maps legacy serialized `none` values to
`Custom("")` when resuming existing rollouts.
- Regenerate the config and app-server schemas.
## Configuration examples
The distinction is whether `multi_agent_mode_hint_text` is present. An
empty string is still a configured value and intentionally suppresses
the built-in mode instructions.
### Unset: preserve existing effort-derived behavior
```toml
[features.multi_agent_v2]
enabled = true
# multi_agent_mode_hint_text is omitted
```
- Ultra reasoning uses the built-in proactive delegation instructions.
- Other reasoning efforts use the built-in explicit-request-only
instructions.
### Empty: suppress all mode hint text
```toml
[features.multi_agent_v2]
enabled = true
multi_agent_mode_hint_text = ""
```
This selects effective mode `custom` at every reasoning effort and
injects an empty mode body, suppressing both built-in policies.
### Set: always use the configured text
```toml
[features.multi_agent_v2]
enabled = true
multi_agent_mode_hint_text = "Delegate to subagents when it will materially improve the result."
```
This selects effective mode `custom` at every reasoning effort and
injects the configured text verbatim instead of either built-in policy.
## Verification
- `just test -p codex-core multi_agent_mode`
- Covers a configured hint across High and Ultra reasoning efforts and
verifies the full custom hint is recorded for both turns.
- Covers an empty-string override suppressing both built-in instruction
bodies.
- `just test -p codex-protocol -p codex-app-server-protocol`
- Covers legacy `none` turn-context deserialization as `Custom("")` and
verifies the regenerated schemas.
|
||
|
Michael Bolin
|
beca198b8a |
telemetry: log structured direct tool-call timing (#30334)
## Why App-server deployments can consume structured JSON logs for operational measurements without requiring an OTEL exporter. Existing tool-result telemetry reports the handler outcome, but it does not separate time spent waiting to dispatch from time spent executing the handler. A compact completion event for the outer, direct tool call lets consumers measure those phases and correlate them with a conversation and turn. Code-mode calls are intentionally excluded so nested runtime calls do not create overlapping events that are easy to double-count. ## What changed - Added a [`ToolCallTimingGuard`](https://github.com/openai/codex/blob/141110a73cff041c42fd51f47e363e83df1dde51/codex-rs/core/src/tools/parallel.rs#L32) around direct tool calls. Event-only strings and timing state are captured only when the `codex_core::tools::parallel` `INFO` target is enabled. - Added a [`codex.tool_call`](https://github.com/openai/codex/blob/141110a73cff041c42fd51f47e363e83df1dde51/codex-rs/core/src/tools/parallel.rs#L313) completion event with conversation, turn, tool, call, trace, dispatch, handler, and total timing fields. - Recorded the execution-start marker after the dispatch lock is acquired. Event emission snapshots that marker once so a concurrently starting dispatch cannot produce internally inconsistent fields. - Limited the event to `ToolCallSource::Direct`; [unit coverage](https://github.com/openai/codex/blob/141110a73cff041c42fd51f47e363e83df1dde51/codex-rs/core/src/tools/parallel.rs#L365) verifies code-mode calls are ignored. - Added [cancellation coverage](https://github.com/openai/codex/blob/141110a73cff041c42fd51f47e363e83df1dde51/codex-rs/core/src/tools/parallel.rs#L408) that holds the execution gate and verifies a call cancelled before admission emits exactly one dispatch-only timing event. - Added reusable [`JsonLogCapture`](https://github.com/openai/codex/blob/141110a73cff041c42fd51f47e363e83df1dde51/codex-rs/app-server/tests/common/json_logging.rs#L15) support, a [JSON-logging-specific `TestAppServer` constructor](https://github.com/openai/codex/blob/141110a73cff041c42fd51f47e363e83df1dde51/codex-rs/app-server/tests/common/test_app_server.rs#L172), and an [end-to-end app-server test](https://github.com/openai/codex/blob/141110a73cff041c42fd51f47e363e83df1dde51/codex-rs/app-server/tests/suite/logging.rs#L52) that drives a direct `exec_command` through the public v2 JSON-RPC API and validates the emitted JSON event. Exec-server-specific request and process timing remains in the stacked PR #30901. ## Suggested logging filter ```bash LOG_FORMAT=json \ RUST_LOG='warn,codex_core::tools::parallel=info' \ codex app-server ``` ## Event example Identifier and timing values are illustrative. ### `codex.tool_call` ```json { "timestamp": "2026-06-27T03:45:20.443Z", "level": "INFO", "fields": { "message": "tool call completed", "event.name": "codex.tool_call", "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736", "conversation.id": "67e55044-10b1-426f-9247-bb680e5fe0c8", "turn_id": "019f04f8-6ac2-78f1-8625-f04a6d35af18", "tool_name": "exec_command", "call_id": "call_7b8483", "tool_source": "direct", "execution_started": true, "dispatch_duration_ms": 12, "handler_duration_ms": 431, "total_duration_ms": 443 }, "target": "codex_core::tools::parallel" } ``` If execution never starts, `execution_started` is `false`, `handler_duration_ms` is `0`, and `dispatch_duration_ms` covers the full observed lifetime. If a duration cannot be represented as an unsigned 64-bit millisecond value, all three duration fields are omitted rather than populated with a sentinel that could corrupt downstream calculations. ## Test plan - `just test -p codex-core tool_call_timing_guard_ignores_code_mode_source` - `just test -p codex-core cancellation_before_dispatch_admission_logs_dispatch_only_timing` - `just test -p codex-app-server app_server_emits_structured_tool_call_timing_event` --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/openai/codex/pull/30334). * __->__ #30334 |
||
|
Dylan Hurd
|
b35d4b6b9d |
fix(websockets) ignore metadata for incremental requests (#30770)
## Summary The Responses API may or may not send back metadata on response items. So when comparing request data to the last response, we should discard it and only consider the content. This change results in a higher success rate of incremental requests. Notably, there is one subtle change to the behavior here - we are now ignoring metadata when comparing the previous request items to the new request items as well. This should be fine, since the metadata is explicitly out of scope for the comparison, regardless of whether the item is an item from the request or response. ## Testing - [x] Added unit test coverage for both cases of metadata existence and absence. |
||
|
xli-oai
|
6ff670bd03 |
[codex] emit per-request TTFT completion telemetry (#30883)
## Why Codex telemetry pipeline needs a per-request TTFT value. The existing `codex.turn_ttft` is recorded once per turn, so it cannot represent later inference requests in the same turn and can miss the beginning of hidden reasoning. This restores the low-volume per-request signal proposed in https://github.com/bk-nvidia/codex/pull/3 without bringing back per-WebSocket-event TRACE logging. ## What changed - start a timer when each mapped Responses stream begins - latch the timer on the first `response.output_item.added`, including an empty hidden-reasoning item - attach `ttft_ms` to the existing `codex.sse_event` / `response.completed` telemetry record - cover the new completion field with an integration test ## Semantics The value is per inference request, not per turn. It measures mapped-stream-to-first-output-item latency, matching the customer-proposed metric. For HTTP, the stream is already established before timing begins, so request setup and response-header latency are excluded. `response.output_item.added` is a client-visible proxy for the start of hidden reasoning; this does not claim access to the server's internal first raw-token timestamp. ## Validation - `just test -p codex-otel` (47 passed) - `just test -p codex-core process_sse_emits_completed_telemetry` (1 passed after the final timer-placement change) - attempted `just test -p codex-core`: 2,855 passed and 53 failed because of unrelated local-environment failures (missing `test_stdio_server` fixture binary, shell startup noise, and timing-sensitive tests); the focused telemetry test passed in that run as well |
||
|
Michael Bolin
|
129ea2aaf5 |
Log multi-agent communication lifecycle (#30872)
## Why [#30867](https://github.com/openai/codex/pull/30867) makes `submit_inter_agent_communication` the common outbound sink for multi-agent v2 communications. This follow-up uses that single point to log every communication lifecycle without requiring new hooks as spawn, messaging, follow-up, or result paths evolve. For each communication, the logs need to identify its type, sender and receiver threads, and content, while correlating the successful send with receipt by the destination mailbox. The logging path must not query externally supplied time providers because those calls can be expensive for app-server clients. ## What changed - Added structured `INFO` events on the OpenTelemetry-exported `codex_otel.agent_communication` target for `spawn`, `message`, `followup`, and `result` communications. - Logged successful sends from `submit_inter_agent_communication` with the communication kind, sender and receiver thread IDs, content, and submission ID. - Logged receives after the communication has been enqueued in the receiver mailbox, using the same submission ID. - Avoided time-provider calls and other asynchronous work in the logging path. - Narrowed ordinary spawn and send-input APIs to `Vec<UserInput>` so `Op::InterAgentCommunication` cannot bypass the context-bearing centralized path. The refactor does not change submission IDs, capacity checks, last-task bookkeeping, mailbox ordering, protocol types, rollout data, or model-visible context. ## Event shape Illustrative JSON representation of the two independently emitted records: ```json [ { "event.name": "codex.agent_communication", "communication_id": "019f20e1-40d1-7890-a123-456789abcdef", "kind": "spawn", "state": "send", "sender_thread_id": "019f20df-fbe1-7890-a123-456789abcdef", "receiver_thread_id": "019f20e1-3f79-7890-a123-456789abcdef", "content": "inspect the repository" }, { "event.name": "codex.agent_communication", "communication_id": "019f20e1-40d1-7890-a123-456789abcdef", "state": "receive" } ] ``` Consumers join the receive record to the send record by `communication_id` for the immutable communication metadata. ## Testing - Extended the existing end-to-end multi-agent v2 spawn test to verify content, both thread IDs, and a correlated send/receive submission ID. - Re-ran focused control and handler coverage for direct messages, follow-up tasks, and completion results. |
||
|
Michael Bolin
|
a98a21798c |
Consolidate multi-agent v2 communication sends (#30867)
## Why Multi-agent v2 communications currently use separate outbound paths: direct messages, follow-up tasks, and completion results go through `send_inter_agent_communication`, while a spawn's initial message goes through the generic input submission path. That split makes it difficult to add complete communication lifecycle logging in one place. This refactor makes `submit_inter_agent_communication` the common sink for those paths, preparing the follow-up observability work discussed in [#30516](https://github.com/openai/codex/pull/30516). ## What changed - Routed all current outbound `InterAgentCommunication` paths in `AgentControl`—direct messages, follow-up tasks, completion results, and multi-agent v2 spawn initial messages—through `submit_inter_agent_communication`. - Centralized the actual submission and last-task-message bookkeeping there, providing one place for the follow-up PR to instrument communication creation and successful enqueue. - Left non-communication input handling and the multi-agent v1 spawn flow unchanged. ## Testing - `just test -p codex-core 'agent::control::tests::'` (51 passed) - `just test -p codex-core 'suite::subagent_notifications::encrypted_multi_agent_v2_spawn_sends_agent_message_to_child'` (passed) --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/openai/codex/pull/30867). * #30872 * __->__ #30867 |
||
|
Shijie Rao
|
80f54d1266 |
[codex] Treat max as a first-class reasoning effort (#30467)
## Why The Bedrock GPT-5.6 catalog advertises `max`, but Codex treated it as an opaque custom effort. That made the reasoning picker render it as lowercase `max` while known efforts use productized labels. Making `max` a known effort aligns catalog data, parsing, and UI presentation without changing the `max` wire value or persisted representation. ## What changed - Add first-class `ReasoningEffort::Max` parsing and serialization. - Use the typed effort in the Bedrock catalog and render it as `Max` in the TUI. - Preserve forward-compatible custom-effort coverage with a genuinely unknown `future` value. ### Before <img width="559" height="124" alt="Screenshot 2026-06-28 at 12 08 47 PM" src="https://github.com/user-attachments/assets/7c43cf4f-020b-4605-9239-0a9c97eb7364" /> ### After <img width="558" height="107" alt="Screenshot 2026-06-28 at 12 09 10 PM" src="https://github.com/user-attachments/assets/b9cc5ded-c940-43b4-b024-bba25abe0a17" /> |
||
|
Ahmed Ibrahim
|
8dac605901 |
[codex] Restore v1 delegation guidance (#30511)
## Summary - restore the v1 clarification that requests for depth, research, or investigation do not authorize subagent spawning - restore guidance for keeping critical-path, urgent, tightly coupled, or difficult work local - update the focused v1 tool-search and spawn-description coverage ## Why PR #27919 simplified the v1 `spawn_agent` prompt by removing its delegation decision guidance. That left the authorization rule intact, but removed the instructions that constrained what should be delegated after spawning was authorized. Restore those guardrails while preserving later support for explicit delegation authorization from applicable AGENTS.md and skill instructions. Multi-agent v2 prompts are unchanged. ## User impact Models using the v1 multi-agent tool surface receive clearer guidance to delegate independent side work while keeping blocking work on the main rollout. ## Validation - `just fmt` - `git diff --check` - tests not run locally per repository guidance; CI will validate the focused coverage |
||
|
ani-oai
|
6b5f5743b3 |
[codex] Use model metadata for skills usage instructions (#29740)
## Summary - add a false-by-default `include_skills_usage_instructions` model metadata field - enable the field for the bundled `gpt-5.5` model metadata - consume the metadata in both core and extension skill rendering - remove hardcoded legacy-model matching and its marker plumbing |
||
|
xl-openai
|
e428a12d22 |
[codex] Enable remote plugins by default (#30297)
## Summary - enable the remote plugin feature by default - promote the remote plugin feature from under development to stable - preserve the existing `features.remote_plugin` override for explicitly disabling it - keep legacy disabled-path coverage explicit in TUI and app-server tests ## Impact Remote plugin functionality is enabled by default for configurations that do not set the feature flag. The existing Codex backend authentication gate still applies. ## Validation - `just fmt` - `just test -p codex-features` - `just test -p codex-tui plugins_popup_remote_section_fallback_states_snapshot` - targeted `codex-app-server` plugin-list and skills-list tests - `git diff --check` The full TUI and app-server suites were also exercised locally. All remote-plugin-related coverage passed; unrelated local sandbox/test-binary failures remain outside this change. |
||
|
Michael Bolin
|
d2885dc3cd |
core: stabilize synthesized call output IDs (#30327)
## Why Response item IDs represent stable conversation identity. `ContextManager::for_prompt` repairs an unmatched call by synthesizing an `"aborted"` output in the disposable prompt projection, but that output previously had no ID. Assigning a fresh ID on every prompt build would make retries and resumes change otherwise identical model context and reduce prompt-cache reuse. The concrete bug is that these normalization-created outputs bypass the regular item-ID allocation path. Even with item IDs enabled, a prompt could therefore contain an identified call paired with a synthetic output whose `id` was missing. This change closes that gap by deriving the output ID from the source call's item ID. For legacy calls that have no item ID, the output remains ID-less because there is no stable source identity to derive from. The originating call already has a stable item ID under the item-ID model introduced in #28814. A prompt-only output can therefore derive stable identity from that call without mutating canonical history or persisted rollouts. This addresses the failure exposed by #30311 while keeping normalization read-only outside its detached prompt snapshot. UUIDv5 is intentional here because it is the standard namespaced, deterministic UUID construction. Using the output kind and source call ID as the name produces the same UUID on every projection while keeping output kinds in separate name domains. UUIDv7 would introduce randomness and time, so keeping it stable would require persisting the synthetic repair. UUIDv5 uses SHA-1 internally, but this is only an identity mapping—not an authenticity or security boundary. ## What changed - Derive a deterministic UUIDv5 ID for each synthesized call output from the source call item ID. - Use the Responses API prefix appropriate for function, custom-tool, tool-search, and local-shell outputs. - Preserve the existing insertion position immediately after the unmatched call. - Keep synthesized outputs prompt-only; no rollout, task-lifecycle, compaction, or raw-response behavior changes. ## Testing - `just test -p codex-core for_prompt_assigns_stable_id_to_synthetic_output_without_reordering_history` - `just test -p codex-core synthetic_call_output_id_is_stable_across_resumes` - `just test -p codex-core normalize_adds_missing_output` - `just test -p codex-core response_item_ids` |
||
|
nhamidi-oai
|
328e95110c |
Preserve namespaces on custom tool calls (#30302)
## Summary - Preserve the optional namespace on custom tool calls during response deserialization and app-server replay. - Use the namespaced tool identifier for streaming argument handling and tool dispatch. - Regenerate app-server protocol schemas. - Add regression tests covering namespace serialization and routing. ## Testing - Ran affected protocol and app-server test suites. - Ran the full core test suite; two load-sensitive timing tests passed when rerun individually. - Ran Clippy and formatting checks. - Verified with a local end-to-end app-server replay that the namespace is preserved through the complete request/response flow. |
||
|
Adam Perry @ OpenAI
|
3ae0543fdd |
core: overlap diff root discovery with world state (#30286)
## Why Remote diff-root discovery is independent of world-state construction, but it ran afterward and added filesystem metadata latency before the first model request. Overlap the independent work so thread-cold turns do not pay those waits serially. ## What - Run `record_context_updates_and_set_reference_context_item` and `turn_diff_display_roots` with `tokio::join!`. - Reuse the same resolved display roots when constructing `TurnDiffTracker`; no cache or behavior lifecycle changes are introduced. ## Validation A synthetic executor-skill benchmark with artificial network delay: thread-cold model-request p50 improved from about 1.79 s to 1.58 s. |
||
|
richardopenai
|
d4ec08b8f0 |
[codex] consume pushed exec-server process events (#30273)
## Summary - complete unified-exec processes from the ordered event stream instead of issuing a final zero-wait `process/read` - add optional executor sandbox-denial state to `process/exited` - retain `process/read` as a retained-output and compatibility fallback for receiver lag, sequence gaps, and legacy servers - recover sandbox-denial state across transport reconnection - cover the real `TestCodex` remote-exec path without adding a public test-only event constructor ## Why A successful one-shot tool call currently receives its output and terminal notifications, then pays another wide-area `process/read` round trip before returning. Staging traces showed that remote response wait accounted for more than 99.8% of RPC time; local serialization, queueing, and deserialization were below 0.6 ms. ## Measured impact A direct staging A/B used the same build and route and changed only completion mode. Each arm ran three times with 30 one-shot `/usr/bin/true` calls per run. The table reports the median of the three per-run percentiles. | Metric | Final `process/read` | Pushed events | Change | | --- | ---: | ---: | ---: | | End-to-end completion p50 | 159.5 ms | 118.7 ms | -40.8 ms (-25.6%) | | End-to-end completion p95 | 182.4 ms | 131.7 ms | -50.6 ms (-27.8%) | | Completion-wait p50 | 80.1 ms | 41.5 ms | -38.5 ms (-48.1%) | | Final `process/read` RPC p50 | 79.9 ms | eliminated | -79.9 ms | TCP_NODELAY was enabled in both A/B arms, so its effect cancels out. The successful, complete, in-order event path issued zero final `process/read` calls. ## Compatibility and recovery - new servers send `sandboxDenied` on `process/exited` - legacy servers omit it, which triggers one compatibility `process/read` - broadcast lag or a sequence gap triggers a retained-output read - recovery remains bounded by the server's existing 1 MiB retained-output window - complete, in-order event streams issue no completion read - sandbox denial is attached to the exit event before consumers can observe process completion - server-first and client-first rollouts remain wire-compatible; server-first realizes the latency win immediately ## Integration coverage The `TestCodex` suite exercises four distinct remote-exec contracts: - complete pushed output/exit/close with zero reads - direct pushed sandbox denial with zero reads - legacy missing denial metadata with exactly one compatibility read - count-bounded replay eviction recovered from retained output without duplication ## Validation - `just test -p codex-core exec_command_consumes_pushed_remote_process_events`: 4 passed - `just test -p codex-core unified_exec::process_tests::`: 4 passed - `just test -p codex-exec-server`: 294 passed, 2 skipped - `just test -p codex-exec-server-protocol`: 5 passed - `just test -p codex-rmcp-client`: 89 passed, 2 skipped - focused Bazel `//codex-rs/core:core-all-test`: passed across 16 shards - scoped `just fix` passed for core and exec-server - `just fmt` passed The complete workspace suite was not rerun; focused Cargo and Bazel coverage passed for the changed behavior. |
||
|
Owen Lin
|
a107b84967 |
feat(protocol): define missing rollout turn items (#30282)
## Description
This PR adds canonical core `TurnItem` shapes for command execution,
dynamic tool calls, collab agent tool calls, and sub-agent activity, to
be stored in the rollout file soon.
It also teaches app-server protocol / `ThreadHistoryBuilder` how to
render those items, and adds the small legacy fanout helpers needed for
existing event-based consumers. No core producer or rollout persistence
behavior changes here, that will be done in a followup.
## Making ThreadHistoryBuilder stateless
This is the first PR in a stack to make `ThreadHistoryBuilder` stateless
enough that we can materialize app-server `ThreadItem`s from only a
given slice of `RolloutItem` history, without ever needing to replay the
whole thread from the beginning.
The persisted legacy `RolloutItem::EventMsg` records are mostly shaped
like live UI events, not like materialized `ThreadItem`s. They work if
we replay the full rollout in order, but they often do not contain
enough stable identity or complete item state to project an arbitrary
suffix on its own.
A few examples:
- `UserMessageEvent` and `AgentMessageEvent` have content, but
historically do not carry the persisted app-server item ID that should
become the SQLite primary key.
- `AgentReasoningEvent` and `AgentReasoningRawContentEvent` are
fragments. `ThreadHistoryBuilder` currently merges them into the last
reasoning item, which means a slice starting in the middle of reasoning
cannot know whether to append to an earlier item or create a new one.
- `WebSearchEndEvent`, `McpToolCallEndEvent`, collab end events, and
similar legacy events can often render a final-looking item, but they
usually rely on prior replay state to know which turn owns the item.
- Begin/end legacy events are partial views of one logical item. The
builder correlates them by `call_id` and mutates prior state to
synthesize the final `ThreadItem`.
That is the problem this direction fixes. A persisted canonical
lifecycle record looks much closer to the read model we actually want
later:
```rust
ItemCompletedEvent {
turn_id,
item: TurnItem { id, ...full snapshot... },
completed_at_ms,
}
```
Once rollout has explicit `turn_id`, stable `item.id`, and a canonical
completed item snapshot, the future SQLite projector can reduce only the
new rollout suffix and upsert the affected `thread_items` rows. It no
longer needs to synthesize `item-N`, infer item ownership from the
active turn, or replay earlier events just to reconstruct the current
item snapshot.
## What changed
- Added core `TurnItem` variants and item structs for command execution,
dynamic tool calls, collab agent tool calls, and sub-agent activity.
- Added conversions from those canonical items back into the legacy
event shapes where current consumers still need them.
- Added app-server v2 `ThreadItem` conversion for the new core item
variants.
- Taught `ThreadHistoryBuilder` and rollout persistence metrics to
recognize the new item variants.
## Follow-up
The next PR https://github.com/openai/codex/pull/30283 switches the live
core producers for these item families onto canonical `ItemStarted` /
`ItemCompleted` events.
|
||
|
Abdulrahman Alfozan
|
c55ce3b51b |
Close thread persistence when submission channel closes (#30173)
### Summary Release live thread persistence when a session ends because its submission channel closes. This prevents a later same-process resume from failing with `thread ... already has a live local writer`. ### Details The issue is in the `codex-core` session teardown path used by Codex hosts, rather than in Managed Agents API or exec-server itself. Explicit shutdown already closes the `LiveThread`, which releases the process-scoped writer held by `LocalThreadStore`. The submission-channel-close fallback ran runtime and extension teardown but skipped that persistence shutdown, leaving the thread ID registered as having a live writer. This change: - closes the `LiveThread` on the channel-close fallback path; - preserves the existing teardown order used by explicit shutdowns; - extends the lifecycle regression test to assert that the thread store receives `shutdown_thread`. Context: [original report](https://openai.slack.com/archives/C0B4NBHQGTV/p1782136364948039), [recent occurrence 1](https://openai.slack.com/archives/C0B4NBHQGTV/p1782434817895839?thread_ts=1782136364.948039&cid=C0B4NBHQGTV), [recent occurrence 2](https://openai.slack.com/archives/C0B4NBHQGTV/p1782335107474429?thread_ts=1782136364.948039&cid=C0B4NBHQGTV) ### Testing - `just test -p codex-core submission_loop_channel_close_runs_full_thread_teardown` - `just test -p codex-core --lib` (1,989 passed; 3 skipped) - `just fix -p codex-core` - `just fmt` - Native code review: no findings I also attempted `just test -p codex-core`. The new regression passed; 79 unrelated integration tests failed in the local harness, primarily because helper binaries such as `test_stdio_server` were unavailable, plus local proxy/shell timing failures. |
||
|
Owen Lin
|
f72976a5f1 |
feat(app-server): add optional turn_id to thread/fork (#30277)
## Description This adds stable optional `turnId` support to `thread/fork`. When supplied, the fork copies persisted history through that terminal turn, inclusive, and drops later turns from the new thread. Omitting or passing `null` preserves the existing full-history fork behavior, including the interruption marker when the stored source history ends mid-turn. ## Why We're deprecating `thread/rollback` and this will help certain UX use cases work around it by using `thread/fork` + `turn_id` instead. |
||
|
Charles Du
|
79a8ffdbf7 |
[codex] allow AGENTS.md and skills to authorize delegation (#30274)
Prompt update of MAv2 to include agents.md and skills more explicitly should mimic: https://github.com/openai/codex/pull/27919 |
||
|
hefuc-oai
|
d9cf931d0e |
[codex] Add managed new-thread model settings (#29683)
## Why Admins need persistent defaults for the model, reasoning effort, and service tier shown when the Desktop App creates a new thread. These are initialization defaults rather than runtime constraints: the App should use them to initialize its draft while still allowing a user to make an explicit selection. The app-server therefore needs to expose the managed values before thread creation without changing `thread/start` behavior for other clients. ## What changed - Parse `model`, `model_reasoning_effort`, and `service_tier` from `[models.new_thread]` in `requirements.toml`. - Compose the `models` requirements through the existing requirements-layer precedence rules. - Expose the resolved values through `configRequirements/read` as `requirements.models.newThread`. - Add the corresponding app-server protocol types and regenerate the JSON and TypeScript schema fixtures. - Document the new `configRequirements/read` fields in the app-server README. ## Scope This PR is data plumbing only. It does not apply these values during `thread/start` and does not change thread creation for existing app-server clients, resumed or forked sessions, internal or subagent sessions, `codex exec`, or the TUI. A companion Desktop App change owns draft initialization, sends the effective settings for ordinary and prewarmed starts, and preserves explicit user changes. ## Validation - Requirements deserialization coverage for `[models.new_thread]` - Requirements-layer precedence coverage - App-server API mapping coverage - `configRequirements/read` integration coverage - Regenerated app-server JSON and TypeScript schema fixtures |
||
|
Owen Lin
|
f91334380e |
fix main (#30276)
Introduced by a merge race around thread.history_mode. |
||
|
Owen Lin
|
5267e805fb |
feat(app-server): add history_mode to thread (#29927)
## Description This PR adds a new `historyMode = "legacy" | "paginated"` to `Thread`. This will be stored in `SessionMeta` in the JSONL rollout file and as a new column in the SQLite thread_metadata table, and exposed on `thread/start` and on the `Thread` object in app-server. ## What changed - Added canonical `ThreadHistoryMode` with `legacy` and `paginated`, defaulting old and new SessionMeta to `legacy`. - Carried `history_mode` through core session config, ThreadStore stored metadata, local/in-memory stores, rollout metadata extraction, and the existing SQLite `threads` table. - Added experimental `historyMode` to app-server v2 `Thread` and `thread/start`. - Made paginated stored threads metadata-discoverable but unsupported for legacy full-history reads, `load_history`, live resume, and create paths. - Regenerated app-server schema fixtures and added protocol/state/thread-store/app-server coverage for persistence and fail-closed behavior. ## Compatibility floor Because users may be running various versions of Codex binaries on the same machine (TUI, Codex App, etc.), we will need to establish a compatibility floor for upcoming paginated threads, which will change how thread storage reads and writes work. The overall plan here: ``` Release N: - Add historyMode to SessionMeta / Thread / SQLite metadata. - Teach binaries to understand paginated threads. - If a binary sees `historyMode="paginated"` but does not support the paginated contract, it refuses to resume/mutate the thread. - Default remains `"legacy"`. Release N+1: - First-party clients start opting into paginated threads where appropriate. - Internal dogfood / staged rollout. - Measure old-client usage and paginated-thread unsupported errors. Release N+2: - Only after Release N+ is overwhelmingly deployed, make paginated the default. - Accept that a small tail of N-1-or-older binaries may not understand paginated threads. ``` The important behavior change is fail-closed handling for a binary that encounters a persisted `paginated` thread before it knows how to fully support paginated history. In app-server, if a thread is `paginated`, we will: - allow metadata-only discovery paths like `thread/list` and `thread/read(includeTurns=false)`, so clients can still see the thread and inspect its `historyMode` - reject legacy full-history/live-thread paths like `thread/read(includeTurns=true)` and `thread/resume` with an unsupported JSON-RPC error - avoid silently treating an unknown or future `historyMode` as `legacy` Under the hood, the ThreadStore layer also rejects legacy operations that would need to load or replay the full thread history for a paginated thread. That gives us the behavior we want for Release N: future paginated threads are visible, but this binary fails closed instead of trying to operate on them as if they were legacy threads. |
||
|
jif
|
6d2168f06a |
Reuse MCP runtimes when selected availability changes nothing (#30148)
## Why
MCP runtime reuse was keyed by every ready selected-capability
environment, even when an environment contributed no MCP servers or
connectors.
For example:
1. a global stdio MCP is running;
2. a selected remote environment contains only a skill;
3. that environment becomes ready;
4. the MCP and connector projection stays exactly the same;
5. Codex nevertheless rebuilds the MCP manager and restarts the global
stdio process.
That restart can interrupt active calls and discard process-local state
even though nothing about MCP changed.
## What changes
When selected-environment availability changes, Codex now resolves the
candidate MCP and connector projection before deciding whether to
replace the runtime:
- if the winning MCP servers or their ownership change, rebuild as
before;
- if the selected connector snapshot changes, rebuild as before;
- if an enabled MCP is explicitly bound to an environment whose
availability changed, rebuild as before;
- otherwise, keep the exact live manager and processes, and update only
the availability input remembered by the snapshot.
```text
ready selected environments: [] -> [skills-env]
resolved MCP servers: {global_probe} -> {global_probe}
resolved connectors: {} -> {}
result: reuse manager; keep the same process
```
The comparison uses the resolved winning servers and their sources, so
plugin/config ownership remains part of the runtime identity.
## Existing stack coverage
The integration PR directly below this one already covers both rebuild
boundaries: a selected MCP becomes callable and a selected connector
tool becomes model-visible when their environment becomes available. It
also verifies that an unchanged selected MCP runtime keeps its process.
This PR does not add another remote-attachment integration scenario for
the no-change optimization. `environment/add` returns before readiness,
and app-server does not currently expose a deterministic readiness
signal for an environment that contributes only skills. Keeping a
fixed-delay test would add flake risk; adding a new readiness API would
be outside this fix.
## Scope and assumptions
- This does not change skill discovery, World State rendering, or plugin
metadata caching.
- This does not add file watching or hot reload behavior.
- This does not change disconnect/reconnect handling.
- Selected environment IDs and their capability contents retain the
stack's existing stability assumption.
- Delayed `required = true` executor MCP behavior remains out of scope.
|
||
|
Channing Conger
|
7d8906b478 |
[codex] wire process-owned code mode host into core (#30142)
## Summary - add the `code_mode_host` feature flag and select `ProcessOwnedCodeModeSessionProvider` in `CodeModeService` when enabled - initialize code-mode sessions lazily so a missing host reports a tool error without failing thread startup - resolve `codex-code-mode-host` beside the running Codex binary by default while preserving `CODEX_CODE_MODE_HOST_PATH` as an override - add unit and end-to-end coverage for host resolution and graceful missing-host behavior ## Why This wires the process-owned session client from #30112 into the core service behind an opt-in rollout gate. Packaged Codex installations can place the helper in the same `bin` directory as the main executable without relying on `PATH`, while development and custom installations can continue to override the helper path. ## Stack - Depends on #30112 - Base branch: `cconger/process-owned-session-runtime-4-client` ## Validation Build `codex` and `codex-code-mode-host` `CODEX_CODE_MODE_HOST_PATH="$PWD/target/debug/codex-code-mode-host" ./target/debug/codex --enable code_mode_host` |
||
|
kbazzi
|
92d2e1df70 |
Retry failed Codex Apps MCP startup (#29920)
## Problem The built-in Codex Apps MCP client shares a future for the full startup operation: connect, complete `initialize`, fetch the initial tools, and return a usable client. Sharing deduplicates startup work, but it also memoizes terminal errors. After a transient connection, handshake, or initial `tools/list` failure, later tool builds observe the same failed future. The thread cannot reconnect after the backend recovers and continues serving its startup-time cached tool snapshot, which may be empty or stale. ## Fix When Apps MCP startup ends in an error, Codex starts bounded recovery without putting startup latency on tool-router construction: 1. The current tool build immediately continues with the cached startup snapshot. 2. After the initial failure is reported, Codex starts one fresh full startup attempt in the background. 3. Concurrent tool builds share that in-flight attempt and also continue with cached tools. 4. On success, the recovered client becomes active, refreshes the Apps tools cache, emits a `Ready` startup status, and is reused by later operations. 5. On failure, the cache remains unchanged and later tool builds may start another background attempt after exponential cooldown: 1s, 2s, 4s, 8s, 16s, then 30s maximum. Each recreated startup performs a fresh MCP `initialize` and uncached `tools/list`. The MCP client retains its existing bounded retries for retryable `initialize` and `tools/list` failures. This avoids adding the Apps startup timeout to every request during a sustained outage. ## Scope This is limited to the built-in Codex Apps MCP client: - no reconnects for user-configured MCP servers; - no cache deletion; and - no proactive refresh for a healthy client with stale tools. ## Tests Coverage verifies: - tool builds return cached tools without waiting for a blocked reconnect; - concurrent tool builds start only one background reconnect; - failed reconnects preserve cached tools and respect exponential cooldown; - a recovered client is retained and reused; and - a long-lived thread exposes recovered app tools on a later follow-up. Validation: - `just test -p codex-mcp` — 95 passed - `just test -p codex-core later_follow_up_uses_background_recovered_apps_after_mid_thread_startup_failures --no-capture` — passed - `just fix -p codex-mcp` - `just fmt` |
||
|
Tom
|
f5f812389e |
[codex] fix terminal rollout event durability (#30144)
Currently session code does not flush the thread store after appending the `TurnComplete` / `TurnAborted` events. This isn't a problem in practice for local storage because append_items itself effectively blocks, but any thread stores that buffer in append_items and only commit on flush effectively never get these events persisted. The fix adds explicit rollout flushes at the terminal emitters after normal completion and interruption. Added test cases that assert the number of flushes when completing or aborting turns. These are admittedly a little brittle and I'm open to better ideas on how to add automated testing. |
||
|
Won Park
|
0d4351c1b8 |
[codex] allow CCA image generation and web search extensions (#29909)
## Summary - allow the standalone image-generation and web-search extensions for the actor-authorized provider shape used by CCA - preserve builtin `image_generation` and `web_search` for older models and existing flows - keep ordinary non-OpenAI providers excluded from both extensions - remove only the image extension local managed-AuthManager requirement that CCA cannot satisfy - share actor-authorization detection through `ModelProviderInfo` - keep Core tests focused on routing behavior and cover header-shape edge cases in `model-provider-info` - add a Responses Lite regression that verifies both `image_gen.imagegen` and `web.run` ## Why CCA uses a provider named `local` with `requires_openai_auth: false` and a non-empty `x-openai-actor-authorization` header. Core accepts that provider shape, but both extension provider-name gates rejected it; image generation additionally required a Codex-managed login. The standalone paths must coexist with existing builtin tools. New Responses Lite models can receive `image_gen.imagegen` and `web.run`, while older models continue using builtin tools. ## Impact This enables both standalone extensions for CCA once installed downstream, without removing or changing builtin-tool compatibility for older models. ## Validation - `just test -p codex-core responses_lite_exposes_standalone_tools_for_actor_authorized_provider` - `just test -p codex-core responses_lite_uses_standalone_web_search_and_image_generation` - `just test -p codex-core hosted_tools_follow_provider_auth_model_and_config_gates` - `just test -p codex-image-generation-extension` - `just test -p codex-web-search-extension` - `just test -p codex-model-provider-info` - `just fmt` - `git diff --check` |
||
|
Martin Au-YeungandMartin Au-Yeung
|
ec300bc7bd |
Expose MCP app identity in app context (#29934)
## Why MCP tool-call events need to expose trusted app identity and action metadata directly so v2 clients do not have to infer it from tool names or resource URIs. ## What changed - Add optional `appName`, `templateId`, and `actionName` fields to MCP tool-call `appContext`. - Populate `appName` and `templateId` from trusted Codex Apps metadata, and derive `actionName` from the trusted app resource metadata. - Preserve all three fields through core events, legacy protocol events, persisted thread history, resume redaction, and app-server v2 responses. - Document the public `appContext` fields in `codex-rs/app-server/README.md`. - Regenerate app-server JSON and TypeScript schemas and add coverage for serialization, persistence, redaction, and metadata propagation. ## Validation - `just test -p codex-app-server-protocol mcp_tool_call` - `just test -p codex-core mcp_tool_call_item_metadata_only_trusts_codex_apps_identity mcp_tool_call_item_includes_app_identity` - `just write-app-server-schema` --------- Co-authored-by: Martin Au-Yeung <280153141+martinauyeung-oai@users.noreply.github.com> |
||
|
jif
|
fb8598df3f |
Keep MCP elicitation routable across runtime refreshes (#30127)
## Why
An MCP tool call can still be waiting for an elicitation response when
an environment update replaces the thread's MCP runtime.
Before this change:
```text
runtime A starts a tool call and asks the user
environment becomes ready, so runtime B is published
client answers the prompt through runtime B
runtime B cannot find runtime A's pending responder
```
The response is lost and the original tool call stays blocked.
## What changed
All MCP runtimes for one thread now share a small elicitation router:
```text
runtime A ---\
shared router: response token -> exact pending responder
runtime B ---/
```
When Codex surfaces an MCP elicitation, it assigns a unique opaque
response token. The router records which pending request owns that
token. A replacement runtime reuses the same router, so the latest
runtime can deliver a response to a request started by the previous
runtime.
The Codex-owned token also prevents two runtime connections that reuse
the same MCP server request ID from receiving each other's responses.
This does not retain or search old MCP managers. Only the pending
responder map is shared.
## Covered scenario
The integration test exercises the complete failure mode:
1. A thread starts while its selected environment is still unavailable.
2. A configured MCP server starts a tool call and asks the client for
input.
3. The environment becomes ready, causing Codex to publish a replacement
MCP runtime.
4. The client answers the original prompt after the replacement.
5. The original tool call receives that answer and completes.
A focused routing test also creates two runtimes with the same server
request ID and verifies that each response reaches the exact request
that emitted its token.
## Scope
This PR changes only elicitation response routing across MCP runtime
replacement. It does not change when runtimes are rebuilt, which
environments contribute MCP configuration, or how environment
availability is detected.
|
||
|
jif
|
723b23efd0 |
Reinject missing World State fragments on resume (#30152)
## Why World State restores its structured snapshot on resume so unchanged sections do not have to be rendered again. That is safe only when the model-visible fragment represented by the snapshot is still present in retained history. For selected executor skills, the failing selected-capability scenario exposed this state: ```text persisted World State: selected skill catalog is known retained model history: selected skill catalog message is missing next diff: unchanged, so emit nothing ``` The model resumes without being told about the selected skill catalog. ## What changed World State contributions may now optionally describe the concrete model-visible fragment that must remain in retained history. When a persisted snapshot is present: ```text matching retained fragment exists -> trust snapshot, emit nothing matching retained fragment missing -> treat section as absent, render current state once ``` The skills extension uses this for non-empty selected-environment catalogs by matching its exact rendered catalog body. Empty or hidden catalogs do not require a fragment. ## Scope This does not clear or rebuild the whole World State baseline. It does not change skill discovery, cache invalidation, environment availability, or MCP runtime behavior. It only keeps a persisted section snapshot and its retained model context consistent across resume/history reconstruction. ## Coverage A focused World State regression test verifies both sides: - a missing retained fragment is rendered again - a matching retained fragment avoids duplicate injection |
||
|
jif
|
3095ea9c3d |
Project selected plugin runtime by environment availability (#30093)
## Why
Selected plugin metadata is stable, but MCP processes are live runtime
state. They need different lifetimes:
- the MCP extension caches manifest, MCP, and connector declarations for
each stable selected root;
- each model step projects that cached metadata through the roots that
resolved as ready for that exact step;
- the MCP manager is rebuilt only when that availability projection
changes.
This matches executor skills: both features consume the same resolved
step roots instead of inferring readiness from the turn's selected
environments.
## Behavior
```text
E1 not ready for this step
-> no E1 MCP servers or connectors
-> cached plugin metadata stays in ext/mcp
E1 becomes ready
-> reuse cached metadata
-> publish one MCP runtime containing E1 capabilities
same ready roots on the next step
-> reuse the exact runtime; no rediscovery and no MCP restart
resume
-> create new extension thread state and a new MCP runtime
```
All model-facing consumers use the same step snapshot:
```text
resolved selected roots
|
v
extension MCP/connector projection
|
v
{ MCP config, connector snapshot, MCP manager }
|
+-> advertise model tools
+-> build app/connector tools
+-> execute MCP calls
```
## Cache contract
The existing MCP extension owns a cache keyed by the full
`SelectedCapabilityRoot`:
```rust
let state = thread_store.get_or_init(SelectedExecutorPluginMcpState::default);
```
The cache lives with extension thread state. Environment availability
filters projection but does not invalidate metadata. Resume creates new
thread state. There is no file watcher or executor generation because
contents behind a stable environment/root are assumed stable.
## What changes
- Keeps executor plugin discovery and cached metadata in `ext/mcp`.
- Caches MCP and connector declarations together per selected root.
- Uses the step's already-resolved capability roots, including lazy
environments that are not turn environments.
- Reuses the current MCP runtime when the ready-root projection is
unchanged.
- Uses the same step MCP manager and connector snapshot for
model-visible tools and execution.
- Resolves direct thread-scoped MCP requests from the current
selected-root projection.
## Deliberately out of scope
- `app/list` remains based on the latest global host-plugin state; this
PR does not make its response or notifications thread-specific.
- `required = true` startup semantics do not apply to delayed executor
MCP activation.
- No filesystem/content invalidation.
- No transport-disconnect watcher.
- No executor generations or environment replacement semantics.
- No client sharing across complete manager replacements.
## Stack
1. Extension-owned World State sections.
2. Project executor skills through World State.
3. Pin one MCP runtime to each model step.
4. **This PR:** project selected MCP and connector state from
extension-owned metadata.
5. Integration coverage for selected capability availability and resume.
## Verification
-
`selected_plugin_servers_use_managed_requirements_for_the_selected_root_id`
- The stacked integration PR covers unavailable to ready activation,
unchanged-runtime reuse, skills, MCP tools, connector attribution, and
cold resume.
|
||
|
jif
|
ee9e0f6387 |
Pin MCP runtimes to model steps (#30101)
## Why
An MCP refresh can replace the session's current manager while a model
step is still running. The step must execute calls through the same
manager whose tools it advertised.
## Boundary
```text
current session MCP runtime
|
| capture once for this model step
v
StepContext.mcp
- exact MCP config
- exact connection manager
- exact runtime environment context
```
```rust
pub struct McpRuntimeSnapshot {
config: Arc<McpConfig>,
manager: Arc<McpConnectionManager>,
runtime_context: McpRuntimeContext,
}
```
## Example
```text
step A captures runtime A and advertises A's tools
refresh publishes runtime B
step A tool call -> runtime A
next step -> runtime B
```
Capturing the snapshot is only an `Arc` clone. It does not restart MCPs
or make an RPC.
## What changes
- Captures one MCP runtime in `StepContext`.
- Uses it for tool planning, tool calls, resources, approvals, connector
attribution, and elicitation.
- Publishes replacement runtimes atomically.
- Lets an old runtime live only while an in-flight step or request still
holds its `Arc`.
Most of this diff is mechanical routing from the session-global manager
to `step_context.mcp`; it does not introduce selected-plugin discovery
yet.
## What does not change
- No plugin or extension migration.
- No new MCP cache policy.
- No environment file watching.
- No client sharing between separate managers.
## Stack
1. Extension-owned World State sections.
2. Project executor skills through World State.
3. **This PR:** pin one MCP runtime to each model step.
4. Project selected MCP/app/connector metadata by environment
availability.
5. One end-to-end integration scenario.
|
||
|
jif
|
5eebeb8169 |
Project executor skills through World State (#30088)
## Why A selected executor environment can be unavailable in one model step and ready in the next. The model should see its skills only while that environment is ready, without rescanning stable files on every sample. The product assumption is simple: - an environment ID names one stable logical environment; - the selected root contents do not change during the thread. ## Behavior ```text E1 unavailable -> do not show E1 skills E1 ready -> discover once, cache, show through World State E1 unavailable -> hide skills, keep cache E1 ready again -> reuse cache, show skills again resume -> create a new thread cache and discover again ``` The cache key is the full `SelectedCapabilityRoot`. Availability does not invalidate it; dropping the extension's thread state does. The step supplies the ready selected roots directly. They do not have to be turn environments: ```text turn environment: laptop selected root: worker:/plugins/lint-fix worker ready -> lint-fix skills are visible ``` ## What changes - Keeps executor skill catalogs in the existing skills extension. - Passes the roots resolved as ready for the step into World State contributors. - Loads each ready selected root at most once per thread. - Contributes the executor catalog as the `skills` World State section. - Uses the exact step catalog for explicit skill selection and body reads. - Leaves host and orchestrator skill behavior where it already lives. Taking a step snapshot itself does not add an RPC. Executor filesystem calls happen only on the first discovery of a stable root for that thread. ## What does not change - No filesystem watcher or content-based invalidation. - No retry/generation framework. - No skill runtime migration into core. - No general rewrite of the skills extension. ## Stack 1. Extension-owned World State sections. 2. **This PR:** project cached executor skills through World State. 3. Pin one MCP runtime to each model step. 4. Project selected MCP/app/connector metadata by environment availability. 5. One end-to-end integration scenario. |
||
|
chiam-oai
|
e2746fd7e9 |
Recognize Work web and mobile thread originators (#29988)
## Summary - recognize `codex_work_web` and `codex_work_mobile` as supported `thread/start.serviceName` values - use the recognized value as the thread-scoped originator, with the same persistence and request propagation added for `codex_work_desktop` - cover precedence over persisted and inherited originators This is the Codex consumer for the service names introduced by [openai/openai#1073178](https://github.com/openai/openai/pull/1073178). ## Rollout / Compatibility The producer is ChatGPT's app-server integration in openai/openai#1073178. This PR is the Codex app-server consumer that converts those service names into the outgoing per-thread `originator`. Until this change is deployed, the new service names are ignored and Codex continues using its fallback originator. Deploy this mapper and the matching codex-backend compatibility change in [openai/openai#1073594](https://github.com/openai/openai/pull/1073594) while the existing Flora egress overwrite remains in place. Remove that overwrite in [openai/openai#1073197](https://github.com/openai/openai/pull/1073197) only after both consumers are deployed. ## Validation - `just test -p codex-core effective_originator_prefers_thread_scoped_sources_before_env_originator` - `just fix -p codex-core` - `just fmt` |
||
|
jif
|
c9e6d9783d |
Let extensions contribute World State sections (#30100)
## Why #29856 already owns the durable thread intent and exact environment binding. This PR adds only the small missing extension boundary: an extension can contribute one named World State section, while core still owns persistence, diffing, and model-visible fragment types. This lets skills stay in the skills extension instead of moving their runtime into core. ## Shape ```text extension-owned state | | contribute section id + JSON snapshot + renderer v core World State | | compare with the previous snapshot v no message, or one incremental model-visible update ``` The extension API is deliberately small: ```rust fn contribute_world_state(...) -> Vec<WorldStateSectionContribution> ``` Core adapts the rendered result to `ContextualUserFragment`, records the snapshot, and keeps the existing compaction/resume behavior. ## What changes - Adds extension-owned World State section contributions. - Calls those contributors from the existing per-step World State builder. - Restores durable selected capability roots into extension thread state on resume. - Keeps the actual model-context fragment and rollout machinery in core. ## What does not change - No skill or MCP implementation moves out of its extension. - No new file watcher, generation, or RPC. - No generic migration of existing World State sections. - No change to the stable environment-ID assumption from #29856. ## Example ```text step 1 snapshot: skills = [] step 2 snapshot: skills = [executor-demo:deploy] core asks the skills extension to render only that change. ``` ## Stack 1. **This PR:** let extensions contribute World State sections. 2. Project executor skills through the skills extension. 3. Pin one MCP runtime to each model step. 4. Project selected MCP/app/connector metadata by environment availability. 5. One end-to-end integration scenario. |
||
|
felixxia-oai
|
db541f4553 |
[codex] Add managed MCP server matchers (#29648)
## Summary
This PR extends the existing managed `mcp_servers` identity requirement
so that one name-qualified rule can use either:
- the released exact command or URL identity;
- an exact stdio executable with an exact-length, ordered argument
matcher list; or
- a direct MCP URL matcher.
Matcher-based rules stay under the released `identity` key and use the
same `McpServerRequirement` abstraction and `mcp_servers.<server_name>`
namespace.
## Behavior
Policy activation and name qualification are unchanged:
- If `mcp_servers` is absent, ordinary configured MCP servers remain
unrestricted.
- If `mcp_servers` is present, a server needs a matching same-name
requirement.
- `mcp_servers = {}` continues to deny every configured MCP server.
- Existing exact identity requirements keep their released semantics.
Plugin-bundled MCP servers use the same requirement shapes under
`plugins.<plugin_name>.mcp_servers.<server_name>`. Top-level non-empty
rules continue to govern only ordinary configured servers; plugin rules
remain explicitly plugin-scoped. The existing globally empty
`mcp_servers = {}` plugin kill switch is preserved.
Requirements layers continue to use the existing regular TOML merge
behavior. Atomic replacement of named MCP requirements is intentionally
out of scope here and is tracked independently in #30118.
## Requirement contract
The released exact identity contract remains valid:
```toml
[mcp_servers.docs.identity]
command = "codex-mcp"
[mcp_servers.remote.identity]
url = "https://example.com/mcp"
```
Command identities continue to check only `command`; they do not inspect
arguments, `cwd`, `env`, or `env_vars`.
A command matcher uses an exact executable plus an exact-length, ordered
argument list. Each argument position supports `exact`, `prefix`, or
full-value `regex` matching:
```toml
[mcp_servers.internal_mcp_proxy.identity]
command = { executable = "company-cli", args = [
{ match = "exact", value = "mcp" },
{ match = "exact", value = "proxy" },
{ match = "exact", value = "--server" },
{ match = "regex", expression = '^https://[A-Za-z0-9-]+\.mcp\.internal\.example\.com(?::443)?(?:/.*)?$' },
] }
```
Direct streamable HTTP MCP definitions can use the same value matcher
types through `identity.url`:
```toml
[mcp_servers.internal_http.identity]
url = {
match = "regex",
expression = '^https://[A-Za-z0-9-]+\.mcp\.internal\.example\.com(?:/.*)?$',
}
```
Plugin-bundled MCP matchers use the same contract inside the
plugin-qualified allowlist:
```toml
[plugins."sample@test".mcp_servers.internal_mcp_proxy.identity]
command = { executable = "company-cli", args = [
{ match = "exact", value = "mcp" },
{ match = "exact", value = "proxy" },
] }
```
Regexes are validated while managed requirements are loaded, and regex
matching must cover the complete value. Command matchers constrain only
the executable and arguments.
## Why
Enterprise administrators need to allow MCP servers by executable and
positional-argument shape, including fixed arguments plus constrained
values such as internal MCP URLs passed to a proxy.
## Validation
- `just fmt`
- `git diff --check`
- `just test -p codex-config` (198 passed)
- `just test -p codex-core mcp_servers_by_matchers --lib` (2 passed)
|
||
|
Owen Lin
|
703793c22e |
feat(core, mcp): cache codex_apps tools in memory (#29003)
## Description This makes Codex Apps tool reads use a shared in-memory snapshot instead of rereading the disk cache every time `list_all_tools()` runs. Disk still seeds the cache on startup and gets updated after successful fetches, but it is no longer the live read path. The core change is that `McpManager` now owns a process-scoped `CodexAppsToolsCache`. Codex threads in the same app-server process now share this Codex Apps in-memory tools snapshot. The snapshot is keyed by the Codex home plus the Codex Apps identity: the active Codex auth user/workspace and the effective Codex Apps MCP source config. There's already code to hard-refresh the cache, so we respect it in this PR. ## Local benchmark I ran a local steady-state microbenchmark of the exact repeated Codex Apps cached-tools read this PR removes, using the same real local cache payload in both trees: `3,678,138` bytes and `381` tools. The cache file was already warm in the OS page cache, so this measures same-process reread/deserialization work rather than cold-disk latency or full turn latency. Each run is 25 iterations (mimicking a turn that makes 25 inference calls). | Version | Run 1 | Run 2 | Avg | |---|---:|---:|---:| | `origin/main` disk read + JSON deserialize + `filter_tools` | `50.755 ms` | `52.894 ms` | `51.825 ms` | | This branch in-memory `current_tools` + `filter_tools` | `0.740 ms` | `0.778 ms` | `0.759 ms` | That removes about `51 ms` from each repeated Codex Apps cached-tools read on this machine, roughly `68x` faster for that subpath. It is useful evidence for the hot path this PR changes, but not a claim that every production turn gets `51 ms` faster; end-to-end impact also depends on the rest of `list_all_tools()` and tool-payload construction. This is on my M2 Max macbook, so with a slower disk this would be much worse (and indeed we did see this really blew up turn runtime with a slow disk). |
||
|
rka-oai
|
adccb464d0 |
[codex] impl delivery_mode: current time reminders on response boundaries (#30033)
## Summary - track user-like input and tool-output boundaries in current-time reminder state - gate reminder injection when delivery_mode is after_user_or_tool_output - preserve interval debounce and forced reminders after context-window changes ## Why Training can request reminders only after user or tool-output items while keeping the existing canonical pre-inference history-injection path. ## Validation - just test -p codex-core current_time_reminders_can_follow_only_user_or_tool_outputs - just test -p codex-core current_time_reminders_follow_time_interval_and_persist_in_history - just test -p codex-core current_time_reminder_is_refreshed_after_compaction - just fix -p codex-core |
||
|
rka-oai
|
e8d4a1a411 |
[codex] add current time reminder delivery mode config (#30031)
```python delivery_mode = "any_inference" # default delivery_mode = "after_user_or_tool_output" # new mode ``` ## Validation - just test -p codex-core load_config_resolves_current_time_reminder - just test -p codex-core lock_contains_prompts_and_materializes_features |
||
|
Michael Bolin
|
c65cfeab14 |
core: expose permission profile to shell tools (#29941)
## tl;dr
Inject a `CODEX_PERMISSION_PROFILE` environment variable with the name
of the current permission profile when invoking a shell tool.
## Why
Shell tool owners may need to launch nested commands under the same
named permission profile, including through `codex sandbox -P PROFILE
--include-managed-config`. Until now, child processes could observe
sandbox and network metadata but could not identify the active named
permission profile.
The `--include-managed-config` flag is essential when a helper
reconstructs the sandbox from a profile name: it ensures the nested
sandbox also loads managed enterprise requirements. Without it, using
the inherited profile could unintentionally create a sandbox that does
not enforce the organization's managed restrictions.
The new environment value is intentionally informational and **must not
be treated as trusted input**. Any process in the ancestry can overwrite
an environment variable, so a consumer that passes this value to `codex
sandbox -P` must first validate it against the profiles that helper is
authorized to use.
## Example Use Case
Suppose an organization provides a trusted `remote-bash` wrapper that
lets Codex run a command on an approved build host. The local shell
command uses the named `:workspace` permission profile:
```toml
default_permissions = ":workspace"
```
The command exposed to the model is a small zsh wrapper. It deliberately
delegates with `exec`, preserving the original arguments and process
environment:
```zsh
#!/usr/bin/env zsh
exec /opt/codex-tools/remote_bash.py "$@"
```
The model invokes the public wrapper, not its Python implementation:
```sh
/opt/codex-tools/remote-bash \
--host builder.example.com \
-- printf '%s' 'hello world'
```
Only the inner implementation is authorized to escape the local sandbox:
```starlark
prefix_rule(
pattern=["/opt/codex-tools/remote_bash.py"],
decision="allow",
)
```
With zsh-fork, execution begins with `remote-bash` inside the
`:workspace` sandbox. When the wrapper calls `exec`, the exact prefix
rule matches `remote_bash.py`, so that inner script is restarted
unsandboxed. The escalated process inherits:
```text
CODEX_PERMISSION_PROFILE=:workspace
```
Inheritance does not make the value trustworthy. `remote_bash.py`
independently allowlists both the remote host and the permission profile
before using either value. In particular, a forged value such as
`:danger-full-access` is rejected before it can reach `codex sandbox
-P`:
```python
import argparse
import os
import shlex
import sys
ALLOWED_HOSTS = {"builder.example.com"}
ALLOWED_PROFILES = {":workspace"}
parser = argparse.ArgumentParser()
parser.add_argument("--host", required=True)
separator = sys.argv.index("--")
args = parser.parse_args(sys.argv[1:separator])
command = sys.argv[separator + 1:]
if args.host not in ALLOWED_HOSTS:
parser.error("host is not allowlisted")
if not command:
parser.error("the remote command must not be empty")
profile = os.environ.get("CODEX_PERMISSION_PROFILE")
if not profile:
raise SystemExit("CODEX_PERMISSION_PROFILE must not be empty")
if profile not in ALLOWED_PROFILES:
raise SystemExit("CODEX_PERMISSION_PROFILE is not allowlisted")
remote_command = shlex.join(command)
sandbox_command = shlex.join([
"codex", "sandbox", "-P", profile,
"--include-managed-config", "--",
"bash", "-lc", remote_command,
])
print(shlex.join(["ssh", args.host, sandbox_command]))
```
This builds each command layer as an argument vector and uses
`shlex.join()` at the boundary, rather than interpolating untrusted
shell text. After validation and parsing, the nested command has this
structure:
```text
ssh argv:
["ssh", "builder.example.com", SANDBOX_COMMAND]
SANDBOX_COMMAND argv:
["codex", "sandbox", "-P", ":workspace",
"--include-managed-config", "--",
"bash", "-lc", "printf %s 'hello world'"]
bash -lc payload argv:
["printf", "%s", "hello world"]
```
A production implementation could execute that SSH command. The
integration fixture prints it and parses the result back into arguments,
verifying the complete flow:
```text
model invokes outer wrapper
-> zsh-fork starts wrapper under :workspace
-> wrapper execs allowlisted Python script
-> prefix rule restarts Python script unsandboxed
-> Python script inherits CODEX_PERMISSION_PROFILE=:workspace
-> Python script verifies :workspace is allowlisted
-> remote command runs codex sandbox -P :workspace
with --include-managed-config
-> nested sandbox honors managed enterprise requirements
```
This gives the trusted helper access to resources outside the local
sandbox—such as SSH credentials—while ensuring that it can select only
an explicitly authorized profile and that work on the remote host
remains subject to the organization's managed requirements.
## What changed
- Inject `CODEX_PERMISSION_PROFILE` after shell environment policy
evaluation so the active profile wins over inherited or configured stale
values.
- Apply the variable to both `shell_command` and unified `exec_command`,
including local, zsh-fork, and remote exec-server paths.
- Remove stale values when the session has no active named profile.
- Preserve the current profile value when loading a shell snapshot so a
parent snapshot cannot restore an older profile.
## Testing
- Added classic-shell integration coverage proving an exact prefix rule
can run a `require_escalated` script outside the `:workspace` sandbox
while preserving `CODEX_PERMISSION_PROFILE=:workspace`.
- Added zsh-fork integration coverage in which the model invokes an
outer zsh wrapper, an inner allowlisted `remote_bash.py` runs
unsandboxed, and its printed SSH command reconstructs the inherited
`:workspace` sandbox with `--include-managed-config` while preserving
every argument after `--`.
- The example helper treats `CODEX_PERMISSION_PROFILE` as untrusted and
validates it against `ALLOWED_PROFILES` before constructing the nested
command.
- Assert that the reconstructed sandbox command includes
`--include-managed-config` so nested use of the inherited profile cannot
bypass managed enterprise requirements.
- Added coverage for overriding and removing stale profile values.
- Verified `shell_command` receives the selected active profile.
- Added shell snapshot coverage using `printenv
CODEX_PERMISSION_PROFILE`.
|