[Codex Thread
019edd6d-6f14-74e2-853c-345d1803d4a6](https://codex-thread-link.openai.chatgpt-team.site/thread/019edd6d-6f14-74e2-853c-345d1803d4a6)
## Stack
Review and merge in order. Every layer is independently correct and
documents its safe stopping point.
1. [openai/codex#30292](https://github.com/openai/codex/pull/30292) —
aggregate File/Secrets store locking
2. [openai/codex#30293](https://github.com/openai/codex/pull/30293) —
resolve and lifecycle-pin the exact OAuth store
3. [openai/codex#30416](https://github.com/openai/codex/pull/30416) —
serialized authoritative refresh transaction
4. [openai/codex#30294](https://github.com/openai/codex/pull/30294) —
Codex-owned transport refresh and one-shot 401 recovery
5. [openai/codex#30295](https://github.com/openai/codex/pull/30295) —
login/logout transaction serialization
6. [openai/codex#30296](https://github.com/openai/codex/pull/30296) —
diagnostic-only Auto store drift reporting
**This PR is layer 2.**
## Why
`Auto` is keyring-first with a File fallback, but re-evaluating that
policy during transport reconstruction or persistence can make one MCP
client read from one store and later write to another. With rotating
refresh tokens, the second store may contain an older token. This layer
makes the source selected at client startup explicit and keeps that
authority stable for the client lifecycle.
## What this PR does
- Keeps `resolve_oauth_tokens_from_store_policy` as the single
configured-policy entry point and returns both credentials and the
concrete File or Keyring source that supplied them.
- Puts exact `load`, `save`, and `delete` operations on
`ResolvedOAuthCredentialStore`, making “resolve configured policy” and
“use the selected authority” distinct at call sites.
- Pins the first concrete source in `pinned_credential_store` in the
transport recipe, so initialization retries and session reconstruction
cannot re-evaluate `Auto` and adopt another store.
- Gives `OAuthPersistor` the resolved store and keeps subsequent
persistence and removal on that authority.
- Uses a typed keyring-load error to distinguish aggregate-store
coordination failures from ordinary backend failures; a coordination
failure is surfaced instead of triggering File fallback.
- Keeps login-time `Auto` behavior unchanged: prefer Keyring, fall back
to File when unavailable, and clean up legacy File state after a
successful keyring save.
- Adds structured server/backend context when fallback cleanup fails.
## Explicit decisions and non-goals
- The selection is lifecycle-local and in memory. This PR does not add a
durable backend selector, migration, reconciliation registry, or global
source of truth outside `CODEX_HOME`.
- `Auto` may choose File at the start of a later process if keyring
availability changes. Once this client resolves, a selected-store
failure is returned instead of hot-switching.
- Different `CODEX_HOME` instances remain independent even when they can
access the same Direct keyring credential.
- Cross-process refresh serialization is intentionally not part of this
layer.
## Safe stopping point
This PR can merge alone. A single MCP client no longer hot-switches
credential stores across transport rebuilds or persistence. Two
processes can still refresh the same selected credential concurrently
until layer 3.
## Review size
The net layer is 9 files, +668/−144. The production change remains
focused on store resolution and lifecycle pinning; the largest follow-up
is integration coverage that drives real session recovery.
## Validation
- `just test -p codex-rmcp-client` (99 passed; 5 expected skips)
- Real-client 404 recovery coverage with different Keyring and File
tokens; captured bearer headers prove the stale File token is never sent
- Mutation check: removing the lifecycle pin makes that integration
regression fail by observing the stale File token
## Why
Remote MCP stdio buffers executor-provided stdout until a JSON-RPC line
completes and stderr until a diagnostic line completes. A malicious or
broken executor can keep sending chunks without a newline, making
orchestrator memory grow without bound.
Executor event boundaries are arbitrary, so a valid message before an
oversized line must not be lost merely because both arrived in one
event.
## What changed
- Limit one stdout JSON-RPC line to 8 MiB.
- Limit one stderr diagnostic line to 1 MiB.
- Check each logical line before copying its bytes into the retained
buffer.
- Preserve complete stdout messages that precede an oversized line and
deliver them before closing the stream.
- Close the MCP transport on overflow; existing transport cleanup
terminates the executor-managed process.
- Preserve multiple lines per chunk, CRLF framing, and final
unterminated messages at EOF.
- Box the pending stdio transport so the added framing state does not
inflate every `ClientState` value.
- Add regression coverage for cross-chunk overflow, bounded lines in a
larger chunk, and a valid prefix before an oversized line.
## Scope
This bounds pre-newline buffering for executor-backed MCP stdio only. It
does not change local stdio framing or address memory amplification
while parsing a complete JSON message.
## Summary
MCP operation timeout errors currently print the full debug precision of
the remaining timeout budget. That makes a configured 30-second timeout
show up as something like `29.999999875s`.
This PR rounds the displayed duration to a whole unit, so the error is
clean and stable:
```text
timed out awaiting tools/list after 30s
```
The timeout behavior itself is unchanged; this only affects the
human-facing error text. A regression test covers the reported
`tools/list` case.
## Why
#28522 routes selected-plugin HTTP MCP traffic through the owning
executor, but OAuth bootstrap and refresh still used host-local clients.
Executor-only servers therefore cannot complete discovery or login
through the same network boundary as the MCP connection.
## What changed
- adapt `codex_exec_server::HttpClient` to RMCP 1.8's `OAuthHttpClient`
contract
- let RMCP own discovery, dynamic registration, PKCE, token exchange,
and refresh
- route auth status, persisted-token startup, and app-server login
through the server runtime while preserving the existing local discovery
path
- add optional `threadId` to `mcpServer/oauth/login` and echo it in the
completion notification
- implement RMCP's redirect policy and 1 MiB OAuth response limit over
executor HTTP
- cover selected-thread OAuth discovery and login through an
executor-only route
Depends on #28522.
## Why
ChatGPT session authentication was inferred from the reserved Codex Apps
server name. That couples credential routing to Codex Apps-specific
behavior and prevents other MCP endpoints hosted by ChatGPT from
explicitly using the current session.
The opt-in also needs a clear security boundary: an arbitrary MCP
configuration must not be able to redirect ChatGPT credentials to
another origin.
## What changed
- Add `use_chatgpt_auth` to HTTP MCP server configuration, defaulting to
`false`.
- Honor the setting only when the parsed server URL has the same HTTP(S)
origin as the configured `chatgpt_base_url`; otherwise remove the
capability before startup.
- Resolve bearer tokens and static or environment-backed authorization
headers before selecting authentication, with configured authorization
taking precedence over ChatGPT session auth.
- Enable the setting for the built-in Codex Apps and hosted plugin
runtime endpoints while keeping Codex Apps caching and tool
normalization scoped to the reserved server.
- Persist the setting through MCP config rewrite paths and expose it in
the generated config schema.
- Load the current login state for `codex mcp list` so reported auth
status matches runtime behavior.
## Verification
Core integration coverage exercises the complete streamable HTTP MCP
startup path and verifies that:
- a same-origin opted-in server receives the current ChatGPT access
token;
- an explicitly configured authorization header takes precedence;
- a different-origin server completes MCP initialization and tool
listing without receiving any ChatGPT authorization header.
## Summary
- add trace-level instrumentation around per-server MCP setup, client
construction, initialization, and initial tool listing
- trace Codex Apps tool and server-info cache loads
- attach `server_name` to server-scoped spans so slow startup work can
be attributed to a specific MCP server
## Why
`session_init.mcp_manager_init` can occasionally be slow, but its
existing coarse span does not identify whether time is spent loading the
Codex Apps cache, constructing a client, initializing a transport, or
listing tools. These definition-level spans provide that breakdown
without changing startup behavior.
## Validation
- `just test -p codex-mcp` (87 passed)
- `just test -p codex-rmcp-client` (86 passed, 2 skipped)
## Summary
- Update `rmcp` and `rmcp-macros` from 1.7.0 to 1.8.0.
- Adapt to the new shared `peer_info` return type.
- Box OAuth status discovery at the MCP boundary to keep the expanded
future type from overflowing Rust's trait recursion limit.
This brings in custom OAuth HTTP client support from
[modelcontextprotocol/rust-sdk#908](https://github.com/modelcontextprotocol/rust-sdk/pull/908).
## Why
Remote stdio MCP servers can run in an environment whose path convention
differs from the Codex host. A Windows cwd such as
`C:\Users\openai\share` is absolute for the executor but was rejected by
a POSIX orchestrator.
Built on #29501, now merged, which only clarifies the host-native
`PathUri` constructor name.
## What changed
- Deserialize MCP cwd values as `LegacyAppPathString` so config does not
apply host path rules.
- Interpret that spelling as host-native for local launches and convert
it to `PathUri` at executor launch.
- Skip host filesystem and command resolution checks for remote stdio in
`codex doctor`.
- Add host-independent config and executor-boundary coverage using the
foreign path convention for each test platform.
## Validation
- `just test -p codex-utils-path-uri -p codex-config -p codex-mcp -p
codex-rmcp-client` (408 passed)
- `just test -p codex-cli -p codex-rmcp-client` (372 passed)
- `cargo check --workspace --tests`
- `just test` (11,311 passed; 43 unrelated environment/timing failures)
- `just fix -p codex-cli -p codex-config -p codex-core -p codex-mcp -p
codex-mcp-extension -p codex-rmcp-client -p codex-tui`
## Summary
- store MCP OAuth credentials in the configured auth credential backend
- support encrypted-local OAuth storage, including legacy keyring
migration
- propagate the credential backend through MCP refresh, session, CLI,
and app-server paths
## Stack
1. #27504 — config and feature flag
2. #27535 — auth-specific secret namespaces
3. #27539 — encrypted CLI auth storage
4. this PR — encrypted MCP OAuth storage
This is a parallel review stack; the original #17931 remains unchanged.
## Tests
- `just test -p codex-rmcp-client` (the transport round-trip test passed
after building the required `codex` binary and retrying)
- `just test -p codex-mcp`
- `just test -p codex-app-server
refresh_config_uses_latest_auth_keyring_backend`
- `just test -p codex-core
refresh_mcp_servers_is_deferred_until_next_turn`
- `just test -p codex-cli mcp`
- `just fix -p codex-rmcp-client -p codex-mcp -p codex-core -p codex-cli
-p codex-app-server -p codex-protocol`
- `just bazel-lock-check`
## Summary
- Retry transient streamable HTTP failures during RMCP startup when the
failure happens while sending the initialize request.
- Retry transient streamable HTTP failures for tools/list, which is
read-only and safe to replay.
- Cover both retryable HTTP statuses and request-layer failures where no
HTTP status is returned.
- Surface retryable HTTP statuses from the streamable HTTP adapter as
typed client errors.
- Add integration coverage for initialize retry, tools/list retry,
no-status request failure retry, and non-retryable initialize status.
## Root cause
The observed codex_apps failures can happen before normal tool
execution: RMCP startup fails while sending initialize, or the first
read-only tools/list fails after startup. Retrying hosted_apps_bridge
tools/call would not cover initialize and would risk replaying
side-effecting tool calls. This change retries the streamable HTTP
handshake itself, recreates the transport between initialize attempts,
and retries only tools/list among post-initialize service operations.
## Validation
- cargo fmt --package codex-rmcp-client
- cargo test -p codex-rmcp-client --test streamable_http_recovery
WIll make it easier to uprev when the new draft spec is supported.
Also updates reqwest where needed for compatibility but doesn't update
it everywhere since this is already a large diff.
The new version of rmcp handles certain kinds of authentication failures
differently, this patch includes support for identifying the failing scope
in a WWW-Authenticate header.
## DISCLAIMER
This is experimental and no production service must rely on this
## Why
Built-in MCPs are product-owned runtime capabilities, but they were
previously flattened into the same config-backed stdio path as
user-configured servers. That made them depend on a hidden `codex
builtin-mcp` re-exec path, exposed them through config-oriented CLI
flows, and erased distinctions the runtime needs to preserve—most
notably whether an MCP call should count as external context for
memory-mode pollution.
## What changed
- Model product-owned built-ins separately from config-backed MCP
servers via `BuiltinMcpServer` and `EffectiveMcpServer`.
- Launch built-ins in process through a reusable async transport instead
of the hidden `builtin-mcp` stdio subcommand.
- Keep config-oriented CLI operations such as `codex mcp
list/get/login/logout` scoped to configured servers, while merging
built-ins only into the effective runtime server set.
- Retain server metadata after launch so parallel-tool support and
context classification come from the live server set; built-in
`memories` is now classified as local Codex state rather than external
context.
## Test plan
- `cargo test -p codex-mcp`
- `cargo test -p codex-core --test suite
builtin_memories_mcp_call_does_not_mark_thread_memory_mode_polluted_when_configured`
---------
Co-authored-by: Codex <noreply@openai.com>
## Why
Several bug reports describe thread shutdown (including subagent
threads) leaving stdio MCP server processes behind. These reports all
point at the same lifecycle gap: Codex launches stdio MCP servers, but
the session-level shutdown path does not explicitly close MCP clients or
terminate the server process tree.
Fixes#12491Fixes#12976Fixes#18881Fixes#19469
## History
This is best understood as a regression/coverage gap in MCP session
lifecycle management, not as stdio MCP cleanup being absent all along.
#10710 added process-group cleanup for stdio MCP servers, but that
cleanup only runs when the `RmcpClient`/transport is dropped. The older
reports (#12491 and #12976) came after that cleanup existed, which
suggests the remaining problem was that some higher-level shutdown paths
kept the MCP manager alive or replaced it without explicitly draining
clients. The newer reports (#18881 and #19469) exposed the same family
around manager replacement and shutdown.
## What changed
- Added an explicit stdio MCP process handle in `codex-rmcp-client` so
local MCP servers terminate their process group and executor-backed MCP
servers call the executor process terminator.
- Added `RmcpClient::shutdown()` and manager-level MCP shutdown draining
so session shutdown, channel-close fallback, MCP refresh, and connector
probing stop owned MCP clients.
- Added regression coverage that starts a stdio MCP server, begins an
in-flight blocking tool call, shuts down the client, and asserts the
server process exits.
## Verification
- `cargo test -p codex-rmcp-client`
- `cargo test -p codex-mcp`
- `just fix -p codex-rmcp-client`
- `just fix -p codex-mcp`
- `just fix -p codex-core`
- Manual before/after validation with a temporary repro script:
- Pre-fix binary from `HEAD^` (`fed0a8f4fa`): reproduced the leak with
surviving MCP server and child PIDs, `survivors=[77583, 77592]`,
`leaked=true`.
- Post-fix binary from this branch (`67e318148b`): verified both MCP
processes were gone after interrupting `codex exec`, `survivors=[]`,
`leaked=false`.
### Why
The RMCP layer needs a Streamable HTTP client that can talk either
directly over `reqwest` or through the executor HTTP runner without
duplicating MCP session logic higher in the stack. This PR adds that
client-side transport boundary so remote Streamable HTTP MCP can reuse
the same RMCP flow as the local path.
### What
- Add a shared `rmcp-client/src/streamable_http/` module with:
- `transport_client.rs` for the local-or-remote transport enum
- `local_client.rs` for the direct `reqwest` implementation
- `remote_client.rs` for the executor-backed implementation
- `common.rs` for the small shared Streamable HTTP helpers
- Teach `RmcpClient` to build Streamable HTTP transports in either local
or remote mode while keeping the existing OAuth ownership in RMCP.
- Translate remote POST, GET, and DELETE session operations into
executor `http/request` calls.
- Preserve RMCP session expiry handling and reconnect behavior for the
remote transport.
- Add remote transport coverage in
`rmcp-client/tests/streamable_http_remote.rs` and keep the shared test
support in `rmcp-client/tests/streamable_http_test_support.rs`.
### Verification
- `cargo check -p codex-rmcp-client`
- online CI
### Stack
1. #18581 protocol
2. #18582 runner
3. #18583 RMCP client
4. #18584 manager wiring and local/remote coverage
---------
Co-authored-by: Codex <noreply@openai.com>
This is the second cleanup in the await-holding lint stack. The
higher-level goal, following https://github.com/openai/codex/pull/18178
and https://github.com/openai/codex/pull/18398, is to enable Clippy
coverage for guards held across `.await` points without carrying broad
suppressions.
The stack is working toward enabling Clippy's
[`await_holding_lock`](https://rust-lang.github.io/rust-clippy/master/index.html#await_holding_lock)
lint and the configurable
[`await_holding_invalid_type`](https://rust-lang.github.io/rust-clippy/master/index.html#await_holding_invalid_type)
lint for Tokio guard types.
Several existing fields used `tokio::sync::Mutex<()>` only as
one-at-a-time async gates. Those guards intentionally lived across
`.await` while an operation was serialized. A mutex over `()` suggests
protected data and trips the await-holding lint shape; a single-permit
`tokio::sync::Semaphore` expresses the intended serialization directly.
## What changed
- Replace `Mutex<()>` serialization gates with `Semaphore::new(1)` for
agent identity ensure, exec policy updates, guardian review session
reuse, plugin remote sync, managed network proxy refresh, auth token
refresh, and RMCP session recovery.
- Update call sites from `lock().await` / `try_lock()` to
`acquire().await` / `try_acquire()`.
- Map closed-semaphore errors into the existing local error types, even
though these semaphores are owned for the lifetime of their managers.
- Update session test builders for the new
`managed_network_proxy_refresh_lock` type.
## Verification
- The split stack was verified at the final lint-enabling head with
`just clippy`.
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/18403).
* #18698
* #18423
* #18418
* __->__ #18403
## Summary
- Add the executor-backed RMCP stdio transport.
- Wire MCP stdio placement through the executor environment config.
- Cover local and executor-backed stdio paths with the existing MCP test
helpers.
## Stack
```text
o #18027 [6/6] Fail exec client operations after disconnect
│
@ #18212 [5/6] Wire executor-backed MCP stdio
│
o #18087 [4/6] Abstract MCP stdio server launching
│
o #18020 [3/6] Add pushed exec process events
│
o #18086 [2/6] Support piped stdin in exec process API
│
o #18085 [1/6] Add MCP server environment config
│
o main
```
---------
Co-authored-by: Codex <noreply@openai.com>
## Summary
- Move local MCP stdio process startup behind a launcher trait.
- Preserve existing local stdio behavior while making transport creation
explicit.
## Stack
```text
o #18027 [6/6] Fail exec client operations after disconnect
│
o #18212 [5/6] Wire executor-backed MCP stdio
│
@ #18087 [4/6] Abstract MCP stdio server launching
│
o #18020 [3/6] Add pushed exec process events
│
o #18086 [2/6] Support piped stdin in exec process API
│
o #18085 [1/6] Add MCP server environment config
│
o main
```
---------
Co-authored-by: Codex <noreply@openai.com>
## Summary
Stop counting elicitation time towards mcp tool call time. There are
some tradeoffs here, but in general I don't think time spent waiting for
elicitations should count towards tool call time, or at least not
directly towards timeouts.
Elicitations are not exactly like exec_command escalation requests, but
I would argue it's ~roughly equivalent.
## Testing
- [x] Added unit tests
- [x] Tested locally
## Why
This is a follow-up to #15360. That change fixed the `arg0` helper
setup, but `rmcp-client` still coerced stdio transport environment
values into UTF-8 `String`s before program resolution and process spawn.
If `PATH` or another inherited environment value contains non-UTF-8
bytes, that loses fidelity before it reaches `which` and `Command`.
## What changed
- change `create_env_for_mcp_server()` to return `HashMap<OsString,
OsString>` and read inherited values with `std::env::var_os()`
- change `TransportRecipe::Stdio.env`, `RmcpClient::new_stdio_client()`,
and `program_resolver::resolve()` to keep stdio transport env values in
`OsString` form within `rmcp-client`
- keep the `codex-core` config boundary stringly, but convert configured
stdio env values to `OsString` once when constructing the transport
- update the rmcp-client stdio test fixtures and callers to use
`OsString` env maps
- add a Unix regression test that verifies `create_env_for_mcp_server()`
preserves a non-UTF-8 `PATH`
## How to verify
- `cargo test -p codex-rmcp-client`
- `cargo test -p codex-core mcp_connection_manager`
- `just argument-comment-lint`
Targeted coverage in this change includes
`utils::tests::create_env_preserves_path_when_it_is_not_utf8`, while the
updated stdio transport path is exercised by the existing rmcp-client
tests that construct `RmcpClient::new_stdio_client()`.
## Summary
Some background. We're looking to instrument GA turns end to end. Right
now a big gap is grouping mcp tool calls with their codex sessions. We
send session id and turn id headers to the responses call but not the
mcp/wham calls.
Ideally we could pass the args as headers like with responses, but given
the setup of the rmcp client, we can't send as headers without either
changing the rmcp package upstream to allow per request headers or
introducing a mutex which break concurrency. An earlier attempt made the
assumption that we had 1 client per thread, which allowed us to set
headers at the start of a turn. @pakrym mentioned that this assumption
might break in the near future.
So the solution now is to package the turn metadata/session id into the
_meta field in the post body and pull out in codex-backend.
- send turn metadata to MCP servers via `tools/call` `_meta` instead of
assuming per-thread request headers on shared clients
- preserve the existing `_codex_apps` metadata while adding
`x-codex-turn-metadata` for all MCP tool calls
- extend tests to cover both custom MCP servers and the codex apps
search flow
---------
Co-authored-by: Codex <noreply@openai.com>
## Summary
- forward request-scoped task headers through MCP tool metadata lookups
and tool calls
- apply those headers to streamable HTTP initialize, tools/list, and
tools/call requests
- update affected rmcp/core tests for the new request_headers plumbing
## Testing
- cargo test -p codex-rmcp-client
- cargo test -p codex-core (fails on pre-existing unrelated error in
core/src/auth_env_telemetry.rs: missing websocket_connect_timeout_ms in
ModelProviderInfo initializer)
- just fix -p codex-rmcp-client
- just fix -p codex-core (hits the same unrelated auth_env_telemetry.rs
error)
- just fmt
---------
Co-authored-by: Codex <noreply@openai.com>
## Why
Once the repo-local lint exists, `codex-rs` needs to follow the
checked-in convention and CI needs to keep it from drifting. This commit
applies the fallback `/*param*/` style consistently across existing
positional literal call sites without changing those APIs.
The longer-term preference is still to avoid APIs that require comments
by choosing clearer parameter types and call shapes. This PR is
intentionally the mechanical follow-through for the places where the
existing signatures stay in place.
After rebasing onto newer `main`, the rollout also had to cover newly
introduced `tui_app_server` call sites. That made it clear the first cut
of the CI job was too expensive for the common path: it was spending
almost as much time installing `cargo-dylint` and re-testing the lint
crate as a representative test job spends running product tests. The CI
update keeps the full workspace enforcement but trims that extra
overhead from ordinary `codex-rs` PRs.
## What changed
- keep a dedicated `argument_comment_lint` job in `rust-ci`
- mechanically annotate remaining opaque positional literals across
`codex-rs` with exact `/*param*/` comments, including the rebased
`tui_app_server` call sites that now fall under the lint
- keep the checked-in style aligned with the lint policy by using
`/*param*/` and leaving string and char literals uncommented
- cache `cargo-dylint`, `dylint-link`, and the relevant Cargo
registry/git metadata in the lint job
- split changed-path detection so the lint crate's own `cargo test` step
runs only when `tools/argument-comment-lint/*` or `rust-ci.yml` changes
- continue to run the repo wrapper over the `codex-rs` workspace, so
product-code enforcement is unchanged
Most of the code changes in this commit are intentionally mechanical
comment rewrites or insertions driven by the lint itself.
## Verification
- `./tools/argument-comment-lint/run.sh --workspace`
- `cargo test -p codex-tui-app-server -p codex-tui`
- parsed `.github/workflows/rust-ci.yml` locally with PyYAML
---
* -> #14652
* #14651
## Stacked PRs
This work is now effectively split across two steps:
- #14178: add custom CA support for browser and device-code login flows,
docs, and hermetic subprocess tests
- #14239: extend that shared custom CA handling across Codex HTTPS
clients and secure websocket TLS
Note: #14240 was merged into this branch while it was stacked on top of
this PR. This PR now subsumes that websocket follow-up and should be
treated as the combined change.
Builds on top of #14178.
## Problem
Custom CA support landed first in the login path, but the real
requirement is broader. Codex constructs outbound TLS clients in
multiple places, and both HTTPS and secure websocket paths can fail
behind enterprise TLS interception if they do not honor
`CODEX_CA_CERTIFICATE` or `SSL_CERT_FILE` consistently.
This PR broadens the shared custom-CA logic beyond login and applies the
same policy to websocket TLS, so the enterprise-proxy story is no longer
split between “HTTPS works” and “websockets still fail”.
## What This Delivers
Custom CA support is no longer limited to login. Codex outbound HTTPS
clients and secure websocket connections can now honor the same
`CODEX_CA_CERTIFICATE` / `SSL_CERT_FILE` configuration, so enterprise
proxy/intercept setups work more consistently end-to-end.
For users and operators, nothing new needs to be configured beyond the
same CA env vars introduced in #14178. The change is that more of Codex
now respects them, including websocket-backed flows that were previously
still using default trust roots.
I also manually validated the proxy path locally with mitmproxy using:
`CODEX_CA_CERTIFICATE=~/.mitmproxy/mitmproxy-ca-cert.pem
HTTPS_PROXY=http://127.0.0.1:8080 just codex`
with mitmproxy installed via `brew install mitmproxy` and configured as
the macOS system proxy.
## Mental model
`codex-client` is now the owner of shared custom-CA policy for outbound
TLS client construction. Reqwest callers start from the builder
configuration they already need, then pass that builder through
`build_reqwest_client_with_custom_ca(...)`. Websocket callers ask the
same module for a rustls client config when a custom CA bundle is
configured.
The env precedence is the same everywhere:
- `CODEX_CA_CERTIFICATE` wins
- otherwise fall back to `SSL_CERT_FILE`
- otherwise use system roots
The helper is intentionally narrow. It loads every usable certificate
from the configured PEM bundle into the appropriate root store and
returns either a configured transport or a typed error that explains
what went wrong.
## Non-goals
This does not add handshake-level integration tests against a live TLS
endpoint. It does not validate that the configured bundle forms a
meaningful certificate chain. It also does not try to force every
transport in the repo through one abstraction; it extends the shared CA
policy across the reqwest and websocket paths that actually needed it.
## Tradeoffs
The main tradeoff is centralizing CA behavior in `codex-client` while
still leaving adoption up to call sites. That keeps the implementation
additive and reviewable, but it means the rule "outbound Codex TLS that
should honor enterprise roots must use the shared helper" is still
partly enforced socially rather than by types.
For websockets, the shared helper only builds an explicit rustls config
when a custom CA bundle is configured. When no override env var is set,
websocket callers still use their ordinary default connector path.
## Architecture
`codex-client::custom_ca` now owns CA bundle selection, PEM
normalization, mixed-section parsing, certificate extraction, typed
CA-loading errors, and optional rustls client-config construction for
websocket TLS.
The affected consumers now call into that shared helper directly rather
than carrying login-local CA behavior:
- backend-client
- cloud-tasks
- RMCP client paths that use `reqwest`
- TUI voice HTTP paths
- `codex-core` default reqwest client construction
- `codex-api` websocket clients for both responses and realtime
websocket connections
The subprocess CA probe, env-sensitive integration tests, and shared PEM
fixtures also live in `codex-client`, which is now the actual owner of
the behavior they exercise.
## Observability
The shared CA path logs:
- which environment variable selected the bundle
- which path was loaded
- how many certificates were accepted
- when `TRUSTED CERTIFICATE` labels were normalized
- when CRLs were ignored
- where client construction failed
Returned errors remain user-facing and include the relevant env var,
path, and remediation hint. That same error model now applies whether
the failure surfaced while building a reqwest client or websocket TLS
configuration.
## Tests
Pure unit tests in `codex-client` cover env precedence and PEM
normalization behavior. Real client construction remains in subprocess
tests so the suite can control process env and avoid the macOS seatbelt
panic path that motivated the hermetic test split.
The subprocess coverage verifies:
- `CODEX_CA_CERTIFICATE` precedence over `SSL_CERT_FILE`
- fallback to `SSL_CERT_FILE`
- single-cert and multi-cert bundles
- malformed and empty-file errors
- OpenSSL `TRUSTED CERTIFICATE` handling
- CRL tolerance for well-formed CRL sections
The websocket side is covered by the existing `codex-api` / `codex-core`
websocket test suites plus the manual mitmproxy validation above.
---------
Co-authored-by: Ivan Zakharchanka <3axap4eHko@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
## Why
to support a new bring your own search tool in Responses
API(https://developers.openai.com/api/docs/guides/tools-tool-search#client-executed-tool-search)
we migrating our bm25 search tool to use official way to execute search
on client and communicate additional tools to the model.
## What
- replace the legacy `search_tool_bm25` flow with client-executed
`tool_search`
- add protocol, SSE, history, and normalization support for
`tool_search_call` and `tool_search_output`
- return namespaced Codex Apps search results and wire namespaced
follow-up tool calls back into MCP dispatch
## Summary
- add one-time session recovery in `RmcpClient` for streamable HTTP MCP
`404` session expiry
- rebuild the transport and retry the failed operation once after
reinitializing the client state
- extend the test server and integration coverage for `404`, `401`,
single-retry, and non-session failure scenarios
## Testing
- just fmt
- cargo test -p codex-rmcp-client (the post-rebase run lost its final
summary in the terminal; the suite had passed earlier before the rebase)
- just fix -p codex-rmcp-client
This PR changes stdio MCP child processes to run in their own process
group
* Add guarded teardown in codex-rmcp-client: send SIGTERM to the group
first, then SIGKILL after a short grace period.
* Add terminate_process_group helper in process_group.rs.
* Add Unix regression test in process_group_cleanup.rs to verify wrapper
+ grandchild are reaped on client drop.
Addresses reported MCP process/thread storm: #10581
We started working with MCP in Codex before
https://crates.io/crates/rmcp was mature, so we had our own crate for
MCP types that was generated from the MCP schema:
https://github.com/openai/codex/blob/8b95d3e082376f4cb23e92641705a22afb28a9da/codex-rs/mcp-types/README.md
Now that `rmcp` is more mature, it makes more sense to use their MCP
types in Rust, as they handle details (like the `_meta` field) that our
custom version ignored. Though one advantage that our custom types had
is that our generated types implemented `JsonSchema` and `ts_rs::TS`,
whereas the types in `rmcp` do not. As such, part of the work of this PR
is leveraging the adapters between `rmcp` types and the serializable
types that are API for us (app server and MCP) introduced in #10356.
Note this PR results in a number of changes to
`codex-rs/app-server-protocol/schema`, which merit special attention
during review. We must ensure that these changes are still
backwards-compatible, which is possible because we have:
```diff
- export type CallToolResult = { content: Array<ContentBlock>, isError?: boolean, structuredContent?: JsonValue, };
+ export type CallToolResult = { content: Array<JsonValue>, structuredContent?: JsonValue, isError?: boolean, _meta?: JsonValue, };
```
so `ContentBlock` has been replaced with the more general `JsonValue`.
Note that `ContentBlock` was defined as:
```typescript
export type ContentBlock = TextContent | ImageContent | AudioContent | ResourceLink | EmbeddedResource;
```
so the deletion of those individual variants should not be a cause of
great concern.
Similarly, we have the following change in
`codex-rs/app-server-protocol/schema/typescript/Tool.ts`:
```
- export type Tool = { annotations?: ToolAnnotations, description?: string, inputSchema: ToolInputSchema, name: string, outputSchema?: ToolOutputSchema, title?: string, };
+ export type Tool = { name: string, title?: string, description?: string, inputSchema: JsonValue, outputSchema?: JsonValue, annotations?: JsonValue, icons?: Array<JsonValue>, _meta?: JsonValue, };
```
so:
- `annotations?: ToolAnnotations` ➡️ `JsonValue`
- `inputSchema: ToolInputSchema` ➡️ `JsonValue`
- `outputSchema?: ToolOutputSchema` ➡️ `JsonValue`
and two new fields: `icons?: Array<JsonValue>, _meta?: JsonValue`
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/10349).
* #10357
* __->__ #10349
* #10356
In order to make Codex work with connectors, we add a built-in gateway
MCP that acts as a transparent proxy between the client and the
connectors. The gateway MCP collects actions that are accessible to the
user and sends them down to the user, when a connector action is chosen
to be called, the client invokes the action through the gateway MCP as
well.
- [x] Add the system built-in gateway MCP to list and run connectors.
- [x] Add the app server methods and protocol
Historically, `accept_elicitation_for_prompt_rule()` was flaky because
we were using a notification to update the sandbox followed by a `shell`
tool request that we expected to be subject to the new sandbox config,
but because [rmcp](https://crates.io/crates/rmcp) MCP servers delegate
each incoming message to a new Tokio task, messages are not guaranteed
to be processed in order, so sometimes the `shell` tool call would run
before the notification was processed.
Prior to this PR, we relied on a generous `sleep()` between the
notification and the request to reduce the change of the test flaking
out.
This PR implements a proper fix, which is to use a _request_ instead of
a notification for the sandbox update so that we can wait for the
response to the sandbox request before sending the request to the
`shell` tool call. Previously, `rmcp` did not support custom requests,
but I fixed that in
https://github.com/modelcontextprotocol/rust-sdk/pull/590, which made it
into the `0.12.0` release (see #8288).
This PR updates `shell-tool-mcp` to expect
`"codex/sandbox-state/update"` as a _request_ instead of a notification
and sends the appropriate ack. Note this behavior is tied to our custom
`codex/sandbox-state` capability, which Codex honors as an MCP client,
which is why `core/src/mcp_connection_manager.rs` had to be updated as
part of this PR, as well.
This PR also updates the docs at `shell-tool-mcp/README.md`.
This introduces a new feature to Codex when it operates as an MCP
_client_ where if an MCP _server_ replies that it has an entry named
`"codex/sandbox-state"` in its _server capabilities_, then Codex will
send it an MCP notification with the following structure:
```json
{
"method": "codex/sandbox-state/update",
"params": {
"sandboxPolicy": {
"type": "workspace-write",
"network-access": false,
"exclude-tmpdir-env-var": false
"exclude-slash-tmp": false
},
"codexLinuxSandboxExe": null,
"sandboxCwd": "/Users/mbolin/code/codex2"
}
}
```
or with whatever values are appropriate for the initial `sandboxPolicy`.
**NOTE:** Codex _should_ continue to send the MCP server notifications
of the same format if these things change over the lifetime of the
thread, but that isn't wired up yet.
The result is that `shell-tool-mcp` can consume these values so that
when it calls `codex_core::exec::process_exec_tool_call()` in
`codex-rs/exec-server/src/posix/escalate_server.rs`, it is now sure to
call it with the correct values (whereas previously we relied on
hardcoded values).
While I would argue this is a supported use case within the MCP
protocol, the `rmcp` crate that we are using today does not support
custom notifications. As such, I had to patch it and I submitted it for
review, so hopefully it will be accepted in some form:
https://github.com/modelcontextprotocol/rust-sdk/pull/556
To test out this change from end-to-end:
- I ran `cargo build` in `~/code/codex2/codex-rs/exec-server`
- I built the fork of Bash in `~/code/bash/bash`
- I added the following to my `~/.codex/config.toml`:
```toml
# Use with `codex --disable shell_tool`.
[mcp_servers.execshell]
args = ["--bash", "/Users/mbolin/code/bash/bash"]
command = "/Users/mbolin/code/codex2/codex-rs/target/debug/codex-exec-mcp-server"
```
- From `~/code/codex2/codex-rs`, I ran `just codex --disable shell_tool`
- When the TUI started up, I verified that the sandbox mode is
`workspace-write`
- I ran `/mcp` to verify that the shell tool from the MCP is there:
<img width="1387" height="1400" alt="image"
src="https://github.com/user-attachments/assets/1a8addcc-5005-4e16-b59f-95cfd06fd4ab"
/>
- Then I asked it:
> what is the output of `gh issue list`
because this should be auto-approved with our existing dummy policy:
https://github.com/openai/codex/blob/af63e6eccc35783f1bf4dca3c61adb090efb6b8a/codex-rs/exec-server/src/posix.rs#L157-L164
And it worked:
<img width="1387" height="1400" alt="image"
src="https://github.com/user-attachments/assets/7568d2f7-80da-4d68-86d0-c265a6f5e6c1"
/>
## What?
Fixes MCP server initialization failures on Windows when using
script-based tools like `npx`, `pnpm`, and `yarn` that rely on
`.cmd`/`.bat` files rather than `.exe` binaries.
Fixes#2945
## Why?
Windows users encounter "program not found" errors when configuring MCP
servers with commands like `npx` in their `~/.codex/config.toml`. This
happens because:
- Tools like `npx` are batch scripts (`npx.cmd`) on Windows, not
executable binaries
- Rust's `std::process::Command` bypasses the shell and cannot execute
these scripts directly
- The Windows shell normally handles this by checking `PATHEXT` for
executable extensions
Without this fix, Windows users must specify full paths or add `.cmd`
extensions manually, which breaks cross-platform compatibility.
## How?
Added platform-specific program resolution using the `which` crate to
find the correct executable path:
- **Windows**: Resolves programs through PATH/PATHEXT to find
`.cmd`/`.bat` scripts
- **Unix**: Returns the program unchanged (no-op, as Unix handles
scripts natively)
### Changes
- Added `which = "6"` dependency to `mcp-client/Cargo.toml`
- Implemented `program_resolver` module in `mcp_client.rs` with
platform-specific resolution
- Added comprehensive tests for both Windows and Unix behavior
### Testing
Added platform-specific tests to verify:
- Unix systems execute scripts without extensions
- Windows fails without proper extensions
- Windows succeeds with explicit extensions
- Cross-platform resolution enables successful execution
**Tested on:**
- Windows 11 (NT 10.0.26100.0 x64)
- PowerShell 5.1 & 7+, CMD, Git Bash
- MCP servers: playwright, context7, supabase
- WSL (verified no regression)
**Local checks passed:**
```bash
cargo test && cargo clippy --tests && cargo fmt -- --config imports_granularity=Item
```
### Results
**Before:**
```
🖐 MCP client for `playwright` failed to start: program not found
```
**After:**
```
🖐 MCP client for `playwright` failed to start: request timed out
```
Windows users can now use simple commands like `npx` in their config
without specifying full paths or extensions. The timeout issue is a
separate concern that will be addressed in a follow-up PR.
---------
Co-authored-by: Eric Traut <etraut@openai.com>
This makes stdio mcp servers more flexible by allowing users to specify
the cwd to run the server command from and adding additional environment
variables to be passed through to the server.
Example config using the test server in this repo:
```toml
[mcp_servers.test_stdio]
cwd = "/Users/<user>/code/codex/codex-rs"
command = "cargo"
args = ["run", "--bin", "test_stdio_server"]
env_vars = ["MCP_TEST_VALUE"]
```
@bolinfest I know you hate these env var tests but let's roll with this
for now. I may take a stab at the env guard + serial macro at some
point.
This adds two new config fields to streamable http mcp servers:
`http_headers`: a map of key to value
`env_http_headers` a map of key to env var which will be resolved at
request time
All headers will be passed to all MCP requests to that server just like
authorization headers.
There is a test ensuring that headers are not passed to other servers.
Fixes#5180
This lets users/companies explicitly choose whether to force/disallow
the keyring/fallback file storage for mcp credentials.
People who develop with Codex will want to use this until we sign
binaries or else each ad-hoc debug builds will require keychain access
on every build. I don't love this and am open to other ideas for how to
handle that.
```toml
mcp_oauth_credentials_store = "auto"
mcp_oauth_credentials_store = "file"
mcp_oauth_credentials_store = "keyrung"
```
Defaults to `auto`
`http_config.auth_header` automatically added `Bearer `. By adding it
ourselves, we were sending `Bearer Bearer <token>`.
I confirmed that the GitHub MCP initialization 400s before and works
now.
I also optimized the oauth flow to not check the keyring if you
explicitly pass in a bearer token.
This PR adds oauth login support to streamable http servers when
`experimental_use_rmcp_client` is enabled.
This PR is large but represents the minimal amount of work required for
this to work. To keep this PR smaller, login can only be done with
`codex mcp login` and `codex mcp logout` but it doesn't appear in `/mcp`
or `codex mcp list` yet. Fingers crossed that this is the last large MCP
PR and that subsequent PRs can be smaller.
Under the hood, credentials are stored using platform credential
managers using the [keyring crate](https://crates.io/crates/keyring).
When the keyring isn't available, it falls back to storing credentials
in `CODEX_HOME/.credentials.json` which is consistent with how other
coding agents handle authentication.
I tested this on macOS, Windows, WSL (ubuntu), and Linux. I wasn't able
to test the dbus store on linux but did verify that the fallback works.
One quirk is that if you have credentials, during development, every
build will have its own ad-hoc binary so the keyring won't recognize the
reader as being the same as the write so it may ask for the user's
password. I may add an override to disable this or allow
users/enterprises to opt-out of the keyring storage if it causes issues.
<img width="5064" height="686" alt="CleanShot 2025-09-30 at 19 31 40"
src="https://github.com/user-attachments/assets/9573f9b4-07f1-4160-83b8-2920db287e2d"
/>
<img width="745" height="486" alt="image"
src="https://github.com/user-attachments/assets/9562649b-ea5f-4f22-ace2-d0cb438b143e"
/>
This PR adds support for streamable HTTP MCP servers when the
`experimental_use_rmcp_client` is enabled.
To set one up, simply add a new mcp server config with the url:
```
[mcp_servers.figma]
url = "http://127.0.0.1:3845/mcp"
```
It also supports an optional `bearer_token` which will be provided in an
authorization header. The full oauth flow is not supported yet.
The config parsing will throw if it detects that the user mixed and
matched config fields (like command + bearer token or url + env).
The best way to review it is to review `core/src` and then
`rmcp-client/src/rmcp_client.rs` first. The rest is tests and
propagating the `Transport` struct around the codebase.
Example with the Figma MCP:
<img width="5084" height="1614" alt="CleanShot 2025-09-26 at 13 35 40"
src="https://github.com/user-attachments/assets/eaf2771e-df3e-4300-816b-184d7dec5a28"
/>
stevenlee-oai