Commit Graph
72 Commits
Author SHA1 Message Date
Francis Chalissery e776b27eb0 Add frozen cloud skill snapshots to the app server (#55631)
## What changed

- Add `--cloud-skills-snapshot PATH` to load a cloud skill catalog and resource contents from JSON once at app-server startup, without contacting a live skill service.
- Expose snapshot skills through the native `skills.list` and `skills.read` interface, preserving aliases and prompt visibility. Reads remain available after the snapshot file is removed.
- Validate package boundaries, required resources, duplicate packages, and size limits; fail startup for missing or invalid snapshots.
- Keep frozen cloud catalog and read caches independent of live MCP resource changes.

## Testing

Add provider, cache, app-server, and core scenario tests covering validation, exact resource contents, authority boundaries, startup failures, and listing and reading frozen skills alongside a named executor.

GitOrigin-RevId: d438a5d678121db3e17515fc5edcd00d4e20b900
2026-10-20 17:01:52 +00:00
smurching-oai 7943713993 Reload the model catalog after account changes (#54169)
## Why

The app server retained its startup model provider for catalog requests, so signing in or out could leave `model/list` using stale provider settings or requiring a restart after managed provider requirements changed.

## What changed

- Invalidate the catalog after successful login and logout, then reload the effective configuration on the next model list or background refresh. Rebuild the models manager when the provider or catalog changes, while preserving existing threads' routes.
- Retain the original project context and the TUI's `--oss` provider selection across reloads. Retry configuration loading after falling back to defaults so temporary failures do not pin the fallback catalog.
- Keep provider policy checks for cached catalogs and apply runtime `api_key_model_discovery` toggles to the active catalog manager.

## Testing

Add and extend tests for provider selection and definition changes after login, catalog reloads after logout, OAuth gateway transitions, fallback recovery, project context, and runtime model discovery toggles.

GitOrigin-RevId: bce2e23ff567e1a52eac2b6ad736a6e63ff61e2b
2026-10-15 03:06:16 +00:00
Eddie Chen cfc946f4e1 Expose and persist turn lineage across the app server (#51415)
## Why

Clients starting delegated work need to attribute descendant turns to the original turn and retain that attribution through completion and history reads.

## What changed

- Accept optional `parentTurnId` and `rootTurnId` in `turn/start`, and expose `rootTurnId` on returned turns and lifecycle notifications.
- Default new turns to their own root when none is supplied. Preserve the active turn's root when steering, ignoring incoming lineage fields.
- Carry root IDs through completion, interruption, recovery, and fork boundaries, and persist them for paginated history reads. Leave unknown roots null in older history.
- Update protocol schemas and generated TypeScript and Python bindings.

## Testing

Extend coverage for default and explicit roots, steering without changing lineage, interrupted and recovered turns, history reads after restart in both history modes, nested fork pagination, and compatibility with older database writers.

GitOrigin-RevId: f1a570a12bd11742929fd2e8b017e5d35aa07dcb
2026-10-06 16:56:39 +00:00
Dmitriy Kovalenko 3d6428510f Warn users when SQLite diagnostic log writes fail (#47887)
## Why

Failed SQLite writes leave diagnostic logs incomplete. Notify users before they close Codex so they can submit logs or run `codex doctor`.

## What changed

- Add a best-effort warning on the first log write failure in standalone and in-process app servers, while continuing to retain failure diagnostics in memory.
- Handle failures that occurred before the in-process warning reporter was attached.
- Tailor the warning's guidance to whether submitting logs is enabled.

## Testing

Add coverage for warning deduplication, both guidance variants, retained diagnostics after the notification sender is dropped, and SQLite flush failures before and after reporter attachment. Extend log upload integration coverage and add a TUI warnings snapshot.

GitOrigin-RevId: 12a74126d13c3568430e539a180cca47fa931fa9
2026-09-24 17:04:50 +00:00
acrognale-oai 9d4d34d436 Apply shared network policy throughout embedded Codex startup (#47411)
## Why

Application network restrictions need to cover clients created by the TUI and `codex exec` before embedded app-server startup, and remain effective when configuration is rebuilt.

## What changed

- Introduce `EmbeddedNetworkPolicy` to share live policy between caller-owned transports and the embedded app-server.
- Apply loaded requirements before creating telemetry, background HTTP clients, and executor connections. Limit authentication and cloud configuration bootstrap requests to exact endpoints under local policy.
- Preserve the shared policy across TUI session changes, permission-profile rebuilds, and worktree cloud loaders so existing clients observe policy reloads and failures.
- Document application network policy behavior and transport coverage.

## Testing

Add regression coverage for policy activation, reload failure and recovery, retained policy across TUI rebuilds, rejected OAuth and cloud-policy redirects, and workspace network restrictions with `CODEX_API_KEY`. Also cover retaining the caller-selected project during embedded startup.

GitOrigin-RevId: 629009f087b8cb21fc34a1ee7ea6fe95d9e0f2b5
2026-09-23 00:49:19 +00:00
acrognale-oai 22a3f6d5d8 Enforce application network policy across app-server requests (#47407)
## What changed

- Load `application.network` requirements at startup and explicit config/account reloads, applying local policy before authentication and cloud configuration bootstrap.
- Block traffic when policy loading fails, cancel requests that a new policy no longer permits, and reject configuration loads based on superseded policy snapshots. Local requirements edits take effect on the next explicit reload or restart.
- Bind authenticated clients to the current account and revoke them when account ownership changes. Carry policy enforcement through backend requests, realtime connections, workload identity exchanges, and code-mode gRPC streams while preserving gRPC trailers.
- Treat denied credential refreshes as policy errors without retrying them or invalidating stored credentials.

## Testing

Add regression coverage for startup restrictions, reload ordering, cancellation after restrictive or malformed requirements edits, account revocation, gRPC destination denial, workload identity cancellation, and credential preservation after denied OAuth refresh.

GitOrigin-RevId: 0cb4c720a906397e9df06675f01ecf60d741101f
2026-09-23 00:47:07 +00:00
joeytrasatti-openai 3319d9b296 Add app-server APIs for stored thread attachments (#44564)
## What changed

- Add `thread/attachment/add`, `thread/attachment/list`, and `thread/attachment/remove` to manage durable resource references without loading the owning thread or changing conversation history.
- Identify attachments by thread, `attachmentType`, and `identityKey`. Repeated adds return the existing attachment; repeated removals succeed without emitting another update. Listing supports cursor pagination.
- Broadcast `thread/attachment/updated` after creation or deletion, after responding to the requester. Serialize attachment mutations with thread lifecycle operations and reject stores that do not support attachments.
- Update protocol schemas, TypeScript and Python bindings, documentation, and TUI notification handling so attachment updates do not add untracked threads to the agent picker.

## Testing

Add coverage for unloaded threads, pagination, idempotent addition and removal, reattachment, invalid inputs, unsupported stores, response-before-notification ordering across clients, and TUI notification routing.

GitOrigin-RevId: 0d29cf903afd4e931a70a9cf96300475213b34aa
2026-09-10 15:52:09 +00:00
riley-oai ca6fb194b6 Wire app-server user verification RPCs to the native provider (#43568)
## Why

The existing `userVerification/*` handlers always returned unavailability, leaving local clients unable to use the native verification provider.

## What changed

- Dispatch status, enrollment, deletion, and challenge signing through an account-scoped provider. Enrollment creates or reuses a local key; it does not register credentials with a server.
- Restrict enrollment, deletion, and signing to stdio and in-process connections, rejecting WebSocket and remote-control callers before parameter validation.
- Bound native work to one operation per service with a 120-second timeout, and propagate in-process caller cancellation. Recheck connection, cancellation, and authentication state before delivering results, including after waiting for response queue capacity.
- Add device-gated verification capability wiring for the experimental in-process `codex-tui` client, with automatic activation still disabled by default.

## Testing

Add tests for local key reuse and deletion, decoded challenge signing, cancellation, authentication changes during verification and response queuing, concurrent worker rejection, and stdio/WebSocket RPC behavior.

GitOrigin-RevId: 7eac15a5ffb049d27bf8fedcfbae19a49a3f6a3d
2026-09-07 19:34:23 +00:00
riley-oai 555b82afa9 Add opt-in MCP user-verification transport (#43352)
## Why

Device-authenticated verification needs an app response outside automated approval and review. Previously, MCP user-verification requests were always cancelled.

## What changed

- Add typed `openai/userVerification` elicitations carrying a title, description, and challenge through core and app-server.
- Route verification to one connection enabled by trusted host activation. Restrict responses to that owner, exclude requests from replay, and cancel on disconnect or authentication changes, including account switches away and back.
- Return proofs in `content`, discard response metadata, and cancel malformed acceptances. Suppress verification response logging and analytics, and send only a generic verification notice to realtime.
- Keep the mode experimental and exclude it from stable schema exports. Capability advertisement remains disabled, and the TUI cancels verification requests.

## Testing

Add coverage for owner isolation, disconnect and authentication races, proof response handling, automated-review bypass, inactive-host cancellation, stable schema filtering, and realtime payload exclusion. Update the MCP integration test to verify that an activated request holds the tool until the client responds.

GitOrigin-RevId: c8c7b6691a3f7756682bbcca9def1df0d1feb906
2026-09-07 04:33:22 +00:00
rka-oai 2c79ee6dac Add structured asynchronous user input requests (#42178)
## What changed

- Replace `send_user_message_async` with `request_user_input_async`, accepting one or more questions with optional suggested answers while allowing the turn to continue.
- Attach structured question metadata to asynchronous agent messages while retaining readable fallback text, and preserve it through app-server events, thread history, and generated schemas.
- Continue enabling the new tool for model catalogs that advertise either the old or new tool name.

## Testing

- Cover tool registration, question validation, emitted items, continued turn execution, history preservation, and backwards-compatible deserialization.

GitOrigin-RevId: ffc49aebde2e854c3c50a122aa52805fec3fc6b3
2026-09-02 02:09:35 +00:00
willwang-openai 68c9556cdf Upgrade Git marketplaces from merged configuration (#42149)
## Why

Configured Git marketplaces may be defined outside the user config, but those definitions could not download or update their snapshots.

## What changed

- Select marketplace source, ref, and sparse-path settings from the effective merged configuration for startup synchronization and explicit upgrades.
- Preserve the initiating operation's configuration loader and overrides when reloading before activation.
- Roll back activation if the marketplace definition changed or configuration can no longer be loaded, without copying settings into the user config.

## Testing

- Cover system-defined marketplaces, CLI overrides, startup fallback after invalid user configuration, and rollback after configuration changes.

GitOrigin-RevId: 53d1eecf68a91ce2aa8430e5749d78697bfcd821
2026-09-01 22:37:26 +00:00
victor-openai 097825f75a Add app-server MCP event streaming (#39761)
## What changed

- Add experimental `mcpServer/event/stream/start` and `mcpServer/event/stream/stop` requests for hosted apps, plus `mcpServer/event/stream/notification` forwarding.
- Scope subscriptions to the owning app-server connection and subscribed thread, enforce unique IDs and a per-connection limit, and clean them up when the thread is unsubscribed or the connection closes.
- Wait for the MCP active notification before completing startup, retry streams that close immediately, and terminate them when authentication or hosted runtime ownership changes.

## Testing

- Add an app-server integration test covering activation, event forwarding, duplicate subscription rejection, and explicit cancellation.

GitOrigin-RevId: 8a6fc1615adfc5af7e67def3b824fa5909ab3e7b
2026-08-20 17:49:35 +00:00
rka-oai fb356f3d2c Add async delivery metadata to agent messages (#39312)
## What changed

- Add an optional `delivery` field to agent message events and app-server
  `agentMessage` items, with `"async"` identifying a user-visible message sent
  without ending the current turn.
- Preserve the marker through legacy event conversion, thread history
  materialization, replay, and generated JSON and TypeScript schemas.
- Require delivery of async agent message completion notifications on the
  in-process app-server transport.

## Testing

- Cover preservation of async delivery metadata in thread history.
- Cover delivery-required classification for async agent messages.

GitOrigin-RevId: f92ba25de1293ee271404badfdf0d117d6530329
2026-08-18 23:05:52 +00:00
efrazer-oai 9341b38310 Add experimental thread queue APIs to app server (#38456)
## What changed

- Add experimental `thread/queue/add`, `list`, `update`, `delete`, `reorder`, and `start` requests for persistent queued user submissions.
- Automatically dispatch queued submissions in FIFO order after completed or failed turns, while leaving the queue paused after interruption.
- Emit `thread/queue/changed` notifications and expose paginated queue contents with stable submission and client message IDs.
- Export the new protocol types through the generated JSON Schema and TypeScript bindings.

## Testing

- Add app-server integration coverage for capability gating, CRUD, pagination, capacity limits, notifications, automatic dispatch, cold-thread resume, interruption, and explicit queue starts.

GitOrigin-RevId: 89caf5577192199bda77cf6c372a59d26cdd910e
2026-08-13 23:38:02 +00:00
cooper-oai 990218bbbd Fail closed when workload identity initialization fails (#38424)
## What changed

- Treat workload identity environment markers as an explicit authentication selection, even when another process credential is present.
- Return initialization errors from `AuthManager` and propagate them through commands and services instead of continuing with an unusable authentication state.
- Make `codex login status` validate workload identity, keep the TUI on an embedded app server for local workload identity, and reject workload identity in `codex mcp-server`, where it is unsupported.

## Testing

- Cover workload identity precedence and partial configuration errors.
- Verify login status reports an unreadable identity assertion and app-server routing enforces the supported workload identity topology.

GitOrigin-RevId: efc6b6b4cd4d61652617de82aaa3d7ffc75d6618
2026-08-13 19:35:22 +00:00
stevenlee-oai 285abc0368 Configure PSP routing through the feature system (#38056)
## What changed

- Add the under-development `psp` feature and expose it in the config schema.
- Use the feature to attach the PSP cookie to first-party ChatGPT clients.
- Remove the hidden `--psp` flag and its process-scoped configuration plumbing.
- Preserve configured ChatGPT cookies when creating the PSP client used for GET and POST requests.

## Testing

- Update the config manager service test to verify that enabling `features.psp` retains the setting in the effective config and configures the expected ChatGPT cookie.

GitOrigin-RevId: 53acb5495d2ff71e4ed25f674a0cff787aea474a
2026-08-11 19:16:15 +00:00
Abhinav 2994f545a7 Enforce managed authentication requirements locally (#37132)
## Why

Authentication restrictions must apply before stored or environment-provided credentials can be used, including during bootstrap before cloud requirements are fetched.

## What changed

- Add local `requirements.toml` allowlists for login methods and ChatGPT workspaces. Ignore these fields in cloud-provided requirements.
- Combine managed workspace allowlists with existing workspace restrictions by intersection, and fail closed when the resulting policy permits no usable login method.
- Centralize policy checks in the authentication manager so CLI, TUI, app-server, external-auth, and credential-loading paths consistently reject disallowed authentication before token hydration or network requests.

## Testing

- Cover policy composition, workspace intersection, invalid stored and external credentials, bootstrap enforcement, and login endpoint restrictions.

GitOrigin-RevId: efef22b248f3c3333e9aa55423e539efa2d2dd48
2026-08-05 18:09:26 +00:00
stevenlee-oai 1d952f027e Add process-scoped PSP routing for ChatGPT requests (#36986)
## What changed

- Add a hidden global `--psp` runtime flag and propagate it through TUI, exec,
  app-server, remote-control, and in-process startup paths.
- Attach the `oai-chat-psp=true` cookie to first-party ChatGPT requests when
  enabled, using a cached cookie-aware client with sensitive request logging
  disabled.
- Keep the routing selection out of persistent configuration layers while
  preserving it across config refreshes and agent role changes.

## Testing

- Cover global flag parsing, app-server propagation, config-layer isolation,
  and preservation across config rebuilds and role changes.

GitOrigin-RevId: 05cdc61ffd7162d8e48fc1e166f4732113e5a816
2026-08-04 22:23:40 +00:00
acrognale-oai c41a38dd10 Reduce response serialization and rollout scan overhead (#36006)
## What changed

- Keep `ClientResponsePayload` typed through the app-server outgoing queue and serialize it at the transport boundary, avoiding an intermediate `serde_json::Value` while preserving the JSON-RPC wire shape.
- Return a JSON-RPC internal error when a response cannot be serialized, including over remote control, and skip unserializable analytics responses.
- Scan reverse JSONL input a chunk at a time with a larger buffer instead of processing each byte individually.

## Testing

- Cover typed response serialization and serialization failures for direct and remote-control transports.
- Cover filtering unserializable responses from analytics.

GitOrigin-RevId: cc4fd7a43461a273315bb95261bd1ae3ff65fbf7
2026-07-29 18:29:26 +00:00
joeytrasatti-openai 07490c7523 Box app-server event payloads (#35854)
## What changed

- Store `ServerNotification` and `ServerRequest` payloads behind `Box` in
  `AppServerEvent`, `InProcessServerEvent`, and TUI `ThreadBufferedEvent` values.
- Update app-server delivery, TUI routing and replay, exec, onboarding, and tests
  to borrow or consume the boxed payloads as appropriate.

GitOrigin-RevId: 7d0dab9aba47487114128ce55c914af5bb46c5b5
2026-07-29 00:30:11 +00:00
Felipe Coury 5a3808b77d Shut down the in-process outbound router explicitly (#35523)
## Why

Detached processor work can retain outgoing message senders, so waiting for
channel closure can keep the in-process app-server's outbound router alive
during shutdown.

## What changed

Add an explicit shutdown signal for the outbound router and prioritize it over
pending outgoing messages. Send that signal after processor tasks are stopped
so runtime shutdown no longer depends on every sender being dropped.

## Testing

Add a paused-time test that verifies the router exits and closes its channel
even when an outgoing sender is still retained.

GitOrigin-RevId: a8f88d88a0950f50339b1c56909fedb781f178d4
2026-07-26 22:11:26 +00:00
Channing Conger f61b51ddd9 Support remote code-mode hosts in app-server (#35098)
## What changed

- Add `--code-mode-host ws://...` and `wss://...` support to `codex app-server`, gated by the `code_mode_host` feature. When omitted, app-server continues to start a local host.
- Share one remote WebSocket connection across the process's threads, using the configured HTTP client's proxy and TLS policy and preserving the existing framed host protocol.
- Reject invalid host URLs, bound WebSocket frame sizes, close connections cleanly, and return an error when a connection exceeds 1,024 pending delegate calls without disconnecting it.

## Testing

- Cover CLI validation, WebSocket protocol execution and shutdown, connection sharing across app-server threads, and delegate-call capacity recovery.

GitOrigin-RevId: 715e82d4d9db1e7e2f91b754a777dcab504e2ae4
2026-07-24 04:37:01 +00:00
Ben Romano 88f1cd9664 Flush analytics before in-process app server shutdown (#34831)
## Why

The in-process app server can shut down while analytics for a completed turn are
still queued, preventing the completed-turn and accepted-line events from being
delivered.

## What changed

- Add a bounded analytics queue flush that waits for all preceding facts to be
  processed.
- Flush analytics after the in-process runtime drains its outbound work and
  before acknowledging shutdown.
- Extend the in-process shutdown timeout budgets to cover runtime draining and
  the best-effort analytics flush.

## Testing

- Verify that the flush waits for preceding facts and is a no-op when analytics
  is disabled.
- Verify that shutdown waits for delayed analytics delivery, including the
  completed-turn and accepted-line events produced by a file change.

GitOrigin-RevId: 57ca24e535e5e2f3f3202b2e5498f075676c910a
2026-07-22 21:35:16 +00:00
Owen Lin c39520f3d1 Timestamp app-server notifications at emission (#32905)
## What changed

- Add an optional `emittedAtMs` field to server notification envelopes and expose the envelope in the generated JSON and TypeScript schemas.
- Populate the Unix timestamp when app-server emits a notification, before transport routing, so every connection in a fan-out receives the same value.
- Keep the field optional for clients decoding notifications from older app-server versions.

## Testing

- Verify notification serialization includes `emittedAtMs`.
- Verify targeted fan-out reuses one timestamp across connections.

GitOrigin-RevId: b379cb11e8e91c558bcd3d46ce459311a6e81e54
2026-07-14 00:38:04 +00:00
Eric Traut 8f5bb6171e Remove TUI exec-policy core exports (#31179)
## Why

The TUI still reached through `codex_app_server_client::legacy_core` to
validate exec-policy rules during local startup. The app server already
owns this validation and reports parse failures through `configWarning`,
so the duplicate TUI preflight preserved an unnecessary core dependency
and could inspect the wrong machine when connected to a remote app
server.

## What changed

- Remove the TUI's direct exec-policy startup check.
- Remove the two exec-policy re-exports from `legacy_core`.
- Rely on the app server's existing config-warning flow for malformed
custom rules in both embedded and remote sessions.
2026-07-06 10:22:25 -07:00
charlesgong-openai fc1fb682a7 [codex] Add external agent import result accounting (#28008)
## Why

External-agent imports can complete synchronously or continue in the
background for plugins/sessions. Clients need a stable import id to
correlate the immediate response with the eventual completion
notification, and the completion payload needs enough accounting to show
which artifact types succeeded or failed without hiding partial
failures.

## What Changed

- `externalAgentConfig/import` now returns an `importId`;
`externalAgentConfig/import/completed` includes the same `importId` plus
type-level `itemResults`.
- Completed `itemResults` report `successCount`, `errorCount`,
`successes`, and `rawErrors` for each migrated item type.
- Added protocol/schema/TypeScript types for import successes, raw
errors, and type-level results. No progress notification is included in
the final PR.
- `ExternalAgentConfigService::import` now returns an outcome object
with synchronous item results and pending plugin imports.
- Plugin import outcomes track succeeded/failed marketplaces, plugin
ids, and raw errors. Plugin failures can be reported in completed
accounting while later migration items continue.
- Non-plugin synchronous import failures still fail the request, so
invalid config/skills-style failures are not reported as a successful
import response.
- Session imports now return item results. Successful imports include
the source session path and imported thread id; prepare, persist,
ledger, and source-validation failures become raw errors in completion
accounting where the import can continue.
- The request processor generates the `importId`, aggregates synchronous
results with background plugin/session results, and sends a single
completed notification when all selected work is done.
- App-server docs and generated schema fixtures were updated for the new
response/completed payload shapes.

## Validation

- `just test -p codex-app-server-protocol`
- `just test -p codex-app-server-client event_requires_delivery`
- `CODEX_SQLITE_HOME=/private/tmp/codex-app-server-review-sync-error
just test -p codex-app-server
external_agent_config_import_returns_error_for_failed_sync_import`
- `CODEX_SQLITE_HOME=/private/tmp/codex-app-server-review-external-agent
just test -p codex-app-server external_agent_config`

Note: local sandbox validation used `CODEX_SQLITE_HOME` because the
default sqlite state path is read-only in this environment.
2026-06-15 13:25:42 -07:00
stefanstokic-oai b4445f2758 [codex] add /import for external agents (#27071)
## Why

External-agent import should be discoverable and deliberate without
blocking startup or claiming the public `codex [PROMPT]` CLI namespace.
The slash command keeps the flow local to the interactive TUI and reuses
the existing app-server import API.

## What changed

- add the user-facing `/import` slash command
- detect external-agent importable items only when the command is
invoked
- run imports through the embedded local app-server
- show start and completion messages, refresh configuration, and block
duplicate imports while one is pending
- reject the flow for unsupported remote and local-daemon sessions

## Validation

- `just test -p codex-tui external_agent_config_migration` (10 passed)
- manually exercised an isolated TUI fixture with existing
external-agent setup and session data using a fresh `CODEX_HOME`
- verified picker customization, plugin and session detection, import
completion, repeated invocation, and imported-session resume context
- the broader `just test -p codex-tui` run passed 2,805 tests, with 2
unrelated guardian feature-flag failures and 4 skipped tests

## Draft follow-ups

- review whether completion messaging should remain attached to the
initiating chat if the user switches chats during an import
- review shutdown semantics for an in-progress background import

## Stack

1. [#27064](https://github.com/openai/codex/pull/27064): remove the
startup migration flow
2. [#27065](https://github.com/openai/codex/pull/27065): extract the
picker renderer
3. [#27070](https://github.com/openai/codex/pull/27070): add the
external-agent import picker UX
4. [#27071](https://github.com/openai/codex/pull/27071): expose the flow
through `/import`

**This PR is stack item 4.** Draft while the lower stack dependencies
are reviewed.
2026-06-10 15:53:15 -04:00
joeflorencio-openai d45cd26248 Switch runtime to cloud config bundle (#24622)
## Summary

- Adapts the moved `codex-cloud-config` crate from the legacy cloud
requirements endpoint to the new config bundle endpoint.
- Switches runtime consumers from `CloudRequirementsLoader` to
`CloudConfigBundleLoader` so one shared bundle supplies cloud-delivered
config and requirements.
- Removes the legacy cloud requirements domain loader path.

## Details

This intentionally keeps `codex-cloud-config` monolithic for review
lineage: the previous PR establishes the crate move, and this PR shows
the behavior change against that moved implementation. A follow-up PR
splits the module back into focused files.

The new bundle path preserves the important cloud requirements loader
semantics where intended: account-scoped signed cache, 30 minute TTL, 5
minute refresh cadence, retry/backoff, auth recovery, and fail-closed
startup loading. The cached payload changes from a single requirements
TOML string to the backend-delivered bundle, and validation rejects
malformed config or requirements fragments before cache write/use.
2026-06-02 13:18:59 -07:00
Eric Traut 771a4e74ac Add thread/settings/update app-server API (#23502)
## Why

App-server clients need a way to update a thread's next-turn settings
without starting a turn, adding transcript content, or waiting for turn
lifecycle events. This gives settings UI a direct path for durable
thread settings while clients observe the eventual effective state
through a notification.

This is a simplified rework of PR
https://github.com/openai/codex/pull/22509. In particular, it changes
the `thread/settings/update` api to return immediately rather than
waiting and returning the effective (updated) thread settings. This
makes the new api consistent with `turn/start` and greatly reduces the
complexity of the implementation relative to the earlier attempt.

## What Changed

- Adds experimental `thread/settings/update` with partial-update request
fields and an empty acknowledgment response.
- Adds experimental `thread/settings/updated`, carrying full effective
`ThreadSettings` and scoped by `threadId` to subscribed clients for the
affected thread.
- Shares durable settings validation with `turn/start`, including
`sandboxPolicy` plus `permissions` rejection and `serviceTier: null`
clearing.
- Emits the same settings notification when `turn/start` overrides
change the stored effective thread settings.
- Regenerates app-server protocol schema fixtures and updates
`app-server/README.md`.
2026-05-20 11:03:20 -07:00
Owen Lin 2b3b220605 revert: mark Feature::RemoteControl as removed (#22520)
reverts: https://github.com/openai/codex/pull/22386
2026-05-13 17:32:15 +00:00
Michael Bolin 889ee018e7 config: add strict config parsing (#20559)
## Why

Codex intentionally ignores unknown `config.toml` fields by default so
older and newer config files keep working across versions. That leniency
also makes typo detection hard because misspelled or misplaced keys
disappear silently.

This change adds an opt-in strict config mode so users and tooling can
fail fast on unrecognized config fields without changing the default
permissive behavior.

This feature is possible because `serde_ignored` exposes the exact
signal Codex needs: it lets Codex run ordinary Serde deserialization
while recording fields Serde would otherwise ignore. That avoids
requiring `#[serde(deny_unknown_fields)]` across every config type and
keeps strict validation opt-in around the existing config model.

## What Changed

### Added strict config validation

- Added `serde_ignored`-based validation for `ConfigToml` in
`codex-rs/config/src/strict_config.rs`.
- Combined `serde_ignored` with `serde_path_to_error` so strict mode
preserves typed config error paths while also collecting fields Serde
would otherwise ignore.
- Added strict-mode validation for unknown `[features]` keys, including
keys that would otherwise be accepted by `FeaturesToml`'s flattened
boolean map.
- Kept typed config errors ahead of ignored-field reporting, so
malformed known fields are reported before unknown-field diagnostics.
- Added source-range diagnostics for top-level and nested unknown config
fields, including non-file managed preference source names.

### Kept parsing single-pass per source

- Reworked file and managed-config loading so strict validation reuses
the already parsed `TomlValue` for that source.
- For actual config files and managed config strings, the loader now
reads once, parses once, and validates that same parsed value instead of
deserializing multiple times.
- Validated `-c` / `--config` override layers with the same
base-directory context used for normal relative-path resolution, so
unknown override keys are still reported when another override contains
a relative path.

### Scoped `--strict-config` to config-heavy entry points

- Added support for `--strict-config` on the main config-loading entry
points where it is most useful:
  - `codex`
  - `codex resume`
  - `codex fork`
  - `codex exec`
  - `codex review`
  - `codex mcp-server`
  - `codex app-server` when running the server itself
  - the standalone `codex-app-server` binary
  - the standalone `codex-exec` binary
- Commands outside that set now reject `--strict-config` early with
targeted errors instead of accepting it everywhere through shared CLI
plumbing.
- `codex app-server` subcommands such as `proxy`, `daemon`, and
`generate-*` are intentionally excluded from the first rollout.
- When app-server strict mode sees invalid config, app-server exits with
the config error instead of logging a warning and continuing with
defaults.
- Introduced a dedicated `ReviewCommand` wrapper in `codex-rs/cli`
instead of extending shared `ReviewArgs`, so `--strict-config` stays on
the outer config-loading command surface and does not become part of the
reusable review payload used by `codex exec review`.

### Coverage

- Added tests for top-level and nested unknown config fields, unknown
`[features]` keys, typed-error precedence, source-location reporting,
and non-file managed preference source names.
- Added CLI coverage showing invalid `--enable`, invalid `--disable`,
and unknown `-c` overrides still error when `--strict-config` is
present, including compound-looking feature names such as
`multi_agent_v2.subagent_usage_hint_text`.
- Added integration coverage showing both `codex app-server
--strict-config` and standalone `codex-app-server --strict-config` exit
with an error for unknown config fields instead of starting with
fallback defaults.
- Added coverage showing unsupported command surfaces reject
`--strict-config` with explicit errors.

## Example Usage

Run Codex with strict config validation enabled:

```shell
codex --strict-config
```

Strict config mode is also available on the supported config-heavy
subcommands:

```shell
codex --strict-config exec "explain this repository"
codex review --strict-config --uncommitted
codex mcp-server --strict-config
codex app-server --strict-config --listen off
codex-app-server --strict-config --listen off
```

For example, if `~/.codex/config.toml` contains a typo in a key name:

```toml
model = "gpt-5"
approval_polic = "on-request"
```

then `codex --strict-config` reports the misspelled key instead of
silently ignoring it. The path is shortened to `~` here for readability:

```text
$ codex --strict-config
Error loading config.toml:
~/.codex/config.toml:2:1: unknown configuration field `approval_polic`
  |
2 | approval_polic = "on-request"
  | ^^^^^^^^^^^^^^
```

Without `--strict-config`, Codex keeps the existing permissive behavior
and ignores the unknown key.

Strict config mode also validates ad-hoc `-c` / `--config` overrides:

```text
$ codex --strict-config -c foo=bar
Error: unknown configuration field `foo` in -c/--config override

$ codex --strict-config -c features.foo=true
Error: unknown configuration field `features.foo` in -c/--config override
```

Invalid feature toggles are rejected too, including values that look
like nested config paths:

```text
$ codex --strict-config --enable does_not_exist
Error: Unknown feature flag: does_not_exist

$ codex --strict-config --disable does_not_exist
Error: Unknown feature flag: does_not_exist

$ codex --strict-config --enable multi_agent_v2.subagent_usage_hint_text
Error: Unknown feature flag: multi_agent_v2.subagent_usage_hint_text
```

Unsupported commands reject the flag explicitly:

```text
$ codex --strict-config cloud list
Error: `--strict-config` is not supported for `codex cloud`
```

## Verification

The `codex-cli` `strict_config` tests cover invalid `--enable`, invalid
`--disable`, the compound `multi_agent_v2.subagent_usage_hint_text`
case, unknown `-c` overrides, app-server strict startup failure through
`codex app-server`, and rejection for unsupported commands such as
`codex cloud`, `codex mcp`, `codex remote-control`, and `codex
app-server proxy`.

The config and config-loader tests cover unknown top-level fields,
unknown nested fields, unknown `[features]` keys, source-location
reporting, non-file managed config sources, and `-c` validation for keys
such as `features.foo`.

The app-server test suite covers standalone `codex-app-server
--strict-config` startup failure for an unknown config field.

## Documentation

The Codex CLI docs on developers.openai.com/codex should mention
`--strict-config` as an opt-in validation mode for supported
config-heavy entry points once this ships.
2026-05-13 16:08:05 +00:00
Owen Lin 2237a13cf1 mark Feature::RemoteControl as removed (#22386)
## Why

`remote_control` can appear in `config.toml`, CLI feature overrides, and
the app-server config APIs. Before this PR, app-server startup treated
`config.features.enabled(Feature::RemoteControl)` as the signal to start
remote control ([base
code](https://github.com/openai/codex/blob/5e3ee5eddfa5333f2e0b011880abf0cbf92bd295/codex-rs/app-server/src/lib.rs#L678-L680)).
That meant a user with:

```toml
[features]
remote_control = true
```

would accidentally opt every app-server process into remote control.
Remote-control startup should instead be a per-process launch decision
made by CLI flags.

## What Changed

- Marks `Feature::RemoteControl` as `Stage::Removed`, keeping
`remote_control` as a known compatibility key while making it
config-inert.
- Adds a hidden `--remote-control` process flag to `codex app-server`
and standalone `codex-app-server`.
- Plumbs that flag through
`AppServerRuntimeOptions.remote_control_enabled` and makes app-server
startup use only that runtime option to decide whether to start remote
control.
- Removes the app-server config mutation hook that reloaded config and
toggled remote control at runtime.
- Updates managed daemon spawning to use `codex app-server
--remote-control --listen unix://` instead of `--enable remote_control`.

Config APIs can still list, read, write, and set `remote_control`; those
operations just no longer affect remote-control process enrollment.
2026-05-13 00:52:45 +00:00
Ruslan Nigmatullin e64a8979b0 device-key: clean up unused crate (#21487) 2026-05-07 09:01:44 -07:00
pakrym-oai a8488fec5e Revert state DB injection and agent graph store (#21481)
## Why

Reverts #20689 to restore the previous optional state DB plumbing. The
conflict resolution keeps the newer installation ID and session/thread
identity changes that landed after #20689, while removing the mandatory
state DB and agent graph store dependency from ThreadManager
construction.

## What changed

- Restored `Option<StateDbHandle>` through app-server, MCP server,
prompt debug, and test entry points.
- Removed the `codex-core` dependency on `codex-agent-graph-store` and
reverted descendant lookup back to the existing state DB path when
available.
- Kept newer `installation_id` forwarding by passing it beside the
optional DB handle.
- Kept local thread-name updates working when the optional state DB
handle is absent.

## Validation

- `git diff --check`
- `cargo test -p codex-thread-store`
- `cargo test -p codex-state -p codex-rollout -p
codex-app-server-protocol`
- Attempted `env CARGO_INCREMENTAL=0 cargo test -p codex-core -p
codex-app-server -p codex-app-server-client -p codex-mcp-server -p
codex-thread-manager-sample -p codex-tui`; blocked locally by a rustc
ICE while compiling `v8 v146.4.0` with `rustc 1.93.0 (254b59607
2026-01-19)` on `aarch64-apple-darwin`.
2026-05-06 22:48:29 -07:00
jif-oaiandCodex 8f3bb355f4 Move installation ID resolution out of core startup (#21182)
## Summary

- resolve or inject the installation ID before core startup and pass it
through `ThreadManager`, `CodexSpawnArgs`, and `Session` as a plain
`String`
- keep child sessions on the parent installation ID instead of
rediscovering it inside core
- propagate installation ID startup failures in `mcp-server` instead of
panicking

## Why

Core was still touching the filesystem on the session startup path to
discover `installation_id`. This moves that work to the outer host
boundary so core no longer depends on `codex_home` reads during session
construction.

---------

Co-authored-by: Codex <noreply@openai.com>
2026-05-06 10:48:54 +00:00
Rasmus Rygaard 7e310bc7f3 Inject state DB, agent graph store (#20689)
## Why

We want the agent graph store to be passed down the stack as a real
dependency, the same way we already treat the thread store.

This will let us inject the agent graph store as a real dependency and
support implementations other than the local SQLite-backed one. Right
now most code instantiates a state DB and an agent graph store
just-in-time. Ideally, we would not depend on the state DB directly but
only read through the higher-level interfaces.

This change makes the dependency boundaries explicit and moves state DB
initialization to process bootstrap instead of hiding it inside local
store implementations.

## What changed

- `ThreadManager` now requires a `StateDbHandle` and an
`AgentGraphStore` at construction time instead of treating them as
optional internals.
- The local store constructors no longer lazily initialize SQLite.
Callers now initialize the state DB once per process and use that shared
handle to build:
  - `LocalThreadStore`
  - `LocalAgentGraphStore`
- App bootstraps (`app-server`, `mcp-server`, `prompt_debug`, and the
thread-manager sample) now initialize the state DB up front and inject
the resulting handle down the stack.
- `app-server` now consistently uses its process-scoped state DB handle
instead of reopening SQLite or trying to recover it from loaded threads.
- Device-key storage now reuses the shared state DB handle instead of
maintaining its own lazy opener.
- The thread archive / descendant traversal paths now use the injected
`AgentGraphStore` instead of reaching through local
thread-store-specific state.

## Verification

- `cargo check -p codex-core -p codex-thread-store -p codex-app-server
-p codex-mcp-server -p codex-thread-manager-sample --tests`
- `cargo test -p codex-thread-store`
- `cargo test -p codex-core
thread_manager_accepts_separate_agent_graph_store_and_thread_store --
--nocapture`
- `cargo test -p codex-app-server
thread_archive_archives_spawned_descendants -- --nocapture`
2026-05-05 21:45:29 +00:00
rhan-oai 9e0c191c13 add turn items view to app-server turns (#21063)
## Why

`Turn.items` currently overloads an empty array to mean either that no
items exist or that the server intentionally did not load them for this
response. That ambiguity blocks future lazy-loading work where clients
need to distinguish unloaded, summary, and fully hydrated turn payloads.

## What changed

- add a new `TurnItemsView` enum with `notLoaded`, `summary`, and `full`
variants
- add required `itemsView` metadata to app-server `Turn` payloads
- mark reconstructed persisted history as `full` and live shell-style
turn payloads as `notLoaded`
- keep current `thread/turns/list` behavior unchanged and document that
it still returns `full` turns today
- regenerate the JSON and TypeScript protocol fixtures

## Verification

- `just write-app-server-schema`
- `cargo test -p codex-app-server-protocol`
- `cargo test -p codex-app-server thread_read_can_include_turns`
- `cargo test -p codex-app-server
thread_turns_list_can_page_backward_and_forward`
- `cargo test -p codex-app-server
thread_resume_rejects_history_when_thread_is_running`
- `just fix -p codex-app-server-protocol`
- `just fix -p codex-app-server`
- `just fmt`
2026-05-05 19:17:16 +00:00
pakrym-oai b6d4c4ea6b [codex] Use shared app-server JSON-RPC error helpers (#21221)
## Why

App-server had repeated hand-built JSON-RPC error objects for standard
error shapes. Using the shared helpers keeps the common
`invalid_request`, `invalid_params`, and `internal_error` construction
in one place and reduces the chance of new call sites drifting from the
common error payload shape.

## What changed

- Replaced manual standard JSON-RPC error object creation with
`internal_error(...)`, `invalid_request(...)`, and `invalid_params(...)`
across app-server request processors and runtime paths.
- Removed local duplicate helper definitions from search and review
request handling.
- Preserved existing structured `data` payloads by creating the shared
helper error first and then attaching the existing metadata.
- Left custom non-standard errors and raw error-code assertions intact.

## Validation

- `cargo test -p codex-app-server`
2026-05-05 12:13:59 -07:00
Ruslan Nigmatullin 4d201e340e state: pass state db handles through consumers (#20561)
## Why

SQLite state was still being opened from consumer paths, including lazy
`OnceCell`-backed thread-store call sites. That let one process
construct multiple state DB connections for the same Codex home, which
makes SQLite lock contention and `database is locked` failures much
easier to hit.

State DB lifetime should be chosen by main-like entrypoints and tests,
then passed through explicitly. Consumers should use the supplied
`Option<StateDbHandle>` or `StateDbHandle` and keep their existing
filesystem fallback or error behavior when no handle is available.

The startup path also needs to keep the rollout crate in charge of
SQLite state initialization. Opening `codex_state::StateRuntime`
directly bypasses rollout metadata backfill, so entrypoints should
initialize through `codex_rollout::state_db` and receive a handle only
after required rollout backfills have completed.

## What Changed

- Initialize the state DB in main-like entrypoints for CLI, TUI,
app-server, exec, MCP server, and the thread-manager sample.
- Pass `Option<StateDbHandle>` through `ThreadManager`,
`LocalThreadStore`, app-server processors, TUI app wiring, rollout
listing/recording, personality migration, shell snapshot cleanup,
session-name lookup, and memory/device-key consumers.
- Remove the lazy local state DB wrapper from the thread store so
non-test consumers use only the supplied handle or their existing
fallback path.
- Make `codex_rollout::state_db::init` the local state startup path: it
opens/migrates SQLite, runs rollout metadata backfill when needed, waits
for concurrent backfill workers up to a bounded timeout, verifies
completion, and then returns the initialized handle.
- Keep optional/non-owning SQLite helpers, such as remote TUI local
reads, as open-only paths that do not run startup backfill.
- Switch app-server startup from direct
`codex_state::StateRuntime::init` to the rollout state initializer so
app-server cannot skip rollout backfill.
- Collapse split rollout lookup/list APIs so callers use the normal
methods with an optional state handle instead of `_with_state_db`
variants.
- Restore `getConversationSummary(ThreadId)` to delegate through
`ThreadStore::read_thread` instead of a LocalThreadStore-specific
rollout path special case.
- Keep DB-backed rollout path lookup keyed on the DB row and file
existence, without imposing the filesystem filename convention on
existing DB rows.
- Verify readable DB-backed rollout paths against `session_meta.id`
before returning them, so a stale SQLite row that points at another
thread's JSONL falls back to filesystem search and read-repairs the DB
row.
- Keep `debug prompt-input` filesystem-only so a one-off debug command
does not initialize or backfill SQLite state just to print prompt input.
- Keep goal-session test Codex homes alive only in the goal-specific
helper, rather than leaking tempdirs from the shared session test
helper.
- Update tests and call sites to pass explicit state handles where DB
behavior is expected and explicit `None` where filesystem-only behavior
is intended.

## Validation

- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo check -p
codex-rollout -p codex-thread-store -p codex-app-server -p codex-core -p
codex-tui -p codex-exec -p codex-cli --tests`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p
codex-rollout state_db_`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p
codex-rollout find_thread_path`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p
codex-rollout find_thread_path -- --nocapture`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p
codex-rollout try_init_ -- --nocapture`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p
codex-rollout`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo clippy -p
codex-rollout --lib -- -D warnings`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p
codex-thread-store
read_thread_falls_back_when_sqlite_path_points_to_another_thread --
--nocapture`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p
codex-thread-store`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p codex-core
shell_snapshot`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p codex-core
--test all personality_migration`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p codex-core
--test all rollout_list_find`
- `RUST_MIN_STACK=8388608 CODEX_SKIP_VENDORED_BWRAP=1
CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p codex-core
--test all rollout_list_find::find_prefers_sqlite_path_by_id --
--nocapture`
- `RUST_MIN_STACK=8388608 CODEX_SKIP_VENDORED_BWRAP=1
CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p codex-core
--test all rollout_list_find -- --nocapture`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p codex-core
interrupt_accounts_active_goal_before_pausing`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p
codex-app-server get_auth_status -- --test-threads=1`
- `CODEX_SKIP_VENDORED_BWRAP=1
CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p
codex-app-server --lib`
- `CODEX_SKIP_VENDORED_BWRAP=1
CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo check -p codex-rollout
-p codex-app-server --tests`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db just fix -p codex-rollout
-p codex-thread-store -p codex-core -p codex-app-server -p codex-tui -p
codex-exec -p codex-cli`
- `CODEX_SKIP_VENDORED_BWRAP=1
CARGO_TARGET_DIR=/tmp/codex-target-state-db just fix -p codex-rollout -p
codex-app-server`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db just fix -p
codex-rollout`
- `CODEX_SKIP_VENDORED_BWRAP=1
CARGO_TARGET_DIR=/tmp/codex-target-state-db just fix -p codex-core`
- `just argument-comment-lint -p codex-core`
- `just argument-comment-lint -p codex-rollout`

Focused coverage added in `codex-rollout`:

- `recorder::tests::state_db_init_backfills_before_returning` verifies
the rollout metadata row exists before startup init returns.
- `state_db::tests::try_init_waits_for_concurrent_startup_backfill`
verifies startup waits for another worker to finish backfill instead of
disabling the handle for the process.
-
`state_db::tests::try_init_times_out_waiting_for_stuck_startup_backfill`
verifies startup does not hang indefinitely on a stuck backfill lease.
-
`tests::find_thread_path_accepts_existing_state_db_path_without_canonical_filename`
verifies DB-backed lookup accepts valid existing rollout paths even when
the filename does not include the thread UUID.
-
`tests::find_thread_path_falls_back_when_db_path_points_to_another_thread`
verifies DB-backed lookup ignores a stale row whose existing path
belongs to another thread and read-repairs the row after filesystem
fallback.

Focused coverage updated in `codex-core`:

- `rollout_list_find::find_prefers_sqlite_path_by_id` now uses a
DB-preferred rollout file with matching `session_meta.id`, so it still
verifies that valid SQLite paths win without depending on stale/empty
rollout contents.

`cargo test -p codex-app-server thread_list_respects_search_term_filter
-- --test-threads=1 --nocapture` was attempted locally but timed out
waiting for the app-server test harness `initialize` response before
reaching the changed thread-list code path.

`bazel test //codex-rs/thread-store:thread-store-unit-tests
--test_output=errors` was attempted locally after the thread-store fix,
but this container failed before target analysis while fetching `v8+`
through BuildBuddy/direct GitHub. The equivalent local crate coverage,
including `cargo test -p codex-thread-store`, passes.

A plain local `cargo check -p codex-rollout -p codex-app-server --tests`
also requires system `libcap.pc` for `codex-linux-sandbox`; the
follow-up app-server check above used `CODEX_SKIP_VENDORED_BWRAP=1` in
this container.
2026-05-04 11:46:03 -07:00
pakrym-oai 33b19bcfde [codex] Split app-server request processors (#20940)
## Why

The app-server request path had grown around a large
`CodexMessageProcessor` plus separate API wrapper/helper modules. That
made the dependency graph hard to see and forced unrelated request
families to share broad processor state.

This PR makes the split mechanical and command-prefix oriented so
request families own only the dependencies they use.

## What changed

- Replaced `CodexMessageProcessor` with command-prefix request
processors under `app-server/src/request_processors/`.
- Removed the old config, device-key, external-agent-config, and fs API
wrapper files by moving their API handling into processors.
- Split apps, plugins, marketplace, catalog, account, MCP, command exec,
fs, git, feedback, thread, turn, thread goals, and Windows sandbox
handling into dedicated processors.
- Kept shared lifecycle, summary conversion, token usage replay, and
shared error mapping only where multiple processors use them; single-use
helpers were inlined into their owning processor.
- Removed the fallback processor path and moved processor tests to
`_tests` files.

## Validation

- `cargo test -p codex-app-server`
- `cargo check -p codex-app-server`
- `just fix -p codex-app-server`
2026-05-04 09:34:11 -07:00
rhan-oai 0690ab0842 [codex-analytics] ingest server requests and responses (#17088)
## Why

Codex analytics needs a typed seam for app-server-originated
request/response traffic so future tool-approval analytics can consume
those facts without adding bespoke callsite tracking each time. Server
responses arrive as JSON-RPC `id + result` payloads, so analytics has to
reconstruct the matching typed response from the original typed request
while that request context still exists in app-server.

This also puts analytics on the app-server outbound path, which needs to
avoid keeping the runtime alive during shutdown. The final ownership fix
keeps the normal strong auth-manager retention in analytics and makes
the external-auth refresh bridge hold a weak back-reference to
`OutgoingMessageSender`, breaking the runtime cycle at the bridge
boundary instead of exposing retention policy through the analytics
client API.

## What changed

- Adds typed `ServerRequest` and `ServerResponse` analytics facts, plus
`AnalyticsEventsClient::track_server_request` and
`track_server_response`.
- Renames the existing client-side facts to `ClientRequest` and
`ClientResponse` so reducers can distinguish client-to-server traffic
from server-to-client traffic.
- Adds `ServerRequest::response_from_result`, allowing a stored typed
request to decode the matching typed server response from a raw JSON-RPC
result payload.
- Threads `AnalyticsEventsClient` through `OutgoingMessageSender` and
records targeted server requests, replayed targeted requests, and
matching targeted responses with the responding connection id needed for
correlation.
- Intentionally leaves broadcast server requests/responses out of
analytics for now because the current model is per connection, while
broadcasts fan one logical request out across multiple connections.
- Breaks the app-server shutdown cycle by storing
`Weak<OutgoingMessageSender>` in `ExternalAuthRefreshBridge` and
upgrading it only when an external-auth refresh is actually requested.
- Keeps reducer ingestion of the new server-side facts as no-ops for
now; this PR is plumbing for later tool-approval analytics work.

## Verification

- `cargo test -p codex-analytics`
- `cargo test -p codex-app-server outgoing_message::tests::`
- Covers typed-response reconstruction plus the targeted, replayed,
broadcast-exclusion, and response-attribution analytics paths.

## Follow-up

This PR intentionally stops at ingestion plumbing, so `ServerRequest`
and `ServerResponse` facts are still reducer no-ops. Once a follow-up PR
adds real downstream analytics output for those facts:

- replace the temporary pre-reducer observation seam with reducer tests
for the emitted event shape;
- add end-to-end coverage in `app-server/tests/suite/v2/analytics.rs`
for the real app-server workflow and captured analytics payload;
- remove the temporary sender-level observer tests added here in favor
of the real-output coverage above.

---

[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/17088).
* #18748
* #18747
* #17090
* #17089
* #20241
* #20239
* __->__ #17088
2026-04-29 19:56:41 +00:00
Ruslan Nigmatullin 0700f979ba app-server: run initialized rpcs with keyed serialization (#17373)
## Why

Initialized app-server RPCs no longer need to bottleneck behind one
request processor path. Running them concurrently improves
responsiveness, but several request families still mutate shared state
or depend on ordered side effects. Those stateful families need an
auditable serialization contract so concurrency does not reorder thread,
config, auth, command, watcher, MCP, or similar state transitions.

This PR keeps that boundary explicit: stateful work is serialized by the
smallest useful key, while intentionally read-only or externally
concurrent work remains unkeyed. In particular, `thread/list` and
`thread/turns/list` explicitly have no serialization because they
primarily read append-only rollout storage and should continue to be
served concurrently.

## What changed

- Adds `ClientRequest::serialization_scope()` in `app-server-protocol`
and requires every client request definition to declare its
serialization behavior.
- Introduces keyed request scopes for thread, thread path, command exec
process, fuzzy search session, fs watch, MCP OAuth, and global state
buckets such as config, account auth, memory, and device keys.
- Routes initialized app-server RPCs through per-key FIFO serialization
while allowing unkeyed initialized requests to run concurrently.
- Cancels in-flight initialized RPC work when the connection disconnects
or the app-server exits so spawned request tasks do not outlive their
session.
- Adds focused coverage for representative keyed and unkeyed
serialization scopes, including explicitly concurrent
`thread/turns/list` behavior.

## Validation

- Added protocol tests for representative keyed serialization scopes and
intentionally unkeyed request families.
- Added app-server request serialization tests covering per-key FIFO
behavior, concurrent unkeyed execution, disconnect shutdown, and config
read-after-write ordering.
- Local focused protocol validation after the latest rebase is currently
blocked by packageproxy failing to resolve locked `rustls-webpki
0.103.13`; CI is expected to provide the full validation signal.
2026-04-28 12:23:34 -07:00
efrazer-oai 2009f6e894 refactor: make auth loading async (#19762)
## Summary

Auth loading used to expose synchronous construction helpers in several
places even though some auth sources now need async work. This PR makes
the auth-loading surface async and updates the callers to await it.

This is intentionally only plumbing. It does not change how
AgentIdentity tokens are decoded, how task runtime ids are allocated, or
how JWT signatures are verified.

## Stack

1. **This PR:** [refactor: make auth loading
async](https://github.com/openai/codex/pull/19762)
2. [refactor: load AgentIdentity runtime
eagerly](https://github.com/openai/codex/pull/19763)
3. [feat: verify AgentIdentity JWTs with
JWKS](https://github.com/openai/codex/pull/19764)

## Important call sites

| Area | Change |
| --- | --- |
| `codex-login` auth loading | `CodexAuth` and `AuthManager`
construction paths now await auth loading. |
| app-server startup | Auth manager construction is awaited during
initialization. |
| CLI/TUI/exec/MCP/chatgpt callers | Existing auth-loading calls now
await the same behavior. |
| cloud requirements storage loader | The loader becomes async so it can
share the same auth construction path. |
| auth tests | Tests that load auth now run in async contexts. |

## Testing

Tests: targeted Rust auth test compilation, formatter, scoped Clippy
fix, and Bazel lock check.
2026-04-27 11:00:27 -07:00
pakrym-oai 9c3abcd46c [codex] Move config loading into codex-config (#19487)
## Why

Config loading had become split across crates: `codex-config` owned the
config types and merge logic, while `codex-core` still owned the loader
that assembled the layer stack. This change consolidates that
responsibility in `codex-config`, so the crate that defines config
behavior also owns how configs are discovered and loaded.

To make that move possible without reintroducing the old dependency
cycle, the shell-environment policy types and helpers that
`codex-exec-server` needs now live in `codex-protocol` instead of
flowing through `codex-config`.

This also makes the migrated loader tests more deterministic on machines
that already have managed or system Codex config installed by letting
tests override the system config and requirements paths instead of
reading the host's `/etc/codex`.

## What Changed

- moved the config loader implementation from `codex-core` into
`codex-config::loader` and deleted the old `core::config_loader` module
instead of leaving a compatibility shim
- moved shell-environment policy types and helpers into
`codex-protocol`, then updated `codex-exec-server` and other downstream
crates to import them from their new home
- updated downstream callers to use loader/config APIs from
`codex-config`
- added test-only loader overrides for system config and requirements
paths so loader-focused tests do not depend on host-managed config state
- cleaned up now-unused dependency entries and platform-specific cfgs
that were surfaced by post-push CI

## Testing

- `cargo test -p codex-config`
- `cargo test -p codex-core config_loader_tests::`
- `cargo test -p codex-protocol -p codex-exec-server -p
codex-cloud-requirements -p codex-rmcp-client --lib`
- `cargo test --lib -p codex-app-server-client -p codex-exec`
- `cargo test --no-run --lib -p codex-app-server`
- `cargo test -p codex-linux-sandbox --lib`
- `cargo shear`
- `just bazel-lock-check`

## Notes

- I did not chase unrelated full-suite failures outside the migrated
loader surface.
- `cargo test -p codex-core --lib` still hits unrelated proxy-sensitive
failures on this machine, and Windows CI still shows unrelated
long-running/timeouting test noise outside the loader migration itself.
2026-04-26 15:10:53 -07:00
Michael Bolin ac2bffa443 test: harden app-server integration tests (#19683)
## Why

Windows Bazel runs in the permissions stack exposed that app-server
integration tests were launching normal plugin startup warmups in every
subprocess. Those warmups can call
`https://chatgpt.com/backend-api/plugins/featured` when a test is not
specifically exercising plugin startup, which adds slow background work,
noisy stderr, and dependence on external network state. The relevant
startup/featured-plugin behavior was introduced across #15042 and
#15264.

A few app-server tests also had long optional waits or unbounded cleanup
paths, making failures expensive to diagnose and contributing to slow
Windows shards. One external-agent config test from #18246 used a
GitHub-style marketplace source, which was enough to exercise the
pending remote-import path but also meant the background completion task
could attempt a real clone.

## What Changed

- Adds explicit `AppServerRuntimeOptions` / `PluginStartupTasks`
plumbing and a hidden debug-only
`--disable-plugin-startup-tasks-for-tests` app-server flag, so
integration tests can suppress startup plugin warmups without adding a
production env-var gate.
- Has the app-server test harness pass that hidden flag by default,
while opting plugin-startup coverage back in for tests that
intentionally exercise startup sync and featured-plugin warmup behavior.
- Lowers normal app-server subprocess logging from `info`/`debug` to
`warn` to avoid multi-megabyte stderr output in Bazel logs.
- Prevents the external-agent config test from attempting a real
marketplace clone by using an invalid non-local source while still
exercising the pending-import completion path.
- Bounds optional filesystem/realtime waits and fake WebSocket
test-server shutdown so failures produce targeted timeouts instead of
hanging a shard.
- Fixes the Unix script-resolution test in `rmcp-client` to exercise
PATH resolution directly and include the actual spawn error in failures.

## Verification

- `cargo check -p codex-app-server`
- `cargo clippy -p codex-app-server --tests -- -D warnings`
- `cargo test -p codex-rmcp-client
program_resolver::tests::test_unix_executes_script_without_extension`
- `cargo test -p codex-app-server --test all
external_agent_config_import_sends_completion_notification_after_pending_plugins_finish
-- --nocapture`
- `cargo test -p codex-app-server --test all
plugin_list_uses_warmed_featured_plugin_ids_cache_on_first_request --
--nocapture`
- Windows Local Bazel passed with this test-hardening bundle before it
was extracted from #19606.

---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/19683).
* #19395
* #19394
* #19393
* #19392
* #19606
* __->__ #19683
2026-04-26 12:43:16 -07:00
starr-openaiandCodex ddbe2536be Support multiple managed environments (#18401)
## Summary
- refactor EnvironmentManager to own keyed environments with
default/local lookup helpers
- keep remote exec-server client creation lazy until exec/fs use
- preserve disabled agent environment access separately from internal
local environment access

## Validation
- not run (per Codex worktree instruction to avoid tests/builds unless
requested)

---------

Co-authored-by: Codex <noreply@openai.com>
2026-04-21 15:29:35 -07:00
Ruslan Nigmatullin 69c3d12274 app-server: implement device key v2 methods (#18430)
## Why

The device-key protocol needs an app-server implementation that keeps
local key operations behind the same request-processing boundary as
other v2 APIs.

app-server owns request dispatch, transport policy, documentation, and
JSON-RPC error shaping. `codex-device-key` owns key binding, validation,
platform provider selection, and signing mechanics. Keeping the adapter
thin makes the boundary easier to review and avoids moving local
key-management details into thread orchestration code.

## What changed

- Added `DeviceKeyApi` as the app-server adapter around
`DeviceKeyStore`.
- Converted protocol protection policies, payload variants, algorithms,
and protection classes to and from the device-key crate types.
- Encoded SPKI public keys and DER signatures as base64 protocol fields.
- Routed `device/key/create`, `device/key/public`, and `device/key/sign`
through `MessageProcessor`.
- Rejected remote transports before provider access while allowing local
`stdio` and in-process callers to reach the device-key API.
- Added stdio, in-process, and websocket tests for device-key validation
and transport policy.
- Documented the device-key methods in the app-server v2 method list.

## Test coverage

- `device_key_create_rejects_empty_account_user_id`
- `in_process_allows_device_key_requests_to_reach_device_key_api`
- `device_key_methods_are_rejected_over_websocket`

## Stack

This is PR 3 of 4 in the device-key app-server stack. It is stacked on
#18429.

## Validation

- `cargo test -p codex-app-server device_key`
- `just fix -p codex-app-server`
2026-04-21 14:07:08 -07:00
pakrym-oai ffa6944587 Load app-server config through ConfigManager (#18870)
## Summary
- Load app-server startup config through `ConfigManager` instead of
direct `ConfigBuilder` calls.
- Move `ConfigManager` constructor-owned state (`cli_overrides`, runtime
feature map, cloud requirements loader) behind internal manager fields.
- Pass `ConfigManager` into `MessageProcessor` directly instead of
reconstructing it from raw args.

## Tests
- `cargo check -p codex-app-server`
- `cargo test -p codex-app-server`
- `just fix -p codex-app-server`
- `just fmt`
2026-04-21 14:01:02 -07:00
Rasmus Rygaard 7b994100b3 Add session config loader interface (#18208)
## Why

Cloud-hosted sessions need a way for the service that starts or manages
a thread to provide session-owned config without treating all config as
if it came from the same user/project/workspace TOML stack.

The important boundary is ownership: some values should be controlled by
the session/orchestrator, some by the authenticated user, and later some
may come from the executor. The earlier broad config-store shape made
that boundary too fuzzy and overlapped heavily with the existing
filesystem-backed config loader. This PR starts with the smaller piece
we need now: a typed session config loader that can feed the existing
config layer stack while preserving the normal precedence and merge
behavior.

## What Changed

- Added `ThreadConfigLoader` and related typed payloads in
`codex-config`.
- `SessionThreadConfig` currently supports `model_provider`,
`model_providers`, and feature flags.
- `UserThreadConfig` is present as an ownership boundary, but does not
yet add TOML-backed fields.
- `NoopThreadConfigLoader` preserves existing behavior when no external
loader is configured.
  - `StaticThreadConfigLoader` supports tests and simple callers.

- Taught thread config sources to produce ordinary `ConfigLayerEntry`
values so the existing `ConfigLayerStack` remains the place where
precedence and merging happen.

- Wired the loader through `ConfigBuilder`, the config loader, and
app-server startup paths so app-server can provide session-owned config
before deriving a thread config.

- Added coverage for:
  - translating typed thread config into config layers,
- inserting thread config layers into the stack at the right precedence,
- applying session-provided model provider and feature settings when
app-server derives config from thread params.

## Follow-Ups

This intentionally stops short of adding the remote/service transport.
The next pieces are expected to be:

1. Define the proto/API shape for this interface.
2. Add a client implementation that can source session config from the
service side.

## Verification

- Added unit coverage in `codex-config` for the loader and layer
conversion.
- Added `codex-core` config loader coverage for thread config layer
precedence.
- Added app-server coverage that verifies session thread config wins
over request-provided config for model provider and feature settings.
2026-04-20 23:05:49 +00:00
pakrym-oai bd61737e8a Async config loading (#18022)
Parts of config will come from executor. Prepare for that by making
config loading methods async.
2026-04-15 19:18:38 -07:00