Commit Graph
34 Commits
Author SHA1 Message Date
Adam Perry @ OpenAI 7e8143a897 Tie executor attachments to environment selection lifetimes (#59043)
## Why

Removing an executor must detach its selection even when older turn snapshots still reference it. Connection events queued for a removed attachment must not leak into a later selection of the same executor.

## What changed

- Introduce `EnvironmentAttachment` with a detach signal shared by turn snapshots and a guard owned by the selection.
- Preserve attachments across workspace and configuration changes, while giving reselections and child threads independent attachment lifetimes.
- Stop forwarding connection events on detach and discard buffered events from detached attachments.
- Activate initial attachments after `SessionConfigured`, using configured bindings for startup discovery.

## Testing

Add coverage for discarding buffered connection events on reselection and extend snapshot tests to verify immediate detachment, fresh attachments on reselection, and independent parent and child lifetimes.

GitOrigin-RevId: 6cf882e0d17a296030c13d80c6e2866c80e04638
2026-11-03 02:37:20 +00:00
andrewgu-oai 7a5860da35 Add hook configuration to environment attachments (#56302)
## What changed

- Add `environment_hooks` to `EnvironmentConfig` for hooks from an environment's effective managed requirements. Initialize it empty for inferred thread configurations and report the handler count in debug output.
- Move `HookEventsToml`, `MatcherGroup`, and `HookHandlerConfig` into `codex_protocol::hooks`, with re-exports from `codex_config::hook_config`.
- Validate MCP hook input recursively to reject nulls and integers outside the signed 64-bit range without converting through `toml::Value`.

GitOrigin-RevId: 410506c8e2b89c9519c81d86b44ccd760d751693
2026-10-23 00:02:29 +00:00
Winston Howes bd274564e1 Add opt-in credential masking for proxied sandboxed sessions (#52302)
## What changed

- Add the disabled-by-default `features.credential_masking` flag to enable credential brokerage through an already enabled network proxy, retaining custom credential providers and managed configuration precedence.
- Prevent project configuration from overriding credential masking and apply existing credential-broker protections to project shell settings when masking is enabled.
- Keep brokerage inactive when the proxy is disabled. Warn at startup and after permission-profile changes when masking is inactive because sandboxing or the proxy is disabled, or provider environment keys conflict.

## Testing

Add coverage for configuration precedence, project protections in both configuration loaders, permission-profile transitions, inactive-masking warnings, and shell snapshot masking of built-in and custom credentials.

GitOrigin-RevId: 4bcb293356e8c1524bdaa4f1b95cbc1e935a5271
2026-10-08 22:27:00 +00:00
Leonardo Di Donato 9b738582b1 Expose originating tool call metadata in command environments (#52119)
## Why

Command processes need thread and tool call labels for correlation, without stale values from inherited environments or cached shell snapshots.

## What changed

- Export the originating `exec_command` call ID as `CODEX_TOOL_CALL_ID`; later `write_stdin` calls leave it unchanged.
- Apply executor metadata as `CODEX_THREAD_ID` and `CODEX_TOOL_CALL_ID` after environment policies, overlays, and snapshot replay. Clear omitted fields when metadata is present; without metadata, preserve thread context and remove the call ID.
- Exclude both IDs from newly captured snapshots and omit the call ID from capture and validation helpers.
- Preserve empty call IDs, omit NUL-containing IDs, and handle case-insensitive environment names on Windows. Document that these labels support correlation, not authorization.

## Testing

Add regression coverage for metadata precedence, missing and invalid IDs, snapshot capture and replay, local and remote execution with pipes and TTYs, sandboxed remote execution, and preservation of the originating call ID across later input.

GitOrigin-RevId: 1027212b5d495d8e7cfac0f498c3134e060aa897
2026-10-08 13:45:19 +00:00
iceweasel-oai 622e9e3696 Preserve sandbox overrides across execution requests (#51786)
## What changed

Move `SandboxOverride` into the shared protocol and carry the controller's selection through tool attempts, execution requests, and exec-server process and filesystem sandbox preparation. Include it as `sandboxOverride` in filesystem sandbox contexts for observation without granting broader permissions or altering sandbox commands.

Default missing values to `NoOverride` and omit that value when serializing to preserve compatibility with existing requests.

## Testing

Extend tests to cover override propagation, wire round-trips for all three variants, and deserialization of requests without `sandboxOverride`.

GitOrigin-RevId: 2f438cf4aef1134e3149959d14aa731682168ae5
2026-10-07 17:59:22 +00:00
Adam Perry @ OpenAI 9479e1fdb7 Bind capability roots to environment selections (#51493)
## Why

Capability discovery needs the roots captured with each environment snapshot. Threads sharing an executor can select different roots, and changing environment order must preserve executor skill aliases.

## What changed

- Attach the receiving thread's capability roots when constructing `TurnEnvironmentSelection` at startup and settings updates, including roots loaded from saved history.
- Resolve capability roots from environment snapshots while preserving their original order. Retain startup roots so deselecting and reselecting an environment restores its capabilities.
- Match inherited configuration by environment and workspace so differences in capability root metadata do not interrupt configuration propagation.

## Testing

Add regression coverage for root ordering across environment states and configuration changes, independent root selections on a shared executor, reselection with startup or saved roots, active turn updates, and pending configuration failures reaching child and grandchild threads.

GitOrigin-RevId: 4f2e4bb414d27195a6752406117a6050718a1198
2026-10-06 22:39:32 +00:00
jif b63a296775 Honor shell environment policy in legacy snapshots (#48099)
## Why

Legacy shell snapshots could retain exports excluded by `shell_environment_policy` and restore captured values over explicit overrides when replayed.

## What changed

- Build snapshots with the resolved environment's policy and omit filtered exports and original values for variables overridden by `set`.
- Skip eager snapshot creation when configuration is unavailable, and reject snapshots whose policy differs from the current turn's policy.

## Testing

Add a regression test covering default and custom exclusions, `include_only`, preservation of multiline values, and `set` overrides that survive snapshot replay. Verify excluded and overridden captured values are absent from the snapshot file.

GitOrigin-RevId: 4b785130f6c5a4d268ab86fd36ab9d77d8c4a3b9
2026-09-25 14:15:05 +00:00
Charlie Marsh 60f0ab6cbf Use the shared process launcher for Unix shell snapshots (#47611)
## What changed

Launch Unix shell snapshot commands through `codex_utils_pty::Command` with `ProcessMode::NewSession`, replacing the custom `pre_exec` hook. Pass the complete, scrubbed child environment and preserve the original command arguments so NUL bytes are rejected before execution.

## Testing

Add a Unix regression test asserting that a snapshot script containing a NUL byte returns `InvalidInput` without creating the file it would otherwise write.

GitOrigin-RevId: ef68a6c9f64d7a541498d3ecc4113fad8ff7ed14
2026-09-23 18:24:42 +00:00
sayan-oai 8a3c4ea3b5 Apply inherited environment settings at turn boundaries (#47428)
## Why

Thread permission updates could affect later steps of a running turn. Keep inherited environment settings consistent throughout the turn and apply saved updates when the next turn starts.

## What changed

Store active thread defaults alongside selected environments and refresh them when activating a new turn, even when its environment selections are unchanged. Resolve new selections from those active defaults while preserving owner-provided configuration, selection workspace roots, and the local Windows sandbox type.

## Testing

Add a regression test that changes permissions during a paused turn and verifies that both steps retain restricted filesystem access while the next turn receives unrestricted access. Update session and MCP elicitation tests to check that permission changes take effect on the next turn.

GitOrigin-RevId: 37ee44198fa40c138d140974dc90c4177d8c1f4c
2026-09-23 02:18:40 +00:00
iceweasel-oai 418199f6ad Add an opt-in preference for the local MXC sandbox (#47375)
## What changed

Add the default-off `features.prefer_mxc` flag to select MXC for local Windows execution when native support is available and effective network settings do not explicitly forbid local binding. Otherwise, retain the configured backend and legacy setup behavior.

Resolve the preference during config loading and use the effective local backend for execution, network proxy setup, sandbox diagnostics, and `windowsSandbox/readiness`. Preserve the configured backend for remote executors and `config/read`. Explicit `windows.sandbox = "mxc"` remains strict, and command failures do not trigger backend fallback.

## Testing

Add coverage for preference resolution, startup readiness without rewriting configuration, local and remote backend selection across environment updates and child threads, and workspace-write permission context in an agent turn.

GitOrigin-RevId: b51178f0371d38ebb58e41af54a068fca67fa316
2026-09-22 21:57:52 +00:00
Charlie Marsh daed4b9755 Split shell snapshot credential tests into focused cases (#46715)
## What changed

Extract shared proxy setup into `credential_snapshot_proxy()` and split POSIX startup protection and inherited credential alias checks into separate tests, each with its own temporary directory and proxy. Preserve the existing assertions.

GitOrigin-RevId: 1799526c7c649c9bacd30a8cc4fa7038e70f192d
2026-09-19 20:41:55 +00:00
iceweasel-oai a633ebc124 Always use private desktops for legacy Windows sandboxes (#46554)
## What changed

- Remove the private-desktop opt-out from elevated and unelevated Windows sandbox launches.
- Remove `windows.sandbox_private_desktop` and its managed requirement and API fields. Warn users to remove the obsolete setting.
- Require a private desktop name when launching through the Windows sandbox wrapper and command runner.

## Testing

Add coverage for the obsolete-setting migration warning and update wrapper tests to verify a live private desktop is passed and a missing desktop name is rejected.

GitOrigin-RevId: c7135f8d211aac8d812180691c2c1e433d77cde4
2026-09-19 01:21:32 +00:00
iceweasel-oai 74af2496c9 Default local binding to true for MXC managed networking (#46523)
## Why

MXC's native host-loopback access is bidirectional, so it cannot enforce `allow_local_binding = false`. Treating an omitted setting as `false` prevents managed networking from working with the default configuration.

## What changed

- Preserve an omitted `allow_local_binding` until the executor's sandbox policy is known. Default to `true` for Windows MXC and `false` elsewhere, including remote execution.
- Reject an effective `false` for MXC managed networking after applying policy restrictions, without enabling disabled networking.
- Use the executor's resolved value for remote network approval decisions while preserving explicit controller restrictions.
- Document that local binding permits local servers and direct host-loopback connections and skips additional private-network destination checks; proxy domain rules still apply.

## Testing

Add coverage for per-executor defaults, explicit values, MXC rejection of `false`, and remote network review cleanup with the resolved policy.

GitOrigin-RevId: 0fa7c1eaec68bebada2b8f7af45688315eea70a9
2026-09-19 00:00:01 +00:00
Sean Huang 3724dc8361 Share platform identity across path, network, and sandbox configuration (#46334)
## What changed

- Add `Platform` to `codex-utils-path-uri` with metadata parsing, native platform detection, and path convention mapping. Preserve missing or unrecognized metadata as `Unknown`.
- Replace `NetworkProxyExecutorOs` with the shared type and keep executor-specific socket path validation in the network proxy.
- Extract `effective_sandbox_mode` with explicit platform and Windows sandbox level inputs, preserving the native Windows fallback from `workspace-write` to `read-only` when the sandbox is disabled.

## Testing

Add unit tests for platform metadata, path conventions, native platform detection, and sandbox mode selection across platforms and Windows sandbox levels.

GitOrigin-RevId: 4fe0972e3a91040e35f2a6dfa5bcdf6c9a29be88
2026-09-18 00:51:34 +00:00
Sean Huang ea218f5cd8 Validate network socket policies using the executor OS (#46302)
## Why

A controller and its executor can run different operating systems. Validating socket paths against the controller's OS can reject absolute paths that are valid on the executor, such as Windows paths on a Linux controller.

## What changed

- Thread `NetworkProxyExecutorOs` through network policy validation, proxy construction, and policy updates.
- Require allowed socket paths to be NUL-free and absolute for the executor OS, while preserving deny entries unchanged.
- Accept either Unix or Windows absolute syntax when executor metadata omits the OS, then validate against the executor's own OS at launch.
- Keep native path normalization and socket support checks at execution time.

## Testing

Add coverage for cross-platform absolute path syntax, invalid allow entries, preserved deny entries, and remote policy round trips that retain executor semantics through domain edits and proxy construction.

GitOrigin-RevId: 1ebc09cbce7138f60ec5fd62875591a3df52d067
2026-09-17 20:59:55 +00:00
iceweasel-oai 8b78600dc8 Enable MXC selection through Windows sandbox configuration (#46271)
## What changed

- Accept `windows.sandbox = "mxc"` and preserve the selected backend through environment configuration, command execution, patch writes, and sandbox metadata.
- Treat MXC as enabled in the TUI and report Windows sandbox readiness as `ready`, avoiding legacy setup prompts.
- Keep `allowed_sandbox_implementations` scoped to the legacy elevated and unelevated backends without restricting MXC.
- Default `windows.sandbox_private_desktop` to `false` for MXC while retaining `true` for legacy sandboxes.

## Testing

Add coverage for MXC configuration precedence, legacy requirement handling, sandbox selection, and TUI state. Add a Wine integration test that verifies command and patch routing fails when native MXC is unavailable and reports `windows_mxc` in turn metadata.

GitOrigin-RevId: e2162447d0750f60753864c92a20e02a7f297bca
2026-09-17 18:20:28 +00:00
iceweasel-oai a5c15ab5c0 Wire Windows sandbox selection into managed proxy routing (#45757)
## What changed

- Track the Windows sandbox implementation separately from its legacy setup level.
- Select dedicated proxy listeners for `WindowsMxc` on Windows across sessions, app-server command execution, and the sandbox CLI.
- Rebuild the session proxy when routing changes, retaining its network policy decider.
- Add a sandbox CLI execution path through the MXC sandbox transform when `WindowsMxc` is selected.

## Testing

Extend configuration assertions for the selected sandbox type and the session refresh test to verify dedicated routing and policy decider retention on Windows.

GitOrigin-RevId: 2a22192636c542022ec81a378a16c1bb6867e145
2026-09-15 18:41:13 +00:00
iceweasel-oai c379459bba Wire the Windows MXC sandbox into command execution (#45176)
## What changed

- Add explicit MXC backend selection and carry its identity through exec-server process reporting and sandbox violation classification.
- Launch MXC through the Codex executable with the effective permission profile and command environment.
- Reject exec-server MXC requests when native MXC is unavailable or when they request a TTY, an `arg0` override, or managed networking. Reject private desktop isolation during MXC preparation.
- Allow an explicitly empty child environment and avoid exposing request payload values in launcher decode errors.

## Testing

Add coverage for sandbox selection and unsupported-request rejection, plus Windows RPC tests for stdin writes and temporary-directory permissions derived from the command environment. Native MXC tests skip when MXC is unavailable.

GitOrigin-RevId: 3626ff0f9ad7f9b812ce09b68c31ea9a5a9c72b1
2026-09-13 06:07:03 +00:00
Winston Howes 38cbebaf3f Support configured credential providers across shell snapshots (#44072)
## Why

Credential brokerage must retain trusted destination hints even when shell environment policy hides them from child processes. Snapshot replay also needs to preserve credential aliases when tokens are short or captured dummy values change.

## What changed

- Add `features.network_proxy.credentials` and preserve provider settings when toggling the proxy. Prevent project configuration from overriding providers or their credential and destination environment variables.
- Use provider metadata throughout snapshot capture and replay, and pass hidden destination context to the broker without exposing it in the child environment.
- Rewrite aliases containing short credentials or previously captured dummies, respecting environment filters and explicit overrides.
- Preserve case-distinct `shell_environment_policy.set` keys so Windows credential ambiguity checks survive configuration merging and permission profile changes.
- Restore independent MITM settings when credential brokerage is disabled.

## Testing

Add regression coverage for custom providers, hidden destination hints, Bash and Zsh alias replay, configuration edits and merging, Windows ambiguity handling, and credential protection during approved execution.

GitOrigin-RevId: b2bd935d2d8555be56baf8feb1a8e5b3a0c666bc
2026-09-09 06:04:59 +00:00
Winston Howes ec512d2347 Harden credential handling in shell snapshots and replay (#44040)
## Why

Shell quoting can hide credentials from raw-text checks, and startup files can restore real credentials after the broker replaces them with dummy values.

## What changed

- Decode shell literals without evaluating them and reject snapshots containing credentials in executable source, including aliases, functions, and heredocs.
- Preserve credential policy overrides, explicit unsets, and aliases whose source variables were removed. Support credential aliases in Zsh tied arrays while rejecting credentials that span array elements.
- Guard snapshot replay against credential restoration through shell startup files and preserve unrelated `ENV` settings.
- Apply Windows environment-key casing rules to credential overrides, suppress unredacted sandbox diagnostics during snapshot capture, and clear inherited environment variables before launching escalated commands.

## Testing

Add regression coverage for shell quoting and escaped credentials, Zsh tied arrays, startup-file replay, readonly credentials, policy overrides, and sensitive capture timeout and cancellation handling.

GitOrigin-RevId: 58274c07c715423241d26ce6dd2c2b4230cf0f64
2026-09-09 04:00:10 +00:00
Winston Howes 808b3411fd Cache protected shell snapshots and harden capture cleanup (#43954)
## Why

Credential-brokered commands rebuild shell snapshots on every invocation. Snapshot capture failures can also include credential-bearing startup output, and descendants holding output pipes need cleanup even after the shell exits.

## What changed

- Cache successful protected snapshots per environment, keyed by working directory, shell, login mode, and sandbox configuration. Rebuild missing or stale snapshots while keeping concurrent captures independently cancellable.
- Invalidate snapshots when shell settings or credential broker configuration change, and retry capture once if broker configuration changes during startup.
- Restore credentials and associated provider context while respecting environment policy filters and explicit overrides.
- Keep full-buffer capture subject to timeout and cancellation through output draining, clean up descendants on capture expiration or drain failure, and omit startup output from snapshot errors.

## Testing

Add regression coverage for snapshot reuse, invalidation, recovery after storage failure, concurrent cancellation, credential-safe errors, and descendant cleanup. Verify that successful background startup and output beyond the shell output cap remain supported.

GitOrigin-RevId: 7c7c8455bf6e3b9ca173dbb4277783469f0b45e6
2026-09-09 00:42:57 +00:00
Winston Howes 2e220af1f6 Protect shell snapshots when credential brokerage is enabled (#43909)
## Why

Shell startup files can introduce credentials or overwrite brokered dummy values. Snapshot capture and replay need to respect the command's sandbox and environment policy without persisting real credentials.

## What changed

- Capture and validate brokered snapshots per command using its requested shell, working directory, login mode, sandbox, and environment policy.
- Discover shell-initialized credentials and prepare snapshot exports for brokered replay. Preserve shell functions and aliases while preventing startup files from overwriting dummy credentials during replay.
- Rebuild snapshots as credential broker state changes and retain each command's snapshot for the lifetime of its process.
- Reject brokered POSIX shell execution when a protected snapshot cannot be created, and reject brokerage with `shell_zsh_fork`. Restore real credentials and remove brokerage markers for unsupported PowerShell and Cmd execution; escalated commands bypass protected snapshots.

## Testing

Add regression coverage for credential discovery and redaction, environment filtering, readonly exports, shell function and heredoc replay, command working directories and login modes, sandbox enforcement, and missing or disabled snapshots.

GitOrigin-RevId: 685356cb9e80f11cf69d053e4d239ee6ccf43a9e
2026-09-08 20:59:50 +00:00
Winston Howes 1530f828cb Preserve complete shell snapshot exports through filtering and replay (#43907)
## Why

Line-based export parsing can truncate multiline values or mistake their contents for declarations. Snapshot replay also needs to restore Bash options before parsing functions that use extended glob syntax.

## What changed

- Capture shell state, aliases, and complete export records with NUL-delimited boundaries, separating capture, credential preparation, and rendering.
- Apply credential and environment policy to whole exports, preserving multiline values and making credential aliases follow allowed overrides.
- Restore Bash `shopt` options before functions, including for Bash running as `sh`.
- Support non-evaluating POSIX `ENV` path expansion and preserve native environment metadata and unset exports.
- Move core and exec-server snapshot consumers to the shared capture parser. Allow capture overhead while enforcing the exec-server's 512 KiB state-and-environment limit before filtering.

## Testing

Add regression coverage for complete capture records, multiline export filtering and replay, credential alias overrides, `ENV` expansion, and `PATH` export state. Extend exec-server tests to cover Bash-backed `sh`, `extglob`, `nocasematch`, `set -u`, and large environments.

GitOrigin-RevId: 7589ab4d137529a395dcee10b5162718de24a621
2026-09-08 20:41:01 +00:00
jif d1d51f6315 Move shell snapshot tests into shell-command (#39480)
## What changed

- Co-locate the Bash and zsh snapshot-script tests with
  `codex-shell-command`, where `snapshot_script` is implemented.
- Add `tempfile` as a dev dependency for the relocated tests.
- Keep the higher-level snapshot lifecycle tests in `codex-core`.

GitOrigin-RevId: 6d5e9ab95962722f7c89ea40512bd70ee81d8da2
2026-08-19 12:56:08 +00:00
jif 681c82f497 Move shell snapshot scripts into codex-shell-command (#39281)
## What changed

- Add a public `snapshot_script` helper to `codex-shell-command` for Bash,
  Zsh, POSIX shell, and PowerShell, returning `None` for Command Prompt.
- Update `codex-core` and its shell snapshot tests to use the shared helper
  without changing snapshot behavior.

GitOrigin-RevId: 779b9dc964635b717922ea9f5646b94c4dbb0da1
2026-08-18 21:11:30 +00:00
Anton Panasenko 2deed3fb9c Preserve zsh tied PATH exports in shell snapshots (#34293)
## Why

zsh prints tied scalar/array exports such as `PATH` and `path` with `-T`
metadata. The snapshot filter rejected that syntax, so commands available only
through the configured `PATH` could not be resolved from a restored snapshot.

## What changed

- Recognize writable tied exports and retain them in zsh shell snapshots.
- Skip readonly exports so restoring a snapshot does not make variables such as
  `PATH` immutable.

## Testing

Added macOS coverage for restoring a tied `PATH`, handling entries with spaces,
deduplicating path entries, excluding readonly tied exports, and resolving a
command available only through the restored snapshot.

GitOrigin-RevId: 297cfae3368230c7f6fe5ef84f0a10b35a2c802c
2026-07-20 08:29:12 +00:00
pakrym-oai 022f1221e8 [codex] Bind shell snapshots to retained thread environments (#28421)
## Why

Shell snapshots are currently session-scoped even though shell and cwd
are properties of a selected turn environment. That makes snapshot
refresh depend on separate session-cwd plumbing, prevents retained
environments from retaining their snapshot work, and can make snapshot
construction use a different shell than command execution.

This follows #27955 by making the retained thread-environment service
own environment snapshot lifecycles. Session configuration remains the
requested selection state, while `ThreadEnvironments` remains the source
of successfully resolved environments.

## What changed

- Configure the shell-snapshot builder before initial environment
resolution.
- Start each local environment snapshot task when its `TurnEnvironment`
is built and retain that shared task while environment ID and cwd still
match.
- Inherit retained environment snapshots into spawned child threads.
- Carry the selected `TurnEnvironment` through shell runtimes so
snapshot construction and command execution use the same
environment-specific shell and cwd.
- Load project instructions and warm plugins/skills after initial
environment resolution.
- Continue decoding invalid UTF-8 instruction files lossily without
emitting a startup warning.
- Keep requested selections in `SessionConfiguration`; failed or
duplicate resolutions only affect the resolved environment snapshot.

## Validation

- `cargo check -p codex-core --tests`
- `just test -p codex-home instructions` (6 passed)
- Focused environment, instruction, shell-snapshot, and user-shell tests
(84 passed)
- Focused shell-snapshot, user-shell, and unified-exec tests (126
passed; two event-timing tests passed on retry)
2026-06-15 20:10:53 -07:00
pakrym-oai 1aa3e432cc [codex] simplify shell snapshot ownership (#27756)
## Why

Shell snapshot lifecycle state was split between `Shell` and
`SessionServices`: `Shell` carried the receiver while session code
exposed and forwarded the raw sender. That coupled shell identity to
mutable snapshot state and made refresh, inheritance, and file lifetime
harder to reason about.

## What changed

- make each `Arc<ShellSnapshot>` represent one cwd-specific snapshot
generation
- store the active generation in `SessionServices` with `ArcSwapOption`
- have construction start the background build and expose only a
cwd-validated snapshot path
- use `ShellSnapshotFile` ownership to delete snapshot files
automatically
- pass snapshot paths explicitly to shell runtimes instead of storing
snapshot state on `Shell`
- preserve inherited and in-flight generations by pinning their `Arc`
while they are in use

## Test plan

- `cargo check -p codex-core --lib`
- `just test -p codex-core 'shell_snapshot::tests'`
- `just test -p codex-core
shell_command_snapshot_still_intercepts_apply_patch`
- `just test -p codex-core
shell_snapshot_deleted_after_shutdown_with_skills`
2026-06-15 08:18:13 -07:00
Ruslan Nigmatullin 4d201e340e state: pass state db handles through consumers (#20561)
## Why

SQLite state was still being opened from consumer paths, including lazy
`OnceCell`-backed thread-store call sites. That let one process
construct multiple state DB connections for the same Codex home, which
makes SQLite lock contention and `database is locked` failures much
easier to hit.

State DB lifetime should be chosen by main-like entrypoints and tests,
then passed through explicitly. Consumers should use the supplied
`Option<StateDbHandle>` or `StateDbHandle` and keep their existing
filesystem fallback or error behavior when no handle is available.

The startup path also needs to keep the rollout crate in charge of
SQLite state initialization. Opening `codex_state::StateRuntime`
directly bypasses rollout metadata backfill, so entrypoints should
initialize through `codex_rollout::state_db` and receive a handle only
after required rollout backfills have completed.

## What Changed

- Initialize the state DB in main-like entrypoints for CLI, TUI,
app-server, exec, MCP server, and the thread-manager sample.
- Pass `Option<StateDbHandle>` through `ThreadManager`,
`LocalThreadStore`, app-server processors, TUI app wiring, rollout
listing/recording, personality migration, shell snapshot cleanup,
session-name lookup, and memory/device-key consumers.
- Remove the lazy local state DB wrapper from the thread store so
non-test consumers use only the supplied handle or their existing
fallback path.
- Make `codex_rollout::state_db::init` the local state startup path: it
opens/migrates SQLite, runs rollout metadata backfill when needed, waits
for concurrent backfill workers up to a bounded timeout, verifies
completion, and then returns the initialized handle.
- Keep optional/non-owning SQLite helpers, such as remote TUI local
reads, as open-only paths that do not run startup backfill.
- Switch app-server startup from direct
`codex_state::StateRuntime::init` to the rollout state initializer so
app-server cannot skip rollout backfill.
- Collapse split rollout lookup/list APIs so callers use the normal
methods with an optional state handle instead of `_with_state_db`
variants.
- Restore `getConversationSummary(ThreadId)` to delegate through
`ThreadStore::read_thread` instead of a LocalThreadStore-specific
rollout path special case.
- Keep DB-backed rollout path lookup keyed on the DB row and file
existence, without imposing the filesystem filename convention on
existing DB rows.
- Verify readable DB-backed rollout paths against `session_meta.id`
before returning them, so a stale SQLite row that points at another
thread's JSONL falls back to filesystem search and read-repairs the DB
row.
- Keep `debug prompt-input` filesystem-only so a one-off debug command
does not initialize or backfill SQLite state just to print prompt input.
- Keep goal-session test Codex homes alive only in the goal-specific
helper, rather than leaking tempdirs from the shared session test
helper.
- Update tests and call sites to pass explicit state handles where DB
behavior is expected and explicit `None` where filesystem-only behavior
is intended.

## Validation

- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo check -p
codex-rollout -p codex-thread-store -p codex-app-server -p codex-core -p
codex-tui -p codex-exec -p codex-cli --tests`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p
codex-rollout state_db_`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p
codex-rollout find_thread_path`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p
codex-rollout find_thread_path -- --nocapture`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p
codex-rollout try_init_ -- --nocapture`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p
codex-rollout`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo clippy -p
codex-rollout --lib -- -D warnings`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p
codex-thread-store
read_thread_falls_back_when_sqlite_path_points_to_another_thread --
--nocapture`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p
codex-thread-store`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p codex-core
shell_snapshot`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p codex-core
--test all personality_migration`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p codex-core
--test all rollout_list_find`
- `RUST_MIN_STACK=8388608 CODEX_SKIP_VENDORED_BWRAP=1
CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p codex-core
--test all rollout_list_find::find_prefers_sqlite_path_by_id --
--nocapture`
- `RUST_MIN_STACK=8388608 CODEX_SKIP_VENDORED_BWRAP=1
CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p codex-core
--test all rollout_list_find -- --nocapture`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p codex-core
interrupt_accounts_active_goal_before_pausing`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p
codex-app-server get_auth_status -- --test-threads=1`
- `CODEX_SKIP_VENDORED_BWRAP=1
CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo test -p
codex-app-server --lib`
- `CODEX_SKIP_VENDORED_BWRAP=1
CARGO_TARGET_DIR=/tmp/codex-target-state-db cargo check -p codex-rollout
-p codex-app-server --tests`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db just fix -p codex-rollout
-p codex-thread-store -p codex-core -p codex-app-server -p codex-tui -p
codex-exec -p codex-cli`
- `CODEX_SKIP_VENDORED_BWRAP=1
CARGO_TARGET_DIR=/tmp/codex-target-state-db just fix -p codex-rollout -p
codex-app-server`
- `CARGO_TARGET_DIR=/tmp/codex-target-state-db just fix -p
codex-rollout`
- `CODEX_SKIP_VENDORED_BWRAP=1
CARGO_TARGET_DIR=/tmp/codex-target-state-db just fix -p codex-core`
- `just argument-comment-lint -p codex-core`
- `just argument-comment-lint -p codex-rollout`

Focused coverage added in `codex-rollout`:

- `recorder::tests::state_db_init_backfills_before_returning` verifies
the rollout metadata row exists before startup init returns.
- `state_db::tests::try_init_waits_for_concurrent_startup_backfill`
verifies startup waits for another worker to finish backfill instead of
disabling the handle for the process.
-
`state_db::tests::try_init_times_out_waiting_for_stuck_startup_backfill`
verifies startup does not hang indefinitely on a stuck backfill lease.
-
`tests::find_thread_path_accepts_existing_state_db_path_without_canonical_filename`
verifies DB-backed lookup accepts valid existing rollout paths even when
the filename does not include the thread UUID.
-
`tests::find_thread_path_falls_back_when_db_path_points_to_another_thread`
verifies DB-backed lookup ignores a stale row whose existing path
belongs to another thread and read-repairs the row after filesystem
fallback.

Focused coverage updated in `codex-core`:

- `rollout_list_find::find_prefers_sqlite_path_by_id` now uses a
DB-preferred rollout file with matching `session_meta.id`, so it still
verifies that valid SQLite paths win without depending on stale/empty
rollout contents.

`cargo test -p codex-app-server thread_list_respects_search_term_filter
-- --test-threads=1 --nocapture` was attempted locally but timed out
waiting for the app-server test harness `initialize` response before
reaching the changed thread-list code path.

`bazel test //codex-rs/thread-store:thread-store-unit-tests
--test_output=errors` was attempted locally after the thread-store fix,
but this container failed before target analysis while fetching `v8+`
through BuildBuddy/direct GitHub. The equivalent local crate coverage,
including `cargo test -p codex-thread-store`, passes.

A plain local `cargo check -p codex-rollout -p codex-app-server --tests`
also requires system `libcap.pc` for `codex-linux-sandbox`; the
follow-up app-server check above used `CODEX_SKIP_VENDORED_BWRAP=1` in
this container.
2026-05-04 11:46:03 -07:00
pakrym-oai dd1321d11b Spread AbsolutePathBuf (#17792)
Mechanical change to promote absolute paths through code.
2026-04-14 14:26:10 -07:00
Michael Bolin 9313c49e4c fix: close Bazel argument-comment-lint CI gaps (#16253)
## Why

The Bazel-backed `argument-comment-lint` CI path had two gaps:

- Bazel wildcard target expansion skipped inline unit-test crates from
`src/` modules because the generated `*-unit-tests-bin` `rust_test`
targets are tagged `manual`.
- `argument-comment-mismatch` was still only a warning in the Bazel and
packaged-wrapper entrypoints, so a typoed `/*param_name*/` comment could
still pass CI even when the lint detected it.

That left CI blind to real linux-sandbox examples, including the missing
`/*local_port*/` comment in
`codex-rs/linux-sandbox/src/proxy_routing.rs` and typoed argument
comments in `codex-rs/linux-sandbox/src/landlock.rs`.

## What Changed

- Added `tools/argument-comment-lint/list-bazel-targets.sh` so Bazel
lint runs cover `//codex-rs/...` plus the manual `rust_test`
`*-unit-tests-bin` targets.
- Updated `just argument-comment-lint`, `rust-ci.yml`, and
`rust-ci-full.yml` to use that helper.
- Promoted both `argument-comment-mismatch` and
`uncommented-anonymous-literal-argument` to errors in every strict
entrypoint:
  - `tools/argument-comment-lint/lint_aspect.bzl`
  - `tools/argument-comment-lint/src/bin/argument-comment-lint.rs`
  - `tools/argument-comment-lint/wrapper_common.py`
- Added wrapper/bin coverage for the stricter lint flags and documented
the behavior in `tools/argument-comment-lint/README.md`.
- Fixed the now-covered callsites in
`codex-rs/linux-sandbox/src/proxy_routing.rs`,
`codex-rs/linux-sandbox/src/landlock.rs`, and
`codex-rs/core/src/shell_snapshot_tests.rs`.

This keeps the Bazel target expansion narrow while making the Bazel and
prebuilt-linter paths enforce the same strict lint set.

## Verification

- `python3 -m unittest discover -s tools/argument-comment-lint -p
'test_*.py'`
- `cargo +nightly-2025-09-18 test --manifest-path
tools/argument-comment-lint/Cargo.toml`
- `just argument-comment-lint`
2026-03-30 11:59:50 -07:00
Michael Bolin 61dfe0b86c chore: clean up argument-comment lint and roll out all-target CI on macOS (#16054)
## Why

`argument-comment-lint` was green in CI even though the repo still had
many uncommented literal arguments. The main gap was target coverage:
the repo wrapper did not force Cargo to inspect test-only call sites, so
examples like the `latest_session_lookup_params(true, ...)` tests in
`codex-rs/tui_app_server/src/lib.rs` never entered the blocking CI path.

This change cleans up the existing backlog, makes the default repo lint
path cover all Cargo targets, and starts rolling that stricter CI
enforcement out on the platform where it is currently validated.

## What changed

- mechanically fixed existing `argument-comment-lint` violations across
the `codex-rs` workspace, including tests, examples, and benches
- updated `tools/argument-comment-lint/run-prebuilt-linter.sh` and
`tools/argument-comment-lint/run.sh` so non-`--fix` runs default to
`--all-targets` unless the caller explicitly narrows the target set
- fixed both wrappers so forwarded cargo arguments after `--` are
preserved with a single separator
- documented the new default behavior in
`tools/argument-comment-lint/README.md`
- updated `rust-ci` so the macOS lint lane keeps the plain wrapper
invocation and therefore enforces `--all-targets`, while Linux and
Windows temporarily pass `-- --lib --bins`

That temporary CI split keeps the stricter all-targets check where it is
already cleaned up, while leaving room to finish the remaining Linux-
and Windows-specific target-gated cleanup before enabling
`--all-targets` on those runners. The Linux and Windows failures on the
intermediate revision were caused by the wrapper forwarding bug, not by
additional lint findings in those lanes.

## Validation

- `bash -n tools/argument-comment-lint/run.sh`
- `bash -n tools/argument-comment-lint/run-prebuilt-linter.sh`
- shell-level wrapper forwarding check for `-- --lib --bins`
- shell-level wrapper forwarding check for `-- --tests`
- `just argument-comment-lint`
- `cargo test` in `tools/argument-comment-lint`
- `cargo test -p codex-terminal-detection`

## Follow-up

- Clean up remaining Linux-only target-gated callsites, then switch the
Linux lint lane back to the plain wrapper invocation.
- Clean up remaining Windows-only target-gated callsites, then switch
the Windows lint lane back to the plain wrapper invocation.
2026-03-27 19:00:44 -07:00
jif-oai d484bb57d9 feat: add suffix to shell snapshot name (#14938)
https://github.com/openai/codex/issues/14906
2026-03-17 17:59:27 +00:00
Michael Bolin 0c8a36676a fix: move inline codex-rs/core unit tests into sibling files (#14444)
## Why
PR #13783 moved the `codex.rs` unit tests into `codex_tests.rs`. This
applies the same extraction pattern across the rest of `codex-rs/core`
so the production modules stay focused on runtime code instead of large
inline test blocks.

Keeping the tests in sibling files also makes follow-up edits easier to
review because product changes no longer have to share a file with
hundreds or thousands of lines of test scaffolding.

## What changed
- replaced each inline `mod tests { ... }` in `codex-rs/core/src/**`
with a path-based module declaration
- moved each extracted unit test module into a sibling `*_tests.rs`
file, using `mod_tests.rs` for `mod.rs` modules
- preserved the existing `cfg(...)` guards and module-local structure so
the refactor remains structural rather than behavioral

## Testing
- `cargo test -p codex-core --lib` (`1653 passed; 0 failed; 5 ignored`)
- `just fix -p codex-core`
- `cargo fmt --check`
- `cargo shear`
2026-03-12 08:16:36 -07:00