Commit Graph
20 Commits
Author SHA1 Message Date
Rasmus Rygaard 66f0220c56 [codex] Report the exec-server working directory (#29666)
## Summary

- add the exec-server working directory to `environment/info` as an
optional `PathUri`
- populate it from the executor process's current directory
- preserve compatibility with older responses that omit `cwd`

## Why

Remote clients currently have no executor-native default working
directory. This forces callers such as app-server-backend to assume
`/workspace`, which fails for laptop environments. Reporting the cwd
alongside the detected shell lets clients use the path convention and
location of the actual executor.

## Impact

This is backward-compatible: the new response field is optional, and
clients can continue handling responses from older exec servers. A
follow-up app-server-backend change will consume the value for cwd-less
`command/exec` requests.

## Validation

- `just test -p codex-exec-server` (275 passed, 2 skipped)
2026-06-23 13:39:13 -07:00
Rasmus Rygaard d69828e18b Revert "Handle additional tools in rollout persistence metrics" (#29680)
Reverts openai/codex#29669
2026-06-23 18:35:54 +00:00
Rasmus Rygaard 7e310bc7f3 Inject state DB, agent graph store (#20689)
## Why

We want the agent graph store to be passed down the stack as a real
dependency, the same way we already treat the thread store.

This will let us inject the agent graph store as a real dependency and
support implementations other than the local SQLite-backed one. Right
now most code instantiates a state DB and an agent graph store
just-in-time. Ideally, we would not depend on the state DB directly but
only read through the higher-level interfaces.

This change makes the dependency boundaries explicit and moves state DB
initialization to process bootstrap instead of hiding it inside local
store implementations.

## What changed

- `ThreadManager` now requires a `StateDbHandle` and an
`AgentGraphStore` at construction time instead of treating them as
optional internals.
- The local store constructors no longer lazily initialize SQLite.
Callers now initialize the state DB once per process and use that shared
handle to build:
  - `LocalThreadStore`
  - `LocalAgentGraphStore`
- App bootstraps (`app-server`, `mcp-server`, `prompt_debug`, and the
thread-manager sample) now initialize the state DB up front and inject
the resulting handle down the stack.
- `app-server` now consistently uses its process-scoped state DB handle
instead of reopening SQLite or trying to recover it from loaded threads.
- Device-key storage now reuses the shared state DB handle instead of
maintaining its own lazy opener.
- The thread archive / descendant traversal paths now use the injected
`AgentGraphStore` instead of reaching through local
thread-store-specific state.

## Verification

- `cargo check -p codex-core -p codex-thread-store -p codex-app-server
-p codex-mcp-server -p codex-thread-manager-sample --tests`
- `cargo test -p codex-thread-store`
- `cargo test -p codex-core
thread_manager_accepts_separate_agent_graph_store_and_thread_store --
--nocapture`
- `cargo test -p codex-app-server
thread_archive_archives_spawned_descendants -- --nocapture`
2026-05-05 21:45:29 +00:00
Rasmus Rygaard 782191547c Add agent graph store interface (#19229)
## Summary

Persisted subagent parent/child topology currently leaks through
`StateRuntime`'s SQLite-specific thread-spawn helpers. This PR
introduces a narrow `AgentGraphStore` boundary so follow-up work can
route graph operations through a local or remote store without coupling
orchestration code directly to the state DB graph API.

## Changes

- Adds the new `codex-agent-graph-store` crate.
- Defines a flat `AgentGraphStore` trait for the v1 graph surface:
upsert edge, set edge status, list direct children, and list
descendants.
- Adds public graph types for `ThreadSpawnEdgeStatus`,
`AgentGraphStoreError`, and `AgentGraphStoreResult`.
- Implements `LocalAgentGraphStore` on top of an existing
`codex_state::StateRuntime`, preserving today's SQLite-backed
`thread_spawn_edges` behavior.
- Registers the crate in Cargo/Bazel metadata.

This PR only adds the local contract and implementation; call-site
migration and the remote gRPC store are left to the follow-up PRs in the
stack.

## Testing

- `cargo test -p codex-agent-graph-store`

The new unit tests cover local parity with the existing `StateRuntime`
graph methods, `Open`/`Closed` filtering, status updates, and stable
breadth-first descendant ordering.
2026-04-29 22:48:26 +00:00
Rasmus Rygaard 5378cccd8a Refactor log DB into LogWriter interface (#19234)
## Why

This prepares feedback log capture for a future remote app-server hook
sink without changing the current local SQLite upload path. The
important boundary is now intentionally small: a log sink is a tracing
`Layer` that can also flush entries it has accepted.

That keeps the existing SQLite implementation simple while giving the
upcoming gRPC sink a place to fit beside it. SQLite and gRPC have
different worker/write semantics, so this PR avoids introducing a shared
buffered-sink abstraction and instead lets each `LogWriter` own the
buffering mechanics it needs.

## What Changed

- Added `LogSinkQueueConfig` with the existing local defaults: queue
capacity `512`, batch size `128`, and flush interval `2s`.
- Added `LogDbLayer::start_with_config(...)` while preserving
`LogDbLayer::start(...)` and `log_db::start(...)` defaults.
- Introduced the `LogWriter` trait as the minimal shared interface:
`tracing_subscriber::Layer` plus `flush()`.
- Made `LogDbLayer` implement `LogWriter`.
- Kept tracing event formatting inside `LogDbLayer`; it still creates
one `LogEntry` per tracing event before queueing it for SQLite.
- Kept normal event capture best-effort and non-blocking via bounded
`try_send`.

## Behavior Notes

This does not change the SQLite schema, retention behavior,
`/feedback/upload`, or Sentry upload behavior. Normal log events still
drop when the queue is full; explicit `flush()` still waits for queue
capacity and receiver processing before returning.

## Verification

- `cargo test -p codex-state log_db`
- `cargo test -p codex-state`
- `just fix -p codex-state`

The added tests cover configured batch-size flushing, configured
interval flushing, queue-full drops, and the flush barrier semantics.
2026-04-24 16:27:39 -07:00
Rasmus Rygaard f11583b8f6 Add remote thread config endpoint (#18908)
## Why

App-server needs a way to fetch thread-scoped config from the remote
thread config service when the user config opts into that behavior. This
mirrors the existing experimental remote thread store endpoint while
keeping local/noop behavior as the default.

Startup paths also need to avoid silently dropping the remote config
endpoint after the first config load. The stdio app-server path
discovers the endpoint from the initial config and installs the real
thread config loader for later config builds, while in-process clients
used by TUI/exec now select the same remote loader directly from their
provided config.

## What changed

- Added `experimental_thread_config_endpoint` to `ConfigToml`, `Config`,
and `core/config.schema.json`.
- Added config parsing coverage for the new setting.
- Updated app-server startup to select `RemoteThreadConfigLoader` from
the initially loaded config, falling back to `NoopThreadConfigLoader`
when unset.
- Let `ConfigManager` replace its thread config loader after startup
discovery so later config loads use the selected loader.
- Updated in-process app-server client startup to pass
`RemoteThreadConfigLoader` when its config has
`experimental_thread_config_endpoint` set.

## Verification

- Added `experimental_thread_config_endpoint_loads_from_config_toml`.
- Added
`runtime_start_args_use_remote_thread_config_loader_when_configured`.
- Ran `cargo check -p codex-app-server --lib`.
- Ran `cargo test -p codex-app-server-client`.
2026-04-23 11:46:06 -07:00
Rasmus Rygaard 0b4f694347 Add remote thread config loader protos (#18892)
## Why

Thread-scoped config needs a stable boundary between the app/session
owner and the config stack. Instead of having call sites manually copy
thread config fields into individual overrides, this adds the proto and
Rust plumbing needed for a `ThreadConfigLoader` implementation to return
typed sources that can be translated into ordinary config layer entries.

Keeping the remote payload typed also makes precedence easier to reason
about: session-owned thread config maps back to the existing session
config source, while user-owned thread config is represented separately
without introducing a new config-layer source until it has TOML-backed
fields.

## What changed

- Added the `codex.thread_config.v1` protobuf service and generated Rust
module for loading thread config sources.
- Added `RemoteThreadConfigLoader`, which calls the gRPC service, parses
`SessionThreadConfig` / `UserThreadConfig`, and validates provider
fields such as `wire_api`, auth timeout, and absolute auth cwd.
- Added proto generation tooling under
`config/scripts/generate-proto.sh` and
`config/examples/generate-proto.rs`.
- Added `ThreadConfigLoader::load_config_layers`, plus static/no-op
loader helpers, so tests and callers can use the same typed loader
interface while config-layer translation stays centralized.

## Verification

- `cargo test -p codex-config thread_config`
2026-04-23 10:06:05 -07:00
Rasmus Rygaard 7b994100b3 Add session config loader interface (#18208)
## Why

Cloud-hosted sessions need a way for the service that starts or manages
a thread to provide session-owned config without treating all config as
if it came from the same user/project/workspace TOML stack.

The important boundary is ownership: some values should be controlled by
the session/orchestrator, some by the authenticated user, and later some
may come from the executor. The earlier broad config-store shape made
that boundary too fuzzy and overlapped heavily with the existing
filesystem-backed config loader. This PR starts with the smaller piece
we need now: a typed session config loader that can feed the existing
config layer stack while preserving the normal precedence and merge
behavior.

## What Changed

- Added `ThreadConfigLoader` and related typed payloads in
`codex-config`.
- `SessionThreadConfig` currently supports `model_provider`,
`model_providers`, and feature flags.
- `UserThreadConfig` is present as an ownership boundary, but does not
yet add TOML-backed fields.
- `NoopThreadConfigLoader` preserves existing behavior when no external
loader is configured.
  - `StaticThreadConfigLoader` supports tests and simple callers.

- Taught thread config sources to produce ordinary `ConfigLayerEntry`
values so the existing `ConfigLayerStack` remains the place where
precedence and merging happen.

- Wired the loader through `ConfigBuilder`, the config loader, and
app-server startup paths so app-server can provide session-owned config
before deriving a thread config.

- Added coverage for:
  - translating typed thread config into config layers,
- inserting thread config layers into the stack at the right precedence,
- applying session-provided model provider and feature settings when
app-server derives config from thread params.

## Follow-Ups

This intentionally stops short of adding the remote/service transport.
The next pieces are expected to be:

1. Define the proto/API shape for this interface.
2. Add a client implementation that can source session config from the
service side.

## Verification

- Added unit coverage in `codex-config` for the loader and layer
conversion.
- Added `codex-core` config loader coverage for thread config layer
precedence.
- Added app-server coverage that verifies session thread config wins
over request-provided config for model provider and feature settings.
2026-04-20 23:05:49 +00:00
Rasmus Rygaard d013576f8b Redirect debug client output to a file (#17234)
In the app-server debug client, allow redirecting output to a file in
addition to just stdout. Shell redirecting works OK but is a bit weird
with the interactive mode of the debug client since a bunch of newlines
get dumped into the shell. With async messages from MCPs starting it's
also tricky to actually type in a prompt.
2026-04-14 09:53:17 -07:00
Rasmus Rygaard 53d5972226 Reapply "Pass more params to compaction" (#14298) (#14521)
This reverts commit 8af97ce4b0.

Confirmed that this runs locally without the previous issues with tool
use
2026-03-12 23:27:21 +00:00
Rasmus Rygaard 7f22329389 Revert "Pass more params to compaction" (#14298) 2026-03-11 12:33:10 -07:00
Rasmus Rygaard 2621ba17e3 Pass more params to compaction (#14247)
Pass more params to /compact. This should give us parity with the
/responses endpoint to improve caching.

I'm torn about the MCP await. Blocking will give us parity but it seems
like we explicitly don't block on MCPs. Happy either way
2026-03-11 12:33:09 -07:00
Rasmus Rygaard 73eaebbd1c Propagate session ID when compacting (#12802)
We propagate the session ID when sending requests for inference but we
don't do the same for compaction requests. This makes it hard to link
compaction requests to their session for debugging purposes
2026-02-25 19:17:38 +00:00
Rasmus Rygaard b2d3843109 Translate websocket errors (#10937)
When getting errors over a websocket connection, translate the error
into our regular API error format
2026-02-09 17:53:09 +00:00
Rasmus Rygaard df000da917 Add a codex.rate_limits event for websockets (#10324)
When communicating over websockets, we can't rely on headers to deliver
rate limit information. This PR adds a `codex.rate_limits` event that
the server can pass to the client to inform them about rate limit usage.
The client parses this data the same way we parse rate limit headers in
HTTP mode.

This PR also wires up the etag and reasoning headers for websockets
2026-02-04 06:01:47 -08:00
Rasmus Rygaard e3d39013d3 Handle exec shutdown on Interrupt (fixes immortal codex exec with websockets) (#10519)
### Motivation
- Ensure `codex exec` exits when a running turn is interrupted (e.g.,
Ctrl-C) so the CLI is not "immortal" when websockets/streaming are used.

### Description
- Return `CodexStatus::InitiateShutdown` when handling
`EventMsg::TurnAborted` in
`exec/src/event_processor_with_human_output.rs` so human-output exec
mode shuts down after an interrupt.
- Treat `protocol::EventMsg::TurnAborted` as
`CodexStatus::InitiateShutdown` in
`exec/src/event_processor_with_jsonl_output.rs` so JSONL output mode
behaves the same.
- Applied formatting with `just fmt`.

### Testing
- Ran `just fmt` successfully.
- Ran `cargo test -p codex-exec`; many unit tests ran and the test
command completed, but the full test run in this environment produced
`35 passed, 11 failed` where the failures are due to Landlock sandbox
panics and 403 responses in the test harness (environmental/integration
issues) and are not caused by the interrupt/shutdown changes.

------
[Codex
Task](https://chatgpt.com/codex/tasks/task_i_698165cec4e083258d17702bd29014c1)
2026-02-03 14:38:21 -08:00
Rasmus Rygaard 39e09c289d Add a wrapper around raw response items (#5923)
We currently have nested enums when sending raw response items in the
app-server protocol. This makes downstream schemas confusing because we
need to embed `type`-discriminated enums within each other.

This PR adds a small wrapper around the response item so we can keep the
schemas separate
2025-10-29 20:32:40 +00:00
Rasmus Rygaard a1635eea25 [app-server] Annotate more exported types with a title (#5879)
Follow-up to https://github.com/openai/codex/pull/5063

Refined the app-server export pipeline so JSON Schema variants and
discriminator fields are annotated with descriptive, stable titles
before writing the bundle. This eliminates anonymous enum names in the
generated Pydantic models (goodbye Type7) while keeping downstream
tooling simple. Added shared helpers to derive titles and literals, and
reused them across the traversal logic for clarity. Running just fix -p
codex-app-server-protocol, just fmt, and cargo test -p
codex-app-server-protocol validates the change.
2025-10-28 16:35:12 -07:00
Rasmus Rygaard 846960ae3d Generate JSON schema for app-server protocol (#5063)
Add annotations and an export script that let us generate app-server
protocol types as typescript and JSONSchema.

The script itself is a bit hacky because we need to manually label some
of the types. Unfortunately it seems that enum variants don't get good
names by default and end up with something like `EventMsg1`,
`EventMsg2`, etc. I'm not an expert in this by any means, but since this
is only run manually and we already need to enumerate the types required
to describe the protocol, it didn't seem that much worse. An ideal
solution here would be to have some kind of root that we could generate
schemas for in one go, but I'm not sure if that's compatible with how we
generate the protocol today.
2025-10-20 11:45:11 -07:00
Rasmus Rygaard cc46d3e359 Update model in code to o4-mini (#39)
Old docs had o3 as the default
2025-04-16 11:17:52 -07:00