## Why
`WorldState` currently remembers its model-visible diff baseline only in
memory. That leaves no durable source for restoring the exact baseline
after resume, fork, rollback, or compaction.
This is the second PR in the WorldState persistence stack, built on
#29833 and following #29249. It records durable state transitions; the
next PR will replay them during rollout reconstruction.
## What
- Add a `world_state` rollout item containing either a full snapshot or
an RFC 7386 JSON Merge Patch.
- Persist a full snapshot after initial context and after compaction
establishes a new context window.
- Persist non-empty patches when later sampling steps or turns advance
the WorldState baseline.
- Write model-visible history before its matching WorldState record, so
an interrupted write can only cause a safe repeated update on replay.
- Preserve WorldState records for full-history forks while excluding
them from thread previews, metadata, and app-server history
materialization.
Older binaries read rollout lines independently, so they skip the
unknown `world_state` records while retaining the rest of the thread.
## Testing
- `just test -p codex-core
snapshot_merge_patch_changes_and_removes_nested_values`
- `just test -p codex-core
world_state_baseline_deduplicates_until_history_is_replaced`
- `just test -p codex-core
deferred_executor_compaction_preserves_then_updates_environment_once`
- `just test -p codex-protocol`
- `just test -p codex-rollout`
- `just test -p codex-state`
- `just test -p codex-thread-store`
- `just test -p codex-app-server-protocol`
Pick up the AgentGraphStore migration.
- Inject an explicit optional agent graph store into `ThreadManager`
- Move all calls to spawn, close, recursive resume, and
subtree/archive/delete/feedback traversal through it
- Keep using `LocalAgentGraphStore` when SQLite is available
This required some changes to the interface to deal with futures:
- The interface now matches `ThreadStore`'s object-safe pattern by
returning a boxed `AgentGraphStoreFuture` directly, allowing
`ThreadManager` to hold `Arc<dyn AgentGraphStore>`
*Slight behavior change!* Unfiltered subtree enumeration now performs a
single all-status breadth-first traversal, so a closed grandchild
beneath an open edge is included; the previous Open-then-Closed
traversals could not cross mixed-status paths and silently omitted it.
## Why
Inter-agent messages are recorded in live history as
`ResponseItem::AgentMessage`, but rollouts stored
`InterAgentCommunication` and rebuilt the response item during resume.
This made the rollout differ from the actual Responses history.
## What changed
- store the prepared `agent_message` response item directly
- keep `trigger_turn` in a small local metadata record for fork
truncation
- keep reading older `inter_agent_communication` rollout items
## Summary
Resuming a persisted thread currently deep-clones its complete rollout
history several times. `InitialHistory` is retained for the app-server
response, copied into thread persistence, and copied again by read-only
accessors. These copies scale with the complete rollout rather than the
bounded model context and add measurable latency for large sessions.
This change stores resumed rollout history in `Arc<Vec<RolloutItem>>`.
Rollout loading wraps the parsed vector once, while app-server response
construction, session initialization, and thread persistence share it
through inexpensive `Arc` clones. Read-only history access now returns a
borrowed slice, and fork paths use `Arc::unwrap_or_clone` where they
genuinely need mutable ownership. Rollout reconstruction also consumes
its temporary context instead of cloning the reconstructed model
history.
The serialized representation remains unchanged. In an artificial 123 MB
rollout benchmark, sharing resumed history reduced cold resume latency
by roughly 9–10%. The affected crates compile with their test targets,
all 80 thread-store tests pass, and the Bazel dependency lock remains
valid.
When using Responses Lite, we should all use `additional_tools` and a
developer item instead of the top level tools array & instructions
field. This keeps things 1-to-1.
Forced namespacing for _all_ tools will land in a following PR after
some coordination & fixes in Responses API (around collisions & return
items).
The goal is to eventually expand the scope of this to _all_ requests
from codex, but that will require larger coordination across providers &
slower rollout.
## Why
Multi-agent delegation policy was split across `multiAgentMode`,
`features.multi_agent_mode`, and `usage_hint_enabled`. These controls
could disagree: a requested mode could be downgraded by the feature
flag, and disabling usage hints also disabled mode instructions.
Some clients also need multi-agent tools without adding
delegation-policy text to model context. The previous two-mode API could
not express that directly.
## What changed
`multiAgentMode` is now the only live delegation-policy control:
| Mode | Behavior |
| --- | --- |
| `none` | Keep multi-agent tools available without adding mode
instructions. |
| `explicitRequestOnly` | Only delegate after an explicit user request.
|
| `proactive` | Delegate when parallel work materially improves speed or
quality. |
- new threads default to `explicitRequestOnly`; omitting the mode on
later turns keeps the current value
- thread start, resume, fork, and settings responses always report the
concrete current mode instead of `null`
- mode selection remains sticky across turns and resume
- usage-hint text no longer controls whether mode instructions apply
- `features.multi_agent_mode` and `usage_hint_enabled` remain accepted
as ignored compatibility settings so existing configs continue to load
- app-server documentation and generated schemas describe the three-mode
API
## Tests
- `just test -p codex-core multi_agent_mode`
- `just test -p codex-core multi_agent_v2_config_from_feature_table`
- `just test -p codex-core spawn_agent_description`
- `just test -p codex-features`
- `just test -p codex-app-server-protocol`
- `just test -p codex-app-server multi_agent_mode`
## Why
Once multi-agent mode can be selected per turn, clients also need to
choose the initial selection when creating a thread and observe that
selection through lifecycle and settings APIs.
The selected value is intentionally distinct from the effective
model-visible value: no client selection is represented as `null`, even
though an eligible multi-agent v2 turn derives `explicitRequestOnly` as
its effective default.
## What changed
- Add the optional experimental `thread/start.multiAgentMode` parameter
and pass it through thread creation.
- Preserve an omitted initial value as an unset selection rather than
eagerly storing `explicitRequestOnly`.
- Apply an explicit `thread/start` selection to the first turn through
the session configuration established at thread creation.
- Restore the latest persisted effective mode as the selected baseline
on cold resume when rollout history contains one.
- Inherit the optional selected mode from a loaded parent when creating
related runtime threads.
- Return the current selected `multiAgentMode` from `thread/start`,
`thread/resume`, `thread/fork`, and thread settings, using `null` when
no mode is selected.
- Keep lifecycle reporting independent from model capability and feature
eligibility; core turn construction remains responsible for calculating
and persisting the effective mode.
## Not covered
- Clearing an existing loaded-session selection back to unset through
`turn/start`; omitted or `null` currently retains the session's
selection.
- A TUI control, slash command, or `config.toml` preference.
## Verification
- `CARGO_INCREMENTAL=0 just test -p codex-app-server-protocol`
- `CARGO_INCREMENTAL=0 just test -p codex-app-server multi_agent_mode`
The focused app-server coverage verifies explicit `thread/start`
initialization, first-turn prompting, nullable reporting for an omitted
selection, and retention of selections that are not currently
runtime-eligible.
## Stack
Stacked on #28685. This PR contains only the thread initialization and
lifecycle/settings API layer.
## Why
Shell snapshots are currently session-scoped even though shell and cwd
are properties of a selected turn environment. That makes snapshot
refresh depend on separate session-cwd plumbing, prevents retained
environments from retaining their snapshot work, and can make snapshot
construction use a different shell than command execution.
This follows #27955 by making the retained thread-environment service
own environment snapshot lifecycles. Session configuration remains the
requested selection state, while `ThreadEnvironments` remains the source
of successfully resolved environments.
## What changed
- Configure the shell-snapshot builder before initial environment
resolution.
- Start each local environment snapshot task when its `TurnEnvironment`
is built and retain that shared task while environment ID and cwd still
match.
- Inherit retained environment snapshots into spawned child threads.
- Carry the selected `TurnEnvironment` through shell runtimes so
snapshot construction and command execution use the same
environment-specific shell and cwd.
- Load project instructions and warm plugins/skills after initial
environment resolution.
- Continue decoding invalid UTF-8 instruction files lossily without
emitting a startup warning.
- Keep requested selections in `SessionConfiguration`; failed or
duplicate resolutions only affect the resolved environment snapshot.
## Validation
- `cargo check -p codex-core --tests`
- `just test -p codex-home instructions` (6 passed)
- Focused environment, instruction, shell-snapshot, and user-shell tests
(84 passed)
- Focused shell-snapshot, user-shell, and unified-exec tests (126
passed; two event-timing tests passed on retry)
## Description
This PR adds an optional `metadata` field to `ResponseItem` for
Responses API calls. Only mechanical plumbing, no actual values
populated and sent yet. Turns out just adding a new field to
`ResponseItem` has quite a large blast radius already.
This change is backwards compatible because `metadata` is optional and
omitted when absent, so existing response items and rollout history
without it still deserialize and requests that do not set it keep the
same wire shape. For provider compatibility, we strip out `metadata`
before non-OpenAI Responses requests so Azure and AWS Bedrock never see
this field.
My followup PR here will actually make use of it to start storing and
passing along `turn_id`: https://github.com/openai/codex/pull/28360
## What changed
- Added `ResponseItemMetadata` with optional `turn_id`, plus optional
`metadata` on Responses API item variants and inter-agent communication.
- Preserved item metadata through response-item rewrites such as
truncation, missing tool-output synthesis, compaction history
rebuilding, visible-history conversion, rollout/resume, and generated
app-server schemas/types.
- Strip item metadata from non-OpenAI Responses requests while
preserving it for OpenAI-shaped requests.
- Updated the mechanical fixture/test construction churn required by the
new optional field.
## Why
Multi-agent v2 `send_message` deliveries already reach the receiving
model as typed `agent_message` items with encrypted content.
Child-completion notifications are generated by Codex itself, so their
content is plaintext and previously fell back to a serialized JSON
envelope inside an assistant message.
With plaintext `input_text` supported for `agent_message`, both delivery
paths can use the same model-visible type while preserving explicit
author and recipient metadata.
## What changed
- add plaintext `input_text` support to `AgentMessageInputContent` and
regenerate the affected app-server schemas
- preserve `InterAgentCommunication` as structured mailbox input instead
of converting it to assistant text
- record delivered communications as typed `agent_message` history items
- persist a dedicated rollout item so local delivery metadata such as
`trigger_turn` remains available without leaking into the Responses
request
- reconstruct typed agent messages on resume and preserve fork-turn
truncation behavior
- remove request-time assistant-content parsing
- preserve plaintext and encrypted inter-agent deliveries in stage-one
memory inputs
- normalize and link plaintext and encrypted agent messages in rollout
traces without treating inbound messages as child results
- cover the real MultiAgent V2 child-completion path end to end with
deterministic mailbox synchronization
## Verification
- `just test -p codex-core
plaintext_multi_agent_v2_completion_sends_agent_message`
- `just test -p codex-core input_queue_drains_mailbox_in_delivery_order
record_initial_history_reconstructs_typed_inter_agent_message
fork_turn_positions_use_inter_agent_delivery_metadata`
- `just test -p codex-memories-write
serializes_inter_agent_communications_for_memory`
- `just test -p codex-rollout-trace
agent_messages_preserve_routing_and_content
sub_agent_started_activity_creates_spawn_edge`
- `just test -p codex-rollout-trace
agent_result_edge_falls_back_to_child_thread_without_result_message`
- `just test -p codex-protocol -p codex-rollout -p
codex-app-server-protocol`
## Why
Multi-agent v2 residency is intended to keep only the threads that need
to be live. The existing rollout resume path still walked persisted open
descendants and reopened the entire descendant tree when resuming a v2
root, which turns resume into an eager reload of work that should stay
unloaded until it is explicitly needed.
The interrupted-agent path has a related residency issue. Interrupted
agents remain open by design, so an idle interrupted resident should be
eligible for eviction just like an idle completed or errored resident.
Otherwise a resident set full of interrupted agents can consume every v2
slot and block later spawns or reloads with `AgentLimitReached`.
## What Changed
- Return early from `resume_agent_from_rollout` after resuming a v2
thread so persisted v2 descendants are not reopened eagerly.
- Treat idle `Interrupted` v2 residents as unloadable in the LRU
residency path.
- Add focused coverage for v2 root resume leaving descendants unloaded
and for eviction of an idle interrupted v2 resident when a new slot is
needed.
## Verification
Added targeted `codex-core` tests covering:
- v2 root resume with persisted descendants, verifying only the root is
loaded after resume.
- residency eviction of an idle interrupted v2 agent when the resident
set is full.
## Why
Multi-Agent V2 concurrency should count active non-root turns, not
resident or durable agent threads. The limit is intentionally best
effort: admission checks are synchronous, but concurrent successful
checks may overshoot slightly.
## What changed
- Keep one root-derived execution limit on the shared `AgentControl`.
- Count active V2 subagent turns with an RAII guard owned by
`RunningTask`.
- Check capacity before spawning or starting an idle agent, including
direct app-server `turn/start` submissions.
- Preserve queued delivery for agents that are already running.
- Exempt automatic idle continuations so `/goal` work is not dropped
when capacity is temporarily full.
- Keep root and V1 turns outside this limiter.
## Test coverage
- `execution_guards_count_active_v2_subagent_turns`
- `execution_guards_ignore_root_and_v1_turns`
- `v2_nested_spawn_checks_shared_active_execution_capacity`
## Why
Multi-agent v2 treats agents as durable logical agents, not just live
entries in `ThreadManager`. After the reload-on-delivery change, a v2
agent can be addressed even if its thread is not currently loaded.
This PR adds the next layer: loaded v2 subagents can be paged out of
`ThreadManager` when the session has too many resident agents. That
keeps residency separate from logical identity and prepares the stack
for making v2 concurrency count active execution instead of existing
agents.
## What Changed
- Add an `AgentControl`-scoped LRU for resident v2 subagents.
- Reserve residency before spawning or reloading a v2 subagent.
- If resident capacity is full, unload the least-recently-used idle v2
subagent from `ThreadManager`.
- Keep `ThreadManager` as a primitive loaded-thread store; it does not
own the LRU policy.
- Keep unloaded agents registered and durable so they can be reloaded by
the delivery path.
- Preserve the existing v2 cap semantics by using the derived non-root
v2 cap for residency.
Eviction is intentionally conservative. A thread is unloadable only when
it is a v2 subagent, has completed or errored, has no active turn, and
has no pending mailbox work. Before removal, the rollout is materialized
and flushed.
## Assumptions And Non-Goals
- PR #26623 provides the reload-on-delivery path for unloaded v2 agents.
- `ThreadManager` membership means loaded/resident, not logical agent
existence.
- `AgentRegistry` remains the logical identity/metadata source for v2
agents that may be unloaded.
- `list_agents` remains a recent/resident view for now.
- This does not change active execution concurrency; that is the next
PR.
- This does not change `close_agent` semantics.
- This does not change or remove `resume_agent`.
- This does not add a new residency config knob.
## Stack
1. V2 durable lookup and reload on delivery (#26623) - reload unloaded
v2 agents before delivering follow-up/input.
2. V2 residency LRU (this PR) - unload idle resident v2 agents from
`ThreadManager` when resident capacity is full.
3. V2 active-execution concurrency - count running non-root v2 turns
instead of logical agents.
4. V2 close/interrupt semantics - make v2 close interrupt the current
turn without deleting durable identity.
5. V2 resume cleanup - remove the manual resume surface for v2 while
keeping internal reload support.
## Validation
- Added focused coverage for the residency LRU eviction path.
- Local clippy/check/tests were not run; CI will cover them.
## Summary
This is the first small step toward making multi-agent v2 agents durable
logical agents whose `ThreadManager` residency is only an implementation
detail.
This PR adds a narrow v2 reload-on-delivery hook:
- If a known v2 agent target is already loaded, delivery is unchanged.
- If the target is still registered but missing from `ThreadManager`,
delivery reloads that exact v2 thread from durable rollout history
before submitting the message.
- If the target is unknown, closed, missing from storage, or not a v2
thread, delivery still fails as not found.
The reload is wired only into existing-agent delivery paths: v2
`send_message` / `followup_task`, and legacy `send_input` when its
target is a known v2 agent.
## Stack
1. **Reload on delivery**: load known unloaded v2 agents before
`followup_task`, `send_message`, or `send_input` delivery. This PR.
2. **Residency LRU**: unload idle resident v2 agents from
`ThreadManager` without making them closed or unreachable.
3. **Execution concurrency**: count active non-root turns, not logical
agents or resident idle threads.
4. **Close semantics**: make v2 close interrupt-only and leave durable
agent identity intact.
5. **Resume cleanup**: remove user-facing v2 resume semantics;
addressing an unloaded durable agent reloads it implicitly.
## Validation
- Ran `just fmt`.
- Left broader tests and clippy to CI.
## Summary
Mechanically splits `AgentControl` into focused modules so later agent
runtime changes are easier to review. The shared lookup, messaging, and
completion logic remains in `control.rs`, while spawn-specific code and
V1 legacy close/resume behavior move into dedicated files.
## Changes
- Extract spawn-agent code into `agent/control/spawn.rs`.
- Extract V1-only legacy close/resume behavior into
`agent/control/legacy.rs`.
- Keep shared control-plane behavior in `agent/control.rs`.
- Preserve existing behavior; this PR is intended to be mechanical.
## Stack
1. This PR - Mechanical `AgentControl` split: extracts spawn and V1
legacy code without behavior changes.
2. #26614 - Execution slot accounting: separates logical agents from
active execution slots.
3. #26611 - Residency and reload runtime: adds resident-agent LRU,
eviction/reload, durable lookup, and V2 delivery through reload.
4. #26612 - V2 tool semantics: narrows `close_agent` to interrupt-only
and updates V2 tool coverage.
sayan-oai