## Why
Persisting user input received during an active turn currently blocks the next model request. Stores that support background persistence can overlap this checkpoint with inference.
## What changed
- Add `PersistContext::SteeredUserInput` and `allows_background_persistence()` so stores may enqueue these checkpoints, with durability and error reporting enforced by later flush or shutdown operations.
- Use the new context for accepted steered user input and apply the same metadata handling as turn-start persistence.
- Keep tool outputs synchronous, including in mixed input batches, and allow stores to retain synchronous persistence for all contexts.
## Testing
Add gated-store integration tests covering background user-input persistence, synchronous stores, and synchronous tool-output checkpoints. Verify that the next request includes the steered input and waits for persistence when required.
GitOrigin-RevId: c60b7b6c9b483245fd3169306bcf0de248ccdf35
## Why
Session initialization can be cancelled after a persistence writer is installed but before acquisition returns. Cleanup must wait for acquisition to finish so it can discard the writer.
## What changed
Extend `LiveThreadInitGuard` to own in-flight acquisition and finish it before discarding persistence, including when the guard is dropped. Use the guard for thread creation, resume, and inherited model context initialization.
## Testing
Add a regression test that cancels acquisition after writer installation, verifies cleanup waits for handoff, and confirms the writer is removed afterward.
GitOrigin-RevId: 09041fa5cd675d082a86a4cc8a719c572afe11a8
## Why
Thread stores may filter a metadata patch to a no-op. Requiring every successful
update to return a `StoredThread` forces those implementations to materialize a
thread even when the caller does not need one.
## What changed
- Let `ThreadStore::update_thread_metadata` return `None` after a successful
update that did not materialize a thread.
- Preserve the materialized-thread contract of `LiveThread` and `ThreadManager`
metadata updates by reading the thread when the store returns `None`.
- Keep completion-only metadata updates from issuing that fallback read.
## Testing
Cover materialized in-memory updates and verify that fallback reads occur only
for callers that require the updated thread.
GitOrigin-RevId: 3059efc9d5b69e0b46e35fb5b5aa9638086654f2
## What changed
- Add `PersistContext` to the thread-store persistence contract so stores can
identify persistence requested immediately before model sampling. A
`TurnStart` request may be enqueued in the background when later flush and
shutdown operations fence it and report any failure; all other persistence
remains synchronous.
- Use the turn-start context only for the initial input of a regular turn, while
admission acknowledgments, steered input, and other persistence paths keep
the standard durability behavior.
- Flush pending metadata before turn-start persistence and ensure shutdown is
still attempted if that metadata update fails.
## Testing
- Verify that developer instructions and user input are persisted before the
model request completes for a default paginated-history thread.
GitOrigin-RevId: 162f0ec796a61aebe66ca0b909fbcccec3047b85
## What changed
- Add `codex-history` for model-history and persisted-rollout domain types, including `RolloutItem`, `RolloutLine`, `CompactedItem`, and initial/resumed history state.
- Re-export the persisted types from `codex-rollout` and update consumers to use the new crate boundary instead of `codex-protocol`.
- Preserve existing rollout serialization, including legacy numeric compacted-window IDs.
## Testing
- Add `codex-history` tests for rollout JSON round trips, compacted-history compatibility, persisted history modes, and multi-agent version selection.
GitOrigin-RevId: 944daa9297ddd231d3aebbdcb05fff4adf8b4e1b
## Why
Paginated rollout history may contain only a bounded suffix. Deriving display
metadata from that suffix can replace the thread's original preview, title, and
first user message when the thread is read or resumed.
## What changed
- Use SQLite as the canonical source of display metadata for paginated thread
reads, including reads by an explicit rollout path.
- Seed resume metadata tracking from the persisted values so appended history
fills only missing fields instead of overwriting existing ones.
- Prefer an explicitly requested rollout path over a stale path stored in
SQLite while retaining the canonical metadata.
## Testing
Added coverage for reads by ID and path, bounded-history resumes across app
server restarts, preservation of existing metadata, initialization of missing
fields, and stale persisted rollout paths.
GitOrigin-RevId: b393d0727afcba70fda7f50a6f2dcca0632745b4
## What changed
- Inherit paginated history mode when spawning or forking a subagent from a paginated parent.
- Load the parent's model context when forking and persist it as an inherited rollout prefix, while excluding that prefix from the child's projected turns, items, and metadata.
- Record the boundary between inherited context and child history, and reject partially initialized paginated subagent rollouts on resume.
- Resume paginated subagents from their stored model context instead of legacy rollout history.
## Testing
Added coverage for fresh and forked paginated subagents, compacted-history forks, model-context resume, inherited-prefix projection, metadata isolation, and incomplete-prefix detection.
GitOrigin-RevId: e57dc37c29aa4aaaf57b052b991be591d730e5ba
## Description
This PR makes new threads with `history_mode = "paginated"` persist
`ItemCompleted(item: <turn_item>)` in their rollout JSONL file.
Legacy threads keep persisting the existing legacy events. Because the
format is selected per thread, a rollout is either legacy or paginated;
we do not need to support mixed rollouts containing both
representations.
This PR depends on [#31473](https://github.com/openai/codex/pull/31473).
## Why
Paginated thread history needs stable turn/item IDs and completed item
snapshots so the later SQLite projector can materialize appended rollout
JSONL without rebuilding the whole thread.
Keeping the legacy persistence policy unchanged avoids changing
historical rollouts or the readers that still consume them.
## What changed
- Made rollout filtering history-mode aware. Paginated threads keep
completed canonical `ItemCompleted` events and drop their redundant
legacy projections; legacy threads keep the existing event set.
- Made forks inherit the source thread history mode, so copied legacy
history is never filtered as paginated.
- Made paginated threads assign IDs to locally-created response items
even when `Feature::ItemIds` is off, and reject streamed output items
that arrive without server IDs.
- Updated legacy turn replay, rollout list/search, and SQLite metadata
extraction to understand completed canonical user-message items.
- Add 1%-sampled rollout persistence metrics that report per-item and
per-thread JSON byte totals before and after filtering when metrics
export is enabled.
- Tag each item with its exact response or event variant, including
nested turn-item kinds for conditionally persisted completion events, so
aggregate cloud-storage impact can be estimated by policy choice.
## Summary
Resuming a persisted thread currently deep-clones its complete rollout
history several times. `InitialHistory` is retained for the app-server
response, copied into thread persistence, and copied again by read-only
accessors. These copies scale with the complete rollout rather than the
bounded model context and add measurable latency for large sessions.
This change stores resumed rollout history in `Arc<Vec<RolloutItem>>`.
Rollout loading wraps the parsed vector once, while app-server response
construction, session initialization, and thread persistence share it
through inexpensive `Arc` clones. Read-only history access now returns a
borrowed slice, and fork paths use `Arc::unwrap_or_clone` where they
genuinely need mutable ownership. Rollout reconstruction also consumes
its temporary context instead of cloning the reconstructed model
history.
The serialized representation remains unchanged. In an artificial 123 MB
rollout benchmark, sharing resumed history reduced cold resume latency
by roughly 9–10%. The affected crates compile with their test targets,
all 80 thread-store tests pass, and the Bazel dependency lock remains
valid.
## Why
#28152 jumped the gun on moving the rollout format to store URIs, and
would likely break compat with some features that don't go through the
same types as the core logic.
## What
Make `TurnContextItem.cwd` an `AbsolutePathBuf` again, remove test added
for `PathUri` serialization in rollouts. Also drops a bunch of error
paths that are no longer needed.
## Why
Model-visible `<environment_context>` should match the environment of
the executor, not of the app server.
Stacked on #28146.
## What
- Keep selected environment cwd values as `PathUri` while building
environment context.
- Render cwd text using the path convention represented by the URI, with
the canonical URI as a fallback.
- Preserve compatibility with legacy `TurnContextItem.cwd` values when
reconstructing and diffing context.
- Extend the Wine-backed remote Windows test to assert that the model
sees `powershell` and `C:\windows`.
Move the application of the persistence policy into the thread store, so
thread stores can get raw append items rather than canonical append
items. This will enable store-specific projections over the raw input
items.
## Summary
Remove the dead experimental `persistExtendedHistory` app-server flag
and collapse rollout persistence to the single policy app-server already
used.
## What Changed
- Removed `persistExtendedHistory` from v2 thread start/resume/fork
params and deleted its deprecation notice path.
- Removed the persistence-mode enums and plumbing through core, rollout,
and thread-store.
- Made rollout filtering mode-free, keeping the existing limited
persisted-history behavior.
## Test Plan
- `just write-app-server-schema`
- `cargo nextest run --no-fail-fast -p codex-app-server-protocol
schema_fixtures`
- `cargo nextest run --no-fail-fast -p codex-app-server
thread_shell_command_history_responses_exclude_persisted_command_executions`
- `cargo nextest run --no-fail-fast -p codex-rollout -p
codex-thread-store`
- final `rg` for removed flag/type names
- make ThreadStore::update_thread_metadata accept a broad range of
metadata patches
- keep ThreadStore::append_items as raw canonical history append (no
metadata side effects)
- in the local store, write these metadata updates to a combination of
sqlite and rollout jsonl files for backwards-compat. It special cases
which fields need to go into jsonl vs sqlite vs whatever, confining the
awkwardness to just this implementation
- in remote stores we can simply persist the metadata directly to a
database, no special casing required.
- move the "implicit metadata updates triggered by appending rollout
items" from the RolloutRecorder (which is local-threadstore-specific) to
the LiveThread layer above the ThreadStore, inside of a private helper
utility called ThreadMetadataSync. LiveThread calls ThreadStore
append_items and update_metadata separately.
- Add a generic update metadata method to ThreadManager that works on
both live threads and "cold" threads
- Call that ThreadManager method from app server code, so app server
doesn't need to worry about whether the thread is live or not
Migrate token usage replay, rollback responses, and detached review
setup (a special case of forking) to be served from ThreadStore reads
rather direct rollout files.
- replay restored token usage from already-loaded `RolloutItem` history
instead of reopening `Thread.path`
- rebuild rollback responses from loaded `ThreadStore` snapshots and
history
- start detached reviews from store-backed parent history and stored
review-thread metadata
- remove obsolete app-server rollout-summary helper code that became
dead after the store-backed migration
- preserve response/notification ordering for resume, fork, rollback,
and detached review flows
- add integration test coverage for the affected paths
- Route `thread/metadata/update` through
`ThreadStore::update_thread_metadata`.
- Add `LocalThreadStore` git metadata patch support for set, partial
update, and clear semantics.
- Add some unit tests for the new thread store code
- Remove a lot of dead code/tests!
Begin migrating the thread write codepaths to ThreadStore.
This starts using ThreadStore inside of core session code, not only in
the app server code.
Rework the interfaces around thread recording/persistence. We're left
with the following:
* `ThreadManager`: owns the process-level registry of loaded threads and
handles cross-thread orchestration: start, resume, fork, lookup, remove,
and route ops to running CodexThreads.
* `CodexThread`: represents one loaded/running thread from the outside.
It is the handle app-server and callers use to submit ops, inspect
session metadata, and shut the thread down.
* `LiveThread`: session-owned persistence lifecycle handle for one
active thread. Core session code uses it to append rollout items,
materialize lazy persistence, flush, shutdown, discard init-failed
writers, and load that thread’s persisted history.
* `ThreadStore`: storage backend abstraction. It answers “how are
threads persisted, read, listed, updated, archived?” Local and remote
implementations live behind this trait.
* `LocalThreadStore`: local ThreadStore implementation. It owns the
file/sqlite-specific details and keeps RolloutRecorder as a local
implementation detail.
This is a few too many Thread abstractions for my liking, but they do
all represent different concepts / needs / layers.
Migration note: in places where the core code explicitly requires a
path, rather than a thread ID, throw an error if we're running with a
remote store.
Cover the new local live-writer lifecycle with focused tests and
preserve app-server thread-start behavior, including ephemeral pathless
sessions.
Bryan Ashley