Commit Graph
231 Commits
Author SHA1 Message Date
Eric Traut 80e0b51c9e Make session lookup pagination stable and report listing failures (#51230)
## Why

Session activity can move unread threads ahead of a pagination cursor, and timestamp-only cursors can skip threads with equal creation times. Either can prevent a session label lookup from detecting duplicate labels. Database failures must also remain distinguishable from an exhausted listing.

## What changed

- Resolve session labels using database-only listings ordered by `CreatedAt`, with thread IDs breaking timestamp ties. Accept unique matches across pages with valid timestamp-and-ID cursors, while retaining ambiguity errors for older cursors that cannot establish uniqueness.
- Preserve timestamp index ranges for database continuation queries and timestamp-only cursors for listings that may fall back to the filesystem.
- Propagate database-only listing and name-resolution failures. Keep best-effort name enrichment for ordinary listings and retry latest-session lookup with scan-and-repair after a database-only error.
- Keep loaded sessions selectable in the resume and fork pickers when a later page fails, stop further pagination, and display `Could not load more sessions`.

## Testing

Add regression coverage for creation-time continuation after session activity, cursor query plans, filesystem fallback, database and name-query errors, cross-page label uniqueness, older server cursors, and selection after picker pagination failures.

GitOrigin-RevId: 264ab2e69b8ad12d9edaa31fd253cb9805a46a90
2026-10-06 02:14:22 +00:00
Owen Lin 820f85cf59 Truncate oversized MCP results in paginated thread history (#50458)
## Why

Paginated thread history can persist large MCP results in completed tool-call items. Bound these stored results to avoid retaining multi-megabyte payloads.

## What changed

Apply a 64 KiB preview budget when persisting oversized MCP results in paginated history. Replace the result with a truncated text preview of its serialized JSON, preserving `is_error` and clearing `structuredContent` and `_meta`. Leave small results unchanged.

Share the truncation helper with the existing MCP tool-call event path.

## Testing

Add helper tests for unchanged small results and oversized results with structured content and metadata. Add a thread-history test verifying that persisted MCP results contain a bounded preview with the beginning, end, and truncation marker while preserving tool arguments.

GitOrigin-RevId: 09dce73c70bfd357c9b9630f456e9e5c1d8492cf
2026-10-02 23:15:02 +00:00
Owen Lin 3c3a990da0 Measure rollout persistence size reductions (#50454)
## What changed

- Record original and persisted item sizes in `codex.rollout.persistence.item_bytes_v2`, distinguished by stage, and record positive size reductions in `codex.rollout.persistence.bytes_removed`. Count the full original size as removed for dropped items.
- Use explicit histogram boundaries from 1 KiB to 64 MiB for item sizes, bytes removed, and `codex.rollout.persistence.turn_bytes_v2`. Larger values remain represented in the overflow bucket and histogram sum.
- Align item and turn size stages and report a measurement error when a kept item's persisted size cannot be measured.

GitOrigin-RevId: 6487acf1c6f2c041ab9680ab398491d3c1541388
2026-10-02 23:06:56 +00:00
pakrym-oai 602d2add6e Persist additional tool definitions in rollout history (#50435)
## Why

`additional_tools` response items were excluded from rollout persistence, dropping their tool definitions from saved history.

## What changed

- Persist `additional_tools` items in both legacy and paginated history modes.
- Expose the variant in generated JSON schemas, TypeScript types, and Python models, with an optional `id`, a `role`, and arbitrary JSON tool definitions.

## Testing

Add a recorder round-trip test that verifies tool definitions survive recording and loading in both history modes without parse errors.

GitOrigin-RevId: 0e7c7b1c10401ec6513fb744de642a1cf9e461d7
2026-10-02 21:42:45 +00:00
Owen Lin bee28e8a06 Cap persisted command output in paginated history at 64 KiB (#50427)
## Why

Paginated history previously persisted completed commands' full `aggregated_output`, allowing large command outputs to inflate stored history.

## What changed

- Limit persisted command `aggregated_output` to 64 KiB, including a truncation marker. Preserve the beginning and end of the output without splitting UTF-8 characters.
- Apply the shared persistence policy to rollout recording, thread-store appends, and legacy-to-paginated history migration.
- Measure persisted payload bytes after truncation so persistence metrics reflect the stored representation.

## Testing

Add coverage for the byte limit, UTF-8 boundaries, unchanged inputs that fit, and markers that exceed the budget. Verify capped output in paginated thread history and migrated legacy history, and verify persistence metrics use the truncated payload size.

GitOrigin-RevId: 01d420d396ad34b42dc5dc208c0bf88d890639ee
2026-10-02 21:00:06 +00:00
jif d6c3b448a4 Test session index thread-name append and removal (#49959)
## What changed

Add a test that appends an original and updated name for one thread alongside another thread's entry. Verify that lookup returns the updated name and that removing the first thread's name entries leaves only the other thread's entry.

GitOrigin-RevId: cedf260a639f0c0e76aeda0d446559c3821bd911
2026-10-01 11:09:12 +00:00
Charlie Marsh 4f699cd642 Move session index I/O off async runtime threads (#49708)
## Why

Session index updates perform synchronous file I/O and wait on a blocking mutex inside async functions, blocking runtime threads.

## What changed

Run appends and thread-name removals in `spawn_blocking` workers. Acquire a Tokio mutex before dispatch and keep its owned guard in the worker so queued and running updates remain serialized even if the awaiting task is cancelled. Move the lookup's file-existence check into its blocking worker as well.

## Testing

Add tests for lock retention after cancellation of running and queued updates, write ordering after cancellation, and removal that preserves other thread names and malformed lines.

GitOrigin-RevId: 368745d8e5bbf24c527facc0a62ddd0fb76f0ffb
2026-09-30 18:51:29 +00:00
Charlie Marsh 5aa92804d2 Batch rollout listing scans on cancellable blocking workers (#49694)
## What changed

Move directory enumeration and candidate metadata lookups into blocking workers, batching updated-time candidate collection into one worker per scan. Check cancellation between entries and metadata lookups so dropping the async waiter stops further scanning.

Share synchronous rollout metadata resolution with the async helper, preserving plain-file precedence over compressed siblings. Retain scan limits, stable file ordering, and directory-level early stopping for creation-time listings.

## Testing

Add tests for cancellation during directory enumeration and candidate discovery in both flat and date-partitioned layouts, including plain-sibling metadata precedence and filtering of unrelated files, directories, and symlinks.

GitOrigin-RevId: 324289003b29a7d2b3220da03e3c002ab5e23eec
2026-09-30 18:03:48 +00:00
npancha-openai d1c4e3c3e6 Keep compressed rollout snippet searches on one blocking worker (#49692)
## Why

Compressed snippet searches schedule a blocking task for every line read. Keeping the scan on one worker avoids repeated task scheduling.

## What changed

Use `RolloutLineReader::find_map` to scan compressed rollouts on a single blocking worker, stopping at the first matching snippet. Preserve read metrics and stop delivering records when the search is canceled, including cancellation during a read.

## Testing

Add tests for first-match snippets, case-insensitive literal matching, malformed records, missing matches, stream errors before a match, and cancellation between lines or during a read.

GitOrigin-RevId: 295206b1a00e861bc44606bd843f6ed8c3e6aae1
2026-09-30 17:57:58 +00:00
felixxia-oai 92bc601ad6 Reuse unchanged history snapshots when resuming threads (#49600)
## Why

Resuming a thread rereads persisted history even when a preloaded snapshot is unchanged. Reusing that snapshot must still account for writes committed before writer ownership is acquired.

## What changed

- Carry optional, non-serialized revisions from stored history and model context through thread and agent resume paths.
- Validate the revision under writer ownership and reuse the shared snapshot when it matches. Reload stored history when the revision is missing or changed, while preserving explicit history overrides.
- Resolve the selected rollout before loading context, validate compressed snapshots before materialization, and enforce archived-thread restrictions on canonical paths.

## Testing

Extend coverage for snapshot reuse and late-write refresh in legacy and paginated histories, compressed rollout reuse, archived rollout symlinks, and cold child restoration that includes edits made after preloading history.

GitOrigin-RevId: e1fd7aee6ef2b30f41951e6b7e6b01c3e1420cab
2026-09-30 11:59:13 +00:00
Brent Traut 8c3612fb63 Use memrchr to find newlines in reverse JSONL scans (#49444)
Replace the byte-by-byte reverse newline search with `memchr::memrchr` and add `memchr` as a dependency of `codex-rollout`.

GitOrigin-RevId: b07e42e15efb0d63d7b00ee9a2ee29c446f4cf59
2026-09-30 02:19:47 +00:00
jif d5e6526362 Scan the session index backwards for batch thread name lookups (#49297)
## Why

Batch thread name lookups read the entire append-only session index even when all requested names could be found near the end.

## What changed

Use `ReverseJsonlScanner` in `spawn_blocking` for `find_thread_names_by_ids`, stopping once every requested ID has a usable name. Preserve the latest nonempty, trimmed name for each ID while skipping malformed entries.

## Testing

Add regression coverage for renamed threads, whitespace-only names, Unicode, reads across chunk boundaries, malformed entries, a partial final write, and missing IDs.

GitOrigin-RevId: 2ab9254ba8b718fe3e02c47fc9b0de20c09fd10f
2026-09-29 16:16:39 +00:00
Eric Traut fbc169827e Suppress Windows console windows for background subprocesses (#49164)
## Why

Background helpers launched from a detached Windows process can allocate console windows. Job Object launches and containment fallbacks need to preserve console suppression as well.

## What changed

- Add `codex_utils_process::background_command` to set `CREATE_NO_WINDOW` on Windows, and use it across background subprocess launches, including Git, plugins, authentication, diagnostics, and sandbox helpers.
- Centralize console suppression in Job Object launches. Keep `CREATE_NO_WINDOW` when retrying without containment, and clear `CREATE_SUSPENDED` so the fallback child can run.
- Preserve console inheritance for interactive shell launches and interactive AWS authentication refreshes.

## Testing

Add Windows regression tests covering console-free launches from detached parents through standard and Tokio commands, stdout/stderr capture, successful containment, Job Object creation and assignment failures, and sandbox setup refresh helpers.

GitOrigin-RevId: b2c238fa7d4f9fbd868e5adf7d4be9958ff01e6c
2026-09-29 04:38:37 +00:00
Won Park d7748e1185 Add opt-in structured errors for Guardian circuit-breaker interruptions (#48796)
## Why

Guardian denial-limit interruptions lack a structured error identifying the cause. Make this opt-in because older clients may not recognize the new error in shared history.

## What changed

- Add `auto_review.circuit_break_action = "strict"` to attach `TooManyDenials` to the interrupted turn. The default leaves the error unset.
- Expose `turn.error.codexErrorInfo = "tooManyDenials"` in app-server notifications and saved history, and update protocol schemas and generated types.
- Preserve the warning, denial limit, and interrupted status without emitting a separate error notification.

## Testing

Add an integration test covering omitted, `default`, and `strict` settings, checking warnings, live turn errors, the absence of separate error notifications, and persisted turn errors after restart.

GitOrigin-RevId: cb5d379e64728d4d7b9daf834c13756e4a70db31
2026-09-27 22:05:19 +00:00
acrognale-oai 7596ab19a6 Keep archived threads with empty previews visible in listings (#48199)
## Why

Archived threads without previews were filtered out of thread listings, even though they had valid session metadata.

## What changed

- Include archived threads with empty previews in both SQLite and rollout-file listings.
- Add archive-specific indexes for creation, update, and recency ordering so these listings can use indexed pagination.

## Testing

Add regression coverage for paginated archive listings with and without SQLite, archive query plans that avoid temporary sorting, and metadata reads that preserve source and file timestamps for empty archived threads.

GitOrigin-RevId: e29d0257b2e029bea5fdf94eed6dab1006d2a8f2
2026-09-25 20:11:47 +00:00
jif d6c4c6aea4 Classify rollout read failures by reason and progress (#47565)
## Why

Rollout read metrics record the failure stage and I/O error kind, but these do not distinguish specific zstd failures, task join failures, or a busy reader, nor indicate whether any line was read before failure.

## What changed

Add bounded `reason` and `read_progress` labels to rollout read counters and duration metrics, captured at the first failure. Distinguish OS errors, reader-busy and task-join failures, and known zstd error categories, with `stream_error` as the fallback.

Record whether the failure occurred before or after the first line. Classify blocking-task join failures during both open and read. Keep error messages, file paths, and rollout contents out of metric labels.

GitOrigin-RevId: 6db302900f504bc2860504d81c18017adcef3efc
2026-09-23 15:01:19 +00:00
Owen Lin e7bbc79f48 Resume model context from the latest compaction boundary (#47365)
## Why

Resuming a compacted thread should use the compaction and its newer records without depending on older turn contexts or restoring stale state from before the compaction.

## What changed

- Stop paginated model-context scans at the newest compaction with `replacement_history` and `window_number`. Fall back to full replay if the newest compaction lacks either field.
- Align reconstruction with that boundary and restore `last_started_turn_id`, previous turn settings, and the multi-agent version from compaction resume metadata. Allow newer completed turns to override the saved settings.
- Keep canonical session metadata alongside the bounded replay and use the same scan boundary for subagent rollout migration.

## Testing

Add coverage for cold resume after mid-turn compaction, bounded/full replay equivalence, missing companion state, settings precedence, and incomplete-compaction fallback.

GitOrigin-RevId: f836464bb5cadd9be4a4e05b5d1679223713fc87
2026-09-22 21:17:08 +00:00
Owen Lin 286d4ecf44 Persist resume metadata in compaction checkpoints (#47323)
## Why

Compaction companion records do not represent all resume state, including the last started turn and previous turn settings. The selected multi-agent runtime also needs a persisted value when no companion turn context is present.

## What changed

- Add optional `resume_metadata` to `CompactedItem`, recording the multi-agent version, last started turn ID, and previous turn settings. Preserve the distinction between absent metadata and explicitly empty values while retaining compatibility with older records.
- Capture this metadata under the settings persistence lock alongside the compaction checkpoint.
- Rewrite inherited metadata to use the child agent's runtime, and clear previous turn settings when the child must rebuild context.
- Clear pending continuation state before delivering acceptance of standalone settings updates, while holding the persistence lock.

## Testing

Add coverage for metadata round trips with empty values, legacy reader compatibility, checkpoint companion records, inherited fork metadata, and continuation invalidation while settings event delivery is blocked.

GitOrigin-RevId: f30b7feaa7e2fdb5b65fc3c53b37fd7a7fe27903
2026-09-22 16:35:30 +00:00
Charlie Marsh 4f6c106cdd Use compact display metadata for Guardian review threads (#47302)
## Why

Guardian review threads use synthetic approval prompts as user messages. Deriving thread titles and previews from those prompts exposes large, unhelpful summaries in thread listings.

## What changed

- Default Guardian threads to the title "Guardian review" and preview "Approval review" across metadata creation, rollout reads, and resume.
- Skip synthetic user messages and goal updates when deriving Guardian display metadata, leaving `first_user_message` unset for new threads.
- Preserve custom names, including legacy indexed names, when listing, reading, resuming, and migrating threads to paginated history.

## Testing

Add regression coverage for compact metadata, resume without session metadata, custom names before and after migration, and app-server persistence, listing, and rename/resume behavior.

GitOrigin-RevId: 42aa707db20c3e57ffa5fd4c91ee313710be10d7
2026-09-22 15:02:54 +00:00
Jeremy Rose 23ae65bb06 Persist thread creator identity in rollouts and SQLite (#47113)
## What changed

- Capture optional `creator_user_id` and `creator_account_id` from authentication when creating a thread, including forks.
- Carry these fields through thread storage, rollout metadata, and a SQLite migration. Leave them absent when unavailable or when reading older threads.
- Preserve recorded creator values across resume and metadata updates. On revert, recover missing rollout values from SQLite before writing replacement metadata.

## Testing

Add coverage for ChatGPT and API-key sessions, resume under another account, forks using current authentication, metadata extraction and upserts, and repeated reverts after creator fields are removed from a rollout.

GitOrigin-RevId: 9839fa42f9392f12226002e2c27d05d5656f5695
2026-09-21 21:22:01 +00:00
peilin-openai ac98537678 Track cumulative MCP attribution across requests and thread history (#47081)
## What changed

- Record each MCP source and its first turn, including connector and plugin identifiers when available. Cover direct and Code Mode calls, including returned errors and results that are not printed.
- Persist cumulative attribution checkpoints and restore them across compaction, resume, and forks, including forks limited to recent turns.
- Attach fresh attribution to Responses API `client_metadata` under `mcp_attribution`, outside model-visible content. Restrict transmission to allowed OpenAI HTTPS destinations for both HTTP and WebSocket requests.
- Report `attribution_error` for legacy history without checkpoints, malformed or conflicting attribution, and serialized attribution exceeding 16 KiB.

## Testing

Add coverage for source deduplication, checkpoint persistence and restoration, recent-turn forks, direct and Code Mode calls, destination filtering, and the serialized size limit.

GitOrigin-RevId: fd285e152ca9e24bfa000debb20013709d67a86e
2026-09-21 18:19:07 +00:00
jif 23a6b705ba Add diagnostics for rollout compression metadata failures (#47026)
## What changed

- Tag metadata scan failures with bounded reasons for invalid history modes, responses before metadata, missing metadata, or I/O errors.
- Emit at most five metadata warnings per compression run with the rollout ID, error category, file size, and modification age, excluding paths and error messages. Include the total metadata failure count in the completion log.
- Preserve underlying metadata errors while redacting their details from `MetadataReadError` debug output.

GitOrigin-RevId: 13f8ec265405b8f8bb23a02ee37c1aabdce806eb
2026-09-21 13:18:50 +00:00
3d5b66c655 Handle unknown error classifications and configure gateway OAuth (#46482)
Preserve known error classifications, including BioPolicy, while falling back to Other for unknown values so saved sessions can still be read.

Add optional gateway OAuth settings to model provider configuration.

Co-authored-by: Owen Lin <owen@openai.com>
Co-authored-by: alexsong-oai <alexsong@openai.com>
GitOrigin-RevId: 64bcf45ca042efca7645c798db3746375426b0fe
2026-09-18 22:29:16 +00:00
jif 821ad43f9d Tag rollout compression metrics by trigger (#46047)
## Why

Compression metrics do not distinguish startup runs from runs requested through
`rollout/compress`, making their outcomes indistinguishable by entry point.

## What changed

Pass `RolloutCompressionTrigger` from startup and RPC callers through the
compression worker. Add a `trigger` tag with `startup` or `rpc` to run, file,
scan-error, and temporary-file cleanup metrics, including durations, byte counts,
and compression ratios.

GitOrigin-RevId: f23d9da791b666a7311f2323f6bead6570067798
2026-09-16 22:33:15 +00:00
jif da18000cae Measure rollout read and materialization durations (#45966)
## What changed

- Emit one read counter and duration observation per rollout reader, tagged by format, outcome, failure stage, and error kind. Distinguish EOF, partial reads, and failures, including failed opens.
- Accumulate time spent in completed open/retry and read calls, excluding caller processing and canceled calls, without exporting a metric for each JSONL record.
- Record materialization duration when decompressing a rollout for append, for both successful decompression and failures.

GitOrigin-RevId: 3d1e257863ffa696a63040172df54d8234d56ed5
2026-09-16 15:36:43 +00:00
jif 2d90e054d6 Expose partial completion in rollout compression metrics (#45964)
## Why

A compression run can report `completed` after hitting its time budget or encountering file, scan, or cleanup errors. The run metrics do not distinguish these cases from a clean, complete pass.

## What changed

- Add `completion_reason` (`time_budget` or `scan_finished`) and `file_errors`, `scan_errors`, and `cleanup_errors` tags to completed run counts and durations, preserving `status=completed`.
- Record scan failures and additional stale-temp cleanup failures with operation stages and bounded I/O error kinds, and track recoverable errors in the run summary.

GitOrigin-RevId: bb6c33890371a2cdb02bc6ccc82a5df343b823b5
2026-09-16 15:26:30 +00:00
jif 3fa9039bd7 Label rollout compression failures by stage and I/O error kind (#45461)
## Why

Rollout compression failure counters report only that an operation failed,
without identifying the failing stage or I/O error kind.

## What changed

- Add `stage` and `error_kind` labels to failure counters for compression runs,
  individual files, materialization for append, and stale temporary file cleanup.
- Record failures at their source, including lock acquisition and task joins,
  and avoid counting file compression failures twice.
- Use static stage labels and a fixed set of error categories, keeping error
  messages, paths, and rollout contents out of metric tags.

GitOrigin-RevId: ac2bfc7ae4cec4e9f60de9557b03345a3986aa53
2026-09-14 16:37:47 +00:00
Owen Lin 3052bbcf8c Remove the deprecated thread/rollback API (#44915)
## What changed

Remove `thread/rollback`, its request and response types, generated bindings, and the core `Op::ThreadRollback` operation. Requests now follow the generic unknown-method rejection path. Document `thread/revert` as the alternative for paginated threads.

Keep historical `ThreadRolledBack` markers and legacy error deserialization so existing rollouts remain compatible with replay and migration.

## Testing

Adapt retained-context and Guardian history tests to append legacy rollback markers and resume threads, preserving coverage of surviving instructions, answers, and review history.

GitOrigin-RevId: b3da1becdf86b1869275aacb0ffc2817cee5af2e
2026-09-11 21:32:04 +00:00
Owen Lin 196964ef10 Preserve root turn attribution in turn-start events (#44611)
## Why

Persisted turn-start events lack the originating root turn ID, so they cannot directly associate child turns with the root turn that initiated them.

## What changed

- Add optional `root_turn_id` to `TurnStartedEvent`. Use the inherited root turn ID when available, otherwise the turn's own ID.
- Centralize turn-start emission for regular turns, shell commands, and manual compaction so they record attribution consistently.
- Carry attribution into `ThreadHistoryTurnChange` when processing turn-start events and retain it through completion in the history builder.
- Keep older records compatible by defaulting missing attribution to `None`.

## Testing

Add coverage for root and child turn attribution, persistence when tool collisions fail a turn before sampling, history change propagation, and deserialization of older turn-start records.

GitOrigin-RevId: 54a80b828ca7d1c4c19f13d468c01d052e1b9130
2026-09-10 18:24:13 +00:00
Matthew Zeng b5544d5732 Persist disabled plugin IDs in thread settings (#44332)
## What changed

- Add `disabled_plugin_ids` to thread startup options, settings overrides, snapshots, and persisted turn contexts. Omitted overrides preserve the current list; an empty list clears it.
- Restore the selection on resume and capture it before truncating history for a fork. Prefer the owning thread's latest settings snapshot, falling back only to the latest turn context so stale history cannot resurrect an older selection.
- Freeze the selection at turn admission and retain compatibility with histories that lack the new fields.

## Testing

Add coverage for owned settings, explicit clears, latest-turn fallback, and legacy snapshots. Extend settings tests to cover plugin selection during runtime restoration and compaction, and verify initial turn persistence without extra settings checkpoints in both history modes.

GitOrigin-RevId: 75ffbff5763cd93cc5ac69d73b08b237f59bc009
2026-09-09 23:23:59 +00:00
jif a2e83a783e Continue rollout searches when a compressed rollout cannot be searched (#44226)
## Why

An error searching a single compressed rollout could abort the entire search and prevent results from other rollouts from being returned.

## What changed

Log a warning with the rollout path and error, skip that compressed rollout, and continue searching. Apply this behavior to both the compressed scan used alongside `ripgrep` and the fallback scan used when `ripgrep` is unavailable.

GitOrigin-RevId: 07a0d7109456a374244bb84f32cf18e60a69c206
2026-09-09 17:31:15 +00:00
jif 0df1daf526 Attach compressed rollouts to diagnostic reports as JSONL (#44175)
## Why

Diagnostic attachment reads assume the queued file path still exists and contains plain bytes. Compressed rollouts can therefore be omitted when only the logical `.jsonl` path is available, or attached as compressed data when a `.jsonl.zst` path is supplied.

## What changed

- Read rollout attachments through a bounded decoder that resolves plain or compressed representations without materializing a durable JSONL file.
- Use canonical `.jsonl` filenames for attachments and app-server report metadata, while preserving filename overrides.
- Apply size limits to decoded bytes and preserve JSONL prefix truncation.

## Testing

Add regression tests for compressed attachments, representation changes after queuing, plain-sibling preference, filename overrides, decoded size limits, truncation, nonregular files, and unrelated `.zst` attachments.

GitOrigin-RevId: b30f7dd08a741b0c99283460a1ce8933d2920ddf
2026-09-09 15:12:19 +00:00
jif 73a1148c9c Coordinate rollout compression with active thread writers (#44138)
## Why

Cold rollout files can still have active writers. Compression must not replace their files while writes are pending, including background work that outlives the local thread store.

## What changed

- Share cross-process writer locks between the rollout compressor and local thread store. Skip busy threads and recheck file state under coordination before publishing compressed files.
- Retain writer ownership through recorder background I/O and compressed rollout materialization. Wait for writers to exit on shutdown or discard, and stop queued file work before deletion.
- Route rollout metadata patches through owned recorders, reject competing writers before updating SQLite, and serialize metadata appends with shutdown.

## Testing

Add regression coverage for compression with live and detached writers, parallel cold-file compression, publication locking, metadata ownership conflicts, updates to compressed rollouts, concurrent shutdown, and discarding deferred items.

GitOrigin-RevId: c52f4037eb2019e8095b554e29bf531af95aa4bd
2026-09-09 12:30:07 +00:00
rhan-oai 6515a72db7 Preserve runtime workspace roots across thread resume (#43848)
## Why

Resuming a thread should retain its selected workspace folders, including additional roots and explicit empty selections. Resume overrides also need to survive a subsequent resume when no turn has run.

## What changed

- Persist `runtime_workspace_roots` in startup metadata and thread settings snapshots, separately from explicit environment selections and permission-profile roots.
- Restore roots from the latest snapshot owned by the resumed thread, falling back to owned startup metadata only when no snapshot exists. Honor explicit `runtimeWorkspaceRoots` overrides, retarget the old `cwd` root when `cwd` changes, deduplicate roots, and validate restored paths for the current host.
- Checkpoint effective settings on resume and restored settings after revert. Reload resume configuration if saved workspace roots change during loading.
- Normalize Windows rollout path spellings when matching thread search results, preserving selection of the correct rollout after revert, including compressed rollouts.

## Testing

Add regression coverage for workspace restoration, empty and explicit overrides, foreign paths, compaction and revert, resume checkpoints without recency changes, concurrent settings persistence, and rollout search path matching.

GitOrigin-RevId: d98d9d34dd63934d441120916c61c12b69e7f062
2026-09-08 16:24:38 +00:00
Eric Traut d0a8dcd157 Limit archive rollout reads to requested threads (#43494)
## Why

Archiving threads reads metadata from every active rollout, including unrelated compressed files.

## What changed

Add `RolloutReferenceIndex::scan_unarchived_threads` and use it when archiving. Filter canonical filenames by the requested thread IDs before reading contents, while retaining metadata-based ownership checks.

## Testing

Add tests covering reverted rollouts, requested descendants, compressed rollouts, exclusion of archived and unrelated files, and candidate ownership checks.

GitOrigin-RevId: a64f7fa25e5493ed20312f1d8216422752272bae
2026-09-07 15:19:21 +00:00
felixxia-oai 16ff14c266 Retain inherited Guardian instructions in standalone forks (#43478)
## Why

A standalone root forked from a worker needs its inherited user instructions for Guardian review, including after compaction. Parent input counters belong to a different thread, and omitted parent answers may have restricted an inherited authorization.

## What changed

- Adopt inherited user instructions into thread-owned retained context for standalone roots, recovering surviving checkpoint messages when needed.
- Keep inherited instructions before local evidence without sharing local acceptance counters, and preserve that distinction during reconciliation and rollback.
- Mark parent verified answers as incomplete and treat recovered checkpoint instructions as incomplete excerpts.

## Testing

Add coverage for standalone forks from copied history and truncated checkpoints, retention through compaction and resume, inherited/local ordering collisions, and rollback scope.

GitOrigin-RevId: bf716e224a1d29d6c528ef0dd7cdb118a413a08c
2026-09-07 13:58:57 +00:00
faizan-oai 3921a30d6b Persist Daybreak preferences in thread metadata (#42854)
## Why

Clients need a per-thread Daybreak toggle that survives restarts and can be
restored on resume without changing the access program of an active or future
turn.

## What changed

- Add experimental `daybreakEnabled` support to `thread/metadata/update` and
  expose the saved value on thread responses.
- Store the preference in SQLite, including for threads without a first turn,
  and preserve it through reads, resumes, reconciliation, and restarts.
- Copy the preference when forking a thread while keeping subsequent updates
  independent.
- Keep `daybreakEnabled` separate from the per-turn `cyberAccessProgram` field;
  app-server does not derive one from the other.

## Testing

- Cover persistence across restarts, cold reads, reconciliation, and forks.
- Verify metadata updates do not alter the access program of active or later
  turns.

GitOrigin-RevId: bcf987a9fcd7b0ac8a88b1c8f0df06b33ed8ad6e
2026-09-04 21:17:25 +00:00
felixxia-oai 80d7ca34bc Retain user instructions in guardian thread context (#42762)
## Why

Guardian review needs the original user instructions alongside verified answers, even after compaction, resume, or rollback. A summarized or legacy checkpoint cannot establish a complete instruction history.

## What changed

- Capture text user messages in retained context when `guardian_thread_context` is enabled, preserving their message identity through compaction and replay.
- Keep retained instructions and verified answers in arrival order, with independent size and completeness tracking for each family.
- Roll back retained evidence at the exact user-message boundary, including steers that share a turn ID, and apply the same semantics during rollout migration.
- Treat checkpoints without retained user messages as incomplete while preserving their verified answers.

## Testing

- Cover capture, compaction, resume, rollback, child forks, legacy checkpoints, storage bounds, and rollout migration.

GitOrigin-RevId: 72706731b54e5ac8645db61b446d2401afb42235
2026-09-04 13:24:42 +00:00
Can Sar 728cb12fe5 Expose thread originators through the app-server API (#42458)
## What changed

- Add the creation-time `originator` to app-server thread responses and `thread/started` notifications.
- Persist originators in thread metadata and preserve the first recorded value across list, read, resume, rollout, and SQLite paths.
- Add an `originators` allowlist to `thread/list` for hosted backends. The local app-server accepts omitted or empty values but rejects nonempty filters.

## Testing

- Cover originator persistence across server restarts and list, read, and resume requests.
- Verify local `thread/list` handling for nonempty, omitted, null, and empty originator filters.
- Heap-allocate large TUI dispatcher futures in affected tests to keep them within the Windows test-thread stack.

GitOrigin-RevId: 73ad6e61b8fdd2d93a22d1f971723a19ec7be3fc
2026-09-03 06:02:25 +00:00
Owen Lin 69cebb5d15 Route rollout reads through the canonical JSON decoder (#42378)
## Why

Directly deserializing the flattened `RolloutLine` envelope can reject nested
decimal values, preventing affected paginated sessions from resuming.

## What changed

- Add canonical string, byte, and reverse-scanner helpers that decode rollout
  records through `serde_json::Value` before decoding the flattened item.
- Route rollout readers across session discovery, history, migration, search,
  thread storage, and transcript previews through those helpers.
- Remove `Deserialize` from `RolloutLine` so new readers cannot bypass the
  canonical persistence decoder.

## Testing

Add coverage that resumes a paginated rollout after a token-count record with
a decimal rate-limit value and verifies that ordinal sequencing continues.

GitOrigin-RevId: 49abac1e0751c073daa5a93a840d8a483fd2d013
2026-09-02 22:45:19 +00:00
felixxia-oai 0d502a4230 Support durable reasoning configuration updates (#42328)
## Why

Reasoning configuration changes need to retain their position and trusted provenance when model history is persisted and replayed. Client-injected history must not be able to forge these controls.

## What changed

- Add a typed `configuration_update` response item carrying reasoning effort, including custom model-defined values.
- Persist harness-authored updates with provenance and preserve them across history reconstruction, thread resume, raw response notifications, and agent forks.
- Exclude untrusted configuration updates from model history, strip client-supplied provenance metadata, and reject configuration updates supplied as turn input.
- Export the new item through the JSON and TypeScript app-server schemas and classify it in telemetry and persistence metrics.

## Testing

- Cover serialization, provenance persistence, history filtering and rollback, resume reconstruction, and injection attempts before and after restart.

GitOrigin-RevId: eb5559d2b52b7a931621e7c9812f009ff9fb8939
2026-09-02 18:52:44 +00:00
felixxia-oai 5971d42847 Preserve verified answers across history compaction (#42293)
## Why

Compaction replaces the model history window, but host-verified user answers may
still be needed for the rest of the owning thread.

## What changed

- Add bounded, model-invisible retained context for verified
  `request_user_input` answers.
- Persist retained context in rollout events and compaction checkpoints, and
  restore it when a thread resumes.
- Remove answers when their source turns are rolled back, and prevent
  parent-local answers from being inherited by child forks.

## Testing

Add lifecycle coverage for deduplication and size limits, compaction and resume,
rollback, rollout migration, and child forks.

GitOrigin-RevId: c32900d3e8120a9899b71f0576683acd67a4b898
2026-09-02 15:07:11 +00:00
rka-oai 2c79ee6dac Add structured asynchronous user input requests (#42178)
## What changed

- Replace `send_user_message_async` with `request_user_input_async`, accepting one or more questions with optional suggested answers while allowing the turn to continue.
- Attach structured question metadata to asynchronous agent messages while retaining readable fallback text, and preserve it through app-server events, thread history, and generated schemas.
- Continue enabling the new tool for model catalogs that advertise either the old or new tool name.

## Testing

- Cover tool registration, question validation, emitted items, continued turn execution, history preservation, and backwards-compatible deserialization.

GitOrigin-RevId: ffc49aebde2e854c3c50a122aa52805fec3fc6b3
2026-09-02 02:09:35 +00:00
Brent Traut 986ff1cc7c Expose model settings in app-server thread metadata (#42151)
## What changed

- Add nullable `model` and `reasoningEffort` fields to the shared app-server `Thread` object and generated schemas.
- Report current settings for loaded threads and the latest persisted settings for unloaded threads across thread read, list, start, resume, rollback, metadata update, and notification paths.
- Keep unavailable legacy or filesystem-only settings nullable, and allow metadata reads without loading a thread or dispatching queued work.

## Testing

- Cover live-setting precedence, persisted settings after restart, null legacy values, and model metadata returned by thread lifecycle operations.

GitOrigin-RevId: 22ebe8a8d1910a820d90a97730d812a00d8c1372
2026-09-01 22:56:18 +00:00
jif 28097e98eb Preserve Guardian history across thread reconstruction (#42065)
## Why

Guardian review evidence must remain available after a compacted thread is resumed or forked, while rolled-back or parent-local evidence must not grant authorization in a different context.

## What changed

- Store the bounded, model-invisible Guardian transcript alongside compacted rollout history and restore it from the newest surviving checkpoint during replay.
- Trim Guardian history at rollback boundaries, clearing it when the boundary has already been evicted.
- Preserve the checkpoint for user-initiated forks, but remove it when spawning a subagent so parent review evidence is not inherited.
- Keep the new rollout field optional for compatibility with existing compacted records and legacy readers.

## Testing

- Cover Guardian history across compaction, restart, paginated and pathless stores, user forks, rollback, bounded replay, serialization, and subagent forks.

GitOrigin-RevId: 6ab076d5e0c3aac4d6a41ebd50ba2b200c90213e
2026-09-01 14:33:58 +00:00
jif 9d0eae74cd Include shared histories in rollout compression (#42039)
## What changed

- Make `local_thread_store_compression` compress cold rollout files across shared and forked histories without a separate compression mode.
- Retire `local_thread_store_shared_compression` while continuing to accept it in strict configuration without changing compression behavior.
- Read rollout files through the compressed-rollout reader when `codex exec resume` determines the latest turn's working directory.

## Testing

- Cover compression and restoration of an archived fork chain.
- Cover cwd-based resume selection when only the compressed rollout remains.
- Cover the removed compatibility key in strict configuration.

GitOrigin-RevId: 8edebf61eccc1f79cabb8e641f8982c76f337bf1
2026-09-01 12:23:20 +00:00
Owen Lin 5f79a92e39 Persist response token usage in rollout history (#41912)
## Why

Resumed threads need to continue per-turn and per-thread token totals without scanning arbitrarily far beyond the latest compaction checkpoint.

## What changed

- Add durable `TokenUsageRecord` rollout items with response, turn, thread, session, and root-turn attribution.
- Restore accumulated usage on resume and snapshot the latest record plus the compaction response ID in compaction checkpoints.
- Preserve root-turn lineage in persisted turn context, while ensuring forked child threads start with their own usage totals.

## Testing

- Cover usage accumulation across multiple responses and resumed turns.
- Cover local and remote compaction checkpoints, invalid remote compaction output, rollout reconstruction, and fork isolation.

GitOrigin-RevId: ef9e0c4a9102a08a2c382be4cdac68c84353c90a
2026-08-31 21:15:38 +00:00
Eric Traut 2f0a5d5516 Avoid scanning archived rollouts when archiving threads (#41908)
## Why

Archiving a thread only needs to locate rollout files that have not already
been moved, so reading the entire rollout archive on every request is
unnecessary.

## What changed

- Add `RolloutReferenceIndex::scan_unarchived` to scan only the active
  `sessions` directory.
- Use the active-only index when archiving threads while retaining full scans
  for operations that require archived-history reference counts.
- Cover discovery of compressed and uncompressed active rollouts and exclusion
  of archived and unrelated rollouts.

GitOrigin-RevId: 46367127578f20bfa46a8db5c6f2b7f59d29f7a9
2026-08-31 20:40:05 +00:00
Bryan Ashley f88ff940c0 Load bounded context after empty wake turns (#41901)
## Why

Paginated threads can accumulate wake turns with no user-message boundary. The
reverse context scan could not use those turns as a bounded replay cutoff, even
when a compaction and its resume state provided everything needed to reconstruct
the latest model context.

## What changed

Treat a full world-state snapshot that survives the latest compaction as a
durable baseline when it is paired with a compatible turn context. This lets the
scan stop at empty wake turns while still scanning further back for patches,
snapshots before compaction, or missing and mismatched turn contexts.

## Testing

Add coverage for repeated empty wake turns and verify that bounded replay
matches full rollout reconstruction across successive compaction windows.

GitOrigin-RevId: ae90c5f939c660ea539fbfc11128cc7259efaa9c
2026-08-31 19:30:05 +00:00
jif 1cc81ca89a Support compression for shared rollout lineages (#41357)
## Why

Rollout compression skips files that participate in paginated fork lineages because lineage readers rely on byte offsets into the original JSONL. This prevents shared rollout history from being compressed.

## What changed

- Add a seekable rollout reader that preserves logical JSONL offsets for plain and zstd-compressed files without materializing compressed data beside the rollout.
- Use the reader for lineage validation, model-context replay, and SQLite history projection, including when compression races with a read.
- Add the opt-in `local_thread_store_shared_compression` feature. The default mode continues to leave shared lineages uncompressed for compatibility, while the new mode includes referenced sources and fork pointers.
- Record the uncompressed size in new zstd frames so common prefix-bound checks can avoid decoding the full file.

## Testing

Add coverage for seekable compressed reads, prefix validation across zstd frame shapes, compressed lineage replay and projection, concurrent forks, and compression-mode behavior.

GitOrigin-RevId: 83e519f7eabf7448977eff37c55899858cc17a07
2026-08-28 16:04:26 +00:00