Commit Graph
242 Commits
Author SHA1 Message Date
ben zhang 9d3ae38e13 Limit unarchive scans to the requested thread's archived rollouts (#54060)
## Why

Unarchiving a thread previously read rollout metadata across both active and archived history, including unrelated threads.

## What changed

Use `RolloutReferenceIndex::scan_archived_threads` to restrict discovery to archived files whose canonical filenames match the requested thread. Continue validating ownership through session metadata and include compressed and reverted rollouts.

## Testing

Add coverage for compressed and replacement archived rollouts, metadata ownership mismatches, unrelated filenames, and exclusion of active rollouts.

GitOrigin-RevId: 8784f3502b839dcb4a3839744010e350d0196545
2026-10-14 18:19:02 +00:00
bbrown-oai c6327eba6a Filter rollout filenames before checking compressed siblings (#53997)
## Why

Filesystem lookups by thread or rollout ID check whether unrelated compressed rollouts have an uncompressed sibling. These checks are unnecessary when the filename does not match the requested ID.

## What changed

Apply the thread or rollout ID filter before resolving plain/compressed file precedence. Preserve preference for uncompressed files and selection of the newest rollout for a thread.

## Testing

Extend lookup tests to cover unrelated compressed files, uncompressed-file precedence, missing rollout IDs, and rollout ID tie-breaking for files with the same timestamp.

GitOrigin-RevId: 5bdb56ebec8031fceb5f6adbfcf644ad010e52f2
2026-10-14 14:35:58 +00:00
jif 78dc250384 Stop rollout reads when the calling task is cancelled (#53947)
## Why

Blocking rollout reads can continue decompressing and scanning history after the calling async task is dropped.

## What changed

Propagate caller cancellation through rollout readers, prefix validation, model-context scans, history materialization, and working-directory lookup. Check cancellation during compressed input and output, bounded reads, seeks, and between history records. Return an error on cancellation so partial history cannot be mistaken for a complete read.

## Testing

Add regression coverage for dropping a caller during decompression, cancelling compressed input and output (including skippable frames), cancelling reads and seeks on plain and decoded snapshots, and cancelling history materialization between records.

GitOrigin-RevId: e3dac952789b53402f2a5f17c2a2232506feb9d6
2026-10-14 10:46:14 +00:00
Dan Gillespie efa159f7aa Reuse compiled search patterns across compressed rollouts (#53732)
## Why

Compressed rollout searches recompiled the same query patterns for every file.

## What changed

Lazily compile and reuse the JSON-escaped and content search patterns across each scan, including the fallback scan when `ripgrep` is unavailable. Keep compilation failures within the existing per-file error handling and open files before compiling patterns.

## Testing

Add regression coverage verifying that an oversized query does not fail the compressed rollout scan and that a missing file still returns `NotFound` before query compilation.

GitOrigin-RevId: 0d41cbca6e1f716fc234c268493d8dd94c34975e
2026-10-13 17:54:31 +00:00
pakrym-oai 5ef96ab278 Add opt-in output token replay for OpenAI requests (#52742)
## What changed

- Add the disabled-by-default `output_token_replay` feature to request `output.encrypted_content` from OpenAI providers.
- Preserve encrypted message and tool-call output, message and function-call status, and output-text `annotations` and `logprobs` through history and replay. Update the generated protocol schemas and SDK types.
- Strip output ciphertext from requests to other providers while retaining it in stored history, including after replay is disabled. Remove encrypted content from memory extraction inputs.

## Testing

Add integration coverage for the feature/provider combinations, metadata replay across follow-up requests, and history preservation when resuming with replay disabled.

GitOrigin-RevId: 77729a021b6771933402aebe74ece8eb39c057e2
2026-10-10 01:25:00 +00:00
Nick Steele a6baf8867c Signal abandonment of unanswered MCP elicitations (#51611)
## Why

Published MCP elicitation requests could lose their response waiter without notifying the event consumer that the request was no longer answerable.

## What changed

- Add `ElicitationAbandoned` with the original server name and response token for requests published through unbounded event channels.
- Emit abandonment exactly once when a waiter is dropped, response delivery fails, or the MCP runtime shuts down.
- Close the elicitation router during shutdown, reject new requests on unbounded sources, and wait for terminal deliveries before shutdown completes. Preserve bounded sources' request-only behavior and backpressure.

## Testing

Add router tests for ordered, exactly-once abandonment, closed receivers, failed response delivery, shutdown coordination, and bounded-channel backpressure. Add a core integration scenario verifying that a pending form's abandonment precedes `ShutdownComplete`.

GitOrigin-RevId: 53fe9f4a8b32f7324be180288aa492f31f4d2057
2026-10-07 07:57:14 +00:00
Brent Traut 406eb53c07 Distinguish thread listing failures from exhausted history (#51602)
## Why

DB-only listing and pagination failures must not look like exhausted history. This hardens the DB-only listing added in https://github.com/openai/codex/pull/18502 and page refilling added in https://github.com/openai/codex/pull/7509.

## What changed

- Return JSON-RPC error `-32603` when `useStateDbOnly: true` cannot query the local state database, including requests with an empty `cwd` array.
- Track all cursors encountered while filling a page and return `-32603` for repeated cursors, including on full pages.
- Document that callers can retry these errors; a successful response with `nextCursor: null` still indicates exhaustion. Default scan-and-repair requests retain their filesystem fallback.

## Testing

Add regression tests for missing or mismatched state databases versus healthy empty history, with active and archived listings and null or empty `cwd` filters. Cover repeated cursors during page refilling and on full pages using the in-memory thread store.

GitOrigin-RevId: 09653a84fc65f47316d8bbae9e4cf3dd8ce9948a
2026-10-07 06:34:22 +00:00
Charlie Marsh a9bc7bebfa Load rollout history on a single blocking worker (#51499)
## What changed

Move full rollout history reading and parsing into one blocking worker for both plain `.jsonl` and compressed `.jsonl.zst` files. Use the shared cancellation-aware line iterator so canceled loads stop before processing another record, while preserving rollout items, thread IDs, and parse-error counts.

## Testing

Add regression tests for cancellation between records and equivalent history-loading results across plain and compressed files, including Unicode, CRLF line endings, malformed records, empty files, and invalid UTF-8.

GitOrigin-RevId: 65e1bd7ebcdbbd0ab5f560e73b2c7f074f627f8f
2026-10-06 23:04:02 +00:00
pakrym-oai 28cd3e2501 Remove obsolete fields from persisted turn context (#51492)
## What changed

Remove `workspace_roots`, `current_date`, `timezone`, `network`, `personality`, and `multi_agent_mode` from `TurnContextItem` and stop recording them in rollouts. Update the precomputed protocol schema accordingly.

Make `summary` optional when reading turn context and write `"none"` as a compatibility placeholder for older readers, rather than persisting the active reasoning-summary setting.

## Testing

Update protocol tests to cover omitted `summary` fields and compatibility-placeholder serialization. Extend compaction tests to verify that model requests retain the updated reasoning-summary setting after compaction while the rollout checkpoint records the placeholder.

GitOrigin-RevId: d49eabaa1b340be1d79f7f09115281a307ea8a64
2026-10-06 22:39:06 +00:00
Eddie Chen cfc946f4e1 Expose and persist turn lineage across the app server (#51415)
## Why

Clients starting delegated work need to attribute descendant turns to the original turn and retain that attribution through completion and history reads.

## What changed

- Accept optional `parentTurnId` and `rootTurnId` in `turn/start`, and expose `rootTurnId` on returned turns and lifecycle notifications.
- Default new turns to their own root when none is supplied. Preserve the active turn's root when steering, ignoring incoming lineage fields.
- Carry root IDs through completion, interruption, recovery, and fork boundaries, and persist them for paginated history reads. Leave unknown roots null in older history.
- Update protocol schemas and generated TypeScript and Python bindings.

## Testing

Extend coverage for default and explicit roots, steering without changing lineage, interrupted and recovered turns, history reads after restart in both history modes, nested fork pagination, and compatibility with older database writers.

GitOrigin-RevId: f1a570a12bd11742929fd2e8b017e5d35aa07dcb
2026-10-06 16:56:39 +00:00
Eddie Chen 551bd409eb Preserve turn attribution across recovery and compaction (#51402)
## Why

Recovering an interrupted turn replaced its original trigger with `retry` and restored only its root lineage from model context. Turns suspended before that context was written could lose their attribution entirely.

## What changed

- Persist regular-turn attribution in `TurnStartedEvent` and compaction resume metadata, including the trigger, parent and root turn IDs, and initiating agent path.
- Restore matching attribution during recovery, keep unknown triggers absent, and retain the model-context root fallback for older records.
- Reconstruct attribution from surviving turns across rollback and late completion or abort events, and clear inherited attribution when spawning a forked agent.

## Testing

Add coverage for repeated cold recovery, suspension before context persistence, legacy rollouts, fresh attribution on subsequent turns, compaction checkpoints, rollback, and late terminal events.

GitOrigin-RevId: 85b5763e8fbd48a1ea2c07cfda8196e63c2cd33a
2026-10-06 15:50:27 +00:00
Eric Traut 80e0b51c9e Make session lookup pagination stable and report listing failures (#51230)
## Why

Session activity can move unread threads ahead of a pagination cursor, and timestamp-only cursors can skip threads with equal creation times. Either can prevent a session label lookup from detecting duplicate labels. Database failures must also remain distinguishable from an exhausted listing.

## What changed

- Resolve session labels using database-only listings ordered by `CreatedAt`, with thread IDs breaking timestamp ties. Accept unique matches across pages with valid timestamp-and-ID cursors, while retaining ambiguity errors for older cursors that cannot establish uniqueness.
- Preserve timestamp index ranges for database continuation queries and timestamp-only cursors for listings that may fall back to the filesystem.
- Propagate database-only listing and name-resolution failures. Keep best-effort name enrichment for ordinary listings and retry latest-session lookup with scan-and-repair after a database-only error.
- Keep loaded sessions selectable in the resume and fork pickers when a later page fails, stop further pagination, and display `Could not load more sessions`.

## Testing

Add regression coverage for creation-time continuation after session activity, cursor query plans, filesystem fallback, database and name-query errors, cross-page label uniqueness, older server cursors, and selection after picker pagination failures.

GitOrigin-RevId: 264ab2e69b8ad12d9edaa31fd253cb9805a46a90
2026-10-06 02:14:22 +00:00
Owen Lin 820f85cf59 Truncate oversized MCP results in paginated thread history (#50458)
## Why

Paginated thread history can persist large MCP results in completed tool-call items. Bound these stored results to avoid retaining multi-megabyte payloads.

## What changed

Apply a 64 KiB preview budget when persisting oversized MCP results in paginated history. Replace the result with a truncated text preview of its serialized JSON, preserving `is_error` and clearing `structuredContent` and `_meta`. Leave small results unchanged.

Share the truncation helper with the existing MCP tool-call event path.

## Testing

Add helper tests for unchanged small results and oversized results with structured content and metadata. Add a thread-history test verifying that persisted MCP results contain a bounded preview with the beginning, end, and truncation marker while preserving tool arguments.

GitOrigin-RevId: 09dce73c70bfd357c9b9630f456e9e5c1d8492cf
2026-10-02 23:15:02 +00:00
Owen Lin 3c3a990da0 Measure rollout persistence size reductions (#50454)
## What changed

- Record original and persisted item sizes in `codex.rollout.persistence.item_bytes_v2`, distinguished by stage, and record positive size reductions in `codex.rollout.persistence.bytes_removed`. Count the full original size as removed for dropped items.
- Use explicit histogram boundaries from 1 KiB to 64 MiB for item sizes, bytes removed, and `codex.rollout.persistence.turn_bytes_v2`. Larger values remain represented in the overflow bucket and histogram sum.
- Align item and turn size stages and report a measurement error when a kept item's persisted size cannot be measured.

GitOrigin-RevId: 6487acf1c6f2c041ab9680ab398491d3c1541388
2026-10-02 23:06:56 +00:00
pakrym-oai 602d2add6e Persist additional tool definitions in rollout history (#50435)
## Why

`additional_tools` response items were excluded from rollout persistence, dropping their tool definitions from saved history.

## What changed

- Persist `additional_tools` items in both legacy and paginated history modes.
- Expose the variant in generated JSON schemas, TypeScript types, and Python models, with an optional `id`, a `role`, and arbitrary JSON tool definitions.

## Testing

Add a recorder round-trip test that verifies tool definitions survive recording and loading in both history modes without parse errors.

GitOrigin-RevId: 0e7c7b1c10401ec6513fb744de642a1cf9e461d7
2026-10-02 21:42:45 +00:00
Owen Lin bee28e8a06 Cap persisted command output in paginated history at 64 KiB (#50427)
## Why

Paginated history previously persisted completed commands' full `aggregated_output`, allowing large command outputs to inflate stored history.

## What changed

- Limit persisted command `aggregated_output` to 64 KiB, including a truncation marker. Preserve the beginning and end of the output without splitting UTF-8 characters.
- Apply the shared persistence policy to rollout recording, thread-store appends, and legacy-to-paginated history migration.
- Measure persisted payload bytes after truncation so persistence metrics reflect the stored representation.

## Testing

Add coverage for the byte limit, UTF-8 boundaries, unchanged inputs that fit, and markers that exceed the budget. Verify capped output in paginated thread history and migrated legacy history, and verify persistence metrics use the truncated payload size.

GitOrigin-RevId: 01d420d396ad34b42dc5dc208c0bf88d890639ee
2026-10-02 21:00:06 +00:00
jif d6c3b448a4 Test session index thread-name append and removal (#49959)
## What changed

Add a test that appends an original and updated name for one thread alongside another thread's entry. Verify that lookup returns the updated name and that removing the first thread's name entries leaves only the other thread's entry.

GitOrigin-RevId: cedf260a639f0c0e76aeda0d446559c3821bd911
2026-10-01 11:09:12 +00:00
Charlie Marsh 4f699cd642 Move session index I/O off async runtime threads (#49708)
## Why

Session index updates perform synchronous file I/O and wait on a blocking mutex inside async functions, blocking runtime threads.

## What changed

Run appends and thread-name removals in `spawn_blocking` workers. Acquire a Tokio mutex before dispatch and keep its owned guard in the worker so queued and running updates remain serialized even if the awaiting task is cancelled. Move the lookup's file-existence check into its blocking worker as well.

## Testing

Add tests for lock retention after cancellation of running and queued updates, write ordering after cancellation, and removal that preserves other thread names and malformed lines.

GitOrigin-RevId: 368745d8e5bbf24c527facc0a62ddd0fb76f0ffb
2026-09-30 18:51:29 +00:00
Charlie Marsh 5aa92804d2 Batch rollout listing scans on cancellable blocking workers (#49694)
## What changed

Move directory enumeration and candidate metadata lookups into blocking workers, batching updated-time candidate collection into one worker per scan. Check cancellation between entries and metadata lookups so dropping the async waiter stops further scanning.

Share synchronous rollout metadata resolution with the async helper, preserving plain-file precedence over compressed siblings. Retain scan limits, stable file ordering, and directory-level early stopping for creation-time listings.

## Testing

Add tests for cancellation during directory enumeration and candidate discovery in both flat and date-partitioned layouts, including plain-sibling metadata precedence and filtering of unrelated files, directories, and symlinks.

GitOrigin-RevId: 324289003b29a7d2b3220da03e3c002ab5e23eec
2026-09-30 18:03:48 +00:00
npancha-openai d1c4e3c3e6 Keep compressed rollout snippet searches on one blocking worker (#49692)
## Why

Compressed snippet searches schedule a blocking task for every line read. Keeping the scan on one worker avoids repeated task scheduling.

## What changed

Use `RolloutLineReader::find_map` to scan compressed rollouts on a single blocking worker, stopping at the first matching snippet. Preserve read metrics and stop delivering records when the search is canceled, including cancellation during a read.

## Testing

Add tests for first-match snippets, case-insensitive literal matching, malformed records, missing matches, stream errors before a match, and cancellation between lines or during a read.

GitOrigin-RevId: 295206b1a00e861bc44606bd843f6ed8c3e6aae1
2026-09-30 17:57:58 +00:00
felixxia-oai 92bc601ad6 Reuse unchanged history snapshots when resuming threads (#49600)
## Why

Resuming a thread rereads persisted history even when a preloaded snapshot is unchanged. Reusing that snapshot must still account for writes committed before writer ownership is acquired.

## What changed

- Carry optional, non-serialized revisions from stored history and model context through thread and agent resume paths.
- Validate the revision under writer ownership and reuse the shared snapshot when it matches. Reload stored history when the revision is missing or changed, while preserving explicit history overrides.
- Resolve the selected rollout before loading context, validate compressed snapshots before materialization, and enforce archived-thread restrictions on canonical paths.

## Testing

Extend coverage for snapshot reuse and late-write refresh in legacy and paginated histories, compressed rollout reuse, archived rollout symlinks, and cold child restoration that includes edits made after preloading history.

GitOrigin-RevId: e1fd7aee6ef2b30f41951e6b7e6b01c3e1420cab
2026-09-30 11:59:13 +00:00
Brent Traut 8c3612fb63 Use memrchr to find newlines in reverse JSONL scans (#49444)
Replace the byte-by-byte reverse newline search with `memchr::memrchr` and add `memchr` as a dependency of `codex-rollout`.

GitOrigin-RevId: b07e42e15efb0d63d7b00ee9a2ee29c446f4cf59
2026-09-30 02:19:47 +00:00
jif d5e6526362 Scan the session index backwards for batch thread name lookups (#49297)
## Why

Batch thread name lookups read the entire append-only session index even when all requested names could be found near the end.

## What changed

Use `ReverseJsonlScanner` in `spawn_blocking` for `find_thread_names_by_ids`, stopping once every requested ID has a usable name. Preserve the latest nonempty, trimmed name for each ID while skipping malformed entries.

## Testing

Add regression coverage for renamed threads, whitespace-only names, Unicode, reads across chunk boundaries, malformed entries, a partial final write, and missing IDs.

GitOrigin-RevId: 2ab9254ba8b718fe3e02c47fc9b0de20c09fd10f
2026-09-29 16:16:39 +00:00
Eric Traut fbc169827e Suppress Windows console windows for background subprocesses (#49164)
## Why

Background helpers launched from a detached Windows process can allocate console windows. Job Object launches and containment fallbacks need to preserve console suppression as well.

## What changed

- Add `codex_utils_process::background_command` to set `CREATE_NO_WINDOW` on Windows, and use it across background subprocess launches, including Git, plugins, authentication, diagnostics, and sandbox helpers.
- Centralize console suppression in Job Object launches. Keep `CREATE_NO_WINDOW` when retrying without containment, and clear `CREATE_SUSPENDED` so the fallback child can run.
- Preserve console inheritance for interactive shell launches and interactive AWS authentication refreshes.

## Testing

Add Windows regression tests covering console-free launches from detached parents through standard and Tokio commands, stdout/stderr capture, successful containment, Job Object creation and assignment failures, and sandbox setup refresh helpers.

GitOrigin-RevId: b2c238fa7d4f9fbd868e5adf7d4be9958ff01e6c
2026-09-29 04:38:37 +00:00
Won Park d7748e1185 Add opt-in structured errors for Guardian circuit-breaker interruptions (#48796)
## Why

Guardian denial-limit interruptions lack a structured error identifying the cause. Make this opt-in because older clients may not recognize the new error in shared history.

## What changed

- Add `auto_review.circuit_break_action = "strict"` to attach `TooManyDenials` to the interrupted turn. The default leaves the error unset.
- Expose `turn.error.codexErrorInfo = "tooManyDenials"` in app-server notifications and saved history, and update protocol schemas and generated types.
- Preserve the warning, denial limit, and interrupted status without emitting a separate error notification.

## Testing

Add an integration test covering omitted, `default`, and `strict` settings, checking warnings, live turn errors, the absence of separate error notifications, and persisted turn errors after restart.

GitOrigin-RevId: cb5d379e64728d4d7b9daf834c13756e4a70db31
2026-09-27 22:05:19 +00:00
acrognale-oai 7596ab19a6 Keep archived threads with empty previews visible in listings (#48199)
## Why

Archived threads without previews were filtered out of thread listings, even though they had valid session metadata.

## What changed

- Include archived threads with empty previews in both SQLite and rollout-file listings.
- Add archive-specific indexes for creation, update, and recency ordering so these listings can use indexed pagination.

## Testing

Add regression coverage for paginated archive listings with and without SQLite, archive query plans that avoid temporary sorting, and metadata reads that preserve source and file timestamps for empty archived threads.

GitOrigin-RevId: e29d0257b2e029bea5fdf94eed6dab1006d2a8f2
2026-09-25 20:11:47 +00:00
jif d6c4c6aea4 Classify rollout read failures by reason and progress (#47565)
## Why

Rollout read metrics record the failure stage and I/O error kind, but these do not distinguish specific zstd failures, task join failures, or a busy reader, nor indicate whether any line was read before failure.

## What changed

Add bounded `reason` and `read_progress` labels to rollout read counters and duration metrics, captured at the first failure. Distinguish OS errors, reader-busy and task-join failures, and known zstd error categories, with `stream_error` as the fallback.

Record whether the failure occurred before or after the first line. Classify blocking-task join failures during both open and read. Keep error messages, file paths, and rollout contents out of metric labels.

GitOrigin-RevId: 6db302900f504bc2860504d81c18017adcef3efc
2026-09-23 15:01:19 +00:00
Owen Lin e7bbc79f48 Resume model context from the latest compaction boundary (#47365)
## Why

Resuming a compacted thread should use the compaction and its newer records without depending on older turn contexts or restoring stale state from before the compaction.

## What changed

- Stop paginated model-context scans at the newest compaction with `replacement_history` and `window_number`. Fall back to full replay if the newest compaction lacks either field.
- Align reconstruction with that boundary and restore `last_started_turn_id`, previous turn settings, and the multi-agent version from compaction resume metadata. Allow newer completed turns to override the saved settings.
- Keep canonical session metadata alongside the bounded replay and use the same scan boundary for subagent rollout migration.

## Testing

Add coverage for cold resume after mid-turn compaction, bounded/full replay equivalence, missing companion state, settings precedence, and incomplete-compaction fallback.

GitOrigin-RevId: f836464bb5cadd9be4a4e05b5d1679223713fc87
2026-09-22 21:17:08 +00:00
Owen Lin 286d4ecf44 Persist resume metadata in compaction checkpoints (#47323)
## Why

Compaction companion records do not represent all resume state, including the last started turn and previous turn settings. The selected multi-agent runtime also needs a persisted value when no companion turn context is present.

## What changed

- Add optional `resume_metadata` to `CompactedItem`, recording the multi-agent version, last started turn ID, and previous turn settings. Preserve the distinction between absent metadata and explicitly empty values while retaining compatibility with older records.
- Capture this metadata under the settings persistence lock alongside the compaction checkpoint.
- Rewrite inherited metadata to use the child agent's runtime, and clear previous turn settings when the child must rebuild context.
- Clear pending continuation state before delivering acceptance of standalone settings updates, while holding the persistence lock.

## Testing

Add coverage for metadata round trips with empty values, legacy reader compatibility, checkpoint companion records, inherited fork metadata, and continuation invalidation while settings event delivery is blocked.

GitOrigin-RevId: f30b7feaa7e2fdb5b65fc3c53b37fd7a7fe27903
2026-09-22 16:35:30 +00:00
Charlie Marsh 4f6c106cdd Use compact display metadata for Guardian review threads (#47302)
## Why

Guardian review threads use synthetic approval prompts as user messages. Deriving thread titles and previews from those prompts exposes large, unhelpful summaries in thread listings.

## What changed

- Default Guardian threads to the title "Guardian review" and preview "Approval review" across metadata creation, rollout reads, and resume.
- Skip synthetic user messages and goal updates when deriving Guardian display metadata, leaving `first_user_message` unset for new threads.
- Preserve custom names, including legacy indexed names, when listing, reading, resuming, and migrating threads to paginated history.

## Testing

Add regression coverage for compact metadata, resume without session metadata, custom names before and after migration, and app-server persistence, listing, and rename/resume behavior.

GitOrigin-RevId: 42aa707db20c3e57ffa5fd4c91ee313710be10d7
2026-09-22 15:02:54 +00:00
Jeremy Rose 23ae65bb06 Persist thread creator identity in rollouts and SQLite (#47113)
## What changed

- Capture optional `creator_user_id` and `creator_account_id` from authentication when creating a thread, including forks.
- Carry these fields through thread storage, rollout metadata, and a SQLite migration. Leave them absent when unavailable or when reading older threads.
- Preserve recorded creator values across resume and metadata updates. On revert, recover missing rollout values from SQLite before writing replacement metadata.

## Testing

Add coverage for ChatGPT and API-key sessions, resume under another account, forks using current authentication, metadata extraction and upserts, and repeated reverts after creator fields are removed from a rollout.

GitOrigin-RevId: 9839fa42f9392f12226002e2c27d05d5656f5695
2026-09-21 21:22:01 +00:00
peilin-openai ac98537678 Track cumulative MCP attribution across requests and thread history (#47081)
## What changed

- Record each MCP source and its first turn, including connector and plugin identifiers when available. Cover direct and Code Mode calls, including returned errors and results that are not printed.
- Persist cumulative attribution checkpoints and restore them across compaction, resume, and forks, including forks limited to recent turns.
- Attach fresh attribution to Responses API `client_metadata` under `mcp_attribution`, outside model-visible content. Restrict transmission to allowed OpenAI HTTPS destinations for both HTTP and WebSocket requests.
- Report `attribution_error` for legacy history without checkpoints, malformed or conflicting attribution, and serialized attribution exceeding 16 KiB.

## Testing

Add coverage for source deduplication, checkpoint persistence and restoration, recent-turn forks, direct and Code Mode calls, destination filtering, and the serialized size limit.

GitOrigin-RevId: fd285e152ca9e24bfa000debb20013709d67a86e
2026-09-21 18:19:07 +00:00
jif 23a6b705ba Add diagnostics for rollout compression metadata failures (#47026)
## What changed

- Tag metadata scan failures with bounded reasons for invalid history modes, responses before metadata, missing metadata, or I/O errors.
- Emit at most five metadata warnings per compression run with the rollout ID, error category, file size, and modification age, excluding paths and error messages. Include the total metadata failure count in the completion log.
- Preserve underlying metadata errors while redacting their details from `MetadataReadError` debug output.

GitOrigin-RevId: 13f8ec265405b8f8bb23a02ee37c1aabdce806eb
2026-09-21 13:18:50 +00:00
3d5b66c655 Handle unknown error classifications and configure gateway OAuth (#46482)
Preserve known error classifications, including BioPolicy, while falling back to Other for unknown values so saved sessions can still be read.

Add optional gateway OAuth settings to model provider configuration.

Co-authored-by: Owen Lin <owen@openai.com>
Co-authored-by: alexsong-oai <alexsong@openai.com>
GitOrigin-RevId: 64bcf45ca042efca7645c798db3746375426b0fe
2026-09-18 22:29:16 +00:00
jif 821ad43f9d Tag rollout compression metrics by trigger (#46047)
## Why

Compression metrics do not distinguish startup runs from runs requested through
`rollout/compress`, making their outcomes indistinguishable by entry point.

## What changed

Pass `RolloutCompressionTrigger` from startup and RPC callers through the
compression worker. Add a `trigger` tag with `startup` or `rpc` to run, file,
scan-error, and temporary-file cleanup metrics, including durations, byte counts,
and compression ratios.

GitOrigin-RevId: f23d9da791b666a7311f2323f6bead6570067798
2026-09-16 22:33:15 +00:00
jif da18000cae Measure rollout read and materialization durations (#45966)
## What changed

- Emit one read counter and duration observation per rollout reader, tagged by format, outcome, failure stage, and error kind. Distinguish EOF, partial reads, and failures, including failed opens.
- Accumulate time spent in completed open/retry and read calls, excluding caller processing and canceled calls, without exporting a metric for each JSONL record.
- Record materialization duration when decompressing a rollout for append, for both successful decompression and failures.

GitOrigin-RevId: 3d1e257863ffa696a63040172df54d8234d56ed5
2026-09-16 15:36:43 +00:00
jif 2d90e054d6 Expose partial completion in rollout compression metrics (#45964)
## Why

A compression run can report `completed` after hitting its time budget or encountering file, scan, or cleanup errors. The run metrics do not distinguish these cases from a clean, complete pass.

## What changed

- Add `completion_reason` (`time_budget` or `scan_finished`) and `file_errors`, `scan_errors`, and `cleanup_errors` tags to completed run counts and durations, preserving `status=completed`.
- Record scan failures and additional stale-temp cleanup failures with operation stages and bounded I/O error kinds, and track recoverable errors in the run summary.

GitOrigin-RevId: bb6c33890371a2cdb02bc6ccc82a5df343b823b5
2026-09-16 15:26:30 +00:00
jif 3fa9039bd7 Label rollout compression failures by stage and I/O error kind (#45461)
## Why

Rollout compression failure counters report only that an operation failed,
without identifying the failing stage or I/O error kind.

## What changed

- Add `stage` and `error_kind` labels to failure counters for compression runs,
  individual files, materialization for append, and stale temporary file cleanup.
- Record failures at their source, including lock acquisition and task joins,
  and avoid counting file compression failures twice.
- Use static stage labels and a fixed set of error categories, keeping error
  messages, paths, and rollout contents out of metric tags.

GitOrigin-RevId: ac2bfc7ae4cec4e9f60de9557b03345a3986aa53
2026-09-14 16:37:47 +00:00
Owen Lin 3052bbcf8c Remove the deprecated thread/rollback API (#44915)
## What changed

Remove `thread/rollback`, its request and response types, generated bindings, and the core `Op::ThreadRollback` operation. Requests now follow the generic unknown-method rejection path. Document `thread/revert` as the alternative for paginated threads.

Keep historical `ThreadRolledBack` markers and legacy error deserialization so existing rollouts remain compatible with replay and migration.

## Testing

Adapt retained-context and Guardian history tests to append legacy rollback markers and resume threads, preserving coverage of surviving instructions, answers, and review history.

GitOrigin-RevId: b3da1becdf86b1869275aacb0ffc2817cee5af2e
2026-09-11 21:32:04 +00:00
Owen Lin 196964ef10 Preserve root turn attribution in turn-start events (#44611)
## Why

Persisted turn-start events lack the originating root turn ID, so they cannot directly associate child turns with the root turn that initiated them.

## What changed

- Add optional `root_turn_id` to `TurnStartedEvent`. Use the inherited root turn ID when available, otherwise the turn's own ID.
- Centralize turn-start emission for regular turns, shell commands, and manual compaction so they record attribution consistently.
- Carry attribution into `ThreadHistoryTurnChange` when processing turn-start events and retain it through completion in the history builder.
- Keep older records compatible by defaulting missing attribution to `None`.

## Testing

Add coverage for root and child turn attribution, persistence when tool collisions fail a turn before sampling, history change propagation, and deserialization of older turn-start records.

GitOrigin-RevId: 54a80b828ca7d1c4c19f13d468c01d052e1b9130
2026-09-10 18:24:13 +00:00
Matthew Zeng b5544d5732 Persist disabled plugin IDs in thread settings (#44332)
## What changed

- Add `disabled_plugin_ids` to thread startup options, settings overrides, snapshots, and persisted turn contexts. Omitted overrides preserve the current list; an empty list clears it.
- Restore the selection on resume and capture it before truncating history for a fork. Prefer the owning thread's latest settings snapshot, falling back only to the latest turn context so stale history cannot resurrect an older selection.
- Freeze the selection at turn admission and retain compatibility with histories that lack the new fields.

## Testing

Add coverage for owned settings, explicit clears, latest-turn fallback, and legacy snapshots. Extend settings tests to cover plugin selection during runtime restoration and compaction, and verify initial turn persistence without extra settings checkpoints in both history modes.

GitOrigin-RevId: 75ffbff5763cd93cc5ac69d73b08b237f59bc009
2026-09-09 23:23:59 +00:00
jif a2e83a783e Continue rollout searches when a compressed rollout cannot be searched (#44226)
## Why

An error searching a single compressed rollout could abort the entire search and prevent results from other rollouts from being returned.

## What changed

Log a warning with the rollout path and error, skip that compressed rollout, and continue searching. Apply this behavior to both the compressed scan used alongside `ripgrep` and the fallback scan used when `ripgrep` is unavailable.

GitOrigin-RevId: 07a0d7109456a374244bb84f32cf18e60a69c206
2026-09-09 17:31:15 +00:00
jif 0df1daf526 Attach compressed rollouts to diagnostic reports as JSONL (#44175)
## Why

Diagnostic attachment reads assume the queued file path still exists and contains plain bytes. Compressed rollouts can therefore be omitted when only the logical `.jsonl` path is available, or attached as compressed data when a `.jsonl.zst` path is supplied.

## What changed

- Read rollout attachments through a bounded decoder that resolves plain or compressed representations without materializing a durable JSONL file.
- Use canonical `.jsonl` filenames for attachments and app-server report metadata, while preserving filename overrides.
- Apply size limits to decoded bytes and preserve JSONL prefix truncation.

## Testing

Add regression tests for compressed attachments, representation changes after queuing, plain-sibling preference, filename overrides, decoded size limits, truncation, nonregular files, and unrelated `.zst` attachments.

GitOrigin-RevId: b30f7dd08a741b0c99283460a1ce8933d2920ddf
2026-09-09 15:12:19 +00:00
jif 73a1148c9c Coordinate rollout compression with active thread writers (#44138)
## Why

Cold rollout files can still have active writers. Compression must not replace their files while writes are pending, including background work that outlives the local thread store.

## What changed

- Share cross-process writer locks between the rollout compressor and local thread store. Skip busy threads and recheck file state under coordination before publishing compressed files.
- Retain writer ownership through recorder background I/O and compressed rollout materialization. Wait for writers to exit on shutdown or discard, and stop queued file work before deletion.
- Route rollout metadata patches through owned recorders, reject competing writers before updating SQLite, and serialize metadata appends with shutdown.

## Testing

Add regression coverage for compression with live and detached writers, parallel cold-file compression, publication locking, metadata ownership conflicts, updates to compressed rollouts, concurrent shutdown, and discarding deferred items.

GitOrigin-RevId: c52f4037eb2019e8095b554e29bf531af95aa4bd
2026-09-09 12:30:07 +00:00
rhan-oai 6515a72db7 Preserve runtime workspace roots across thread resume (#43848)
## Why

Resuming a thread should retain its selected workspace folders, including additional roots and explicit empty selections. Resume overrides also need to survive a subsequent resume when no turn has run.

## What changed

- Persist `runtime_workspace_roots` in startup metadata and thread settings snapshots, separately from explicit environment selections and permission-profile roots.
- Restore roots from the latest snapshot owned by the resumed thread, falling back to owned startup metadata only when no snapshot exists. Honor explicit `runtimeWorkspaceRoots` overrides, retarget the old `cwd` root when `cwd` changes, deduplicate roots, and validate restored paths for the current host.
- Checkpoint effective settings on resume and restored settings after revert. Reload resume configuration if saved workspace roots change during loading.
- Normalize Windows rollout path spellings when matching thread search results, preserving selection of the correct rollout after revert, including compressed rollouts.

## Testing

Add regression coverage for workspace restoration, empty and explicit overrides, foreign paths, compaction and revert, resume checkpoints without recency changes, concurrent settings persistence, and rollout search path matching.

GitOrigin-RevId: d98d9d34dd63934d441120916c61c12b69e7f062
2026-09-08 16:24:38 +00:00
Eric Traut d0a8dcd157 Limit archive rollout reads to requested threads (#43494)
## Why

Archiving threads reads metadata from every active rollout, including unrelated compressed files.

## What changed

Add `RolloutReferenceIndex::scan_unarchived_threads` and use it when archiving. Filter canonical filenames by the requested thread IDs before reading contents, while retaining metadata-based ownership checks.

## Testing

Add tests covering reverted rollouts, requested descendants, compressed rollouts, exclusion of archived and unrelated files, and candidate ownership checks.

GitOrigin-RevId: a64f7fa25e5493ed20312f1d8216422752272bae
2026-09-07 15:19:21 +00:00
felixxia-oai 16ff14c266 Retain inherited Guardian instructions in standalone forks (#43478)
## Why

A standalone root forked from a worker needs its inherited user instructions for Guardian review, including after compaction. Parent input counters belong to a different thread, and omitted parent answers may have restricted an inherited authorization.

## What changed

- Adopt inherited user instructions into thread-owned retained context for standalone roots, recovering surviving checkpoint messages when needed.
- Keep inherited instructions before local evidence without sharing local acceptance counters, and preserve that distinction during reconciliation and rollback.
- Mark parent verified answers as incomplete and treat recovered checkpoint instructions as incomplete excerpts.

## Testing

Add coverage for standalone forks from copied history and truncated checkpoints, retention through compaction and resume, inherited/local ordering collisions, and rollback scope.

GitOrigin-RevId: bf716e224a1d29d6c528ef0dd7cdb118a413a08c
2026-09-07 13:58:57 +00:00
faizan-oai 3921a30d6b Persist Daybreak preferences in thread metadata (#42854)
## Why

Clients need a per-thread Daybreak toggle that survives restarts and can be
restored on resume without changing the access program of an active or future
turn.

## What changed

- Add experimental `daybreakEnabled` support to `thread/metadata/update` and
  expose the saved value on thread responses.
- Store the preference in SQLite, including for threads without a first turn,
  and preserve it through reads, resumes, reconciliation, and restarts.
- Copy the preference when forking a thread while keeping subsequent updates
  independent.
- Keep `daybreakEnabled` separate from the per-turn `cyberAccessProgram` field;
  app-server does not derive one from the other.

## Testing

- Cover persistence across restarts, cold reads, reconciliation, and forks.
- Verify metadata updates do not alter the access program of active or later
  turns.

GitOrigin-RevId: bcf987a9fcd7b0ac8a88b1c8f0df06b33ed8ad6e
2026-09-04 21:17:25 +00:00
felixxia-oai 80d7ca34bc Retain user instructions in guardian thread context (#42762)
## Why

Guardian review needs the original user instructions alongside verified answers, even after compaction, resume, or rollback. A summarized or legacy checkpoint cannot establish a complete instruction history.

## What changed

- Capture text user messages in retained context when `guardian_thread_context` is enabled, preserving their message identity through compaction and replay.
- Keep retained instructions and verified answers in arrival order, with independent size and completeness tracking for each family.
- Roll back retained evidence at the exact user-message boundary, including steers that share a turn ID, and apply the same semantics during rollout migration.
- Treat checkpoints without retained user messages as incomplete while preserving their verified answers.

## Testing

- Cover capture, compaction, resume, rollback, child forks, legacy checkpoints, storage bounds, and rollout migration.

GitOrigin-RevId: 72706731b54e5ac8645db61b446d2401afb42235
2026-09-04 13:24:42 +00:00
Can Sar 728cb12fe5 Expose thread originators through the app-server API (#42458)
## What changed

- Add the creation-time `originator` to app-server thread responses and `thread/started` notifications.
- Persist originators in thread metadata and preserve the first recorded value across list, read, resume, rollout, and SQLite paths.
- Add an `originators` allowlist to `thread/list` for hosted backends. The local app-server accepts omitted or empty values but rejects nonempty filters.

## Testing

- Cover originator persistence across server restarts and list, read, and resume requests.
- Verify local `thread/list` handling for nonempty, omitted, null, and empty originator filters.
- Heap-allocate large TUI dispatcher futures in affected tests to keep them within the Windows test-thread stack.

GitOrigin-RevId: 73ad6e61b8fdd2d93a22d1f971723a19ec7be3fc
2026-09-03 06:02:25 +00:00