Commit Graph
107 Commits
Author SHA1 Message Date
Henry Levy d78f68f8d1 Allow per-turn overrides for direct-only tool namespaces (#59481)
## What changed

- Add `direct_only_tool_namespaces` to `TurnStartOptions`. Omission inherits the thread configuration; an empty list clears direct-only routing for that turn. Steering ignores the option.
- Persist explicit overrides in turn context and restore them when recovering an interrupted turn, including turns started by standalone tool output.
- Keep subsequent turns and spawned or resumed child agents on the thread's routing defaults.

## Testing

Add coverage for routing changes across turns in Responses and Responses Lite, recovery without carrying overrides into subsequent turns, and child-agent inheritance.

GitOrigin-RevId: cea1a997424572d4cd372cd15b098626904c6796
2026-11-04 19:58:49 +00:00
Ruslan Nigmatullin 19438adcb6 Introduce a dedicated context window ID type (#59328)
## What changed

Add `ContextWindowId` as a string-backed protocol type with UUIDv7 generation and parsing. Use it throughout compaction state, rollout reconstruction, token budget context, and request metadata in place of raw UUIDs, preserving the serialized string format and UUIDv7 validation.

GitOrigin-RevId: 786b74fcc331707ca5d6765f429b6e5e322d9328
2026-11-04 06:05:16 +00:00
aojiao-oai 16ddd538bd Expose assistant content token counts in completion notifications (#59041)
## What changed

Add optional `contentTokenCount` metadata to `item/completed` notifications, carrying the producer-supplied token count for each completed assistant message in default and plan modes. Counts include any sampled footer and are `null` when unavailable.

Keep this telemetry separate from stored thread items and omit it from subsequent model requests. Update the protocol schemas and generated TypeScript and Python types.

## Testing

Add coverage for per-message counts, including zero, in default and plan modes; missing and malformed metadata; backward-compatible notification decoding; and exclusion from subsequent model requests.

GitOrigin-RevId: 35973037a42650fa2d7569da8e88cb81215cf971
2026-11-03 02:26:29 +00:00
Eric Traut 6ba3a84680 Restore spawned descendants when unarchiving a thread (#57635)
## Why

Archiving a thread also archives its spawned descendants, but unarchiving it previously restored only the parent.

## What changed

- Restore archived spawned descendants when handling `thread/unarchive`, including descendants archived independently of the parent. Skip active descendants and continue past descendant restoration failures; fall back to restoring only the parent if descendant discovery fails.
- Emit `thread/unarchived` for each successfully restored thread. Hold the thread-list mutation permit through archive and unarchive responses and notifications to preserve ordering with other list mutations.
- Add `ThreadStore::unarchive_threads` and share rollout discovery across the subtree in the local store.

## Testing

Extend coverage for subtree restoration, independently archived children, descendant restoration failures, and failed descendant discovery. Store tests cover skipping active descendants and restoring archived descendants without hydrating their metadata.

GitOrigin-RevId: c54ca099c92bcb2df5e595fa252d7fad7c018760
2026-10-28 04:05:31 +00:00
Brent Traut 30587d7885 Add experimental attached thread forks (#56351)
## What changed

- Add `attachToSource` to `thread/fork` to persist a fork attached to its source without emitting `thread/started`. Attached forks require local storage and cannot use `ephemeral` or `deferGoalContinuation`.
- Hide attached threads from ordinary listings and add `attachedToThreadId` to `thread/list` for explicit discovery, including empty threads. Keep this relationship separate from spawned-agent parent and ancestor filters.
- Persist attachment metadata in rollouts and SQLite so it survives restarts and database recovery. Reject source deletion until its attached threads are deleted.

## Testing

Add coverage for legacy and paginated history, restart and missing or corrupt database recovery, filtered listings and pagination, resume, conflicting options, and deletion safeguards across store instances.

GitOrigin-RevId: 589f3174fc36f6054e75d65818360695645a5913
2026-10-23 04:52:47 +00:00
Charlie Marsh ffda54f5b8 Reuse current model context across paginated resume config loading (#56224)
## Why

Resuming a paginated thread rechecks stored history after loading configuration. Unchanged model context can be reused instead of loading it again.

## What changed

Retain the prepared model context across configuration loading and reuse it when the thread ID, rollout path, and history revision still match. Add `ThreadStore::is_model_context_current` to validate revisions, with a fresh read when validation fails or is unavailable.

## Testing

Add coverage for revision invalidation after history is appended or removed. Extend resume tests to cover paginated history, concurrent metadata and lifecycle changes, workspace-root changes, and preservation of saved user messages in model requests.

GitOrigin-RevId: bad2ca0caddb63523264ac530ffb801db3961f2b
2026-10-22 17:15:06 +00:00
viktor-oai a3b685331f Add local diagnostic ZIP exports to the app server (#55940)
## What changed

Add a save RPC that accepts the existing report options and an absolute `savePath` on the server host. Save the report, selected attachments, and a manifest without uploading, and return `savedPath`, `threadId`, and `promptHash`. Local exports remain available when uploads are disabled by configuration.

Publish archives atomically without replacing existing files. Bound uncompressed contents to 200 MiB and attachments to 128, preserve duplicate attachment names safely, and fail without publishing a partial ZIP when a selected file cannot be read. Honor `includeLogs` and omit automatic rollouts for untouched threads without materializing them.

Update protocol schemas and generated TypeScript and Python types.

## Testing

Add coverage for local-only saves, disabled uploads, path validation, overwrite protection, log consent, archive budgets, duplicate names, missing files, and rollout selection before materialization and after writer shutdown.

GitOrigin-RevId: 222c0fe7ab923904de9c8a9f12ad27f7a0104875
2026-10-21 16:43:30 +00:00
jif 46d252f8bf Make the startup migration contention test deterministic (#55524)
## Why

The maintenance-lock test relied on a 50 ms timeout to assume startup migration had encountered contention, and its completion timeout included the production retry delay.

## What changed

Add a per-store, test-only retry gate to observe lock contention and explicitly resume migration after releasing the maintenance lock. Verify that rollout metadata stays in `Legacy` mode while the lock is held and becomes `Paginated` after migration completes.

GitOrigin-RevId: 657fa6ec6f9494d9e66976560c8158fde8ffb495
2026-10-20 09:53:33 +00:00
Owen Lin b7ba02f880 Support store-owned preparation when resuming threads (#54059)
## Why

Thread stores need a way to reuse history and writer state loaded while preparing a resume, rather than loading them again during session startup.

## What changed

- Add an opaque, single-use `ThreadStoreResumePreparation` and an optional preparation in `StartThreadOptions`.
- Pass preparation through session startup to `ThreadStore::resume_thread_with_preparation`, which defaults to an unsupported-operation error.
- Reject preparation for ephemeral threads or threads without resumed history. Continue using `resume_thread` when no preparation is supplied.

## Testing

Add a regression test verifying that supplied preparation selects the prepared store path and propagates its unsupported-operation error instead of falling back to ordinary resume.

GitOrigin-RevId: 26537d9b83443bd56341d6afc713f32b999dc67c
2026-10-14 18:02:25 +00:00
pakrym-oai 5ef96ab278 Add opt-in output token replay for OpenAI requests (#52742)
## What changed

- Add the disabled-by-default `output_token_replay` feature to request `output.encrypted_content` from OpenAI providers.
- Preserve encrypted message and tool-call output, message and function-call status, and output-text `annotations` and `logprobs` through history and replay. Update the generated protocol schemas and SDK types.
- Strip output ciphertext from requests to other providers while retaining it in stored history, including after replay is disabled. Remove encrypted content from memory extraction inputs.

## Testing

Add integration coverage for the feature/provider combinations, metadata replay across follow-up requests, and history preservation when resuming with replay disabled.

GitOrigin-RevId: 77729a021b6771933402aebe74ece8eb39c057e2
2026-10-10 01:25:00 +00:00
pakrym-oai 28cd3e2501 Remove obsolete fields from persisted turn context (#51492)
## What changed

Remove `workspace_roots`, `current_date`, `timezone`, `network`, `personality`, and `multi_agent_mode` from `TurnContextItem` and stop recording them in rollouts. Update the precomputed protocol schema accordingly.

Make `summary` optional when reading turn context and write `"none"` as a compatibility placeholder for older readers, rather than persisting the active reasoning-summary setting.

## Testing

Update protocol tests to cover omitted `summary` fields and compatibility-placeholder serialization. Extend compaction tests to verify that model requests retain the updated reasoning-summary setting after compaction while the rollout checkpoint records the placeholder.

GitOrigin-RevId: d49eabaa1b340be1d79f7f09115281a307ea8a64
2026-10-06 22:39:06 +00:00
Eddie Chen cfc946f4e1 Expose and persist turn lineage across the app server (#51415)
## Why

Clients starting delegated work need to attribute descendant turns to the original turn and retain that attribution through completion and history reads.

## What changed

- Accept optional `parentTurnId` and `rootTurnId` in `turn/start`, and expose `rootTurnId` on returned turns and lifecycle notifications.
- Default new turns to their own root when none is supplied. Preserve the active turn's root when steering, ignoring incoming lineage fields.
- Carry root IDs through completion, interruption, recovery, and fork boundaries, and persist them for paginated history reads. Leave unknown roots null in older history.
- Update protocol schemas and generated TypeScript and Python bindings.

## Testing

Extend coverage for default and explicit roots, steering without changing lineage, interrupted and recovered turns, history reads after restart in both history modes, nested fork pagination, and compatibility with older database writers.

GitOrigin-RevId: f1a570a12bd11742929fd2e8b017e5d35aa07dcb
2026-10-06 16:56:39 +00:00
Eddie Chen 551bd409eb Preserve turn attribution across recovery and compaction (#51402)
## Why

Recovering an interrupted turn replaced its original trigger with `retry` and restored only its root lineage from model context. Turns suspended before that context was written could lose their attribution entirely.

## What changed

- Persist regular-turn attribution in `TurnStartedEvent` and compaction resume metadata, including the trigger, parent and root turn IDs, and initiating agent path.
- Restore matching attribution during recovery, keep unknown triggers absent, and retain the model-context root fallback for older records.
- Reconstruct attribution from surviving turns across rollback and late completion or abort events, and clear inherited attribution when spawning a forked agent.

## Testing

Add coverage for repeated cold recovery, suspension before context persistence, legacy rollouts, fresh attribution on subsequent turns, compaction checkpoints, rollback, and late terminal events.

GitOrigin-RevId: 85b5763e8fbd48a1ea2c07cfda8196e63c2cd33a
2026-10-06 15:50:27 +00:00
Benjamin Carlsson 09bced5ad9 Populate thread previews from delegated task inputs (#50462)
## Why

Delegated turns can begin without a user message, leaving threads without a task preview. Populate the preview from delegated input so these threads are discoverable before a user follow-up.

## What changed

- Extract previews from completed `create_thread` and `send_message_to_thread` outputs in the `codex_app` and `codex_tui` namespaces.
- Unwrap recognized `<codex_delegation>` inputs and decode their escaped task text for display, preserving model content. Keep plain or unrecognized wrappers as text and ignore blank inputs.
- Fill missing previews in state metadata extraction and live thread metadata updates without replacing existing previews or setting the title or `first_user_message`.

## Testing

Add tests for delegation decoding and tool filtering, preview-only metadata updates, preservation of later user-message metadata, and database listing before a user follow-up in both legacy and paginated history modes.

GitOrigin-RevId: f7b3c60ef75e08c76b4084d043e3967bf2bade22
2026-10-02 23:34:11 +00:00
joeytrasatti-openai 2635431edb Add attachment owner lookup to the app-server (#50094)
## Why

`thread/attachment/list` lists attachments for a known thread. Clients also need to find the threads associated with a given attachment identity.

## What changed

Add `thread/attachmentOwner/list` to return owning thread IDs and archive status for an exact `attachmentType` and `identityKey`. Support cursor pagination and an optional `archived` filter, including both archived and non-archived threads by default.

Wire the request through the thread store and update documentation, protocol schemas, and generated TypeScript and Python types. The lookup is limited to the configured thread store and reports current membership, not an atomic resource-cleanup check.

## Testing

Extend app-server integration tests to cover exact type-and-key matching, multiple owners, archive filtering, and invalid cursor rejection.

GitOrigin-RevId: 30f4a20e00243d0023246d465a9b95213b3adeb6
2026-10-01 20:47:05 +00:00
felixxia-oai 92bc601ad6 Reuse unchanged history snapshots when resuming threads (#49600)
## Why

Resuming a thread rereads persisted history even when a preloaded snapshot is unchanged. Reusing that snapshot must still account for writes committed before writer ownership is acquired.

## What changed

- Carry optional, non-serialized revisions from stored history and model context through thread and agent resume paths.
- Validate the revision under writer ownership and reuse the shared snapshot when it matches. Reload stored history when the revision is missing or changed, while preserving explicit history overrides.
- Resolve the selected rollout before loading context, validate compressed snapshots before materialization, and enforce archived-thread restrictions on canonical paths.

## Testing

Extend coverage for snapshot reuse and late-write refresh in legacy and paginated histories, compressed rollout reuse, archived rollout symlinks, and cold child restoration that includes edits made after preloading history.

GitOrigin-RevId: e1fd7aee6ef2b30f41951e6b7e6b01c3e1420cab
2026-09-30 11:59:13 +00:00
felixxia-oai d7b0d4aa66 Return authoritative replay history when resuming a thread (#49599)
## Why

A stored history snapshot can become stale before a resumed thread acquires writer ownership. Resume must include writes committed before that acquisition.

## What changed

Make `ThreadStore::resume_thread` return the replay context and use it for session restoration and metadata initialization. The local store refreshes stored snapshots while holding writer ownership, reads through the installed writer to honor explicit rollout paths, and releases the writer if loading context fails. Explicit histories without a canonical session header retain their override semantics.

## Testing

Extend resume tests to cover edits made after the initial snapshot in both legacy and paginated history modes, stale SQLite rollout paths, and preservation of explicitly supplied history.

GitOrigin-RevId: 2361f524d324c859f5dfcf2863320e2045736c2e
2026-09-30 11:58:50 +00:00
jif c0d26949be Avoid full metadata rewrites for thread timestamp updates (#48983)
## Why

Updates that only change `updated_at` should not rewrite unrelated thread metadata or its indexes.

## What changed

Use `touch_thread_updated_at` for timestamp-only patches when the SQLite row exists and the rollout path is unchanged. Preserve the full update path for missing rows, changed rollout paths, and patches containing other metadata, including staged changes.

## Testing

Add regression coverage for legacy and paginated histories that verifies missing-row reconstruction, timestamp-only writes using SQLite triggers, preservation of other metadata, and application and removal of staged metadata.

GitOrigin-RevId: d5bc70d57e591e3eb8e9521f0c31a7ef75f244d3
2026-09-28 13:43:55 +00:00
acrognale-oai 7596ab19a6 Keep archived threads with empty previews visible in listings (#48199)
## Why

Archived threads without previews were filtered out of thread listings, even though they had valid session metadata.

## What changed

- Include archived threads with empty previews in both SQLite and rollout-file listings.
- Add archive-specific indexes for creation, update, and recency ordering so these listings can use indexed pagination.

## Testing

Add regression coverage for paginated archive listings with and without SQLite, archive query plans that avoid temporary sorting, and metadata reads that preserve source and file timestamps for empty archived threads.

GitOrigin-RevId: e29d0257b2e029bea5fdf94eed6dab1006d2a8f2
2026-09-25 20:11:47 +00:00
Owen Lin 12cb14f7b7 Default local threads to paginated history (#47900)
## What changed

- Make `LocalThreadStore` default to paginated history while retaining explicit legacy mode.
- Explicitly use paginated history for Guardian review sessions and allow in-memory stores to create paginated threads.
- Rename full-history resume, fork, and subagent helpers to identify them as legacy APIs, and document the paginated alternatives.
- Reject `thread/revert` for ephemeral threads and prevent the TUI from replaying or attaching to ephemeral agent threads.

## Testing

Add coverage for preserving the default and explicitly selected history modes and conversation context across restart, and for retaining context after rejected ephemeral reverts. Extend Guardian checkpoint coverage across both history modes and verify that saved reviews use paginated history.

GitOrigin-RevId: 870239eb3757018159ee4f230a009c97d38ac5b7
2026-09-24 18:07:37 +00:00
Owen Lin 5121aeff13 Overlap subagent spawn persistence and clean up cancelled children (#47348)
## Why

Forked subagent startup serializes history persistence and spawn-edge writes. Cancellation during startup can also leave a provisional child behind.

## What changed

- Defer copied-history durability so it can run alongside spawn-edge persistence, while still awaiting both before submitting the child's initial input.
- Commit agent and residency reservations and notify clients only after initial input is accepted.
- Guard provisional children so cancellation or startup failure removes and shuts them down, discards their stored state, and closes their spawn edge after any pending edge write finishes.

## Testing

Add coverage for cancellation at the child-history durability barrier, checking child termination, removal, and edge closure. Extend fork-context tests in legacy and paginated history modes to verify inherited context can be read from storage after spawn acknowledgment without flushing the live child first.

GitOrigin-RevId: e02dda57efc32b7237bb7af0ce5e5844922890c3
2026-09-22 19:39:18 +00:00
dermanyang-oai b781733f95 Separate thread preparation from execution persistence (#47199)
## Why

Injecting context before the first turn currently forces a full flush. Stores need a preparation checkpoint that can leave disposable context in memory, while local storage must keep prepared context durable before the first user input.

## What changed

- Add `PersistContext::ThreadPreparation` and `CodexThread::checkpoint_preparation()`, and use the checkpoint in `inject_response_items()`.
- Allow stores to defer rollout-derived metadata writes through `record_thread_metadata()`, including during preparation and execution checkpoints. Keep standard metadata updates synchronous.
- Flush local storage at preparation checkpoints so injected context survives restart.

## Testing

Add tests showing that preparation and first sampling can proceed without waiting for durable metadata. Cover prepared context in first and follow-up requests, plus durability before first input and restoration after restart, in both legacy and paginated history modes.

GitOrigin-RevId: 74f6bb1a5f8d47192364b8bba6afd703a3b7d688
2026-09-22 05:53:29 +00:00
Jeremy Rose 23ae65bb06 Persist thread creator identity in rollouts and SQLite (#47113)
## What changed

- Capture optional `creator_user_id` and `creator_account_id` from authentication when creating a thread, including forks.
- Carry these fields through thread storage, rollout metadata, and a SQLite migration. Leave them absent when unavailable or when reading older threads.
- Preserve recorded creator values across resume and metadata updates. On revert, recover missing rollout values from SQLite before writing replacement metadata.

## Testing

Add coverage for ChatGPT and API-key sessions, resume under another account, forks using current authentication, metadata extraction and upserts, and repeated reverts after creator fields are removed from a rollout.

GitOrigin-RevId: 9839fa42f9392f12226002e2c27d05d5656f5695
2026-09-21 21:22:01 +00:00
jif 73129716c2 Delete persisted message boards with their root threads (#47029)
## Why

Permanent thread deletion needs to remove persisted agent message board data even when the message board feature has been disabled.

## What changed

- Add a cleanup hook for single and batch thread deletion, invoked after reference and writer checks but before rollout removal so cleanup failures remain retryable.
- Remove the deleted roots' channels, posts, and subscriptions regardless of the current feature setting.
- Record board tombstones and check them within write transactions to prevent delayed writes from recreating deleted data.

## Testing

Add an app-server regression test that creates a board, archives its root thread, restarts with the feature disabled, and verifies that permanent deletion removes the persisted channels, posts, and subscriptions.

GitOrigin-RevId: 4ea71559a8df0814dffda23a056a9bbaf6e0a0cb
2026-09-21 13:28:02 +00:00
joeytrasatti-openai 19286b8819 Copy current thread attachments into non-ephemeral forks (#45579)
## What changed

Copy the source thread's current attachments when creating a non-ephemeral fork, including forks at an earlier turn. Copies receive new attachment IDs and creation timestamps while preserving resource identities and payloads. Attachment membership can then change independently on either thread; referenced resources are not copied.

Await the atomic copy before publishing the fork. If copying fails, log the error and allow the conversation fork to succeed without attachments. Resuming a fork does not copy attachments again.

Document that clients should use `forkedFromId` on `thread/started` and fetch the fork's attachments with `thread/attachment/list`; copying does not emit per-attachment updates.

## Testing

Add coverage for atomic rollback, independent attachment membership, inheritance across history cutoffs, successful conversation forks after copy failures, and resuming without restoring removed attachments.

GitOrigin-RevId: 939d2c6a3073ccfcd922380c6beda45d3deac606
2026-09-15 02:33:27 +00:00
faizan-oai ef8b356c22 Allow setting daybreakEnabled when starting a thread (#45513)
## What changed

Add experimental `thread/start.daybreakEnabled` so clients can set the initial preference for persistent threads. Omitted or null values leave it unset; explicit values are rejected for ephemeral threads.

Return the choice in the start response, `thread/started`, and reads before persistence. Stage it with the initial thread metadata and save it when the thread is persisted. Later changes still use `thread/metadata/update`. The preference does not select `turn/start.cyberAccessProgram` or grant access.

## Testing

Add coverage for true, false, and unset values in responses, notifications, reads, and reads after persistence and restart, plus rejection for ephemeral threads. Update existing metadata and access-program tests to exercise threads with an initial preference.

GitOrigin-RevId: 3bff3dc55a18436067bc2a3f156f5abf52d7321b
2026-09-14 20:57:48 +00:00
jif 21b1ef18c6 Retain thread persistence acquisition through session cancellation (#45487)
## Why

Session initialization can be cancelled after a persistence writer is installed but before acquisition returns. Cleanup must wait for acquisition to finish so it can discard the writer.

## What changed

Extend `LiveThreadInitGuard` to own in-flight acquisition and finish it before discarding persistence, including when the guard is dropped. Use the guard for thread creation, resume, and inherited model context initialization.

## Testing

Add a regression test that cancels acquisition after writer installation, verifies cleanup waits for handoff, and confirms the writer is removed afterward.

GitOrigin-RevId: 09041fa5cd675d082a86a4cc8a719c572afe11a8
2026-09-14 18:40:02 +00:00
Owen Lin 196964ef10 Preserve root turn attribution in turn-start events (#44611)
## Why

Persisted turn-start events lack the originating root turn ID, so they cannot directly associate child turns with the root turn that initiated them.

## What changed

- Add optional `root_turn_id` to `TurnStartedEvent`. Use the inherited root turn ID when available, otherwise the turn's own ID.
- Centralize turn-start emission for regular turns, shell commands, and manual compaction so they record attribution consistently.
- Carry attribution into `ThreadHistoryTurnChange` when processing turn-start events and retain it through completion in the history builder.
- Keep older records compatible by defaulting missing attribution to `None`.

## Testing

Add coverage for root and child turn attribution, persistence when tool collisions fail a turn before sampling, history change propagation, and deserialization of older turn-start records.

GitOrigin-RevId: 54a80b828ca7d1c4c19f13d468c01d052e1b9130
2026-09-10 18:24:13 +00:00
joeytrasatti-openai 2df0b747ba Add thread attachment operations with coordinated deletion (#44350)
## Why

Attachment mutations must coordinate with thread deletion so queued requests cannot use metadata belonging to a deleted thread.

## What changed

- Add typed `ThreadStore` operations for idempotent attachment creation, paginated listing, and removal, with capability detection and unsupported defaults.
- Implement attachment operations for SQLite-backed `LocalThreadStore`, including unloaded threads, and preserve invalid-request and missing-thread errors.
- Coordinate attachment mutations with lifecycle locks and move SQLite cleanup into thread-store deletion. Remove state rows only after rollout deletion, preserving retries after cleanup failures.
- Give the in-memory store ownership of its associated SQLite cleanup and remove duplicate cleanup from the app-server.

## Testing

Add coverage for attachment pagination and idempotency, unsupported stores, error categories, lifecycle exclusion, mutations queued behind single and batch deletion, cleanup failure retries, and in-memory state cleanup. Extend the app-server descendant deletion test to verify SQLite rows are removed.

GitOrigin-RevId: f0157538c7c5c2ee1362ce40eac157101c5d4505
2026-09-10 00:30:27 +00:00
Matthew Zeng b5544d5732 Persist disabled plugin IDs in thread settings (#44332)
## What changed

- Add `disabled_plugin_ids` to thread startup options, settings overrides, snapshots, and persisted turn contexts. Omitted overrides preserve the current list; an empty list clears it.
- Restore the selection on resume and capture it before truncating history for a fork. Prefer the owning thread's latest settings snapshot, falling back only to the latest turn context so stale history cannot resurrect an older selection.
- Freeze the selection at turn admission and retain compatibility with histories that lack the new fields.

## Testing

Add coverage for owned settings, explicit clears, latest-turn fallback, and legacy snapshots. Extend settings tests to cover plugin selection during runtime restoration and compaction, and verify initial turn persistence without extra settings checkpoints in both history modes.

GitOrigin-RevId: 75ffbff5763cd93cc5ac69d73b08b237f59bc009
2026-09-09 23:23:59 +00:00
jif 73a1148c9c Coordinate rollout compression with active thread writers (#44138)
## Why

Cold rollout files can still have active writers. Compression must not replace their files while writes are pending, including background work that outlives the local thread store.

## What changed

- Share cross-process writer locks between the rollout compressor and local thread store. Skip busy threads and recheck file state under coordination before publishing compressed files.
- Retain writer ownership through recorder background I/O and compressed rollout materialization. Wait for writers to exit on shutdown or discard, and stop queued file work before deletion.
- Route rollout metadata patches through owned recorders, reject competing writers before updating SQLite, and serialize metadata appends with shutdown.

## Testing

Add regression coverage for compression with live and detached writers, parallel cold-file compression, publication locking, metadata ownership conflicts, updates to compressed rollouts, concurrent shutdown, and discarding deferred items.

GitOrigin-RevId: c52f4037eb2019e8095b554e29bf531af95aa4bd
2026-09-09 12:30:07 +00:00
rhan-oai 6515a72db7 Preserve runtime workspace roots across thread resume (#43848)
## Why

Resuming a thread should retain its selected workspace folders, including additional roots and explicit empty selections. Resume overrides also need to survive a subsequent resume when no turn has run.

## What changed

- Persist `runtime_workspace_roots` in startup metadata and thread settings snapshots, separately from explicit environment selections and permission-profile roots.
- Restore roots from the latest snapshot owned by the resumed thread, falling back to owned startup metadata only when no snapshot exists. Honor explicit `runtimeWorkspaceRoots` overrides, retarget the old `cwd` root when `cwd` changes, deduplicate roots, and validate restored paths for the current host.
- Checkpoint effective settings on resume and restored settings after revert. Reload resume configuration if saved workspace roots change during loading.
- Normalize Windows rollout path spellings when matching thread search results, preserving selection of the correct rollout after revert, including compressed rollouts.

## Testing

Add regression coverage for workspace restoration, empty and explicit overrides, foreign paths, compaction and revert, resume checkpoints without recency changes, concurrent settings persistence, and rollout search path matching.

GitOrigin-RevId: d98d9d34dd63934d441120916c61c12b69e7f062
2026-09-08 16:24:38 +00:00
faizan-oai 3921a30d6b Persist Daybreak preferences in thread metadata (#42854)
## Why

Clients need a per-thread Daybreak toggle that survives restarts and can be
restored on resume without changing the access program of an active or future
turn.

## What changed

- Add experimental `daybreakEnabled` support to `thread/metadata/update` and
  expose the saved value on thread responses.
- Store the preference in SQLite, including for threads without a first turn,
  and preserve it through reads, resumes, reconciliation, and restarts.
- Copy the preference when forking a thread while keeping subsequent updates
  independent.
- Keep `daybreakEnabled` separate from the per-turn `cyberAccessProgram` field;
  app-server does not derive one from the other.

## Testing

- Cover persistence across restarts, cold reads, reconciliation, and forks.
- Verify metadata updates do not alter the access program of active or later
  turns.

GitOrigin-RevId: bcf987a9fcd7b0ac8a88b1c8f0df06b33ed8ad6e
2026-09-04 21:17:25 +00:00
rka-oai 2c79ee6dac Add structured asynchronous user input requests (#42178)
## What changed

- Replace `send_user_message_async` with `request_user_input_async`, accepting one or more questions with optional suggested answers while allowing the turn to continue.
- Attach structured question metadata to asynchronous agent messages while retaining readable fallback text, and preserve it through app-server events, thread history, and generated schemas.
- Continue enabling the new tool for model catalogs that advertise either the old or new tool name.

## Testing

- Cover tool registration, question validation, emitted items, continued turn execution, history preservation, and backwards-compatible deserialization.

GitOrigin-RevId: ffc49aebde2e854c3c50a122aa52805fec3fc6b3
2026-09-02 02:09:35 +00:00
Owen Lin 5f79a92e39 Persist response token usage in rollout history (#41912)
## Why

Resumed threads need to continue per-turn and per-thread token totals without scanning arbitrarily far beyond the latest compaction checkpoint.

## What changed

- Add durable `TokenUsageRecord` rollout items with response, turn, thread, session, and root-turn attribution.
- Restore accumulated usage on resume and snapshot the latest record plus the compaction response ID in compaction checkpoints.
- Preserve root-turn lineage in persisted turn context, while ensuring forked child threads start with their own usage totals.

## Testing

- Cover usage accumulation across multiple responses and resumed turns.
- Cover local and remote compaction checkpoints, invalid remote compaction output, rollout reconstruction, and fork isolation.

GitOrigin-RevId: ef9e0c4a9102a08a2c382be4cdac68c84353c90a
2026-08-31 21:15:38 +00:00
faizan-oai 94967e03e5 Add per-turn cyber access program selection (#40687)
## What changed

- Add the experimental `turn/start.cyberAccessProgram` option with `standard`,
  `daybreakBlue`, and `daybreakRed` values.
- Forward the selection as `access_programs.cyber` on Responses, WebSocket,
  and remote-compaction requests made with ChatGPT authentication, while
  omitting it for API-key and custom-provider requests.
- Preserve the per-turn selection across recovery, compaction, and child-agent
  turns without making it a persistent thread setting.

## Testing

- Cover app-server forwarding, authentication boundaries, WebSocket reuse,
  turn recovery, compaction, and child-agent inheritance.

GitOrigin-RevId: d2eb468f365b2214c5099bc21741cf30f8bd2eb5
2026-08-25 19:54:52 +00:00
guinness-oai 41ca88e95c Persist realtime events in the thread timeline (#40508)
## Why

Clients need a bounded view of realtime conversations that preserves the order of speech, agent work, and turn lifecycle events without loading the full thread history.

## What changed

- Persist realtime session boundaries, transcript segments, and promoted agent items in paginated thread rollouts.
- Add the experimental `thread/timeline/list` API to page ordinary items, realtime items, and turn boundaries in canonical rollout order. Include `activeRealtimeSessionAtPageStart` so each page can be rendered independently.
- Emit typed `thread/realtime/item/started`, `thread/realtime/item/transcript/delta`, and `thread/realtime/item/completed` notifications around durable realtime items.
- Keep the existing `thread/items/list` response and legacy thread history behavior unchanged.

## Testing

- Cover mixed timeline pagination, realtime-session state at page boundaries, rollout replay, subagent history prefixes, transcript streaming, steering, and promoted agent artifacts.

GitOrigin-RevId: faa2c420192d63e060e8fb32446e3c797422656e
2026-08-25 01:03:20 +00:00
Owen Lin 90c67e6f33 Classify rollout migration failures (#39784)
## Why

Rollout migration outcomes expose free-form error messages, making failures hard to group without parsing text.

## What changed

- Add a serialized `RolloutMigrationFailureReason` to failed outcomes and classify failures across reading, conversion, SQLite materialization, publishing, and recovery.
- Add the failure reason as a low-cardinality tag on per-thread migration telemetry.

## Testing

- Cover missing SQLite metadata and invalid session metadata classifications.

GitOrigin-RevId: 2bff98483b484f4d02b72d52f39950ff99ebe429
2026-08-20 19:24:09 +00:00
rka-oai 763787d061 Support standalone named function call outputs (#39782)
## Why

External tool events may need to enter thread history without a preceding function call and therefore do not have a `call_id`.

## What changed

- Allow `function_call_output` items to omit `call_id` and carry optional `name` and `namespace` fields.
- Preserve named standalone outputs during history normalization and agent forks while retaining existing pairing behavior for outputs with a `call_id`.
- Accept, persist, and forward these outputs through `thread/inject_items`, and update the app-server schemas and documentation.

## Testing

- Cover paired and standalone JSON round trips, history normalization, agent forks, and injected thread history.

GitOrigin-RevId: a3258163a7dc93777c7c3023116fe204819bdbb0
2026-08-20 19:19:41 +00:00
rka-oai fb356f3d2c Add async delivery metadata to agent messages (#39312)
## What changed

- Add an optional `delivery` field to agent message events and app-server
  `agentMessage` items, with `"async"` identifying a user-visible message sent
  without ending the current turn.
- Preserve the marker through legacy event conversion, thread history
  materialization, replay, and generated JSON and TypeScript schemas.
- Require delivery of async agent message completion notifications on the
  in-process app-server transport.

## Testing

- Cover preservation of async delivery metadata in thread history.
- Cover delivery-required classification for async agent messages.

GitOrigin-RevId: f92ba25de1293ee271404badfdf0d117d6530329
2026-08-18 23:05:52 +00:00
Shijie Rao 230791fd1f Persist active permission profiles in turn context (#39145)
## What changed

- Add the optional `active_permission_profile` field to `TurnContextItem` to record the built-in or named profile that produced the effective permission profile.
- Populate the field when serializing a turn context while preserving compatibility with records where it is absent.

## Testing

- Add coverage verifying that turn-context serialization stores the active permission profile.

GitOrigin-RevId: d69142ab1b26595ab406235f89818ef41370e963
2026-08-18 05:46:08 +00:00
Anton Panasenko 3b4569a920 Add experimental app-server project APIs (#38940)
## What changed

- Add SQLite-backed `project/list`, `project/read`, `project/create`,
  `project/import`, `project/update`, `project/move`, and `project/delete`
  endpoints with ordered roots, metadata, manual positioning, pagination, and
  idempotent creation.
- Add project assignment to thread start, metadata updates, list filtering,
  and fork inheritance. Project deletion clears assignments without deleting
  threads.
- Emit `project/changed` and `thread/project/updated` notifications after
  committed changes, and export the new protocol schemas and TypeScript types.

## Testing

- Cover project lifecycle and ordering, idempotency, atomic imports, cursor and
  filter validation, thread assignment, deletion, and fork inheritance.

GitOrigin-RevId: 47ae621be01ab8ef70a74a629fb29a5c5709ea33
2026-08-17 04:01:37 +00:00
Anton Panasenko fcdae21073 Support metadata staging for reserved thread IDs (#38819)
## Why

Callers need to associate host-owned state with a thread before Core starts it.

## What changed

- Add `ThreadManager::reserve_thread_id` and allow new threads to start with the reserved ID. Reject reserved IDs when resuming an existing thread.
- Let thread stores stage metadata for a reserved ID. The local store merges staged values into the first successful metadata update and then removes them.
- Clear staged metadata when an unmaterialized thread shuts down or is discarded, and require the state database for staging.

## Testing

- Cover reserved ID generation and resume validation.
- Cover metadata persistence, merging, rollout-compatible updates, cleanup, and invalid staging requests.

GitOrigin-RevId: 3f9bc48f0f7764011cc90ea01349e497dda0a9e3
2026-08-16 00:57:28 +00:00
Rasmus Rygaard 42bb50d502 Allow metadata updates without materializing threads (#38413)
## Why

Thread stores may filter a metadata patch to a no-op. Requiring every successful
update to return a `StoredThread` forces those implementations to materialize a
thread even when the caller does not need one.

## What changed

- Let `ThreadStore::update_thread_metadata` return `None` after a successful
  update that did not materialize a thread.
- Preserve the materialized-thread contract of `LiveThread` and `ThreadManager`
  metadata updates by reading the thread when the store returns `None`.
- Keep completion-only metadata updates from issuing that fallback read.

## Testing

Cover materialized in-memory updates and verify that fallback reads occur only
for callers that require the updated thread.

GitOrigin-RevId: 3059efc9d5b69e0b46e35fb5b5aa9638086654f2
2026-08-13 18:12:27 +00:00
Owen Lin b1373b74a2 Add durable reverts for paginated threads (#38292)
## What changed

- Add `ThreadStore::revert_thread` to retain history before a selected turn by creating a new immutable rollout and atomically switching the thread's stored rollout path.
- Preserve the logical thread ID and session metadata across repeated reverts.
- Track every rollout owned by a thread so archive, unarchive, and delete operations handle the complete lineage and restore moved files when metadata updates fail.
- Prevent deletion when forks still reference any rollout in the thread's lineage.

## Testing

- Cover repeated reverts, compressed source rollouts, lifecycle operations over reverted histories, reference-aware deletion, and cross-process unarchive locking.

GitOrigin-RevId: 1c05a6db3d0036ee91b562263d7ca200c6a0ea71
2026-08-13 01:56:24 +00:00
Owen Lin 8d4d57387a Resolve paginated thread history by rollout ID (#38244)
## Why

`thread/revert` keeps the logical thread ID while selecting a new immutable
rollout. Using the thread ID for persisted history can therefore read or update
the wrong rollout.

## What changed

- Centralize resolution of a thread's current rollout across live writers,
  SQLite metadata, and filesystem fallbacks.
- Key history materialization, lineage traversal, paging, turn lookup, and fork
  boundaries by immutable rollout ID while retaining the thread ID for the
  logical thread.
- Treat SQLite's selected rollout as authoritative for paginated threads so a
  filesystem scan cannot select an older rollout after `thread/revert`.

## Testing

Add a regression test showing that update-ordinal item paging reads from the
selected rollout ID when it differs from the thread ID.

GitOrigin-RevId: 1101e548a6e6c6d319996e24cb650e5b03ab7fbb
2026-08-12 20:31:52 +00:00
rka-oai 3a6f747d77 Preserve harness metadata across conversation history (#38058)
## What changed

- Wrap response items with optional harness-owned metadata while keeping the persisted response payload shape backward compatible.
- Store compacted-history metadata in an aligned sidecar and reject malformed sidecars.
- Carry metadata through history normalization, compaction, resume, fork, truncation, migration, and rollout recording without sending it to model providers.

## Testing

- Cover legacy rollout compatibility, metadata serialization and validation, compaction retention, and resume/fork reconstruction.

GitOrigin-RevId: 17a6127c9048ff984cb4df4d97c0751a38c29f5f
2026-08-11 19:30:12 +00:00
Bryan Ashley 722784e936 Distinguish turn-start thread persistence (#37926)
## What changed

- Add `PersistContext` to the thread-store persistence contract so stores can
  identify persistence requested immediately before model sampling. A
  `TurnStart` request may be enqueued in the background when later flush and
  shutdown operations fence it and report any failure; all other persistence
  remains synchronous.
- Use the turn-start context only for the initial input of a regular turn, while
  admission acknowledgments, steered input, and other persistence paths keep
  the standard durability behavior.
- Flush pending metadata before turn-start persistence and ensure shutdown is
  still attempted if that metadata update fails.

## Testing

- Verify that developer instructions and user input are persisted before the
  model request completes for a default paginated-history thread.

GitOrigin-RevId: 162f0ec796a61aebe66ca0b909fbcccec3047b85
2026-08-11 03:35:45 +00:00
Adam Perry @ OpenAI 63002bdb26 Extract persisted history types into a dedicated crate (#37871)
## What changed

- Add `codex-history` for model-history and persisted-rollout domain types, including `RolloutItem`, `RolloutLine`, `CompactedItem`, and initial/resumed history state.
- Re-export the persisted types from `codex-rollout` and update consumers to use the new crate boundary instead of `codex-protocol`.
- Preserve existing rollout serialization, including legacy numeric compacted-window IDs.

## Testing

- Add `codex-history` tests for rollout JSON round trips, compacted-history compatibility, persisted history modes, and multi-agent version selection.

GitOrigin-RevId: 944daa9297ddd231d3aebbdcb05fff4adf8b4e1b
2026-08-10 19:26:52 +00:00
Owen Lin 4bb7ee3472 Add rollout migration tooling and background migration (#37348)
## What changed

- Add `codex migrate-rollouts` with dry-run inspection by default, explicit `--apply`, thread filtering, optional I/O throttling, progress output, and JSON or verbose reports.
- Add the disabled-by-default `background_paginated_rollout_migration` feature to migrate legacy local sessions at startup before rollout compression begins.
- Persist a migration cursor and skipped-file fingerprints so later startups avoid full rescans while retrying changed files and recovering pending migrations.
- Coordinate migration with live writers and emit metrics for manual and startup runs.

## Testing

- Cover startup cursor advancement and lookback, pending migration recovery, live-writer coordination, changed empty rollouts, and progress reporting.

GitOrigin-RevId: 276ac506c50ebec5140fd319faca1bb998172061
2026-08-07 01:26:42 +00:00