Commit Graph
103 Commits
Author SHA1 Message Date
Dmitriy Kovalenko ca0bbbdf0c Optimize default and attached thread listings with targeted indexes (#61727)
## Why

Default thread listings should stay in timestamp order without scanning subagent threads. Attached-thread listings should use both parent and archive filters even without planner statistics.

## What changed

- Add partial indexes for visible interactive threads sorted by creation, update, or recency timestamps.
- Use literal source predicates for default interactive listings so SQLite can select the partial indexes.
- Extend the attached-parent index to include archive status.

## Testing

Add a query-plan test verifying that the default sidebar listing uses the interactive update-time index. Extend app-server coverage to include `chatgpt` custom-source threads in default interactive listings.

GitOrigin-RevId: 5844622b7af629cad71e15ec95a8960739d6d447
2026-11-16 02:53:45 +00:00
Brent Traut 32cec881b7 Add thread ID inclusion filters to thread/list (#61446)
## Why

Clients reconciling saved thread IDs need to retrieve matching summaries within the current list filters, including threads beyond the first page of history.

## What changed

- Add `includedThreadIds` with up to 100 entries, applied before pagination alongside existing filters. Exclusions take precedence, and results retain the requested list sort.
- Treat omitted or null inclusions as unrestricted and an empty array as matching nothing. Reject invalid IDs and oversized arrays without truncating them.
- Use indexed metadata and primary-key lookups for local inclusion queries, without scanning rollouts for repairs. IDs absent from the index do not match.
- Update protocol schemas, generated bindings, and documentation, including guidance to verify server support before treating missing saved IDs as removals.

## Testing

Add coverage for saved IDs across project moves and archiving, pagination, exclusions, validation limits, and combinations with search, account, and relationship filters. Add a SQLite query-plan test for primary-key lookups across scopes.

GitOrigin-RevId: 90b02a12aee5716f872a9d5d6168d23cf86c6333
2026-11-14 02:23:37 +00:00
Adam Perry @ OpenAI a4b69a0ae9 Add session-scoped stores for stable model-facing IDs (#61339)
## What changed

- Add `SessionModelIdStore` with SQLite and in-memory implementations for looking up and assigning aliases by session, namespace, and external key.
- Commit SQLite assignments before returning them, and recheck under a write transaction so concurrent writers reuse the same alias.
- Support snapshotting and initializing alias maps from generator state and ordered keys, and remove mappings when their thread is deleted.
- Expose alias stores through `ThreadStore` and add distinct types for registered environment IDs and model-facing `env` aliases.

## Testing

Add a test using two SQLite runtime instances to verify that concurrent assignments return the same alias, different keys receive different aliases, and lookups and snapshots preserve the assignments.

GitOrigin-RevId: 2668f605251ca76d4891f70f94bfb4151ba8142c
2026-11-13 18:43:58 +00:00
Brent Traut 77fde36ae1 Allow searching persisted attached threads (#60574)
## Why

Title and content searches exclude attached threads, preventing clients from finding their saved conversations alongside ordinary threads.

## What changed

- Add `includeAttachedThreads` to `thread/list` and `thread/search`. For `thread/list`, require a non-empty `searchTerm` and preserve the usual provider and source filters.
- Expose `attachedToThreadId` on thread responses to identify the owning parent independently of fork and subagent lineage.
- Support inclusive searches through SQLite and filesystem fallback, repairing missing database entries with saved thread names during title searches.

## Testing

Add regression coverage for opt-in title and content searches, attached-parent metadata after restart, default search filters, filesystem searches without SQLite, and discovery and repair of unindexed title matches.

GitOrigin-RevId: ac12ec3c730ad7f1722bdf8e21600d17f12017f0
2026-11-10 11:17:06 +00:00
ioanbudea-OAI ab541fb990 Encapsulate thread metadata construction (#60524)
## What changed

Move `ThreadMetadataBuilder` into its own module, make its fields private, and replace direct field assignments with fluent `with_*` methods. Initialize `updated_at` and `recency_at` from the creation time and default the approval mode to `OnRequest`.

Update `apply_rollout_items` to accept a boxed `ThreadMetadata` seed, keeping callers' async frames small while preserving existing database fields except for the seed rollout path. Add `build_thread_metadata` to use the state runtime's configured provider, and use `thread_id_from_rollout_path` directly for the doctor's filename fallback.

GitOrigin-RevId: a27e4c6b22832e902922dbdd60d7f5c11b4a1a83
2026-11-10 04:56:49 +00:00
Dmitriy Kovalenko caae161dd0 Keep thread storage available when diagnostic logs fail (#60466)
## Why

Failure to open the diagnostic logs database should not disable thread storage or database-only history listing. Thread deletion must still preserve saved conversations when diagnostic log cleanup cannot complete.

## What changed

- Allow state initialization to continue after non-corruption errors opening the logs database, while preserving corruption recovery.
- Report the original diagnostic database error after tracing is installed in the TUI and app server, and return it from log operations.
- Reject thread deletion before destructive cleanup when diagnostic storage failed to initialize.
- Continue periodic cleanup of legacy logs when the current logs database is unavailable.

## Testing

Extend coverage for unavailable log directories, database-only history listing, startup diagnostics, deletion preserving saved conversations, corrupt log headers, and periodic legacy log cleanup during a diagnostic storage outage.

GitOrigin-RevId: a5184580ae6c39ed0c4eefa8cb48ee7731a991c3
2026-11-09 23:22:08 +00:00
JonLuca De Caro df769889c0 Reduce SQLite write overhead for thread activity and history (#60006)
## Why

Token-usage observations previously required full metadata writes. Concurrent history writers could also occupy pool connections while waiting for SQLite's single writer.

## What changed

- Update only timestamps and token totals for eligible activity patches, preserving concurrent metadata edits. Retain metadata repair for missing rows, changed sources, missing spawn edges, history promotion, and preview backfill.
- Serialize local history writes before acquiring transaction connections so readers can use the remaining pool connections.
- Use `ROWS UNBOUNDED PRECEDING` for cumulative log-byte limits so SQLite can compute byte and row limits in one pass.

## Testing

Add coverage for token updates preserving concurrent edits, clamping negative totals, and rejecting stale source or missing spawn metadata. Extend metadata repair tests across legacy and paginated history modes to cover token updates, preview backfill, and spawn-edge repair.

GitOrigin-RevId: 4a30fd42a7498c616ed13adb46fc3d7e0226a94d
2026-11-06 22:45:37 +00:00
Dmitriy Kovalenko c60cda34a3 Make observed thread metadata writes atomic and retryable (#59016)
## Why

A failed memory-mode update could leave other metadata changes applied. Failed observed metadata writes for paginated threads also need to remain pending so persistence can retry them.

## What changed

- Apply observed metadata, memory mode, and optional recency advancement in one SQLite upsert, preserving monotonic recency.
- Require successful SQLite writes for paginated thread metadata when a state runtime is configured, retaining failed writes for retry.
- Use SQLite busy-retry helpers for read-state writes and immediate transactions.

## Testing

Add regression tests that reject SQLite writes to verify pending metadata survives a failed persist and a rejected memory-mode update leaves existing metadata unchanged. Both tests verify a subsequent retry succeeds.

GitOrigin-RevId: 652bf6ebee0197976852aaf9f096b907697fd590
2026-11-03 00:16:13 +00:00
jif 8806313db4 Scope local history discovery to the current ChatGPT account (#58918)
## What changed

- Add `currentAccountOnly` to `thread/list` and `thread/search`, defaulting to `false`, and enable it for TUI history discovery, resume pickers, and task mentions.
- Mark newly created threads as account-scoped and require each recorded creator user or account ID to match the current login. Older threads and threads without creator IDs remain shared; API-key logins continue to see all local history.
- Preserve account scope through resume and revert, including fallback to original rollout metadata when an older binary drops the marker. Forks receive new-thread scope.
- Skip unreadable rollout headers in scoped discovery and filter search candidates before reading snippets. Direct access by thread ID remains available.

## Testing

Add coverage for account matching, shared legacy history, list page refilling, search pagination, API-key visibility, and scope preservation across reverts. Extend rollout lookup coverage to compressed active and archived files.

GitOrigin-RevId: aa374f5e69165be5a71c0716ded1b105f5fb51e0
2026-11-02 16:00:20 +00:00
jif 020e0f7708 Preserve timestamp ties when paginating threads (#58886)
## Why

Timestamp-only cursors can skip threads that share the last result's timestamp, including rollout files created in the same second.

## What changed

- Include thread IDs as tie-breakers in time-sorted listing and search cursors, and use them when resuming filesystem scans and database queries.
- Share filesystem sort-key and cursor construction so continuation cursors match the listing order, including filename-based creation times.
- Preserve support for legacy timestamp-only cursors and their existing behavior of skipping ties.

## Testing

Update regression coverage for pagination through filesystem timestamp ties, legacy cursors, thread IDs in database fallback cursors, and search cursors for all time-based sort keys.

GitOrigin-RevId: 455b95452849e6ffe20a27d7bdcec864f2e8135a
2026-11-02 13:40:04 +00:00
andrewgu-oai c62e9df39f Preserve reasoning token accounting across resume and fork (#57839)
## Why

Resuming or forking a thread loses the session flag indicating that reported token usage already includes retained reasoning. Counting that reasoning again can trigger automatic compaction too early.

## What changed

Persist `last_usage_includes_reasoning` with `TokenUsageInfo` and use it when estimating context usage. Record the flag with completed sampling and compaction usage, and mark local estimates as including reasoning. Older records without the field retain the existing accounting behavior.

## Testing

Add coverage for saved inclusive, exclusive, and legacy token counts. Extend the encrypted-reasoning compaction test to verify the compaction threshold for live, forked, and cold-resumed threads.

GitOrigin-RevId: 9f22262a693111ec0406dce11804eb4844ad8983
2026-10-28 21:22:06 +00:00
Brent Traut e6c5a83799 Include directly attached threads in diagnostic uploads (#57824)
## What changed

When a diagnostic upload includes logs, include saved threads attached directly to the reported thread in the existing bounded thread selection alongside spawned descendants. Query attached threads from the state database, including archived threads, so their rollouts, logs, and retained review failures can be collected. Log a warning and continue if the lookup fails.

## Testing

Extend the upload integration test to verify that the rollout archive includes the reported and directly attached threads, excludes unrelated threads, and is omitted when logs are disabled.

GitOrigin-RevId: 1bcd2794938339fb4b1c433c3731f507601cd8a4
2026-10-28 20:21:30 +00:00
Steve Coffey f088e0c4ff Allow replacing dynamic tools between model steps (#57553)
## What changed

Add `CodexThread::replace_dynamic_tools` to replace the tool catalog without starting a turn. Updates apply to subsequent model steps and future turns while preserving in-flight calls; an empty list clears the catalog.

Require host opt-in through `allow_persistent_dynamic_tool_updates` for persistent threads. Persist replacements before publishing them, restore the latest catalog on resume, and preserve replacements and explicit clears through compaction. Keep explicit resume overrides separate from the persisted catalog.

## Testing

Add coverage for pending calls during replacement, replacement and clearing across restart, compaction recovery, resume overrides, submission ordering after caller cancellation, host opt-in, and persistence failures.

GitOrigin-RevId: fd2634906895a9b72ae7892ddd9435dced45eccb
2026-10-27 19:50:41 +00:00
Brent Traut 30587d7885 Add experimental attached thread forks (#56351)
## What changed

- Add `attachToSource` to `thread/fork` to persist a fork attached to its source without emitting `thread/started`. Attached forks require local storage and cannot use `ephemeral` or `deferGoalContinuation`.
- Hide attached threads from ordinary listings and add `attachedToThreadId` to `thread/list` for explicit discovery, including empty threads. Keep this relationship separate from spawned-agent parent and ancestor filters.
- Persist attachment metadata in rollouts and SQLite so it survives restarts and database recovery. Reject source deletion until its attached threads are deleted.

## Testing

Add coverage for legacy and paginated history, restart and missing or corrupt database recovery, filtered listings and pagination, resume, conflicting options, and deletion safeguards across store instances.

GitOrigin-RevId: 589f3174fc36f6054e75d65818360695645a5913
2026-10-23 04:52:47 +00:00
Charlie Marsh c7234c1174 Box session metadata and compaction payloads in RolloutItem (#55733)
## What changed

Store `RolloutItem::SessionMeta` and `RolloutItem::Compacted` payloads in `Box` values and update their producers, consumers, and test fixtures. Preserve the existing serialized rollout format.

GitOrigin-RevId: 59fcc4beb47d23160872dcb6e95ace9b142d3d11
2026-10-21 02:41:05 +00:00
willwang-openai 7ae8319917 Retry SQLite operations on transient writer contention (#54369)
## Why

Competing SQLite writers can cause database opening and writes to fail with `SQLITE_BUSY`. Deferred write transactions can also encounter contention after taking a read snapshot.

## What changed

- Add up to two retries with 100 ms and 200 ms delays for SQLite busy errors during pool opening, writer admission, and standalone writes.
- Use a shared `BEGIN IMMEDIATE` helper across state, thread history, and agent message board writes to acquire the writer slot before reading.
- Preserve bound arguments when retrying standalone writes, and commit writes using `RETURNING` before exposing their results. Retry transaction admission without replaying an already-started transaction.

## Testing

Add tests using real SQLite writer contention to verify recovery after a competing writer releases its lock. Queue tests cover enqueue, update, and delete, checking preserved payloads, single revision increments, and no duplicate messages or unintended deletions.

GitOrigin-RevId: ac5bd31a87c67cc47fb385bf3fa5435a54ff70f0
2026-10-15 18:06:57 +00:00
Brent Traut b6e8a2e207 Add durable thread read state with revision-checked updates (#52337)
## Why

A stale read acknowledgement must not clear unread results published after the client’s snapshot or overwrite another read-state edit. Read state also needs to survive thread metadata rebuilds.

## What changed

- Add `thread_read_receipts` and state APIs to query read state, publish thread attention, and mark threads read or unread.
- Track the first unread turn and require matching revisions for explicit edits, returning the current state on conflicts. Allow manually marking even an empty thread unread.
- Retain the last published turn across acknowledgements to suppress duplicate delivery. Keep receipts independent of rebuildable metadata and remove them on thread deletion.
- Return initial read states without creating receipt rows.

## Testing

Add tests covering stale edits, thread-specific initial revisions, repeated unread marks, duplicate publication, persistence across metadata rebuilds and restarts, clearing a reverted turn’s unread position, and deleted-thread handling.

GitOrigin-RevId: fee8c05c20903b1d7aea7e8b61c9248c7f623a72
2026-10-09 00:41:27 +00:00
Eric Traut 80e0b51c9e Make session lookup pagination stable and report listing failures (#51230)
## Why

Session activity can move unread threads ahead of a pagination cursor, and timestamp-only cursors can skip threads with equal creation times. Either can prevent a session label lookup from detecting duplicate labels. Database failures must also remain distinguishable from an exhausted listing.

## What changed

- Resolve session labels using database-only listings ordered by `CreatedAt`, with thread IDs breaking timestamp ties. Accept unique matches across pages with valid timestamp-and-ID cursors, while retaining ambiguity errors for older cursors that cannot establish uniqueness.
- Preserve timestamp index ranges for database continuation queries and timestamp-only cursors for listings that may fall back to the filesystem.
- Propagate database-only listing and name-resolution failures. Keep best-effort name enrichment for ordinary listings and retry latest-session lookup with scan-and-repair after a database-only error.
- Keep loaded sessions selectable in the resume and fork pickers when a later page fails, stop further pagination, and display `Could not load more sessions`.

## Testing

Add regression coverage for creation-time continuation after session activity, cursor query plans, filesystem fallback, database and name-query errors, cross-page label uniqueness, older server cursors, and selection after picker pagination failures.

GitOrigin-RevId: 264ab2e69b8ad12d9edaa31fd253cb9805a46a90
2026-10-06 02:14:22 +00:00
acrognale-oai 7596ab19a6 Keep archived threads with empty previews visible in listings (#48199)
## Why

Archived threads without previews were filtered out of thread listings, even though they had valid session metadata.

## What changed

- Include archived threads with empty previews in both SQLite and rollout-file listings.
- Add archive-specific indexes for creation, update, and recency ordering so these listings can use indexed pagination.

## Testing

Add regression coverage for paginated archive listings with and without SQLite, archive query plans that avoid temporary sorting, and metadata reads that preserve source and file timestamps for empty archived threads.

GitOrigin-RevId: e29d0257b2e029bea5fdf94eed6dab1006d2a8f2
2026-09-25 20:11:47 +00:00
Charlie Marsh 4f6c106cdd Use compact display metadata for Guardian review threads (#47302)
## Why

Guardian review threads use synthetic approval prompts as user messages. Deriving thread titles and previews from those prompts exposes large, unhelpful summaries in thread listings.

## What changed

- Default Guardian threads to the title "Guardian review" and preview "Approval review" across metadata creation, rollout reads, and resume.
- Skip synthetic user messages and goal updates when deriving Guardian display metadata, leaving `first_user_message` unset for new threads.
- Preserve custom names, including legacy indexed names, when listing, reading, resuming, and migrating threads to paginated history.

## Testing

Add regression coverage for compact metadata, resume without session metadata, custom names before and after migration, and app-server persistence, listing, and rename/resume behavior.

GitOrigin-RevId: 42aa707db20c3e57ffa5fd4c91ee313710be10d7
2026-09-22 15:02:54 +00:00
Jeremy Rose 23ae65bb06 Persist thread creator identity in rollouts and SQLite (#47113)
## What changed

- Capture optional `creator_user_id` and `creator_account_id` from authentication when creating a thread, including forks.
- Carry these fields through thread storage, rollout metadata, and a SQLite migration. Leave them absent when unavailable or when reading older threads.
- Preserve recorded creator values across resume and metadata updates. On revert, recover missing rollout values from SQLite before writing replacement metadata.

## Testing

Add coverage for ChatGPT and API-key sessions, resume under another account, forks using current authentication, metadata extraction and upserts, and repeated reverts after creator fields are removed from a rollout.

GitOrigin-RevId: 9839fa42f9392f12226002e2c27d05d5656f5695
2026-09-21 21:22:01 +00:00
rhan-oai 6515a72db7 Preserve runtime workspace roots across thread resume (#43848)
## Why

Resuming a thread should retain its selected workspace folders, including additional roots and explicit empty selections. Resume overrides also need to survive a subsequent resume when no turn has run.

## What changed

- Persist `runtime_workspace_roots` in startup metadata and thread settings snapshots, separately from explicit environment selections and permission-profile roots.
- Restore roots from the latest snapshot owned by the resumed thread, falling back to owned startup metadata only when no snapshot exists. Honor explicit `runtimeWorkspaceRoots` overrides, retarget the old `cwd` root when `cwd` changes, deduplicate roots, and validate restored paths for the current host.
- Checkpoint effective settings on resume and restored settings after revert. Reload resume configuration if saved workspace roots change during loading.
- Normalize Windows rollout path spellings when matching thread search results, preserving selection of the correct rollout after revert, including compressed rollouts.

## Testing

Add regression coverage for workspace restoration, empty and explicit overrides, foreign paths, compaction and revert, resume checkpoints without recency changes, concurrent settings persistence, and rollout search path matching.

GitOrigin-RevId: d98d9d34dd63934d441120916c61c12b69e7f062
2026-09-08 16:24:38 +00:00
jif 3f76e88a48 Add configurable memory versions with isolated storage (#43797)
## What changed

- Add `memories.version` with `v1` as the default and `v2` as an opt-in selection.
- Route memory generation, summaries, and retrieval tools through the selected version. Store `v2` artifacts in `memories_v2` and lazily create a separate SQLite database, keeping jobs and outputs isolated while sharing the thread catalog.
- Preserve the initial memory version across extension config updates so summaries and retrieval tools use the same namespace.
- Clear both versions on memory reset and remove thread memory from both stores on thread deletion. Recognize both artifact roots in shell usage telemetry.

## Testing

Add regression coverage for version defaults and validation, independent job claims and outputs, deletion and reset across versions, and shell usage classification for both memory roots.

GitOrigin-RevId: aa799bf87ab6ec10c8f5668213931e57547d080b
2026-09-08 10:55:48 +00:00
faizan-oai 3921a30d6b Persist Daybreak preferences in thread metadata (#42854)
## Why

Clients need a per-thread Daybreak toggle that survives restarts and can be
restored on resume without changing the access program of an active or future
turn.

## What changed

- Add experimental `daybreakEnabled` support to `thread/metadata/update` and
  expose the saved value on thread responses.
- Store the preference in SQLite, including for threads without a first turn,
  and preserve it through reads, resumes, reconciliation, and restarts.
- Copy the preference when forking a thread while keeping subsequent updates
  independent.
- Keep `daybreakEnabled` separate from the per-turn `cyberAccessProgram` field;
  app-server does not derive one from the other.

## Testing

- Cover persistence across restarts, cold reads, reconciliation, and forks.
- Verify metadata updates do not alter the access program of active or later
  turns.

GitOrigin-RevId: bcf987a9fcd7b0ac8a88b1c8f0df06b33ed8ad6e
2026-09-04 21:17:25 +00:00
Can Sar 728cb12fe5 Expose thread originators through the app-server API (#42458)
## What changed

- Add the creation-time `originator` to app-server thread responses and `thread/started` notifications.
- Persist originators in thread metadata and preserve the first recorded value across list, read, resume, rollout, and SQLite paths.
- Add an `originators` allowlist to `thread/list` for hosted backends. The local app-server accepts omitted or empty values but rejects nonempty filters.

## Testing

- Cover originator persistence across server restarts and list, read, and resume requests.
- Verify local `thread/list` handling for nonempty, omitted, null, and empty originator filters.
- Heap-allocate large TUI dispatcher futures in affected tests to keep them within the Windows test-thread stack.

GitOrigin-RevId: 73ad6e61b8fdd2d93a22d1f971723a19ec7be3fc
2026-09-03 06:02:25 +00:00
felixxia-oai 5971d42847 Preserve verified answers across history compaction (#42293)
## Why

Compaction replaces the model history window, but host-verified user answers may
still be needed for the rest of the owning thread.

## What changed

- Add bounded, model-invisible retained context for verified
  `request_user_input` answers.
- Persist retained context in rollout events and compaction checkpoints, and
  restore it when a thread resumes.
- Remove answers when their source turns are rolled back, and prevent
  parent-local answers from being inherited by child forks.

## Testing

Add lifecycle coverage for deduplication and size limits, compaction and resume,
rollback, rollout migration, and child forks.

GitOrigin-RevId: c32900d3e8120a9899b71f0576683acd67a4b898
2026-09-02 15:07:11 +00:00
Owen Lin 5f79a92e39 Persist response token usage in rollout history (#41912)
## Why

Resumed threads need to continue per-turn and per-thread token totals without scanning arbitrarily far beyond the latest compaction checkpoint.

## What changed

- Add durable `TokenUsageRecord` rollout items with response, turn, thread, session, and root-turn attribution.
- Restore accumulated usage on resume and snapshot the latest record plus the compaction response ID in compaction checkpoints.
- Preserve root-turn lineage in persisted turn context, while ensuring forked child threads start with their own usage totals.

## Testing

- Cover usage accumulation across multiple responses and resumed turns.
- Cover local and remote compaction checkpoints, invalid remote compaction output, rollout reconstruction, and fork isolation.

GitOrigin-RevId: ef9e0c4a9102a08a2c382be4cdac68c84353c90a
2026-08-31 21:15:38 +00:00
Brandon Zhang 0d654e6536 Track window and fork positions in turn metadata (#40987)
## What changed

- Add the zero-based `window_number` to Responses turn metadata and include it in prewarm requests.
- Persist `forked_from_ordinal_exclusive` separately from the physical history base so fork lineage remains accurate after reverts and cold resumes.
- Report context-inheriting subagents through `parent_thread_id` without also emitting fork lineage.
- Reserve the new metadata keys while accepting and filtering previously valid configured values so Codex-owned values take precedence.

## Testing

- Cover window numbering across compaction, resume, and fork operations.
- Cover fork cutoff persistence across direct forks, legacy rollouts, reverts into inherited history, and cold resumes.

GitOrigin-RevId: 288f6cd632254814c1f166668b33828082d799a1
2026-08-26 22:37:59 +00:00
Adam Perry @ OpenAI 1bc02aea50 Sanitize credentials from Git remote metadata (#40713)
## Why

Git remote URLs can embed usernames, passwords, or tokens. Codex carries these
URLs through turn metadata and persisted thread metadata, so credentials must be
removed before a remote enters those paths.

## What changed

- Add `SanitizedGitUrl`, which parses Git URL and SCP-style remotes, strips
  authentication data, and preserves the conventional `git` SSH user.
- Use sanitized remotes when collecting Git metadata, enriching model requests,
  discovering cloud environments, updating thread metadata, and reading legacy
  rollouts. Reject or omit malformed remotes instead of retaining their raw
  contents.
- Keep API, schema, and TypeScript representations as strings while enforcing
  sanitization in Rust.

## Testing

- Cover URL schemes, SCP and IPv6 forms, remote helpers, encoded paths,
  malformed values, and legacy deserialization.
- Verify credentials do not appear in model requests, API responses, SQLite, or
  rollout files.

GitOrigin-RevId: 6435efc4c45bfbfad4723ce7a0457cb00175f25c
2026-08-25 21:34:05 +00:00
guinness-oai 41ca88e95c Persist realtime events in the thread timeline (#40508)
## Why

Clients need a bounded view of realtime conversations that preserves the order of speech, agent work, and turn lifecycle events without loading the full thread history.

## What changed

- Persist realtime session boundaries, transcript segments, and promoted agent items in paginated thread rollouts.
- Add the experimental `thread/timeline/list` API to page ordinary items, realtime items, and turn boundaries in canonical rollout order. Include `activeRealtimeSessionAtPageStart` so each page can be rendered independently.
- Emit typed `thread/realtime/item/started`, `thread/realtime/item/transcript/delta`, and `thread/realtime/item/completed` notifications around durable realtime items.
- Keep the existing `thread/items/list` response and legacy thread history behavior unchanged.

## Testing

- Cover mixed timeline pagination, realtime-session state at page boundaries, rollout replay, subagent history prefixes, transcript streaming, steering, and promoted agent artifacts.

GitOrigin-RevId: faa2c420192d63e060e8fb32446e3c797422656e
2026-08-25 01:03:20 +00:00
joeytrasatti-openai 1b450c7912 Persist thread section moves before the first turn (#39523)
## Why

New non-ephemeral threads have no persisted rollout or preview until their first
turn, so moving them into a section could leave them absent from section-filtered
thread lists.

## What changed

- Materialize and flush a non-ephemeral thread before applying an explicit
  section move.
- Include explicitly sectioned threads with empty previews in section-filtered
  queries, and update the partial indexes to cover them.

## Testing

- Add an app-server test that moves a newly started thread into and out of the
  pinned section before its first turn.
- Cover empty-preview threads in recency and section-position query tests.

GitOrigin-RevId: aa3c4daf9890f06c08b93cd60f146099cbc6cfb2
2026-08-19 17:57:40 +00:00
Owen Lin f950a1ba0b Preserve thread names during rollout migration (#39273)
## Why

Legacy threads and paginated threads read their displayed names from different metadata. Moving a rollout to paginated history must carry over its legacy-visible name so renamed threads do not lose it.

## What changed

- Load legacy indexed names once for an apply run.
- When promoting a thread, preserve an existing canonical name or copy its distinct legacy title, falling back to the legacy name index.
- Repair missing names when migration encounters an already-paginated rollout.

## Testing

Added migration tests covering preservation from both legacy name sources and repair on a rerun.

GitOrigin-RevId: 5fc526570b1cf358a159ef174c159ac091ccc40f
2026-08-18 20:07:26 +00:00
Anton Panasenko 3b4569a920 Add experimental app-server project APIs (#38940)
## What changed

- Add SQLite-backed `project/list`, `project/read`, `project/create`,
  `project/import`, `project/update`, `project/move`, and `project/delete`
  endpoints with ordered roots, metadata, manual positioning, pagination, and
  idempotent creation.
- Add project assignment to thread start, metadata updates, list filtering,
  and fork inheritance. Project deletion clears assignments without deleting
  threads.
- Emit `project/changed` and `thread/project/updated` notifications after
  committed changes, and export the new protocol schemas and TypeScript types.

## Testing

- Cover project lifecycle and ordering, idempotency, atomic imports, cursor and
  filter validation, thread assignment, deletion, and fork inheritance.

GitOrigin-RevId: 47ae621be01ab8ef70a74a629fb29a5c5709ea33
2026-08-17 04:01:37 +00:00
jif 72fa74fbc9 Persist security risk scores in rollout history (#38363)
## What changed

- Add a `SecurityRiskScore` rollout item containing a category and numeric score.
- Persist the item in both thread history modes while excluding it from model context, user-visible thread history, search text, forks, and reconstructed conversation history.
- Re-export the score type from the extension API.

## Testing

- Cover serialization, persistence and loading, thread history projection, session reconstruction, append planning, and memory filtering.

GitOrigin-RevId: 1926fe366aeaa75052708a6da589f45a38eefb52
2026-08-13 12:04:29 +00:00
Owen Lin b1373b74a2 Add durable reverts for paginated threads (#38292)
## What changed

- Add `ThreadStore::revert_thread` to retain history before a selected turn by creating a new immutable rollout and atomically switching the thread's stored rollout path.
- Preserve the logical thread ID and session metadata across repeated reverts.
- Track every rollout owned by a thread so archive, unarchive, and delete operations handle the complete lineage and restore moved files when metadata updates fail.
- Prevent deletion when forks still reference any rollout in the thread's lineage.

## Testing

- Cover repeated reverts, compressed source rollouts, lifecycle operations over reverted histories, reference-aware deletion, and cross-process unarchive locking.

GitOrigin-RevId: 1c05a6db3d0036ee91b562263d7ca200c6a0ea71
2026-08-13 01:56:24 +00:00
joeytrasatti-openai 1549756b78 Add appearance metadata to thread sections (#37898)
## What changed

- Add optional `icon` and `color` appearance fields to custom thread sections and expose them through the app-server protocol.
- Persist appearance metadata in SQLite and include it when listing sections or returning threads.
- Let `threadSection/update` preserve an omitted appearance, clear it with `null`, or replace it with a new value.
- Reject appearance fields larger than 64 bytes.

## Testing

- Cover protocol compatibility, persistence across restart, update and clear behavior, and field-length validation.

GitOrigin-RevId: 8f55ce4f42f8bd17aa5416848a85845f681953df
2026-08-10 23:32:42 +00:00
Owen Lin 6bb6e9045f Add legacy rollout migration to paginated history (#37175)
## What changed

- Add `LocalThreadStore::migrate_rollouts` with dry-run and apply modes,
  optional thread selection, throughput limiting, and per-rollout outcomes.
- Canonicalize legacy JSONL records into paginated history while preserving
  model-visible conversation context, including compressed rollouts and copied
  fork history.
- Publish replacements atomically and journal migrations so interrupted SQLite
  projections can be recovered. Coordinate migration with rollout compression
  and skip rollouts with active writers.
- Make paginated history a one-way SQLite promotion so stale metadata cannot
  downgrade a migrated thread.

## Testing

- Cover legacy record normalization, dry runs, idempotency, malformed input,
  compressed rollouts, active writers, interrupted migration recovery, and a
  cold app-server resume after migration.

GitOrigin-RevId: b9991b659f28ebb52da39ce62e8b4e0bae2ba2bb
2026-08-05 22:57:27 +00:00
efrazer-oai b87981a510 Add durable per-thread user submission queues (#36952)
## What changed

- Add a storage-neutral `QueueStore` interface backed by a dedicated SQLite
  database for ordered, thread-scoped user submissions.
- Support enqueueing, paginated listing, editing, deletion, and atomic
  reordering, with a limit of 100 pending items per thread.
- Remove queued submissions when their thread is deleted.

## Testing

- Cover concurrent inserts, ordering and pagination, edits, deletion,
  cross-thread isolation, queue limits, and thread cleanup.

GitOrigin-RevId: 0a8e95ac71137afc6f3a039d4b5fc8603110c891
2026-08-04 19:23:29 +00:00
rka-oai bb5054fe47 Capture rollout budget units from response usage (#36641)
## What changed

- Parse `codex_rollout_budget_units` from completed Responses API usage into
  `TokenUsage`.
- Keep the provider-only value out of serialized protocol, JSON schema, and
  TypeScript representations.

## Testing

- Cover the value in unit and end-to-end SSE response parsing tests.

GitOrigin-RevId: a53d4202beb9c8985d25894cb10e7c01f20b2a44
2026-08-03 00:30:57 +00:00
joeytrasatti-openai ad6fc66b6d Add persisted manual ordering for thread sections (#36007)
## What changed

- Add `thread/section/move` to atomically move a thread into, within, or out of a section. Threads can be inserted before an existing member or appended, and moves within a section preserve `sectionEnteredAt`.
- Add `section_position` sorting to `thread/list`, with ascending order as its default, and expose `sectionEnteredAt` in thread responses.
- Persist section positions and entry times in SQLite, including migration of existing section members into recency order. Section membership is no longer updated through `thread/metadata/update`.

## Testing

- Cover section moves, reordering, pagination, persistence across restarts and rollout reconciliation, concurrent updates, and rank renumbering.

GitOrigin-RevId: aec6d7ddedca5277029b5caf5c074975397e956c
2026-07-29 18:37:22 +00:00
joeytrasatti-openai 85c6da1c79 Add persisted sections for organizing threads (#35722)
## What changed

- Replace the `isPinned` thread metadata and filters with an optional persisted
  `section` and `sectionId`.
- Add the paginated `threadSection/list` app-server method so clients can
  discover sections even when they contain no threads.
- Seed a stable `Pinned` section, validate section assignments, and support
  filtering for a specific section or for unsectioned threads.

## Testing

- Cover section protocol serialization, listing and pagination, metadata
  updates, filtering, persistence, migration compatibility, and operation
  without SQLite state.

GitOrigin-RevId: 7972b5471d29317b9387bfd90aa9f573f691ad4c
2026-07-28 05:26:16 +00:00
Tamir Duberstein 63682c4e1a Include empty-preview threads in relationship listings (#35691)
## What changed

- Include threads without preview text when listing direct children or descendants from the persisted spawn graph.
- Continue filtering those threads out of the global thread list.

## Testing

- Extend the relationship pagination test to cover an empty-preview child in direct-child and descendant results.

GitOrigin-RevId: c2e7bb04c5f64558a48b1d7877bd24edaa364bb4
2026-07-28 00:24:52 +00:00
Adam Perry @ OpenAI c769a05340 Honor the configured SQLite home across state consumers (#34994)
## Why

Codex and SQLite data can use separate home directories, but state consumers
could reconstruct database paths from the Codex home instead of consistently
using the resolved SQLite configuration.

## What changed

- Pass `SqliteConfig` through the core, rollout, state runtime, and thread store
  instead of passing a directory and rebuilding the configuration downstream.
- Use that shared configuration for state, logs, memories, goals, and paginated
  thread-history database access, including integrity checks and cleanup.
- Reject state database handles whose SQLite configuration does not match the
  requesting store.

## Testing

Add coverage with separate Codex and SQLite homes that verifies startup
backfill, thread listing, and paginated history all use the configured SQLite
directory.

GitOrigin-RevId: 1de1cdd1d6ff1d70bbb6c360c8352e6543fb8ebf
2026-07-23 19:19:35 +00:00
joeytrasatti-openai 400ee190c3 Add persisted thread pinning to the app server (#34840)
## What changed

- Add `isPinned` to thread responses and allow `thread/metadata/update` to pin or unpin stored threads.
- Add an `isPinned` filter to `thread/list`, including cursor-based pagination and combinations with relationship filters.
- Persist pin state in SQLite without modifying rollout files, default existing threads to unpinned, and preserve pins through reconciliation and archive transitions.

## Testing

- Cover protocol serialization, database migration and indexing, pin updates, filtered pagination, rollout reconciliation, and archive/unarchive behavior.

GitOrigin-RevId: c2eacabd6dae465e2ae6ce8a169e00740960cdb7
2026-07-22 22:54:28 +00:00
jif 687f05cb94 Remove CSV-backed agent jobs (#34413)
## What changed

- Remove the `spawn_agents_on_csv` and `report_agent_job_result` tools and their agent-job runtime and state models.
- Drop the legacy `agent_jobs` and `agent_job_items` tables during state database migration.
- Keep `features.enable_fanout` and `agents.job_max_runtime_seconds` accepted as no-op compatibility settings while omitting them from the generated configuration schema.

## Testing

- Verify upgrades remove both legacy agent-job tables.
- Verify the removed feature and configuration keys still parse without taking effect.

GitOrigin-RevId: 8cc3337da78c67162229f02f40a747f503542646
2026-07-20 21:00:17 +00:00
Brent Traut 5a208c1fc3 Persist names for paginated threads (#34229)
## Why

Paginated threads need explicit user-facing names to remain distinct from
derived titles and previews without requiring rollout metadata writes.

## What changed

- Add a nullable `name` column to persisted thread metadata and use it as the
  canonical name for paginated threads.
- Route paginated name updates through the state database while retaining the
  name index as best-effort compatibility data and preserving legacy thread
  behavior.
- Return persisted names when reading, listing, searching, and resuming
  paginated threads, and include them in state-database search matching.

## Testing

- Cover paginated name updates, reads, list/search results, and metadata-only
  resume behavior, including stale or unavailable compatibility data.

GitOrigin-RevId: 5ac4dfa2a83682b43d59ecc87e0f515580164dca
2026-07-19 21:13:13 +00:00
Owen Lin bcdc695877 Track inherited paginated rollout prefixes (#33930)
## What changed

- Add `HistoryPosition` to identify a source thread and an exclusive rollout
  prefix by ordinal and JSONL byte offset.
- Add optional `history_base` metadata to `SessionMeta` so a thread can record
  an inherited prefix of another paginated rollout.
- Default missing `history_base` metadata to `None` and omit it when unset to
  preserve compatibility with existing rollout metadata.

## Testing

- Verify that legacy session metadata without `history_base` deserializes with
  no inherited history position.

GitOrigin-RevId: 3b2b07a2dfee793d3b37b0a60529f856f167dbc1
2026-07-18 00:06:49 +00:00
Eric Traut 2edad72de3 Track prompt cache write token usage (#33454)
## What changed

- Parse `cache_write_tokens` from response input token details and carry it through token usage aggregation.
- Expose `cache_write_input_tokens` in protocol, app-server, exec, and TypeScript SDK usage events, defaulting it to zero for compatibility with older payloads.
- Report cache-write usage in analytics, metrics, traces, and rollout data alongside existing cache-read usage.

## Testing

- Add coverage for response parsing, usage event output, aggregation, analytics, metrics, and telemetry fields.

GitOrigin-RevId: 634e4fbe3086a8923c0ee13b622573e7d9d221bd
2026-07-16 00:22:53 +00:00
Owen Lin 2be648ba4a Preserve paginated history for spawned subagents (#33432)
## What changed

- Inherit paginated history mode when spawning or forking a subagent from a paginated parent.
- Load the parent's model context when forking and persist it as an inherited rollout prefix, while excluding that prefix from the child's projected turns, items, and metadata.
- Record the boundary between inherited context and child history, and reject partially initialized paginated subagent rollouts on resume.
- Resume paginated subagents from their stored model context instead of legacy rollout history.

## Testing

Added coverage for fresh and forked paginated subagents, compacted-history forks, model-context resume, inherited-prefix projection, metadata isolation, and incomplete-prefix detection.

GitOrigin-RevId: e57dc37c29aa4aaaf57b052b991be591d730e5ba
2026-07-15 21:25:24 +00:00
Owen Lin 5267e805fb feat(app-server): add history_mode to thread (#29927)
## Description

This PR adds a new `historyMode = "legacy" | "paginated"` to `Thread`.
This will be stored in `SessionMeta` in the JSONL rollout file and as a
new column in the SQLite thread_metadata table, and exposed on
`thread/start` and on the `Thread` object in app-server.

## What changed

- Added canonical `ThreadHistoryMode` with `legacy` and `paginated`,
defaulting old and new SessionMeta to `legacy`.
- Carried `history_mode` through core session config, ThreadStore stored
metadata, local/in-memory stores, rollout metadata extraction, and the
existing SQLite `threads` table.
- Added experimental `historyMode` to app-server v2 `Thread` and
`thread/start`.
- Made paginated stored threads metadata-discoverable but unsupported
for legacy full-history reads, `load_history`, live resume, and create
paths.
- Regenerated app-server schema fixtures and added
protocol/state/thread-store/app-server coverage for persistence and
fail-closed behavior.

## Compatibility floor
Because users may be running various versions of Codex binaries on the
same machine (TUI, Codex App, etc.), we will need to establish a
compatibility floor for upcoming paginated threads, which will change
how thread storage reads and writes work.

The overall plan here:
```
Release N:
- Add historyMode to SessionMeta / Thread / SQLite metadata.
- Teach binaries to understand paginated threads.
- If a binary sees `historyMode="paginated"` but does not support the paginated contract, it refuses to resume/mutate the thread.
- Default remains `"legacy"`.

Release N+1:
- First-party clients start opting into paginated threads where appropriate.
- Internal dogfood / staged rollout.
- Measure old-client usage and paginated-thread unsupported errors.

Release N+2:
- Only after Release N+ is overwhelmingly deployed, make paginated the default.
- Accept that a small tail of N-1-or-older binaries may not understand paginated threads.
```

The important behavior change is fail-closed handling for a binary that
encounters a persisted `paginated` thread before it knows how to fully
support paginated history. In app-server, if a thread is `paginated`, we
will:

- allow metadata-only discovery paths like `thread/list` and
`thread/read(includeTurns=false)`, so clients can still see the thread
and inspect its `historyMode`
- reject legacy full-history/live-thread paths like
`thread/read(includeTurns=true)` and `thread/resume` with an unsupported
JSON-RPC error
- avoid silently treating an unknown or future `historyMode` as `legacy`

Under the hood, the ThreadStore layer also rejects legacy operations
that would need to load or replay the full thread history for a
paginated thread. That gives us the behavior we want for Release N:
future paginated threads are visible, but this binary fails closed
instead of trying to operate on them as if they were legacy threads.
2026-06-26 09:12:42 -07:00