Commit Graph

142 Commits

Author SHA1 Message Date
Shijie Rao
230791fd1f Persist active permission profiles in turn context (#39145)
## What changed

- Add the optional `active_permission_profile` field to `TurnContextItem` to record the built-in or named profile that produced the effective permission profile.
- Populate the field when serializing a turn context while preserving compatibility with records where it is absent.

## Testing

- Add coverage verifying that turn-context serialization stores the active permission profile.

GitOrigin-RevId: d69142ab1b26595ab406235f89818ef41370e963
2026-08-18 05:46:08 +00:00
Eric Traut
eeb82a156d Dispatch queued messages written by other processes (#39034)
## Why

Durable thread queues can be updated through another SQLite connection, but loaded idle threads need to notice those updates before they can dispatch the new messages.

## What changed

- Track a durable revision for each thread queue, including backfilling existing queues and updating revisions on inserts, updates, and deletes.
- Poll SQLite's data version and query revisions to find changed queues for loaded threads.
- Wake idle threads with pending external messages, discover queued work when threads are loaded or resumed, and retry each thread independently so one blocked queue does not stall others.

## Testing

- Cover cross-runtime queue writes, edits, independent dispatch, wake retries, and resumed threads.
- Cover migration backfills and revision tracking for queue updates and deletes.

GitOrigin-RevId: 906d902bbf6a760be206a1b2c08fd71427b63c46
2026-08-17 17:19:46 +00:00
Anton Panasenko
3b4569a920 Add experimental app-server project APIs (#38940)
## What changed

- Add SQLite-backed `project/list`, `project/read`, `project/create`,
  `project/import`, `project/update`, `project/move`, and `project/delete`
  endpoints with ordered roots, metadata, manual positioning, pagination, and
  idempotent creation.
- Add project assignment to thread start, metadata updates, list filtering,
  and fork inheritance. Project deletion clears assignments without deleting
  threads.
- Emit `project/changed` and `thread/project/updated` notifications after
  committed changes, and export the new protocol schemas and TypeScript types.

## Testing

- Cover project lifecycle and ordering, idempotency, atomic imports, cursor and
  filter validation, thread assignment, deletion, and fork inheritance.

GitOrigin-RevId: 47ae621be01ab8ef70a74a629fb29a5c5709ea33
2026-08-17 04:01:37 +00:00
Anton Panasenko
fcdae21073 Support metadata staging for reserved thread IDs (#38819)
## Why

Callers need to associate host-owned state with a thread before Core starts it.

## What changed

- Add `ThreadManager::reserve_thread_id` and allow new threads to start with the reserved ID. Reject reserved IDs when resuming an existing thread.
- Let thread stores stage metadata for a reserved ID. The local store merges staged values into the first successful metadata update and then removes them.
- Clear staged metadata when an unmaterialized thread shuts down or is discarded, and require the state database for staging.

## Testing

- Cover reserved ID generation and resume validation.
- Cover metadata persistence, merging, rollout-compatible updates, cleanup, and invalid staging requests.

GitOrigin-RevId: 3f9bc48f0f7764011cc90ea01349e497dda0a9e3
2026-08-16 00:57:28 +00:00
efrazer-oai
9341b38310 Add experimental thread queue APIs to app server (#38456)
## What changed

- Add experimental `thread/queue/add`, `list`, `update`, `delete`, `reorder`, and `start` requests for persistent queued user submissions.
- Automatically dispatch queued submissions in FIFO order after completed or failed turns, while leaving the queue paused after interruption.
- Emit `thread/queue/changed` notifications and expose paginated queue contents with stable submission and client message IDs.
- Export the new protocol types through the generated JSON Schema and TypeScript bindings.

## Testing

- Add app-server integration coverage for capability gating, CRUD, pagination, capacity limits, notifications, automatic dispatch, cold-thread resume, interruption, and explicit queue starts.

GitOrigin-RevId: 89caf5577192199bda77cf6c372a59d26cdd910e
2026-08-13 23:38:02 +00:00
Rasmus Rygaard
42bb50d502 Allow metadata updates without materializing threads (#38413)
## Why

Thread stores may filter a metadata patch to a no-op. Requiring every successful
update to return a `StoredThread` forces those implementations to materialize a
thread even when the caller does not need one.

## What changed

- Let `ThreadStore::update_thread_metadata` return `None` after a successful
  update that did not materialize a thread.
- Preserve the materialized-thread contract of `LiveThread` and `ThreadManager`
  metadata updates by reading the thread when the store returns `None`.
- Keep completion-only metadata updates from issuing that fallback read.

## Testing

Cover materialized in-memory updates and verify that fallback reads occur only
for callers that require the updated thread.

GitOrigin-RevId: 3059efc9d5b69e0b46e35fb5b5aa9638086654f2
2026-08-13 18:12:27 +00:00
Tamir Duberstein
2bd8727a0c Preserve floating-point values when decoding rollout lines (#38399)
## Why

Serde's generic buffering for flattened and internally tagged fields is incompatible with `serde_json`'s `arbitrary_precision` representation. As a result, persisted rollout items containing nested floating-point values can fail to decode.

## What changed

- Decode the rollout envelope at the JSON persistence boundary before deserializing the item payload.
- Use the shared decoder for both session resume and thread-history projection.
- Cover token-count floats, response-item metadata, field ordering, and ignored metadata in the decoder compatibility test.

GitOrigin-RevId: 59c7f6fbcf83162b5ed6980a69381fe45fc20a0a
2026-08-13 16:03:20 +00:00
jif
72fa74fbc9 Persist security risk scores in rollout history (#38363)
## What changed

- Add a `SecurityRiskScore` rollout item containing a category and numeric score.
- Persist the item in both thread history modes while excluding it from model context, user-visible thread history, search text, forks, and reconstructed conversation history.
- Re-export the score type from the extension API.

## Testing

- Cover serialization, persistence and loading, thread history projection, session reconstruction, append planning, and memory filtering.

GitOrigin-RevId: 1926fe366aeaa75052708a6da589f45a38eefb52
2026-08-13 12:04:29 +00:00
Owen Lin
b1373b74a2 Add durable reverts for paginated threads (#38292)
## What changed

- Add `ThreadStore::revert_thread` to retain history before a selected turn by creating a new immutable rollout and atomically switching the thread's stored rollout path.
- Preserve the logical thread ID and session metadata across repeated reverts.
- Track every rollout owned by a thread so archive, unarchive, and delete operations handle the complete lineage and restore moved files when metadata updates fail.
- Prevent deletion when forks still reference any rollout in the thread's lineage.

## Testing

- Cover repeated reverts, compressed source rollouts, lifecycle operations over reverted histories, reference-aware deletion, and cross-process unarchive locking.

GitOrigin-RevId: 1c05a6db3d0036ee91b562263d7ca200c6a0ea71
2026-08-13 01:56:24 +00:00
Owen Lin
8d4d57387a Resolve paginated thread history by rollout ID (#38244)
## Why

`thread/revert` keeps the logical thread ID while selecting a new immutable
rollout. Using the thread ID for persisted history can therefore read or update
the wrong rollout.

## What changed

- Centralize resolution of a thread's current rollout across live writers,
  SQLite metadata, and filesystem fallbacks.
- Key history materialization, lineage traversal, paging, turn lookup, and fork
  boundaries by immutable rollout ID while retaining the thread ID for the
  logical thread.
- Treat SQLite's selected rollout as authoritative for paginated threads so a
  filesystem scan cannot select an older rollout after `thread/revert`.

## Testing

Add a regression test showing that update-ordinal item paging reads from the
selected rollout ID when it differs from the thread ID.

GitOrigin-RevId: 1101e548a6e6c6d319996e24cb650e5b03ab7fbb
2026-08-12 20:31:52 +00:00
rka-oai
3a6f747d77 Preserve harness metadata across conversation history (#38058)
## What changed

- Wrap response items with optional harness-owned metadata while keeping the persisted response payload shape backward compatible.
- Store compacted-history metadata in an aligned sidecar and reject malformed sidecars.
- Carry metadata through history normalization, compaction, resume, fork, truncation, migration, and rollout recording without sending it to model providers.

## Testing

- Cover legacy rollout compatibility, metadata serialization and validation, compaction retention, and resume/fork reconstruction.

GitOrigin-RevId: 17a6127c9048ff984cb4df4d97c0751a38c29f5f
2026-08-11 19:30:12 +00:00
Tamir Duberstein
ed390a5dc4 Filter live rollout items in place (#38034)
## What changed

Retain persistable items directly in the owned append buffer instead of cloning
them into a second vector. Preserve the existing history-mode filtering and
empty-append behavior.

GitOrigin-RevId: 5300312545492bef28f63521427048a27d46a9d1
2026-08-11 16:27:58 +00:00
Tamir Duberstein
4496ba3fd5 Use session metadata to validate thread history paths (#38033)
## What changed

Validate a SQLite thread's rollout path by reading its `SessionMetaLine` and
comparing the recorded thread ID directly. This avoids constructing a
`StoredThread` solely to decide whether the path can supply the requested
history.

GitOrigin-RevId: 8eb1e329514cefc056e44ab4a76d58ff50314a97
2026-08-11 16:20:12 +00:00
arun eswara
edcec13372 Expose image generation usage-limit failures (#38024)
## What changed

- Add structured `usageLimitExceeded` metadata to failed image-generation
  items, including the image limit ID and optional reset timestamp.
- Preserve the failure metadata in completion events, app-server thread
  history, thread reads and resumes, and migrated legacy rollouts.
- Export the new failure type in the generated JSON and TypeScript schemas.

## Testing

- Cover usage-limit responses through item completion, persistence, and thread
  resume.
- Verify legacy rollout migration retains image-generation failure metadata.

GitOrigin-RevId: 91c31a2a47456b6b916ee03002ff314b05946dd4
2026-08-11 15:40:52 +00:00
Bryan Ashley
722784e936 Distinguish turn-start thread persistence (#37926)
## What changed

- Add `PersistContext` to the thread-store persistence contract so stores can
  identify persistence requested immediately before model sampling. A
  `TurnStart` request may be enqueued in the background when later flush and
  shutdown operations fence it and report any failure; all other persistence
  remains synchronous.
- Use the turn-start context only for the initial input of a regular turn, while
  admission acknowledgments, steered input, and other persistence paths keep
  the standard durability behavior.
- Flush pending metadata before turn-start persistence and ensure shutdown is
  still attempted if that metadata update fails.

## Testing

- Verify that developer instructions and user input are persisted before the
  model request completes for a default paginated-history thread.

GitOrigin-RevId: 162f0ec796a61aebe66ca0b909fbcccec3047b85
2026-08-11 03:35:45 +00:00
joeytrasatti-openai
1549756b78 Add appearance metadata to thread sections (#37898)
## What changed

- Add optional `icon` and `color` appearance fields to custom thread sections and expose them through the app-server protocol.
- Persist appearance metadata in SQLite and include it when listing sections or returning threads.
- Let `threadSection/update` preserve an omitted appearance, clear it with `null`, or replace it with a new value.
- Reject appearance fields larger than 64 bytes.

## Testing

- Cover protocol compatibility, persistence across restart, update and clear behavior, and field-length validation.

GitOrigin-RevId: 8f55ce4f42f8bd17aa5416848a85845f681953df
2026-08-10 23:32:42 +00:00
Adam Perry @ OpenAI
63002bdb26 Extract persisted history types into a dedicated crate (#37871)
## What changed

- Add `codex-history` for model-history and persisted-rollout domain types, including `RolloutItem`, `RolloutLine`, `CompactedItem`, and initial/resumed history state.
- Re-export the persisted types from `codex-rollout` and update consumers to use the new crate boundary instead of `codex-protocol`.
- Preserve existing rollout serialization, including legacy numeric compacted-window IDs.

## Testing

- Add `codex-history` tests for rollout JSON round trips, compacted-history compatibility, persisted history modes, and multi-agent version selection.

GitOrigin-RevId: 944daa9297ddd231d3aebbdcb05fff4adf8b4e1b
2026-08-10 19:26:52 +00:00
Owen Lin
4bb7ee3472 Add rollout migration tooling and background migration (#37348)
## What changed

- Add `codex migrate-rollouts` with dry-run inspection by default, explicit `--apply`, thread filtering, optional I/O throttling, progress output, and JSON or verbose reports.
- Add the disabled-by-default `background_paginated_rollout_migration` feature to migrate legacy local sessions at startup before rollout compression begins.
- Persist a migration cursor and skipped-file fingerprints so later startups avoid full rescans while retrying changed files and recovering pending migrations.
- Coordinate migration with live writers and emit metrics for manual and startup runs.

## Testing

- Cover startup cursor advancement and lookback, pending migration recovery, live-writer coordination, changed empty rollouts, and progress reporting.

GitOrigin-RevId: 276ac506c50ebec5140fd319faca1bb998172061
2026-08-07 01:26:42 +00:00
Brent Traut
547080e4d6 Prefer persisted cwd when reading local threads (#37198)
## Why

The `cwd` recorded in a rollout can become stale after thread metadata is
updated, causing thread reads and lists to disagree with the persisted state.

## What changed

- Overlay a non-empty state database `cwd` when reading a thread by ID.
- Apply the same overlay when reading by rollout path, but only when the
  metadata points to the requested rollout.
- Recompute the permission profile against the selected `cwd` and continue to
  fall back to rollout metadata when the persisted `cwd` is empty or belongs to
  another rollout.

## Testing

- Cover stale, empty, and mismatched persisted `cwd` metadata.
- Verify that `thread/read`, state-only `thread/list`, and `thread/resume`
  report the persisted thread `cwd` while the resumed session uses its requested
  live `cwd`.

GitOrigin-RevId: 50a3f187618ee3cadbad95196ee2976e2d5e9a5a
2026-08-06 02:26:24 +00:00
Owen Lin
aac9f84247 Preserve legacy semantics during rollout migration (#37191)
## Why

Legacy rollouts can contain historical rollbacks, compaction checkpoints, and
subagent copies of parent history. Migrating those records verbatim can change
the visible conversation or the model context used when a thread resumes, and
can retain large amounts of duplicated subagent history.

## What changed

- Plan and replay historical rollbacks by logical turn, preserving surviving
  lifecycle events and the correct compaction anchor for future resumes.
- Migrate subagent rollouts from a safe bounded replay when possible and record
  the history boundary so copied parent context is not projected as child turns.
- Include archived and compressed rollouts while keeping their storage form and
  location intact.
- Tolerate malformed, partial, oversized, and empty JSONL records during the
  migration scan.

## Testing

Add coverage for rollback and compaction combinations, subagent replay bounds,
archived and compressed rollouts, recovery, concurrent maintenance, and damaged
JSONL input.

GitOrigin-RevId: 3948e16d43f75df8571b0a92af12a181d1d08fdf
2026-08-06 00:54:14 +00:00
Owen Lin
6bb6e9045f Add legacy rollout migration to paginated history (#37175)
## What changed

- Add `LocalThreadStore::migrate_rollouts` with dry-run and apply modes,
  optional thread selection, throughput limiting, and per-rollout outcomes.
- Canonicalize legacy JSONL records into paginated history while preserving
  model-visible conversation context, including compressed rollouts and copied
  fork history.
- Publish replacements atomically and journal migrations so interrupted SQLite
  projections can be recovered. Coordinate migration with rollout compression
  and skip rollouts with active writers.
- Make paginated history a one-way SQLite promotion so stale metadata cannot
  downgrade a migrated thread.

## Testing

- Cover legacy record normalization, dry runs, idempotency, malformed input,
  compressed rollouts, active writers, interrupted migration recovery, and a
  cold app-server resume after migration.

GitOrigin-RevId: b9991b659f28ebb52da39ce62e8b4e0bae2ba2bb
2026-08-05 22:57:27 +00:00
efrazer-oai
b87981a510 Add durable per-thread user submission queues (#36952)
## What changed

- Add a storage-neutral `QueueStore` interface backed by a dedicated SQLite
  database for ordered, thread-scoped user submissions.
- Support enqueueing, paginated listing, editing, deletion, and atomic
  reordering, with a limit of 100 pending items per thread.
- Remove queued submissions when their thread is deleted.

## Testing

- Cover concurrent inserts, ordering and pagination, edits, deletion,
  cross-thread isolation, queue limits, and thread cleanup.

GitOrigin-RevId: 0a8e95ac71137afc6f3a039d4b5fc8603110c891
2026-08-04 19:23:29 +00:00
Owen Lin
dc60dadce6 Enforce single-writer ownership for all thread histories (#36389)
## Why

Legacy thread histories did not use the cross-process writer ownership guard already applied to paginated histories.

## What changed

- Acquire and retain a writer lock whenever a legacy or paginated thread is created or resumed.
- Apply the same ownership checks when archiving or deleting either history mode.
- Release ownership when initialization fails or the active thread shuts down, allowing another process to resume it.

## Testing

- Cover competing create and resume operations for both history modes.
- Cover archive and deletion conflicts for legacy and paginated descendants.

GitOrigin-RevId: 1be79b01966ad2634ed79eb96c0aa0db8eafe2fa
2026-07-31 20:49:08 +00:00
Owen Lin
332eac4b85 Load turn summaries with paginated queries (#36384)
## Why

Loading the summary view issued a separate item query for every returned turn.

## What changed

Join each turn's first user and final agent items into the paginated turn query
when `StoredTurnItemsView::Summary` is requested. Preserve the existing fallback
that resolves inherited summary items for synthetic fork-boundary rows.

GitOrigin-RevId: add7f2e03c0809bbee85dc9f07bb6f91af92dfe7
2026-07-31 20:12:28 +00:00
joeytrasatti-openai
c42ea41ee0 Add thread section management APIs (#36380)
## What changed

- Add `threadSection/create`, `threadSection/update`, and `threadSection/delete` app-server methods, with generated protocol schemas and TypeScript bindings.
- Persist custom sections in SQLite with stable UUIDv7 identities, trim and validate display names, and prevent renaming or deleting the built-in pinned section.
- Delete sections transactionally and return their active and archived threads to the unsectioned list.
- Serialize section mutations with section listing so concurrent requests observe ordered updates.

## Testing

- Cover section persistence across restarts, ordered renames, member cleanup on deletion, invalid requests, unsupported stores, and thread-store/state-runtime behavior.

GitOrigin-RevId: dc9a4be283c21dbc93ab1829eafc4825d8df6575
2026-07-31 19:08:45 +00:00
Owen Lin
6256a7ccc7 Make thread history projection resilient to malformed rollouts (#36188)
## Why

A failed rollout append can leave a rejected line before a valid same-ordinal
retry. Advancing the byte checkpoint past that line without also advancing the
ordinal checkpoint can prevent subsequent history from being projected.

## What changed

- Defer rejected and unknown rollout lines until a later valid ordinal shows
  whether they were replaced by a retry or consumed an ordinal.
- Advance skipped ordinal ranges together with the durable byte checkpoint,
  while rejecting gaps that cannot be explained by rejected lines.
- Prefer item event timestamps and parse the rollout timestamp only when an
  item needs it as a fallback.

## Testing

Added coverage for same-ordinal retries, inferred ordinal gaps, unknown and
malformed records, invalid fallback timestamps, and unexplained gaps.

GitOrigin-RevId: 374beda316c605d562bd9526abed19916598d69f
2026-07-30 16:49:12 +00:00
jif
88ec932e96 Handle fractional rate limits in thread history projection (#36083)
## Why

Paginated thread history could reject valid rollout records when rate-limit
percentages were encoded as floating-point JSON values, preventing later final
answers from being projected.

## What changed

Deserialize each rollout line through `serde_json::Value` before converting it
to `RolloutLine`, preserving floating-point rate-limit values during history
materialization.

## Testing

Added coverage for projecting fractional, integral, very small, and large
floating-point rate-limit percentages across both catch-up materialization and
subsequent history appends.

GitOrigin-RevId: 68da907987ffe59f4a3033ef90f3b88c39734cb1
2026-07-30 04:55:09 +00:00
joeytrasatti-openai
ad6fc66b6d Add persisted manual ordering for thread sections (#36007)
## What changed

- Add `thread/section/move` to atomically move a thread into, within, or out of a section. Threads can be inserted before an existing member or appended, and moves within a section preserve `sectionEnteredAt`.
- Add `section_position` sorting to `thread/list`, with ascending order as its default, and expose `sectionEnteredAt` in thread responses.
- Persist section positions and entry times in SQLite, including migration of existing section members into recency order. Section membership is no longer updated through `thread/metadata/update`.

## Testing

- Cover section moves, reordering, pagination, persistence across restarts and rollout reconciliation, concurrent updates, and rank renumbering.

GitOrigin-RevId: aec6d7ddedca5277029b5caf5c074975397e956c
2026-07-29 18:37:22 +00:00
Owen Lin
fa1d4c40d0 Gate paginated thread history on the state database (#35787)
## Why

Local thread stores without an initialized state database should not implicitly
create SQLite files or partially delete threads that have materialized history.

## What changed

- Report paginated history listing as unsupported when no state database is
  available, and skip history projection and materialization in that mode.
- Reject paginated forks without a state database.
- Validate access to materialized history before deleting rollout files, so a
  failed deletion preserves both the rollout and its history rows.

## Testing

Added coverage that a store without a state database creates no SQLite files
and preserves materialized history when deletion is unsupported. Existing
projection tests now initialize the state runtime explicitly.

GitOrigin-RevId: 2eaa0f5f0de8d5e4d84375ec50b310746a0ee68e
2026-07-28 16:00:39 +00:00
joeytrasatti-openai
85c6da1c79 Add persisted sections for organizing threads (#35722)
## What changed

- Replace the `isPinned` thread metadata and filters with an optional persisted
  `section` and `sectionId`.
- Add the paginated `threadSection/list` app-server method so clients can
  discover sections even when they contain no threads.
- Seed a stable `Pinned` section, validate section assignments, and support
  filtering for a specific section or for unsectioned threads.

## Testing

- Cover section protocol serialization, listing and pagination, metadata
  updates, filtering, persistence, migration compatibility, and operation
  without SQLite state.

GitOrigin-RevId: 7972b5471d29317b9387bfd90aa9f573f691ad4c
2026-07-28 05:26:16 +00:00
Owen Lin
8d6a91b1ff Preserve item timestamps in thread history projections (#35689)
## What changed

- Add optional start and completion timestamps to `ThreadHistoryItemChange`.
- Populate them from canonical `ItemCompleted` records during stateless thread history projection.
- Treat a zero completion timestamp as absent, and leave timestamps unset for legacy builder events that do not provide them.

## Testing

- Cover projected item timestamps, including missing start times and zero completion times.

GitOrigin-RevId: 2187530f489ad3d86b6ab65faaaab1eab5357f1b
2026-07-28 00:14:22 +00:00
Owen Lin
4d1f66bf81 Preserve paginated thread metadata across resumes (#35678)
## Why

Paginated rollout history may contain only a bounded suffix. Deriving display
metadata from that suffix can replace the thread's original preview, title, and
first user message when the thread is read or resumed.

## What changed

- Use SQLite as the canonical source of display metadata for paginated thread
  reads, including reads by an explicit rollout path.
- Seed resume metadata tracking from the persisted values so appended history
  fills only missing fields instead of overwriting existing ones.
- Prefer an explicitly requested rollout path over a stale path stored in
  SQLite while retaining the canonical metadata.

## Testing

Added coverage for reads by ID and path, bounded-history resumes across app
server restarts, preservation of existing metadata, initialization of missing
fields, and stale persisted rollout paths.

GitOrigin-RevId: b393d0727afcba70fda7f50a6f2dcca0632745b4
2026-07-27 22:12:15 +00:00
Owen Lin
af7f6f4d34 Include item start times in completion events (#35363)
## What changed

- Add an optional `started_at_ms` field to `ItemCompletedEvent`, retaining
  compatibility with older persisted events that lack it.
- Track the first start timestamp for each in-flight item and attach it when
  emitting and persisting the completion event. If no start was recorded, use
  the completion timestamp as a fallback.
- Emit a complete start/completion lifecycle for subagent activity items.

## Testing

- Cover concurrent item timing, repeated starts, turn-boundary cleanup, the
  missing-start fallback, subagent activity, and persisted web-search events.

GitOrigin-RevId: e7cec9c4f1ef6ba67f287e81fb4d7d856fcf87a7
2026-07-25 16:06:54 +00:00
Owen Lin
05f000263b Support paginated thread forks (#35220)
## Why

`thread/fork` rejected threads using paginated history, so they could not use the same fork workflow as legacy threads.

## What changed

- Create paginated forks by referencing a frozen source-history prefix while persisting only child-owned records in the new rollout.
- Support latest, `lastTurnId`, and `beforeTurnId` boundaries, including inherited lineage, model-context reconstruction, interruption markers, and approval-reviewer inheritance.
- Make paginated reads and occurrence search traverse referenced history while excluding source records added after the fork.
- Coordinate fork preparation with archive and delete operations, and materialize compressed rollouts before they become fork references.

## Testing

Added app-server and thread-store coverage for boundary selection, active turns, inherited reads and search, compressed lineage, and concurrent source lifecycle operations.

GitOrigin-RevId: 66052d69517d7fb97a3b7817ab77a39c5ad995fd
2026-07-24 17:31:55 +00:00
Owen Lin
963316583b Enforce writer ownership for thread archive and deletion (#35031)
## Why

Paginated threads allow only one app-server process to write at a time. Archive and delete operations must not mutate a thread or its spawned descendants while another process owns any of them.

## What changed

- Acquire paginated writer locks for the full archive or deletion set before changing rollout files, including descendants whose rollout has not materialized yet.
- Add a batch archive store operation so ownership conflicts are detected before any thread in the subtree is archived.
- Return ownership conflicts as JSON-RPC invalid-request errors and document the behavior.

## Testing

- Cover archive and deletion conflicts for owned threads and descendants, including unmaterialized rollouts.
- Verify deletion still removes rollouts with unreadable metadata.

GitOrigin-RevId: f62a3e0c1ee2631cea48dd9145b588a1486e2425
2026-07-23 23:18:06 +00:00
richardopenai
b834702b27 Support incremental replay of updated thread items (#35013)
## Why

Persisted item snapshots can be updated after their initial projection while retaining their creation order. Callers need a way to read those updates incrementally without replaying every item.

## What changed

- Track the rollout ordinal of each item's latest snapshot separately from its creation ordinal.
- Let thread-store callers filter items after an exclusive update watermark and sort pages by either creation or update ordinal.
- Keep existing app-server reads creation-ordered, reject incremental reads across forked lineages, and preserve compatibility with older writers.

## Testing

- Cover update-watermark filtering, pagination, sort directions, per-turn reads, cursor validation, forked lineages, migration compatibility, and refreshed item snapshots.

GitOrigin-RevId: d57de849abec27f788e9c384dbff073a0aca1053
2026-07-23 21:17:45 +00:00
Adam Perry @ OpenAI
c769a05340 Honor the configured SQLite home across state consumers (#34994)
## Why

Codex and SQLite data can use separate home directories, but state consumers
could reconstruct database paths from the Codex home instead of consistently
using the resolved SQLite configuration.

## What changed

- Pass `SqliteConfig` through the core, rollout, state runtime, and thread store
  instead of passing a directory and rebuilding the configuration downstream.
- Use that shared configuration for state, logs, memories, goals, and paginated
  thread-history database access, including integrity checks and cleanup.
- Reject state database handles whose SQLite configuration does not match the
  requesting store.

## Testing

Add coverage with separate Codex and SQLite homes that verifies startup
backfill, thread listing, and paginated history all use the configured SQLite
directory.

GitOrigin-RevId: 1de1cdd1d6ff1d70bbb6c360c8352e6543fb8ebf
2026-07-23 19:19:35 +00:00
Owen Lin
5c94796dc9 Enforce single-writer ownership for paginated threads (#34986)
## Why

Only one app-server process should keep a paginated thread open for writing at
a time, while other processes must still be able to read it.

## What changed

- Acquire a per-thread filesystem lock when creating or resuming a paginated
  thread, and retain it for the lifetime of the live recorder.
- Reject a competing `thread/resume` request with JSON-RPC error `-32600` and
  release ownership when the writer is discarded, deleted, or shut down.
- Remove stale lock files without disturbing active writers.

## Testing

Added coverage for competing processes, ownership transfer after shutdown,
stale-lock cleanup, and lock release during discard and deletion.

GitOrigin-RevId: 81d49a2fffc73ba8135752b4944f27feafa4c0d6
2026-07-23 18:22:41 +00:00
joeytrasatti-openai
400ee190c3 Add persisted thread pinning to the app server (#34840)
## What changed

- Add `isPinned` to thread responses and allow `thread/metadata/update` to pin or unpin stored threads.
- Add an `isPinned` filter to `thread/list`, including cursor-based pagination and combinations with relationship filters.
- Persist pin state in SQLite without modifying rollout files, default existing threads to unpinned, and preserve pins through reconciliation and archive transitions.

## Testing

- Cover protocol serialization, database migration and indexing, pin updates, filtered pagination, rollout reconciliation, and archive/unarchive behavior.

GitOrigin-RevId: c2eacabd6dae465e2ae6ce8a169e00740960cdb7
2026-07-22 22:54:28 +00:00
Adam Perry @ OpenAI
946ed315a4 Centralize SQLite connection configuration (#34808)
## What changed

- Add `SqliteConfig` to own the resolved SQLite home, runtime database paths,
  and shared read/write and read-only pool settings.
- Pass the configuration through state and local thread-store consumers,
  replacing standalone path helpers and duplicated connection setup.

GitOrigin-RevId: 7351b08da94b5c8b6c0bbe492f86aeeca0699d3c
2026-07-22 18:45:47 +00:00
Owen Lin
5bfd74d36c Load paginated model context across rollout lineages (#34621)
## What changed

- Resolve the full rollout lineage when loading model context for paginated threads.
- Reverse-scan each lineage segment up to its recorded byte boundary, stopping at segment metadata and preserving the requested thread's canonical session metadata.
- Add a bounded reverse-scanner constructor for scanning a frozen JSONL prefix.

## Testing

- Cover frozen-prefix reverse scans and nested lineages whose root rollout has been archived.

GitOrigin-RevId: 8b97da43a263b7bd09cb7883450bb19eba5f195f
2026-07-21 21:58:41 +00:00
Owen Lin
0b175e6439 Protect fork history references during rollout cleanup (#34566)
## Why

Paginated forks can retain a `history_base` pointer to another rollout. Deleting or compressing either side of that relationship can leave the fork without the rollout history it references.

## What changed

- Add a rollout reference index that scans active, archived, and compressed rollout metadata.
- Reject deletion when a rollout is still referenced by a fork outside the requested deletion set.
- Delete spawned thread subtrees as one store operation so internal references do not block deleting the complete subtree, while external references preflight the request before any files are removed.
- Skip referenced rollouts, fork-pointer rollouts, and rollouts with unreadable metadata during background compression.

## Testing

- Cover reference indexing across active, archived, compressed, duplicate, and self-referencing rollouts.
- Verify deletion rejects external references without partially deleting a subtree and permits batch deletion of internally referenced threads.
- Verify compression preserves fork pointer chains and continues past unreadable metadata.

GitOrigin-RevId: a8672727f8706e743d7fb3ca895a1d9ceac98a3c
2026-07-21 15:18:56 +00:00
Owen Lin
7bb13ab846 Page through inherited thread history (#34563)
## What changed

- Resolve a thread's rollout lineage when listing paginated turns and items.
- Page across parent, child, and nested fork segments in either direction while respecting each segment's ordinal bounds.
- Encode the physical segment in history cursors and reject cursors that belong to another requested thread or fall outside its lineage.
- Load turn summaries and filtered items from inherited segments.

## Testing

- Added coverage for forward and backward pagination across parent and child history, inherited summary and filtered items, cursor isolation, and nested forks.

GitOrigin-RevId: b803fff781fcb6e21d0e151189290148fc8d8b18
2026-07-21 15:14:55 +00:00
Owen Lin
175f82147f Record rollout boundaries for materialized turns (#34562)
## What changed

- Store the starting byte offset and terminal ordinal and byte offset for each turn projected into SQLite.
- Advance offsets across blank and rejected physical lines while leaving incomplete trailing lines for the next materialization pass.
- Stop later rollout records from changing a turn after it reaches a terminal state.

## Testing

- Cover active and completed turn positions, trailing partial lines, skipped invalid lines, subagent history prefixes, and records received after terminal state.

GitOrigin-RevId: a7da2f0b00c0bf47b841db874c79a0d495c18acd
2026-07-21 15:10:42 +00:00
Owen Lin
b7e39aa316 Resolve paginated rollout lineages (#34407)
## What changed

- Add a shared local thread-store resolver that follows `history_base` links into ordered, bounded rollout segments, including archived ancestors.
- Support resolving a lineage at an explicit `HistoryPosition`.
- Reject malformed lineages with cycles, missing or mismatched source rollouts, non-paginated sources, or invalid cutoff bounds.

## Testing

- Add unit coverage for nested and archived lineages, explicit history positions, cycles, missing sources, and out-of-bounds offsets.

GitOrigin-RevId: a9063ad87e81d9865bd17fc588ea5d8e9ac74c0a
2026-07-20 20:33:51 +00:00
Owen Lin
2793c826e8 Enable memories for paginated threads (#34386)
## Why

Paginated threads keep metadata updates in SQLite, while their rollout retains the initial `memory_mode`. Reconciliation could therefore overwrite the current setting with a stale value, and memory processing excluded these threads entirely.

## What changed

- Include paginated threads in stage 1 memory job selection and global memory output lookup.
- Preserve the SQLite `memory_mode` when reconciling or backfilling an existing paginated thread, while continuing to seed missing rows and restore legacy threads from rollouts.
- Omit `memory_mode` alongside Git metadata when flushing resumed paginated history.

## Testing

Added coverage for preserving disabled memory mode during reconciliation and backfill, selecting eligible paginated threads, and omitting initial metadata on paginated resume.

GitOrigin-RevId: 2a6e16068e69680728757fbec27aeefae45b8110
2026-07-20 17:41:33 +00:00
Owen Lin
19b2273d8a Keep paginated thread Git metadata in SQLite (#34382)
## Why

Paginated rollouts contain only the initial Git metadata tuple. Reusing that
tuple during later reads, reconciliation, or resume can overwrite newer SQLite
values and undo explicit field clears.

## What changed

- Treat SQLite as authoritative for Git metadata once a thread is paginated.
- Apply paginated Git metadata patches directly to SQLite without appending
  compatibility metadata to the rollout.
- Preserve explicit null values when reading by rollout path or reconciling the
  initial session metadata, and ignore the initial Git tuple during resume.

## Testing

Added coverage for updating and clearing paginated Git metadata, preserving the
result through reconciliation and rollout-path reads, and avoiding stale Git
metadata updates on resume.

GitOrigin-RevId: 4d3e68cb24965b93d337d923092a98bac9b352b5
2026-07-20 17:23:18 +00:00
Brent Traut
5a208c1fc3 Persist names for paginated threads (#34229)
## Why

Paginated threads need explicit user-facing names to remain distinct from
derived titles and previews without requiring rollout metadata writes.

## What changed

- Add a nullable `name` column to persisted thread metadata and use it as the
  canonical name for paginated threads.
- Route paginated name updates through the state database while retaining the
  name index as best-effort compatibility data and preserving legacy thread
  behavior.
- Return persisted names when reading, listing, searching, and resuming
  paginated threads, and include them in state-database search matching.

## Testing

- Cover paginated name updates, reads, list/search results, and metadata-only
  resume behavior, including stale or unavailable compatibility data.

GitOrigin-RevId: 5ac4dfa2a83682b43d59ecc87e0f515580164dca
2026-07-19 21:13:13 +00:00
Owen Lin
bcdc695877 Track inherited paginated rollout prefixes (#33930)
## What changed

- Add `HistoryPosition` to identify a source thread and an exclusive rollout
  prefix by ordinal and JSONL byte offset.
- Add optional `history_base` metadata to `SessionMeta` so a thread can record
  an inherited prefix of another paginated rollout.
- Default missing `history_base` metadata to `None` and omit it when unset to
  preserve compatibility with existing rollout metadata.

## Testing

- Verify that legacy session metadata without `history_base` deserializes with
  no inherited history position.

GitOrigin-RevId: 3b2b07a2dfee793d3b37b0a60529f856f167dbc1
2026-07-18 00:06:49 +00:00
sayan-oai
15d7d2a733 Handle audio inputs and Bazel unit test arguments (#33929)
## What changed

- Exclude remote and local audio inputs from thread-history searchable text.
- Add a `unit_test_args` option to `codex_rust_crate` and forward it to the
  generated unit-test wrapper.

GitOrigin-RevId: f0aeb7f811163c3c80aa6f8ef4104650e698347c
2026-07-17 23:41:01 +00:00