151 Commits

Author SHA1 Message Date
andrewgu-oai
53cec04646 Optimize case-insensitive thread history matching (#39802)
## What changed

Map lowercase match offsets back to the original thread text with monotonic
span cursors. This avoids rescanning all character spans for every occurrence
while preserving the original byte ranges returned for matches.

GitOrigin-RevId: 363fba5d56813c3aeb2a7ede1d7275147a9ef51a
2026-08-20 21:50:51 +00:00
Owen Lin
90c67e6f33 Classify rollout migration failures (#39784)
## Why

Rollout migration outcomes expose free-form error messages, making failures hard to group without parsing text.

## What changed

- Add a serialized `RolloutMigrationFailureReason` to failed outcomes and classify failures across reading, conversion, SQLite materialization, publishing, and recovery.
- Add the failure reason as a low-cardinality tag on per-thread migration telemetry.

## Testing

- Cover missing SQLite metadata and invalid session metadata classifications.

GitOrigin-RevId: 2bff98483b484f4d02b72d52f39950ff99ebe429
2026-08-20 19:24:09 +00:00
rka-oai
763787d061 Support standalone named function call outputs (#39782)
## Why

External tool events may need to enter thread history without a preceding function call and therefore do not have a `call_id`.

## What changed

- Allow `function_call_output` items to omit `call_id` and carry optional `name` and `namespace` fields.
- Preserve named standalone outputs during history normalization and agent forks while retaining existing pairing behavior for outputs with a `call_id`.
- Accept, persist, and forward these outputs through `thread/inject_items`, and update the app-server schemas and documentation.

## Testing

- Cover paired and standalone JSON round trips, history normalization, agent forks, and injected thread history.

GitOrigin-RevId: a3258163a7dc93777c7c3023116fe204819bdbb0
2026-08-20 19:19:41 +00:00
joeflorencio-openai
ce950dcf26 Add managed developer instructions to requirements (#39755)
## What changed

- Add `additional_developer_instructions` to managed requirements and expose it through `configRequirements/read` as `additionalDeveloperInstructions`, independently of ordinary developer instructions.
- Include the managed instructions in model context, emit explicit replacement or removal messages when requirements change, and preserve the current value across compaction, resume, and agent forks without duplication.
- Reject managed instructions whose rendered context exceeds 10,000 estimated tokens.

## Testing

- Cover requirements layering and API serialization, context updates and removal, size validation, repeated model requests, compaction and resume, rollout migration, and agent forks.

GitOrigin-RevId: bc0b70fb7988944c2f68176dff55f5ed61eb46c8
2026-08-20 17:36:22 +00:00
Tamir Duberstein
db675cc005 Use stored item types when materializing turn summaries (#39514)
## What changed

- Select user and agent summary items using the materialized `item_type` column.
- Fall back to the type in `item_json` when `item_type` is empty so rows written by older clients still produce the correct summary.

## Testing

- Extend the summary materialization test to cover items inserted without a stored item type.

GitOrigin-RevId: 45ca1107900e9b8d46561cea210a27ecd661cedc
2026-08-19 16:20:21 +00:00
rka-oai
fb356f3d2c Add async delivery metadata to agent messages (#39312)
## What changed

- Add an optional `delivery` field to agent message events and app-server
  `agentMessage` items, with `"async"` identifying a user-visible message sent
  without ending the current turn.
- Preserve the marker through legacy event conversion, thread history
  materialization, replay, and generated JSON and TypeScript schemas.
- Require delivery of async agent message completion notifications on the
  in-process app-server transport.

## Testing

- Cover preservation of async delivery metadata in thread history.
- Cover delivery-required classification for async agent messages.

GitOrigin-RevId: f92ba25de1293ee271404badfdf0d117d6530329
2026-08-18 23:05:52 +00:00
Owen Lin
f950a1ba0b Preserve thread names during rollout migration (#39273)
## Why

Legacy threads and paginated threads read their displayed names from different metadata. Moving a rollout to paginated history must carry over its legacy-visible name so renamed threads do not lose it.

## What changed

- Load legacy indexed names once for an apply run.
- When promoting a thread, preserve an existing canonical name or copy its distinct legacy title, falling back to the legacy name index.
- Repair missing names when migration encounters an already-paginated rollout.

## Testing

Added migration tests covering preservation from both legacy name sources and repair on a rerun.

GitOrigin-RevId: 5fc526570b1cf358a159ef174c159ac091ccc40f
2026-08-18 20:07:26 +00:00
Jeremy Rose
fa9a05f2d2 Deduplicate rollout moves when archiving threads (#39256)
## Why

SQLite metadata can identify a rollout through a non-canonical path. When that
path resolves to a rollout already found on disk, archive and unarchive
operations could schedule the same source file to be moved twice.

## What changed

- Deduplicate archive and unarchive moves by canonical source path.
- Cover both operations with metadata-backed tests that use equivalent rollout
  paths.

GitOrigin-RevId: d190915264214413667fc2c83d360a7e8504d0aa
2026-08-18 18:38:12 +00:00
jif
a397079287 Preserve MCP resource origins across compaction (#39192)
## Why

Compaction can remove the tool-call events that associate an MCP app widget with
the tool, account, and URI needed to authorize later resource reads.

## What changed

- Store a bounded MCP resource-origin checkpoint with each compacted rollout.
- Restore the checkpoint when resuming a session so existing widgets remain
  readable after compaction and restart.
- Discard saved origins when rollback rewrites history across the compaction
  boundary, and reject oversized or invalid checkpoints during restoration.

## Testing

- Extend the app-server MCP resource-origin test to compact a paginated thread,
  continue the conversation, and read the original widgets before and after a
  restart.
- Verify rollback clears provenance that no longer matches retained history.

GitOrigin-RevId: 7f91c6202fe8d9be21c78dda42fa205937c44263
2026-08-18 12:37:41 +00:00
Shijie Rao
230791fd1f Persist active permission profiles in turn context (#39145)
## What changed

- Add the optional `active_permission_profile` field to `TurnContextItem` to record the built-in or named profile that produced the effective permission profile.
- Populate the field when serializing a turn context while preserving compatibility with records where it is absent.

## Testing

- Add coverage verifying that turn-context serialization stores the active permission profile.

GitOrigin-RevId: d69142ab1b26595ab406235f89818ef41370e963
2026-08-18 05:46:08 +00:00
Eric Traut
eeb82a156d Dispatch queued messages written by other processes (#39034)
## Why

Durable thread queues can be updated through another SQLite connection, but loaded idle threads need to notice those updates before they can dispatch the new messages.

## What changed

- Track a durable revision for each thread queue, including backfilling existing queues and updating revisions on inserts, updates, and deletes.
- Poll SQLite's data version and query revisions to find changed queues for loaded threads.
- Wake idle threads with pending external messages, discover queued work when threads are loaded or resumed, and retry each thread independently so one blocked queue does not stall others.

## Testing

- Cover cross-runtime queue writes, edits, independent dispatch, wake retries, and resumed threads.
- Cover migration backfills and revision tracking for queue updates and deletes.

GitOrigin-RevId: 906d902bbf6a760be206a1b2c08fd71427b63c46
2026-08-17 17:19:46 +00:00
Anton Panasenko
3b4569a920 Add experimental app-server project APIs (#38940)
## What changed

- Add SQLite-backed `project/list`, `project/read`, `project/create`,
  `project/import`, `project/update`, `project/move`, and `project/delete`
  endpoints with ordered roots, metadata, manual positioning, pagination, and
  idempotent creation.
- Add project assignment to thread start, metadata updates, list filtering,
  and fork inheritance. Project deletion clears assignments without deleting
  threads.
- Emit `project/changed` and `thread/project/updated` notifications after
  committed changes, and export the new protocol schemas and TypeScript types.

## Testing

- Cover project lifecycle and ordering, idempotency, atomic imports, cursor and
  filter validation, thread assignment, deletion, and fork inheritance.

GitOrigin-RevId: 47ae621be01ab8ef70a74a629fb29a5c5709ea33
2026-08-17 04:01:37 +00:00
Anton Panasenko
fcdae21073 Support metadata staging for reserved thread IDs (#38819)
## Why

Callers need to associate host-owned state with a thread before Core starts it.

## What changed

- Add `ThreadManager::reserve_thread_id` and allow new threads to start with the reserved ID. Reject reserved IDs when resuming an existing thread.
- Let thread stores stage metadata for a reserved ID. The local store merges staged values into the first successful metadata update and then removes them.
- Clear staged metadata when an unmaterialized thread shuts down or is discarded, and require the state database for staging.

## Testing

- Cover reserved ID generation and resume validation.
- Cover metadata persistence, merging, rollout-compatible updates, cleanup, and invalid staging requests.

GitOrigin-RevId: 3f9bc48f0f7764011cc90ea01349e497dda0a9e3
2026-08-16 00:57:28 +00:00
efrazer-oai
9341b38310 Add experimental thread queue APIs to app server (#38456)
## What changed

- Add experimental `thread/queue/add`, `list`, `update`, `delete`, `reorder`, and `start` requests for persistent queued user submissions.
- Automatically dispatch queued submissions in FIFO order after completed or failed turns, while leaving the queue paused after interruption.
- Emit `thread/queue/changed` notifications and expose paginated queue contents with stable submission and client message IDs.
- Export the new protocol types through the generated JSON Schema and TypeScript bindings.

## Testing

- Add app-server integration coverage for capability gating, CRUD, pagination, capacity limits, notifications, automatic dispatch, cold-thread resume, interruption, and explicit queue starts.

GitOrigin-RevId: 89caf5577192199bda77cf6c372a59d26cdd910e
2026-08-13 23:38:02 +00:00
Rasmus Rygaard
42bb50d502 Allow metadata updates without materializing threads (#38413)
## Why

Thread stores may filter a metadata patch to a no-op. Requiring every successful
update to return a `StoredThread` forces those implementations to materialize a
thread even when the caller does not need one.

## What changed

- Let `ThreadStore::update_thread_metadata` return `None` after a successful
  update that did not materialize a thread.
- Preserve the materialized-thread contract of `LiveThread` and `ThreadManager`
  metadata updates by reading the thread when the store returns `None`.
- Keep completion-only metadata updates from issuing that fallback read.

## Testing

Cover materialized in-memory updates and verify that fallback reads occur only
for callers that require the updated thread.

GitOrigin-RevId: 3059efc9d5b69e0b46e35fb5b5aa9638086654f2
2026-08-13 18:12:27 +00:00
Tamir Duberstein
2bd8727a0c Preserve floating-point values when decoding rollout lines (#38399)
## Why

Serde's generic buffering for flattened and internally tagged fields is incompatible with `serde_json`'s `arbitrary_precision` representation. As a result, persisted rollout items containing nested floating-point values can fail to decode.

## What changed

- Decode the rollout envelope at the JSON persistence boundary before deserializing the item payload.
- Use the shared decoder for both session resume and thread-history projection.
- Cover token-count floats, response-item metadata, field ordering, and ignored metadata in the decoder compatibility test.

GitOrigin-RevId: 59c7f6fbcf83162b5ed6980a69381fe45fc20a0a
2026-08-13 16:03:20 +00:00
jif
72fa74fbc9 Persist security risk scores in rollout history (#38363)
## What changed

- Add a `SecurityRiskScore` rollout item containing a category and numeric score.
- Persist the item in both thread history modes while excluding it from model context, user-visible thread history, search text, forks, and reconstructed conversation history.
- Re-export the score type from the extension API.

## Testing

- Cover serialization, persistence and loading, thread history projection, session reconstruction, append planning, and memory filtering.

GitOrigin-RevId: 1926fe366aeaa75052708a6da589f45a38eefb52
2026-08-13 12:04:29 +00:00
Owen Lin
b1373b74a2 Add durable reverts for paginated threads (#38292)
## What changed

- Add `ThreadStore::revert_thread` to retain history before a selected turn by creating a new immutable rollout and atomically switching the thread's stored rollout path.
- Preserve the logical thread ID and session metadata across repeated reverts.
- Track every rollout owned by a thread so archive, unarchive, and delete operations handle the complete lineage and restore moved files when metadata updates fail.
- Prevent deletion when forks still reference any rollout in the thread's lineage.

## Testing

- Cover repeated reverts, compressed source rollouts, lifecycle operations over reverted histories, reference-aware deletion, and cross-process unarchive locking.

GitOrigin-RevId: 1c05a6db3d0036ee91b562263d7ca200c6a0ea71
2026-08-13 01:56:24 +00:00
Owen Lin
8d4d57387a Resolve paginated thread history by rollout ID (#38244)
## Why

`thread/revert` keeps the logical thread ID while selecting a new immutable
rollout. Using the thread ID for persisted history can therefore read or update
the wrong rollout.

## What changed

- Centralize resolution of a thread's current rollout across live writers,
  SQLite metadata, and filesystem fallbacks.
- Key history materialization, lineage traversal, paging, turn lookup, and fork
  boundaries by immutable rollout ID while retaining the thread ID for the
  logical thread.
- Treat SQLite's selected rollout as authoritative for paginated threads so a
  filesystem scan cannot select an older rollout after `thread/revert`.

## Testing

Add a regression test showing that update-ordinal item paging reads from the
selected rollout ID when it differs from the thread ID.

GitOrigin-RevId: 1101e548a6e6c6d319996e24cb650e5b03ab7fbb
2026-08-12 20:31:52 +00:00
rka-oai
3a6f747d77 Preserve harness metadata across conversation history (#38058)
## What changed

- Wrap response items with optional harness-owned metadata while keeping the persisted response payload shape backward compatible.
- Store compacted-history metadata in an aligned sidecar and reject malformed sidecars.
- Carry metadata through history normalization, compaction, resume, fork, truncation, migration, and rollout recording without sending it to model providers.

## Testing

- Cover legacy rollout compatibility, metadata serialization and validation, compaction retention, and resume/fork reconstruction.

GitOrigin-RevId: 17a6127c9048ff984cb4df4d97c0751a38c29f5f
2026-08-11 19:30:12 +00:00
Tamir Duberstein
ed390a5dc4 Filter live rollout items in place (#38034)
## What changed

Retain persistable items directly in the owned append buffer instead of cloning
them into a second vector. Preserve the existing history-mode filtering and
empty-append behavior.

GitOrigin-RevId: 5300312545492bef28f63521427048a27d46a9d1
2026-08-11 16:27:58 +00:00
Tamir Duberstein
4496ba3fd5 Use session metadata to validate thread history paths (#38033)
## What changed

Validate a SQLite thread's rollout path by reading its `SessionMetaLine` and
comparing the recorded thread ID directly. This avoids constructing a
`StoredThread` solely to decide whether the path can supply the requested
history.

GitOrigin-RevId: 8eb1e329514cefc056e44ab4a76d58ff50314a97
2026-08-11 16:20:12 +00:00
arun eswara
edcec13372 Expose image generation usage-limit failures (#38024)
## What changed

- Add structured `usageLimitExceeded` metadata to failed image-generation
  items, including the image limit ID and optional reset timestamp.
- Preserve the failure metadata in completion events, app-server thread
  history, thread reads and resumes, and migrated legacy rollouts.
- Export the new failure type in the generated JSON and TypeScript schemas.

## Testing

- Cover usage-limit responses through item completion, persistence, and thread
  resume.
- Verify legacy rollout migration retains image-generation failure metadata.

GitOrigin-RevId: 91c31a2a47456b6b916ee03002ff314b05946dd4
2026-08-11 15:40:52 +00:00
Bryan Ashley
722784e936 Distinguish turn-start thread persistence (#37926)
## What changed

- Add `PersistContext` to the thread-store persistence contract so stores can
  identify persistence requested immediately before model sampling. A
  `TurnStart` request may be enqueued in the background when later flush and
  shutdown operations fence it and report any failure; all other persistence
  remains synchronous.
- Use the turn-start context only for the initial input of a regular turn, while
  admission acknowledgments, steered input, and other persistence paths keep
  the standard durability behavior.
- Flush pending metadata before turn-start persistence and ensure shutdown is
  still attempted if that metadata update fails.

## Testing

- Verify that developer instructions and user input are persisted before the
  model request completes for a default paginated-history thread.

GitOrigin-RevId: 162f0ec796a61aebe66ca0b909fbcccec3047b85
2026-08-11 03:35:45 +00:00
joeytrasatti-openai
1549756b78 Add appearance metadata to thread sections (#37898)
## What changed

- Add optional `icon` and `color` appearance fields to custom thread sections and expose them through the app-server protocol.
- Persist appearance metadata in SQLite and include it when listing sections or returning threads.
- Let `threadSection/update` preserve an omitted appearance, clear it with `null`, or replace it with a new value.
- Reject appearance fields larger than 64 bytes.

## Testing

- Cover protocol compatibility, persistence across restart, update and clear behavior, and field-length validation.

GitOrigin-RevId: 8f55ce4f42f8bd17aa5416848a85845f681953df
2026-08-10 23:32:42 +00:00
Adam Perry @ OpenAI
63002bdb26 Extract persisted history types into a dedicated crate (#37871)
## What changed

- Add `codex-history` for model-history and persisted-rollout domain types, including `RolloutItem`, `RolloutLine`, `CompactedItem`, and initial/resumed history state.
- Re-export the persisted types from `codex-rollout` and update consumers to use the new crate boundary instead of `codex-protocol`.
- Preserve existing rollout serialization, including legacy numeric compacted-window IDs.

## Testing

- Add `codex-history` tests for rollout JSON round trips, compacted-history compatibility, persisted history modes, and multi-agent version selection.

GitOrigin-RevId: 944daa9297ddd231d3aebbdcb05fff4adf8b4e1b
2026-08-10 19:26:52 +00:00
Owen Lin
4bb7ee3472 Add rollout migration tooling and background migration (#37348)
## What changed

- Add `codex migrate-rollouts` with dry-run inspection by default, explicit `--apply`, thread filtering, optional I/O throttling, progress output, and JSON or verbose reports.
- Add the disabled-by-default `background_paginated_rollout_migration` feature to migrate legacy local sessions at startup before rollout compression begins.
- Persist a migration cursor and skipped-file fingerprints so later startups avoid full rescans while retrying changed files and recovering pending migrations.
- Coordinate migration with live writers and emit metrics for manual and startup runs.

## Testing

- Cover startup cursor advancement and lookback, pending migration recovery, live-writer coordination, changed empty rollouts, and progress reporting.

GitOrigin-RevId: 276ac506c50ebec5140fd319faca1bb998172061
2026-08-07 01:26:42 +00:00
Brent Traut
547080e4d6 Prefer persisted cwd when reading local threads (#37198)
## Why

The `cwd` recorded in a rollout can become stale after thread metadata is
updated, causing thread reads and lists to disagree with the persisted state.

## What changed

- Overlay a non-empty state database `cwd` when reading a thread by ID.
- Apply the same overlay when reading by rollout path, but only when the
  metadata points to the requested rollout.
- Recompute the permission profile against the selected `cwd` and continue to
  fall back to rollout metadata when the persisted `cwd` is empty or belongs to
  another rollout.

## Testing

- Cover stale, empty, and mismatched persisted `cwd` metadata.
- Verify that `thread/read`, state-only `thread/list`, and `thread/resume`
  report the persisted thread `cwd` while the resumed session uses its requested
  live `cwd`.

GitOrigin-RevId: 50a3f187618ee3cadbad95196ee2976e2d5e9a5a
2026-08-06 02:26:24 +00:00
Owen Lin
aac9f84247 Preserve legacy semantics during rollout migration (#37191)
## Why

Legacy rollouts can contain historical rollbacks, compaction checkpoints, and
subagent copies of parent history. Migrating those records verbatim can change
the visible conversation or the model context used when a thread resumes, and
can retain large amounts of duplicated subagent history.

## What changed

- Plan and replay historical rollbacks by logical turn, preserving surviving
  lifecycle events and the correct compaction anchor for future resumes.
- Migrate subagent rollouts from a safe bounded replay when possible and record
  the history boundary so copied parent context is not projected as child turns.
- Include archived and compressed rollouts while keeping their storage form and
  location intact.
- Tolerate malformed, partial, oversized, and empty JSONL records during the
  migration scan.

## Testing

Add coverage for rollback and compaction combinations, subagent replay bounds,
archived and compressed rollouts, recovery, concurrent maintenance, and damaged
JSONL input.

GitOrigin-RevId: 3948e16d43f75df8571b0a92af12a181d1d08fdf
2026-08-06 00:54:14 +00:00
Owen Lin
6bb6e9045f Add legacy rollout migration to paginated history (#37175)
## What changed

- Add `LocalThreadStore::migrate_rollouts` with dry-run and apply modes,
  optional thread selection, throughput limiting, and per-rollout outcomes.
- Canonicalize legacy JSONL records into paginated history while preserving
  model-visible conversation context, including compressed rollouts and copied
  fork history.
- Publish replacements atomically and journal migrations so interrupted SQLite
  projections can be recovered. Coordinate migration with rollout compression
  and skip rollouts with active writers.
- Make paginated history a one-way SQLite promotion so stale metadata cannot
  downgrade a migrated thread.

## Testing

- Cover legacy record normalization, dry runs, idempotency, malformed input,
  compressed rollouts, active writers, interrupted migration recovery, and a
  cold app-server resume after migration.

GitOrigin-RevId: b9991b659f28ebb52da39ce62e8b4e0bae2ba2bb
2026-08-05 22:57:27 +00:00
efrazer-oai
b87981a510 Add durable per-thread user submission queues (#36952)
## What changed

- Add a storage-neutral `QueueStore` interface backed by a dedicated SQLite
  database for ordered, thread-scoped user submissions.
- Support enqueueing, paginated listing, editing, deletion, and atomic
  reordering, with a limit of 100 pending items per thread.
- Remove queued submissions when their thread is deleted.

## Testing

- Cover concurrent inserts, ordering and pagination, edits, deletion,
  cross-thread isolation, queue limits, and thread cleanup.

GitOrigin-RevId: 0a8e95ac71137afc6f3a039d4b5fc8603110c891
2026-08-04 19:23:29 +00:00
Owen Lin
dc60dadce6 Enforce single-writer ownership for all thread histories (#36389)
## Why

Legacy thread histories did not use the cross-process writer ownership guard already applied to paginated histories.

## What changed

- Acquire and retain a writer lock whenever a legacy or paginated thread is created or resumed.
- Apply the same ownership checks when archiving or deleting either history mode.
- Release ownership when initialization fails or the active thread shuts down, allowing another process to resume it.

## Testing

- Cover competing create and resume operations for both history modes.
- Cover archive and deletion conflicts for legacy and paginated descendants.

GitOrigin-RevId: 1be79b01966ad2634ed79eb96c0aa0db8eafe2fa
2026-07-31 20:49:08 +00:00
Owen Lin
332eac4b85 Load turn summaries with paginated queries (#36384)
## Why

Loading the summary view issued a separate item query for every returned turn.

## What changed

Join each turn's first user and final agent items into the paginated turn query
when `StoredTurnItemsView::Summary` is requested. Preserve the existing fallback
that resolves inherited summary items for synthetic fork-boundary rows.

GitOrigin-RevId: add7f2e03c0809bbee85dc9f07bb6f91af92dfe7
2026-07-31 20:12:28 +00:00
joeytrasatti-openai
c42ea41ee0 Add thread section management APIs (#36380)
## What changed

- Add `threadSection/create`, `threadSection/update`, and `threadSection/delete` app-server methods, with generated protocol schemas and TypeScript bindings.
- Persist custom sections in SQLite with stable UUIDv7 identities, trim and validate display names, and prevent renaming or deleting the built-in pinned section.
- Delete sections transactionally and return their active and archived threads to the unsectioned list.
- Serialize section mutations with section listing so concurrent requests observe ordered updates.

## Testing

- Cover section persistence across restarts, ordered renames, member cleanup on deletion, invalid requests, unsupported stores, and thread-store/state-runtime behavior.

GitOrigin-RevId: dc9a4be283c21dbc93ab1829eafc4825d8df6575
2026-07-31 19:08:45 +00:00
Owen Lin
6256a7ccc7 Make thread history projection resilient to malformed rollouts (#36188)
## Why

A failed rollout append can leave a rejected line before a valid same-ordinal
retry. Advancing the byte checkpoint past that line without also advancing the
ordinal checkpoint can prevent subsequent history from being projected.

## What changed

- Defer rejected and unknown rollout lines until a later valid ordinal shows
  whether they were replaced by a retry or consumed an ordinal.
- Advance skipped ordinal ranges together with the durable byte checkpoint,
  while rejecting gaps that cannot be explained by rejected lines.
- Prefer item event timestamps and parse the rollout timestamp only when an
  item needs it as a fallback.

## Testing

Added coverage for same-ordinal retries, inferred ordinal gaps, unknown and
malformed records, invalid fallback timestamps, and unexplained gaps.

GitOrigin-RevId: 374beda316c605d562bd9526abed19916598d69f
2026-07-30 16:49:12 +00:00
jif
88ec932e96 Handle fractional rate limits in thread history projection (#36083)
## Why

Paginated thread history could reject valid rollout records when rate-limit
percentages were encoded as floating-point JSON values, preventing later final
answers from being projected.

## What changed

Deserialize each rollout line through `serde_json::Value` before converting it
to `RolloutLine`, preserving floating-point rate-limit values during history
materialization.

## Testing

Added coverage for projecting fractional, integral, very small, and large
floating-point rate-limit percentages across both catch-up materialization and
subsequent history appends.

GitOrigin-RevId: 68da907987ffe59f4a3033ef90f3b88c39734cb1
2026-07-30 04:55:09 +00:00
joeytrasatti-openai
ad6fc66b6d Add persisted manual ordering for thread sections (#36007)
## What changed

- Add `thread/section/move` to atomically move a thread into, within, or out of a section. Threads can be inserted before an existing member or appended, and moves within a section preserve `sectionEnteredAt`.
- Add `section_position` sorting to `thread/list`, with ascending order as its default, and expose `sectionEnteredAt` in thread responses.
- Persist section positions and entry times in SQLite, including migration of existing section members into recency order. Section membership is no longer updated through `thread/metadata/update`.

## Testing

- Cover section moves, reordering, pagination, persistence across restarts and rollout reconciliation, concurrent updates, and rank renumbering.

GitOrigin-RevId: aec6d7ddedca5277029b5caf5c074975397e956c
2026-07-29 18:37:22 +00:00
Owen Lin
fa1d4c40d0 Gate paginated thread history on the state database (#35787)
## Why

Local thread stores without an initialized state database should not implicitly
create SQLite files or partially delete threads that have materialized history.

## What changed

- Report paginated history listing as unsupported when no state database is
  available, and skip history projection and materialization in that mode.
- Reject paginated forks without a state database.
- Validate access to materialized history before deleting rollout files, so a
  failed deletion preserves both the rollout and its history rows.

## Testing

Added coverage that a store without a state database creates no SQLite files
and preserves materialized history when deletion is unsupported. Existing
projection tests now initialize the state runtime explicitly.

GitOrigin-RevId: 2eaa0f5f0de8d5e4d84375ec50b310746a0ee68e
2026-07-28 16:00:39 +00:00
joeytrasatti-openai
85c6da1c79 Add persisted sections for organizing threads (#35722)
## What changed

- Replace the `isPinned` thread metadata and filters with an optional persisted
  `section` and `sectionId`.
- Add the paginated `threadSection/list` app-server method so clients can
  discover sections even when they contain no threads.
- Seed a stable `Pinned` section, validate section assignments, and support
  filtering for a specific section or for unsectioned threads.

## Testing

- Cover section protocol serialization, listing and pagination, metadata
  updates, filtering, persistence, migration compatibility, and operation
  without SQLite state.

GitOrigin-RevId: 7972b5471d29317b9387bfd90aa9f573f691ad4c
2026-07-28 05:26:16 +00:00
Owen Lin
8d6a91b1ff Preserve item timestamps in thread history projections (#35689)
## What changed

- Add optional start and completion timestamps to `ThreadHistoryItemChange`.
- Populate them from canonical `ItemCompleted` records during stateless thread history projection.
- Treat a zero completion timestamp as absent, and leave timestamps unset for legacy builder events that do not provide them.

## Testing

- Cover projected item timestamps, including missing start times and zero completion times.

GitOrigin-RevId: 2187530f489ad3d86b6ab65faaaab1eab5357f1b
2026-07-28 00:14:22 +00:00
Owen Lin
4d1f66bf81 Preserve paginated thread metadata across resumes (#35678)
## Why

Paginated rollout history may contain only a bounded suffix. Deriving display
metadata from that suffix can replace the thread's original preview, title, and
first user message when the thread is read or resumed.

## What changed

- Use SQLite as the canonical source of display metadata for paginated thread
  reads, including reads by an explicit rollout path.
- Seed resume metadata tracking from the persisted values so appended history
  fills only missing fields instead of overwriting existing ones.
- Prefer an explicitly requested rollout path over a stale path stored in
  SQLite while retaining the canonical metadata.

## Testing

Added coverage for reads by ID and path, bounded-history resumes across app
server restarts, preservation of existing metadata, initialization of missing
fields, and stale persisted rollout paths.

GitOrigin-RevId: b393d0727afcba70fda7f50a6f2dcca0632745b4
2026-07-27 22:12:15 +00:00
Owen Lin
af7f6f4d34 Include item start times in completion events (#35363)
## What changed

- Add an optional `started_at_ms` field to `ItemCompletedEvent`, retaining
  compatibility with older persisted events that lack it.
- Track the first start timestamp for each in-flight item and attach it when
  emitting and persisting the completion event. If no start was recorded, use
  the completion timestamp as a fallback.
- Emit a complete start/completion lifecycle for subagent activity items.

## Testing

- Cover concurrent item timing, repeated starts, turn-boundary cleanup, the
  missing-start fallback, subagent activity, and persisted web-search events.

GitOrigin-RevId: e7cec9c4f1ef6ba67f287e81fb4d7d856fcf87a7
2026-07-25 16:06:54 +00:00
Owen Lin
05f000263b Support paginated thread forks (#35220)
## Why

`thread/fork` rejected threads using paginated history, so they could not use the same fork workflow as legacy threads.

## What changed

- Create paginated forks by referencing a frozen source-history prefix while persisting only child-owned records in the new rollout.
- Support latest, `lastTurnId`, and `beforeTurnId` boundaries, including inherited lineage, model-context reconstruction, interruption markers, and approval-reviewer inheritance.
- Make paginated reads and occurrence search traverse referenced history while excluding source records added after the fork.
- Coordinate fork preparation with archive and delete operations, and materialize compressed rollouts before they become fork references.

## Testing

Added app-server and thread-store coverage for boundary selection, active turns, inherited reads and search, compressed lineage, and concurrent source lifecycle operations.

GitOrigin-RevId: 66052d69517d7fb97a3b7817ab77a39c5ad995fd
2026-07-24 17:31:55 +00:00
Owen Lin
963316583b Enforce writer ownership for thread archive and deletion (#35031)
## Why

Paginated threads allow only one app-server process to write at a time. Archive and delete operations must not mutate a thread or its spawned descendants while another process owns any of them.

## What changed

- Acquire paginated writer locks for the full archive or deletion set before changing rollout files, including descendants whose rollout has not materialized yet.
- Add a batch archive store operation so ownership conflicts are detected before any thread in the subtree is archived.
- Return ownership conflicts as JSON-RPC invalid-request errors and document the behavior.

## Testing

- Cover archive and deletion conflicts for owned threads and descendants, including unmaterialized rollouts.
- Verify deletion still removes rollouts with unreadable metadata.

GitOrigin-RevId: f62a3e0c1ee2631cea48dd9145b588a1486e2425
2026-07-23 23:18:06 +00:00
richardopenai
b834702b27 Support incremental replay of updated thread items (#35013)
## Why

Persisted item snapshots can be updated after their initial projection while retaining their creation order. Callers need a way to read those updates incrementally without replaying every item.

## What changed

- Track the rollout ordinal of each item's latest snapshot separately from its creation ordinal.
- Let thread-store callers filter items after an exclusive update watermark and sort pages by either creation or update ordinal.
- Keep existing app-server reads creation-ordered, reject incremental reads across forked lineages, and preserve compatibility with older writers.

## Testing

- Cover update-watermark filtering, pagination, sort directions, per-turn reads, cursor validation, forked lineages, migration compatibility, and refreshed item snapshots.

GitOrigin-RevId: d57de849abec27f788e9c384dbff073a0aca1053
2026-07-23 21:17:45 +00:00
Adam Perry @ OpenAI
c769a05340 Honor the configured SQLite home across state consumers (#34994)
## Why

Codex and SQLite data can use separate home directories, but state consumers
could reconstruct database paths from the Codex home instead of consistently
using the resolved SQLite configuration.

## What changed

- Pass `SqliteConfig` through the core, rollout, state runtime, and thread store
  instead of passing a directory and rebuilding the configuration downstream.
- Use that shared configuration for state, logs, memories, goals, and paginated
  thread-history database access, including integrity checks and cleanup.
- Reject state database handles whose SQLite configuration does not match the
  requesting store.

## Testing

Add coverage with separate Codex and SQLite homes that verifies startup
backfill, thread listing, and paginated history all use the configured SQLite
directory.

GitOrigin-RevId: 1de1cdd1d6ff1d70bbb6c360c8352e6543fb8ebf
2026-07-23 19:19:35 +00:00
Owen Lin
5c94796dc9 Enforce single-writer ownership for paginated threads (#34986)
## Why

Only one app-server process should keep a paginated thread open for writing at
a time, while other processes must still be able to read it.

## What changed

- Acquire a per-thread filesystem lock when creating or resuming a paginated
  thread, and retain it for the lifetime of the live recorder.
- Reject a competing `thread/resume` request with JSON-RPC error `-32600` and
  release ownership when the writer is discarded, deleted, or shut down.
- Remove stale lock files without disturbing active writers.

## Testing

Added coverage for competing processes, ownership transfer after shutdown,
stale-lock cleanup, and lock release during discard and deletion.

GitOrigin-RevId: 81d49a2fffc73ba8135752b4944f27feafa4c0d6
2026-07-23 18:22:41 +00:00
joeytrasatti-openai
400ee190c3 Add persisted thread pinning to the app server (#34840)
## What changed

- Add `isPinned` to thread responses and allow `thread/metadata/update` to pin or unpin stored threads.
- Add an `isPinned` filter to `thread/list`, including cursor-based pagination and combinations with relationship filters.
- Persist pin state in SQLite without modifying rollout files, default existing threads to unpinned, and preserve pins through reconciliation and archive transitions.

## Testing

- Cover protocol serialization, database migration and indexing, pin updates, filtered pagination, rollout reconciliation, and archive/unarchive behavior.

GitOrigin-RevId: c2eacabd6dae465e2ae6ce8a169e00740960cdb7
2026-07-22 22:54:28 +00:00
Adam Perry @ OpenAI
946ed315a4 Centralize SQLite connection configuration (#34808)
## What changed

- Add `SqliteConfig` to own the resolved SQLite home, runtime database paths,
  and shared read/write and read-only pool settings.
- Pass the configuration through state and local thread-store consumers,
  replacing standalone path helpers and duplicated connection setup.

GitOrigin-RevId: 7351b08da94b5c8b6c0bbe492f86aeeca0699d3c
2026-07-22 18:45:47 +00:00
Owen Lin
5bfd74d36c Load paginated model context across rollout lineages (#34621)
## What changed

- Resolve the full rollout lineage when loading model context for paginated threads.
- Reverse-scan each lineage segment up to its recorded byte boundary, stopping at segment metadata and preserving the requested thread's canonical session metadata.
- Add a bounded reverse-scanner constructor for scanning a frozen JSONL prefix.

## Testing

- Cover frozen-prefix reverse scans and nested lineages whose root rollout has been archived.

GitOrigin-RevId: 8b97da43a263b7bd09cb7883450bb19eba5f195f
2026-07-21 21:58:41 +00:00