Commit Graph

174 Commits

Author SHA1 Message Date
Can Sar
728cb12fe5 Expose thread originators through the app-server API (#42458)
## What changed

- Add the creation-time `originator` to app-server thread responses and `thread/started` notifications.
- Persist originators in thread metadata and preserve the first recorded value across list, read, resume, rollout, and SQLite paths.
- Add an `originators` allowlist to `thread/list` for hosted backends. The local app-server accepts omitted or empty values but rejects nonempty filters.

## Testing

- Cover originator persistence across server restarts and list, read, and resume requests.
- Verify local `thread/list` handling for nonempty, omitted, null, and empty originator filters.
- Heap-allocate large TUI dispatcher futures in affected tests to keep them within the Windows test-thread stack.

GitOrigin-RevId: 73ad6e61b8fdd2d93a22d1f971723a19ec7be3fc
2026-09-03 06:02:25 +00:00
Owen Lin
69cebb5d15 Route rollout reads through the canonical JSON decoder (#42378)
## Why

Directly deserializing the flattened `RolloutLine` envelope can reject nested
decimal values, preventing affected paginated sessions from resuming.

## What changed

- Add canonical string, byte, and reverse-scanner helpers that decode rollout
  records through `serde_json::Value` before decoding the flattened item.
- Route rollout readers across session discovery, history, migration, search,
  thread storage, and transcript previews through those helpers.
- Remove `Deserialize` from `RolloutLine` so new readers cannot bypass the
  canonical persistence decoder.

## Testing

Add coverage that resumes a paginated rollout after a token-count record with
a decimal rate-limit value and verifies that ordinal sequencing continues.

GitOrigin-RevId: 49abac1e0751c073daa5a93a840d8a483fd2d013
2026-09-02 22:45:19 +00:00
dhruvgupta-oai
5037919777 Add Luna Reserve usage fallback to the TUI (#42372)
## Why

When ordinary included usage is exhausted, eligible users need a way to keep working with the backend-provided Luna Reserve quota and return to their previous model once ordinary usage recovers.

## What changed

- Automatically switch eligible TUI tasks to Luna Reserve after an authoritative account usage read, preserving the prior model and reasoning effort per task.
- Restrict model selection while on Reserve to its supported reasoning levels, keep queued turns on the accepted model, and restore the saved model after a fresh read confirms recovery.
- Surface Reserve usage in the composer and `/status`, and expose `normalModelSlug` in app-server rate-limit snapshots so clients can use the associated model's display metadata without changing the request model.
- Poll account limits more frequently near exhaustion and retain compatibility with app servers that only accept the legacy null request parameters.

## Testing

- Added coverage for Reserve entry, queued-turn handling, task reconstruction, recovery authorization, account changes, model selection, usage rendering, and banner dismissal.

GitOrigin-RevId: 46a435e9531051f8cec3f6a46c4b9f7cfa479747
2026-09-02 21:50:51 +00:00
Owen Lin
095ac4f131 Keep SQLite history projection moving past invalid records (#42369)
## Why

A malformed or discontinuous rollout record can otherwise block later valid
thread history from being materialized in SQLite.

## What changed

- Skip malformed JSON, unknown records, missing or regressed ordinals, ordinal
  gaps, and invalid timestamps while advancing the projection checkpoint.
- Keep unknown-record ordinals available for a subsequent valid retry.
- Record projection outcomes and anomaly kinds with OpenTelemetry counters.

## Testing

- Cover malformed lines, missing, duplicate, and forward-gap ordinals, and valid
  retries after unknown records.

GitOrigin-RevId: 9ae616b44513e35a1fa209e54f05f0612abcbd86
2026-09-02 21:29:57 +00:00
felixxia-oai
8e3b180d49 Preserve retained answers across steer rollbacks (#42298)
## Why

A steer creates a new instruction boundary without creating a new turn ID. Retained `request_user_input` answers that share that turn ID must not all be discarded when only the later steer is rolled back.

## What changed

- Associate retained answers with their originating function call and instruction boundary.
- Keep answers whose source calls survive rollback, including through compaction, resume, and rollout migration.
- Remove answers whose source calls belong to the rolled-back instruction.

## Testing

- Cover live rollback and resume with answers before and after a steer.
- Cover rollout migration when delayed answers share a turn ID but originate from different calls.

GitOrigin-RevId: 225e7eb7b6a3ae5a5e7fa6fcb80cb7d259db90d0
2026-09-02 15:46:33 +00:00
felixxia-oai
5971d42847 Preserve verified answers across history compaction (#42293)
## Why

Compaction replaces the model history window, but host-verified user answers may
still be needed for the rest of the owning thread.

## What changed

- Add bounded, model-invisible retained context for verified
  `request_user_input` answers.
- Persist retained context in rollout events and compaction checkpoints, and
  restore it when a thread resumes.
- Remove answers when their source turns are rolled back, and prevent
  parent-local answers from being inherited by child forks.

## Testing

Add lifecycle coverage for deduplication and size limits, compaction and resume,
rollback, rollout migration, and child forks.

GitOrigin-RevId: c32900d3e8120a9899b71f0576683acd67a4b898
2026-09-02 15:07:11 +00:00
rka-oai
2c79ee6dac Add structured asynchronous user input requests (#42178)
## What changed

- Replace `send_user_message_async` with `request_user_input_async`, accepting one or more questions with optional suggested answers while allowing the turn to continue.
- Attach structured question metadata to asynchronous agent messages while retaining readable fallback text, and preserve it through app-server events, thread history, and generated schemas.
- Continue enabling the new tool for model catalogs that advertise either the old or new tool name.

## Testing

- Cover tool registration, question validation, emitted items, continued turn execution, history preservation, and backwards-compatible deserialization.

GitOrigin-RevId: ffc49aebde2e854c3c50a122aa52805fec3fc6b3
2026-09-02 02:09:35 +00:00
Brent Traut
986ff1cc7c Expose model settings in app-server thread metadata (#42151)
## What changed

- Add nullable `model` and `reasoningEffort` fields to the shared app-server `Thread` object and generated schemas.
- Report current settings for loaded threads and the latest persisted settings for unloaded threads across thread read, list, start, resume, rollback, metadata update, and notification paths.
- Keep unavailable legacy or filesystem-only settings nullable, and allow metadata reads without loading a thread or dispatching queued work.

## Testing

- Cover live-setting precedence, persisted settings after restart, null legacy values, and model metadata returned by thread lifecycle operations.

GitOrigin-RevId: 22ebe8a8d1910a820d90a97730d812a00d8c1372
2026-09-01 22:56:18 +00:00
Owen Lin
c5b5fa8089 Support thread forks from symlinked session roots (#42135)
## Why

Forking a resumed paginated thread failed when the managed `sessions` directory was a symlink, even though the rollout remained under that managed root.

## What changed

- Validate rollout lineage references against the canonical `sessions` and `archived_sessions` roots.
- Accept rollouts beneath symlinked managed roots while rejecting nested symlinks that escape those roots.

## Testing

- Cover forking a resumed rollout under a symlinked `sessions` root.
- Cover rejection of rollout lineage paths that escape through a nested symlink.

GitOrigin-RevId: 0128ecde7087120ef510baa3ed62cdbb33073117
2026-09-01 21:03:30 +00:00
jif
28097e98eb Preserve Guardian history across thread reconstruction (#42065)
## Why

Guardian review evidence must remain available after a compacted thread is resumed or forked, while rolled-back or parent-local evidence must not grant authorization in a different context.

## What changed

- Store the bounded, model-invisible Guardian transcript alongside compacted rollout history and restore it from the newest surviving checkpoint during replay.
- Trim Guardian history at rollback boundaries, clearing it when the boundary has already been evicted.
- Preserve the checkpoint for user-initiated forks, but remove it when spawning a subagent so parent review evidence is not inherited.
- Keep the new rollout field optional for compatibility with existing compacted records and legacy readers.

## Testing

- Cover Guardian history across compaction, restart, paginated and pathless stores, user forks, rollback, bounded replay, serialization, and subagent forks.

GitOrigin-RevId: 6ab076d5e0c3aac4d6a41ebd50ba2b200c90213e
2026-09-01 14:33:58 +00:00
Owen Lin
5f79a92e39 Persist response token usage in rollout history (#41912)
## Why

Resumed threads need to continue per-turn and per-thread token totals without scanning arbitrarily far beyond the latest compaction checkpoint.

## What changed

- Add durable `TokenUsageRecord` rollout items with response, turn, thread, session, and root-turn attribution.
- Restore accumulated usage on resume and snapshot the latest record plus the compaction response ID in compaction checkpoints.
- Preserve root-turn lineage in persisted turn context, while ensuring forked child threads start with their own usage totals.

## Testing

- Cover usage accumulation across multiple responses and resumed turns.
- Cover local and remote compaction checkpoints, invalid remote compaction output, rollout reconstruction, and fork isolation.

GitOrigin-RevId: ef9e0c4a9102a08a2c382be4cdac68c84353c90a
2026-08-31 21:15:38 +00:00
Eric Traut
2f0a5d5516 Avoid scanning archived rollouts when archiving threads (#41908)
## Why

Archiving a thread only needs to locate rollout files that have not already
been moved, so reading the entire rollout archive on every request is
unnecessary.

## What changed

- Add `RolloutReferenceIndex::scan_unarchived` to scan only the active
  `sessions` directory.
- Use the active-only index when archiving threads while retaining full scans
  for operations that require archived-history reference counts.
- Cover discovery of compressed and uncompressed active rollouts and exclusion
  of archived and unrelated rollouts.

GitOrigin-RevId: 46367127578f20bfa46a8db5c6f2b7f59d29f7a9
2026-08-31 20:40:05 +00:00
Bryan Ashley
f88ff940c0 Load bounded context after empty wake turns (#41901)
## Why

Paginated threads can accumulate wake turns with no user-message boundary. The
reverse context scan could not use those turns as a bounded replay cutoff, even
when a compaction and its resume state provided everything needed to reconstruct
the latest model context.

## What changed

Treat a full world-state snapshot that survives the latest compaction as a
durable baseline when it is paired with a compatible turn context. This lets the
scan stop at empty wake turns while still scanning further back for patches,
snapshots before compaction, or missing and mismatched turn contexts.

## Testing

Add coverage for repeated empty wake turns and verify that bounded replay
matches full rollout reconstruction across successive compaction windows.

GitOrigin-RevId: ae90c5f939c660ea539fbfc11128cc7259efaa9c
2026-08-31 19:30:05 +00:00
rhan-oai
f5636bb733 Restore thread cwd from owned settings snapshots (#41567)
## Why

Resuming a thread without an explicit `cwd` should restore that thread's latest
retained setting. Forked history can contain settings copied from another
thread, while compaction can move the latest setting outside the replay window.

## What changed

- Record the owning thread ID on new settings snapshots and only use snapshots
  owned by the resumed thread when restoring `cwd`. Older snapshots without an
  owner remain readable but do not override the startup `cwd`.
- Checkpoint the current settings after compaction and serialize checkpoints
  with settings updates so the retained history contains an accepted, current
  snapshot.
- Cover resume behavior across compaction, forks, reverts, legacy histories,
  and concurrent settings updates.

GitOrigin-RevId: 1efc9cf55472d6b88c465c4efe44672a232dbf64
2026-08-29 19:57:06 +00:00
jif
1cc81ca89a Support compression for shared rollout lineages (#41357)
## Why

Rollout compression skips files that participate in paginated fork lineages because lineage readers rely on byte offsets into the original JSONL. This prevents shared rollout history from being compressed.

## What changed

- Add a seekable rollout reader that preserves logical JSONL offsets for plain and zstd-compressed files without materializing compressed data beside the rollout.
- Use the reader for lineage validation, model-context replay, and SQLite history projection, including when compression races with a read.
- Add the opt-in `local_thread_store_shared_compression` feature. The default mode continues to leave shared lineages uncompressed for compatibility, while the new mode includes referenced sources and fork pointers.
- Record the uncompressed size in new zstd frames so common prefix-bound checks can avoid decoding the full file.

## Testing

Add coverage for seekable compressed reads, prefix validation across zstd frame shapes, compressed lineage replay and projection, concurrent forks, and compression-mode behavior.

GitOrigin-RevId: 83e519f7eabf7448977eff37c55899858cc17a07
2026-08-28 16:04:26 +00:00
Anton Panasenko
f6726e8986 Add recency sorting to project/list (#41223)
## What changed

- Include `recencyAt`, derived from the newest non-archived assigned thread, in project responses.
- Let `project/list` sort by `position` or `recencyAt` in either direction, with recency as the default descending order for that key and projects without activity placed last.
- Encode the sort key, direction, and millisecond-precision anchor in new cursors while retaining existing ascending-position cursors.
- Add an index for project recency lookups and document the API's sorting and pagination behavior.

## Testing

- Cover recency updates when threads are assigned, archived, unarchived, or deleted.
- Cover ordering, null handling, tie-breaking, cursor pagination, invalid sort combinations, and cursor mismatches.

GitOrigin-RevId: 6165e2b9976b0b45cfa16bcd42978b67811f6df5
2026-08-27 23:43:46 +00:00
rka-oai
e56e4922eb Support standalone tool outputs in turn/start (#41002)
## What changed

- Add an optional `toolOutput` to `turn/start` for starting or steering a turn with a named function-call output instead of user input.
- Emit standalone outputs as `functionCallOutput` thread items, persist them in durable history, and include them in resumed threads.
- Validate that tool outputs have a name, are not combined with nonempty `input`, satisfy the text-size limit, and use supported image URLs.

## Testing

- Cover standalone tool-output notifications, model input, and resumed history in both legacy and paginated history modes.
- Cover the text-size limit for structured tool output.

GitOrigin-RevId: a1eeec7b9b7de67b1a1c5e829a54552b85ec118b
2026-08-26 23:58:10 +00:00
Brandon Zhang
0d654e6536 Track window and fork positions in turn metadata (#40987)
## What changed

- Add the zero-based `window_number` to Responses turn metadata and include it in prewarm requests.
- Persist `forked_from_ordinal_exclusive` separately from the physical history base so fork lineage remains accurate after reverts and cold resumes.
- Report context-inheriting subagents through `parent_thread_id` without also emitting fork lineage.
- Reserve the new metadata keys while accepting and filtering previously valid configured values so Codex-owned values take precedence.

## Testing

- Cover window numbering across compaction, resume, and fork operations.
- Cover fork cutoff persistence across direct forks, legacy rollouts, reverts into inherited history, and cold resumes.

GitOrigin-RevId: 288f6cd632254814c1f166668b33828082d799a1
2026-08-26 22:37:59 +00:00
Chris Dong
346c4db7c2 Track image generation request IDs in analytics (#40714)
## What changed

- Read `x-codex-imagegen-request-id` from image generation and edit responses.
- Propagate the ID to `codex_image_generation_event` analytics.
- Keep the ID in process only, excluding it from extension item serialization, generated types, app-server wire data, and rollout history.

## Testing

- Cover response-header extraction and missing-header behavior in the images client.
- Verify end-to-end analytics emission and the unchanged image-generation item wire shape.

GitOrigin-RevId: 9b8e09b7d655c0c2faa84f0eee99aabae583668d
2026-08-25 21:38:10 +00:00
Adam Perry @ OpenAI
1bc02aea50 Sanitize credentials from Git remote metadata (#40713)
## Why

Git remote URLs can embed usernames, passwords, or tokens. Codex carries these
URLs through turn metadata and persisted thread metadata, so credentials must be
removed before a remote enters those paths.

## What changed

- Add `SanitizedGitUrl`, which parses Git URL and SCP-style remotes, strips
  authentication data, and preserves the conventional `git` SSH user.
- Use sanitized remotes when collecting Git metadata, enriching model requests,
  discovering cloud environments, updating thread metadata, and reading legacy
  rollouts. Reject or omit malformed remotes instead of retaining their raw
  contents.
- Keep API, schema, and TypeScript representations as strings while enforcing
  sanitization in Rust.

## Testing

- Cover URL schemes, SCP and IPv6 forms, remote helpers, encoded paths,
  malformed values, and legacy deserialization.
- Verify credentials do not appear in model requests, API responses, SQLite, or
  rollout files.

GitOrigin-RevId: 6435efc4c45bfbfad4723ce7a0457cb00175f25c
2026-08-25 21:34:05 +00:00
faizan-oai
94967e03e5 Add per-turn cyber access program selection (#40687)
## What changed

- Add the experimental `turn/start.cyberAccessProgram` option with `standard`,
  `daybreakBlue`, and `daybreakRed` values.
- Forward the selection as `access_programs.cyber` on Responses, WebSocket,
  and remote-compaction requests made with ChatGPT authentication, while
  omitting it for API-key and custom-provider requests.
- Preserve the per-turn selection across recovery, compaction, and child-agent
  turns without making it a persistent thread setting.

## Testing

- Cover app-server forwarding, authentication boundaries, WebSocket reuse,
  turn recovery, compaction, and child-agent inheritance.

GitOrigin-RevId: d2eb468f365b2214c5099bc21741cf30f8bd2eb5
2026-08-25 19:54:52 +00:00
guinness-oai
41ca88e95c Persist realtime events in the thread timeline (#40508)
## Why

Clients need a bounded view of realtime conversations that preserves the order of speech, agent work, and turn lifecycle events without loading the full thread history.

## What changed

- Persist realtime session boundaries, transcript segments, and promoted agent items in paginated thread rollouts.
- Add the experimental `thread/timeline/list` API to page ordinary items, realtime items, and turn boundaries in canonical rollout order. Include `activeRealtimeSessionAtPageStart` so each page can be rendered independently.
- Emit typed `thread/realtime/item/started`, `thread/realtime/item/transcript/delta`, and `thread/realtime/item/completed` notifications around durable realtime items.
- Keep the existing `thread/items/list` response and legacy thread history behavior unchanged.

## Testing

- Cover mixed timeline pagination, realtime-session state at page boundaries, rollout replay, subagent history prefixes, transcript streaming, steering, and promoted agent artifacts.

GitOrigin-RevId: faa2c420192d63e060e8fb32446e3c797422656e
2026-08-25 01:03:20 +00:00
Owen Lin
465eafacbc Harden startup rollout migration against concurrent updates (#40499)
## Why

Another Codex process can write, archive, or compress a rollout while startup
migration is inspecting it. This can leave a discovered path stale or make an
in-progress rollout look empty or busy.

## What changed

- Wait for rollout maintenance to finish before starting background migration.
- Re-read empty rollouts under their writer lock and retry busy rollouts on a
  later startup.
- Find archived or compressed rollouts again when their paths change after
  discovery, while keeping terminal failures from blocking the startup cursor.

## Testing

Added coverage for maintenance contention, writer-owned empty rollouts, busy
rollout retries after archive and compression moves, permanent failure skips,
pending recovery, and paths that move after discovery.

GitOrigin-RevId: 9708f212f2bbff88bd47a01ec4872bf6ce9af535
2026-08-25 00:03:23 +00:00
andrewgu-oai
53cec04646 Optimize case-insensitive thread history matching (#39802)
## What changed

Map lowercase match offsets back to the original thread text with monotonic
span cursors. This avoids rescanning all character spans for every occurrence
while preserving the original byte ranges returned for matches.

GitOrigin-RevId: 363fba5d56813c3aeb2a7ede1d7275147a9ef51a
2026-08-20 21:50:51 +00:00
Owen Lin
90c67e6f33 Classify rollout migration failures (#39784)
## Why

Rollout migration outcomes expose free-form error messages, making failures hard to group without parsing text.

## What changed

- Add a serialized `RolloutMigrationFailureReason` to failed outcomes and classify failures across reading, conversion, SQLite materialization, publishing, and recovery.
- Add the failure reason as a low-cardinality tag on per-thread migration telemetry.

## Testing

- Cover missing SQLite metadata and invalid session metadata classifications.

GitOrigin-RevId: 2bff98483b484f4d02b72d52f39950ff99ebe429
2026-08-20 19:24:09 +00:00
rka-oai
763787d061 Support standalone named function call outputs (#39782)
## Why

External tool events may need to enter thread history without a preceding function call and therefore do not have a `call_id`.

## What changed

- Allow `function_call_output` items to omit `call_id` and carry optional `name` and `namespace` fields.
- Preserve named standalone outputs during history normalization and agent forks while retaining existing pairing behavior for outputs with a `call_id`.
- Accept, persist, and forward these outputs through `thread/inject_items`, and update the app-server schemas and documentation.

## Testing

- Cover paired and standalone JSON round trips, history normalization, agent forks, and injected thread history.

GitOrigin-RevId: a3258163a7dc93777c7c3023116fe204819bdbb0
2026-08-20 19:19:41 +00:00
joeflorencio-openai
ce950dcf26 Add managed developer instructions to requirements (#39755)
## What changed

- Add `additional_developer_instructions` to managed requirements and expose it through `configRequirements/read` as `additionalDeveloperInstructions`, independently of ordinary developer instructions.
- Include the managed instructions in model context, emit explicit replacement or removal messages when requirements change, and preserve the current value across compaction, resume, and agent forks without duplication.
- Reject managed instructions whose rendered context exceeds 10,000 estimated tokens.

## Testing

- Cover requirements layering and API serialization, context updates and removal, size validation, repeated model requests, compaction and resume, rollout migration, and agent forks.

GitOrigin-RevId: bc0b70fb7988944c2f68176dff55f5ed61eb46c8
2026-08-20 17:36:22 +00:00
Tamir Duberstein
db675cc005 Use stored item types when materializing turn summaries (#39514)
## What changed

- Select user and agent summary items using the materialized `item_type` column.
- Fall back to the type in `item_json` when `item_type` is empty so rows written by older clients still produce the correct summary.

## Testing

- Extend the summary materialization test to cover items inserted without a stored item type.

GitOrigin-RevId: 45ca1107900e9b8d46561cea210a27ecd661cedc
2026-08-19 16:20:21 +00:00
rka-oai
fb356f3d2c Add async delivery metadata to agent messages (#39312)
## What changed

- Add an optional `delivery` field to agent message events and app-server
  `agentMessage` items, with `"async"` identifying a user-visible message sent
  without ending the current turn.
- Preserve the marker through legacy event conversion, thread history
  materialization, replay, and generated JSON and TypeScript schemas.
- Require delivery of async agent message completion notifications on the
  in-process app-server transport.

## Testing

- Cover preservation of async delivery metadata in thread history.
- Cover delivery-required classification for async agent messages.

GitOrigin-RevId: f92ba25de1293ee271404badfdf0d117d6530329
2026-08-18 23:05:52 +00:00
Owen Lin
f950a1ba0b Preserve thread names during rollout migration (#39273)
## Why

Legacy threads and paginated threads read their displayed names from different metadata. Moving a rollout to paginated history must carry over its legacy-visible name so renamed threads do not lose it.

## What changed

- Load legacy indexed names once for an apply run.
- When promoting a thread, preserve an existing canonical name or copy its distinct legacy title, falling back to the legacy name index.
- Repair missing names when migration encounters an already-paginated rollout.

## Testing

Added migration tests covering preservation from both legacy name sources and repair on a rerun.

GitOrigin-RevId: 5fc526570b1cf358a159ef174c159ac091ccc40f
2026-08-18 20:07:26 +00:00
Jeremy Rose
fa9a05f2d2 Deduplicate rollout moves when archiving threads (#39256)
## Why

SQLite metadata can identify a rollout through a non-canonical path. When that
path resolves to a rollout already found on disk, archive and unarchive
operations could schedule the same source file to be moved twice.

## What changed

- Deduplicate archive and unarchive moves by canonical source path.
- Cover both operations with metadata-backed tests that use equivalent rollout
  paths.

GitOrigin-RevId: d190915264214413667fc2c83d360a7e8504d0aa
2026-08-18 18:38:12 +00:00
jif
a397079287 Preserve MCP resource origins across compaction (#39192)
## Why

Compaction can remove the tool-call events that associate an MCP app widget with
the tool, account, and URI needed to authorize later resource reads.

## What changed

- Store a bounded MCP resource-origin checkpoint with each compacted rollout.
- Restore the checkpoint when resuming a session so existing widgets remain
  readable after compaction and restart.
- Discard saved origins when rollback rewrites history across the compaction
  boundary, and reject oversized or invalid checkpoints during restoration.

## Testing

- Extend the app-server MCP resource-origin test to compact a paginated thread,
  continue the conversation, and read the original widgets before and after a
  restart.
- Verify rollback clears provenance that no longer matches retained history.

GitOrigin-RevId: 7f91c6202fe8d9be21c78dda42fa205937c44263
2026-08-18 12:37:41 +00:00
Shijie Rao
230791fd1f Persist active permission profiles in turn context (#39145)
## What changed

- Add the optional `active_permission_profile` field to `TurnContextItem` to record the built-in or named profile that produced the effective permission profile.
- Populate the field when serializing a turn context while preserving compatibility with records where it is absent.

## Testing

- Add coverage verifying that turn-context serialization stores the active permission profile.

GitOrigin-RevId: d69142ab1b26595ab406235f89818ef41370e963
2026-08-18 05:46:08 +00:00
Eric Traut
eeb82a156d Dispatch queued messages written by other processes (#39034)
## Why

Durable thread queues can be updated through another SQLite connection, but loaded idle threads need to notice those updates before they can dispatch the new messages.

## What changed

- Track a durable revision for each thread queue, including backfilling existing queues and updating revisions on inserts, updates, and deletes.
- Poll SQLite's data version and query revisions to find changed queues for loaded threads.
- Wake idle threads with pending external messages, discover queued work when threads are loaded or resumed, and retry each thread independently so one blocked queue does not stall others.

## Testing

- Cover cross-runtime queue writes, edits, independent dispatch, wake retries, and resumed threads.
- Cover migration backfills and revision tracking for queue updates and deletes.

GitOrigin-RevId: 906d902bbf6a760be206a1b2c08fd71427b63c46
2026-08-17 17:19:46 +00:00
Anton Panasenko
3b4569a920 Add experimental app-server project APIs (#38940)
## What changed

- Add SQLite-backed `project/list`, `project/read`, `project/create`,
  `project/import`, `project/update`, `project/move`, and `project/delete`
  endpoints with ordered roots, metadata, manual positioning, pagination, and
  idempotent creation.
- Add project assignment to thread start, metadata updates, list filtering,
  and fork inheritance. Project deletion clears assignments without deleting
  threads.
- Emit `project/changed` and `thread/project/updated` notifications after
  committed changes, and export the new protocol schemas and TypeScript types.

## Testing

- Cover project lifecycle and ordering, idempotency, atomic imports, cursor and
  filter validation, thread assignment, deletion, and fork inheritance.

GitOrigin-RevId: 47ae621be01ab8ef70a74a629fb29a5c5709ea33
2026-08-17 04:01:37 +00:00
Anton Panasenko
fcdae21073 Support metadata staging for reserved thread IDs (#38819)
## Why

Callers need to associate host-owned state with a thread before Core starts it.

## What changed

- Add `ThreadManager::reserve_thread_id` and allow new threads to start with the reserved ID. Reject reserved IDs when resuming an existing thread.
- Let thread stores stage metadata for a reserved ID. The local store merges staged values into the first successful metadata update and then removes them.
- Clear staged metadata when an unmaterialized thread shuts down or is discarded, and require the state database for staging.

## Testing

- Cover reserved ID generation and resume validation.
- Cover metadata persistence, merging, rollout-compatible updates, cleanup, and invalid staging requests.

GitOrigin-RevId: 3f9bc48f0f7764011cc90ea01349e497dda0a9e3
2026-08-16 00:57:28 +00:00
efrazer-oai
9341b38310 Add experimental thread queue APIs to app server (#38456)
## What changed

- Add experimental `thread/queue/add`, `list`, `update`, `delete`, `reorder`, and `start` requests for persistent queued user submissions.
- Automatically dispatch queued submissions in FIFO order after completed or failed turns, while leaving the queue paused after interruption.
- Emit `thread/queue/changed` notifications and expose paginated queue contents with stable submission and client message IDs.
- Export the new protocol types through the generated JSON Schema and TypeScript bindings.

## Testing

- Add app-server integration coverage for capability gating, CRUD, pagination, capacity limits, notifications, automatic dispatch, cold-thread resume, interruption, and explicit queue starts.

GitOrigin-RevId: 89caf5577192199bda77cf6c372a59d26cdd910e
2026-08-13 23:38:02 +00:00
Rasmus Rygaard
42bb50d502 Allow metadata updates without materializing threads (#38413)
## Why

Thread stores may filter a metadata patch to a no-op. Requiring every successful
update to return a `StoredThread` forces those implementations to materialize a
thread even when the caller does not need one.

## What changed

- Let `ThreadStore::update_thread_metadata` return `None` after a successful
  update that did not materialize a thread.
- Preserve the materialized-thread contract of `LiveThread` and `ThreadManager`
  metadata updates by reading the thread when the store returns `None`.
- Keep completion-only metadata updates from issuing that fallback read.

## Testing

Cover materialized in-memory updates and verify that fallback reads occur only
for callers that require the updated thread.

GitOrigin-RevId: 3059efc9d5b69e0b46e35fb5b5aa9638086654f2
2026-08-13 18:12:27 +00:00
Tamir Duberstein
2bd8727a0c Preserve floating-point values when decoding rollout lines (#38399)
## Why

Serde's generic buffering for flattened and internally tagged fields is incompatible with `serde_json`'s `arbitrary_precision` representation. As a result, persisted rollout items containing nested floating-point values can fail to decode.

## What changed

- Decode the rollout envelope at the JSON persistence boundary before deserializing the item payload.
- Use the shared decoder for both session resume and thread-history projection.
- Cover token-count floats, response-item metadata, field ordering, and ignored metadata in the decoder compatibility test.

GitOrigin-RevId: 59c7f6fbcf83162b5ed6980a69381fe45fc20a0a
2026-08-13 16:03:20 +00:00
jif
72fa74fbc9 Persist security risk scores in rollout history (#38363)
## What changed

- Add a `SecurityRiskScore` rollout item containing a category and numeric score.
- Persist the item in both thread history modes while excluding it from model context, user-visible thread history, search text, forks, and reconstructed conversation history.
- Re-export the score type from the extension API.

## Testing

- Cover serialization, persistence and loading, thread history projection, session reconstruction, append planning, and memory filtering.

GitOrigin-RevId: 1926fe366aeaa75052708a6da589f45a38eefb52
2026-08-13 12:04:29 +00:00
Owen Lin
b1373b74a2 Add durable reverts for paginated threads (#38292)
## What changed

- Add `ThreadStore::revert_thread` to retain history before a selected turn by creating a new immutable rollout and atomically switching the thread's stored rollout path.
- Preserve the logical thread ID and session metadata across repeated reverts.
- Track every rollout owned by a thread so archive, unarchive, and delete operations handle the complete lineage and restore moved files when metadata updates fail.
- Prevent deletion when forks still reference any rollout in the thread's lineage.

## Testing

- Cover repeated reverts, compressed source rollouts, lifecycle operations over reverted histories, reference-aware deletion, and cross-process unarchive locking.

GitOrigin-RevId: 1c05a6db3d0036ee91b562263d7ca200c6a0ea71
2026-08-13 01:56:24 +00:00
Owen Lin
8d4d57387a Resolve paginated thread history by rollout ID (#38244)
## Why

`thread/revert` keeps the logical thread ID while selecting a new immutable
rollout. Using the thread ID for persisted history can therefore read or update
the wrong rollout.

## What changed

- Centralize resolution of a thread's current rollout across live writers,
  SQLite metadata, and filesystem fallbacks.
- Key history materialization, lineage traversal, paging, turn lookup, and fork
  boundaries by immutable rollout ID while retaining the thread ID for the
  logical thread.
- Treat SQLite's selected rollout as authoritative for paginated threads so a
  filesystem scan cannot select an older rollout after `thread/revert`.

## Testing

Add a regression test showing that update-ordinal item paging reads from the
selected rollout ID when it differs from the thread ID.

GitOrigin-RevId: 1101e548a6e6c6d319996e24cb650e5b03ab7fbb
2026-08-12 20:31:52 +00:00
rka-oai
3a6f747d77 Preserve harness metadata across conversation history (#38058)
## What changed

- Wrap response items with optional harness-owned metadata while keeping the persisted response payload shape backward compatible.
- Store compacted-history metadata in an aligned sidecar and reject malformed sidecars.
- Carry metadata through history normalization, compaction, resume, fork, truncation, migration, and rollout recording without sending it to model providers.

## Testing

- Cover legacy rollout compatibility, metadata serialization and validation, compaction retention, and resume/fork reconstruction.

GitOrigin-RevId: 17a6127c9048ff984cb4df4d97c0751a38c29f5f
2026-08-11 19:30:12 +00:00
Tamir Duberstein
ed390a5dc4 Filter live rollout items in place (#38034)
## What changed

Retain persistable items directly in the owned append buffer instead of cloning
them into a second vector. Preserve the existing history-mode filtering and
empty-append behavior.

GitOrigin-RevId: 5300312545492bef28f63521427048a27d46a9d1
2026-08-11 16:27:58 +00:00
Tamir Duberstein
4496ba3fd5 Use session metadata to validate thread history paths (#38033)
## What changed

Validate a SQLite thread's rollout path by reading its `SessionMetaLine` and
comparing the recorded thread ID directly. This avoids constructing a
`StoredThread` solely to decide whether the path can supply the requested
history.

GitOrigin-RevId: 8eb1e329514cefc056e44ab4a76d58ff50314a97
2026-08-11 16:20:12 +00:00
arun eswara
edcec13372 Expose image generation usage-limit failures (#38024)
## What changed

- Add structured `usageLimitExceeded` metadata to failed image-generation
  items, including the image limit ID and optional reset timestamp.
- Preserve the failure metadata in completion events, app-server thread
  history, thread reads and resumes, and migrated legacy rollouts.
- Export the new failure type in the generated JSON and TypeScript schemas.

## Testing

- Cover usage-limit responses through item completion, persistence, and thread
  resume.
- Verify legacy rollout migration retains image-generation failure metadata.

GitOrigin-RevId: 91c31a2a47456b6b916ee03002ff314b05946dd4
2026-08-11 15:40:52 +00:00
Bryan Ashley
722784e936 Distinguish turn-start thread persistence (#37926)
## What changed

- Add `PersistContext` to the thread-store persistence contract so stores can
  identify persistence requested immediately before model sampling. A
  `TurnStart` request may be enqueued in the background when later flush and
  shutdown operations fence it and report any failure; all other persistence
  remains synchronous.
- Use the turn-start context only for the initial input of a regular turn, while
  admission acknowledgments, steered input, and other persistence paths keep
  the standard durability behavior.
- Flush pending metadata before turn-start persistence and ensure shutdown is
  still attempted if that metadata update fails.

## Testing

- Verify that developer instructions and user input are persisted before the
  model request completes for a default paginated-history thread.

GitOrigin-RevId: 162f0ec796a61aebe66ca0b909fbcccec3047b85
2026-08-11 03:35:45 +00:00
joeytrasatti-openai
1549756b78 Add appearance metadata to thread sections (#37898)
## What changed

- Add optional `icon` and `color` appearance fields to custom thread sections and expose them through the app-server protocol.
- Persist appearance metadata in SQLite and include it when listing sections or returning threads.
- Let `threadSection/update` preserve an omitted appearance, clear it with `null`, or replace it with a new value.
- Reject appearance fields larger than 64 bytes.

## Testing

- Cover protocol compatibility, persistence across restart, update and clear behavior, and field-length validation.

GitOrigin-RevId: 8f55ce4f42f8bd17aa5416848a85845f681953df
2026-08-10 23:32:42 +00:00
Adam Perry @ OpenAI
63002bdb26 Extract persisted history types into a dedicated crate (#37871)
## What changed

- Add `codex-history` for model-history and persisted-rollout domain types, including `RolloutItem`, `RolloutLine`, `CompactedItem`, and initial/resumed history state.
- Re-export the persisted types from `codex-rollout` and update consumers to use the new crate boundary instead of `codex-protocol`.
- Preserve existing rollout serialization, including legacy numeric compacted-window IDs.

## Testing

- Add `codex-history` tests for rollout JSON round trips, compacted-history compatibility, persisted history modes, and multi-agent version selection.

GitOrigin-RevId: 944daa9297ddd231d3aebbdcb05fff4adf8b4e1b
2026-08-10 19:26:52 +00:00
Owen Lin
4bb7ee3472 Add rollout migration tooling and background migration (#37348)
## What changed

- Add `codex migrate-rollouts` with dry-run inspection by default, explicit `--apply`, thread filtering, optional I/O throttling, progress output, and JSON or verbose reports.
- Add the disabled-by-default `background_paginated_rollout_migration` feature to migrate legacy local sessions at startup before rollout compression begins.
- Persist a migration cursor and skipped-file fingerprints so later startups avoid full rescans while retrying changed files and recovering pending migrations.
- Coordinate migration with live writers and emit metrics for manual and startup runs.

## Testing

- Cover startup cursor advancement and lookback, pending migration recovery, live-writer coordination, changed empty rollouts, and progress reporting.

GitOrigin-RevId: 276ac506c50ebec5140fd319faca1bb998172061
2026-08-07 01:26:42 +00:00