Commit Graph

239 Commits

Author SHA1 Message Date
efrazer-oai
9341b38310 Add experimental thread queue APIs to app server (#38456)
## What changed

- Add experimental `thread/queue/add`, `list`, `update`, `delete`, `reorder`, and `start` requests for persistent queued user submissions.
- Automatically dispatch queued submissions in FIFO order after completed or failed turns, while leaving the queue paused after interruption.
- Emit `thread/queue/changed` notifications and expose paginated queue contents with stable submission and client message IDs.
- Export the new protocol types through the generated JSON Schema and TypeScript bindings.

## Testing

- Add app-server integration coverage for capability gating, CRUD, pagination, capacity limits, notifications, automatic dispatch, cold-thread resume, interruption, and explicit queue starts.

GitOrigin-RevId: 89caf5577192199bda77cf6c372a59d26cdd910e
2026-08-13 23:38:02 +00:00
jif
72fa74fbc9 Persist security risk scores in rollout history (#38363)
## What changed

- Add a `SecurityRiskScore` rollout item containing a category and numeric score.
- Persist the item in both thread history modes while excluding it from model context, user-visible thread history, search text, forks, and reconstructed conversation history.
- Re-export the score type from the extension API.

## Testing

- Cover serialization, persistence and loading, thread history projection, session reconstruction, append planning, and memory filtering.

GitOrigin-RevId: 1926fe366aeaa75052708a6da589f45a38eefb52
2026-08-13 12:04:29 +00:00
Owen Lin
b1373b74a2 Add durable reverts for paginated threads (#38292)
## What changed

- Add `ThreadStore::revert_thread` to retain history before a selected turn by creating a new immutable rollout and atomically switching the thread's stored rollout path.
- Preserve the logical thread ID and session metadata across repeated reverts.
- Track every rollout owned by a thread so archive, unarchive, and delete operations handle the complete lineage and restore moved files when metadata updates fail.
- Prevent deletion when forks still reference any rollout in the thread's lineage.

## Testing

- Cover repeated reverts, compressed source rollouts, lifecycle operations over reverted histories, reference-aware deletion, and cross-process unarchive locking.

GitOrigin-RevId: 1c05a6db3d0036ee91b562263d7ca200c6a0ea71
2026-08-13 01:56:24 +00:00
rka-oai
3a6f747d77 Preserve harness metadata across conversation history (#38058)
## What changed

- Wrap response items with optional harness-owned metadata while keeping the persisted response payload shape backward compatible.
- Store compacted-history metadata in an aligned sidecar and reject malformed sidecars.
- Carry metadata through history normalization, compaction, resume, fork, truncation, migration, and rollout recording without sending it to model providers.

## Testing

- Cover legacy rollout compatibility, metadata serialization and validation, compaction retention, and resume/fork reconstruction.

GitOrigin-RevId: 17a6127c9048ff984cb4df4d97c0751a38c29f5f
2026-08-11 19:30:12 +00:00
Tamir Duberstein
dad1db87bb Limit TUI streaming traces in SQLite logs (#38036)
## Why

Per-chunk streaming traces can flood the bounded SQLite log queue.

## What changed

- Persist `codex_tui::streaming::controller` and
  `codex_tui::streaming::table_holdback` events at `DEBUG` and above.
- Keep `TRACE` events enabled for other TUI streaming targets.

## Testing

Extended the SQLite log filter test to cover dropped streaming traces, retained
debug diagnostics, and unaffected streaming targets.

GitOrigin-RevId: e7eb999a0770eba6cce5f389005896e64928f02b
2026-08-11 17:07:24 +00:00
joeytrasatti-openai
1549756b78 Add appearance metadata to thread sections (#37898)
## What changed

- Add optional `icon` and `color` appearance fields to custom thread sections and expose them through the app-server protocol.
- Persist appearance metadata in SQLite and include it when listing sections or returning threads.
- Let `threadSection/update` preserve an omitted appearance, clear it with `null`, or replace it with a new value.
- Reject appearance fields larger than 64 bytes.

## Testing

- Cover protocol compatibility, persistence across restart, update and clear behavior, and field-length validation.

GitOrigin-RevId: 8f55ce4f42f8bd17aa5416848a85845f681953df
2026-08-10 23:32:42 +00:00
Adam Perry @ OpenAI
63002bdb26 Extract persisted history types into a dedicated crate (#37871)
## What changed

- Add `codex-history` for model-history and persisted-rollout domain types, including `RolloutItem`, `RolloutLine`, `CompactedItem`, and initial/resumed history state.
- Re-export the persisted types from `codex-rollout` and update consumers to use the new crate boundary instead of `codex-protocol`.
- Preserve existing rollout serialization, including legacy numeric compacted-window IDs.

## Testing

- Add `codex-history` tests for rollout JSON round trips, compacted-history compatibility, persisted history modes, and multi-agent version selection.

GitOrigin-RevId: 944daa9297ddd231d3aebbdcb05fff4adf8b4e1b
2026-08-10 19:26:52 +00:00
dshakiba-OAI
62b7386b07 Limit payload traces in diagnostic logs (#37497)
## Why

High-volume request and streamed-response payloads can overwhelm the SQLite log database and diagnostic ring buffer used for reports.

## What changed

- Limit HTTP transport, SSE, and WebSocket diagnostics to `DEBUG` in persistent log sinks while leaving unrelated trace diagnostics available.
- Log known unhandled response events and delta events at `TRACE`, and surface unexpected event kinds at `DEBUG` without including their payloads.
- Report structured parse-error metadata instead of logging an unparseable SSE payload.

## Testing

- Cover filtering for transport, SSE, WebSocket, and unrelated trace records in both report and SQLite log sinks.
- Cover unknown and unsupported delta response events.

GitOrigin-RevId: 6d9121e093ddadf6834da394df544d9c09d8aeb9
2026-08-07 19:56:29 +00:00
Owen Lin
4bb7ee3472 Add rollout migration tooling and background migration (#37348)
## What changed

- Add `codex migrate-rollouts` with dry-run inspection by default, explicit `--apply`, thread filtering, optional I/O throttling, progress output, and JSON or verbose reports.
- Add the disabled-by-default `background_paginated_rollout_migration` feature to migrate legacy local sessions at startup before rollout compression begins.
- Persist a migration cursor and skipped-file fingerprints so later startups avoid full rescans while retrying changed files and recovering pending migrations.
- Coordinate migration with live writers and emit metrics for manual and startup runs.

## Testing

- Cover startup cursor advancement and lookback, pending migration recovery, live-writer coordination, changed empty rollouts, and progress reporting.

GitOrigin-RevId: 276ac506c50ebec5140fd319faca1bb998172061
2026-08-07 01:26:42 +00:00
Owen Lin
6bb6e9045f Add legacy rollout migration to paginated history (#37175)
## What changed

- Add `LocalThreadStore::migrate_rollouts` with dry-run and apply modes,
  optional thread selection, throughput limiting, and per-rollout outcomes.
- Canonicalize legacy JSONL records into paginated history while preserving
  model-visible conversation context, including compressed rollouts and copied
  fork history.
- Publish replacements atomically and journal migrations so interrupted SQLite
  projections can be recovered. Coordinate migration with rollout compression
  and skip rollouts with active writers.
- Make paginated history a one-way SQLite promotion so stale metadata cannot
  downgrade a migrated thread.

## Testing

- Cover legacy record normalization, dry runs, idempotency, malformed input,
  compressed rollouts, active writers, interrupted migration recovery, and a
  cold app-server resume after migration.

GitOrigin-RevId: b9991b659f28ebb52da39ce62e8b4e0bae2ba2bb
2026-08-05 22:57:27 +00:00
efrazer-oai
b87981a510 Add durable per-thread user submission queues (#36952)
## What changed

- Add a storage-neutral `QueueStore` interface backed by a dedicated SQLite
  database for ordered, thread-scoped user submissions.
- Support enqueueing, paginated listing, editing, deletion, and atomic
  reordering, with a limit of 100 pending items per thread.
- Remove queued submissions when their thread is deleted.

## Testing

- Cover concurrent inserts, ordering and pagination, edits, deletion,
  cross-thread isolation, queue limits, and thread cleanup.

GitOrigin-RevId: 0a8e95ac71137afc6f3a039d4b5fc8603110c891
2026-08-04 19:23:29 +00:00
jif
d98eb72e18 Limit RMCP logs persisted to SQLite (#36904)
## What changed

- Persist `rmcp` and `codex_rmcp_client` events only at `INFO` level or above.
- Cover nested targets from both namespaces in the SQLite log filter test.

GitOrigin-RevId: dfeeec302a9562f533a09a735b392d202c1f7801
2026-08-04 14:56:06 +00:00
rka-oai
bb5054fe47 Capture rollout budget units from response usage (#36641)
## What changed

- Parse `codex_rollout_budget_units` from completed Responses API usage into
  `TokenUsage`.
- Keep the provider-only value out of serialized protocol, JSON schema, and
  TypeScript representations.

## Testing

- Cover the value in unit and end-to-end SSE response parsing tests.

GitOrigin-RevId: a53d4202beb9c8985d25894cb10e7c01f20b2a44
2026-08-03 00:30:57 +00:00
joeytrasatti-openai
c42ea41ee0 Add thread section management APIs (#36380)
## What changed

- Add `threadSection/create`, `threadSection/update`, and `threadSection/delete` app-server methods, with generated protocol schemas and TypeScript bindings.
- Persist custom sections in SQLite with stable UUIDv7 identities, trim and validate display names, and prevent renaming or deleting the built-in pinned section.
- Delete sections transactionally and return their active and archived threads to the unsectioned list.
- Serialize section mutations with section listing so concurrent requests observe ordered updates.

## Testing

- Cover section persistence across restarts, ordered renames, member cleanup on deletion, invalid requests, unsupported stores, and thread-store/state-runtime behavior.

GitOrigin-RevId: dc9a4be283c21dbc93ab1829eafc4825d8df6575
2026-07-31 19:08:45 +00:00
joeytrasatti-openai
ad6fc66b6d Add persisted manual ordering for thread sections (#36007)
## What changed

- Add `thread/section/move` to atomically move a thread into, within, or out of a section. Threads can be inserted before an existing member or appended, and moves within a section preserve `sectionEnteredAt`.
- Add `section_position` sorting to `thread/list`, with ascending order as its default, and expose `sectionEnteredAt` in thread responses.
- Persist section positions and entry times in SQLite, including migration of existing section members into recency order. Section membership is no longer updated through `thread/metadata/update`.

## Testing

- Cover section moves, reordering, pagination, persistence across restarts and rollout reconciliation, concurrent updates, and rank renumbering.

GitOrigin-RevId: aec6d7ddedca5277029b5caf5c074975397e956c
2026-07-29 18:37:22 +00:00
charlesgong-openai
9f23e97797 Include session titles in external agent import history (#35870)
## What changed

- Preserve the original title when importing an external-agent session and return it with successful session entries from `externalAgentConfig/import/readHistories`.
- Add dedicated history-record success types so externally completed imports can optionally supply a session title without changing the live import result type.

## Testing

- Verify imported session titles are returned by the app server and persisted when session import ledger entries are created or refreshed.

GitOrigin-RevId: e896688a5763c238f1e1dc8b4672a7aa7bc1ed4c
2026-07-29 02:45:55 +00:00
Adam Perry @ OpenAI
50a7328f50 Enforce centralized SQLite connection creation (#35828)
## Why

Direct SQLx constructors can bypass the shared SQLite configuration in
`codex-state`.

## What changed

- Deny SQLx pool, connection, and database creation methods through the
  workspace Clippy configuration for both Cargo and Bazel builds.
- Exempt `codex-rs/state/src/sqlite.rs`, the centralized connection shim, from
  the lint.
- Document that the deny list must be audited when upgrading SQLx.

GitOrigin-RevId: e20d7e83095727ac446347157782175062a100fc
2026-07-28 20:49:22 +00:00
joeytrasatti-openai
85c6da1c79 Add persisted sections for organizing threads (#35722)
## What changed

- Replace the `isPinned` thread metadata and filters with an optional persisted
  `section` and `sectionId`.
- Add the paginated `threadSection/list` app-server method so clients can
  discover sections even when they contain no threads.
- Seed a stable `Pinned` section, validate section assignments, and support
  filtering for a specific section or for unsectioned threads.

## Testing

- Cover section protocol serialization, listing and pagination, metadata
  updates, filtering, persistence, migration compatibility, and operation
  without SQLite state.

GitOrigin-RevId: 7972b5471d29317b9387bfd90aa9f573f691ad4c
2026-07-28 05:26:16 +00:00
Adam Perry @ OpenAI
3418498f01 Honor the configured SQLite home in the logs client (#35695)
## Why

`just log` derived the logs database path from `CODEX_HOME`, so it could read
the wrong database when `sqlite_home` or `CODEX_SQLITE_HOME` selected a
different location.

## What changed

- Move `logs_client` into `codex-cli` so it can resolve the shared
  `SqliteConfig` through the standard configuration loader.
- Keep `--db` as a direct override that skips config loading and preserves
  native path bytes.
- Update the `just log` recipes to run the client from its new crate.

## Testing

- Add coverage for bypassing invalid Codex config with `--db`.
- Add Unix coverage for non-UTF-8 database paths.

GitOrigin-RevId: fabd64a66543be26a6f5d3b5e509016c3270350e
2026-07-28 01:17:52 +00:00
Tamir Duberstein
63682c4e1a Include empty-preview threads in relationship listings (#35691)
## What changed

- Include threads without preview text when listing direct children or descendants from the persisted spawn graph.
- Continue filtering those threads out of the global thread list.

## Testing

- Extend the relationship pagination test to cover an empty-preview child in direct-child and descendant results.

GitOrigin-RevId: c2e7bb04c5f64558a48b1d7877bd24edaa364bb4
2026-07-28 00:24:52 +00:00
Owen Lin
af7f6f4d34 Include item start times in completion events (#35363)
## What changed

- Add an optional `started_at_ms` field to `ItemCompletedEvent`, retaining
  compatibility with older persisted events that lack it.
- Track the first start timestamp for each in-flight item and attach it when
  emitting and persisting the completion event. If no start was recorded, use
  the completion timestamp as a fallback.
- Emit a complete start/completion lifecycle for subagent activity items.

## Testing

- Cover concurrent item timing, repeated starts, turn-boundary cleanup, the
  missing-start fallback, subagent activity, and persisted web-search events.

GitOrigin-RevId: e7cec9c4f1ef6ba67f287e81fb4d7d856fcf87a7
2026-07-25 16:06:54 +00:00
richardopenai
b834702b27 Support incremental replay of updated thread items (#35013)
## Why

Persisted item snapshots can be updated after their initial projection while retaining their creation order. Callers need a way to read those updates incrementally without replaying every item.

## What changed

- Track the rollout ordinal of each item's latest snapshot separately from its creation ordinal.
- Let thread-store callers filter items after an exclusive update watermark and sort pages by either creation or update ordinal.
- Keep existing app-server reads creation-ordered, reject incremental reads across forked lineages, and preserve compatibility with older writers.

## Testing

- Cover update-watermark filtering, pagination, sort directions, per-turn reads, cursor validation, forked lineages, migration compatibility, and refreshed item snapshots.

GitOrigin-RevId: d57de849abec27f788e9c384dbff073a0aca1053
2026-07-23 21:17:45 +00:00
Adam Perry @ OpenAI
c769a05340 Honor the configured SQLite home across state consumers (#34994)
## Why

Codex and SQLite data can use separate home directories, but state consumers
could reconstruct database paths from the Codex home instead of consistently
using the resolved SQLite configuration.

## What changed

- Pass `SqliteConfig` through the core, rollout, state runtime, and thread store
  instead of passing a directory and rebuilding the configuration downstream.
- Use that shared configuration for state, logs, memories, goals, and paginated
  thread-history database access, including integrity checks and cleanup.
- Reject state database handles whose SQLite configuration does not match the
  requesting store.

## Testing

Add coverage with separate Codex and SQLite homes that verifies startup
backfill, thread listing, and paginated history all use the configured SQLite
directory.

GitOrigin-RevId: 1de1cdd1d6ff1d70bbb6c360c8352e6543fb8ebf
2026-07-23 19:19:35 +00:00
charlesgong-openai
ce803c45ae Record externally completed agent config imports (#34981)
## Why

Clients that complete an external agent config import outside the app server need a way to include the result in import history.

## What changed

- Add `externalAgentConfig/import/recordHistory` to persist completed item results and return a generated import ID.
- Store the optional provider ID for regular imports and expose it when reading import history.
- Update the generated JSON and TypeScript protocol schemas for the new request, response, and history field.

## Testing

- Cover recording and reading an externally completed import through the app-server protocol.
- Cover provider ID persistence and updates in the state runtime.

GitOrigin-RevId: 935401e899ec9eeb38f54c182e45a16642411deb
2026-07-23 17:55:22 +00:00
brisebois-oai
83ff1c2f80 Cache remote plugin catalogs by scope (#34849)
## What changed

- Cache global, user, and workspace remote plugin catalogs on disk with a three-hour TTL and scope- and account-specific keys.
- Serve cached catalogs from `plugin/list`, refresh stale entries in the background, and warm existing caches at app-server startup without blocking list requests.
- Make `forceRefetch: true` bypass cached catalog data and replace an entry only after a successful fetch.
- Invalidate user and workspace catalog caches after remote plugin sharing changes.

## Testing

- Cover cache freshness, scope isolation, legacy global cache compatibility, identity requirements for private catalogs, forced refetches, and non-blocking startup refreshes.

GitOrigin-RevId: 7844e9dfb72e43b26d749b4afd16fd9c1793caba
2026-07-23 00:52:55 +00:00
joeytrasatti-openai
400ee190c3 Add persisted thread pinning to the app server (#34840)
## What changed

- Add `isPinned` to thread responses and allow `thread/metadata/update` to pin or unpin stored threads.
- Add an `isPinned` filter to `thread/list`, including cursor-based pagination and combinations with relationship filters.
- Persist pin state in SQLite without modifying rollout files, default existing threads to unpinned, and preserve pins through reconciliation and archive transitions.

## Testing

- Cover protocol serialization, database migration and indexing, pin updates, filtered pagination, rollout reconciliation, and archive/unarchive behavior.

GitOrigin-RevId: c2eacabd6dae465e2ae6ce8a169e00740960cdb7
2026-07-22 22:54:28 +00:00
Adam Perry @ OpenAI
946ed315a4 Centralize SQLite connection configuration (#34808)
## What changed

- Add `SqliteConfig` to own the resolved SQLite home, runtime database paths,
  and shared read/write and read-only pool settings.
- Pass the configuration through state and local thread-store consumers,
  replacing standalone path helpers and duplicated connection setup.

GitOrigin-RevId: 7351b08da94b5c8b6c0bbe492f86aeeca0699d3c
2026-07-22 18:45:47 +00:00
jif
66bd101fff Avoid unnecessary post-sampling token estimates (#34789)
## Why

The post-sampling trace calculated an estimated token count even when only the
always-on diagnostic and state log sinks were listening, adding unnecessary work
after every sample.

## What changed

- Move the estimate to a dedicated trace target and calculate it only when that
  event is enabled.
- Disable the target for the always-on diagnostic and state log sinks while
  keeping it available to explicitly configured subscribers.

## Testing

- Verify that the combined always-on subscribers do not enable the estimate
  event.

GitOrigin-RevId: ea572584597dda313e7719897d0ab1fdf7362745
2026-07-22 16:50:09 +00:00
Charlie Marsh
fd3c1dc13d Optimize remote compaction history handling (#34431)
## Why

Remote compaction can process large histories. Repeatedly estimating and replacing the full history, and cloning it when tracing is disabled, adds avoidable CPU and memory overhead.

## What changed

- Estimate each history item's token count once, update the total as trailing tool outputs are rewritten, and replace history only after all rewrites are selected.
- Preserve unclamped token totals while calculating removed tokens so saturated estimates do not hide overflow.
- Snapshot compaction input history only when rollout tracing is enabled.
- Reuse the v2 request input instead of cloning it before adding the compaction trigger.

## Testing

- Cover enabled and disabled compaction trace contexts through `is_enabled()` assertions.

GitOrigin-RevId: 8de9c9704ba29532fba05430eb204bbae2e8bf83
2026-07-20 23:30:30 +00:00
jif
687f05cb94 Remove CSV-backed agent jobs (#34413)
## What changed

- Remove the `spawn_agents_on_csv` and `report_agent_job_result` tools and their agent-job runtime and state models.
- Drop the legacy `agent_jobs` and `agent_job_items` tables during state database migration.
- Keep `features.enable_fanout` and `agents.job_max_runtime_seconds` accepted as no-op compatibility settings while omitting them from the generated configuration schema.

## Testing

- Verify upgrades remove both legacy agent-job tables.
- Verify the removed feature and configuration keys still parse without taking effect.

GitOrigin-RevId: 8cc3337da78c67162229f02f40a747f503542646
2026-07-20 21:00:17 +00:00
Adam Perry @ OpenAI
81e89fa5af Require absolute paths for test SQLite configuration (#34411)
## Why

`SqliteConfig` stores its home as an `AbsolutePathBuf`, but its test constructor
previously accepted a `PathBuf` and checked the absolute-path invariant at
runtime.

## What changed

- Make `SqliteConfig::new_for_testing` accept an `AbsolutePathBuf` directly,
  removing its fallible conversion and `expect`.
- Update SQLite test setup to convert temporary directory paths with
  `PathExt::abs` at each call site.

GitOrigin-RevId: 93585b9aea805e2449b3465ac20eeb39417ed555
2026-07-20 20:54:08 +00:00
Owen Lin
2793c826e8 Enable memories for paginated threads (#34386)
## Why

Paginated threads keep metadata updates in SQLite, while their rollout retains the initial `memory_mode`. Reconciliation could therefore overwrite the current setting with a stale value, and memory processing excluded these threads entirely.

## What changed

- Include paginated threads in stage 1 memory job selection and global memory output lookup.
- Preserve the SQLite `memory_mode` when reconciling or backfilling an existing paginated thread, while continuing to seed missing rows and restore legacy threads from rollouts.
- Omit `memory_mode` alongside Git metadata when flushing resumed paginated history.

## Testing

Added coverage for preserving disabled memory mode during reconciliation and backfill, selecting eligible paginated threads, and omitting initial metadata on paginated resume.

GitOrigin-RevId: 2a6e16068e69680728757fbec27aeefae45b8110
2026-07-20 17:41:33 +00:00
nhamidi-oai
6f785632b0 Preserve audio across history and tool outputs (#34385)
## Why

Audio attachments could be dropped from tool outputs and reconstructed thread history, while inline base64 payload size could overstate their model context cost.

## What changed

- Preserve remote and local audio attachments in legacy user-message events and thread history.
- Keep audio from function and custom tool outputs when the model supports it and the output fits the truncation budget; otherwise replace it with an omission marker.
- Estimate inline audio cost from its duration, with a serialized-size fallback for malformed or unsupported data URLs, so compaction and truncation use model-visible cost instead of raw base64 size.
- Replace audio in messages and tool outputs with an explanatory placeholder for models without audio input support.

## Testing

Add coverage for history round trips, dynamic tool responses, modality filtering, duration-based context estimates, output-budget truncation, and remote compaction.

GitOrigin-RevId: 5ed4628850a668f8383b7cd73db71b144cc02cd1
2026-07-20 17:36:44 +00:00
Owen Lin
19b2273d8a Keep paginated thread Git metadata in SQLite (#34382)
## Why

Paginated rollouts contain only the initial Git metadata tuple. Reusing that
tuple during later reads, reconciliation, or resume can overwrite newer SQLite
values and undo explicit field clears.

## What changed

- Treat SQLite as authoritative for Git metadata once a thread is paginated.
- Apply paginated Git metadata patches directly to SQLite without appending
  compatibility metadata to the rollout.
- Preserve explicit null values when reading by rollout path or reconciling the
  initial session metadata, and ignore the initial Git tuple during resume.

## Testing

Added coverage for updating and clearing paginated Git metadata, preserving the
result through reconciliation and rollout-path reads, and avoiding stale Git
metadata updates on resume.

GitOrigin-RevId: 4d3e68cb24965b93d337d923092a98bac9b352b5
2026-07-20 17:23:18 +00:00
Brent Traut
5a208c1fc3 Persist names for paginated threads (#34229)
## Why

Paginated threads need explicit user-facing names to remain distinct from
derived titles and previews without requiring rollout metadata writes.

## What changed

- Add a nullable `name` column to persisted thread metadata and use it as the
  canonical name for paginated threads.
- Route paginated name updates through the state database while retaining the
  name index as best-effort compatibility data and preserving legacy thread
  behavior.
- Return persisted names when reading, listing, searching, and resuming
  paginated threads, and include them in state-database search matching.

## Testing

- Cover paginated name updates, reads, list/search results, and metadata-only
  resume behavior, including stale or unavailable compatibility data.

GitOrigin-RevId: 5ac4dfa2a83682b43d59ecc87e0f515580164dca
2026-07-19 21:13:13 +00:00
Adam Perry @ OpenAI
6bd3f5e3db Centralize SQLite connection configuration (#33938)
## What changed

- Add `SqliteConfig` as the shared entry point for read-write and read-only SQLite pools.
- Apply consistent WAL, synchronization, auto-vacuum, busy-timeout, logging, and pool-size settings to writable Codex databases.
- Route state runtime, audit, CLI, and test database connections through the shared configuration.

## Testing

- Run migration tests against temporary on-disk databases opened through `SqliteConfig`, including the concurrent-writer repair case.

GitOrigin-RevId: e3946b98bde04c47574532ac8b1a7bb2b03edd97
2026-07-18 02:06:13 +00:00
Owen Lin
bcdc695877 Track inherited paginated rollout prefixes (#33930)
## What changed

- Add `HistoryPosition` to identify a source thread and an exclusive rollout
  prefix by ordinal and JSONL byte offset.
- Add optional `history_base` metadata to `SessionMeta` so a thread can record
  an inherited prefix of another paginated rollout.
- Default missing `history_base` metadata to `None` and omit it when unset to
  preserve compatibility with existing rollout metadata.

## Testing

- Verify that legacy session metadata without `history_base` deserializes with
  no inherited history position.

GitOrigin-RevId: 3b2b07a2dfee793d3b37b0a60529f856f167dbc1
2026-07-18 00:06:49 +00:00
Zanie Blue
18110b810f Avoid unnecessary writes during migration repair (#33687)
## Why

Opening a database with current migration history should not require SQLite's
writer slot. The legacy recency repair previously issued an `UPDATE` even when
there was nothing to repair, so it could fail while another connection held the
writer slot.

## What changed

Check `_sqlx_migrations` for the legacy version and checksum before issuing the
repair update, and return early when the current recency migration is already
recorded.

## Testing

Add a WAL-mode regression test that holds the writer slot on a second
connection and verifies that current migration history needs no write access.

GitOrigin-RevId: 76649213912b510df42a836cf8ed7f84a7c70eb5
2026-07-16 20:55:13 +00:00
Eric Traut
2edad72de3 Track prompt cache write token usage (#33454)
## What changed

- Parse `cache_write_tokens` from response input token details and carry it through token usage aggregation.
- Expose `cache_write_input_tokens` in protocol, app-server, exec, and TypeScript SDK usage events, defaulting it to zero for compatibility with older payloads.
- Report cache-write usage in analytics, metrics, traces, and rollout data alongside existing cache-read usage.

## Testing

- Add coverage for response parsing, usage event output, aggregation, analytics, metrics, and telemetry fields.

GitOrigin-RevId: 634e4fbe3086a8923c0ee13b622573e7d9d221bd
2026-07-16 00:22:53 +00:00
Owen Lin
2be648ba4a Preserve paginated history for spawned subagents (#33432)
## What changed

- Inherit paginated history mode when spawning or forking a subagent from a paginated parent.
- Load the parent's model context when forking and persist it as an inherited rollout prefix, while excluding that prefix from the child's projected turns, items, and metadata.
- Record the boundary between inherited context and child history, and reject partially initialized paginated subagent rollouts on resume.
- Resume paginated subagents from their stored model context instead of legacy rollout history.

## Testing

Added coverage for fresh and forked paginated subagents, compacted-history forks, model-context resume, inherited-prefix projection, metadata isolation, and incomplete-prefix detection.

GitOrigin-RevId: e57dc37c29aa4aaaf57b052b991be591d730e5ba
2026-07-15 21:25:24 +00:00
Felipe Coury
d88db19144 Preserve thread context when retrying or editing turns (#33211)
## What changed

- Add experimental `beforeTurnId` support to `thread/fork` for copying history strictly before a turn, including an in-progress turn.
- Use before-turn forks for TUI backtracking and safety retries so the replacement prompt retains thread context instead of recreating the branch from the preceding turn or a new thread.
- Add experimental `deferGoalContinuation` support so safety-retry forks can inherit an active goal while waiting for the replacement turn before automatic continuation resumes.
- Hide duplicated nested-review prompts when replaying history or selecting a prompt to edit.

## Testing

- Cover before-turn truncation, fork validation, active-goal inheritance, TUI safety retries, backtracking, and nested-review replay.

GitOrigin-RevId: 4915d85ae1929a0b8c40f3ed7dca29c7a5bdcdf3
2026-07-15 02:35:49 +00:00
charlesgong-openai
3fae30dd85 Preserve plugin install failure subtypes during imports (#33182)
## What changed

- Add `subErrorType` to external agent config import failures and carry it through progress and completion notifications, persisted import histories, and analytics.
- Expose plugin store failure subtypes from `PluginInstallError` so manual installs and imported plugins report details such as `failed_to_copy_plugin_file` while failures without a subtype remain `null`.
- Keep existing persisted import records compatible by defaulting the new field when it is absent.

## Testing

- Cover store I/O subtype extraction, analytics serialization, persisted import histories, and migration notification fixtures.

GitOrigin-RevId: ec1a85623fc826cbd7cfe9f55c2dd2bc86665937
2026-07-14 22:21:27 +00:00
Owen Lin
0ef9fa4d65 Materialize paginated thread history in SQLite (#32923)
## What changed

- Project durable paginated rollout records into rebuildable SQLite tables for turns, items, and projection progress while keeping JSONL as the source of truth.
- Add cursor-based local reads for turn summaries and thread or per-turn items, including archived-thread and history-mode validation.
- Serialize writes, shutdown, and deletion per thread so projection updates cannot race cleanup, and remove projected rows when a thread is deleted.
- Preserve stable item ordering while updating repeated snapshots and expose structured turn error details.

## Testing

- Cover projection, snapshot updates, summary recovery, pagination, failure handling, shutdown flushing, legacy-mode rejection, and deletion races.

GitOrigin-RevId: 34533185ece7f1f5690004c7f12b0f1f34dc2640
2026-07-14 02:00:39 +00:00
Felipe Coury
769a5de257 Make advanced reasoning selection explicit in the TUI (#32746)
## Why

`Max` and `Ultra` consume usage limits faster than standard reasoning levels, so they should not be selected accidentally while navigating the normal effort scale.

## What changed

- Move `Max` and `Ultra` behind a `More reasoning…` entry with a dedicated warning and descriptions.
- Keep the reasoning shortcuts from silently increasing into advanced efforts.
- Apply `Ultra` to the active conversation without changing defaults for new threads, while preserving it across mode switches and thread resumes.
- Record applied thread settings in thread metadata, including explicit clearing of reasoning effort, so resumed threads restore their latest model settings unless the user supplied an override.

## Testing

- Add TUI coverage for the advanced picker, shortcuts, Plan mode, configuration defaults, and resumed conversations.
- Add app-server and thread-store coverage for persisting, clearing, and restoring model and reasoning settings.

GitOrigin-RevId: 6708c7c2e8d38f491bf63000ea34f83476afdfe6
2026-07-13 09:40:51 +00:00
Tom
9993fb838a Improve Responses WebSocket timing telemetry (#32256)
## What changed

- Emit `responsesapi.websocket_timing` payloads as opt-in trace events with request context, while excluding them from diagnostic uploads and persisted logs by default.
- Preserve fractional-millisecond TBT values in telemetry histograms and runtime summaries, rounding only when formatting the TUI label.

## Testing

- Cover timing-log filtering, fractional TBT collection and display, and existing whole-millisecond duration behavior.

GitOrigin-RevId: 5f0a1c60c237abed17cc8b2d569ca77fbbc6ec41
2026-07-10 20:01:47 +00:00
Owen Lin
414217dc8a Add dedicated storage for paginated thread history (#32234)
## Why

Paginated thread history needs its own SQLite database to avoid adding lock
contention to the main state store.

## What changed

- Add the `thread_history_1.sqlite` path and migration scaffolding.
- Create tables and pagination indexes for projected turns and items, plus a
  per-thread projection checkpoint.
- Register the database with runtime diagnostics, Bazel inputs, and database
  telemetry.

GitOrigin-RevId: d194310835f1df2a2a29c7827d77ca37eabbd0a3
2026-07-10 18:27:19 +00:00
jif
bfa3eeb8bf Filter routine Hyper logs from SQLite (#31791)
## Why

The SQLite feedback log currently persists Hyper's TRACE, DEBUG, and
INFO connection-pool bookkeeping. Healthy connects, checkouts, reuse,
and timer activity generated thousands of small rows during active
sessions without adding useful feedback context.

## What changed

- Set the `hyper_util` target prefix to WARN for the SQLite feedback-log
layer.
- Continue persisting Hyper warnings and errors.
- Leave other tracing and OpenTelemetry subscribers unchanged.

This removes routine dependency chatter while preserving actionable
failures.

Related to #28224.
2026-07-09 14:35:09 +01:00
jif
acb78a41a0 Stop persisting RMCP service traces (#31789)
## Why

The SQLite feedback log enables TRACE by default. `rmcp::service`
renders complete MCP service events at TRACE, including large tool
catalogs and other protocol payloads. In a busy local sample, this was
the largest retained source and individual rows reached about 2 MiB.

These rows can quickly consume the per-thread retention budget, displace
smaller diagnostics, and add unnecessary SQLite insert-and-prune work.

## What changed

- Persist `rmcp::service` at INFO and above in the SQLite feedback log.
- Keep service initialization, shutdown, warnings, and errors available
for feedback.
- Leave other tracing and OpenTelemetry subscribers unchanged.

This is intentionally target-specific. Broader persistent-log policy
changes can remain separate.

Related to #28224.
2026-07-09 14:34:47 +01:00
Owen Lin
2342b2c2a6 feat(rollout): persist TurnItems for paginated thread rollouts (#30188)
## Description

This PR makes new threads with `history_mode = "paginated"` persist
`ItemCompleted(item: <turn_item>)` in their rollout JSONL file.

Legacy threads keep persisting the existing legacy events. Because the
format is selected per thread, a rollout is either legacy or paginated;
we do not need to support mixed rollouts containing both
representations.

This PR depends on [#31473](https://github.com/openai/codex/pull/31473).

## Why

Paginated thread history needs stable turn/item IDs and completed item
snapshots so the later SQLite projector can materialize appended rollout
JSONL without rebuilding the whole thread.

Keeping the legacy persistence policy unchanged avoids changing
historical rollouts or the readers that still consume them.

## What changed

- Made rollout filtering history-mode aware. Paginated threads keep
completed canonical `ItemCompleted` events and drop their redundant
legacy projections; legacy threads keep the existing event set.
- Made forks inherit the source thread history mode, so copied legacy
history is never filtered as paginated.
- Made paginated threads assign IDs to locally-created response items
even when `Feature::ItemIds` is off, and reject streamed output items
that arrive without server IDs.
- Updated legacy turn replay, rollout list/search, and SQLite metadata
extraction to understand completed canonical user-message items.
2026-07-08 19:55:03 -07:00
viyatb-oai
0746e8a345 [codex] Preserve reviewer when resuming threads (#30278)
## Why

A thread resumed without an explicit reviewer could pick up the reviewer
from the current config instead of preserving the reviewer already in
use by the thread. After an app restart, this meant a thread running
with auto review could silently switch back to user review, and the next
turn could continue under the wrong reviewer.

## What changed

Persist the effective reviewer with each turn and restore the latest
persisted value when the thread resumes. If the resume request
explicitly provides a reviewer, that value still takes precedence.

## Test plan

- Added a regression test that starts a thread with auto review, records
a turn, restarts with user review in config, resumes without an
override, and verifies that auto review is preserved.
- `just test -p codex-protocol`
- `just test -p codex-state`
- `just test -p codex-rollout`
- `just test -p codex-app-server
thread_resume_preserves_persisted_approvals_reviewer`
- Clippy for the affected crates
2026-07-09 00:58:28 +00:00