Commit Graph
292 Commits
Author SHA1 Message Date
joeytrasatti-openai 26a49b3882 Add paginated reverse lookup for thread attachments (#50083)
## What changed

Add `StateRuntime::list_thread_attachment_threads` to find threads by exact `attachment_type` and `identity_key`, including threads without previews. Return thread IDs and archive state, with filters for all, archived, or non-archived threads.

Order results by thread ID and bind pagination cursors to the attachment identity and archive filter. Add a database index on `(attachment_type, identity_key, thread_id)` to support the lookup.

## Testing

Add coverage for exact identity matching, archive filters, pagination, rejection of cursors reused with different lookup parameters, and membership updates after attachment removal or thread deletion.

GitOrigin-RevId: 21f32eb0cb1845a82d740d67b374080b6e12eb03
2026-10-01 19:55:36 +00:00
Dmitriy Kovalenko 596ae94fb0 Classify SQLite corruption using typed error codes (#49710)
## What changed

Preserve the underlying error in `LocalStateDbStartupError` and use its error chain to decide whether CLI startup failures qualify for automatic database backup and recovery.

Replace corruption checks against error-message text with SQLite result-code classification, including SQLx extended result codes and directly wrapped `SqliteError` values.

## Testing

Update the embedded startup corruption test to verify that the startup error retains a detectable SQLite corruption cause and the failed database path.

GitOrigin-RevId: 6e8ecd0ee67f3c239183ddea3fc75c4e4cd5744a
2026-09-30 19:00:00 +00:00
Dmitriy Kovalenko 3620b2caf8 Detect SQLite corruption during startup and preserve recovery backups (#49701)
## Why

A database can open successfully while containing corruption that startup previously missed. Detecting it before runtime initialization completes allows recoverable databases to be rebuilt while preserving the damaged files.

## What changed

- Run `PRAGMA quick_check(1)` when opening writable databases, with a 100 ms scan budget. Share attempts across cloned `SqliteConfig` instances by file identity so repeated opens skip validation and replacement files are checked again.
- Back up and rebuild runtime databases that support recovery. Report corruption without rebuilding thread history or other databases without an automatic recovery policy. Treat interrupted or locked checks as incomplete rather than confirmed corruption.
- Collect startup backups across recovery retries and include their locations in the app server warning. Explain that saved conversations can restore the thread list and history, while some database-only metadata may be unavailable.
- Record confirmed corruption in `codex.sqlite.corruption.count`.

## Testing

Add tests for recovery policies, backup preservation, validation caching and concurrency, cancelled and interrupted checks, and locked databases. Add app server coverage for recovery warnings and saved conversation restoration, a TUI warning snapshot, and message board post preservation.

GitOrigin-RevId: 43e26328888e0301def642d69bd324acee867566
2026-09-30 18:29:46 +00:00
Dmitriy Kovalenko 8ea2c0e0d4 Prune diagnostic logs periodically by age and database size (#49425)
## Why

Startup-only cleanup leaves expired logs in long-running sessions, and age-based retention alone does not limit database usage.

## What changed

Run cleanup in the background immediately after initialization and every 30 minutes. Start with the existing 10-day retention window, then halve it until occupied SQLite pages fit within a 64 MiB budget or the window reaches one second. Preserve logs within that final second even when they exceed the budget.

Exclude free pages from the budget so SQLite can reuse them, and retain the passive checkpoint after the first sweep. Stop the task when the runtime is dropped or the log pool is closed.

## Testing

Add tests for the 10-day boundary, repeated retention-window halving, free-page accounting, the one-second retention floor, immediate startup cleanup, and periodic cleanup without new writes or a restart. Update existing log tests to use current timestamps so background retention does not remove their fixtures.

GitOrigin-RevId: c6eeb5ec13006d4a03354e08322827429be53ce8
2026-09-30 01:34:12 +00:00
Dmitriy Kovalenko 16a7c0f0eb Filter graceful shutdown guard and trigger traces from SQLite logs (#49414)
## What changed

Set the default SQLite log filter to `DEBUG` for `tokio_graceful::guard` and `tokio_graceful::trigger`, dropping their `TRACE` events while retaining debug and higher-level diagnostics.

## Testing

Extend the SQLite sink filter test to verify that guard and trigger traces are dropped, their debug events are retained, and `tokio_graceful::shutdown` trace events remain available.

GitOrigin-RevId: c98495c834db35f3d8799ab4f0a1f7a82a53c8ef
2026-09-30 00:50:50 +00:00
jif c2837d8ece Batch metadata reads when resolving thread names (#49305)
## Why

Thread name resolution performs a separate SQLite metadata lookup for each thread. Batch these reads to reduce database queries for large result sets.

## What changed

Add `StateRuntime::get_threads` and use it in `resolve_thread_names`. Fetch metadata in batches of at most 999 IDs to stay within SQLite's bind limit, skipping missing or invalid records independently so valid metadata remains available.

## Testing

Add regression coverage for reads spanning multiple batches, invalid and missing records, duplicate IDs, and empty input.

GitOrigin-RevId: a403416f1da5adea56705c70839850760d3eb550
2026-09-29 16:36:07 +00:00
Benjamin Carlsson c2d2f422e6 Preserve SQLite vacuum modes and surface pool initialization errors (#49102)
## Why

Changing an existing database from `FULL` to `INCREMENTAL` auto-vacuum requires a writer lock and can block pool initialization. SQLx also retries connection initialization failures, masking the original error as `PoolTimedOut`.

## What changed

Preserve existing auto-vacuum modes when opening writable pools, while continuing to initialize empty databases with incremental auto-vacuum. Return the first connection initialization error directly to the caller, retaining SQLx's normal error handling for later lazy connections.

## Testing

Add regression tests for fresh database settings, opening an existing `FULL` auto-vacuum database under a write lock, and preserving the WAL conversion lock error while allowing a subsequent open to succeed after the lock is released.

GitOrigin-RevId: b586c68e43a5a0d3b47268378716919d01b749cf
2026-09-28 23:44:39 +00:00
Dmitriy Kovalenko 33a0f766a6 Reclaim unused SQLite log database pages in the background (#49069)
## Why

Deleting logs leaves free SQLite pages available for reuse without shrinking the database file. Reclaim excess free space while preserving a reserve for future writes.

## What changed

- Start a background incremental-vacuum worker with `StateRuntime`, enabling reclamation only for the logs database. Use a per-home file lock to select one worker and wait for it to finish during shutdown.
- Reclaim pages when at least 64 MiB and 25% of the database are free, preserving a reserve of at least 16 MiB or 10% of the database.
- Use short transactions with page and cooperative time budgets, defer work on contention, and reduce batch sizes with exponential backoff when interrupted passes make no progress.
- Record reclamation counts, durations, and reclaimed pages in telemetry.

## Testing

Add a regression test that interrupts a vacuum commit, verifies foreground writes can resume without data loss, then confirms a resumed pass shrinks the database and passes `PRAGMA integrity_check`.

GitOrigin-RevId: 8421f87b0aeef6540eda5690c84216906673c72e
2026-09-28 20:19:10 +00:00
jif 46fdd5ef39 Avoid SQLite stalls from connection setup and stderr span logging (#49032)
## Why

Setting `auto_vacuum` on every connection can wait for a writer lock even when the mode is unchanged. Logging every span entry and exit can also block SQLx workers on undrained stderr while they hold a write transaction.

## What changed

- Set incremental vacuum only for empty databases or databases using full auto-vacuum, before enabling WAL. Preserve existing nonempty databases with auto-vacuum disabled.
- Emit span creation and closure events in app-server text and JSON stderr logs, preserving busy/idle timings and explicit events while omitting span entry and exit records.

## Testing

Add regression tests for opening and expanding writable pools while another connection holds a writer lock, and for transaction completion with undrained stderr in both log formats. Verify vacuum modes and retained log diagnostics.

GitOrigin-RevId: 40d4ef32c7efc49f4240b987277ba45551259335
2026-09-28 17:00:25 +00:00
acrognale-oai 7596ab19a6 Keep archived threads with empty previews visible in listings (#48199)
## Why

Archived threads without previews were filtered out of thread listings, even though they had valid session metadata.

## What changed

- Include archived threads with empty previews in both SQLite and rollout-file listings.
- Add archive-specific indexes for creation, update, and recency ordering so these listings can use indexed pagination.

## Testing

Add regression coverage for paginated archive listings with and without SQLite, archive query plans that avoid temporary sorting, and metadata reads that preserve source and file timestamps for empty archived threads.

GitOrigin-RevId: e29d0257b2e029bea5fdf94eed6dab1006d2a8f2
2026-09-25 20:11:47 +00:00
Dmitriy Kovalenko 3d6428510f Warn users when SQLite diagnostic log writes fail (#47887)
## Why

Failed SQLite writes leave diagnostic logs incomplete. Notify users before they close Codex so they can submit logs or run `codex doctor`.

## What changed

- Add a best-effort warning on the first log write failure in standalone and in-process app servers, while continuing to retain failure diagnostics in memory.
- Handle failures that occurred before the in-process warning reporter was attached.
- Tailor the warning's guidance to whether submitting logs is enabled.

## Testing

Add coverage for warning deduplication, both guidance variants, retained diagnostics after the notification sender is dropped, and SQLite flush failures before and after reporter attachment. Extend log upload integration coverage and add a TUI warnings snapshot.

GitOrigin-RevId: 12a74126d13c3568430e539a180cca47fa931fa9
2026-09-24 17:04:50 +00:00
Dmitriy Kovalenko a285066737 Preserve diagnostic logs when SQLite logging fails (#47886)
## Why

SQLite write failures can leave persisted logs incomplete, while log snapshots taken before collection miss errors raised during that collection.

## What changed

- Report SQLite flush failures directly to the in-memory log buffer without re-entering tracing, using an error category and batch size instead of raw error details.
- Track whether the log writer has lost a batch and make app-server diagnostic uploads use the memory buffer when SQLite logs are incomplete.
- Refresh log bytes after flushing and querying SQLite so the same submission includes collection failures, while preserving captured metadata and thread identity.

## Testing

Add regression tests for failure reporting without recursion, subsequent write recovery, metadata preservation, and healthy and failing SQLite collection. Add an upload integration test that captures attachments locally and checks failure diagnostics, memory-buffer fallback, and omission of raw SQLite error details.

GitOrigin-RevId: 51b0c4b69c977d9f4e74f2f245a9fd90fdb13761
2026-09-24 17:04:24 +00:00
Charlie Marsh 44b857c00e Clean up legacy Guardian thread metadata in SQLite (#47303)
## Why

Guardian prompts are synthetic review context, but legacy thread metadata can duplicate them in titles, previews, and first messages. Keep SQLite metadata small while retaining rollout JSONL as the canonical record.

## What changed

Add a migration for Guardian threads that:

- Replaces blank or prompt-derived titles with `Guardian review`.
- Preserves nonblank names and distinct nonblank titles, using the title as the name when no name exists.
- Sets `preview` to `Approval review` and clears `first_user_message`.

## Testing

Add a migration test covering prompt-derived and blank titles, custom titles and names, and unchanged metadata for a non-Guardian worker thread.

GitOrigin-RevId: 65b5ff5200a39a19fb11f494de6dca7710ff6cc6
2026-09-22 15:05:27 +00:00
Charlie Marsh 4f6c106cdd Use compact display metadata for Guardian review threads (#47302)
## Why

Guardian review threads use synthetic approval prompts as user messages. Deriving thread titles and previews from those prompts exposes large, unhelpful summaries in thread listings.

## What changed

- Default Guardian threads to the title "Guardian review" and preview "Approval review" across metadata creation, rollout reads, and resume.
- Skip synthetic user messages and goal updates when deriving Guardian display metadata, leaving `first_user_message` unset for new threads.
- Preserve custom names, including legacy indexed names, when listing, reading, resuming, and migrating threads to paginated history.

## Testing

Add regression coverage for compact metadata, resume without session metadata, custom names before and after migration, and app-server persistence, listing, and rename/resume behavior.

GitOrigin-RevId: 42aa707db20c3e57ffa5fd4c91ee313710be10d7
2026-09-22 15:02:54 +00:00
chiam-oai 772abc9425 Preserve and expose thread item lifecycle timestamps (#47114)
## What changed

- Persist producer-recorded item start and completion times in thread history, preserving existing non-null values on updates.
- Return nullable `startedAtMs` and `completedAtMs` Unix millisecond timestamps from `thread/items/list`, and update the generated schemas and TypeScript and Python types.
- Leave timestamps unknown for existing rows and older writers.

## Testing

Add coverage for exact timestamp persistence, protocol round trips with present and missing timestamps, paginated item responses, and compatibility with older database writers.

GitOrigin-RevId: 894853481085782ff54db1704799e41446d15e51
2026-09-21 22:23:26 +00:00
Jeremy Rose 23ae65bb06 Persist thread creator identity in rollouts and SQLite (#47113)
## What changed

- Capture optional `creator_user_id` and `creator_account_id` from authentication when creating a thread, including forks.
- Carry these fields through thread storage, rollout metadata, and a SQLite migration. Leave them absent when unavailable or when reading older threads.
- Preserve recorded creator values across resume and metadata updates. On revert, recover missing rollout values from SQLite before writing replacement metadata.

## Testing

Add coverage for ChatGPT and API-key sessions, resume under another account, forks using current authentication, metadata extraction and upserts, and repeated reverts after creator fields are removed from a rollout.

GitOrigin-RevId: 9839fa42f9392f12226002e2c27d05d5656f5695
2026-09-21 21:22:01 +00:00
Ahmed Ibrahim 3627bb91b2 Avoid building a host SQLite driver for SQLx macros (#47084)
## Why

Row derives and migration embedding do not need query macros or a host SQLite driver.

## What changed

- Remove the `macros` feature from `sqlx` and depend directly on `sqlx-macros` with only `derive` and `migrate` enabled.
- Update state and thread-store derives and migration embedding to use `sqlx_macros`.
- Enable `sqlx-sqlite`'s `offline` feature directly to preserve query-description support in the target driver.
- Pin `sqlx`, `sqlx-macros`, and `sqlx-sqlite` to `=0.9.0` to keep their versions aligned.

GitOrigin-RevId: 8a8ef691ff2862a7dad978a578c5206b52dfc0b6
2026-09-21 18:30:31 +00:00
joeytrasatti-openai 19286b8819 Copy current thread attachments into non-ephemeral forks (#45579)
## What changed

Copy the source thread's current attachments when creating a non-ephemeral fork, including forks at an earlier turn. Copies receive new attachment IDs and creation timestamps while preserving resource identities and payloads. Attachment membership can then change independently on either thread; referenced resources are not copied.

Await the atomic copy before publishing the fork. If copying fails, log the error and allow the conversation fork to succeed without attachments. Resuming a fork does not copy attachments again.

Document that clients should use `forkedFromId` on `thread/started` and fetch the fork's attachments with `thread/attachment/list`; copying does not emit per-attachment updates.

## Testing

Add coverage for atomic rollback, independent attachment membership, inheritance across history cutoffs, successful conversation forks after copy failures, and resuming without restoring removed attachments.

GitOrigin-RevId: 939d2c6a3073ccfcd922380c6beda45d3deac606
2026-09-15 02:33:27 +00:00
Matthew Zeng b5544d5732 Persist disabled plugin IDs in thread settings (#44332)
## What changed

- Add `disabled_plugin_ids` to thread startup options, settings overrides, snapshots, and persisted turn contexts. Omitted overrides preserve the current list; an empty list clears it.
- Restore the selection on resume and capture it before truncating history for a fork. Prefer the owning thread's latest settings snapshot, falling back only to the latest turn context so stale history cannot resurrect an older selection.
- Freeze the selection at turn admission and retain compatibility with histories that lack the new fields.

## Testing

Add coverage for owned settings, explicit clears, latest-turn fallback, and legacy snapshots. Extend settings tests to cover plugin selection during runtime restoration and compaction, and verify initial turn persistence without extra settings checkpoints in both history modes.

GitOrigin-RevId: 75ffbff5763cd93cc5ac69d73b08b237f59bc009
2026-09-09 23:23:59 +00:00
joeytrasatti-openai 130d6e4fba Add paginated thread attachment listing to the state runtime (#44330)
## What changed

Add `StateRuntime::list_thread_attachments` with page sizes from 1 to 100 and stable keyset ordering by `created_at` and `id`. Return a continuation cursor when more attachments remain, and reject malformed cursors or cursors belonging to another thread.

## Testing

Add coverage for thread-scoped pagination, cursor validation, and attachment persistence across reopening the database and archiving/unarchiving a thread. Verify listings reflect attachment removal and cascading deletion when a thread is deleted.

GitOrigin-RevId: e07433e1a675669082cc58f86b0dc4535d365ba3
2026-09-09 23:04:22 +00:00
joeytrasatti-openai 589874be81 Add transactional thread attachment mutations to the state runtime (#43949)
## What changed

- Replace the `ThreadArtifact` model and related exports with attachment terminology.
- Add `StateRuntime::add_thread_attachment` and `remove_thread_attachment` using SQLite transactions. Repeated additions for the same thread, attachment type, and identity key return the existing record without changing its payload or creation time.
- Enforce limits of 100 attachments per thread, 64 KiB per serialized payload, and 256 bytes each for nonblank attachment types and identity keys. Removal returns the deleted record or `NotFound` and frees capacity immediately. Both mutations reject unknown threads.

## Testing

Add tests for idempotency, thread isolation, removal outcomes, capacity reuse, invalid inputs, unknown threads, and concurrent additions creating exactly one record.

GitOrigin-RevId: a68fe5076832f071524f1b4fc87d08b96ac270b7
2026-09-09 00:18:35 +00:00
joeytrasatti-openai 9d88e9ae08 Rename thread artifacts to attachments in the state database (#43927)
## What changed

Add a migration that renames `thread_artifacts` to `thread_attachments` and
`artifact_type` to `attachment_type`, and recreates the thread lookup index
under the attachment name.

## Testing

Extend the migration test to verify that existing attachment data survives the
rename, section metadata is preserved, and the version 50 migrator tolerates
the updated database with `ignore_missing` enabled.

GitOrigin-RevId: 43955c4f69b91f92217efc1dc8e6a551d0716059
2026-09-08 22:02:55 +00:00
rhan-oai 6515a72db7 Preserve runtime workspace roots across thread resume (#43848)
## Why

Resuming a thread should retain its selected workspace folders, including additional roots and explicit empty selections. Resume overrides also need to survive a subsequent resume when no turn has run.

## What changed

- Persist `runtime_workspace_roots` in startup metadata and thread settings snapshots, separately from explicit environment selections and permission-profile roots.
- Restore roots from the latest snapshot owned by the resumed thread, falling back to owned startup metadata only when no snapshot exists. Honor explicit `runtimeWorkspaceRoots` overrides, retarget the old `cwd` root when `cwd` changes, deduplicate roots, and validate restored paths for the current host.
- Checkpoint effective settings on resume and restored settings after revert. Reload resume configuration if saved workspace roots change during loading.
- Normalize Windows rollout path spellings when matching thread search results, preserving selection of the correct rollout after revert, including compressed rollouts.

## Testing

Add regression coverage for workspace restoration, empty and explicit overrides, foreign paths, compaction and revert, resume checkpoints without recency changes, concurrent settings persistence, and rollout search path matching.

GitOrigin-RevId: d98d9d34dd63934d441120916c61c12b69e7f062
2026-09-08 16:24:38 +00:00
jif 2cbbf0c9b5 Add memory dual writing and v2 readiness reporting (#43827)
## Why

Allow v2 memories to build in the background while the selected memory version continues supplying context, and let clients check whether v2 has enough consolidated data and a valid summary to use.

## What changed

- Add `memories.dual_write`, disabled by default, to run v1 and v2 extraction and consolidation concurrently with separate stores and directories.
- Add the experimental `memory/status` endpoint, returning `v2ConsolidatedThreads` and `v2Ready`. Readiness requires a valid v2 summary and a consolidation thread count meeting `minConsolidatedThreads` (default 20, supported range 1–4096).
- Persist the largest thread count from a successful consolidation across pruning and clear it on memory reset.
- Share v2 summary validation between consolidation and status reporting, and tag memory metrics with `memory_version`.

## Testing

Add tests for independent dual-write outputs, preserving existing notes without importing them into v2, and readiness behavior across consolidation ownership checks, missing summaries, configurable thresholds, thread deletion, and memory reset.

GitOrigin-RevId: 54417cc60bf419062401355c0da8c79119995f61
2026-09-08 14:23:23 +00:00
jif 3f76e88a48 Add configurable memory versions with isolated storage (#43797)
## What changed

- Add `memories.version` with `v1` as the default and `v2` as an opt-in selection.
- Route memory generation, summaries, and retrieval tools through the selected version. Store `v2` artifacts in `memories_v2` and lazily create a separate SQLite database, keeping jobs and outputs isolated while sharing the thread catalog.
- Preserve the initial memory version across extension config updates so summaries and retrieval tools use the same namespace.
- Clear both versions on memory reset and remove thread memory from both stores on thread deletion. Recognize both artifact roots in shell usage telemetry.

## Testing

Add regression coverage for version defaults and validation, independent job claims and outputs, deletion and reset across versions, and shell usage classification for both memory roots.

GitOrigin-RevId: aa799bf87ab6ec10c8f5668213931e57547d080b
2026-09-08 10:55:48 +00:00
faizan-oai 3921a30d6b Persist Daybreak preferences in thread metadata (#42854)
## Why

Clients need a per-thread Daybreak toggle that survives restarts and can be
restored on resume without changing the access program of an active or future
turn.

## What changed

- Add experimental `daybreakEnabled` support to `thread/metadata/update` and
  expose the saved value on thread responses.
- Store the preference in SQLite, including for threads without a first turn,
  and preserve it through reads, resumes, reconciliation, and restarts.
- Copy the preference when forking a thread while keeping subsequent updates
  independent.
- Keep `daybreakEnabled` separate from the per-turn `cyberAccessProgram` field;
  app-server does not derive one from the other.

## Testing

- Cover persistence across restarts, cold reads, reconciliation, and forks.
- Verify metadata updates do not alter the access program of active or later
  turns.

GitOrigin-RevId: bcf987a9fcd7b0ac8a88b1c8f0df06b33ed8ad6e
2026-09-04 21:17:25 +00:00
Can Sar 728cb12fe5 Expose thread originators through the app-server API (#42458)
## What changed

- Add the creation-time `originator` to app-server thread responses and `thread/started` notifications.
- Persist originators in thread metadata and preserve the first recorded value across list, read, resume, rollout, and SQLite paths.
- Add an `originators` allowlist to `thread/list` for hosted backends. The local app-server accepts omitted or empty values but rejects nonempty filters.

## Testing

- Cover originator persistence across server restarts and list, read, and resume requests.
- Verify local `thread/list` handling for nonempty, omitted, null, and empty originator filters.
- Heap-allocate large TUI dispatcher futures in affected tests to keep them within the Windows test-thread stack.

GitOrigin-RevId: 73ad6e61b8fdd2d93a22d1f971723a19ec7be3fc
2026-09-03 06:02:25 +00:00
felixxia-oai 5971d42847 Preserve verified answers across history compaction (#42293)
## Why

Compaction replaces the model history window, but host-verified user answers may
still be needed for the rest of the owning thread.

## What changed

- Add bounded, model-invisible retained context for verified
  `request_user_input` answers.
- Persist retained context in rollout events and compaction checkpoints, and
  restore it when a thread resumes.
- Remove answers when their source turns are rolled back, and prevent
  parent-local answers from being inherited by child forks.

## Testing

Add lifecycle coverage for deduplication and size limits, compaction and resume,
rollback, rollout migration, and child forks.

GitOrigin-RevId: c32900d3e8120a9899b71f0576683acd67a4b898
2026-09-02 15:07:11 +00:00
Owen Lin 5f79a92e39 Persist response token usage in rollout history (#41912)
## Why

Resumed threads need to continue per-turn and per-thread token totals without scanning arbitrarily far beyond the latest compaction checkpoint.

## What changed

- Add durable `TokenUsageRecord` rollout items with response, turn, thread, session, and root-turn attribution.
- Restore accumulated usage on resume and snapshot the latest record plus the compaction response ID in compaction checkpoints.
- Preserve root-turn lineage in persisted turn context, while ensuring forked child threads start with their own usage totals.

## Testing

- Cover usage accumulation across multiple responses and resumed turns.
- Cover local and remote compaction checkpoints, invalid remote compaction output, rollout reconstruction, and fork isolation.

GitOrigin-RevId: ef9e0c4a9102a08a2c382be4cdac68c84353c90a
2026-08-31 21:15:38 +00:00
rhan-oai f5636bb733 Restore thread cwd from owned settings snapshots (#41567)
## Why

Resuming a thread without an explicit `cwd` should restore that thread's latest
retained setting. Forked history can contain settings copied from another
thread, while compaction can move the latest setting outside the replay window.

## What changed

- Record the owning thread ID on new settings snapshots and only use snapshots
  owned by the resumed thread when restoring `cwd`. Older snapshots without an
  owner remain readable but do not override the startup `cwd`.
- Checkpoint the current settings after compaction and serialize checkpoints
  with settings updates so the retained history contains an accepted, current
  snapshot.
- Cover resume behavior across compaction, forks, reverts, legacy histories,
  and concurrent settings updates.

GitOrigin-RevId: 1efc9cf55472d6b88c465c4efe44672a232dbf64
2026-08-29 19:57:06 +00:00
Anton Panasenko f6726e8986 Add recency sorting to project/list (#41223)
## What changed

- Include `recencyAt`, derived from the newest non-archived assigned thread, in project responses.
- Let `project/list` sort by `position` or `recencyAt` in either direction, with recency as the default descending order for that key and projects without activity placed last.
- Encode the sort key, direction, and millisecond-precision anchor in new cursors while retaining existing ascending-position cursors.
- Add an index for project recency lookups and document the API's sorting and pagination behavior.

## Testing

- Cover recency updates when threads are assigned, archived, unarchived, or deleted.
- Cover ordering, null handling, tie-breaking, cursor pagination, invalid sort combinations, and cursor mismatches.

GitOrigin-RevId: 6165e2b9976b0b45cfa16bcd42978b67811f6df5
2026-08-27 23:43:46 +00:00
Brandon Zhang 0d654e6536 Track window and fork positions in turn metadata (#40987)
## What changed

- Add the zero-based `window_number` to Responses turn metadata and include it in prewarm requests.
- Persist `forked_from_ordinal_exclusive` separately from the physical history base so fork lineage remains accurate after reverts and cold resumes.
- Report context-inheriting subagents through `parent_thread_id` without also emitting fork lineage.
- Reserve the new metadata keys while accepting and filtering previously valid configured values so Codex-owned values take precedence.

## Testing

- Cover window numbering across compaction, resume, and fork operations.
- Cover fork cutoff persistence across direct forks, legacy rollouts, reverts into inherited history, and cold resumes.

GitOrigin-RevId: 288f6cd632254814c1f166668b33828082d799a1
2026-08-26 22:37:59 +00:00
Alexi Christakis 0b94751cc4 Add telemetry for SQLite log persistence (#40726)
## Why

SQLite log persistence needs visibility into batch size, write latency,
failures, and entries dropped before they reach the writer. Exporter
diagnostics must not feed back into the SQLite log sink and keep metric
exports active indefinitely.

## What changed

- Record write count, duration, batch bytes, entry count, largest-entry size,
  and queue drops with bounded outcome tags.
- Add explicit byte histogram boundaries for log batch metrics.
- Filter OTLP transport noise and the benign unsolicited HTTP/2 PING warning
  from SQLite log persistence to prevent export cycles while retaining other
  HTTP/2 warnings.

## Testing

- Cover metric values, tags, queue-drop reasons, and histogram boundaries.
- Verify successful and failed OTLP HTTP and gRPC exports become idle instead
  of generating recurring SQLite log-write metrics.

GitOrigin-RevId: a63224ae15945fe84adc977a8b7d55bb47a4de53
2026-08-25 22:46:49 +00:00
Adam Perry @ OpenAI 1bc02aea50 Sanitize credentials from Git remote metadata (#40713)
## Why

Git remote URLs can embed usernames, passwords, or tokens. Codex carries these
URLs through turn metadata and persisted thread metadata, so credentials must be
removed before a remote enters those paths.

## What changed

- Add `SanitizedGitUrl`, which parses Git URL and SCP-style remotes, strips
  authentication data, and preserves the conventional `git` SSH user.
- Use sanitized remotes when collecting Git metadata, enriching model requests,
  discovering cloud environments, updating thread metadata, and reading legacy
  rollouts. Reject or omit malformed remotes instead of retaining their raw
  contents.
- Keep API, schema, and TypeScript representations as strings while enforcing
  sanitization in Rust.

## Testing

- Cover URL schemes, SCP and IPv6 forms, remote helpers, encoded paths,
  malformed values, and legacy deserialization.
- Verify credentials do not appear in model requests, API responses, SQLite, or
  rollout files.

GitOrigin-RevId: 6435efc4c45bfbfad4723ce7a0457cb00175f25c
2026-08-25 21:34:05 +00:00
chess 32ce703a80 Bound SQLite integrity checks for diagnostic attachments (#40688)
## Why

Diagnostic uploads skip the doctor attachment if report generation exceeds its
overall timeout. SQLite integrity scans can be delayed by large databases or
lock contention.

## What changed

- Give each database integrity scan a one-second deadline when collecting a
  diagnostic attachment.
- Interrupt scans and lock waits at the deadline, preserve any corruption rows
  already returned, and report an incomplete clean scan as a warning.
- Keep direct `codex doctor` integrity checks unbounded.

## Testing

- Cover interrupted scans, lock waits, retries without a deadline, and
  corruption results returned before interruption.

GitOrigin-RevId: 4817fb459ac53953669a9ba5f03397a6fc82c72e
2026-08-25 20:09:55 +00:00
faizan-oai 94967e03e5 Add per-turn cyber access program selection (#40687)
## What changed

- Add the experimental `turn/start.cyberAccessProgram` option with `standard`,
  `daybreakBlue`, and `daybreakRed` values.
- Forward the selection as `access_programs.cyber` on Responses, WebSocket,
  and remote-compaction requests made with ChatGPT authentication, while
  omitting it for API-key and custom-provider requests.
- Preserve the per-turn selection across recovery, compaction, and child-agent
  turns without making it a persistent thread setting.

## Testing

- Cover app-server forwarding, authentication boundaries, WebSocket reuse,
  turn recovery, compaction, and child-agent inheritance.

GitOrigin-RevId: d2eb468f365b2214c5099bc21741cf30f8bd2eb5
2026-08-25 19:54:52 +00:00
Adam Perry @ OpenAI 2f4b105514 Prevent telemetry exporters from feeding back into themselves (#40658)
## Why

Telemetry transport and storage can emit their own diagnostics, causing exports
or SQLite writes to recursively generate more telemetry.

## What changed

- Exclude `h2` spans from trace export so OTLP transport activity cannot trigger
  additional OTLP exports.
- Exclude all `sqlx`, `sqlx_core`, and `sqlx_sqlite` diagnostics from the SQLite
  log sink, including when no external subscriber filter is installed.
- Keep similarly named targets such as `sqlx_application` eligible for storage.

## Testing

- Extend the SQLite log-filter test to cover the broader SQLx exclusion and the
  unrelated-target case.

GitOrigin-RevId: 13a832e1f228e9564a31cc08f00add286342bac1
2026-08-25 17:59:32 +00:00
joeytrasatti-openai 9b2ef38f54 Add persisted thread artifact models (#40509)
## What changed

- Add a `thread_artifacts` SQLite table with per-thread typed identities,
  JSON payloads, cascade deletion, uniqueness enforcement, and an index for
  ordered per-thread reads.
- Export artifact records, attachment and removal outcomes, and paginated
  result models from `codex-state`.

## Testing

- Add a migration test covering preservation of existing thread-section
  metadata and compatibility with the previous migration set.

GitOrigin-RevId: 4e0a8e5bfa168ae2b8871970466e5055e142a4a6
2026-08-25 01:10:10 +00:00
guinness-oai 41ca88e95c Persist realtime events in the thread timeline (#40508)
## Why

Clients need a bounded view of realtime conversations that preserves the order of speech, agent work, and turn lifecycle events without loading the full thread history.

## What changed

- Persist realtime session boundaries, transcript segments, and promoted agent items in paginated thread rollouts.
- Add the experimental `thread/timeline/list` API to page ordinary items, realtime items, and turn boundaries in canonical rollout order. Include `activeRealtimeSessionAtPageStart` so each page can be rendered independently.
- Emit typed `thread/realtime/item/started`, `thread/realtime/item/transcript/delta`, and `thread/realtime/item/completed` notifications around durable realtime items.
- Keep the existing `thread/items/list` response and legacy thread history behavior unchanged.

## Testing

- Cover mixed timeline pagination, realtime-session state at page boundaries, rollout replay, subagent history prefixes, transcript streaming, steering, and promoted agent artifacts.

GitOrigin-RevId: faa2c420192d63e060e8fb32446e3c797422656e
2026-08-25 01:03:20 +00:00
Adam Perry @ OpenAI 493e0efb7b Prevent SQLx warnings from feeding back into SQLite logs (#39592)
## Why

Warnings emitted while querying SQLite or acquiring a connection must not be
written back through the same SQLite log sink.

## What changed

- Disable the `sqlx::query` and `sqlx::pool::acquire` targets for the SQLite
  log filter.
- Keep warnings from other `sqlx` targets so useful diagnostics remain
  available.

## Testing

Extend the log filter test to verify that query and pool-acquisition warnings
are dropped while another `sqlx` warning is retained.

GitOrigin-RevId: cec9d4e07d113de8eccec7445ede0ae68e21886d
2026-08-20 05:16:59 +00:00
joeytrasatti-openai 1b450c7912 Persist thread section moves before the first turn (#39523)
## Why

New non-ephemeral threads have no persisted rollout or preview until their first
turn, so moving them into a section could leave them absent from section-filtered
thread lists.

## What changed

- Materialize and flush a non-ephemeral thread before applying an explicit
  section move.
- Include explicitly sectioned threads with empty previews in section-filtered
  queries, and update the partial indexes to cover them.

## Testing

- Add an app-server test that moves a newly started thread into and out of the
  pinned section before its first turn.
- Cover empty-preview threads in recency and section-position query tests.

GitOrigin-RevId: aa3c4daf9890f06c08b93cd60f146099cbc6cfb2
2026-08-19 17:57:40 +00:00
Eric Traut 701db965e5 Increase SQLite log sink batching (#39294)
## What changed

- Increase the default bounded log queue capacity from 512 to 2,048 entries.
- Raise the default insert batch size from 128 to 512 entries.
- Extend the default periodic flush interval from 2 to 10 seconds.

GitOrigin-RevId: e96be0996152b4a2a84d88c23d36959436685682
2026-08-18 22:05:07 +00:00
Owen Lin f950a1ba0b Preserve thread names during rollout migration (#39273)
## Why

Legacy threads and paginated threads read their displayed names from different metadata. Moving a rollout to paginated history must carry over its legacy-visible name so renamed threads do not lose it.

## What changed

- Load legacy indexed names once for an apply run.
- When promoting a thread, preserve an existing canonical name or copy its distinct legacy title, falling back to the legacy name index.
- Repair missing names when migration encounters an already-paginated rollout.

## Testing

Added migration tests covering preservation from both legacy name sources and repair on a rerun.

GitOrigin-RevId: 5fc526570b1cf358a159ef174c159ac091ccc40f
2026-08-18 20:07:26 +00:00
Shijie Rao 230791fd1f Persist active permission profiles in turn context (#39145)
## What changed

- Add the optional `active_permission_profile` field to `TurnContextItem` to record the built-in or named profile that produced the effective permission profile.
- Populate the field when serializing a turn context while preserving compatibility with records where it is absent.

## Testing

- Add coverage verifying that turn-context serialization stores the active permission profile.

GitOrigin-RevId: d69142ab1b26595ab406235f89818ef41370e963
2026-08-18 05:46:08 +00:00
Eric Traut eeb82a156d Dispatch queued messages written by other processes (#39034)
## Why

Durable thread queues can be updated through another SQLite connection, but loaded idle threads need to notice those updates before they can dispatch the new messages.

## What changed

- Track a durable revision for each thread queue, including backfilling existing queues and updating revisions on inserts, updates, and deletes.
- Poll SQLite's data version and query revisions to find changed queues for loaded threads.
- Wake idle threads with pending external messages, discover queued work when threads are loaded or resumed, and retry each thread independently so one blocked queue does not stall others.

## Testing

- Cover cross-runtime queue writes, edits, independent dispatch, wake retries, and resumed threads.
- Cover migration backfills and revision tracking for queue updates and deletes.

GitOrigin-RevId: 906d902bbf6a760be206a1b2c08fd71427b63c46
2026-08-17 17:19:46 +00:00
Anton Panasenko 3b4569a920 Add experimental app-server project APIs (#38940)
## What changed

- Add SQLite-backed `project/list`, `project/read`, `project/create`,
  `project/import`, `project/update`, `project/move`, and `project/delete`
  endpoints with ordered roots, metadata, manual positioning, pagination, and
  idempotent creation.
- Add project assignment to thread start, metadata updates, list filtering,
  and fork inheritance. Project deletion clears assignments without deleting
  threads.
- Emit `project/changed` and `thread/project/updated` notifications after
  committed changes, and export the new protocol schemas and TypeScript types.

## Testing

- Cover project lifecycle and ordering, idempotency, atomic imports, cursor and
  filter validation, thread assignment, deletion, and fork inheritance.

GitOrigin-RevId: 47ae621be01ab8ef70a74a629fb29a5c5709ea33
2026-08-17 04:01:37 +00:00
Benjamin Carlsson 375996d3f5 Restore thread timestamp maxima independently (#38893)
## What changed

Load the persisted maxima for `updated_at_ms` and `recency_at_ms` with separate
scalar subqueries during state initialization. This restores each timestamp
counter independently when the maxima belong to different threads.

## Testing

Add a regression test that reopens the state database and verifies both
independent maxima are restored.

GitOrigin-RevId: aeb9cc005d52c4a02dc7a220a817e6d40c0e01d4
2026-08-16 19:37:39 +00:00
efrazer-oai 9341b38310 Add experimental thread queue APIs to app server (#38456)
## What changed

- Add experimental `thread/queue/add`, `list`, `update`, `delete`, `reorder`, and `start` requests for persistent queued user submissions.
- Automatically dispatch queued submissions in FIFO order after completed or failed turns, while leaving the queue paused after interruption.
- Emit `thread/queue/changed` notifications and expose paginated queue contents with stable submission and client message IDs.
- Export the new protocol types through the generated JSON Schema and TypeScript bindings.

## Testing

- Add app-server integration coverage for capability gating, CRUD, pagination, capacity limits, notifications, automatic dispatch, cold-thread resume, interruption, and explicit queue starts.

GitOrigin-RevId: 89caf5577192199bda77cf6c372a59d26cdd910e
2026-08-13 23:38:02 +00:00
jif 72fa74fbc9 Persist security risk scores in rollout history (#38363)
## What changed

- Add a `SecurityRiskScore` rollout item containing a category and numeric score.
- Persist the item in both thread history modes while excluding it from model context, user-visible thread history, search text, forks, and reconstructed conversation history.
- Re-export the score type from the extension API.

## Testing

- Cover serialization, persistence and loading, thread history projection, session reconstruction, append planning, and memory filtering.

GitOrigin-RevId: 1926fe366aeaa75052708a6da589f45a38eefb52
2026-08-13 12:04:29 +00:00
Owen Lin b1373b74a2 Add durable reverts for paginated threads (#38292)
## What changed

- Add `ThreadStore::revert_thread` to retain history before a selected turn by creating a new immutable rollout and atomically switching the thread's stored rollout path.
- Preserve the logical thread ID and session metadata across repeated reverts.
- Track every rollout owned by a thread so archive, unarchive, and delete operations handle the complete lineage and restore moved files when metadata updates fail.
- Prevent deletion when forks still reference any rollout in the thread's lineage.

## Testing

- Cover repeated reverts, compressed source rollouts, lifecycle operations over reverted histories, reference-aware deletion, and cross-process unarchive locking.

GitOrigin-RevId: 1c05a6db3d0036ee91b562263d7ca200c6a0ea71
2026-08-13 01:56:24 +00:00