321 Commits
Author SHA1 Message Date
Henry Levy d78f68f8d1 Allow per-turn overrides for direct-only tool namespaces (#59481)
## What changed

- Add `direct_only_tool_namespaces` to `TurnStartOptions`. Omission inherits the thread configuration; an empty list clears direct-only routing for that turn. Steering ignores the option.
- Persist explicit overrides in turn context and restore them when recovering an interrupted turn, including turns started by standalone tool output.
- Keep subsequent turns and spawned or resumed child agents on the thread's routing defaults.

## Testing

Add coverage for routing changes across turns in Responses and Responses Lite, recovery without carrying overrides into subsequent turns, and child-agent inheritance.

GitOrigin-RevId: cea1a997424572d4cd372cd15b098626904c6796
2026-11-04 19:58:49 +00:00
ioanbudea-OAI c141c31323 Preserve requested approval reviewers across policy overrides (#59272)
## Why

A model requirement can force `auto_review` even when the requested reviewer is `user`. Persisting or sending that effective value back as an explicit choice loses the original selection, so removing the requirement cannot restore it on resume.

## What changed

- Retain `requested_approvals_reviewer` separately from the effective reviewer in configuration and thread settings, with a fallback for older histories.
- Restore the requested reviewer on resume while applying current model requirements and reviewer allowlists.
- Let ordinary TUI turns, continuations, retries, and forks inherit the server's reviewer instead of echoing its effective value as an explicit request.
- Preserve a worker's requested reviewer when reloading it rather than inheriting the sender's selection.

## Testing

Extend resume coverage for explicit `auto_review`, removal of a model requirement, and a tightened reviewer allowlist. Update TUI assertions for omitted reviewer overrides and verify worker reload preserves its reviewer.

GitOrigin-RevId: d8c9c3a7b89e55e63f0e8ae210d6a87cb5547933
2026-11-04 00:31:24 +00:00
aojiao-oai 16ddd538bd Expose assistant content token counts in completion notifications (#59041)
## What changed

Add optional `contentTokenCount` metadata to `item/completed` notifications, carrying the producer-supplied token count for each completed assistant message in default and plan modes. Counts include any sampled footer and are `null` when unavailable.

Keep this telemetry separate from stored thread items and omit it from subsequent model requests. Update the protocol schemas and generated TypeScript and Python types.

## Testing

Add coverage for per-message counts, including zero, in default and plan modes; missing and malformed metadata; backward-compatible notification decoding; and exclusion from subsequent model requests.

GitOrigin-RevId: 35973037a42650fa2d7569da8e88cb81215cf971
2026-11-03 02:26:29 +00:00
Dmitriy Kovalenko c60cda34a3 Make observed thread metadata writes atomic and retryable (#59016)
## Why

A failed memory-mode update could leave other metadata changes applied. Failed observed metadata writes for paginated threads also need to remain pending so persistence can retry them.

## What changed

- Apply observed metadata, memory mode, and optional recency advancement in one SQLite upsert, preserving monotonic recency.
- Require successful SQLite writes for paginated thread metadata when a state runtime is configured, retaining failed writes for retry.
- Use SQLite busy-retry helpers for read-state writes and immediate transactions.

## Testing

Add regression tests that reject SQLite writes to verify pending metadata survives a failed persist and a rejected memory-mode update leaves existing metadata unchanged. Both tests verify a subsequent retry succeeds.

GitOrigin-RevId: 652bf6ebee0197976852aaf9f096b907697fd590
2026-11-03 00:16:13 +00:00
Dmitriy Kovalenko 9e77fb4ad5 Make SQLite schema migrations safe for concurrent startup (#59015)
## Why

SQLx's SQLite migration lock is a no-op, allowing concurrent database openers to race between schema version checks and migration changes.

## What changed

Run schema checks and migrations in one transaction, retrying SQLite busy errors from a fresh snapshot. Databases with a current schema need only a read lock, including on NFS. Include the error cause chain when the app server reports SQLite initialization failures.

## Testing

Add regression tests for eight concurrent database openers applying each migration once and for reopening a current schema while another connection holds a writer reservation with NFS settings.

GitOrigin-RevId: 38004d58a5c637ac1b2be53d07fd9a4c8efec7ab
2026-11-03 00:04:40 +00:00
jif 8806313db4 Scope local history discovery to the current ChatGPT account (#58918)
## What changed

- Add `currentAccountOnly` to `thread/list` and `thread/search`, defaulting to `false`, and enable it for TUI history discovery, resume pickers, and task mentions.
- Mark newly created threads as account-scoped and require each recorded creator user or account ID to match the current login. Older threads and threads without creator IDs remain shared; API-key logins continue to see all local history.
- Preserve account scope through resume and revert, including fallback to original rollout metadata when an older binary drops the marker. Forks receive new-thread scope.
- Skip unreadable rollout headers in scoped discovery and filter search candidates before reading snippets. Direct access by thread ID remains available.

## Testing

Add coverage for account matching, shared legacy history, list page refilling, search pagination, API-key visibility, and scope preservation across reverts. Extend rollout lookup coverage to compressed active and archived files.

GitOrigin-RevId: aa374f5e69165be5a71c0716ded1b105f5fb51e0
2026-11-02 16:00:20 +00:00
jif 020e0f7708 Preserve timestamp ties when paginating threads (#58886)
## Why

Timestamp-only cursors can skip threads that share the last result's timestamp, including rollout files created in the same second.

## What changed

- Include thread IDs as tie-breakers in time-sorted listing and search cursors, and use them when resuming filesystem scans and database queries.
- Share filesystem sort-key and cursor construction so continuation cursors match the listing order, including filename-based creation times.
- Preserve support for legacy timestamp-only cursors and their existing behavior of skipping ties.

## Testing

Update regression coverage for pagination through filesystem timestamp ties, legacy cursors, thread IDs in database fallback cursors, and search cursors for all time-based sort keys.

GitOrigin-RevId: 455b95452849e6ffe20a27d7bdcec864f2e8135a
2026-11-02 13:40:04 +00:00
andrewgu-oai c62e9df39f Preserve reasoning token accounting across resume and fork (#57839)
## Why

Resuming or forking a thread loses the session flag indicating that reported token usage already includes retained reasoning. Counting that reasoning again can trigger automatic compaction too early.

## What changed

Persist `last_usage_includes_reasoning` with `TokenUsageInfo` and use it when estimating context usage. Record the flag with completed sampling and compaction usage, and mark local estimates as including reasoning. Older records without the field retain the existing accounting behavior.

## Testing

Add coverage for saved inclusive, exclusive, and legacy token counts. Extend the encrypted-reasoning compaction test to verify the compaction threshold for live, forked, and cold-resumed threads.

GitOrigin-RevId: 9f22262a693111ec0406dce11804eb4844ad8983
2026-10-28 21:22:06 +00:00
Brent Traut e6c5a83799 Include directly attached threads in diagnostic uploads (#57824)
## What changed

When a diagnostic upload includes logs, include saved threads attached directly to the reported thread in the existing bounded thread selection alongside spawned descendants. Query attached threads from the state database, including archived threads, so their rollouts, logs, and retained review failures can be collected. Log a warning and continue if the lookup fails.

## Testing

Extend the upload integration test to verify that the rollout archive includes the reported and directly attached threads, excludes unrelated threads, and is omitted when logs are disabled.

GitOrigin-RevId: 1bcd2794938339fb4b1c433c3731f507601cd8a4
2026-10-28 20:21:30 +00:00
Steve Coffey f088e0c4ff Allow replacing dynamic tools between model steps (#57553)
## What changed

Add `CodexThread::replace_dynamic_tools` to replace the tool catalog without starting a turn. Updates apply to subsequent model steps and future turns while preserving in-flight calls; an empty list clears the catalog.

Require host opt-in through `allow_persistent_dynamic_tool_updates` for persistent threads. Persist replacements before publishing them, restore the latest catalog on resume, and preserve replacements and explicit clears through compaction. Keep explicit resume overrides separate from the persisted catalog.

## Testing

Add coverage for pending calls during replacement, replacement and clearing across restart, compaction recovery, resume overrides, submission ordering after caller cancellation, host opt-in, and persistence failures.

GitOrigin-RevId: fd2634906895a9b72ae7892ddd9435dced45eccb
2026-10-27 19:50:41 +00:00
Charlie Marsh dddc3c0a79 Skip unrelated indexed rollouts during selected thread migration (#57483)
## Why

Migrating selected threads reads rollout headers even for unrelated threads. Use the state database to skip files whose indexed owner and canonical filename agree that they belong to an unselected thread.

## What changed

- Read thread IDs and rollout paths without creating or migrating the state database, and fall back to header inspection if the index cannot be read immediately.
- Keep inspecting selected, unindexed, misnamed, and conflicting paths so rollout headers still determine ownership.
- Recognize stable thread IDs in reverted rollout filenames and match plain and compressed paths when filtering.

## Testing

Add a regression test covering pruning of unrelated indexed rollouts, including archived and compressed files, while retaining header inspection for selected and ambiguously named files.

GitOrigin-RevId: d76827174c00f5bc9e38ad9b69dadeee91f4415f
2026-10-27 15:19:58 +00:00
Dmitriy Kovalenko ca8b7a0737 Allow NFS database migration with shared file permissions (#57226)
## Why

NFS database migration rejected files with group or other access. Allow accessible databases to migrate while making the effects on file access explicit.

## What changed

- Preserve file mode bits during migration while requiring owner read and write permissions on replacement files.
- Before CLI consent, disclose that replacements belong to the current user and do not copy the original owner, group, or extended ACLs. Require explicit confirmation for files with permissions other than `0600`.
- Warn at Linux TUI startup when NFS databases need migration and permission confirmation. Inspect journal mode without accessing WAL shared memory or releasing another connection's locks.
- Allow five seconds to acquire the exclusive database lock during migration.

## Testing

Add CLI coverage for consent, access disclosures, and migration of `0644` files. Extend state tests for permission preservation, rejection of missing owner access, and interrupted migration recovery; snapshot the startup warning.

GitOrigin-RevId: 2d9c3663172c3e631a787dc1dc76db95c39e6aa1
2026-10-26 20:16:30 +00:00
Dmitriy Kovalenko 480ab2d006 Add managed enforcement of DELETE journaling on NFS (#57225)
## Why

Existing databases retain their journal mode, so selecting `DELETE` for new NFS databases leaves existing WAL databases unchanged.

## What changed

- Add the managed `unsafe_enforce_sqlite_journal_mode` requirement to migrate existing NFS databases before opening writable pools and require `DELETE` on each connection.
- Share migration code between startup and `codex migrate-nfs-db`, preserving WAL backups and resuming interrupted replacements, including lazily opened databases.
- Hold the migration lock through pool initialization and keep migration failures out of automatic corruption recovery.
- Preserve local logs behavior and skip legacy NFS logs cleanup when enforcement is enabled.

## Testing

Add coverage for requirement precedence, database settings, data and backup preservation across restarts, interrupted replacement recovery, refusal of active WAL databases, migration lock waiting, and local logs remaining in WAL mode.

GitOrigin-RevId: da41ec1b58b1ee8092591cdf8b358280928ff9d4
2026-10-26 20:10:45 +00:00
Dmitriy Kovalenko 0d7f3ea03b Initialize SQLite databases with DELETE journaling on NFS (#56552)
## What changed

Choose `DELETE` journaling for new SQLite databases on NFS and `WAL` elsewhere, using the database's own directory to detect the filesystem. Probe a separately configured logs directory independently of the state directory.

Preserve journal modes for databases with an existing schema, and apply `synchronous = EXTRA` when initialization results in `DELETE` mode.

## Testing

Extend SQLite tests to cover NFS and non-NFS initialization, interrupted initialization, lazy pool connections, separate logs directories, and preservation of existing settings under a write lock.

GitOrigin-RevId: bde35861e517881b9ce23b28b199475cee70bd76
2026-10-23 18:30:44 +00:00
Brent Traut 30587d7885 Add experimental attached thread forks (#56351)
## What changed

- Add `attachToSource` to `thread/fork` to persist a fork attached to its source without emitting `thread/started`. Attached forks require local storage and cannot use `ephemeral` or `deferGoalContinuation`.
- Hide attached threads from ordinary listings and add `attachedToThreadId` to `thread/list` for explicit discovery, including empty threads. Keep this relationship separate from spawned-agent parent and ancestor filters.
- Persist attachment metadata in rollouts and SQLite so it survives restarts and database recovery. Reject source deletion until its attached threads are deleted.

## Testing

Add coverage for legacy and paginated history, restart and missing or corrupt database recovery, filtered listings and pagination, resume, conflicting options, and deletion safeguards across store instances.

GitOrigin-RevId: 589f3174fc36f6054e75d65818360695645a5913
2026-10-23 04:52:47 +00:00
Dmitriy Kovalenko 17a0e6f63f Route SQLite logs to an explicitly configured log_dir (#55978)
## What changed

- Store `logs_2.sqlite` in `log_dir` when explicitly configured, unless a managed `sqlite_home` requirement applies. Keep other databases in `sqlite_home` and preserve the existing SQLite log location when `log_dir` is unset.
- Initialize and lock the selected log directory only when opening its database, so an unavailable directory does not prevent authentication commands from loading configuration. Share the cleanup lock across configuration clones and allow failed initialization to be retried.
- Leave existing logs in place. Read legacy logs for diagnostic submissions when the current query returns no logs, attaching them as `codex-logs-legacy.log`. Continue retention cleanup and page reclamation for both locations, with independent maintenance locks and filesystem checks.

## Testing

Add coverage for configuration routing and managed requirements, deferred initialization and lock lifetimes, authentication with an unavailable log directory, legacy log attachments, and cleanup of current and legacy databases.

GitOrigin-RevId: 48216694596d738eff1b3c5eef6c4e7ada6d0137
2026-10-21 19:53:00 +00:00
Charlie Marsh c7234c1174 Box session metadata and compaction payloads in RolloutItem (#55733)
## What changed

Store `RolloutItem::SessionMeta` and `RolloutItem::Compacted` payloads in `Box` values and update their producers, consumers, and test fixtures. Preserve the existing serialized rollout format.

GitOrigin-RevId: 59fcc4beb47d23160872dcb6e95ace9b142d3d11
2026-10-21 02:41:05 +00:00
Dmitriy Kovalenko 9c84e89ce0 Skip SQLite page reclamation on Linux NFS mounts (#54417)
## Why

SQLite page reclamation on NFS creates I/O load and contention.

## What changed

Detect whether `sqlite_home` is on NFS on Linux and skip reclamation before opening a maintenance connection. Cache successful filesystem probes across `SqliteConfig` clones while allowing failed probes to retry.

## Testing

Add tests for shared probe caching and recovery after a missing directory is created. Update reclamation coverage to expect no reclaimed pages or file shrinkage on NFS.

GitOrigin-RevId: 4f627cabc6ad712171654c3f8b8659d2f632bbba
2026-10-15 21:07:40 +00:00
Dmitriy Kovalenko 1ef30bcbe7 Preserve existing SQLite journal modes on startup (#54389)
## Why

Opening an initialized database in `DELETE` journal mode previously attempted to convert it to `WAL`, which could fail while another connection held a write lock.

## What changed

- Initialize `WAL` and incremental auto-vacuum only for databases without a schema, including files left by interrupted initialization.
- Preserve journal and vacuum modes for initialized databases, use `synchronous = EXTRA` for `DELETE` connections, and retain `NORMAL` for `WAL`.
- Close failed initialization pools before retrying and restrict background page reclamation to `WAL` databases.

## Testing

Extend SQLite tests to cover interrupted initialization, opening `DELETE` databases under a write lock across all pool connections, persistence after reopening, and lock release after failed initialization.

GitOrigin-RevId: 23acecfe472618dde8545815e9a1058a51591235
2026-10-15 19:15:04 +00:00
willwang-openai 7ae8319917 Retry SQLite operations on transient writer contention (#54369)
## Why

Competing SQLite writers can cause database opening and writes to fail with `SQLITE_BUSY`. Deferred write transactions can also encounter contention after taking a read snapshot.

## What changed

- Add up to two retries with 100 ms and 200 ms delays for SQLite busy errors during pool opening, writer admission, and standalone writes.
- Use a shared `BEGIN IMMEDIATE` helper across state, thread history, and agent message board writes to acquire the writer slot before reading.
- Preserve bound arguments when retrying standalone writes, and commit writes using `RETURNING` before exposing their results. Retry transaction admission without replaying an already-started transaction.

## Testing

Add tests using real SQLite writer contention to verify recovery after a competing writer releases its lock. Queue tests cover enqueue, update, and delete, checking preserved payloads, single revision increments, and no duplicate messages or unintended deletions.

GitOrigin-RevId: ac5bd31a87c67cc47fb385bf3fa5435a54ff70f0
2026-10-15 18:06:57 +00:00
pakrym-oai 5ef96ab278 Add opt-in output token replay for OpenAI requests (#52742)
## What changed

- Add the disabled-by-default `output_token_replay` feature to request `output.encrypted_content` from OpenAI providers.
- Preserve encrypted message and tool-call output, message and function-call status, and output-text `annotations` and `logprobs` through history and replay. Update the generated protocol schemas and SDK types.
- Strip output ciphertext from requests to other providers while retaining it in stored history, including after replay is disabled. Remove encrypted content from memory extraction inputs.

## Testing

Add integration coverage for the feature/provider combinations, metadata replay across follow-up requests, and history preservation when resuming with replay disabled.

GitOrigin-RevId: 77729a021b6771933402aebe74ece8eb39c057e2
2026-10-10 01:25:00 +00:00
Brent Traut 0ada5d8806 Expose experimental durable thread read state in the app server (#52350)
## What changed

- Add experimental `readState` to `thread/read` and a thread-ID-keyed `readStates` map to `thread/list` for eligible durable local user threads. Each receipt contains `firstUnread` and an opaque `revision`.
- Publish new visible terminal results before `turn/completed`, after flushing the rollout. Reading snapshots leaves activity unread; replayed history does not publish new activity.
- Clear unread positions removed by a paginated thread revert while preserving explicit thread-start marks and concurrent receipt updates.

## Testing

Add an integration test covering completed-turn receipts, repeated reads without acknowledgement, matching list receipts, and revert cleanup with a changed revision.

GitOrigin-RevId: e7f6ddfbb1fc2fa95d1ef2876d7f0f7b9f99ff65
2026-10-09 01:22:30 +00:00
Brent Traut b6e8a2e207 Add durable thread read state with revision-checked updates (#52337)
## Why

A stale read acknowledgement must not clear unread results published after the client’s snapshot or overwrite another read-state edit. Read state also needs to survive thread metadata rebuilds.

## What changed

- Add `thread_read_receipts` and state APIs to query read state, publish thread attention, and mark threads read or unread.
- Track the first unread turn and require matching revisions for explicit edits, returning the current state on conflicts. Allow manually marking even an empty thread unread.
- Retain the last published turn across acknowledgements to suppress duplicate delivery. Keep receipts independent of rebuildable metadata and remove them on thread deletion.
- Return initial read states without creating receipt rows.

## Testing

Add tests covering stale edits, thread-specific initial revisions, repeated unread marks, duplicate publication, persistence across metadata rebuilds and restarts, clearing a reverted turn’s unread position, and deleted-thread handling.

GitOrigin-RevId: fee8c05c20903b1d7aea7e8b61c9248c7f623a72
2026-10-09 00:41:27 +00:00
zbarsky-openai 9dd798805f Prevent recursive Tokio tracing in telemetry and SQLite logs (#51775)
## Why

Tokio runtime tracing can cause SQLite log writes to enqueue more writes and exporter activity to generate further OTLP exports.

## What changed

- Exclude `tokio` and `runtime` spans from OTLP export alongside `h2` spans.
- Limit `tokio::` and `runtime::` targets to `INFO` in the default log filter, and reject their `TRACE` events directly in the SQLite sink even without an outer filter.
- Box and pin the session-loop future before spawning it.
- Reorder tracing and Tokio test attributes, restrict the skills concurrency test's subscriber to the catalog-refresh span, and use the default log filter in the SQLite formatter comparison test.

GitOrigin-RevId: f545013a2469088d7f9890c1f46bb93eb5446d2d
2026-10-07 17:28:32 +00:00
jif 30bdfec59d Persist Guardian review failures for reports across restarts (#51651)
## Why

Failed Guardian review evidence was process-local, so restarting Codex lost the context needed for later diagnostic reports.

## What changed

- Persist failed-review records in SQLite before reviewer cleanup, retaining at most 64 records globally, eight per thread, and 8 MiB of evidence. Delete persisted evidence when its reviewed thread is deleted.
- Recover and deduplicate persisted and in-memory records for the reported task tree when uploading `auto-review-failures.jsonl`.
- Bound persistence attempts to 250 ms and retain an in-memory fallback when storage is unavailable. Keep capture gated by user opt-in and non-ephemeral sessions, and export evidence only when logs are included.

## Testing

Add coverage for persistence across reopen and process restart, retention limits, cascading deletion, busy-storage fallback, and report uploads with and without logs, including SQLite log failures.

GitOrigin-RevId: ea2da1818d4a5da3da29f211042ca45e834342bc
2026-10-07 11:18:05 +00:00
pakrym-oai 28cd3e2501 Remove obsolete fields from persisted turn context (#51492)
## What changed

Remove `workspace_roots`, `current_date`, `timezone`, `network`, `personality`, and `multi_agent_mode` from `TurnContextItem` and stop recording them in rollouts. Update the precomputed protocol schema accordingly.

Make `summary` optional when reading turn context and write `"none"` as a compatibility placeholder for older readers, rather than persisting the active reasoning-summary setting.

## Testing

Update protocol tests to cover omitted `summary` fields and compatibility-placeholder serialization. Extend compaction tests to verify that model requests retain the updated reasoning-summary setting after compaction while the rollout checkpoint records the placeholder.

GitOrigin-RevId: d49eabaa1b340be1d79f7f09115281a307ea8a64
2026-10-06 22:39:06 +00:00
Eddie Chen cfc946f4e1 Expose and persist turn lineage across the app server (#51415)
## Why

Clients starting delegated work need to attribute descendant turns to the original turn and retain that attribution through completion and history reads.

## What changed

- Accept optional `parentTurnId` and `rootTurnId` in `turn/start`, and expose `rootTurnId` on returned turns and lifecycle notifications.
- Default new turns to their own root when none is supplied. Preserve the active turn's root when steering, ignoring incoming lineage fields.
- Carry root IDs through completion, interruption, recovery, and fork boundaries, and persist them for paginated history reads. Leave unknown roots null in older history.
- Update protocol schemas and generated TypeScript and Python bindings.

## Testing

Extend coverage for default and explicit roots, steering without changing lineage, interrupted and recovered turns, history reads after restart in both history modes, nested fork pagination, and compatibility with older database writers.

GitOrigin-RevId: f1a570a12bd11742929fd2e8b017e5d35aa07dcb
2026-10-06 16:56:39 +00:00
Eric Traut 80e0b51c9e Make session lookup pagination stable and report listing failures (#51230)
## Why

Session activity can move unread threads ahead of a pagination cursor, and timestamp-only cursors can skip threads with equal creation times. Either can prevent a session label lookup from detecting duplicate labels. Database failures must also remain distinguishable from an exhausted listing.

## What changed

- Resolve session labels using database-only listings ordered by `CreatedAt`, with thread IDs breaking timestamp ties. Accept unique matches across pages with valid timestamp-and-ID cursors, while retaining ambiguity errors for older cursors that cannot establish uniqueness.
- Preserve timestamp index ranges for database continuation queries and timestamp-only cursors for listings that may fall back to the filesystem.
- Propagate database-only listing and name-resolution failures. Keep best-effort name enrichment for ordinary listings and retry latest-session lookup with scan-and-repair after a database-only error.
- Keep loaded sessions selectable in the resume and fork pickers when a later page fails, stop further pagination, and display `Could not load more sessions`.

## Testing

Add regression coverage for creation-time continuation after session activity, cursor query plans, filesystem fallback, database and name-query errors, cross-page label uniqueness, older server cursors, and selection after picker pagination failures.

GitOrigin-RevId: 264ab2e69b8ad12d9edaa31fd253cb9805a46a90
2026-10-06 02:14:22 +00:00
Benjamin Carlsson 09bced5ad9 Populate thread previews from delegated task inputs (#50462)
## Why

Delegated turns can begin without a user message, leaving threads without a task preview. Populate the preview from delegated input so these threads are discoverable before a user follow-up.

## What changed

- Extract previews from completed `create_thread` and `send_message_to_thread` outputs in the `codex_app` and `codex_tui` namespaces.
- Unwrap recognized `<codex_delegation>` inputs and decode their escaped task text for display, preserving model content. Keep plain or unrecognized wrappers as text and ignore blank inputs.
- Fill missing previews in state metadata extraction and live thread metadata updates without replacing existing previews or setting the title or `first_user_message`.

## Testing

Add tests for delegation decoding and tool filtering, preview-only metadata updates, preservation of later user-message metadata, and database listing before a user follow-up in both legacy and paginated history modes.

GitOrigin-RevId: f7b3c60ef75e08c76b4084d043e3967bf2bade22
2026-10-02 23:34:11 +00:00
joeytrasatti-openai 26a49b3882 Add paginated reverse lookup for thread attachments (#50083)
## What changed

Add `StateRuntime::list_thread_attachment_threads` to find threads by exact `attachment_type` and `identity_key`, including threads without previews. Return thread IDs and archive state, with filters for all, archived, or non-archived threads.

Order results by thread ID and bind pagination cursors to the attachment identity and archive filter. Add a database index on `(attachment_type, identity_key, thread_id)` to support the lookup.

## Testing

Add coverage for exact identity matching, archive filters, pagination, rejection of cursors reused with different lookup parameters, and membership updates after attachment removal or thread deletion.

GitOrigin-RevId: 21f32eb0cb1845a82d740d67b374080b6e12eb03
2026-10-01 19:55:36 +00:00
Dmitriy Kovalenko 596ae94fb0 Classify SQLite corruption using typed error codes (#49710)
## What changed

Preserve the underlying error in `LocalStateDbStartupError` and use its error chain to decide whether CLI startup failures qualify for automatic database backup and recovery.

Replace corruption checks against error-message text with SQLite result-code classification, including SQLx extended result codes and directly wrapped `SqliteError` values.

## Testing

Update the embedded startup corruption test to verify that the startup error retains a detectable SQLite corruption cause and the failed database path.

GitOrigin-RevId: 6e8ecd0ee67f3c239183ddea3fc75c4e4cd5744a
2026-09-30 19:00:00 +00:00
Dmitriy Kovalenko 3620b2caf8 Detect SQLite corruption during startup and preserve recovery backups (#49701)
## Why

A database can open successfully while containing corruption that startup previously missed. Detecting it before runtime initialization completes allows recoverable databases to be rebuilt while preserving the damaged files.

## What changed

- Run `PRAGMA quick_check(1)` when opening writable databases, with a 100 ms scan budget. Share attempts across cloned `SqliteConfig` instances by file identity so repeated opens skip validation and replacement files are checked again.
- Back up and rebuild runtime databases that support recovery. Report corruption without rebuilding thread history or other databases without an automatic recovery policy. Treat interrupted or locked checks as incomplete rather than confirmed corruption.
- Collect startup backups across recovery retries and include their locations in the app server warning. Explain that saved conversations can restore the thread list and history, while some database-only metadata may be unavailable.
- Record confirmed corruption in `codex.sqlite.corruption.count`.

## Testing

Add tests for recovery policies, backup preservation, validation caching and concurrency, cancelled and interrupted checks, and locked databases. Add app server coverage for recovery warnings and saved conversation restoration, a TUI warning snapshot, and message board post preservation.

GitOrigin-RevId: 43e26328888e0301def642d69bd324acee867566
2026-09-30 18:29:46 +00:00
Dmitriy Kovalenko 8ea2c0e0d4 Prune diagnostic logs periodically by age and database size (#49425)
## Why

Startup-only cleanup leaves expired logs in long-running sessions, and age-based retention alone does not limit database usage.

## What changed

Run cleanup in the background immediately after initialization and every 30 minutes. Start with the existing 10-day retention window, then halve it until occupied SQLite pages fit within a 64 MiB budget or the window reaches one second. Preserve logs within that final second even when they exceed the budget.

Exclude free pages from the budget so SQLite can reuse them, and retain the passive checkpoint after the first sweep. Stop the task when the runtime is dropped or the log pool is closed.

## Testing

Add tests for the 10-day boundary, repeated retention-window halving, free-page accounting, the one-second retention floor, immediate startup cleanup, and periodic cleanup without new writes or a restart. Update existing log tests to use current timestamps so background retention does not remove their fixtures.

GitOrigin-RevId: c6eeb5ec13006d4a03354e08322827429be53ce8
2026-09-30 01:34:12 +00:00
Dmitriy Kovalenko 16a7c0f0eb Filter graceful shutdown guard and trigger traces from SQLite logs (#49414)
## What changed

Set the default SQLite log filter to `DEBUG` for `tokio_graceful::guard` and `tokio_graceful::trigger`, dropping their `TRACE` events while retaining debug and higher-level diagnostics.

## Testing

Extend the SQLite sink filter test to verify that guard and trigger traces are dropped, their debug events are retained, and `tokio_graceful::shutdown` trace events remain available.

GitOrigin-RevId: c98495c834db35f3d8799ab4f0a1f7a82a53c8ef
2026-09-30 00:50:50 +00:00
jif c2837d8ece Batch metadata reads when resolving thread names (#49305)
## Why

Thread name resolution performs a separate SQLite metadata lookup for each thread. Batch these reads to reduce database queries for large result sets.

## What changed

Add `StateRuntime::get_threads` and use it in `resolve_thread_names`. Fetch metadata in batches of at most 999 IDs to stay within SQLite's bind limit, skipping missing or invalid records independently so valid metadata remains available.

## Testing

Add regression coverage for reads spanning multiple batches, invalid and missing records, duplicate IDs, and empty input.

GitOrigin-RevId: a403416f1da5adea56705c70839850760d3eb550
2026-09-29 16:36:07 +00:00
Benjamin Carlsson c2d2f422e6 Preserve SQLite vacuum modes and surface pool initialization errors (#49102)
## Why

Changing an existing database from `FULL` to `INCREMENTAL` auto-vacuum requires a writer lock and can block pool initialization. SQLx also retries connection initialization failures, masking the original error as `PoolTimedOut`.

## What changed

Preserve existing auto-vacuum modes when opening writable pools, while continuing to initialize empty databases with incremental auto-vacuum. Return the first connection initialization error directly to the caller, retaining SQLx's normal error handling for later lazy connections.

## Testing

Add regression tests for fresh database settings, opening an existing `FULL` auto-vacuum database under a write lock, and preserving the WAL conversion lock error while allowing a subsequent open to succeed after the lock is released.

GitOrigin-RevId: b586c68e43a5a0d3b47268378716919d01b749cf
2026-09-28 23:44:39 +00:00
Dmitriy Kovalenko 33a0f766a6 Reclaim unused SQLite log database pages in the background (#49069)
## Why

Deleting logs leaves free SQLite pages available for reuse without shrinking the database file. Reclaim excess free space while preserving a reserve for future writes.

## What changed

- Start a background incremental-vacuum worker with `StateRuntime`, enabling reclamation only for the logs database. Use a per-home file lock to select one worker and wait for it to finish during shutdown.
- Reclaim pages when at least 64 MiB and 25% of the database are free, preserving a reserve of at least 16 MiB or 10% of the database.
- Use short transactions with page and cooperative time budgets, defer work on contention, and reduce batch sizes with exponential backoff when interrupted passes make no progress.
- Record reclamation counts, durations, and reclaimed pages in telemetry.

## Testing

Add a regression test that interrupts a vacuum commit, verifies foreground writes can resume without data loss, then confirms a resumed pass shrinks the database and passes `PRAGMA integrity_check`.

GitOrigin-RevId: 8421f87b0aeef6540eda5690c84216906673c72e
2026-09-28 20:19:10 +00:00
jif 46fdd5ef39 Avoid SQLite stalls from connection setup and stderr span logging (#49032)
## Why

Setting `auto_vacuum` on every connection can wait for a writer lock even when the mode is unchanged. Logging every span entry and exit can also block SQLx workers on undrained stderr while they hold a write transaction.

## What changed

- Set incremental vacuum only for empty databases or databases using full auto-vacuum, before enabling WAL. Preserve existing nonempty databases with auto-vacuum disabled.
- Emit span creation and closure events in app-server text and JSON stderr logs, preserving busy/idle timings and explicit events while omitting span entry and exit records.

## Testing

Add regression tests for opening and expanding writable pools while another connection holds a writer lock, and for transaction completion with undrained stderr in both log formats. Verify vacuum modes and retained log diagnostics.

GitOrigin-RevId: 40d4ef32c7efc49f4240b987277ba45551259335
2026-09-28 17:00:25 +00:00
acrognale-oai 7596ab19a6 Keep archived threads with empty previews visible in listings (#48199)
## Why

Archived threads without previews were filtered out of thread listings, even though they had valid session metadata.

## What changed

- Include archived threads with empty previews in both SQLite and rollout-file listings.
- Add archive-specific indexes for creation, update, and recency ordering so these listings can use indexed pagination.

## Testing

Add regression coverage for paginated archive listings with and without SQLite, archive query plans that avoid temporary sorting, and metadata reads that preserve source and file timestamps for empty archived threads.

GitOrigin-RevId: e29d0257b2e029bea5fdf94eed6dab1006d2a8f2
2026-09-25 20:11:47 +00:00
Dmitriy Kovalenko 3d6428510f Warn users when SQLite diagnostic log writes fail (#47887)
## Why

Failed SQLite writes leave diagnostic logs incomplete. Notify users before they close Codex so they can submit logs or run `codex doctor`.

## What changed

- Add a best-effort warning on the first log write failure in standalone and in-process app servers, while continuing to retain failure diagnostics in memory.
- Handle failures that occurred before the in-process warning reporter was attached.
- Tailor the warning's guidance to whether submitting logs is enabled.

## Testing

Add coverage for warning deduplication, both guidance variants, retained diagnostics after the notification sender is dropped, and SQLite flush failures before and after reporter attachment. Extend log upload integration coverage and add a TUI warnings snapshot.

GitOrigin-RevId: 12a74126d13c3568430e539a180cca47fa931fa9
2026-09-24 17:04:50 +00:00
Dmitriy Kovalenko a285066737 Preserve diagnostic logs when SQLite logging fails (#47886)
## Why

SQLite write failures can leave persisted logs incomplete, while log snapshots taken before collection miss errors raised during that collection.

## What changed

- Report SQLite flush failures directly to the in-memory log buffer without re-entering tracing, using an error category and batch size instead of raw error details.
- Track whether the log writer has lost a batch and make app-server diagnostic uploads use the memory buffer when SQLite logs are incomplete.
- Refresh log bytes after flushing and querying SQLite so the same submission includes collection failures, while preserving captured metadata and thread identity.

## Testing

Add regression tests for failure reporting without recursion, subsequent write recovery, metadata preservation, and healthy and failing SQLite collection. Add an upload integration test that captures attachments locally and checks failure diagnostics, memory-buffer fallback, and omission of raw SQLite error details.

GitOrigin-RevId: 51b0c4b69c977d9f4e74f2f245a9fd90fdb13761
2026-09-24 17:04:24 +00:00
Charlie Marsh 44b857c00e Clean up legacy Guardian thread metadata in SQLite (#47303)
## Why

Guardian prompts are synthetic review context, but legacy thread metadata can duplicate them in titles, previews, and first messages. Keep SQLite metadata small while retaining rollout JSONL as the canonical record.

## What changed

Add a migration for Guardian threads that:

- Replaces blank or prompt-derived titles with `Guardian review`.
- Preserves nonblank names and distinct nonblank titles, using the title as the name when no name exists.
- Sets `preview` to `Approval review` and clears `first_user_message`.

## Testing

Add a migration test covering prompt-derived and blank titles, custom titles and names, and unchanged metadata for a non-Guardian worker thread.

GitOrigin-RevId: 65b5ff5200a39a19fb11f494de6dca7710ff6cc6
2026-09-22 15:05:27 +00:00
Charlie Marsh 4f6c106cdd Use compact display metadata for Guardian review threads (#47302)
## Why

Guardian review threads use synthetic approval prompts as user messages. Deriving thread titles and previews from those prompts exposes large, unhelpful summaries in thread listings.

## What changed

- Default Guardian threads to the title "Guardian review" and preview "Approval review" across metadata creation, rollout reads, and resume.
- Skip synthetic user messages and goal updates when deriving Guardian display metadata, leaving `first_user_message` unset for new threads.
- Preserve custom names, including legacy indexed names, when listing, reading, resuming, and migrating threads to paginated history.

## Testing

Add regression coverage for compact metadata, resume without session metadata, custom names before and after migration, and app-server persistence, listing, and rename/resume behavior.

GitOrigin-RevId: 42aa707db20c3e57ffa5fd4c91ee313710be10d7
2026-09-22 15:02:54 +00:00
chiam-oai 772abc9425 Preserve and expose thread item lifecycle timestamps (#47114)
## What changed

- Persist producer-recorded item start and completion times in thread history, preserving existing non-null values on updates.
- Return nullable `startedAtMs` and `completedAtMs` Unix millisecond timestamps from `thread/items/list`, and update the generated schemas and TypeScript and Python types.
- Leave timestamps unknown for existing rows and older writers.

## Testing

Add coverage for exact timestamp persistence, protocol round trips with present and missing timestamps, paginated item responses, and compatibility with older database writers.

GitOrigin-RevId: 894853481085782ff54db1704799e41446d15e51
2026-09-21 22:23:26 +00:00
Jeremy Rose 23ae65bb06 Persist thread creator identity in rollouts and SQLite (#47113)
## What changed

- Capture optional `creator_user_id` and `creator_account_id` from authentication when creating a thread, including forks.
- Carry these fields through thread storage, rollout metadata, and a SQLite migration. Leave them absent when unavailable or when reading older threads.
- Preserve recorded creator values across resume and metadata updates. On revert, recover missing rollout values from SQLite before writing replacement metadata.

## Testing

Add coverage for ChatGPT and API-key sessions, resume under another account, forks using current authentication, metadata extraction and upserts, and repeated reverts after creator fields are removed from a rollout.

GitOrigin-RevId: 9839fa42f9392f12226002e2c27d05d5656f5695
2026-09-21 21:22:01 +00:00
Ahmed Ibrahim 3627bb91b2 Avoid building a host SQLite driver for SQLx macros (#47084)
## Why

Row derives and migration embedding do not need query macros or a host SQLite driver.

## What changed

- Remove the `macros` feature from `sqlx` and depend directly on `sqlx-macros` with only `derive` and `migrate` enabled.
- Update state and thread-store derives and migration embedding to use `sqlx_macros`.
- Enable `sqlx-sqlite`'s `offline` feature directly to preserve query-description support in the target driver.
- Pin `sqlx`, `sqlx-macros`, and `sqlx-sqlite` to `=0.9.0` to keep their versions aligned.

GitOrigin-RevId: 8a8ef691ff2862a7dad978a578c5206b52dfc0b6
2026-09-21 18:30:31 +00:00
joeytrasatti-openai 19286b8819 Copy current thread attachments into non-ephemeral forks (#45579)
## What changed

Copy the source thread's current attachments when creating a non-ephemeral fork, including forks at an earlier turn. Copies receive new attachment IDs and creation timestamps while preserving resource identities and payloads. Attachment membership can then change independently on either thread; referenced resources are not copied.

Await the atomic copy before publishing the fork. If copying fails, log the error and allow the conversation fork to succeed without attachments. Resuming a fork does not copy attachments again.

Document that clients should use `forkedFromId` on `thread/started` and fetch the fork's attachments with `thread/attachment/list`; copying does not emit per-attachment updates.

## Testing

Add coverage for atomic rollback, independent attachment membership, inheritance across history cutoffs, successful conversation forks after copy failures, and resuming without restoring removed attachments.

GitOrigin-RevId: 939d2c6a3073ccfcd922380c6beda45d3deac606
2026-09-15 02:33:27 +00:00
Matthew Zeng b5544d5732 Persist disabled plugin IDs in thread settings (#44332)
## What changed

- Add `disabled_plugin_ids` to thread startup options, settings overrides, snapshots, and persisted turn contexts. Omitted overrides preserve the current list; an empty list clears it.
- Restore the selection on resume and capture it before truncating history for a fork. Prefer the owning thread's latest settings snapshot, falling back only to the latest turn context so stale history cannot resurrect an older selection.
- Freeze the selection at turn admission and retain compatibility with histories that lack the new fields.

## Testing

Add coverage for owned settings, explicit clears, latest-turn fallback, and legacy snapshots. Extend settings tests to cover plugin selection during runtime restoration and compaction, and verify initial turn persistence without extra settings checkpoints in both history modes.

GitOrigin-RevId: 75ffbff5763cd93cc5ac69d73b08b237f59bc009
2026-09-09 23:23:59 +00:00
joeytrasatti-openai 130d6e4fba Add paginated thread attachment listing to the state runtime (#44330)
## What changed

Add `StateRuntime::list_thread_attachments` with page sizes from 1 to 100 and stable keyset ordering by `created_at` and `id`. Return a continuation cursor when more attachments remain, and reject malformed cursors or cursors belonging to another thread.

## Testing

Add coverage for thread-scoped pagination, cursor validation, and attachment persistence across reopening the database and archiving/unarchiving a thread. Verify listings reflect attachment removal and cascading deletion when a thread is deleted.

GitOrigin-RevId: e07433e1a675669082cc58f86b0dc4535d365ba3
2026-09-09 23:04:22 +00:00
joeytrasatti-openai 589874be81 Add transactional thread attachment mutations to the state runtime (#43949)
## What changed

- Replace the `ThreadArtifact` model and related exports with attachment terminology.
- Add `StateRuntime::add_thread_attachment` and `remove_thread_attachment` using SQLite transactions. Repeated additions for the same thread, attachment type, and identity key return the existing record without changing its payload or creation time.
- Enforce limits of 100 attachments per thread, 64 KiB per serialized payload, and 256 bytes each for nonblank attachment types and identity keys. Removal returns the deleted record or `NotFound` and frees capacity immediately. Both mutations reject unknown threads.

## Testing

Add tests for idempotency, thread isolation, removal outcomes, capacity reuse, invalid inputs, unknown threads, and concurrent additions creating exactly one record.

GitOrigin-RevId: a68fe5076832f071524f1b4fc87d08b96ac270b7
2026-09-09 00:18:35 +00:00