mirror of https://github.com/openai/codex.git synced 2026-11-07 20:30:33 +00:00

Files

T

JonLuca De Caro df769889c0 Reduce SQLite write overhead for thread activity and history (#60006 )

## Why

Token-usage observations previously required full metadata writes. Concurrent history writers could also occupy pool connections while waiting for SQLite's single writer.

## What changed

- Update only timestamps and token totals for eligible activity patches, preserving concurrent metadata edits. Retain metadata repair for missing rows, changed sources, missing spawn edges, history promotion, and preview backfill.
- Serialize local history writes before acquiring transaction connections so readers can use the remaining pool connections.
- Use `ROWS UNBOUNDED PRECEDING` for cumulative log-byte limits so SQLite can compute byte and row limits in one pass.

## Testing

Add coverage for token updates preserving concurrent edits, clamping negative totals, and rejecting stale source or missing spawn metadata. Extend metadata repair tests across legacy and paginated history modes to cover token updates, preview backfill, and spawn-edge repair.

GitOrigin-RevId: 4a30fd42a7498c616ed13adb46fc3d7e0226a94d

2026-11-06 22:45:37 +00:00

src

Reduce SQLite write overhead for thread activity and history (#60006 )

2026-11-06 22:45:37 +00:00

BUILD.bazel

ThreadStore interface (#17659 )

2026-04-14 13:51:00 -07:00

Cargo.toml

Route response item IDs through a shared generator (#59312 )

2026-11-04 04:02:00 +00:00

README.md

Unify thread metadata updates above store (#22236 )

2026-05-13 00:28:15 +00:00

README.md

Thread Store

codex-thread-store is the storage boundary for Codex threads. It defines the ThreadStore trait plus local and in-memory implementations. Other storage implementations may live outside this repository.

Responsibilities

ThreadStore::append_items is the raw canonical history append API. It does not infer metadata from item contents.
ThreadStore::update_thread_metadata is the only thread metadata write API. It accepts a single literal metadata patch shape, regardless of whether the caller is applying a user/API mutation or facts derived above the store from appended history.
LiveThread is the preferred API for active session persistence. It owns a per-thread metadata sync helper, applies the rollout persistence policy, appends canonical history, and then sends metadata patches through ThreadStore::update_thread_metadata.
ThreadManager routes metadata mutations for loaded and cold threads through one entrypoint. Loaded threads use their LiveThread; cold threads go directly to the store.
LocalThreadStore persists history through codex-rollout JSONL files and persists queryable metadata through the SQLite state database when available. Local explicit metadata mutations also maintain JSONL/name-index compatibility so reading old or SQLite-less local storage keeps working.
RolloutRecorder is the local JSONL writer. It writes already-canonical items for ThreadStore::append_items; it no longer decides metadata updates for live thread-store appends.
core/session creates or resumes LiveThread handles and does not need to know whether persistence is backed by local files or another store.

Direction

New metadata observation semantics should live above ThreadStore. Stores persist explicit metadata fields, but raw history appends remain history-only.