mirror of https://github.com/openai/codex.git synced 2026-09-06 15:29:32 +00:00

Files

jif 1cc81ca89a Support compression for shared rollout lineages (#41357 )

## Why

Rollout compression skips files that participate in paginated fork lineages because lineage readers rely on byte offsets into the original JSONL. This prevents shared rollout history from being compressed.

## What changed

- Add a seekable rollout reader that preserves logical JSONL offsets for plain and zstd-compressed files without materializing compressed data beside the rollout.
- Use the reader for lineage validation, model-context replay, and SQLite history projection, including when compression races with a read.
- Add the opt-in `local_thread_store_shared_compression` feature. The default mode continues to leave shared lineages uncompressed for compatibility, while the new mode includes referenced sources and fork pointers.
- Record the uncompressed size in new zstd frames so common prefix-bound checks can avoid decoding the full file.

## Testing

Add coverage for seekable compressed reads, prefix validation across zstd frame shapes, compressed lineage replay and projection, concurrent forks, and compression-mode behavior.

GitOrigin-RevId: 83e519f7eabf7448977eff37c55899858cc17a07

2026-08-28 16:04:26 +00:00

src

Support compression for shared rollout lineages (#41357 )

2026-08-28 16:04:26 +00:00

BUILD.bazel

ThreadStore interface (#17659 )

2026-04-14 13:51:00 -07:00

Cargo.toml

Expose image generation usage-limit failures (#38024 )

2026-08-11 15:40:52 +00:00

README.md

Unify thread metadata updates above store (#22236 )

2026-05-13 00:28:15 +00:00

README.md

Thread Store

codex-thread-store is the storage boundary for Codex threads. It defines the ThreadStore trait plus local and in-memory implementations. Other storage implementations may live outside this repository.

Responsibilities

ThreadStore::append_items is the raw canonical history append API. It does not infer metadata from item contents.
ThreadStore::update_thread_metadata is the only thread metadata write API. It accepts a single literal metadata patch shape, regardless of whether the caller is applying a user/API mutation or facts derived above the store from appended history.
LiveThread is the preferred API for active session persistence. It owns a per-thread metadata sync helper, applies the rollout persistence policy, appends canonical history, and then sends metadata patches through ThreadStore::update_thread_metadata.
ThreadManager routes metadata mutations for loaded and cold threads through one entrypoint. Loaded threads use their LiveThread; cold threads go directly to the store.
LocalThreadStore persists history through codex-rollout JSONL files and persists queryable metadata through the SQLite state database when available. Local explicit metadata mutations also maintain JSONL/name-index compatibility so reading old or SQLite-less local storage keeps working.
RolloutRecorder is the local JSONL writer. It writes already-canonical items for ThreadStore::append_items; it no longer decides metadata updates for live thread-store appends.
core/session creates or resumes LiveThread handles and does not need to know whether persistence is backed by local files or another store.

Direction

New metadata observation semantics should live above ThreadStore. Stores persist explicit metadata fields, but raw history appends remain history-only.