Add rollout migration tooling and background migration (#37348)

## What changed

- Add `codex migrate-rollouts` with dry-run inspection by default, explicit `--apply`, thread filtering, optional I/O throttling, progress output, and JSON or verbose reports.
- Add the disabled-by-default `background_paginated_rollout_migration` feature to migrate legacy local sessions at startup before rollout compression begins.
- Persist a migration cursor and skipped-file fingerprints so later startups avoid full rescans while retrying changed files and recovering pending migrations.
- Coordinate migration with live writers and emit metrics for manual and startup runs.

## Testing

- Cover startup cursor advancement and lookback, pending migration recovery, live-writer coordination, changed empty rollouts, and progress reporting.

GitOrigin-RevId: 276ac506c50ebec5140fd319faca1bb998172061
This commit is contained in:
Owen Lin
2026-08-07 00:46:38 +00:00
committed by copyberry
parent 69b6152c17
commit 4bb7ee3472
21 changed files with 1569 additions and 24 deletions

View File

@@ -147,6 +147,8 @@ pub enum Feature {
ExternalAgentMemoryImport,
/// Compress cold local thread-store rollout files.
LocalThreadStoreCompression,
/// Migrate legacy local rollout files to paginated history in the background.
BackgroundPaginatedRolloutMigration,
/// Enable the Chronicle sidecar for passive screen-context memories.
Chronicle,
/// Compress request bodies (zstd) when sending streaming requests to codex-backend.
@@ -1009,6 +1011,12 @@ pub const FEATURES: &[FeatureSpec] = &[
stage: Stage::UnderDevelopment,
default_enabled: false,
},
FeatureSpec {
id: Feature::BackgroundPaginatedRolloutMigration,
key: "background_paginated_rollout_migration",
stage: Stage::UnderDevelopment,
default_enabled: false,
},
FeatureSpec {
id: Feature::Chronicle,
key: "chronicle",