jif 47db419304 Batch thread history materialization into SQLite (#55244) ...

## Why

History catch-up collected the entire durable rollout prefix before applying it in one SQLite transaction. Large rollouts therefore required buffering all projection steps and committing them together.

## What changed

Stream projection batches through a bounded queue and commit each batch with its checkpoint. Flush between complete records after reaching 1 MiB of source data or 256 projection steps, keeping ordinal gaps with their following records and allowing oversized records to remain intact.

Wait for batches before opening SQLite transactions so an idle reader does not hold the writer. Preserve committed progress across later failures or cancellation, allowing retries to resume from the last checkpoint.

## Testing

Add coverage for reader and writer failures under backpressure, atomic batch rollback, retries without replaying committed records, and unrelated writes while projection is paused. Verify that producer disconnection and consumer cancellation retain committed progress.

GitOrigin-RevId: 06e79fa4cf273de9abd95faf7eed370f18cedc33

2026-10-19 10:48:13 +00:00

..

rollout_migration

Record local image inputs on completed image-generation items (#54420)

2026-10-15 21:30:56 +00:00

tests

Populate thread previews from delegated task inputs (#50462)

2026-10-02 23:34:11 +00:00

thread_history

Use the turn index for turn-scoped item pagination (#54351)

2026-10-15 16:58:14 +00:00

archive_thread.rs

Coordinate rollout compression with active thread writers (#44138)

2026-09-09 12:30:07 +00:00

compression_writer_tests.rs

Reuse unchanged history snapshots when resuming threads (#49600)

2026-09-30 11:59:13 +00:00

create_thread.rs

Persist thread creator identity in rollouts and SQLite (#47113)

2026-09-21 21:22:01 +00:00

daybreak_metadata_tests.rs

Persist Daybreak preferences in thread metadata (#42854)

2026-09-04 21:17:25 +00:00

delete_thread.rs

Reuse unchanged history snapshots when resuming threads (#49600)

2026-09-30 11:59:13 +00:00

helpers.rs

Limit unarchive scans to the requested thread's archived rollouts (#54060)

2026-10-14 18:19:02 +00:00

history_revision.rs

Reuse unchanged history snapshots when resuming threads (#49600)

2026-09-30 11:59:13 +00:00

list_threads.rs

Make session lookup pagination stable and report listing failures (#51230)

2026-10-06 02:14:22 +00:00

live_writer.rs

Cap persisted command output in paginated history at 64 KiB (#50427)

2026-10-02 21:00:06 +00:00

mod.rs

Support store-owned preparation when resuming threads (#54059)

2026-10-14 18:02:25 +00:00

model_context_tests.rs

Add opt-in output token replay for OpenAI requests (#52742)

2026-10-10 01:25:00 +00:00

model_context.rs

Stop rollout reads when the calling task is cancelled (#53947)

2026-10-14 10:46:14 +00:00

move_thread_to_section.rs

Add persisted manual ordering for thread sections (#36007)

2026-07-29 18:37:22 +00:00

paginated_fork.rs

Resolve paginated thread history by rollout ID (#38244)

2026-08-12 20:31:52 +00:00

pending_thread_metadata_tests.rs

Persist empty paginated threads when naming them (#49785)

2026-10-01 00:57:29 +00:00

pending_thread_metadata.rs

Support metadata staging for reserved thread IDs (#38819)

2026-08-16 00:57:28 +00:00

projects.rs

Add recency sorting to project/list (#41223)

2026-08-27 23:43:46 +00:00

read_thread_tests.rs

Keep archived threads with empty previews visible in listings (#48199)

2026-09-25 20:11:47 +00:00

read_thread.rs

Fix session lookup and resume after rollout compression (#54008)

2026-10-14 15:22:02 +00:00

revert_thread_tests.rs

Expose and persist turn lineage across the app server (#51415)

2026-10-06 16:56:39 +00:00

revert_thread.rs

Reuse unchanged history snapshots when resuming threads (#49600)

2026-09-30 11:59:13 +00:00

rollout_lineage_tests.rs

Reuse unchanged history snapshots when resuming threads (#49600)

2026-09-30 11:59:13 +00:00

rollout_lineage.rs

Stop rollout reads when the calling task is cancelled (#53947)

2026-10-14 10:46:14 +00:00

rollout_migration_tests.rs

Record local image inputs on completed image-generation items (#54420)

2026-10-15 21:30:56 +00:00

rollout_migration.rs

Retain rollout migration locks across worker cancellation (#53794)

2026-10-13 23:06:58 +00:00

search_threads_tests.rs

Preserve runtime workspace roots across thread resume (#43848)

2026-09-08 16:24:38 +00:00

search_threads.rs

Make session lookup pagination stable and report listing failures (#51230)

2026-10-06 02:14:22 +00:00

test_support.rs

Centralize SQLite connection configuration (#34808)

2026-07-22 18:45:47 +00:00

thread_attachments_tests.rs

Add thread attachment operations with coordinated deletion (#44350)

2026-09-10 00:30:27 +00:00

thread_attachments.rs

Add attachment owner lookup to the app-server (#50094)

2026-10-01 20:47:05 +00:00

thread_history_materialization_pipeline.rs

Batch thread history materialization into SQLite (#55244)

2026-10-19 10:48:13 +00:00

thread_history_materialization_tests.rs

Batch thread history materialization into SQLite (#55244)

2026-10-19 10:48:13 +00:00

thread_history_materialization.rs

Batch thread history materialization into SQLite (#55244)

2026-10-19 10:48:13 +00:00

thread_history.rs

Retry SQLite operations on transient writer contention (#54369)

2026-10-15 18:06:57 +00:00

thread_rollout_resolver.rs

Resolve paginated thread history by rollout ID (#38244)

2026-08-12 20:31:52 +00:00

thread_sections_tests.rs

Add appearance metadata to thread sections (#37898)

2026-08-10 23:32:42 +00:00

thread_sections.rs

Add appearance metadata to thread sections (#37898)

2026-08-10 23:32:42 +00:00

timestamp_metadata_tests.rs

Avoid full metadata rewrites for thread timestamp updates (#48983)

2026-09-28 13:43:55 +00:00

unarchive_thread.rs

Limit unarchive scans to the requested thread's archived rollouts (#54060)

2026-10-14 18:19:02 +00:00

update_thread_metadata.rs

Persist empty paginated threads when naming them (#49785)

2026-10-01 00:57:29 +00:00