Commit Graph

2 Commits

Author SHA1 Message Date
jif
2cbbf0c9b5 Add memory dual writing and v2 readiness reporting (#43827)
## Why

Allow v2 memories to build in the background while the selected memory version continues supplying context, and let clients check whether v2 has enough consolidated data and a valid summary to use.

## What changed

- Add `memories.dual_write`, disabled by default, to run v1 and v2 extraction and consolidation concurrently with separate stores and directories.
- Add the experimental `memory/status` endpoint, returning `v2ConsolidatedThreads` and `v2Ready`. Readiness requires a valid v2 summary and a consolidation thread count meeting `minConsolidatedThreads` (default 20, supported range 1–4096).
- Persist the largest thread count from a successful consolidation across pruning and clear it on memory reset.
- Share v2 summary validation between consolidation and status reporting, and tag memory metrics with `memory_version`.

## Testing

Add tests for independent dual-write outputs, preserving existing notes without importing them into v2, and readiness behavior across consolidation ownership checks, missing summaries, configurable thresholds, thread deletion, and memory reset.

GitOrigin-RevId: 54417cc60bf419062401355c0da8c79119995f61
2026-09-08 14:23:23 +00:00
jif-oai
aad59a0916 Move memory state to a dedicated SQLite DB (#24591)
## Summary

Generated memory rows and their stage-one/stage-two job state currently
live in `state_5.sqlite` alongside thread metadata. That makes memory
cleanup and regeneration share the main state schema even though those
rows are memory-pipeline data and can be rebuilt independently from the
durable thread records.

This PR moves the memory-owned tables into a dedicated
`memories_1.sqlite` runtime database while keeping thread metadata in
`state_5.sqlite`.

## Changes

- Adds a separate memories DB runtime, migrator, path helpers, telemetry
kind, and Bazel compile data for `state/memory_migrations`.
- Introduces `MemoryStore` behind `StateRuntime::memories()` and moves
memory table/job operations onto that store.
- Drops the old memory tables from the state DB and recreates their
schema in `state/memory_migrations/0001_memories.sql`.
- Updates memory startup, citation usage tracking, rollout pollution
handling, `debug clear-memories`, and app-server `memory/reset` to
operate through the memories DB.
- Preserves cross-DB behavior by hydrating thread metadata from the
state DB when selecting visible memory outputs and checking stage-one
staleness.

## Verification

- Added/updated `codex-state` tests for deleted-thread memory visibility
and already-polluted phase-two enqueue behavior.
- Updated `debug clear-memories`, app-server `memory/reset`, and
memories startup tests to seed and assert memory rows through
`memories_1.sqlite`.
2026-05-26 20:07:25 +02:00