Commit Graph
8 Commits
Author SHA1 Message Date
dhruvgupta-oai 333d7bde0f Bound memory baseline files and recover incomplete snapshots (#57240)
## Why

Memory baseline snapshots need bounded file reads, and recovery must keep existing memory files visible to consolidation rather than treating them as already processed.

## What changed

- Limit baseline files to 10 MiB, skip oversized working files, and check stored blob sizes before reading them.
- Version baselines and replace missing, outdated, or unusable metadata with an empty baseline, preserving working files as additions. Also invalidate the baseline when a tracked file grows beyond the limit.
- Serialize baseline initialization, resets, and diffs with a cross-process lock. Isolate repository access and pin index writes to the baseline worktree.

## Testing

Add coverage for oversized files, legacy baseline migration, interrupted resets, cross-process locking, and consolidation with unchanged inputs after baseline recovery.

GitOrigin-RevId: 34ecc9e5700a88ed37eff83a90f08d8a9b0d9f29
2026-10-26 20:40:53 +00:00
jif 6500c1f844 Record memory storage size after successful consolidation (#45956)
## What changed

Emit a `codex.memory.storage_bytes` histogram after successful phase-two consolidation, including runs with no workspace changes. Sum regular-file sizes recursively from filesystem metadata, excluding `.git` entries and symbolic links. Stop the phase-two duration timer before measuring storage.

## Testing

Add coverage for both memory versions, nested files, UTF-8 byte counts, Git metadata exclusion, and Unix symbolic links, including broken links and directory loops. Extend the workspace baseline reset test to verify the remaining storage size.

GitOrigin-RevId: 9170c34d656728897790decd2e2cabb38a81836f
2026-09-16 14:49:24 +00:00
jif 2cbbf0c9b5 Add memory dual writing and v2 readiness reporting (#43827)
## Why

Allow v2 memories to build in the background while the selected memory version continues supplying context, and let clients check whether v2 has enough consolidated data and a valid summary to use.

## What changed

- Add `memories.dual_write`, disabled by default, to run v1 and v2 extraction and consolidation concurrently with separate stores and directories.
- Add the experimental `memory/status` endpoint, returning `v2ConsolidatedThreads` and `v2Ready`. Readiness requires a valid v2 summary and a consolidation thread count meeting `minConsolidatedThreads` (default 20, supported range 1–4096).
- Persist the largest thread count from a successful consolidation across pruning and clear it on memory reset.
- Share v2 summary validation between consolidation and status reporting, and tag memory metrics with `memory_version`.

## Testing

Add tests for independent dual-write outputs, preserving existing notes without importing them into v2, and readiness behavior across consolidation ownership checks, missing summaries, configurable thresholds, thread deletion, and memory reset.

GitOrigin-RevId: 54417cc60bf419062401355c0da8c79119995f61
2026-09-08 14:23:23 +00:00
jif 553df1c691 Add dedicated memory v2 consolidation and read prompts (#43813)
## What changed

- Consolidate v2 rollout summaries into `memory_summary.md` without generating `raw_memories.md` or requiring `MEMORY.md`. Validate the summary's required sections and size below 10,000 UTF-8 bytes.
- Add v2 read instructions for selective history retrieval, evidence-grounded preferences, citations, and explicit memory edits. Split injected instructions into fragments to preserve the complete summary within fragment byte limits.
- Record memory citation usage in the store selected by `memories.version`.

## Testing

Add coverage for v2 consolidation without a handbook, summary validation, version-isolated memory reads, and resetting both memory versions while preserving threads.

GitOrigin-RevId: 1d895fb4a23a973f1a45ba03be07a1f480c10227
2026-09-08 12:58:11 +00:00
jif a04940cb12 Reject symbolic links in memory workspaces (#39205)
## What changed

- Reject a symbolic link used as the memory root.
- Recursively remove symbolic links before preparing or seeding the memory workspace.
- Remove and reject symbolic links created during consolidation, including when the worker fails, so they cannot affect files outside the workspace.

## Testing

Add Unix startup tests covering pre-existing extension links and links created by both successful and failed consolidation workers.

GitOrigin-RevId: f96e03812773f7beaa16455dd77370ee47d8a71f
2026-08-18 14:14:57 +00:00
jif 52c9605dc7 Validate memory consolidation artifacts before succeeding (#32193)
## Why

A completed Phase 2 agent run does not guarantee that its required outputs were
created correctly. Treating a clean workspace as success can also preserve a
state where those outputs are missing.

## What changed

- Require `MEMORY.md` to be a file and `memory_summary.md` to start with `v1`
  before marking consolidation successful.
- Fail completed runs with invalid artifacts without resetting the workspace
  baseline, allowing the job to be retried.
- Run consolidation for a clean workspace when its required artifacts are
  invalid instead of taking the no-change success path.

## Testing

Added coverage for rejecting an invalid summary and retrying a clean workspace
whose consolidation artifacts are missing.

GitOrigin-RevId: ac57b2ba9d062c5203ca51afd716795a91814bb3
2026-07-10 15:40:35 +00:00
jif-oai 5b7d6f5c4f feat: house-keeping memories 3 (#20005)
Move stuff in memories, no behavioural change expected
2026-04-28 18:13:35 +02:00
jif-oai bb83eec825 chore: split memories part 1 (#19818)
Extract memories into 2 different crates
2026-04-27 16:01:05 +02:00