Commit Graph
14 Commits
Author SHA1 Message Date
Dmitriy Kovalenko 3620b2caf8 Detect SQLite corruption during startup and preserve recovery backups (#49701)
## Why

A database can open successfully while containing corruption that startup previously missed. Detecting it before runtime initialization completes allows recoverable databases to be rebuilt while preserving the damaged files.

## What changed

- Run `PRAGMA quick_check(1)` when opening writable databases, with a 100 ms scan budget. Share attempts across cloned `SqliteConfig` instances by file identity so repeated opens skip validation and replacement files are checked again.
- Back up and rebuild runtime databases that support recovery. Report corruption without rebuilding thread history or other databases without an automatic recovery policy. Treat interrupted or locked checks as incomplete rather than confirmed corruption.
- Collect startup backups across recovery retries and include their locations in the app server warning. Explain that saved conversations can restore the thread list and history, while some database-only metadata may be unavailable.
- Record confirmed corruption in `codex.sqlite.corruption.count`.

## Testing

Add tests for recovery policies, backup preservation, validation caching and concurrency, cancelled and interrupted checks, and locked databases. Add app server coverage for recovery warnings and saved conversation restoration, a TUI warning snapshot, and message board post preservation.

GitOrigin-RevId: 43e26328888e0301def642d69bd324acee867566
2026-09-30 18:29:46 +00:00
Ahmed Ibrahim 3627bb91b2 Avoid building a host SQLite driver for SQLx macros (#47084)
## Why

Row derives and migration embedding do not need query macros or a host SQLite driver.

## What changed

- Remove the `macros` feature from `sqlx` and depend directly on `sqlx-macros` with only `derive` and `migrate` enabled.
- Update state and thread-store derives and migration embedding to use `sqlx_macros`.
- Enable `sqlx-sqlite`'s `offline` feature directly to preserve query-description support in the target driver.
- Pin `sqlx`, `sqlx-macros`, and `sqlx-sqlite` to `=0.9.0` to keep their versions aligned.

GitOrigin-RevId: 8a8ef691ff2862a7dad978a578c5206b52dfc0b6
2026-09-21 18:30:31 +00:00
Adam Perry @ OpenAI 63002bdb26 Extract persisted history types into a dedicated crate (#37871)
## What changed

- Add `codex-history` for model-history and persisted-rollout domain types, including `RolloutItem`, `RolloutLine`, `CompactedItem`, and initial/resumed history state.
- Re-export the persisted types from `codex-rollout` and update consumers to use the new crate boundary instead of `codex-protocol`.
- Preserve existing rollout serialization, including legacy numeric compacted-window IDs.

## Testing

- Add `codex-history` tests for rollout JSON round trips, compacted-history compatibility, persisted history modes, and multi-agent version selection.

GitOrigin-RevId: 944daa9297ddd231d3aebbdcb05fff4adf8b4e1b
2026-08-10 19:26:52 +00:00
Adam Perry @ OpenAI 3418498f01 Honor the configured SQLite home in the logs client (#35695)
## Why

`just log` derived the logs database path from `CODEX_HOME`, so it could read
the wrong database when `sqlite_home` or `CODEX_SQLITE_HOME` selected a
different location.

## What changed

- Move `logs_client` into `codex-cli` so it can resolve the shared
  `SqliteConfig` through the standard configuration loader.
- Keep `--db` as a direct override that skips config loading and preserves
  native path bytes.
- Update the `just log` recipes to run the client from its new crate.

## Testing

- Add coverage for bypassing invalid Codex config with `--db`.
- Add Unix coverage for non-UTF-8 database paths.

GitOrigin-RevId: fabd64a66543be26a6f5d3b5e509016c3270350e
2026-07-28 01:17:52 +00:00
Adam Perry @ OpenAI 6bd3f5e3db Centralize SQLite connection configuration (#33938)
## What changed

- Add `SqliteConfig` as the shared entry point for read-write and read-only SQLite pools.
- Apply consistent WAL, synchronization, auto-vacuum, busy-timeout, logging, and pool-size settings to writable Codex databases.
- Route state runtime, audit, CLI, and test database connections through the shared configuration.

## Testing

- Run migration tests against temporary on-disk databases opened through `SqliteConfig`, including the concurrent-writer repair case.

GitOrigin-RevId: e3946b98bde04c47574532ac8b1a7bb2b03edd97
2026-07-18 02:06:13 +00:00
Gabriel Peal 73c58011b3 [codex] Pin bundled SQLite to fixed WAL-reset version (#27992)
## Summary

Prevent dependency refreshes from silently downgrading Codex's bundled
SQLite to a release affected by the WAL-reset corruption bug.

SQLx 0.9 accepts a broad `libsqlite3-sys` range. An unrelated lock
refresh therefore moved Codex from `libsqlite3-sys 0.37.0` back to
`0.35.0`, changing the bundled SQLite runtime from 3.51.3 to 3.50.2.
SQLite documents the affected versions and fix in [The WAL Reset
Bug](https://www.sqlite.org/wal.html#the_wal_reset_bug) and the [SQLite
3.51.3 changelog](https://www.sqlite.org/changes.html#version_3_51_3).
2026-06-13 21:28:31 -07:00
0f957a93cd Move git utilities into a dedicated crate (#15564)
- create `codex-git-utils` and move the shared git helpers into it with
file moves preserved for diff readability
- move the `GitInfo` helpers out of `core` so stacked rollout work can
depend on the shared crate without carrying its own git info module

---------

Co-authored-by: Ahmed Ibrahim <219906144+aibrahim-oai@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-03-24 13:26:23 -07:00
jif-oai 70cdb17703 feat: add graph representation of agent network (#15056)
Add a representation of the agent graph. This is now used for:
* Cascade close agents (when I close a parent, it close the kids)
* Cascade resume (oposite)

Later, this will also be used for post-compaction stuffing of the
context

Direct fix for: https://github.com/openai/codex/issues/14458
2026-03-19 10:21:25 +00:00
jif-oai cf143bf71e feat: simplify DB further (#13771) 2026-03-07 03:48:36 -08:00
jif-oai 34c0534f6e feat: drop sqlx logging (#10398) 2026-02-02 19:26:58 +00:00
jif-oai 4ba911d48c chore: improve client (#10149)
<img width="883" height="84" alt="Screenshot 2026-01-29 at 11 13 12"
src="https://github.com/user-attachments/assets/090a2fec-94ed-4c0f-aee5-1653ed8b1439"
/>
2026-01-29 11:25:22 +01:00
jif-oai 6a06726af2 feat: log db client (#10087)
```
just log -h
if [ "${1:-}" = "--" ]; then shift; fi; cargo run -p codex-state --bin logs_client -- "$@"
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.21s
     Running `target/debug/logs_client -h`
Tail Codex logs from state.sqlite with simple filters

Usage: logs_client [OPTIONS]

Options:
      --codex-home <CODEX_HOME>  Path to CODEX_HOME. Defaults to $CODEX_HOME or ~/.codex [env: CODEX_HOME=]
      --db <DB>                  Direct path to the SQLite database. Overrides --codex-home
      --level <LEVEL>            Log level to match exactly (case-insensitive)
      --from <RFC3339|UNIX>      Start timestamp (RFC3339 or unix seconds)
      --to <RFC3339|UNIX>        End timestamp (RFC3339 or unix seconds)
      --module <MODULE>          Substring match on module_path
      --file <FILE>              Substring match on file path
      --backfill <BACKFILL>      Number of matching rows to show before tailing [default: 200]
      --poll-ms <POLL_MS>        Poll interval in milliseconds [default: 500]
  -h, --help                     Print help
  ```
2026-01-29 11:11:47 +01:00
jif-oai 780482da84 feat: add log db (#10086)
Add a log DB. The goal is just to store our logs in a `.sqlite` DB to
make it easier to crawl them and drop the oldest ones.
2026-01-29 10:23:03 +01:00
jif-oai 3878c3dc7c feat: sqlite 1 (#10004)
Add a `.sqlite` database to be used to store rollout metatdata (and
later logs)
This PR is phase 1:
* Add the database and the required infrastructure
* Add a backfill of the database
* Persist the newly created rollout both in files and in the DB
* When we need to get metadata or a rollout, consider the `JSONL` as the
source of truth but compare the results with the DB and show any errors
2026-01-28 15:29:14 +01:00