Commit Graph
12 Commits
Author SHA1 Message Date
richardopenai b834702b27 Support incremental replay of updated thread items (#35013)
## Why

Persisted item snapshots can be updated after their initial projection while retaining their creation order. Callers need a way to read those updates incrementally without replaying every item.

## What changed

- Track the rollout ordinal of each item's latest snapshot separately from its creation ordinal.
- Let thread-store callers filter items after an exclusive update watermark and sort pages by either creation or update ordinal.
- Keep existing app-server reads creation-ordered, reject incremental reads across forked lineages, and preserve compatibility with older writers.

## Testing

- Cover update-watermark filtering, pagination, sort directions, per-turn reads, cursor validation, forked lineages, migration compatibility, and refreshed item snapshots.

GitOrigin-RevId: d57de849abec27f788e9c384dbff073a0aca1053
2026-07-23 21:17:45 +00:00
brisebois-oai 83ff1c2f80 Cache remote plugin catalogs by scope (#34849)
## What changed

- Cache global, user, and workspace remote plugin catalogs on disk with a three-hour TTL and scope- and account-specific keys.
- Serve cached catalogs from `plugin/list`, refresh stale entries in the background, and warm existing caches at app-server startup without blocking list requests.
- Make `forceRefetch: true` bypass cached catalog data and replace an entry only after a successful fetch.
- Invalidate user and workspace catalog caches after remote plugin sharing changes.

## Testing

- Cover cache freshness, scope isolation, legacy global cache compatibility, identity requirements for private catalogs, forced refetches, and non-blocking startup refreshes.

GitOrigin-RevId: 7844e9dfb72e43b26d749b4afd16fd9c1793caba
2026-07-23 00:52:55 +00:00
joeytrasatti-openai 400ee190c3 Add persisted thread pinning to the app server (#34840)
## What changed

- Add `isPinned` to thread responses and allow `thread/metadata/update` to pin or unpin stored threads.
- Add an `isPinned` filter to `thread/list`, including cursor-based pagination and combinations with relationship filters.
- Persist pin state in SQLite without modifying rollout files, default existing threads to unpinned, and preserve pins through reconciliation and archive transitions.

## Testing

- Cover protocol serialization, database migration and indexing, pin updates, filtered pagination, rollout reconciliation, and archive/unarchive behavior.

GitOrigin-RevId: c2eacabd6dae465e2ae6ce8a169e00740960cdb7
2026-07-22 22:54:28 +00:00
Adam Perry @ OpenAI 946ed315a4 Centralize SQLite connection configuration (#34808)
## What changed

- Add `SqliteConfig` to own the resolved SQLite home, runtime database paths,
  and shared read/write and read-only pool settings.
- Pass the configuration through state and local thread-store consumers,
  replacing standalone path helpers and duplicated connection setup.

GitOrigin-RevId: 7351b08da94b5c8b6c0bbe492f86aeeca0699d3c
2026-07-22 18:45:47 +00:00
Charlie Marsh fd3c1dc13d Optimize remote compaction history handling (#34431)
## Why

Remote compaction can process large histories. Repeatedly estimating and replacing the full history, and cloning it when tracing is disabled, adds avoidable CPU and memory overhead.

## What changed

- Estimate each history item's token count once, update the total as trailing tool outputs are rewritten, and replace history only after all rewrites are selected.
- Preserve unclamped token totals while calculating removed tokens so saturated estimates do not hide overflow.
- Snapshot compaction input history only when rollout tracing is enabled.
- Reuse the v2 request input instead of cloning it before adding the compaction trigger.

## Testing

- Cover enabled and disabled compaction trace contexts through `is_enabled()` assertions.

GitOrigin-RevId: 8de9c9704ba29532fba05430eb204bbae2e8bf83
2026-07-20 23:30:30 +00:00
jif 687f05cb94 Remove CSV-backed agent jobs (#34413)
## What changed

- Remove the `spawn_agents_on_csv` and `report_agent_job_result` tools and their agent-job runtime and state models.
- Drop the legacy `agent_jobs` and `agent_job_items` tables during state database migration.
- Keep `features.enable_fanout` and `agents.job_max_runtime_seconds` accepted as no-op compatibility settings while omitting them from the generated configuration schema.

## Testing

- Verify upgrades remove both legacy agent-job tables.
- Verify the removed feature and configuration keys still parse without taking effect.

GitOrigin-RevId: 8cc3337da78c67162229f02f40a747f503542646
2026-07-20 21:00:17 +00:00
Adam Perry @ OpenAI 81e89fa5af Require absolute paths for test SQLite configuration (#34411)
## Why

`SqliteConfig` stores its home as an `AbsolutePathBuf`, but its test constructor
previously accepted a `PathBuf` and checked the absolute-path invariant at
runtime.

## What changed

- Make `SqliteConfig::new_for_testing` accept an `AbsolutePathBuf` directly,
  removing its fallible conversion and `expect`.
- Update SQLite test setup to convert temporary directory paths with
  `PathExt::abs` at each call site.

GitOrigin-RevId: 93585b9aea805e2449b3465ac20eeb39417ed555
2026-07-20 20:54:08 +00:00
Adam Perry @ OpenAI 6bd3f5e3db Centralize SQLite connection configuration (#33938)
## What changed

- Add `SqliteConfig` as the shared entry point for read-write and read-only SQLite pools.
- Apply consistent WAL, synchronization, auto-vacuum, busy-timeout, logging, and pool-size settings to writable Codex databases.
- Route state runtime, audit, CLI, and test database connections through the shared configuration.

## Testing

- Run migration tests against temporary on-disk databases opened through `SqliteConfig`, including the concurrent-writer repair case.

GitOrigin-RevId: e3946b98bde04c47574532ac8b1a7bb2b03edd97
2026-07-18 02:06:13 +00:00
Zanie Blue 18110b810f Avoid unnecessary writes during migration repair (#33687)
## Why

Opening a database with current migration history should not require SQLite's
writer slot. The legacy recency repair previously issued an `UPDATE` even when
there was nothing to repair, so it could fail while another connection held the
writer slot.

## What changed

Check `_sqlx_migrations` for the legacy version and checksum before issuing the
repair update, and return early when the current recency migration is already
recorded.

## Testing

Add a WAL-mode regression test that holds the writer slot on a second
connection and verifies that current migration history needs no write access.

GitOrigin-RevId: 76649213912b510df42a836cf8ed7f84a7c70eb5
2026-07-16 20:55:13 +00:00
Jeremy Rose 7dc7096ae1 [codex] Restore thread recency with compatible migration history (#28671)
## Summary

- Revert #28655, restoring the thread `recencyAt` behavior introduced by
#27910.
- Move `threads_recency_at` to migration 0039 so it no longer collides
with `external_agent_config_imports` at version 0038.
- Repair databases that already applied the recency migration as version
38 by moving the matching migration-history row to version 39 before
SQLx validation. The current version-38 migration can then apply
normally.

## Validation

- `just test -p codex-state
migrations::tests::repairs_recency_migration_that_was_applied_as_version_38`
- `just test -p codex-state -p codex-rollout -p codex-thread-store -p
codex-app-server-protocol -p codex-tui`: 3,439 passed; six TUI tests
could not open the machine's existing read-only incident database at
`~/.codex/sqlite/state_5.sqlite`.
- `just fix -p codex-state`
- `just fmt`
- Verified that state migration versions are unique.
2026-06-17 18:52:18 +00:00
pakrym-oai cb15c64760 Revert thread recencyAt for sidebar ordering (#28655)
## Why

Revert #27910 to remove the newly introduced thread `recencyAt`
persistence and API behavior from `main`.

## What changed

This reverts commit `fac3158c2a783095768076489815f361fa9b0db4`,
including the state migration, thread-store propagation, app-server API
surface, generated schemas, and related tests.

## Validation

Not run before opening; relying on CI for the initial fast signal.
2026-06-16 21:39:30 -07:00
Jeremy Rose fac3158c2a Add thread recencyAt for sidebar ordering (#27910)
## Summary

Add a server-owned `recencyAt` timestamp and `recency_at` thread-list
sort key for product recency ordering while preserving the existing
meaning of `updatedAt` as the latest persisted thread mutation.

This is the server-side alternative to #27697. Rather than narrowing
`updatedAt`, clients can sort the sidebar by `recency_at` and continue
treating `updatedAt` as mutation time.

Paired Codex Apps PR:
[openai/openai#1024599](https://github.com/openai/openai/pull/1024599)

## Contract

- `recencyAt` initializes when a thread is created.
- A turn start advances `recencyAt` monotonically.
- Commentary, agent output, tool results, token/accounting updates, turn
completion, archive, unarchive, resume, and generic metadata writes do
not advance it.
- `updatedAt` retains its existing behavior and continues to advance for
persisted thread mutations.
- Current servers populate `recencyAt`; the response field is optional
in generated TypeScript so clients connected to older servers can fall
back to `updatedAt`.
- Filesystem-only fallback uses existing updated/mtime ordering when
SQLite is unavailable.

## Persistence and compatibility

Migration 0038 adds second- and millisecond-precision recency columns,
backfills them from the existing updated timestamp, creates list
indexes, and includes an insert trigger so older binaries writing to a
migrated database seed recency without causing later mutations to
advance it.

Generic metadata upserts preserve existing recency values. Turn-start
updates use a dedicated monotonic touch, and process-local allocation
keeps millisecond cursor values unique. State DB list, search, read,
filtered-list repair, rollout fallback propagation, and app-server
conversions all carry the new field.

## API

`Thread` responses include:

```ts
recencyAt?: number
```

`thread/list` and `thread/search` accept:

```json
{ "sortKey": "recency_at" }
```

Generated TypeScript and JSON schemas are included.

## Validation

- `just test -p codex-state` — 146 passed
- `just test -p codex-rollout` — 69 passed
- `just test -p codex-thread-store` — 81 passed
- `just test -p codex-app-server-protocol` — 231 passed
- Focused app-server list ordering, response mapping, archive/unarchive,
and resume lifecycle tests passed
- Scoped `just fix` for state, rollout, thread-store,
app-server-protocol, and app-server
- `just fmt`
- `git diff --check`
- Independent correctness, simplicity, elegance, security, and
test-quality reviews; actionable ordering, lifecycle, query-projection,
and timestamp-uniqueness findings were addressed
2026-06-16 17:06:22 -07:00