Commit Graph
11 Commits
Author SHA1 Message Date
Alexi Christakis 0b94751cc4 Add telemetry for SQLite log persistence (#40726)
## Why

SQLite log persistence needs visibility into batch size, write latency,
failures, and entries dropped before they reach the writer. Exporter
diagnostics must not feed back into the SQLite log sink and keep metric
exports active indefinitely.

## What changed

- Record write count, duration, batch bytes, entry count, largest-entry size,
  and queue drops with bounded outcome tags.
- Add explicit byte histogram boundaries for log batch metrics.
- Filter OTLP transport noise and the benign unsolicited HTTP/2 PING warning
  from SQLite log persistence to prevent export cycles while retaining other
  HTTP/2 warnings.

## Testing

- Cover metric values, tags, queue-drop reasons, and histogram boundaries.
- Verify successful and failed OTLP HTTP and gRPC exports become idle instead
  of generating recurring SQLite log-write metrics.

GitOrigin-RevId: a63224ae15945fe84adc977a8b7d55bb47a4de53
2026-08-25 22:46:49 +00:00
jif-oaiandCodex e1c289e11b feat: log client use min log level (#18661)
In the log client, use the log level filter as a minimum severity
instead of exact match

---------

Co-authored-by: Codex <noreply@openai.com>
2026-04-20 14:40:39 +01:00
Charley CunninghamandCodex ebbbc52ce4 Align SQLite feedback logs with feedback formatter (#13494)
## Summary
- store a pre-rendered `feedback_log_body` in SQLite so `/feedback`
exports keep span prefixes and structured event fields
- render SQLite feedback exports with timestamps and level prefixes to
match the old in-memory feedback formatter, while preserving existing
trailing newlines
- count `feedback_log_body` in the SQLite retention budget so structured
or span-prefixed rows still prune correctly
- bound `/feedback` row loading in SQL with the retention estimate, then
apply exact whole-line truncation in Rust so uploads stay capped without
splitting lines

## Details
- add a `feedback_log_body` column to `logs` and backfill it from
`message` for existing rows
- capture span names plus formatted span and event fields at write time,
since SQLite does not retain enough structure to reconstruct the old
formatter later
- keep SQLite feedback queries scoped to the requested thread plus
same-process threadless rows
- restore a SQL-side cumulative `estimated_bytes` cap for feedback
export queries so over-retained partitions do not load every matching
row before truncation
- add focused formatting coverage for exported feedback lines and parity
coverage against `tracing_subscriber`

## Testing
- cargo test -p codex-state
- just fix -p codex-state
- just fmt

codex author: `codex resume 019ca1b0-0ecc-78b1-85eb-6befdd7e4f1f`

---------

Co-authored-by: Codex <noreply@openai.com>
2026-03-18 22:44:31 +00:00
jif-oai 31d4bfdde0 feat: add --search to just log (#11995)
Summary
- extend the log client to accept an optional `--search` substring
filter when querying codex-state logs
- propagate the filter through `LogQuery` and apply it in
`push_log_filters` via `INSTR(message, ...)`
- add an integration test that exercises the new search filtering
behavior

Testing
- Not run (not requested)
2026-02-17 14:19:52 +00:00
Charley Cunningham fce4ad9cf4 Add process_uuid to sqlite logs (#11534)
## Summary
This PR is the first slice of the per-session `/feedback` logging work:
it adds a process-unique identifier to SQLite log rows.

It does **not** change `/feedback` sourcing behavior yet.

## Changes
- Add migration `0009_logs_process_id.sql` to extend `logs` with:
  - `process_uuid TEXT`
  - `idx_logs_process_uuid` index
- Extend state log models:
  - `LogEntry.process_uuid: Option<String>`
  - `LogRow.process_uuid: Option<String>`
- Stamp each log row with a stable per-process UUID in the sqlite log
layer:
  - generated once per process as `pid:<pid>:<uuid>`
- Update sqlite log insert/query paths to persist and read
`process_uuid`:
  - `INSERT INTO logs (..., process_uuid, ...)`
  - `SELECT ..., process_uuid, ... FROM logs`

## Why
App-server runs many sessions in one process. This change provides a
process-scoping primitive we need for follow-up `/feedback` work, so
threadless/process-level logs can be associated with the emitting
process without mixing across processes.

## Non-goals in this PR
- No `/feedback` transport/source changes
- No attachment size changes
- No sqlite retention/trim policy changes

## Testing
- `just fmt`
- CI will run the full checks
2026-02-14 17:27:22 -08:00
jif-oai eff11f792b feat: improve logs client (#10229) 2026-01-30 18:23:18 +01:00
jif-oai 798c4b3260 feat: reduce span exposition (#10171)
This only avoids the creation of duplicates spans
2026-01-29 18:15:22 +00:00
jif-oai e6c4f548ab chore: unify log queries (#10152)
Unify log queries to only have SQLX code in the runtime and use it for
both the log client and for tests
2026-01-29 16:28:15 +00:00
jif-oai 89c5f3c4d4 feat: adding thread ID to logs + filter in the client (#10150) 2026-01-29 16:53:30 +01:00
jif-oai 4ba911d48c chore: improve client (#10149)
<img width="883" height="84" alt="Screenshot 2026-01-29 at 11 13 12"
src="https://github.com/user-attachments/assets/090a2fec-94ed-4c0f-aee5-1653ed8b1439"
/>
2026-01-29 11:25:22 +01:00
jif-oai 780482da84 feat: add log db (#10086)
Add a log DB. The goal is just to store our logs in a `.sqlite` DB to
make it easier to crawl them and drop the oldest ones.
2026-01-29 10:23:03 +01:00