Commit Graph
54 Commits
Author SHA1 Message Date
pmccrary-oai bdfd769640 Track history notes thread hint outcomes (#42247)
## What changed

- Emit a `codex_thread_hint_status` analytics event for each native history-notes thread hint attempt.
- Report whether retrieval was successful or failed along with thread context and timing, without including hint contents.
- Treat valid empty responses as successful retrievals while continuing to omit them from the context window.

## Testing

- Extend the app-server history-notes tests to verify success, empty-result success, and backend failure statuses.

GitOrigin-RevId: b9462f312847e8c871ed2be0c8cf8df0928a7fdd
2026-09-02 09:12:17 +00:00
jif 86b7d12742 Add Guardian V2 analytics events (#42144)
## What changed

- Emit `codex_guardian_v2_classification` events with the outcome, risk
  level, duration, model, and thread attribution.
- Emit `codex_guardian_v2_fast_decision` events when Guardian V2 approves an
  action without a full review.
- Add `guardian_v2_enabled` to turn analytics, accounting for review scope
  and model requirements.

## Testing

- Extend the Guardian V2 app-server coverage to verify event payloads,
  attribution, enablement state, and scoped event emission.

GitOrigin-RevId: 0805975f8e8f9d954aa1058ea8c60545a090ae74
2026-09-01 22:02:42 +00:00
Eddie Chen 2b7c279735 Report turn trigger and source in turn analytics (#42003)
## What changed

- Add `turn_trigger` and `codex_turn_source` to `codex_turn_event` analytics.
- Read the source from effective Responses metadata, preserving configured-over-client precedence and reflecting accepted steer metadata.
- Omit trigger and source values from analytics when they exceed the existing Responses metadata value limit.

## Testing

- Cover event serialization, turn lifecycle emission, configured and client source precedence, size limits, and source updates after steering.

GitOrigin-RevId: bbd45af4c6af52956d2394b722ea78e65e5b9bd7
2026-09-01 07:32:54 +00:00
marksteinbrick-oai 307ce6cda9 Track Guardian reviewer turn and tool analytics (#41023)
## Why

Trusted Guardian reviewer sessions can emit events without an app-server listener, leaving their turns and tool usage absent from analytics.

## What changed

- Translate Guardian session lifecycle and canonical tool item events into analytics notifications while excluding messages, reasoning, and review content.
- Preserve Guardian lineage during prewarming even when inherited client metadata is not yet available, without emitting an incomplete thread initialization event.
- Limit direct session tracking to trusted Guardian children so resumed threads that have an app-server listener do not report duplicate events.

## Testing

- Cover completed, failed, and interrupted Guardian turns, tool counts and timing, lineage, content exclusion, prewarmed lineage, and resumed-thread deduplication.

GitOrigin-RevId: 70ace5f7ed30a7e3b6a9d6097c8b2a6a326dc4f7
2026-08-27 02:19:10 +00:00
Eddie Chen 49880081fe Add root turn IDs to turn and tool analytics (#40486)
## Why

Analytics need to associate subagent activity with the top-level turn that
caused it, while avoiding a stale association when steering makes that origin
ambiguous.

## What changed

- Add `root_turn_id` to turn and tool-item analytics events.
- Read the root from live turn metadata when each event is built, reporting
  `null` when the root is unknown or ambiguous.
- Propagate the metadata through control-tool and code-mode completion facts so
  correlated and background tool events retain the correct root.

## Testing

- Cover root IDs for top-level turns, subagent turns, correlated code-mode
  calls, and tool events emitted after turn completion.
- Verify that queued and subsequent events report `null` after the live root is
  invalidated.

GitOrigin-RevId: 42fd76541c3c716cec74787d47b50adb2d22e6d2
2026-08-24 22:30:59 +00:00
Dylan Hurd c9b19deb09 Distinguish Guardian review threads from subagents (#40221)
## Why

Guardian reviews were reported with the generic `subagent` thread source, making them indistinguishable from other delegated threads in persisted metadata and analytics.

## What changed

- Add the `guardian_review` thread source to the core protocol and app-server schema.
- Assign it to Guardian reviewer sessions and propagate the configured source through turn metadata, rollouts, and analytics.
- Continue treating Guardian reviews like subagents for paginated history and trusted-provenance checks.

## Testing

- Cover protocol round trips, delegate source selection, emitted analytics, request metadata, and persisted rollout metadata.

GitOrigin-RevId: 07480c122715812874c9d3b48bb39fc5c86b2367
2026-08-23 10:06:57 +00:00
jif 992f5c681f Track built-in control tool calls in analytics (#39510)
## What changed

- Emit `codex_control_tool_call_event` for built-in control tools such as
  `request_user_input`, `update_plan`, `view_image`, and goal tools.
- Record correlation and timing metadata plus completed, failed, rejected, or
  interrupted outcomes, and include these calls in turn-level tool counts.
- Keep tool arguments out of control-tool analytics events.

## Testing

- Extend the app-server turn profile test to cover successful and failed
  control tools, event correlation, turn counts, and argument exclusion.

GitOrigin-RevId: c20f73dfb79654e31b4b5040eddb0e0d91cae10d
2026-08-19 15:55:38 +00:00
Abhinav 87070a7792 Enable MCP tool hooks in Codex sessions (#39296)
## What changed

- Execute `mcp_tool` hook handlers through the session's shared MCP runtime, including managed hook configurations.
- Restrict hook calls to already-connected, cataloged, and policy-allowed tools. Unavailable servers fail immediately without starting or reconnecting them.
- Pass session metadata to hook tools, cap hook timeouts by the server timeout, and invoke hooks without model-tool approval or recursive hook dispatch.
- Record handler type and execution mode in hook telemetry, and mark memory as externally influenced after successful or blocking MCP hooks when configured.

## Testing

- Added end-to-end coverage for prompt interpolation, hidden tools, blocking decisions, unavailable servers, thread metadata, approval bypass, and memory-mode handling.
- Added MCP runtime coverage for startup readiness, tool filters, catalog revisions, and managed connector policy.

GitOrigin-RevId: 1647cb801ebc5fa385c7bda9d9c266a389c611b8
2026-08-18 22:10:43 +00:00
Tamir Duberstein 5c6f498b0e Stop generating accepted-line fingerprints (#38473)
## What changed

- Replace accepted-line fingerprint generation with diff-based added and deleted line counts.
- Keep `line_fingerprints` in the analytics payload as a statically empty field for schema compatibility.
- Remove the fingerprint types and public parsing API that are no longer used.

## Testing

- Update diff parsing tests to cover accepted-line counts without generating fingerprints.

GitOrigin-RevId: 4247c76c7b56b1a323f70f960823937352b84f7b
2026-08-14 01:44:33 +00:00
Kyle Brown d6eefb26a6 Add bounded plugin measurement analytics (#38239)
## What changed

- Add `PluginMeasurementsInput` and `PluginMeasurementRow`, plus an analytics
  client method for recording batches and checking whether analytics are enabled.
- Emit each valid row as a `codex_plugin_measurement_event` with thread, turn,
  item, plugin, execution, and operation context.
- Bound batches at 100 rows and reject non-finite values, invalid identifiers,
  and rows with more than eight dimensions.
- Treat measurement events as plugin events when routing API-key analytics.

## Testing

- Cover batch limits, row filtering, serialized event payloads, and API-key
  plugin-event routing.

GitOrigin-RevId: f949ed06246ee1e22466df2660076f2d4f556444
2026-08-12 19:14:09 +00:00
rphilizaire-openai c8f673fddc Track resource-backed skill invocations (#38066)
## What changed

- Emit skill invocation analytics when a resource-backed skill is explicitly selected or its main resource is successfully read from the first page.
- Preserve provider-supplied skill IDs and user scope when available, and derive a stable ID from the main resource otherwise.
- Deduplicate repeated implicit reads of the same main resource within a turn.

## Testing

- Extend the orchestrator skill integration test to cover explicit and implicit invocation events, stable fallback IDs, failed reads, and repeated-read deduplication.

GitOrigin-RevId: 7d8453b31b5a2205847e65cda2c46183d6c97b45
2026-08-11 19:52:54 +00:00
edwardysun3 44d992c14e Track artifact operations from trusted plugin markers (#38057)
## What changed

- Recognize validated create and edit marker commands from the trusted presentations, documents, spreadsheets, and PDF plugins.
- Emit a `codex_artifact_operation` analytics event plus started-count and expected-output-count metrics when a recognized marker starts through unified exec.
- Attribute remote executor commands only when their plugin cache path and script contents match a trusted local plugin, including primary-runtime plugins.

## Testing

- Cover supported marker combinations and reject mismatched plugins, paths, arguments, counts, and output formats.
- Cover local, relocated remote, and Windows-style plugin attribution, including rejection of modified remote scripts.

GitOrigin-RevId: 475cc1689aa7f371a24dec0a8bdb6f7fd5a2e922
2026-08-11 19:26:41 +00:00
jif 155f1ca9e5 Correlate code mode tool analytics with model responses (#36729)
## What changed

- Emit dynamic tool-call analytics for code mode `exec` and `wait` calls, including duration and terminal status.
- Associate code-mode cells and their nested tool calls with the originating and subsequent model response IDs.
- Buffer correlated tool events until the next response is known, while flushing pending events when turns, threads, or the analytics queue close.

## Testing

- Add reducer coverage for correlating `exec`, `wait`, and nested tool events across turns.
- Add an app-server test that verifies production analytics include the cell and response IDs for a code-mode `exec` call.

GitOrigin-RevId: 8b34ab59c1ad68ebbd57f9c8a05f19ab04df48a4
2026-08-03 10:19:04 +00:00
Curtis 'Fjord' Hawthorne 1bef168976 Track image preparation details in turn analytics (#36388)
## What changed

- Record the effective detail setting and source and prepared dimensions for
  successfully decoded images.
- Associate message images with their role and tool-output images with their
  originating call ID.
- Include the collected image preparation metadata in the corresponding turn
  analytics event.

## Testing

- Cover image resize metadata, tool-output attribution, turn aggregation, event
  serialization, and app-server analytics output.

GitOrigin-RevId: dbfb5ba1ce7cb02dd93d9b579c5109817f16aef5
2026-07-31 20:34:41 +00:00
alicec-oai 6219b7c40f Correlate image requests and explicit interrupts with turns (#36092)
## What changed

- Send the active turn ID in the `x-codex-image-turn-id` header for image generation and edit requests.
- Add `explicit_client_interrupt_requested_at_ms` to turn analytics. Record the earliest explicit interrupt request that receives a successful response, while excluding rejected requests.
- Track errors returned by `turn/interrupt` so pending analytics state is cleaned up correctly.

## Testing

- Cover turn ID headers for image generation and editing.
- Cover accepted, retried, and rejected explicit interrupt requests in analytics tests.

GitOrigin-RevId: 80fe996a161b4952b79c5c38238f75b7661b92af
2026-07-30 05:53:21 +00:00
jameswt-oai 0d2a0aa76b Track remote plugin IDs in skill invocation analytics (#35262)
## What changed

- Add `remote_plugin_id` to skill invocation facts and analytics event parameters.
- Propagate the ID for both explicit and implicit plugin skill invocations.

## Testing

- Cover explicit and implicit remote plugin skill invocations with analytics integration tests.

GitOrigin-RevId: 3e1d25b5ff88dcff9c2b6c9fe44bca97e8778221
2026-07-24 23:40:14 +00:00
Adam Perry @ OpenAI 7bafdada8b Separate Codex error details from retry metadata (#34996)
## What changed

- Wrap `CodexErrorDetails` and an optional retry delay in `CodexErr`, allowing any mapped error to preserve server-provided retry timing.
- Generate the payload-free `CodexErrKind` classification alongside the error details and reuse it for analytics.
- Update error handling sites to inspect `CodexErr::details()` while preserving existing display, debug, protocol mapping, and retryability behavior.

## Testing

- Add coverage for legacy debug formatting, error-specific retryability, and retry-delay propagation through API error mapping.

GitOrigin-RevId: d3ab8a305f2a2ee21d0c0a8c8c388b06dda9c59a
2026-07-23 19:32:26 +00:00
Ben Romano bd5b55e403 Track compaction time in turn profiles (#34835)
## What changed

- Add `compaction_ms` to turn profile facts and turn analytics events.
- Measure manual and automatic compaction as an exclusive profile phase instead of including it in surrounding idle time.
- Complete the profile and total turn duration from the same instant so their elapsed times remain consistent.

## Testing

- Cover compaction phase accounting and profile-duration consistency.
- Update turn event serialization coverage for `compaction_ms`.

GitOrigin-RevId: 50f0e1cf19789a807f9f56c2ef763c620a5d641c
2026-07-22 22:13:02 +00:00
charlesgong-openai a30aee8d90 Attribute external agent imports by provider (#34451)
## What changed

- Add an optional opaque `providerId` to `externalAgentConfig/import` for analytics attribution, independently of the `migrationSource` selector.
- Include the provider ID in completed-import and import-failure analytics events.
- Have TUI imports identify their selected migration source as the provider.

## Testing

- Cover provider attribution in app-server import completion and failure tests and analytics serialization tests.

GitOrigin-RevId: 0ee7313a43652e5305939c526bf6b903a79334ad
2026-07-21 02:59:22 +00:00
Eric Traut 2edad72de3 Track prompt cache write token usage (#33454)
## What changed

- Parse `cache_write_tokens` from response input token details and carry it through token usage aggregation.
- Expose `cache_write_input_tokens` in protocol, app-server, exec, and TypeScript SDK usage events, defaulting it to zero for compatibility with older payloads.
- Report cache-write usage in analytics, metrics, traces, and rollout data alongside existing cache-read usage.

## Testing

- Add coverage for response parsing, usage event output, aggregation, analytics, metrics, and telemetry fields.

GitOrigin-RevId: 634e4fbe3086a8923c0ee13b622573e7d9d221bd
2026-07-16 00:22:53 +00:00
charlesgong-openai 3fae30dd85 Preserve plugin install failure subtypes during imports (#33182)
## What changed

- Add `subErrorType` to external agent config import failures and carry it through progress and completion notifications, persisted import histories, and analytics.
- Expose plugin store failure subtypes from `PluginInstallError` so manual installs and imported plugins report details such as `failed_to_copy_plugin_file` while failures without a subtype remain `null`.
- Keep existing persisted import records compatible by defaulting the new field when it is absent.

## Testing

- Cover store I/O subtype extraction, analytics serialization, persisted import histories, and migration notification fixtures.

GitOrigin-RevId: ec1a85623fc826cbd7cfe9f55c2dd2bc86665937
2026-07-14 22:21:27 +00:00
charlesgong-openai 8dc77faf51 Import enabled plugins from known marketplaces (#31672)
## Summary

- discover enabled-plugin marketplace sources from the user-level known
marketplace registry
- preserve scoped settings and supported Git/directory declarations,
falling back to the matching registry `installLocation` for file, URL,
npm-hosted, and inline marketplace sources
- resolve relative registry materialization paths against the
external-agent home
- reuse the existing marketplace add and plugin install flow during
import
- emit `plugin_not_found` tracing and analytics with plugin and
marketplace identity

## Why

Enabled plugins can remain in settings after their marketplace source
has moved into the user-level registry. Import detection previously
skipped those plugins because it only consulted settings-defined
marketplace sources.

## Impact

`/import` can now restore those marketplaces and install their enabled
plugins through the normal plugin installer, including npm, Git, and
local plugin sources. Missing plugins remain visible in import results
and the plugin-install failure analytics stream.
2026-07-09 12:16:15 -04:00
alexsong-oai 841f30598c [codex] Attribute app-server analytics by thread originator (#29935)
## Why

Desktop Work threads and regular Codex threads can share the same
app-server connection. App-server analytics currently copy
`product_client_id` from connection metadata for every thread-scoped
event, so Work thread activity is attributed to the Desktop connection
instead of the thread's resolved originator. This prevents analytics
from distinguishing the two products on a shared connection.

## What changed

- Publish the resolved originator after a thread is materialized,
covering new, resumed, forked, and subagent threads.
- Store that originator in the analytics reducer's existing per-thread
state.
- Override only `app_server_client.product_client_id` for thread, turn,
tool, review, goal, guardian, and compaction events while preserving the
connection's client name, version, and transport metadata.
- Fall back to the connection-wide product client ID when a thread has
no originator override.
- Preserve persisted originators in thread initialization analytics for
resume and fork flows.

## Validation

- `just test -p codex-analytics
thread_originator_overrides_shared_connection_across_thread_events
subagent_events_keep_thread_originator_with_explicit_turn_connection`
- `just test -p codex-app-server
turn_start_tracks_thread_originator_in_analytics
thread_start_tracks_thread_initialized_analytics
thread_fork_tracks_thread_initialized_analytics
thread_resume_tracks_thread_initialized_analytics`
- `just test -p codex-core thread_manager`
2026-06-25 18:15:48 -07:00
Alex Daley 24423f5712 [plugins] Track plugin install requests by ID (#29684)
Summary
- Emit `codex_plugin_install_requested` when a validated plugin install
request is made, before the user accepts or declines the elicitation.
- Record the exact model-visible plugin ID, remote plugin ID, required
connector IDs, stable suggestion ID, and `endpoint_recommendation` vs
`legacy_discovery` source.
- Keep `suggest_reason` out of telemetry and leave connector-only
install requests unchanged.

Rollout
- Backend/schema dependency:
https://github.com/openai/openai/pull/1065270
- Land the backend PR before this producer starts sending the event.

Validation
- `just test -p codex-analytics` (83 passed)
- `just test -p codex-core request_plugin_install` (17 passed)
- `just fix -p codex-analytics`
- `just fix -p codex-core`
- `just fmt`
- `git diff --check`
2026-06-24 21:29:11 +00:00
alexsong-oai 1acb722e8a Support thread-level originator overrides (#29477)
## Why

Work(TPP) threads can be launched from the Desktop app, but if they all
keep the Desktop app's default originator then downstream attribution
cannot distinguish local Work launches from cloud-backed Work launches.
`thread/start.serviceName` already carries that launch signal, while
`SessionMeta.originator` is the durable thread-level value that survives
resume and fork.

This change converts the Desktop Work service names into an effective
originator at thread creation time, persists that originator with the
thread, and keeps using it for later model requests and memory writes.

## What changed

- Map `CODEX_WORK_LOCAL` and `CODEX_WORK_CLOUD` service names to
per-thread originators, while preserving
`CODEX_INTERNAL_ORIGINATOR_OVERRIDE` as the highest-precedence override.
- Persist the effective originator in `SessionMeta.originator`, read it
back on resume/fork, and inherit the parent originator for subagent
spawns when there is no persisted session metadata.
- Handle truncated `SpawnAgentForkMode::LastNTurns` forks by falling
back to the live parent originator when the forked history no longer
includes `SessionMeta`.
- Thread the per-thread originator through Responses headers,
websocket/compaction request paths, thread-store creation, rollout
metadata, and memory stage-one telemetry.

## Verification

- `just test -p codex-core
agent::control::tests::spawn_thread_subagent_inherits_parent_originator_without_fork
agent::control::tests::spawn_thread_subagent_fork_last_n_turns_inherits_parent_originator_without_session_meta
thread_manager::tests::originator_override_precedes_service_name_remapping`
- `just test -p codex-core
agent::control::tests::resume_thread_subagent_restores_stored_metadata_and_effective_multi_agent_mode`
- `just test -p codex-memories-write`
- `just fix -p codex-core -p codex-memories-write`
- `git diff --check`
2026-06-23 17:23:38 -07:00
rka-oai 1ec3def0b5 [codex] rename rollout budget error to session budget error (#29744)
## Summary

- rename the rollout-budget exhaustion error from
`RolloutBudgetExceeded` to `SessionBudgetExceeded`
- expose the matching app-server v2 wire value as
`sessionBudgetExceeded`
- regenerate JSON/TypeScript schema fixtures and update the app-server
docs and focused tests

This is a naming-only follow-up to #29715 based on [Pavel's review
suggestion](https://github.com/openai/codex/pull/29715#discussion_r3463183480).
Runtime behavior is unchanged.

## Tests

- `just test -p codex-core rollout_budget`
- `just test -p codex-app-server-protocol`
- `just fmt`
- `just write-app-server-schema`
2026-06-23 16:49:13 -07:00
rka-oai bbbea91960 [codex] surface rollout budget exhaustion (#29715)
## Summary
- surface shared rollout-budget exhaustion as
`CodexErr::RolloutBudgetExceeded` instead of a generic interrupted turn
- map it through the existing `CodexErrorInfo` and app-server v2
`codexErrorInfo` path
- keep local compaction from retrying after the shared rollout budget is
exhausted

This gives app-server clients a stable `rolloutBudgetExceeded` error
they can classify without guessing from `status="interrupted"`.

## Tests
- `just test -p codex-core rollout_budget`
2026-06-23 15:01:28 -07:00
charlesgong-openai 3959ab0ffc [codex] Track plugin install and import telemetry failures (#28731)
## Summary
- Track plugin install failures through the unified
`codex_plugin_install_failed` event for local installs, remote install
preflight failures, bundle failures, and remote catalog/backend
failures.
- Send classified `error_type` values in plugin install failure
analytics instead of raw error strings.
- Stop sending raw external-agent import errors in analytics while
preserving raw failure details in app-facing import
notifications/history.
- Keep raw plugin/migration diagnostics in `tracing::warn!` logs.
- Keep remote failure plugin names as the existing local placeholder
(`unknown`) and remove the extra telemetry plugin-name override.
- Change `ExternalAgentConfigImportParams.source` from a generated enum
to `string | null`, with legacy `claudeCode` / `claudeCowork` inputs
normalized to existing analytics values.

## Testing
2026-06-17 13:16:34 -07:00
rhan-oai 8d9f33c87c [codex-analytics] Emit structured compaction codex errors (#27082)
## Summary
- replace raw compaction `error` analytics with `codex_error_kind` and
`codex_error_http_status_code`
- derive compaction error telemetry from `CodexErr` using the same
`CodexErrKind` mapping and HTTP status helper used by turn events
- remove the pre-compact hook stop reason from the internal compaction
outcome now that it is no longer emitted as raw analytics text

## Why
Compaction `error` was a raw `CodexErr::to_string()` value, which can
carry free-form provider or user-derived text. Structured Codex error
fields preserve useful low-cardinality telemetry without sending the raw
string.

## Validation
- `just fmt`
- `just test -p codex-analytics`
- `just test -p codex-core
compact::tests::build_token_limited_compacted_history_appends_summary_message`

Attempted `just test -p codex-core`; the changed crate compiled, but the
full target failed in unrelated environment-dependent tests such as
missing helper binaries and shell snapshot timeouts.
2026-06-11 06:07:06 +00:00
rhan-oai 383708e74e [codex-analytics] report cached input tokens for v2 compaction (#27103)
## Summary

- add nullable `cached_input_tokens` to the compaction analytics event
- populate it from response usage for compaction v2
- leave it `null` for other compaction implementations

This adds visibility into prompt-cache usage for v2 compaction without
changing compaction behavior.

## Testing

- `just test -p codex-analytics`
- `just test -p codex-core
collect_compaction_output_accepts_additional_output_items`
2026-06-10 22:47:22 -07:00
Ahmed Ibrahim ba4925b3c2 [codex] Compact when comp_hash changes (#27520)
## Summary
- snapshot `comp_hash` into `TurnContext` when the turn is created and
use that snapshot as the downstream source of truth
- persist the turn hash in rollout context and recover it into
previous-turn settings during resume and fork replay
- compact existing history with the previous model only when both
adjacent turns provide hashes and the values differ
- record `comp_hash_changed` as the compaction reason
- cover ordinary transitions, resume, and missing-hash compatibility
with end-to-end tests

## Why
History produced under one compaction-compatible model configuration may
not be safe to carry directly into another. Compacting at the turn
boundary converts that history before context updates and the new user
message are added. Persisting the turn snapshot in `TurnContextItem`
makes the same protection work after resuming a rollout.

A missing hash is not treated as evidence of incompatibility. `None →
Some`, `Some → None`, and `None → None` do not trigger compaction; only
`Some(previous) → Some(current)` with unequal values does.

## Stack
- depends on #27532
- #27532 is based directly on `main`

## Testing
- `just test -p codex-core pre_sampling_compact_` — 6 passed
- `just test -p codex-core
turn_context_item_uses_turn_context_comp_hash_snapshot` — passed
- `just fix -p codex-core -p codex-protocol -p codex-analytics -p
codex-models-manager`
2026-06-11 04:11:26 +00:00
marksteinbrick-oai 608b8b1cc6 [codex-analytics] emit goal lifecycle analytics (#27078)
## Why
- Currently, there is no analytics event for `/goal` behavior
- Existing events cannot identify goal execution or its resulting
outcome
- The original update in
[#26182](https://github.com/openai/codex/pull/26182) was implemented
before `/goal` moved into `codex-goal-extension`.

## What Changed
- Adds `codex_goal_event` serialization and enrichment to
`codex-analytics`
- Emits goal events from the canonical `codex-goal-extension` mutation
and accounting paths:
  - `created` when a new logical goal is persisted
  - `usage_accounted` when cumulative goal usage is persisted
  - `status_changed` when the stored goal status changes
  - `cleared` when the goal is deleted
- Preserves causal `turn_id` for turn driven events and uses null
attribution for external or idle lifecycle events
- Changes goal deletion to return the deleted row so `cleared` retains
the stable goal ID

## Event Details

Includes standard analytics metadata along with goal specific fields:
- `goal_id`: Stable ID stored in the local SQLite goal row and shared
across the goal's events
- `event_kind`: Observed operation (see the 4 lifecycle events cited in
the above bullet)
- `goal_status`: Resulting or last stored status: `active`, `paused`,
`blocked`, `usage_limited`, etc.
  - `has_token_budget`: Indicates whether a token budget is configured
  - `turn_id`: Causal turn ID, or null when no causal turn exists
- `cumulative_tokens_accounted`: Cumulative tokens on `usage_accounted`
events; null otherwise
- `cumulative_time_accounted_seconds`: Cumulative active time on
`usage_accounted` events; null otherwise

## Validation
- `just test -p codex-analytics -p codex-state -p codex-goal-extension`
- `just test -p codex-core -E 'test(/goal/)'`
- `just test -p codex-app-server`
- `cargo build -p codex-analytics -p codex-core -p codex-state -p
codex-app-server`
2026-06-09 18:45:54 -07:00
rhan-oai ee6c91d5cf [codex-analytics] stop sending codex error subreason (#27060)
## Summary
- stop emitting `codex_error_subreason` on `codex_turn_event`
- remove the transient analytics fact plumbing that copied
`CodexErr::InvalidRequest(String)` into the event
- update analytics serialization coverage accordingly

## Why
`codex_error_subreason` is a free-form copy of `InvalidRequest(String)`,
including raw provider 400 bodies in some paths. That makes it unsafe as
an analytics field because it can carry user-derived or sensitive text.

## Validation
- `just fmt`
- `just test -p codex-analytics`
2026-06-08 21:29:06 +00:00
rhan-oai f1c18df9ae [codex-analytics] report compaction analytics details (#26680)
## Why

Compaction analytics adds retained image count and compaction summary
output tokens for v1.5 specifically.

## What changed

- Add nullable `retained_image_count` and `compaction_summary_tokens`
fields to `codex_compaction_event`.
- Populate them only for `responses_compaction_v2`: retained images come
from the retained v2 compacted history, and summary tokens come from
`response.completed.token_usage.output_tokens`.
- Leave local and legacy remote compaction events as `null` for these
detail fields.

## Verification

- `just fmt`
- `just fix -p codex-core`
- `just test -p codex-core
build_v2_compacted_history_counts_retained_input_images`
- `git diff --check`
2026-06-08 10:52:31 -07:00
Ahmed Ibrahim 8d72fb6de9 [codex] Add turn profiling analytics (#26484)
## Summary

Add flat profiling fields to `codex_turn_event` so analytics can explain
where turn wall-clock time is spent without changing tool execution
behavior.

The profile reports:
- time before the first sampling request
- sampling time across all attempts and follow-ups
- overhead between sampling requests
- time blocked in the post-sampling tool drain
- time after the final sampling request
- sampling request and retry counts

## Implementation

- Extend the existing turn timing state with constant-memory phase
accounting and one RAII phase guard.
- Observe sampling and the existing post-sampling drain only at turn
orchestration boundaries.
- Keep tool runtime, tool futures, response item handling, and turn
lifecycle values unchanged.
- Add the profiling fields directly to the existing analytics turn event
without changing app-server protocol or rollout persistence.
- Use the existing turn `status` to distinguish completed, failed, and
interrupted profiles.

Exact sampling/tool overlap is intentionally omitted because measuring
tool completion accurately would require hooks in the tool execution
path.

## Validation

- Add app-server end-to-end coverage for a single-sampling turn with no
blocking tool work.
- Add app-server end-to-end coverage for `request_user_input` blocking
followed by a second sampling request.
- CI is running on the PR; tests were not executed locally per
repository guidance.
2026-06-05 11:27:10 -07:00
kbazzi 9e41f8ddbe [codex-analytics] emit forked thread id on initialization (#26248)
## Why
- Thread initialization analytics do not identify the source thread for
forked threads.
- The session viewer needs this lineage to construct thread trees.
- Depends on openai/openai#987854. Do not release this change before
that backend schema change is deployed.

## What Changed
- Adds optional `forked_from_thread_id` to `codex_thread_initialized`.
- Populates it from the existing thread fork lineage for app-server and
in-process subagent initialization paths.
- Keeps it null for non-forked threads.

## Verification
- `just fmt`
- `just test -p codex-analytics`
- `just test -p codex-app-server
thread_fork_tracks_thread_initialized_analytics`
2026-06-04 11:24:12 -07:00
knittel-openai b794182ea7 Populate workspace kind on Codex turn events (#25135)
## Summary
- carry `workspace_kind` from Responses API client metadata into the
turn resolved analytics fact
- serialize the optional value on `codex_turn_event`
- cover both the turn metadata source and turn event serialization

The `workspace_kind` tells us whether a thread had a project attached vs
projectless. this is an indicator for who is adopting Codex for
knowledge work outside of coding

## Testing
- `env UV_CACHE_DIR=/private/tmp/uv-cache
/private/tmp/cargo-tools/bin/just fmt`
- `env PATH=/private/tmp/cargo-tools/bin:$PATH
CARGO_HOME=/private/tmp/cargo-home UV_CACHE_DIR=/private/tmp/uv-cache
/private/tmp/cargo-tools/bin/just test -p codex-analytics`
- `env PATH=/private/tmp/cargo-tools/bin:$PATH
CARGO_HOME=/private/tmp/cargo-home UV_CACHE_DIR=/private/tmp/uv-cache
/private/tmp/cargo-tools/bin/just test -p codex-core turn_metadata`

Paired with openai/openai#970661, which keeps forwarding the same
metadata key through Responses API headers.
2026-06-02 12:46:14 -07:00
rhan-oai 8e4b92d294 [codex-analytics] Track CodexErr details in turn analytics (#25707)
## Summary
- add analytics-only `CodexErr` telemetry to `codex_turn_event` while
leaving existing `turn_error` unchanged
- record terminal `CodexErr` facts from core immediately before the
existing turn error event is sent
- emit source-truth `codex_error_*` fields for downstream analytics,
including the raw `CodexErr::InvalidRequest(String)` message as
`codex_error_subreason`

## Validation
- `just test -p codex-analytics`
- attempted `just test -p codex-core`, but the local run timed out
across unrelated integration suites in this environment and is not being
used as validation
2026-06-02 11:40:35 -07:00
marksteinbrick-oai 487521733b [codex-analytics] add grouped session id to runtime events (#24655)
## Why
- Runtime analytics events report `thread_id`, which identifies the
individual thread emitting an event
- They don't report `session_id`, which identifies the shared session
for a root thread and its subagent threads
- Emitting both identifiers allows analytics to group related activity

## What Changed
- Adds `session_id` to relevant analytics events (thread_initalized,
turn, turn_steer, compaction, guardian_review)
- Tracks each thread's session ID in the analytics reducer so subsequent
thread scoped events emit the same value
- Carries the shared session ID through subagent initialization

## Verification
- `just test -p codex-analytics` validates event payloads and subagent
session grouping.
- Focused `codex-app-server` tests validate session IDs for thread,
turn, and steer events.
- Focused `codex-core` tests validate root and subagent session ID
propagation.
2026-05-26 16:38:46 -07:00
rhan-oai 6419402a7c [codex-analytics] split compaction v2 analytics implementation (#24146)
## What changed

- Add a distinct `responses_compaction_v2` value for
`CodexCompactionEvent.implementation`.
- Emit that value from the remote compaction v2 path.
- Keep local compaction as `responses` and legacy `/responses/compact`
as `responses_compact`.

## Why

Remote compaction v2 and local prompt-based compaction were both
reported as `responses`, which made the analytics table collapse two
different compaction mechanisms into one implementation bucket.

## Validation

- `just fmt`
- `just test -p codex-analytics`

`just test -p codex-core` was started locally, but this PR is
intentionally being pushed for CI to finish the remaining validation.
2026-05-22 21:34:22 +00:00
rhan-oai a175ddacc0 [codex-analytics] emit terminal review events (#18748)
## Why

Review telemetry should describe reviews as first-class events, not only
as counters denormalized onto terminal tool-item events. That lets us
analyze guardian and user reviews consistently across command execution,
file changes, permissions, and network access, while still preserving
the terminal item summaries that existing tool analytics need.

To make those review events accurate, analytics also needs the observed
completion time for each review and enough command metadata to
distinguish `shell` from `unified_exec` reviews.

## What changed

- emit generic `codex_review_event` rows for completed user and guardian
reviews, with review subjects, reviewer, trigger, terminal status,
resolution, and observed duration
- reduce approval request / response / abort facts into review events
for command execution, file change, and permissions flows
- keep denormalized review counts, final approval outcome, and
permission-request flags on terminal tool-item events for
item-associated reviews
- plumb review completion timing so user-review responses and aborts use
app-server-observed completion times, while guardian analytics reuse the
same terminal timestamps emitted on guardian assessment events
- carry command approval `source` through the protocol and app-server
layers so review analytics can distinguish `shell` from `unified_exec`
- add analytics coverage for user-review emission, guardian-review
emission, permission reviews that should not denormalize onto tool
items, item-summary isolation across threads, and the serialized
review-event shape

## Verification

- `cargo test -p codex-analytics`

---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/18748).
* __->__ #18748
* #21434
* #18747
* #17090
* #17089
* #20514
2026-05-11 22:13:32 +00:00
alexsong-oai bbb6bf0a37 Emit accepted line fingerprint analytics (#21601)
## Why

Codex assisted-code attribution needs a client-side accepted-code source
that does not upload raw code. This adds a hash-only analytics event
derived from the turn diff so downstream attribution can compare
accepted Codex lines against commit or PR diffs.

## What Changed

- Parse accepted/effective added lines from the final turn diff and emit
`codex_accepted_line_fingerprints` analytics.
- Hash repo, path, and normalized line content before upload; raw code
and raw diffs are not included in the event.
- Chunk large fingerprint payloads and send accepted-line fingerprint
events in isolated requests while preserving normal batching for other
analytics events.
- Canonicalize Git remote URLs before repo hashing so SSH/HTTPS GitHub
remotes join to the same repo hash.
- Add parser coverage for unified diff hunk lines that look like `+++`
or `---` file headers.

## Verification

- `cargo test -p codex-analytics`
- `cargo test -p codex-git-utils canonicalize_git_remote_url`
- `just fix -p codex-analytics`
- `just bazel-lock-check`
- `git diff --check`
2026-05-08 12:16:24 -07:00
rhan-oai 99016ec732 [codex-analytics] plumb protocol-native review timing (#21434)
## Why

We want terminal tool review analytics, but the reducer should not stamp
review timing from its own wall clock.

This PR plumbs review timing through the real protocol and app-server
seams so downstream analytics can consume the emitter's timestamps
directly. Guardian reviews keep their enriched `started_at` /
`completed_at` analytics fields by deriving those legacy second-based
values from the same protocol-native millisecond lifecycle timestamps,
rather than sampling a separate analytics clock.

## What changed

- add `started_at_ms` to user approval request payloads
- add `started_at_ms` / `completed_at_ms` to guardian review
notifications
- preserve Guardian review `started_at` / `completed_at` enrichment from
the protocol-native timing source
- stamp typed `ServerResponse` analytics facts with app-server-observed
`completed_at_ms`
- thread the new timing fields through core, protocol, app-server, TUI,
and analytics fixtures

## Verification

- `cargo test -p codex-app-server outgoing_message --manifest-path
codex-rs/Cargo.toml`
- `cargo test -p codex-app-server-protocol guardian --manifest-path
codex-rs/Cargo.toml`
- `cargo test -p codex-tui guardian --manifest-path codex-rs/Cargo.toml`
- `cargo test -p codex-analytics analytics_client_tests --manifest-path
codex-rs/Cargo.toml`

---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/21434).
* #18748
* __->__ #21434
* #18747
* #17090
* #17089
* #20514
2026-05-07 20:31:41 -07:00
alexsong-oai 3ad7cf0993 Add plugin ID to skill analytics (#20923)
## Summary
- thread plugin skill roots through the skills loader with their plugin
ID
- store plugin ID on loaded skill metadata for plugin-provided skills
- include plugin ID on skill invocation analytics events

## Test plan
- cargo check -p codex-core-skills
- cargo check -p codex-core -p codex-core-plugins -p codex-analytics
- cargo check -p codex-tui
- cargo check -p codex-plugin -p codex-core -p codex-core-plugins -p
codex-analytics
- cargo check -p codex-app-server
- cargo test -p codex-analytics
- HOME=/private/tmp/codex-empty-home cargo test -p codex-core-skills
- just fix -p codex-core-skills
- just fix -p codex-analytics
- just fix -p codex-core-plugins
- just fix -p codex-core
- just fmt
- git diff --check
2026-05-04 20:36:29 -07:00
rhan-oai 72a39e3a96 [app-server] centralize client response analytics (#20059)
## Why

The precursor PR keeps successful client responses typed until
app-server's outgoing response seam. This follow-up uses that seam to
move successful client-response analytics out of individual handlers and
into the shared sender path, while keeping filtering decisions inside
`codex-analytics`.

## What changed

- Emit successful client-response analytics centrally from
`OutgoingMessageSender::send_response`.
- Remove duplicate handler-local response tracking for the current
thread/turn lifecycle responses.
- Keep analytics ingestion selective inside `AnalyticsEventsClient`, so
unrelated client traffic is ignored before cloning or boxing.
- Collapse client-response analytics facts onto one typed path and
normalize payloads in the reducer.
- Add direct client-filter coverage plus sender-level coverage for the
centralized forwarding path.

## Verification

- `cargo test -p codex-analytics`
- `cargo test -p codex-app-server outgoing_message::tests --lib`
2026-04-29 21:22:39 +00:00
rhan-oai 0690ab0842 [codex-analytics] ingest server requests and responses (#17088)
## Why

Codex analytics needs a typed seam for app-server-originated
request/response traffic so future tool-approval analytics can consume
those facts without adding bespoke callsite tracking each time. Server
responses arrive as JSON-RPC `id + result` payloads, so analytics has to
reconstruct the matching typed response from the original typed request
while that request context still exists in app-server.

This also puts analytics on the app-server outbound path, which needs to
avoid keeping the runtime alive during shutdown. The final ownership fix
keeps the normal strong auth-manager retention in analytics and makes
the external-auth refresh bridge hold a weak back-reference to
`OutgoingMessageSender`, breaking the runtime cycle at the bridge
boundary instead of exposing retention policy through the analytics
client API.

## What changed

- Adds typed `ServerRequest` and `ServerResponse` analytics facts, plus
`AnalyticsEventsClient::track_server_request` and
`track_server_response`.
- Renames the existing client-side facts to `ClientRequest` and
`ClientResponse` so reducers can distinguish client-to-server traffic
from server-to-client traffic.
- Adds `ServerRequest::response_from_result`, allowing a stored typed
request to decode the matching typed server response from a raw JSON-RPC
result payload.
- Threads `AnalyticsEventsClient` through `OutgoingMessageSender` and
records targeted server requests, replayed targeted requests, and
matching targeted responses with the responding connection id needed for
correlation.
- Intentionally leaves broadcast server requests/responses out of
analytics for now because the current model is per connection, while
broadcasts fan one logical request out across multiple connections.
- Breaks the app-server shutdown cycle by storing
`Weak<OutgoingMessageSender>` in `ExternalAuthRefreshBridge` and
upgrading it only when an external-auth refresh is actually requested.
- Keeps reducer ingestion of the new server-side facts as no-ops for
now; this PR is plumbing for later tool-approval analytics work.

## Verification

- `cargo test -p codex-analytics`
- `cargo test -p codex-app-server outgoing_message::tests::`
- Covers typed-response reconstruction plus the targeted, replayed,
broadcast-exclusion, and response-attribution analytics paths.

## Follow-up

This PR intentionally stops at ingestion plumbing, so `ServerRequest`
and `ServerResponse` facts are still reducer no-ops. Once a follow-up PR
adds real downstream analytics output for those facts:

- replace the temporary pre-reducer observation seam with reducer tests
for the emitted event shape;
- add end-to-end coverage in `app-server/tests/suite/v2/analytics.rs`
for the real app-server workflow and captured analytics payload;
- remove the temporary sender-level observer tests added here in favor
of the real-output coverage above.

---

[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/17088).
* #18748
* #18747
* #17090
* #17089
* #20241
* #20239
* __->__ #17088
2026-04-29 19:56:41 +00:00
Michael Bolin dda8199b73 permissions: migrate approval and sandbox consumers to profiles (#19393)
## Why

Runtime decisions should not infer permissions from the lossy legacy
sandbox projection once `PermissionProfile` is available. In particular,
`Disabled` and `External` need to remain distinct, and managed profiles
with split filesystem or deny-read rules should not be collapsed before
approval, network, safety, or analytics code makes decisions.

## What Changed

- Changes managed network proxy setup and network approval logic to use
`PermissionProfile` when deciding whether a managed sandbox is active.
- Migrates patch safety, Guardian/user-shell approval paths, Landlock
helper setup, analytics sandbox classification, and selected
turn/session code to profile-backed permissions.
- Validates command-level profile overrides against the constrained
`PermissionProfile` rather than a strict `SandboxPolicy` round trip.
- Preserves configured deny-read restrictions when command profiles are
narrowed.
- Adds coverage for profile-backed trust, network proxy/approval
behavior, patch safety, analytics classification, and command-profile
narrowing.

## Verification

- `cargo test -p codex-core direct_write_roots`
- `cargo test -p codex-core runtime_roots_to_legacy_projection`
- `cargo test -p codex-app-server
requested_permissions_trust_project_uses_permission_profile_intent`




































































---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/19393).
* #19395
* #19394
* __->__ #19393
2026-04-26 15:30:40 -07:00
AbhinavandCodex 8720b7bdce Add codex_hook_run analytics event (#17996)
# Why
Add product analytics for hook handler executions so we can understand
which hooks are running, where they came from, and whether they
completed, failed, stopped, or blocked work.

# What
- add the new `codex_hook_run` analytics event and payload plumbing in
`codex-rs/analytics`
- emit hook-run analytics from the shared hook completion path in
`codex-rs/core`
- classify hook source from the loaded hook path as `system`, `user`,
`project`, or `unknown`

```
{
  "event_type": "codex_hook_run",
  "event_params": {
    "thread_id": "string",
    "turn_id": "string",
    "model_slug": "string",
    "hook_name": "string, // any HookEventName
    "hook_source": "system | user | project | unknown",
    "status": "completed | failed | stopped | blocked"
  }
}
```

---------

Co-authored-by: Codex <noreply@openai.com>
2026-04-16 19:43:16 +00:00
rhan-oai b704df85b8 [codex-analytics] feature plumbing and emittance (#16640)
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/16640).
* #16870
* #16706
* #16641
* __->__ #16640
2026-04-13 23:11:49 -07:00
Owen Lin 58933237cd feat(analytics): add guardian review event schema (#17055)
Just the analytics schema definition for guardian evaluations. No wiring
done yet.
2026-04-10 17:33:58 -07:00