Commit Graph
10 Commits
Author SHA1 Message Date
Kyle Brown 9579479d28 Collect plugin metrics from remote executors (#38283)
## What changed

- Resolve manifest-declared metric operations against the executor filesystem for remote plugin commands.
- Create the measurement sidecar in an executor-native, owner-private temporary directory, stream its bounded output back for validation, and clean up the directory afterward.
- Extend the exec-server protocol with the executor temporary directory and atomic private-directory creation.

## Testing

- Cover remote unified-exec measurements for foreground and background commands.
- Verify private directories use owner-only permissions on Unix and fail closed on unsupported platforms.

GitOrigin-RevId: dbbd0a84717b91237fc5728e510e18994eb46dd4
2026-08-13 00:39:04 +00:00
Kyle Brown c38d59fca0 Add app-server coverage for plugin measurement analytics (#38278)
## What changed

- Add end-to-end tests for curated plugin measurements through classic shell execution and unified exec.
- Cover unified background commands whose measurements arrive after the turn completes.
- Verify command attribution and measurement payloads, including values, dimensions, execution IDs, and thread, turn, and item IDs.

GitOrigin-RevId: 866103aafef8d3454123c40aee7041912ad0b7fc
2026-08-13 00:08:57 +00:00
Kyle Brown 96e8afbfb8 Track plugin metrics for background unified exec commands (#38276)
## Why

Unified exec can yield while a command is still running. Plugin measurement
collection must remain active until that background command exits, including
when its item completion arrives after the turn has completed.

## What changed

- Keep the plugin metrics sidecar with the stored process and let either the
  exit watcher or a poll that observes completion finalize it exactly once.
- Retain completed turn analytics state while tool items are pending so late
  command completion events can be emitted without duplicating the turn event.

## Testing

- Verify that a background command completed after its turn emits a command
  execution event and does not emit a second turn event.

GitOrigin-RevId: ecf715b3e047aa29ca9a417d12d955257fed8557
2026-08-13 00:04:57 +00:00
Kyle Brown 6e7daed1e9 Collect plugin metrics from unified exec commands (#38253)
## What changed

- Create a metrics sidecar for attributed local plugin commands launched through unified exec, and grant the sandbox access to its output file.
- Publish valid measurements when the initial command exits, while discarding the sidecar when the process remains live for later polling.
- Strip inherited or configured `CODEX_PLUGIN_METRICS_OUTPUT` values so each command can only use its own sidecar.

GitOrigin-RevId: 43e669e4c77171aec8a798de0c12ca13bfb8adee
2026-08-12 21:27:30 +00:00
Kyle Brown 9ca0337dbf Collect metrics from plugin shell commands (#38252)
## What changed

- Provide matching local plugin commands with a sandbox-writable temporary output file through `CODEX_PLUGIN_METRICS_OUTPUT` when analytics is enabled.
- Validate successful command output against the plugin's `analytics.yaml` declaration, including measurement names, enum dimensions, finite values, duplicate rows, and size limits, before publishing analytics events.
- Keep the output path reserved from user overrides and clean up the temporary file after execution.

## Testing

- Cover output validation, limits, cleanup, sandbox permissions, environment handling, and path replacement.
- Verify measurement collection through both classic and zsh-fork shell runtimes.

GitOrigin-RevId: 88af0f87dc2f207fcbcca6af498f5c940d79349d
2026-08-12 21:23:21 +00:00
Kyle Brown d6eefb26a6 Add bounded plugin measurement analytics (#38239)
## What changed

- Add `PluginMeasurementsInput` and `PluginMeasurementRow`, plus an analytics
  client method for recording batches and checking whether analytics are enabled.
- Emit each valid row as a `codex_plugin_measurement_event` with thread, turn,
  item, plugin, execution, and operation context.
- Bound batches at 100 rows and reject non-finite values, invalid identifiers,
  and rows with more than eight dimensions.
- Treat measurement events as plugin events when routing API-key analytics.

## Testing

- Cover batch limits, row filtering, serialized event payloads, and API-key
  plugin-event routing.

GitOrigin-RevId: f949ed06246ee1e22466df2660076f2d4f556444
2026-08-12 19:14:09 +00:00
Kyle Brown dc8562d672 Add manifest-defined metrics for trusted plugin scripts (#38238)
## What changed

- Load version 1 `analytics.yaml` manifests from trusted plugin roots and map declared operations, measurements, and enum dimensions to exact script paths.
- Expose resolution types and bind a declared operation to the plugin identity returned by fresh command attribution.
- Reject malformed, oversized, ambiguous, or unsafe manifests without disabling normal script attribution.

## Testing

- Cover exact script resolution, measurement names shared across operations, and invalid manifests including duplicate keys, path traversal, symlink escapes, invalid identifiers, and oversized files.

GitOrigin-RevId: 1e2f221b9f2c3d7faffe578c7a8499ad4ed933ca
2026-08-12 19:09:48 +00:00
Kyle Brown 9fc4e5a7aa Preserve plugin attribution across command approvals (#35029)
## What changed

- Add optional `plugin_id` and `script_path` fields to execution approval and guardian assessment events.
- Propagate validated plugin attribution through delegated approvals, guardian-reviewed command items, app-server notifications, thread history, and rollout traces.
- Preserve attribution on both started and completed command items, including declined commands.

## Testing

- Extend core, app-server, thread-history, and rollout-trace tests to cover plugin attribution propagation and serialization.

GitOrigin-RevId: 723684d010cab04142918d7a95e06ed95d008da1
2026-07-23 23:13:59 +00:00
Kyle Brown 84fa68b429 Attribute command executions to trusted plugin scripts (#35020)
## What changed

- Resolve shell and unified-exec commands against the trusted plugin roots loaded for each turn.
- Add optional `pluginId` and safe plugin-relative `scriptPath` fields to command execution items and legacy execution events, and propagate them through app-server notifications.
- Include the attribution in command execution analytics while rejecting absolute, unsafe, and unattributed script paths.

## Testing

- Cover attribution for cached curated and remote plugin scripts from command execution through core and app-server events.
- Verify analytics serialization and unsafe-path filtering.

GitOrigin-RevId: 02fac3a233284ccfc6642fa502a95f1881dba83d
2026-07-23 22:31:53 +00:00
Kyle Brown 5bdbd3ee90 Add trusted plugin script attribution (#35016)
## What changed

- Build a set of active, verified curated and remote plugin roots from loaded plugins.
- Resolve direct and safely wrapped script commands to a plugin ID and normalized plugin-relative path.
- Leave complex or ambiguous commands, local overrides, missing files, overlapping roots, and symlink escapes unattributed.
- Add a shared validator for the safe cross-platform shape of serialized plugin-relative paths.

## Testing

- Cover trusted-root selection, supported interpreters and shell wrappers, normalized paths, and fail-closed cases.

GitOrigin-RevId: 6e4199a241fd6dfadfec3df0845e7cb615352a49
2026-07-23 21:50:17 +00:00