Commit Graph
91 Commits
Author SHA1 Message Date
felixxia-oai a3ebd19fa4 Move explicit skill selection into the skills crate (#37177)
## What changed

- Add `ExplicitSkillLookup` to decouple explicit mention selection from the
  core skill-loading model.
- Export `collect_explicit_skill_mentions` from `codex-skills` and implement
  the lookup interface for `SkillLoadOutcome`.
- Move the selection tests into `codex-skills` while keeping the prompt-size
  boundary test with prompt injection.

GitOrigin-RevId: 21105af31c9b519c231c5ea02464da560705c293
2026-08-05 23:23:21 +00:00
felixxia-oai e3465b48ad Centralize skill invocation helpers in codex-skills (#37174)
## What changed

- Move tool mention parsing, skill-name counting, and implicit invocation detection into `codex-skills` and expose them through its public API.
- Decouple implicit invocation detection from `SkillLoadOutcome` with an `ImplicitSkillLookup` trait, while preserving the existing `core-skills` interface through re-exports.
- Cover remote plugin attribution for both implicit `SKILL.md` reads and skill script runs.

GitOrigin-RevId: f3da85c9821868638ccfd2fc2e01e0869d2fd437
2026-08-05 22:26:59 +00:00
felixxia-oai bd36d69aae Load host skill roots through the skills extension (#37162)
## What changed

- Load ordinary recursive host skill roots with the skills extension's host loader.
- Keep plugin-specific roots on the existing loader so plugin snapshot caching and namespacing continue to work.
- Merge both kinds of root snapshots through the existing precedence and deduplication logic, including discovery paths and source filesystem mappings.
- Move `HostSkillsSnapshot` into the skills extension as the owner of host skill loading.

## Testing

- Cover merged host and plugin catalogs, precedence for symlinked roots, discovery paths, and end-to-end rendering and invocation of host, repository, and plugin skills.

GitOrigin-RevId: c18080afb917c33c57bc17e36b40dd0f3c72d1d7
2026-08-05 21:17:54 +00:00
xl-openai 72d937ed4d Preserve discovery paths for symlinked skills (#37144)
## Why

Symlinked skills have a canonical `SKILL.md` path that can differ from the path
under the configured skill root. A catalog entry that advertises the discovery
path must still select the canonical skill.

## What changed

- Track each loaded skill's discovery path alongside its canonical identity.
- Render discovery paths in the skill catalog and accept either form for
  structured selections and linked skill mentions.
- Preserve enabled-state checks, deduplication, and product filtering when
  resolving discovery paths.

## Testing

Added loader, catalog, mention-resolution, and end-to-end turn coverage for
skills discovered through directory symlinks.

GitOrigin-RevId: bae539b766cc3782c7c121a4276c337699cbb509
2026-08-05 19:51:51 +00:00
jacobzhou-oai 56b82e676c Enforce Agent Plugin runtime boundaries (#37027)
## What changed

- Track Agent Plugin manifests through plugin, skill, and MCP loading so their capabilities use format-specific behavior without changing legacy plugins.
- Discover only direct-child skills, exclude app and hook capabilities, isolate MCP data, and reject MCP configuration files that are non-regular or resolve outside the plugin root.
- Bound model-visible skill instructions, plugin instructions, MCP descriptions, schemas, individual tools, and the aggregate Agent Plugin MCP tool set.
- Stop MCP and OAuth redirects when Agent Plugins send configured or authorization headers, while retaining existing redirect behavior for legacy MCP servers.

## Testing

- Add coverage for capability filtering, skill discovery, isolated MCP data and reserved-path expansion, unsafe MCP configuration files, context limits, and redirect handling.

GitOrigin-RevId: c9af66b051269f3226628ca280a58d32c808c38f
2026-08-05 04:54:33 +00:00
chess 5d89ab65dc Keep shared skill caches fresh across plugin loads (#37000)
## What changed

- Key cached skill snapshots by filesystem and plugin snapshot identity so compatible config and working-directory loads can share results without reusing stale plugin data.
- Coalesce concurrent loads for the same cache key and replace the cached entry on forced reload.
- Clear both plugin and skill caches when `skills/list` forces a reload, and bypass working-directory caching when effective plugin roots are present.

## Testing

- Cover concurrent cache sharing, filesystem isolation, forced reloads, and refreshed plugin skill metadata in host service and app server tests.

GitOrigin-RevId: 03fed3b40d45bb29206d5a3c3e78f06df04dbb1e
2026-08-05 01:24:35 +00:00
felixxia-oai 1a7519fa07 Move host skill root resolution into the skills extension (#36943)
## What changed

- Move config-layer, user, system, plugin, extra, and repository skill-root
  resolution from `core-skills` into the host skills extension.
- Keep the core loader focused on loading explicit `SkillRoot` values.
- Relocate and expand tests for root precedence, deduplication, repository
  ancestry, plugin metadata, and concurrent probing.

GitOrigin-RevId: 3b95cf28101b8b4d64d54079d202154dad560aab
2026-08-04 18:41:55 +00:00
felixxia-oai 02bc1dd796 Move the host skills service into the skills extension (#36921)
## What changed

- Move host skill discovery, snapshot caching, and configuration handling from
  `codex-core-skills` to `codex-skills-extension`.
- Rename `SkillsService` and `SkillsLoadInput` to `HostSkillsService` and
  `HostSkillsLoadInput` to distinguish the host implementation from other skill
  providers.
- Keep shared loading primitives and skill outcome modeling in
  `codex-core-skills`.

GitOrigin-RevId: d81a21791d2ef8c066e157e7e538b8cb7ee4c24b
2026-08-04 16:28:46 +00:00
felixxia-oai 2999b8e831 Move skill policy resolution into codex-skills (#36913)
## What changed

- Move `resolve_disabled_skill_paths` into `codex-skills` and expose it from the crate.
- Re-export the resolver from `core-skills` to preserve existing callers.
- Split config-layer and policy-resolution coverage into focused unit tests, including ordered path and name overrides.

GitOrigin-RevId: 6528ff99285925132bf80eb33eafb1b62118e033
2026-08-04 15:37:42 +00:00
felixxia-oai 49b0aebd6f Load skill interface metadata in the host loader (#36903)
## What changed

- Move skill interface validation and asset-path resolution into `codex-skills` so both skill loaders share the same behavior.
- Populate host-loaded skills with validated display metadata, prompts, brand colors, and local icons from `agents/openai.yaml`.
- Allow plugin skills to reference icons under the plugin's shared `assets` directory while rejecting absolute paths and paths that escape permitted asset roots.

## Testing

- Add resolver tests for valid, invalid, local, and plugin-shared interface assets.
- Add host loader coverage for interface fields and asset-path restrictions.

GitOrigin-RevId: 3f316191752eeebd15d11ac0bee82201574320ec
2026-08-04 14:51:08 +00:00
felixxia-oai 77ce1d10aa Move direct executor skill discovery into the skills extension (#36880)
## What changed

- Add extension-owned discovery and namespace resolution for skills loaded directly through an `ExecutorFileSystem`.
- Preserve hidden and symlinked skills, nested plugin namespaces, optional `agents/openai.yaml` metadata, product restrictions, and deterministic ordering.
- Reuse the filesystem walk inventory and bound concurrent skill, metadata, and manifest reads.
- Route direct executor catalog loading through the new extension loader.

## Testing

- Cover namespace lookup, metadata probing, walk reuse, concurrent reads, and parity with the existing environment loader.

GitOrigin-RevId: 4e0b821eb84d03f0dc1c2dee7b2b9a072ee3fd44
2026-08-04 11:04:45 +00:00
felixxia-oai 4c25d6cc5c Move executor skill bundle loading into the skills extension (#36877)
## What changed

- Move parsing of pre-discovered executor skill bundles from `core-skills` to
  the skills extension.
- Expose shared `SKILL.md` frontmatter parsing from `codex-skills` so direct
  and pre-discovered loading use the same validation and repair behavior.

## Testing

- Add parity coverage for direct and pre-discovered executor skill catalogs,
  including plugin namespaces, metadata, product policy, and warnings.
- Add a snapshot for the resulting pre-discovered executor catalog.

GitOrigin-RevId: c5f888226fa5600bd8b90f5682400da39a5db5ff
2026-08-04 10:41:01 +00:00
jif 78306a32af Clarify config layer iteration APIs (#36774)
## What changed

- Replace the ordering enum and `include_disabled` boolean with named
  `ConfigLayerStack` iterators for each precedence direction.
- Keep enabled-only iteration separate from `all_layers_*` iteration, which
  includes disabled layers.
- Update config consumers to use the iterator matching their precedence and
  disabled-layer requirements without allocating an intermediate `Vec`.

## Testing

- Add coverage that verifies ordering and disabled-layer filtering for all four
  iterators.

GitOrigin-RevId: d14df3db4a2eae80ba97cdec246bff405e6c5f3d
2026-08-03 17:29:42 +00:00
felixxia-oai 0d109f097c Move skill catalog rendering out of core (#36364)
## What changed

- Make the skills extension own the catalog prompt templates and rendering path.
- Remove the duplicate core fallback that injected available skills into initial context.
- Update core integration tests to install the skills extension explicitly when they expect model-visible skill context.

GitOrigin-RevId: bbe6b15c4e776a09c98b7f4166426fe4e58434ba
2026-07-31 17:12:10 +00:00
felixxia-oai 66d63afd18 Preserve discovery order for host skill path aliases (#36327)
## Why

Host skill path aliases should follow the core skill loader's root discovery
order, independent of how catalog entries are sorted for rendering.

## What changed

- Propagate each host skill root's discovery position into its catalog entries.
- Assign `r0`, `r1`, and subsequent aliases from that position while leaving
  entries without an explicit root order last.

## Testing

Added coverage for alias ordering in extension rendering and across consecutive
production turns.

GitOrigin-RevId: 9085a3843c91bf2ce78a378cf5ce4926e9c38eb8
2026-07-31 13:47:46 +00:00
mandal-OAI ba42e6866c Label explicit skill injection metrics (#36191)
## Why

The `codex.skill.injected` counter needs to distinguish explicitly requested
skills from implicitly invoked skills.

## What changed

Add the `invoke_type=explicit` attribute to skill injection metrics emitted for
explicit skill mentions, including both successful and failed injections.

GitOrigin-RevId: 22023f8d628be0f22b23f5376b668e8d5492c14a
2026-07-30 17:14:40 +00:00
jif 5decb399ae Respect filesystem permissions during capability discovery (#36124)
## Why

Executor capability discovery can traverse plugin and skill roots, including
symlinks. Under restricted filesystem permissions, discovery must not expose
files outside the permitted paths.

## What changed

- Pass each environment's filesystem sandbox context through capability root
  discovery and apply it to metadata, directory walks, and file reads.
- Enable discovery for restricted sessions so permitted executor skills remain
  available while inaccessible roots and symlink targets are omitted.
- Key discovery caches by sandbox context and reject sandboxed discovery on
  executors that do not advertise support for it.
- Split requests with more than 128 roots into supported-size batches.

## Testing

- Cover permitted and denied external symlink targets.
- Verify restricted skill listing excludes inaccessible skills.
- Verify cache separation across permission contexts and discovery of 129 roots.

GitOrigin-RevId: 44d16468ca003403bdb8b71a04ae8c9ff94ed494
2026-07-30 10:01:06 +00:00
jameswt-oai 0d2a0aa76b Track remote plugin IDs in skill invocation analytics (#35262)
## What changed

- Add `remote_plugin_id` to skill invocation facts and analytics event parameters.
- Propagate the ID for both explicit and implicit plugin skill invocations.

## Testing

- Cover explicit and implicit remote plugin skill invocations with analytics integration tests.

GitOrigin-RevId: 3e1d25b5ff88dcff9c2b6c9fe44bca97e8778221
2026-07-24 23:40:14 +00:00
jameswt-oai 07fd04abb1 Propagate remote plugin IDs to skill metadata (#35261)
## What changed

- Carry a plugin's local and remote identities together from plugin loading into
  `SkillMetadata`.
- Resolve remote IDs from the installed-plugin snapshot when available, falling
  back to persisted install metadata only when no snapshot exists.
- Include plugin identity in skill cache keys so identity changes refresh cached
  skill metadata.

## Testing

- Cover snapshot and persisted identity resolution, local marketplace isolation,
  cached skill refreshes, and propagation through plugin skill snapshots.

GitOrigin-RevId: aabeeb631a43361fe817358ace7f1ea8ba5db708
2026-07-24 23:36:11 +00:00
jif 7c71783135 Expose executor skills through skill tools (#35184)
## What changed

- Add executor authority support to `skills.list` and `skills.read` for skills from selected capability roots.
- Allow `skills.read` to load package-relative resources referenced by an executor skill while keeping reads within the selected package.
- Paginate skill listings and resource contents with bounded response sizes and stale-cursor validation.
- Resolve executor filesystems per sampling step so skill tools use the active selected capability roots.

## Testing

- Extend the app-server executor skill test to list an executor skill, read its main `SKILL.md`, and read a paginated reference resource.
- Cover visibility filtering, metadata bounds, pagination, invalid cursors, package boundaries, and resource size limits.

GitOrigin-RevId: 1b5de95d5d1b9f79c34d2879d6e67f24401e9c29
2026-07-24 13:46:20 +00:00
felixxia-oai 5a1c54fc21 Compact host skill paths under metadata pressure (#35172)
## Why

Repeated absolute path prefixes consume the bounded skill catalog and can leave
less room for descriptions or later skills.

## What changed

- Track the discovery root for host skills and render shared roots as `r0`,
  `r1`, and similar aliases when doing so improves the bounded catalog.
- Include a skill-roots table and alias-aware usage instructions whenever
  compact paths are selected.
- Keep absolute, authority-aware locators when aliases provide no benefit or
  the catalog contains non-host skills.

## Testing

Added unit and extension coverage for unpressured catalogs, constrained shared
paths, mixed-source catalogs, and plugin-cache root selection.

GitOrigin-RevId: 4196df2c52be27338796065cb2e6434efcfac6d5
2026-07-24 12:58:41 +00:00
jacobzhou-oai a28374e0db Support Agent Plugins manifests (#35105)
## What changed

- Recognize root `plugin.json` files using the Agent Plugins 1.0 schema and map their portable metadata, `skills/`, and `mcp.json` into Codex plugin manifests.
- Apply Codex-specific apps, hooks, and interface settings from the inline `com.openai` extension, with `.codex-plugin/plugin.json` as a fallback overlay.
- Preserve legacy manifest precedence when a root `plugin.json` is unrelated, and reject unsupported Agent Plugins schema versions.
- Add a direct-child skill discovery mode that excludes nested skills and paths resolving outside the plugin root.

## Testing

- Cover manifest metadata, validation, extension precedence, legacy fallback, and direct-child skill path boundaries.

GitOrigin-RevId: eab24139f13a5cc5cb3ad3fb444d8e904511aca6
2026-07-24 05:59:16 +00:00
Michael Bolin 539c0e1100 Migrate login HTTP construction to HttpClient (#34643)
## Why

`codex-http-client` is the intended owner of direct `reqwest` usage.

## What changed

- Build Codex's default and route-aware clients with `HttpClientBuilder`, and return `HttpClient` directly to API, model discovery, auth, remote control, skills, and extension callers.
- Preserve custom CA, proxy, Cloudflare cookie, sandbox, and request-logging policies while removing `codex-login`'s direct `reqwest` dependency.
- Use shared HTTP types for auth headers, status codes, transport errors, and login-server tests.

## Testing

- Add a Linux integration test verifying that a regular Responses API turn traverses the configured system proxy.

GitOrigin-RevId: 714695fb80762d2083e2f7a6414183e2b3b63854
2026-07-22 00:50:28 +00:00
mandal-OAI 6c00dc087e Sanitize skill names in injection metrics (#34601)
## Why

Skill names may contain characters that are not valid in metric tag values.

## What changed

Pass skill names through `sanitize_metric_tag_value` before using them as the
`skill` tag on `codex.skill.injected` counters for both explicit and implicit
skill invocations.

GitOrigin-RevId: 0ed633fbd468739acc30193b566d830e20bc7860
2026-07-21 19:25:39 +00:00
felixxia-oai 56c11cf658 Move shared skill models into codex-skills (#34429)
## What changed

- Define and export skill metadata, policy, dependency, interface, and configuration rule types from `codex-skills`.
- Update core, plugin, and extension consumers to use the shared models while retaining compatibility re-exports from `codex-core-skills`.
- Share product-restriction handling between host and environment skill metadata.

## Testing

- Add unit coverage for default and explicit implicit-invocation and product policies.

GitOrigin-RevId: 9093469f8fff8e2d902dc9781235f64709827a87
2026-07-20 23:22:48 +00:00
Bryan Ashley 08e30a2e4e Add batched executor capability discovery (#33852)
## Why

Selected capability roots can contribute plugins, MCP servers, connectors, and
skills. Discovering each contribution separately requires repeated access to the
executor filesystem.

## What changed

- Add the `capabilityRoots/discoverV1` exec-server RPC to scan selected roots and
  materialize recognized plugin manifests, configuration files, skill
  instructions, and skill metadata in one bounded request.
- Add the opt-in `executor_capability_discovery` feature, with a thread-scoped
  cache and per-step snapshot shared by MCP and skill discovery.
- Parse MCP, connector, and skill contributions from the materialized snapshot,
  including serving cached skill instructions without another filesystem read.

## Testing

- Cover discovery limits, manifest precedence, root-local failures, cache reuse,
  plugin contributions, and parity with the existing environment skill loader.

GitOrigin-RevId: f98fd2321cafb58c596db02da1f83c09d8eb375d
2026-07-17 15:45:48 +00:00
jif 0f2321f244 Refresh host skill catalogs through world state (#33425)
## Why

Host skills can change after a thread starts. Subsequent turns need the updated
catalog without reinjecting an unchanged catalog on every turn.

## What changed

- Register the host skill provider in app-server and project its catalog through
  a dedicated World State section.
- Render host skills with the existing metadata budget and usage instructions,
  including skill-root aliases when available.
- Suppress the legacy host catalog and contextual fragment when World State owns
  the catalog, while preserving host skills for explicit and shadow selection.

## Testing

Added an app-server test that installs a host skill at runtime and verifies the
catalog updates once and remains stable on the following turn.

GitOrigin-RevId: 4cacc6b2258fc3cab8499fbdf9aa6ab0324ec10d
2026-07-15 20:59:21 +00:00
Adam Perry @ OpenAI 2e4f55608b Scan skill roots concurrently (#33369)
## What changed

- Scan up to eight skill roots concurrently instead of loading them one at a time.
- Share scan capacity across host-skill and plugin-skill loads to bound aggregate filesystem work.
- Merge completed scans in input order so root precedence and error ordering remain deterministic.
- Skip canonicalization and parsing work for empty roots.

## Testing

- Add coverage for out-of-order scan completion and shared-capacity blocking.

GitOrigin-RevId: 33a47aaa02aee7fe31a050bb04d51c1e2043d6ae
2026-07-15 15:56:53 +00:00
jif 0d1733b5e9 Test bounded concurrency in ancestor discovery (#32126)
## What changed

Add regression coverage that verifies ancestor metadata searches:

- keep at most 256 project-root marker or repository skill-root probes in flight;
- start the next probe as capacity becomes available; and
- preserve `AGENTS.md` and repository skill-root discovery order.

GitOrigin-RevId: e7ac90bbeb94573d01e67705001af85ce59165b6
2026-07-10 10:29:30 +00:00
jif 6ad0e943cc Reduce startup latency for ancestor discovery (#32106)
## Why

Serial metadata probes add remote round trips while Codex discovers project
instructions, repository skills, and project-root markers during startup.

## What changed

- Probe ancestor directories concurrently while preserving discovery order.
- Raise the shared find-up probe window to cover typical project hierarchies.
- Refresh project instructions in parallel with plugin and skill warmup.

## Testing

Add coverage that blocks one metadata request and verifies all expected project-root
and `AGENTS.md` ancestor probes have started before it is released.

GitOrigin-RevId: 11449837c7a212ec8ed385e7290476e926d93531
2026-07-10 09:39:08 +00:00
jif 0d18d4f753 test(skills): assert symlinked metadata loading (#31756)
## Why

The host walk-inventory test writes policy metadata for a symlinked
skill, but only asserted the loaded names. It could still pass if
metadata loading regressed.

## What changed

- Compare the complete loaded skill metadata.
- Verify the first skill keeps `allow_implicit_invocation: false` while
the sibling has no policy.

Stacked on #31566.
2026-07-09 10:12:30 +01:00
jif e8eb60092a perf(skills): reuse walk inventory for host loading (#31566)
## Why

CCA thread startup loads repository skills through the primary remote
environment. This still used the older host skill loader, which
recursively issued directory and metadata requests over the exec-server
connection. On a high-latency connection, one skill scan could turn into
hundreds of round trips.

Executor-selected skill roots already avoid this pattern: they use one
bounded filesystem walk, reuse the returned inventory, and overlap
independent reads.

## What changed

- Extract the URI-native walk inventory into a shared private discovery
module.
- Use that discovery path for both environment-owned and host/repository
skill roots.
- Reuse the walk inventory to avoid per-skill metadata existence checks
when the inventory is complete.
- Canonicalize host skill identities and parse skill files with bounded
concurrency.
- Prune hidden host directories during traversal, preserving visible
aliases and walk limits.
- Cache namespace probes by ancestor and keep all remote namespace reads
under the same concurrency bound.
- Keep safe metadata probes for incomplete walks, file symlinks, and
case aliases.

## Dependency

#31570 is now merged. It provides the optional `pruneHiddenDirectories`
walk flag used here. That flag defaults to false and is omitted when
disabled.

No other exec-server API changes are introduced.
2026-07-09 09:54:24 +01:00
jif a52b35fcf6 fs: support pruning hidden directories during walks (#31570)
Why

Filtering hidden directories after a walk is too late: their descendants
consume traversal limits, and canonical directory deduplication can let
a hidden path claim a target before a visible symlink reaches it.

What this changes

- Add an optional pruneHiddenDirectories walk option. It defaults to
false and is omitted from the wire when disabled.
- Return hidden directory entries, but do not traverse them or add their
canonical identities to the visited set.
- Cover the visible-symlink-to-hidden-directory case through both local
and remote filesystem implementations.

This is the small filesystem prerequisite for #31566. Skill-specific
behavior remains in that PR.
2026-07-08 13:49:05 +01:00
Adam Perry @ OpenAI 58b66d39e4 perf(skills): resolve plugin namespaces per root (#31348)
## Why

Loading skills from a remote executor can add a lot to thread start time
when there are many skills. Previous changes added some concurrency for
the file reads themselves, but we're still bottlenecked on the initial
root path discovery.

Using the benchmark from
[#31295](https://github.com/openai/codex/pull/31295), this change
reduces the measured mean of loading 66 skills about 71%. Behavioral
coverage lands separately in
[#31369](https://github.com/openai/codex/pull/31369).

## What

- resolve the scanned root's inherited namespace once
- resolve discovered nested plugin roots once, retaining
nearest-valid-ancestor behavior
- pass an explicit resolved Plain / Plugin namespace into skill parsing
instead of probing per skill
- preserve explicitly provided plugin namespaces as the highest-priority
source
- reuse the same resolver for environment skills, deleting its duplicate
root-probe and ancestor-selection path

## Validation

- just test -p codex-core-skills namespace: 12 passed on both the parent
and optimized branches
2026-07-07 23:00:15 +00:00
Adam Perry @ OpenAI 42156ba007 test(skills): cover plugin namespace loading (#31369)
## Why

Before changing plugin namespace loading performance, lock down the
existing behavior so the same cases can be validated before and after
the optimization.

## What

- cover mixed plain and nested plugin skills under one scan root
- cover inherited, nested, and invalid-manifest namespace precedence
- cover symlinked plugin skill directories, symlinked plain directories,
and scan-root ancestor symlinks

## Validation

- just test -p codex-core-skills namespace on unmodified main: 12 passed
2026-07-07 09:28:52 -07:00
Adam Perry @ OpenAI a86d525e4d core: trace executor skill discovery (#30318)
## Why

Make it easier to measure the performance of different parts of skill
loading.

## What

- Add spans for step-context capture, world-state construction, executor
catalog snapshot/root loading, and environment skill loading.
- Record the discovered environment skill count.
- Trace outbound exec-server requests with client kind and RPC method
fields.
- Update trace propagation tests to assert that requests keep the parent
trace id while creating their own child span.
2026-07-06 18:44:08 +00:00
ani-oai 6b5f5743b3 [codex] Use model metadata for skills usage instructions (#29740)
## Summary

- add a false-by-default `include_skills_usage_instructions` model
metadata field
- enable the field for the bundled `gpt-5.5` model metadata
- consume the metadata in both core and extension skill rendering
- remove hardcoded legacy-model matching and its marker plumbing
2026-06-29 09:44:36 +09:00
jif a938d5f607 Overlap executor skill reads with namespace discovery (#30225)
## Why

Environment skill discovery needs two independent pieces of information:

- plugin namespaces from `plugin.json` files; and
- skill metadata from each `SKILL.md` file.

Today these happen in sequence. Codex waits for every plugin namespace
lookup to finish before it starts reading any skill files. On a remote
executor, that creates an avoidable network-latency barrier.

```text
before: walk -> namespace lookups -> skill reads -> build catalog
after:  walk -> namespace lookups ─┐
             -> skill reads ───────┴-> build catalog
```

## What changes

- Read and parse skill files without waiting for plugin namespace
discovery.
- Resolve root and nested plugin namespaces concurrently.
- Join both results only when constructing the final qualified skill
names.
- Keep the existing 64-skill concurrency bound, output ordering,
warnings, metadata behavior, and namespace rules.

## Testing

The regression test makes plugin manifest lookup wait until a `SKILL.md`
read has started. The old serialized pipeline would time out; the new
pipeline completes and still returns the correctly namespaced skill.

`just test -p codex-core-skills` passes all 111 tests.

## Out of scope

This does not add an exec-server endpoint, batch filesystem calls, or
reduce the number of files transferred. A frontmatter-only read or
server-side skill catalog can remain a separate follow-up if benchmarks
show that transferred bytes are the next bottleneck.
2026-06-26 18:37:59 +00:00
Ahmed Ibrahim 914c8eeb4e [codex] narrow unused skills intro export (#29991)
## Summary

- stop publicly re-exporting the internally used
`SKILLS_INTRO_WITH_ALIASES` constant
- keep the constant and all skills rendering behavior unchanged
- preserve every integration helper, API, fixture, assertion, and module
used by tests

## Scope guardrails

This revision keeps all remote/network-facing functionality and every
line introduced by `jif <jif@openai.com>`.

Following the test-preservation audit, it also restores the in-process
RMCP test transport, the original `codex-mcp` fixture,
`PluginLoadOutcome::effective_skill_roots` and its assertions, the
`EffectiveSkillRoots` API family, the test-only apps renderer, and the
TUI dead-code annotation. Those files now match the PR base exactly.

No test imports or directly references the remaining public skills
export being narrowed.

## Validation

- repository-wide test-reference audit: no test-used code remains
deleted or narrowed
- deleted-line `git blame` audit: zero Jif-authored deletions
- `cargo test -p codex-core-plugins -p codex-mcp -p codex-rmcp-client
--lib`: 467 passed
- `cargo test -p codex-core --lib apps::render`: 2 passed
- `cargo test -p codex-core-skills --lib render::tests`: 19 passed
- `cargo check -p codex-core-skills --all-targets`: passed
- `just fix -p codex-core-skills`: passed
- `just fmt`: passed
- `git diff --check`: passed

The full local `codex-core-skills` suite passed 106/108 tests; two
loader tests detected an ambient repository skills root outside the
package and failed their isolation assertions. The scoped renderer suite
and all-target compile pass, and CI runs in an isolated environment.

Final code delta: 1 insertion, 2 deletions across 2 files.
2026-06-26 05:52:04 -07:00
jif 8ebf71ec25 Reuse walk inventory for environment skill metadata (#30145)
## Why

Environment skill discovery already asks the executor to run one
`fs/walk`. That response contains every regular file path found under
the selected root, including any `agents/openai.yaml` files.

Today Core keeps the discovered `SKILL.md` paths but discards the rest
of that file inventory. It then sends one `fs/getMetadata` request per
skill just to ask whether `agents/openai.yaml` exists. A root with 66
skills and no metadata therefore pays for 66 unnecessary network round
trips.

## What changes

- Keep the `fs/walk` file and directory inventory for the duration of
the scan.
- Associate each discovered `SKILL.md` with metadata that is known
present, known absent, or still requires a fallback probe.
- Read a known `agents/openai.yaml` directly instead of statting it
first.
- Skip the metadata request entirely when a complete walk shows that the
skill has no `agents` directory.
- Read a known `SKILL.md` and `agents/openai.yaml` concurrently.
- Keep parsing and validation in `core-skills`.

The inventory is scan-local. This does not add another cache or change
cache lifetime.

## Network impact

For a complete scan of 66 valid skills with no `agents/openai.yaml`, and
one root `.codex-plugin/plugin.json`:

| Operation | Current | After this PR |
| --- | ---: | ---: |
| `fs/walk` | 1 | 1 |
| Read `SKILL.md` | 66 | 66 |
| Stat `agents/openai.yaml` | 66 | 0 |
| Read `agents/openai.yaml` | 0 | 0 |
| Stat plugin manifest | 1 | 1 |
| Read plugin manifest | 1 | 1 |
| **Total executor RPCs** | **135** | **69** |

This removes exactly 66 request/response exchanges from the common cold
scan. Warm scans remain at zero discovery RPCs because the thread-level
executor catalog cache is unchanged.

When metadata exists, each file still requires one read. This PR removes
only the preceding existence check; it does not batch file contents into
a new RPC.

## Correctness fallbacks

Absence is trusted only when the walk is complete and the metadata
directory was not present. Core keeps the existing `getMetadata`
fallback when:

- the walk was truncated;
- the walk reported an error; or
- an `agents` directory was observed but `openai.yaml` was not, which
preserves support for file symlinks and traversal boundaries.

## Deliberate scope

This PR changes only the environment skill loader and its existing
filesystem-call regression coverage. It does not:

- change `fs/walk` or any exec-server protocol;
- add `readFiles` or a skills-list endpoint;
- change thread caching;
- change local skill discovery;
- change exec-server request concurrency; or
- optimize plugin-manifest lookup.

The plugin-manifest stat is intentionally left in place, which is why
this PR reaches 69 calls rather than the broader 68-call estimate. That
lookup has separate alternate-path, ancestor, and symlink semantics and
should not be mixed into this change.
2026-06-26 01:47:00 +01:00
Adam Perry @ OpenAI 5579792b3b Parallelize environment skill loading (#29990)
## Why

Avoid a request waterfall for loading lots of skills at once by hiding
latency in concurrent tasks.

## What changed

Poll the per-skill parse futures concurrently with an order-preserving
stream capped at 64 in-flight loads. Results retain discovery order, and
the existing filtering, warnings, and final catalog sorting are
unchanged.
2026-06-25 10:02:07 +01:00
jif 96d8e34712 Follow directory symlinks in filesystem walks (#29844)
Stack 3 of 3. Stacked on #29842.

## What changes

Adds an opt-in `followDirectorySymlinks` setting to `fs/walk`.

When enabled, the walk follows directory symlinks but continues to
ignore symlinked files. Canonical directory identities prevent symlink
cycles, while normal paths keep their existing spelling.

Environment skill discovery enables the setting so symlinked skill
directories continue to work with the new single-RPC scan.
2026-06-24 20:52:36 +01:00
jif 8a6a34be75 Fix environment skill discovery after merge (#29887)
## Why

The merge of #29831 with the new `fs/walk` environment discovery path
left three `SkillFileDiscovery` initializers without the new namespace
fields. This makes `codex-core-skills` fail to compile and breaks CI for
every PR based on current `main`.

## What changed

- collect plugin roots from the directory entries already returned by
`fs/walk`
- keep the selected root as the namespace fallback
- initialize empty discovery results with empty namespace sets

This preserves the bounded `fs/walk` implementation while restoring the
namespace caching added by #29831.
2026-06-24 19:08:39 +01:00
jif 390b73133b Cache plugin namespace during executor skill discovery (#29831)
## Why

Executor skill discovery runs before the remote skills catalog is
available. For a remote environment, each `ExecutorFileSystem` operation
becomes an exec-server RPC.

Previously, every discovered `SKILL.md` independently resolved its
plugin namespace by walking its ancestors and probing both supported
manifest locations. In the common `plugin/skills/<skill>/SKILL.md`
layout, that repeats 8 RPCs per skill even though every skill under the
plugin root uses the same namespace. These lookups happen while skills
are parsed, so their cost grows linearly with the skill count and adds
directly to first-turn latency.

A selected capability root can also contain standalone skills, multiple
sibling plugins, nested plugins, or symlinked directories. The
optimization therefore needs to retain the nearest-ancestor namespace
for each skill rather than assuming the selected root represents exactly
one plugin.

## What changed

- record plugin-root candidates from directory entries already returned
during skill discovery
- prune candidates that are not ancestors of any discovered `SKILL.md`
before reading manifests
- resolve each relevant plugin root once, with one fallback lookup per
canonical traversal root for symlinked directories
- select the nearest cached plugin namespace for each discovered skill
- avoid namespace lookup entirely when the root contains no skills

No additional directory traversal is required. Namespace work now scales
with the number of plugin roots that contain discovered skills, rather
than the total number of skills or unrelated sibling plugins. Standalone
and nested-plugin names keep their previous behavior.

## Benchmarks

I used a temporary counting `ExecutorFileSystem` around the real local
filesystem. Each filesystem operation was counted as one remote RPC and
given 1 ms of injected latency. Each variant ran three times; times
below are medians.

### One plugin with 100 skills

| Operation | Before | After | Delta |
| --- | ---: | ---: | ---: |
| `get_metadata` | 1,002 | 303 | -699 |
| `read_file` | 200 | 101 | -99 |
| `read_directory` | 102 | 102 | 0 |
| **Total filesystem RPCs** | **1,304** | **506** | **-798 (-61.2%)** |
| **Median load time** | **2.890 s** | **0.997 s** | **2.90× faster** |

The namespace-specific work drops from 800 RPCs to 2 in this layout.

### Multiple plugins under one selected root

These runs compare the correct pre-optimization implementation with the
final nearest-plugin-root cache. The total plugin skill count stays at
100 while the number of plugin roots changes.

| Layout | Before RPCs | After RPCs | Reduction | Before | After |
Speedup |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| 2 plugins × 50 skills | 1,312 | 530 | 59.6% | 1,819 ms | 711 ms |
2.56× |
| 10 plugins × 10 skills | 1,344 | 578 | 57.0% | 1,850 ms | 778 ms |
2.38× |
| 50 plugins × 2 skills | 1,504 | 818 | 45.6% | 2,094 ms | 1,086 ms |
1.93× |
| 10 plugins × 10 skills + 10 standalone skills | 1,596 | 630 | 60.5% |
2,209 ms | 860 ms | 2.57× |

The remaining cost grows with the number of relevant plugin manifests.
Each relevant manifest is read once instead of once per skill, while
sibling plugins with no discovered skills are not read. Absolute latency
savings depend on the executor's real RPC latency.

## Tests

- `just test -p codex-core-skills` (109 passed across the library and
integration-test binaries)
- one integration test covers standalone, outer-plugin, nested-plugin,
and unused sibling-plugin layouts, and asserts the exact set of
manifests read
2026-06-24 17:14:34 +01:00
jif 69b76e9d07 Use fs/walk for environment skill discovery (#29842)
Stack 2 of 3. Base: #29841. Follow-up: #29844.

## What changes

Environment skill discovery currently walks remote filesystems through
repeated `readDirectory` and `getMetadata` calls. This switches that
scan to the bounded `fs/walk` operation from the base PR.

```text
Before: readDirectory(root) -> getMetadata(...) -> readDirectory(child) -> ...
After:  fs/walk(root, limits) -> filter the result for SKILL.md
```

This makes environment skill discovery one RPC while preserving
traversal warnings and the existing depth and directory limits. The scan
also has an explicit entry limit. The follow-up restores
directory-symlink traversal.
2026-06-24 16:32:35 +01:00
alexsong-oai f959e7fc98 [codex] Emit implicit skill usage for support reads (#29731)
## Summary
- Index all enabled skills for command-based usage detection, regardless
of `allow_implicit_invocation`.
- Preserve `allow_implicit_invocation` for the model-visible implicit
routing list.
- Add regression coverage for a support/preflight skill whose `SKILL.md`
is read and whose script is run while implicit invocation is disabled.

## Root cause
`allow_implicit_invocation` was used for both model routing and
command-based usage-event detection. That meant support skills like
`data-analytics:user-context` could be read or run by other skills, but
those accesses could not emit implicit usage events.

## Validation
- `just fmt`
- `just test -p codex-core-skills
service::tests::skills_for_config_indexes_usage_detection_for_non_implicit_skills`
- `just test -p codex-core-skills` now has the new test passing, but 3
unrelated local tests fail because
`/Users/alexsong/.agents/skills/test/SKILL.md` is invalid/missing YAML
frontmatter.
2026-06-24 08:57:34 +00:00
Adam Perry @ OpenAI 1d65ccabd5 config: own layer provenance types (#29722)
## Why

Config layer provenance describes how effective configuration was
assembled, so it belongs with the config loader rather than in
app-server's serialized API types.

## What changed

- Moved `ConfigLayerSource`, `ConfigLayerMetadata`, and `ConfigLayer`
ownership into `codex-config`.
- Kept app-server's wire payloads unchanged and added explicit
conversions at the app boundary.
- Removed lower-level app-server-protocol dependencies from config
consumers.

## Stack

This is PR 3 of 6, stacked on [PR
#29721](https://github.com/openai/codex/pull/29721). Review only the
delta from `codex/split-auth-domain-types`. Next: [PR
#29723](https://github.com/openai/codex/pull/29723).

## Validation

- `codex-config` coverage passed.
- App-server config-manager and config RPC coverage passed.
2026-06-24 04:03:04 +00:00
jif 220f5b76b2 Load executor skills without host path conversion (#29626)
## Why

After #28918, selected skill roots are `PathUri`, but the executor skill
provider still converts them to the app-server host's `AbsolutePathBuf`.
A foreign Windows root therefore cannot be discovered by a Unix host,
and the inverse has the same problem.

This PR keeps executor skill discovery and reads on the filesystem that
owns the selected root while reusing the existing skill rules.

## What changed

- Generalize the existing skill traversal to operate on `PathUri`
through `ExecutorFileSystem`, preserving its depth, directory, symlink,
and sibling-metadata concurrency behavior.
- Add a small environment skill loader that reuses the shared discovery,
frontmatter validation, dependency parsing, product policy, and
prompt-visibility rules.
- Keep the environment id and entrypoint `PathUri` in the skill catalog,
then route `skills.read` back through the same environment filesystem.
- Preserve the executor's path convention when deriving catalog handles,
including literal backslashes in POSIX filenames.
- Resolve plugin namespaces from nearby manifests through URI-native
filesystem reads.
- Cover foreign Windows roots, executor-owned reads, namespaces,
metadata, policy, and path identity.

```text
selected root (PathUri)
        |
        v
shared discovery over ExecutorFileSystem
        |
        v
environment-bound catalog entry --skills.read--> same ExecutorFileSystem
```

No second filesystem abstraction or duplicate traversal implementation
is introduced.

## Stack

1. #29614 — add lexical `PathUri` containment.
2. #29620 — share URI-native manifest path resolution.
3. #28918 — keep selected plugin roots and resources URI-native.
4. **This PR** — load executor skills without host path conversion.
5. #29628 — resolve executor MCP working directories without host path
conversion.
2026-06-23 23:26:06 +01:00
jif c2fbf4247a Parallelize skill metadata stats (#29326)
## Summary

This switches skill discovery to the simpler same-connection scalar
request shape.

After reading a skills directory, discovery now starts the existing
`fs/getMetadata` calls for all visible entries in that directory before
awaiting the results. There is no JSON-RPC batch frame and no new
filesystem API; remote filesystems use the existing request-id
multiplexing on the same exec-server connection.

This is the scoped alternative to the batch-frame approach in #29074 /
#29075.

## What changed

- Collect visible directory entries before processing them.
- Run their existing `fs.get_metadata(...)` calls with `join_all`.
- Process the results in the original directory order, so skill
discovery behavior stays the same.

## Benchmarks

Fresh local benchmark against generated skill trees over a real
exec-server remote filesystem. The benchmark calls the actual
`load_skills_from_roots` path, so this includes directory reads,
metadata stats, `SKILL.md` reads, and parsing.

Times are p50 milliseconds from 5 samples after 1 warmup, using warmed
runs.

| Scenario | Legacy `main` | Batch frame stack (#29074 / #29075) |
Same-connection scalar stack |
| --- | ---: | ---: | ---: |
| 100 flat skills | 377.4 | 389.0 | 378.6 |
| 500 flat skills | 1983.2 | 1856.6 | 1757.5 |

Takeaway: for the actual skill discovery path, same-connection scalar is
tied with legacy at 100 skills and best at 500 skills. The batch-frame
stack does not show enough win here to justify the extra protocol/API
surface.

Benchmark command:

- `just test -p codex-exec-server benchmark_remote_skill_discovery
--run-ignored ignored-only --no-capture`

Checked locally with:

- `just test -p codex-core-skills`
- `just bazel-lock-update`
- `just bazel-lock-check`
2026-06-21 14:04:18 +02:00
charlesgong-openai 64bdeed9f7 [codex] Preserve skill descriptions outside model context (#29006)
## Why

Skill descriptions are used in model-visible lists: the default
available-skills catalog that supports implicit selection, and the
on-demand `skills.list` tool response used to discover orchestrator
skills. A single overlong description should not consume a
disproportionate share of either list.

Enforcing the 1024-character limit while loading or migrating skills is
the wrong boundary: it rejects otherwise-valid skills and discards
metadata that non-model consumers and full skill reads may need. Skill
metadata and `SKILL.md` content should remain intact; the cap belongs at
model-visible list rendering boundaries.

## What changed

- Preserve full `description` and `metadata.short-description` values
when loading skills.
- Preserve full external-agent command descriptions during
`source-command-*` migration instead of skipping commands solely because
their descriptions exceed 1024 characters.
- Preserve full normalized orchestrator descriptions in the underlying
skills catalog.
- Cap each description at 1024 Unicode characters when rendering the
default available-skills context in `codex-core-skills` and
`codex-skills-extension`.
- Apply the same cap when serializing descriptions in the model-visible
`skills.list` response.
- Render truncated descriptions as 1021 original characters plus `...`.
- Leave explicit `$skill` injection, `skills.read`, underlying metadata,
and on-disk `SKILL.md` files unchanged and full-fidelity.

## Implicit skill selection

Codex injects a bounded catalog containing each implicitly allowed
skill's name, description, and source locator, together with
instructions to use a skill when the task clearly matches its
description. The model makes that semantic choice; after selecting a
skill, it reads the full `SKILL.md` from its filesystem or provider
resource. Explicit `$skill` mentions remain a separate path that injects
the full skill instructions. For orchestrator skills, `skills.list`
provides bounded discovery metadata before `skills.read` returns the
full selected resource.

## Test plan

- `just test -p codex-core-skills`
- `just test -p codex-skills-extension`
- `just test -p codex-external-agent-migration`

The focused regressions verify that overlong metadata is preserved at
load and migration boundaries while default available-skills rendering
and `skills.list` output produce the 1021-character prefix plus `...`.
2026-06-19 12:47:53 -07:00