Commit Graph
18 Commits
Author SHA1 Message Date
Sean Huang 13869ca39e Add portable project trust lookup APIs (#47620)
## What changed

Add `resolve_root_git_project_uri_for_trust` to resolve repository trust roots through `ExecutorFileSystem` using executor path conventions. Share repository and linked-worktree validation with native lookup while preserving native ancestry for paths with opaque URI representations.

Extract `ProjectTrustLookup` to match merged project settings using supplied path spellings and an explicit path convention. Preserve canonical-before-original and cwd-before-repository-root precedence, including cwd entries without a trust level, with ASCII case-insensitive matching for Windows.

## Testing

Add coverage for POSIX, Windows drive and UNC trust roots, project-key precedence and case handling, and native/URI lookup parity. Extend environment-config coverage to verify returned `projects` entries include entries without a trust level.

GitOrigin-RevId: 315434802e86cceee85b14bc1c3767a45e3139f5
2026-09-23 18:40:12 +00:00
Adam Perry @ OpenAI 1bc02aea50 Sanitize credentials from Git remote metadata (#40713)
## Why

Git remote URLs can embed usernames, passwords, or tokens. Codex carries these
URLs through turn metadata and persisted thread metadata, so credentials must be
removed before a remote enters those paths.

## What changed

- Add `SanitizedGitUrl`, which parses Git URL and SCP-style remotes, strips
  authentication data, and preserves the conventional `git` SSH user.
- Use sanitized remotes when collecting Git metadata, enriching model requests,
  discovering cloud environments, updating thread metadata, and reading legacy
  rollouts. Reject or omit malformed remotes instead of retaining their raw
  contents.
- Keep API, schema, and TypeScript representations as strings while enforcing
  sanitization in Rust.

## Testing

- Cover URL schemes, SCP and IPv6 forms, remote helpers, encoded paths,
  malformed values, and legacy deserialization.
- Verify credentials do not appear in model requests, API responses, SQLite, or
  rollout files.

GitOrigin-RevId: 6435efc4c45bfbfad4723ce7a0457cb00175f25c
2026-08-25 21:34:05 +00:00
cgst-oai a26d50852a Require filesystem backends to implement directory walks (#39749)
## What changed

- Make `ExecutorFileSystem::walk` a required backend operation instead of
  providing a fallback built from directory reads and metadata requests.
- Implement bounded local walks on a blocking task with cancellation, symlink
  cycle detection, deterministic ordering, error collection, and response-size
  limits.
- Have remote filesystems use the server's walk operation directly.

## Testing

- Cover local and remote handling of invalid roots and limits, directory
  symlinks, non-UTF-8 names, cancellation, sandbox contexts, and response
  budgets.

GitOrigin-RevId: 7499bf05080c3f9965a5eb7ffd593de604d62c2a
2026-08-20 17:16:52 +00:00
pakrym-oai e3e5ad2847 Harden unsandboxed patch filesystem access (#39659)
## Why

An `apply_patch` path can be replaced with a symlink after verification, allowing an unsandboxed patch operation to reach a different file than the one that was approved.

## What changed

- Add `follow_symlinks` options to executor filesystem reads, writes, metadata lookups, directory creation, and removal, including the corresponding `followSymlinks` protocol fields.
- Implement no-follow filesystem operations on Unix and Windows that reject links in any path component and restrict file access to regular files.
- Run `apply_patch` with symlink traversal disabled when an otherwise-required sandbox is bypassed, while retaining the existing follow-symlink default for standalone callers.

## Testing

- Cover leaf and ancestor symlinks across patch add, update, delete, and move operations, including a path swap after verification.
- Exercise local and remote no-follow filesystem behavior, concurrent directory creation, special-file rejection, and Windows reparse points.

GitOrigin-RevId: 43fd479084891493ce13564fbd894b98f329c6dd
2026-08-20 08:10:08 +00:00
Eric Traut 9ca99b5171 Preserve parent repository discovery through sandbox metadata mounts (#39629)
## Why

The Linux sandbox represents missing protected metadata paths such as `.git`
with empty read-only directories. Repository and project discovery treated any
`.git` directory as a checkout root, so this synthetic directory could hide a
real parent repository and its trust configuration.

## What changed

- Treat a `.git` directory as repository metadata only when it contains
  `HEAD`, while continuing to recognize file-based `.git` entries.
- Protect missing `.git` paths with the same read-only synthetic mounts as
  other workspace metadata without disrupting parent-repository discovery.
- Canonicalize and read-only bind the synthetic-mount registry into the
  sandbox, and make protected-path cleanup handle read-only directory trees.

## Testing

Added coverage for repository, project-root, trust, synthetic-mount, and
Landlock behavior, including nested incomplete `.git` directories and a
redirected `TMPDIR`.

GitOrigin-RevId: 0724c54d56531143bb28011e4ca414cd3b0212be
2026-08-20 06:57:00 +00:00
Jeremy Rose bc3545b805 Validate linked worktrees before inheriting project trust (#39616)
## Why

A checkout could point its `.git` file at a trusted repository's worktree
directory without proving that the repository had registered that checkout.
This could cause project configuration from an unrelated checkout to be treated
as trusted.

## What changed

- Verify the linked worktree's `gitdir` backlink, `commondir`, registered
  checkout, and main checkout ownership before resolving the main repository's
  trust key.
- Reject missing, oversized, symlinked, mismatched, or swapped Git metadata.
- Preserve valid linked worktrees that use path aliases, separate Git
  directories, or non-UTF-8 POSIX paths.

## Testing

Add resolver and config-loading coverage for forged worktrees, metadata races,
case-sensitive paths, moved worktrees, and host MCP startup from project config.

GitOrigin-RevId: 6052a7d10ad2d613436f20175c356abdef8c758e
2026-08-20 06:29:32 +00:00
Adam Hupp b6c3b51533 Coalesce concurrent Git status scans (#37151)
## What changed

- Share an in-flight `git status --porcelain` invocation among concurrent workspace metadata requests for the same canonical repository root.
- Keep scans for different repositories independent, and start a fresh scan after an in-flight request completes or loses all consumers.
- Pass the known repository root into dirty-worktree detection so requests from sibling directories and symlink aliases use the same key.

## Testing

- Add coverage for request coalescing, repository and symlink keying, completed scans, and consumer cancellation.

GitOrigin-RevId: 4c197c1d5bb94ca83c02979be34b17f24aefde08
2026-08-05 20:23:35 +00:00
Adam Perry @ OpenAI 1823c13771 Render turn diffs for foreign environment paths (#34654)
## Why

Turn diffs need to render paths from remote environments whose path convention may differ from the Codex host.

## What changed

- Preserve `PathUri` values through applied patch deltas, display-root discovery, and turn-diff tracking.
- Add host-independent relative-path rendering and normalize Windows drive-letter case in `PathUri`.
- Render paths relative to each environment's repository root while keeping Git diff headers slash-separated.

## Testing

- Cover POSIX, Windows, UNC, encoded-separator, and opaque URI relative paths.
- Verify an applied patch in a remote Windows working directory emits a portable unified diff.

GitOrigin-RevId: 526f70ae36a40627de5496d375310d12295b69bb
2026-07-22 02:19:45 +00:00
jif 39aab9fc45 Pipeline bounded AGENTS.md and Git root probes (#29870)
## Why

When Codex uses a remote `ExecutorFileSystem`, every `get_metadata` call
is an exec-server round trip. Upward discovery currently pays those
round trips serially in two latency-sensitive places:

- session startup, while locating the configured project root before
loading `AGENTS.md`; and
- Git-root discovery, which runs before per-turn Git diff enrichment.

The goal is to remove the serial ancestor dependency without adding a
new filesystem RPC, JSON-RPC batch method, Git executable dependency, or
cache.

## Example

Assume this layout, with `.git` as the configured project-root marker:

```text
/workspace/repo/.git
/workspace/repo/AGENTS.md
/workspace/repo/crates/core/    <- cwd
```

The marker probes have this required precedence:

```text
1. /workspace/repo/crates/core/.git
2. /workspace/repo/crates/.git
3. /workspace/repo/.git
4. /workspace/.git
5. /.git
```

Previously, probe 2 was not sent until probe 1 returned, and probe 3 was
not sent until probe 2 returned. With this change, the client lazily
keeps up to eight ordinary `fs/getMetadata` requests in flight, but
consumes their results in the order above. Codex must still learn that
probes 1 and 2 are absent before accepting probe 3, so the nearest root
always wins. Once probe 3 succeeds, the client has its answer and stops
awaiting probes 4 and 5. Requests that were already sent may still
finish on the worker.

For the marker phase alone, with a 50 ms client-to-worker round trip and
fast local metadata calls, finding the root at probe 3 changes from
roughly three serialized round trips (150 ms) to one round trip plus
worker processing. The later `AGENTS.md` candidate phase remains
separate and ordered.

Only after `/workspace/repo` is selected does `AGENTS.md` discovery
check instruction candidates, in root-to-cwd order:

```text
/workspace/repo/AGENTS.override.md
/workspace/repo/AGENTS.md
/workspace/repo/crates/AGENTS.override.md
/workspace/repo/crates/AGENTS.md
/workspace/repo/crates/core/AGENTS.override.md
/workspace/repo/crates/core/AGENTS.md
```

The first configured candidate found in each directory wins. These
checks remain ordered and no instruction candidate above
`/workspace/repo` is issued. Git-root discovery uses the same bounded
lookup with only `.git` as the marker.

## What changed

- Added a client-side find-up helper that generates `ancestor x marker`
probes lazily, nearest directory first and configured marker order
within each directory.
- Uses an ordered concurrency window of eight scalar metadata requests.
This bounds executor load while preserving nearest-root and marker
precedence.
- Reuses the helper for both configured project-root discovery and
remote Git-root discovery.
- Keeps Git ancestor and marker construction in `AbsolutePathBuf`,
converting only each complete `.git` probe to `PathUri`. This preserves
native paths that require an opaque URI fallback, such as Windows
namespace paths.
- Preserves existing error behavior: `AGENTS.md` discovery propagates
non-`NotFound` metadata errors, while Git discovery treats a failed
marker probe as absent and continues upward.
- Reads each discovered `AGENTS.md` directly instead of statting it a
second time.

No filesystem trait or exec-server protocol method is added. An empty
`project_root_markers` list performs no ancestor-marker I/O and checks
instruction candidates only in `cwd`. This change also deliberately does
not cache roots across turns.

## Symlinks

Upward traversal remains **lexical**. The helper does not canonicalize
`cwd`; it appends marker names to the supplied path and walks that
path's textual parents. The filesystem performs the actual metadata/read
operation, and the current local and exec-server implementations follow
live symlink targets.

For example:

```text
/tmp/pkg -> /workspace/repo/packages/pkg
cwd = /tmp/pkg/src
actual Git marker = /workspace/repo/.git
```

The lexical probes are `/tmp/pkg/src/.git`, `/tmp/pkg/.git`,
`/tmp/.git`, and `/.git`. They do not jump from `/tmp/pkg` to the
target's parent `/workspace/repo`, so this spelling of `cwd` does not
discover `/workspace/repo/.git`. That is the existing behavior and is
unchanged by this PR.

Conversely, if `/tmp/repo -> /workspace/repo`, then probing
`/tmp/repo/.git` follows the directory symlink and finds
`/workspace/repo/.git`; the reported root remains the lexical path
`/tmp/repo`. A live symlink used directly as `.git`, another configured
marker, or `AGENTS.md` is also followed. A symlinked `AGENTS.md` is
loaded when its target is a regular file, while a broken symlink behaves
as `NotFound`.
2026-06-24 22:58:34 +01:00
Tamir DubersteinandChris Bookholt dffc4bf75d [codex] preserve fsmonitor for worktree Git reads (#26880)
Codex forces `core.fsmonitor=false` on internal Git commands so a
repository cannot select an executable fsmonitor helper. This also
disables Git's built-in daemon for `status`, `diff`, and `ls-files`,
turning those worktree reads into full scans in large repositories.

Read the raw effective `core.fsmonitor` value and preserve it only when
Git interprets it as true and advertises built-in daemon support through
`git version --build-options`. Query uncommon boolean spellings back
through Git using the exact effective value. Unset, false, helper paths,
malformed values, probe failures, and unsupported Git builds continue to
force `core.fsmonitor=false`.

Centralize this policy in `git-utils` while keeping process execution in
the existing local and workspace-command adapters. Probe once per
worktree workflow and reuse the result for its Git commands, including
the TUI `/diff` path. Metadata-only commands and repository discovery
remain disabled without probing. Each probe and requested Git process
keeps its own existing timeout, and the decision is not cached because
layered and conditional Git configuration can change while Codex runs.

---------

Co-authored-by: Chris Bookholt <bookholt@openai.com>
2026-06-08 21:32:46 -07:00
starr-openai 9286ff2805 Fix remote turn diff display roots (#23261)
## Why

`TurnDiffTracker` computes a display root so turn diffs can be rendered
repo-relative. For remote exec-server turns, the selected turn `cwd` may
exist only inside the selected environment, but `run_turn` was
discovering the git root through the local host filesystem. When that
lookup failed, nested remote-session diffs fell back to the nested `cwd`
and showed `/tmp/...`-prefixed paths instead of repo-relative paths.

## What changed

- Resolve the diff display root from the primary selected turn
environment when one exists, using that environment's filesystem and
`cwd`.
- Add `codex_git_utils::get_git_repo_root_with_fs(...)` so git-root
discovery can run against an `ExecutorFileSystem`, including remote
environments.
- Reuse that helper from `resolve_root_git_project_for_trust(...)` and
add coverage for `.git` gitdir-pointer detection.

## Validation

- Devbox Bazel: `//codex-rs/core:core-unit-tests
--test_filter=get_git_repo_root_with_fs_detects_gitdir_pointer`
- Devbox Docker-backed remote-env repro: `//codex-rs/core:core-all-test
--test_filter=apply_patch_turn_diff_paths_stay_repo_relative_when_session_cwd_is_nested`
2026-05-18 10:53:49 -07:00
Chris Bookholt 9facdccb37 Ignore configured hooks in git helpers (#22843)
## What
- Internal Git helper commands now ignore configured hook directories
during repository bookkeeping.

## Why
- These helper flows should stay consistent even when a repository has
hook-directory configuration of its own.

## How
- Pass a command-local `core.hooksPath` override in the shared helper
path and the Git-info helper path.
- Add regressions for the baseline index rewrite flow and the metadata
status flow.

## Validation
- `cargo fmt --manifest-path
/Users/bookholt/code/codex/codex-rs/Cargo.toml --all --check`
- `cargo test --manifest-path
/Users/bookholt/code/codex/codex-rs/Cargo.toml -p codex-git-utils`
- `cargo test --manifest-path
/Users/bookholt/code/codex/codex-rs/Cargo.toml -p codex-core
test_get_has_changes_`
2026-05-15 10:07:54 -07:00
Chris Bookholt 6ec8c4a6ec [codex] Ignore fsmonitor config in Git metadata reads (#22652)
## Summary
- keep Git metadata/status subprocesses independent of repository
`core.fsmonitor` configuration
- preserve existing working-tree state reporting while making the helper
behavior more predictable
- add regression coverage for `get_has_changes` when a repository
defines an fsmonitor command

## Validation
- `cargo fmt --all`
- `cargo test -p codex-core test_get_has_changes_`
- `cargo test -p codex-git-utils`
2026-05-14 10:07:43 -07:00
pakrym-oai 9effa0509f Refactor config loading to use filesystem abstraction (#18209)
Initial pass propagating FileSystem through config loading.
2026-04-17 00:51:21 +00:00
pakrym-oai bd61737e8a Async config loading (#18022)
Parts of config will come from executor. Prepare for that by making
config loading methods async.
2026-04-15 19:18:38 -07:00
Michael Bolin 61dfe0b86c chore: clean up argument-comment lint and roll out all-target CI on macOS (#16054)
## Why

`argument-comment-lint` was green in CI even though the repo still had
many uncommented literal arguments. The main gap was target coverage:
the repo wrapper did not force Cargo to inspect test-only call sites, so
examples like the `latest_session_lookup_params(true, ...)` tests in
`codex-rs/tui_app_server/src/lib.rs` never entered the blocking CI path.

This change cleans up the existing backlog, makes the default repo lint
path cover all Cargo targets, and starts rolling that stricter CI
enforcement out on the platform where it is currently validated.

## What changed

- mechanically fixed existing `argument-comment-lint` violations across
the `codex-rs` workspace, including tests, examples, and benches
- updated `tools/argument-comment-lint/run-prebuilt-linter.sh` and
`tools/argument-comment-lint/run.sh` so non-`--fix` runs default to
`--all-targets` unless the caller explicitly narrows the target set
- fixed both wrappers so forwarded cargo arguments after `--` are
preserved with a single separator
- documented the new default behavior in
`tools/argument-comment-lint/README.md`
- updated `rust-ci` so the macOS lint lane keeps the plain wrapper
invocation and therefore enforces `--all-targets`, while Linux and
Windows temporarily pass `-- --lib --bins`

That temporary CI split keeps the stricter all-targets check where it is
already cleaned up, while leaving room to finish the remaining Linux-
and Windows-specific target-gated cleanup before enabling
`--all-targets` on those runners. The Linux and Windows failures on the
intermediate revision were caused by the wrapper forwarding bug, not by
additional lint findings in those lanes.

## Validation

- `bash -n tools/argument-comment-lint/run.sh`
- `bash -n tools/argument-comment-lint/run-prebuilt-linter.sh`
- shell-level wrapper forwarding check for `-- --lib --bins`
- shell-level wrapper forwarding check for `-- --tests`
- `just argument-comment-lint`
- `cargo test` in `tools/argument-comment-lint`
- `cargo test -p codex-terminal-detection`

## Follow-up

- Clean up remaining Linux-only target-gated callsites, then switch the
Linux lint lane back to the plain wrapper invocation.
- Clean up remaining Windows-only target-gated callsites, then switch
the Windows lint lane back to the plain wrapper invocation.
2026-03-27 19:00:44 -07:00
0f957a93cd Move git utilities into a dedicated crate (#15564)
- create `codex-git-utils` and move the shared git helpers into it with
file moves preserved for diff readability
- move the `GitInfo` helpers out of `core` so stacked rollout work can
depend on the shared crate without carrying its own git info module

---------

Co-authored-by: Ahmed Ibrahim <219906144+aibrahim-oai@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-03-24 13:26:23 -07:00
Michael Bolin 0c8a36676a fix: move inline codex-rs/core unit tests into sibling files (#14444)
## Why
PR #13783 moved the `codex.rs` unit tests into `codex_tests.rs`. This
applies the same extraction pattern across the rest of `codex-rs/core`
so the production modules stay focused on runtime code instead of large
inline test blocks.

Keeping the tests in sibling files also makes follow-up edits easier to
review because product changes no longer have to share a file with
hundreds or thousands of lines of test scaffolding.

## What changed
- replaced each inline `mod tests { ... }` in `codex-rs/core/src/**`
with a path-based module declaration
- moved each extracted unit test module into a sibling `*_tests.rs`
file, using `mod_tests.rs` for `mod.rs` modules
- preserved the existing `cfg(...)` guards and module-local structure so
the refactor remains structural rather than behavioral

## Testing
- `cargo test -p codex-core --lib` (`1653 passed; 0 failed; 5 ignored`)
- `just fix -p codex-core`
- `cargo fmt --check`
- `cargo shear`
2026-03-12 08:16:36 -07:00