Commit Graph
10426 Commits
Author SHA1 Message Date
Michael Bolin 23c5e3dec9 Merge a667068f8d into sapling-pr-archive-bolinfest 2026-03-27 20:37:25 -07:00
Michael Bolin a667068f8d ci: add Windows Bazel clippy lane 2026-03-27 20:37:07 -07:00
Michael Bolin 343d1af3da bazel: enable the full Windows gnullvm CI path (#15952)
## Why

This PR is the current, consolidated follow-up to the earlier Windows
Bazel attempt in #11229. The goal is no longer just to get a tiny
Windows smoke job limping along: it is to make the ordinary Bazel CI
path usable on `windows-latest` for `x86_64-pc-windows-gnullvm`, with
the same broad `//...` test shape that macOS and Linux already use.

The earlier smoke-list version of this work was useful as a foothold,
but it was not a good long-term landing point. Windows Bazel kept
surfacing real issues outside that allowlist:

- GitHub's Windows runner exposed runfiles-manifest bugs such as
`FINDSTR: Cannot open D:MANIFEST`, which broke Bazel test launchers even
when the manifest file existed.
- `rules_rs`, `rules_rust`, LLVM extraction, and Abseil still needed
`windows-gnullvm`-specific fixes for our hermetic toolchain.
- the V8 path needed more work than just turning the Windows matrix
entry back on: `rusty_v8` does not ship Windows GNU artifacts in the
same shape we need, and Bazel's in-tree V8 build needed a set of Windows
GNU portability fixes.

Windows performance pressure also pushed this toward a full solution
instead of a permanent smoke suite. During this investigation we hit
targets such as `//codex-rs/shell-command:shell-command-unit-tests` that
were much more expensive on Windows because they repeatedly spawn real
PowerShell parsers (see #16057 for one concrete example of that
pressure). That made it much more valuable to get the real Windows Bazel
path working than to keep iterating on a narrowly curated subset.

The net result is that this PR now aims for the same CI contract on
Windows that we already expect elsewhere: keep standalone
`//third_party/v8:all` out of the ordinary Bazel lane, but allow V8
consumers under `//codex-rs/...` to build and test transitively through
`//...`.

## What Changed

### CI and workflow wiring

- re-enable the `windows-latest` / `x86_64-pc-windows-gnullvm` Bazel
matrix entry in `.github/workflows/bazel.yml`
- move the Windows Bazel output root to `D:\b` and enable `git config
--global core.longpaths true` in
`.github/actions/setup-bazel-ci/action.yml`
- keep the ordinary Bazel target set on Windows aligned with macOS and
Linux by running `//...` while excluding only standalone
`//third_party/v8:all` targets from the normal lane

### Toolchain and module support for `windows-gnullvm`

- patch `rules_rs` so `windows-gnullvm` is modeled as a distinct Windows
exec/toolchain platform instead of collapsing into the generic Windows
shape
- patch `rules_rust` build-script environment handling so llvm-mingw
build-script probes do not inherit unsupported `-fstack-protector*`
flags
- patch the LLVM module archive so it extracts cleanly on Windows and
provides the MinGW libraries this toolchain needs
- patch Abseil so its thread-local identity path matches the hermetic
`windows-gnullvm` toolchain instead of taking an incompatible MinGW
pthread path
- keep both MSVC and GNU Windows targets in the generated Cargo metadata
because the current V8 release-asset story still uses MSVC-shaped names
in some places while the Bazel build targets the GNU ABI

### Windows test-launch and binary-behavior fixes

- update `workspace_root_test_launcher.bat.tpl` to read the runfiles
manifest directly instead of shelling out to `findstr`, which was the
source of the `D:MANIFEST` failures on the GitHub Windows runner
- thread a larger Windows GNU stack reserve through `defs.bzl` so
Bazel-built binaries that pull in V8 behave correctly both under normal
builds and under `bazel test`
- remove the no-longer-needed Windows bootstrap sh-toolchain override
from `.bazelrc`

### V8 / `rusty_v8` Windows GNU support

- export and apply the new Windows GNU patch set from
`patches/BUILD.bazel` / `MODULE.bazel`
- patch the V8 module/rules/source layers so the in-tree V8 build can
produce Windows GNU archives under Bazel
- teach `third_party/v8/BUILD.bazel` to build Windows GNU static
archives in-tree instead of aliasing them to the MSVC prebuilts
- reuse the Linux release binding for the experimental Windows GNU path
where `rusty_v8` does not currently publish a Windows GNU binding
artifact

## Testing

- the primary end-to-end validation for this work is the `Bazel`
workflow plus `v8-canary`, since the hard parts are Windows-specific and
depend on real GitHub runner behavior
- before consolidation back onto this PR, the same net change passed the
full Bazel matrix in [run
23675590471](https://github.com/openai/codex/actions/runs/23675590471)
and passed `v8-canary` in [run
23675590453](https://github.com/openai/codex/actions/runs/23675590453)
- those successful runs included the `windows-latest` /
`x86_64-pc-windows-gnullvm` Bazel job with the ordinary `//...` path,
not the earlier Windows smoke allowlist

---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/15952).
* #16067
* __->__ #15952
2026-03-27 20:37:03 -07:00
Michael Bolin c4f8eb1fbe Merge cff12b0135 into sapling-pr-archive-bolinfest 2026-03-27 20:14:37 -07:00
Michael Bolin cff12b0135 ci: add Windows Bazel clippy lane 2026-03-27 20:14:28 -07:00
Michael Bolin 2619ef6673 merge commit for archive created by Sapling 2026-03-27 19:59:21 -07:00
Michael Bolin bebe223453 ci: enable the Windows Bazel workflow path 2026-03-27 19:59:02 -07:00
Michael Bolin 6c0983aa57 Merge d2e77009da into sapling-pr-archive-bolinfest 2026-03-27 19:58:17 -07:00
Michael Bolin d2e77009da stabilize zsh-fork approvals and resume --last 2026-03-27 19:58:12 -07:00
Michael Bolin adcf9eb053 Merge d6f44bfef0 into sapling-pr-archive-bolinfest 2026-03-27 19:56:11 -07:00
Michael Bolin d6f44bfef0 ci: enable the Windows Bazel workflow path 2026-03-27 19:56:07 -07:00
Michael Bolin cc4af08413 Merge a55041425f into sapling-pr-archive-bolinfest 2026-03-27 19:50:34 -07:00
Michael Bolin a55041425f ci: enable the Windows Bazel workflow path 2026-03-27 19:50:18 -07:00
Michael Bolin 5037a2d199 refactor: rewrite argument-comment lint wrappers in Python (#16063)
## Why

The `argument-comment-lint` entrypoints had grown into two shell
wrappers with duplicated parsing, environment setup, and Cargo
forwarding logic. The recent `--` separator regression was a good
example of the problem: the behavior was subtle, easy to break, and hard
to verify.

This change rewrites those wrappers in Python so the control flow is
easier to follow, the shared behavior lives in one place, and the tricky
argument/defaulting paths have direct test coverage.

## What changed

- replaced `tools/argument-comment-lint/run.sh` and
`tools/argument-comment-lint/run-prebuilt-linter.sh` with Python
entrypoints: `run.py` and `run-prebuilt-linter.py`
- moved shared wrapper behavior into
`tools/argument-comment-lint/wrapper_common.py`, including:
  - splitting lint args from forwarded Cargo args after `--`
- defaulting repo runs to `--manifest-path codex-rs/Cargo.toml
--workspace --no-deps`
- defaulting non-`--fix` runs to `--all-targets` unless the caller
explicitly narrows the target set
  - setting repo defaults for `DYLINT_RUSTFLAGS` and `CARGO_INCREMENTAL`
- kept the prebuilt wrapper thin: it still just resolves the packaged
DotSlash entrypoint, keeps `rustup` shims first on `PATH`, infers
`RUSTUP_HOME` when needed, and then launches the packaged `cargo-dylint`
path
- updated `justfile`, `rust-ci.yml`, and
`tools/argument-comment-lint/README.md` to use the Python entrypoints
- updated `rust-ci` so the package job runs Python syntax checks plus
the new wrapper unit tests, and the OS-specific lint jobs invoke the
wrappers through an explicit Python interpreter

This is a follow-up to #16054: it keeps the current lint semantics while
making the wrapper logic maintainable enough to iterate on safely.

## Validation

- `python3 -m py_compile tools/argument-comment-lint/wrapper_common.py
tools/argument-comment-lint/run.py
tools/argument-comment-lint/run-prebuilt-linter.py
tools/argument-comment-lint/test_wrapper_common.py`
- `python3 -m unittest discover -s tools/argument-comment-lint -p
'test_*.py'`
- `python3 ./tools/argument-comment-lint/run-prebuilt-linter.py -p
codex-terminal-detection -- --lib`
- `python3 ./tools/argument-comment-lint/run.py -p
codex-terminal-detection -- --lib`
2026-03-27 19:42:30 -07:00
Michael Bolin 71f7664099 Merge 77137c412d into sapling-pr-archive-bolinfest 2026-03-27 19:35:10 -07:00
Michael Bolin 77137c412d wip: experiment with windows bazel v8 build 2026-03-27 19:34:32 -07:00
Michael Bolin ada573c8c1 ci: enable the Windows Bazel workflow path 2026-03-27 19:34:32 -07:00
Michael Bolin 142681ef93 shell-command: reuse a PowerShell parser process on Windows (#16057)
## Why

`//codex-rs/shell-command:shell-command-unit-tests` became a real
bottleneck in the Windows Bazel lane because repeated calls to
`is_safe_command_windows()` were starting a fresh PowerShell parser
process for every `powershell.exe -Command ...` assertion.

PR #16056 was motivated by that same bottleneck, but its test-only
shortcut was the wrong layer to optimize because it weakened the
end-to-end guarantee that our runtime path really asks PowerShell to
parse the command the way we expect.

This PR attacks the actual cost center instead: it keeps the real
PowerShell parser in the loop, but turns that parser into a long-lived
helper process so both tests and the runtime safe-command path can reuse
it across many requests.

## What Changed

- add `shell-command/src/command_safety/powershell_parser.rs`, which
keeps one mutex-protected parser process per PowerShell executable path
and speaks a simple JSON-over-stdio request/response protocol
- turn `shell-command/src/command_safety/powershell_parser.ps1` into a
long-running parser server with comments explaining the protocol, the
AST-shape restrictions, and why unsupported constructs are rejected
conservatively
- keep request ids and a one-time respawn path so a dead or
desynchronized cached child fails closed instead of silently returning
mixed parser output
- preserve separate parser processes for `powershell.exe` and
`pwsh.exe`, since they do not accept the same language surface
- avoid a direct `PipelineChainAst` type reference in the PowerShell
script so the parser service still runs under Windows PowerShell 5.1 as
well as newer `pwsh`
- make `shell-command/src/command_safety/windows_safe_commands.rs`
delegate to the new parser utility instead of spawning a fresh
PowerShell process for every parse
- add a Windows-only unit test that exercises multiple sequential
requests against the same parser process

## Testing

- adds a Windows-only parser-reuse unit test in `powershell_parser.rs`
- the main end-to-end verification for this change is the Windows CI
lane, because the new service depends on real `powershell.exe` /
`pwsh.exe` behavior
2026-03-27 19:33:41 -07:00
Michael Bolin a773e162cd Merge d9abab2c05 into sapling-pr-archive-bolinfest 2026-03-27 19:25:12 -07:00
Michael Bolin d9abab2c05 shell-escalation: keep macOS handshake stream alive 2026-03-27 19:25:03 -07:00
Joe LicciniandCodex 71923f43a7 Support Codex CLI stdin piping for codex exec (#15917)
# Summary

Claude Code supports a useful prompt-plus-stdin workflow:

```bash
echo "complex input..." | claude -p "summarize concisely"
```

Codex previously did not support the equivalent `codex exec` form. While
`codex exec` could read the prompt from stdin, it could not combine
piped input with an explicit prompt argument.

This change adds that missing workflow:

```bash
echo "complex input..." | codex exec "summarize concisely"
```

With this change, when `codex exec` receives both a positional prompt
and piped stdin, the prompt remains the instruction and stdin is passed
along as structured `<stdin>...</stdin>` context.

Example:

```bash
curl https://jsonplaceholder.typicode.com/comments \
  | ./target/debug/codex exec --skip-git-repo-check "format the top 20 items into a markdown table" \
  > table.md
```

This PR also adds regression coverage for:
- prompt argument + piped stdin
- legacy stdin-as-prompt behavior
- `codex exec -` forced-stdin behavior
- empty-stdin error cases

---------

Co-authored-by: Codex <noreply@openai.com>
2026-03-28 02:21:22 +00:00
Michael Bolin d2c02f3d64 merge commit for archive created by Sapling 2026-03-27 19:20:21 -07:00
Michael Bolin 215c82b09d refactor: rewrite argument-comment lint wrappers in Python 2026-03-27 19:20:07 -07:00
Michael Bolin d28e030d9f Merge d849d5e545 into sapling-pr-archive-bolinfest 2026-03-27 19:12:41 -07:00
Michael Bolin d849d5e545 refactor: rewrite argument-comment lint wrappers in Python 2026-03-27 19:12:34 -07:00
Michael Bolin b76196f912 Merge f37fa114bc into sapling-pr-archive-bolinfest 2026-03-27 19:11:53 -07:00
Michael Bolin f37fa114bc shell-escalation: keep macOS handshake stream alive 2026-03-27 19:11:47 -07:00
Michael Bolin dccbd68c00 Merge bb7732ada7 into sapling-pr-archive-bolinfest 2026-03-27 19:04:53 -07:00
Michael Bolin bb7732ada7 shell-escalation: keep macOS handshake stream alive 2026-03-27 19:04:45 -07:00
Michael Bolin 61dfe0b86c chore: clean up argument-comment lint and roll out all-target CI on macOS (#16054)
## Why

`argument-comment-lint` was green in CI even though the repo still had
many uncommented literal arguments. The main gap was target coverage:
the repo wrapper did not force Cargo to inspect test-only call sites, so
examples like the `latest_session_lookup_params(true, ...)` tests in
`codex-rs/tui_app_server/src/lib.rs` never entered the blocking CI path.

This change cleans up the existing backlog, makes the default repo lint
path cover all Cargo targets, and starts rolling that stricter CI
enforcement out on the platform where it is currently validated.

## What changed

- mechanically fixed existing `argument-comment-lint` violations across
the `codex-rs` workspace, including tests, examples, and benches
- updated `tools/argument-comment-lint/run-prebuilt-linter.sh` and
`tools/argument-comment-lint/run.sh` so non-`--fix` runs default to
`--all-targets` unless the caller explicitly narrows the target set
- fixed both wrappers so forwarded cargo arguments after `--` are
preserved with a single separator
- documented the new default behavior in
`tools/argument-comment-lint/README.md`
- updated `rust-ci` so the macOS lint lane keeps the plain wrapper
invocation and therefore enforces `--all-targets`, while Linux and
Windows temporarily pass `-- --lib --bins`

That temporary CI split keeps the stricter all-targets check where it is
already cleaned up, while leaving room to finish the remaining Linux-
and Windows-specific target-gated cleanup before enabling
`--all-targets` on those runners. The Linux and Windows failures on the
intermediate revision were caused by the wrapper forwarding bug, not by
additional lint findings in those lanes.

## Validation

- `bash -n tools/argument-comment-lint/run.sh`
- `bash -n tools/argument-comment-lint/run-prebuilt-linter.sh`
- shell-level wrapper forwarding check for `-- --lib --bins`
- shell-level wrapper forwarding check for `-- --tests`
- `just argument-comment-lint`
- `cargo test` in `tools/argument-comment-lint`
- `cargo test -p codex-terminal-detection`

## Follow-up

- Clean up remaining Linux-only target-gated callsites, then switch the
Linux lint lane back to the plain wrapper invocation.
- Clean up remaining Windows-only target-gated callsites, then switch
the Windows lint lane back to the plain wrapper invocation.
2026-03-27 19:00:44 -07:00
Michael Bolin d930261a67 merge commit for archive created by Sapling 2026-03-27 18:22:02 -07:00
Michael Bolin f2ab5f36bb chore: enforce argument-comment lint across all targets 2026-03-27 18:19:24 -07:00
Eric Traut ed977b42ac Fix tui_app_server agent picker closed-state regression (#16014)
Addresses #15992

The app-server TUI was treating tracked agent threads as closed based on
listener-task bookkeeping that does not reflect live thread state during
normal thread switching. That caused the `/agent` picker to gray out
live agents and could show a false "Agent thread ... is closed" replay
message after switching branches.

This PR fixes the picker refresh path to query the app server for each
tracked thread and derive closed vs loaded state from `thread/read`
status, while preserving cached agent metadata for replay-only threads.
2026-03-27 19:05:43 -06:00
Eric Traut 8e24d5aaea Fix tui_app_server resume-by-name lookup regression (#16050)
Addresses #16049

`codex resume <name>` and `/resume <name>` could fail in the app-server
TUI path because name lookup pre-filtered `thread/list` with the backend
`search_term`, but saved thread names are hydrated after listing and are
not part of that search index. Resolve names by scanning listed threads
client-side instead, and add a regression test for saved sessions whose
rollout title does not match the thread name.
2026-03-27 19:04:48 -06:00
Michael Bolin 05a7dad803 merge commit for archive created by Sapling 2026-03-27 17:36:39 -07:00
Michael Bolin f05597bbc0 chore: enforce argument-comment lint across all targets 2026-03-27 17:34:47 -07:00
Michael Bolin 1ca44623cc Merge 6759b7b659 into sapling-pr-archive-bolinfest 2026-03-27 17:24:22 -07:00
Michael Bolin 6759b7b659 shell-command: reuse a PowerShell parser process on Windows 2026-03-27 17:24:10 -07:00
Michael Bolin adb3b736ca Merge 880a15a281 into sapling-pr-archive-bolinfest 2026-03-27 17:21:45 -07:00
Michael Bolin 880a15a281 chore: enforce argument-comment lint across all targets 2026-03-27 17:21:27 -07:00
Michael Bolin 2ffb32db98 ci: run SDK tests with a Bazel-built codex (#16046)
## Why

Before this change, the SDK CI job built `codex` with Cargo before
running the TypeScript package tests. That step has been getting more
expensive as the Rust workspace grows, while the repo already has a
Bazel-backed build path for the CLI.

The SDK tests also need a normal executable path they can spawn
repeatedly. Moving the job to Bazel exposed an extra CI detail: a plain
`bazel-bin/...` lookup is not reliable under the Linux config because
top-level outputs may stay remote and the wrapper emits status lines
around `cquery` output.

## What Changed

- taught `sdk/typescript/tests/testCodex.ts` to honor `CODEX_EXEC_PATH`
before falling back to the local Cargo-style `target/debug/codex` path
- added `--remote-download-toplevel` to
`.github/scripts/run-bazel-ci.sh` so workflows can force Bazel to
materialize top-level outputs on disk after a build
- switched `.github/workflows/sdk.yml` from `cargo build --bin codex` to
the shared Bazel CI setup and `//codex-rs/cli:codex` build target
- changed the SDK workflow to resolve the built CLI with wrapper-backed
`cquery --output=files`, stage the binary into
`${GITHUB_WORKSPACE}/.tmp/sdk-ci/codex`, and point the SDK tests at that
path via `CODEX_EXEC_PATH`
- kept the warm-up step before Jest and the Bazel repository-cache save
step

## Verification

- `bash -n .github/scripts/run-bazel-ci.sh`
- `./.github/scripts/run-bazel-ci.sh -- cquery --output=files --
//codex-rs/cli:codex | grep -E '^(/|bazel-out/)' | tail -n 1`
- `./.github/scripts/run-bazel-ci.sh --remote-download-toplevel -- build
--build_metadata=TAG_job=sdk -- //codex-rs/cli:codex`
- `CODEX_EXEC_PATH="$PWD/.tmp/sdk-ci/codex" pnpm --dir sdk/typescript
test --runInBand`
- `pnpm --dir sdk/typescript lint`
2026-03-27 17:17:22 -07:00
Michael Bolin 298f66831e Merge a547507e48 into sapling-pr-archive-bolinfest 2026-03-27 16:57:25 -07:00
Michael Bolin a547507e48 fix: speed up test that uses PowerShell 2026-03-27 16:57:04 -07:00
Michael Bolin f367c71f6a merge commit for archive created by Sapling 2026-03-27 16:50:04 -07:00
Michael Bolin 28ac0c9171 chore: enforce argument-comment lint across all targets 2026-03-27 16:49:54 -07:00
Michael Bolin fb9bc28e9d merge commit for archive created by Sapling 2026-03-27 16:36:37 -07:00
Michael Bolin c42afa14dd wip: experiment with windows bazel v8 build 2026-03-27 16:36:26 -07:00
Michael Bolin c790a31e72 Merge 05737eb51e into sapling-pr-archive-bolinfest 2026-03-27 16:21:24 -07:00
Michael Bolin 05737eb51e chore: enforce argument-comment lint across all targets 2026-03-27 16:17:50 -07:00
Michael Bolin 2f8f47d1d6 merge commit for archive created by Sapling 2026-03-27 16:16:12 -07:00