mirror of
https://github.com/openai/codex.git
synced 2026-08-29 14:09:35 +00:00
## Why Windows Cargo and Bazel jobs spend significant time in filesystem-heavy build and cache directories. Route those directories through one CI build root so Windows can use its Dev Drive and Unix can use a stable cache root. ## What - Have `setup-ci` define `CI_BUILD_ROOT`, `CARGO_TARGET_DIR`, Bazel cache/output paths, and temp paths. - Require Windows to find or provision a verified Dev Drive instead of falling back to `C:`. - Pass the shared Bazel output base to `setup-bazel` so its explicit `output_base` does not defeat Dev Drive routing. - Point nextest, release, and V8 source-build paths at the shared environment contract. ## Benchmark results One-off cold-cache WPR/ETW traces show the explicit Bazel output-base routing removes the dominant `C:` traffic: | sample | `C:\_bazel` | summed `C:` traffic | traced test step | |---|---:|---:|---:| | shard 1 before | 62.2 GiB | 85.2 GiB | 16m22s | | shard 1 updated | 0 | 16.5 GiB | 12m05s | | shard 3 before | 67.2 GiB | 84.6 GiB | 16m48s | | shard 3 updated | 0 | 13.5 GiB | 11m08s | For a cold x64 V8 source build, the retained build-tail sample showed `D:\cargo-target` at ~1.29 GiB while measured `C:` roots totaled ~0.45 GiB (`C:\Users` ~0.33 GiB, `C:\Program Files` ~0.06 GiB, `C:\Windows` ~0.03 GiB). The full cold build took 2h20m36s. The Bazel timing improvement is directional because both refreshed shards failed tests. The V8 trace is a bounded build-tail sample, not the full build. All final samples had zero lost ETW events; VHDX traffic was excluded from the optimization ranking. Runs: [baseline Bazel](https://github.com/openai/codex/actions/runs/28911908527), [updated Bazel](https://github.com/openai/codex/actions/runs/28917133701), [V8 build tail](https://github.com/openai/codex/actions/runs/28933626678). ## Manual validation - Ran `just fmt`. - Ran `just test-github-scripts` (35 tests). - Parsed GitHub Actions YAML with `yq`. - Ran `git diff --check`. ## Stack - [#31332](https://github.com/openai/codex/pull/31332) — parameterize Cargo target paths - [#31356](https://github.com/openai/codex/pull/31356) — Windows 2025 runner bump - [#31357](https://github.com/openai/codex/pull/31357) — Dev Drive I/O routing
98 lines
3.6 KiB
YAML
98 lines
3.6 KiB
YAML
name: setup-ci
|
|
description: Prepare common tools and environment shared by CI jobs.
|
|
outputs:
|
|
bazel-output-base:
|
|
description: Filesystem path used for Bazel's output base.
|
|
value: ${{ steps.configure_ci_build_paths.outputs.bazel-output-base }}
|
|
cargo-target-dir:
|
|
description: Filesystem path used for Cargo's target directory.
|
|
value: ${{ steps.configure_ci_build_paths.outputs.cargo-target-dir }}
|
|
|
|
runs:
|
|
using: composite
|
|
steps:
|
|
# setup-ci expects either this step or the Unix fallback below to define
|
|
# CI_BUILD_ROOT. Windows puts it on a Dev Drive because Cargo and Bazel
|
|
# spend significant time reading and writing build/cache trees.
|
|
- name: Configure Dev Drive (Windows)
|
|
if: runner.os == 'Windows'
|
|
shell: pwsh
|
|
run: ./.github/scripts/setup-dev-drive.ps1
|
|
|
|
- name: Configure CI build root (Unix)
|
|
if: runner.os != 'Windows'
|
|
shell: bash
|
|
run: echo "CI_BUILD_ROOT=$HOME/.cache/codex-ci" >> "$GITHUB_ENV"
|
|
|
|
- name: Configure CI build paths
|
|
id: configure_ci_build_paths
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
# setup-bazel passes output_base explicitly, so keep both it and the
|
|
# user root under the shared build root. Keep these directory names tiny
|
|
# on every platform so Windows Bazel paths do not overflow argv or
|
|
# confuse test MANIFEST handling.
|
|
bazel_output_base="$CI_BUILD_ROOT/o"
|
|
bazel_output_user_root="$CI_BUILD_ROOT/b"
|
|
bazel_repository_cache="$CI_BUILD_ROOT/bazel-repository-cache"
|
|
bazel_repo_contents_cache="$CI_BUILD_ROOT/bazel-repo-contents-cache-$GITHUB_RUN_ID-$GITHUB_JOB"
|
|
cargo_target_dir="$CI_BUILD_ROOT/cargo-target"
|
|
tmp="$CI_BUILD_ROOT/tmp"
|
|
|
|
build_dirs=(
|
|
"$bazel_output_base"
|
|
"$bazel_output_user_root"
|
|
"$bazel_repository_cache"
|
|
"$bazel_repo_contents_cache"
|
|
"$cargo_target_dir"
|
|
"$tmp"
|
|
)
|
|
mkdir -p "${build_dirs[@]}"
|
|
echo "bazel-output-base=$bazel_output_base" >> "$GITHUB_OUTPUT"
|
|
echo "cargo-target-dir=$cargo_target_dir" >> "$GITHUB_OUTPUT"
|
|
|
|
{
|
|
echo "BAZEL_OUTPUT_BASE=$bazel_output_base"
|
|
echo "BAZEL_OUTPUT_USER_ROOT=$bazel_output_user_root"
|
|
echo "BAZEL_REPOSITORY_CACHE=$bazel_repository_cache"
|
|
echo "BAZEL_REPO_CONTENTS_CACHE=$bazel_repo_contents_cache"
|
|
echo "CARGO_TARGET_DIR=$cargo_target_dir"
|
|
echo "TEMP=$tmp"
|
|
echo "TMP=$tmp"
|
|
} >> "$GITHUB_ENV"
|
|
|
|
- name: Prefer the Git CLI for Cargo git dependencies
|
|
shell: bash
|
|
run: echo "CARGO_NET_GIT_FETCH_WITH_CLI=true" >> "$GITHUB_ENV"
|
|
|
|
- name: Install DotSlash
|
|
uses: facebook/install-dotslash@1e4e7b3e07eaca387acb98f1d4720e0bee8dbb6a # v2
|
|
|
|
- name: Install just
|
|
uses: taiki-e/install-action@44c6d64aa62cd779e873306675c7a58e86d6d532 # v2.62.49
|
|
with:
|
|
tool: just@1.51.0
|
|
|
|
# Some integration tests spawn DotSlash from a stable system path rather
|
|
# than inheriting the action-local PATH entry.
|
|
- name: Make DotSlash available in PATH (Unix)
|
|
if: runner.os != 'Windows'
|
|
shell: bash
|
|
run: |
|
|
if [[ -w /usr/local/bin ]]; then
|
|
cp "$(which dotslash)" /usr/local/bin
|
|
else
|
|
sudo cp "$(which dotslash)" /usr/local/bin
|
|
fi
|
|
|
|
- name: Make DotSlash available in PATH (Windows)
|
|
if: runner.os == 'Windows'
|
|
shell: pwsh
|
|
run: Copy-Item (Get-Command dotslash).Source -Destination "$env:LOCALAPPDATA\Microsoft\WindowsApps\dotslash.exe"
|
|
|
|
- name: Enable Git long paths (Windows)
|
|
if: runner.os == 'Windows'
|
|
shell: pwsh
|
|
run: git config --global core.longpaths true
|