mirror of
https://github.com/openai/codex.git
synced 2026-08-23 13:09:46 +00:00
## Why Windows Cargo and Bazel jobs spend significant time in filesystem-heavy build and cache directories. Route those directories through one CI build root so Windows can use its Dev Drive and Unix can use a stable cache root. ## What - Have `setup-ci` define `CI_BUILD_ROOT`, `CARGO_TARGET_DIR`, Bazel cache/output paths, and temp paths. - Require Windows to find or provision a verified Dev Drive instead of falling back to `C:`. - Pass the shared Bazel output base to `setup-bazel` so its explicit `output_base` does not defeat Dev Drive routing. - Point nextest, release, and V8 source-build paths at the shared environment contract. ## Benchmark results One-off cold-cache WPR/ETW traces show the explicit Bazel output-base routing removes the dominant `C:` traffic: | sample | `C:\_bazel` | summed `C:` traffic | traced test step | |---|---:|---:|---:| | shard 1 before | 62.2 GiB | 85.2 GiB | 16m22s | | shard 1 updated | 0 | 16.5 GiB | 12m05s | | shard 3 before | 67.2 GiB | 84.6 GiB | 16m48s | | shard 3 updated | 0 | 13.5 GiB | 11m08s | For a cold x64 V8 source build, the retained build-tail sample showed `D:\cargo-target` at ~1.29 GiB while measured `C:` roots totaled ~0.45 GiB (`C:\Users` ~0.33 GiB, `C:\Program Files` ~0.06 GiB, `C:\Windows` ~0.03 GiB). The full cold build took 2h20m36s. The Bazel timing improvement is directional because both refreshed shards failed tests. The V8 trace is a bounded build-tail sample, not the full build. All final samples had zero lost ETW events; VHDX traffic was excluded from the optimization ranking. Runs: [baseline Bazel](https://github.com/openai/codex/actions/runs/28911908527), [updated Bazel](https://github.com/openai/codex/actions/runs/28917133701), [V8 build tail](https://github.com/openai/codex/actions/runs/28933626678). ## Manual validation - Ran `just fmt`. - Ran `just test-github-scripts` (35 tests). - Parsed GitHub Actions YAML with `yq`. - Ran `git diff --check`. ## Stack - [#31332](https://github.com/openai/codex/pull/31332) — parameterize Cargo target paths - [#31356](https://github.com/openai/codex/pull/31356) — Windows 2025 runner bump - [#31357](https://github.com/openai/codex/pull/31357) — Dev Drive I/O routing
169 lines
6.1 KiB
YAML
169 lines
6.1 KiB
YAML
name: sdk
|
|
|
|
on:
|
|
workflow_call:
|
|
|
|
jobs:
|
|
python-sdk:
|
|
runs-on:
|
|
group: ${{ github.event.repository.name }}-runners
|
|
labels: ${{ github.event.repository.name }}-linux-x64
|
|
timeout-minutes: 10
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
|
|
persist-credentials: false
|
|
|
|
- name: Test Python SDK
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
|
|
# Run inside a glibc Linux image so dependency resolution exercises
|
|
# the pinned manylinux runtime wheel that users install.
|
|
docker run --rm \
|
|
--user "$(id -u):$(id -g)" \
|
|
-e HOME=/tmp/codex-python-sdk-home \
|
|
-e UV_LINK_MODE=copy \
|
|
-v "${GITHUB_WORKSPACE}:${GITHUB_WORKSPACE}" \
|
|
-w "${GITHUB_WORKSPACE}/sdk/python" \
|
|
python:3.12-slim \
|
|
sh -euxc '
|
|
python -m venv /tmp/uv
|
|
/tmp/uv/bin/python -m pip install uv==0.11.3
|
|
/tmp/uv/bin/uv sync --group dev --frozen
|
|
/tmp/uv/bin/uv run --frozen --no-sync ruff check --output-format=github .
|
|
/tmp/uv/bin/uv run --frozen --no-sync ruff format --check .
|
|
/tmp/uv/bin/uv run --frozen --no-sync pytest
|
|
'
|
|
|
|
- name: Check for a clean worktree
|
|
if: always() && !cancelled()
|
|
uses: ./.github/actions/check-clean-worktree
|
|
|
|
sdks:
|
|
runs-on:
|
|
group: ${{ github.event.repository.name }}-runners
|
|
labels: ${{ github.event.repository.name }}-linux-x64
|
|
timeout-minutes: 10
|
|
environment:
|
|
name: bazel
|
|
deployment: false
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
|
|
persist-credentials: false
|
|
|
|
- name: Install Linux bwrap build dependencies
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
sudo apt-get update -y
|
|
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends pkg-config libcap-dev
|
|
|
|
- name: Setup pnpm
|
|
uses: pnpm/action-setup@a8198c4bff370c8506180b035930dea56dbd5288 # v5
|
|
with:
|
|
run_install: false
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
|
|
with:
|
|
node-version: 22
|
|
cache: pnpm
|
|
|
|
- name: Set up Bazel CI
|
|
id: setup_bazel
|
|
uses: ./.github/actions/setup-bazel-ci
|
|
with:
|
|
target: x86_64-unknown-linux-gnu
|
|
|
|
- name: Build codex with Bazel
|
|
env:
|
|
BUILDBUDDY_API_KEY: ${{ secrets.BUILDBUDDY_API_KEY }}
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
# Use the shared CI wrapper so fork PRs fall back cleanly when
|
|
# BuildBuddy credentials are unavailable. This workflow needs the
|
|
# built `codex` binary on disk afterwards, so ask the wrapper to
|
|
# override CI's default remote_download_minimal behavior.
|
|
./.github/scripts/run-bazel-ci.sh \
|
|
--remote-download-toplevel \
|
|
-- \
|
|
build \
|
|
--build_metadata=COMMIT_SHA=${GITHUB_SHA} \
|
|
--build_metadata=TAG_job=sdk \
|
|
-- \
|
|
//codex-rs/cli:codex
|
|
|
|
# Resolve the exact output file using the same wrapper/config path as
|
|
# the build instead of guessing which Bazel convenience symlink is
|
|
# available on the runner.
|
|
cquery_output="$(
|
|
./.github/scripts/run-bazel-ci.sh \
|
|
-- \
|
|
cquery \
|
|
--output=files \
|
|
-- \
|
|
//codex-rs/cli:codex \
|
|
| grep -E '^(/|bazel-out/)' \
|
|
| tail -n 1
|
|
)"
|
|
if [[ "${cquery_output}" = /* ]]; then
|
|
codex_bazel_output_path="${cquery_output}"
|
|
else
|
|
codex_bazel_output_path="${GITHUB_WORKSPACE}/${cquery_output}"
|
|
fi
|
|
if [[ -z "${codex_bazel_output_path}" ]]; then
|
|
echo "Bazel did not report an output path for //codex-rs/cli:codex." >&2
|
|
exit 1
|
|
fi
|
|
if [[ ! -e "${codex_bazel_output_path}" ]]; then
|
|
echo "Unable to locate the Bazel-built codex binary at ${codex_bazel_output_path}." >&2
|
|
exit 1
|
|
fi
|
|
|
|
# Stage the binary into the workspace and point the SDK tests at that
|
|
# stable path. The tests spawn `codex` directly many times, so using a
|
|
# normal executable path is more reliable than invoking Bazel for each
|
|
# test process.
|
|
install_dir="${GITHUB_WORKSPACE}/.tmp/sdk-ci"
|
|
mkdir -p "${install_dir}"
|
|
install -m 755 "${codex_bazel_output_path}" "${install_dir}/codex"
|
|
echo "CODEX_EXEC_PATH=${install_dir}/codex" >> "$GITHUB_ENV"
|
|
|
|
- name: Warm up Bazel-built codex
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
"${CODEX_EXEC_PATH}" --version
|
|
|
|
- name: Install dependencies
|
|
run: pnpm install --frozen-lockfile
|
|
|
|
- name: Build SDK packages
|
|
run: pnpm -r --filter ./sdk/typescript run build
|
|
|
|
- name: Lint SDK packages
|
|
run: pnpm -r --filter ./sdk/typescript run lint
|
|
|
|
- name: Test SDK packages
|
|
run: pnpm -r --filter ./sdk/typescript run test
|
|
|
|
- name: Save bazel repository cache
|
|
if: always() && !cancelled() && steps.setup_bazel.outputs.cache-hit != 'true'
|
|
continue-on-error: true
|
|
uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
|
|
with:
|
|
path: ${{ steps.setup_bazel.outputs.repository-cache-path }}
|
|
key: bazel-cache-x86_64-unknown-linux-gnu-${{ hashFiles('MODULE.bazel', 'codex-rs/Cargo.lock', 'codex-rs/Cargo.toml') }}
|
|
|
|
- name: Check for a clean worktree
|
|
if: always() && !cancelled()
|
|
uses: ./.github/actions/check-clean-worktree
|