Files
codex/.github/workflows/sdk.yml
Adam Perry @ OpenAI bd5c860abe ci: route build IO through Dev Drives (#31357)
## Why

Windows Cargo and Bazel jobs spend significant time in filesystem-heavy
build and cache directories. Route those directories through one CI
build root so Windows can use its Dev Drive and Unix can use a stable
cache root.

## What

- Have `setup-ci` define `CI_BUILD_ROOT`, `CARGO_TARGET_DIR`, Bazel
cache/output paths, and temp paths.
- Require Windows to find or provision a verified Dev Drive instead of
falling back to `C:`.
- Pass the shared Bazel output base to `setup-bazel` so its explicit
`output_base` does not defeat Dev Drive routing.
- Point nextest, release, and V8 source-build paths at the shared
environment contract.

## Benchmark results

One-off cold-cache WPR/ETW traces show the explicit Bazel output-base
routing removes the dominant `C:` traffic:

| sample | `C:\_bazel` | summed `C:` traffic | traced test step |
|---|---:|---:|---:|
| shard 1 before | 62.2 GiB | 85.2 GiB | 16m22s |
| shard 1 updated | 0 | 16.5 GiB | 12m05s |
| shard 3 before | 67.2 GiB | 84.6 GiB | 16m48s |
| shard 3 updated | 0 | 13.5 GiB | 11m08s |

For a cold x64 V8 source build, the retained build-tail sample showed
`D:\cargo-target` at ~1.29 GiB while measured `C:` roots totaled ~0.45
GiB (`C:\Users` ~0.33 GiB, `C:\Program Files` ~0.06 GiB, `C:\Windows`
~0.03 GiB). The full cold build took 2h20m36s.

The Bazel timing improvement is directional because both refreshed
shards failed tests. The V8 trace is a bounded build-tail sample, not
the full build. All final samples had zero lost ETW events; VHDX traffic
was excluded from the optimization ranking.

Runs: [baseline
Bazel](https://github.com/openai/codex/actions/runs/28911908527),
[updated
Bazel](https://github.com/openai/codex/actions/runs/28917133701), [V8
build tail](https://github.com/openai/codex/actions/runs/28933626678).

## Manual validation

- Ran `just fmt`.
- Ran `just test-github-scripts` (35 tests).
- Parsed GitHub Actions YAML with `yq`.
- Ran `git diff --check`.

## Stack

- [#31332](https://github.com/openai/codex/pull/31332) — parameterize
Cargo target paths
- [#31356](https://github.com/openai/codex/pull/31356) — Windows 2025
runner bump
- [#31357](https://github.com/openai/codex/pull/31357) — Dev Drive I/O
routing
2026-07-08 14:06:37 -07:00

169 lines
6.1 KiB
YAML

name: sdk
on:
workflow_call:
jobs:
python-sdk:
runs-on:
group: ${{ github.event.repository.name }}-runners
labels: ${{ github.event.repository.name }}-linux-x64
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
persist-credentials: false
- name: Test Python SDK
shell: bash
run: |
set -euo pipefail
# Run inside a glibc Linux image so dependency resolution exercises
# the pinned manylinux runtime wheel that users install.
docker run --rm \
--user "$(id -u):$(id -g)" \
-e HOME=/tmp/codex-python-sdk-home \
-e UV_LINK_MODE=copy \
-v "${GITHUB_WORKSPACE}:${GITHUB_WORKSPACE}" \
-w "${GITHUB_WORKSPACE}/sdk/python" \
python:3.12-slim \
sh -euxc '
python -m venv /tmp/uv
/tmp/uv/bin/python -m pip install uv==0.11.3
/tmp/uv/bin/uv sync --group dev --frozen
/tmp/uv/bin/uv run --frozen --no-sync ruff check --output-format=github .
/tmp/uv/bin/uv run --frozen --no-sync ruff format --check .
/tmp/uv/bin/uv run --frozen --no-sync pytest
'
- name: Check for a clean worktree
if: always() && !cancelled()
uses: ./.github/actions/check-clean-worktree
sdks:
runs-on:
group: ${{ github.event.repository.name }}-runners
labels: ${{ github.event.repository.name }}-linux-x64
timeout-minutes: 10
environment:
name: bazel
deployment: false
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
persist-credentials: false
- name: Install Linux bwrap build dependencies
shell: bash
run: |
set -euo pipefail
sudo apt-get update -y
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends pkg-config libcap-dev
- name: Setup pnpm
uses: pnpm/action-setup@a8198c4bff370c8506180b035930dea56dbd5288 # v5
with:
run_install: false
- name: Setup Node.js
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: 22
cache: pnpm
- name: Set up Bazel CI
id: setup_bazel
uses: ./.github/actions/setup-bazel-ci
with:
target: x86_64-unknown-linux-gnu
- name: Build codex with Bazel
env:
BUILDBUDDY_API_KEY: ${{ secrets.BUILDBUDDY_API_KEY }}
shell: bash
run: |
set -euo pipefail
# Use the shared CI wrapper so fork PRs fall back cleanly when
# BuildBuddy credentials are unavailable. This workflow needs the
# built `codex` binary on disk afterwards, so ask the wrapper to
# override CI's default remote_download_minimal behavior.
./.github/scripts/run-bazel-ci.sh \
--remote-download-toplevel \
-- \
build \
--build_metadata=COMMIT_SHA=${GITHUB_SHA} \
--build_metadata=TAG_job=sdk \
-- \
//codex-rs/cli:codex
# Resolve the exact output file using the same wrapper/config path as
# the build instead of guessing which Bazel convenience symlink is
# available on the runner.
cquery_output="$(
./.github/scripts/run-bazel-ci.sh \
-- \
cquery \
--output=files \
-- \
//codex-rs/cli:codex \
| grep -E '^(/|bazel-out/)' \
| tail -n 1
)"
if [[ "${cquery_output}" = /* ]]; then
codex_bazel_output_path="${cquery_output}"
else
codex_bazel_output_path="${GITHUB_WORKSPACE}/${cquery_output}"
fi
if [[ -z "${codex_bazel_output_path}" ]]; then
echo "Bazel did not report an output path for //codex-rs/cli:codex." >&2
exit 1
fi
if [[ ! -e "${codex_bazel_output_path}" ]]; then
echo "Unable to locate the Bazel-built codex binary at ${codex_bazel_output_path}." >&2
exit 1
fi
# Stage the binary into the workspace and point the SDK tests at that
# stable path. The tests spawn `codex` directly many times, so using a
# normal executable path is more reliable than invoking Bazel for each
# test process.
install_dir="${GITHUB_WORKSPACE}/.tmp/sdk-ci"
mkdir -p "${install_dir}"
install -m 755 "${codex_bazel_output_path}" "${install_dir}/codex"
echo "CODEX_EXEC_PATH=${install_dir}/codex" >> "$GITHUB_ENV"
- name: Warm up Bazel-built codex
shell: bash
run: |
set -euo pipefail
"${CODEX_EXEC_PATH}" --version
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Build SDK packages
run: pnpm -r --filter ./sdk/typescript run build
- name: Lint SDK packages
run: pnpm -r --filter ./sdk/typescript run lint
- name: Test SDK packages
run: pnpm -r --filter ./sdk/typescript run test
- name: Save bazel repository cache
if: always() && !cancelled() && steps.setup_bazel.outputs.cache-hit != 'true'
continue-on-error: true
uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
with:
path: ${{ steps.setup_bazel.outputs.repository-cache-path }}
key: bazel-cache-x86_64-unknown-linux-gnu-${{ hashFiles('MODULE.bazel', 'codex-rs/Cargo.lock', 'codex-rs/Cargo.toml') }}
- name: Check for a clean worktree
if: always() && !cancelled()
uses: ./.github/actions/check-clean-worktree