mirror of
https://github.com/openai/codex.git
synced 2026-10-18 17:18:52 +00:00
ci: route build IO through Dev Drives (#31357)
## Why Windows Cargo and Bazel jobs spend significant time in filesystem-heavy build and cache directories. Route those directories through one CI build root so Windows can use its Dev Drive and Unix can use a stable cache root. ## What - Have `setup-ci` define `CI_BUILD_ROOT`, `CARGO_TARGET_DIR`, Bazel cache/output paths, and temp paths. - Require Windows to find or provision a verified Dev Drive instead of falling back to `C:`. - Pass the shared Bazel output base to `setup-bazel` so its explicit `output_base` does not defeat Dev Drive routing. - Point nextest, release, and V8 source-build paths at the shared environment contract. ## Benchmark results One-off cold-cache WPR/ETW traces show the explicit Bazel output-base routing removes the dominant `C:` traffic: | sample | `C:\_bazel` | summed `C:` traffic | traced test step | |---|---:|---:|---:| | shard 1 before | 62.2 GiB | 85.2 GiB | 16m22s | | shard 1 updated | 0 | 16.5 GiB | 12m05s | | shard 3 before | 67.2 GiB | 84.6 GiB | 16m48s | | shard 3 updated | 0 | 13.5 GiB | 11m08s | For a cold x64 V8 source build, the retained build-tail sample showed `D:\cargo-target` at ~1.29 GiB while measured `C:` roots totaled ~0.45 GiB (`C:\Users` ~0.33 GiB, `C:\Program Files` ~0.06 GiB, `C:\Windows` ~0.03 GiB). The full cold build took 2h20m36s. The Bazel timing improvement is directional because both refreshed shards failed tests. The V8 trace is a bounded build-tail sample, not the full build. All final samples had zero lost ETW events; VHDX traffic was excluded from the optimization ranking. Runs: [baseline Bazel](https://github.com/openai/codex/actions/runs/28911908527), [updated Bazel](https://github.com/openai/codex/actions/runs/28917133701), [V8 build tail](https://github.com/openai/codex/actions/runs/28933626678). ## Manual validation - Ran `just fmt`. - Ran `just test-github-scripts` (35 tests). - Parsed GitHub Actions YAML with `yq`. - Ran `git diff --check`. ## Stack - [#31332](https://github.com/openai/codex/pull/31332) — parameterize Cargo target paths - [#31356](https://github.com/openai/codex/pull/31356) — Windows 2025 runner bump - [#31357](https://github.com/openai/codex/pull/31357) — Dev Drive I/O routing
This commit is contained in:
@@ -1,14 +1,15 @@
|
||||
# Configure a fast drive for Windows CI jobs.
|
||||
#
|
||||
# GitHub-hosted Windows runners do not always expose a secondary D: volume. When
|
||||
# they do not, try to create a Dev Drive VHD and fall back to C: if the runner
|
||||
# image does not allow that provisioning path.
|
||||
# they do not, create a Dev Drive VHD. CI depends on this path for its
|
||||
# build directories where CI spends significant time doing I/O, so fail the
|
||||
# job if no real Dev Drive is available.
|
||||
|
||||
function Use-FallbackDrive {
|
||||
param([string]$Reason)
|
||||
function Test-DevDrive {
|
||||
param([string]$Drive)
|
||||
|
||||
Write-Warning "$Reason Falling back to C:"
|
||||
return "C:"
|
||||
& fsutil devdrv query $Drive *> $null
|
||||
return $LASTEXITCODE -eq 0
|
||||
}
|
||||
|
||||
function Invoke-BestEffort {
|
||||
@@ -21,10 +22,14 @@ function Invoke-BestEffort {
|
||||
}
|
||||
}
|
||||
|
||||
if (Test-Path "D:\") {
|
||||
Write-Output "Using existing drive at D:"
|
||||
if ((Test-Path "D:\") -and (Test-DevDrive "D:")) {
|
||||
Write-Output "Using existing Dev Drive at D:"
|
||||
$Drive = "D:"
|
||||
} else {
|
||||
if (Test-Path "D:\") {
|
||||
Write-Output "Existing D: volume is not a Dev Drive; provisioning a new Dev Drive VHD."
|
||||
}
|
||||
|
||||
try {
|
||||
$VhdPath = Join-Path $env:RUNNER_TEMP "codex-dev-drive.vhdx"
|
||||
$SizeBytes = 64GB
|
||||
@@ -42,21 +47,17 @@ if (Test-Path "D:\") {
|
||||
|
||||
$Drive = "$($Volume.DriveLetter):"
|
||||
|
||||
if (-not (Test-DevDrive $Drive)) {
|
||||
throw "Provisioned volume at $Drive did not pass Dev Drive verification."
|
||||
}
|
||||
|
||||
Invoke-BestEffort { fsutil devdrv trust $Drive } "Trusting Dev Drive $Drive"
|
||||
Invoke-BestEffort { fsutil devdrv enable /disallowAv } "Disabling AV filter attachment for Dev Drives"
|
||||
Invoke-BestEffort { fsutil devdrv query $Drive } "Querying Dev Drive $Drive"
|
||||
|
||||
Write-Output "Using Dev Drive at $Drive"
|
||||
} catch {
|
||||
$Drive = Use-FallbackDrive "Failed to create Dev Drive: $($_.Exception.Message)"
|
||||
throw "Failed to create Dev Drive: $($_.Exception.Message)"
|
||||
}
|
||||
}
|
||||
|
||||
$Tmp = "$Drive\codex-tmp"
|
||||
New-Item -Path $Tmp -ItemType Directory -Force | Out-Null
|
||||
|
||||
@(
|
||||
"DEV_DRIVE=$Drive"
|
||||
"TMP=$Tmp"
|
||||
"TEMP=$Tmp"
|
||||
) | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
|
||||
"CI_BUILD_ROOT=$Drive" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
|
||||
|
||||
Reference in New Issue
Block a user