Balance Windows Bazel test shards using duration estimates (#49103)

## Why

Hash-based sharding does not account for differences in test duration. Use estimated test time to balance the Windows Bazel workload across shards.

## What changed

- Add checked-in duration weights and a deterministic selector that assigns the longest targets first to the least-loaded shard.
- Assign every queried target exactly once, default missing weights to one second, and ignore stale weights.
- Run queries and selection in the foreground so failures stop the job before tests run with a partial target list.
- Emit LF-delimited target labels for Git Bash and report estimated shard loads on stderr.

## Testing

Add seven passing selector tests covering deterministic coverage and balancing, missing and stale weights, invalid inputs, duration-file overrides, and CLI output. Run them on the first Windows CI shard.

GitOrigin-RevId: d415ae093bfe57794ee53a2eb9d712022202dad8
This commit is contained in:
jgershen-oai
2026-09-28 23:52:19 +00:00
committed by copyberry
parent c2d2f422e6
commit 7e049b3eaa
4 changed files with 520 additions and 22 deletions
+17 -22
View File
@@ -160,6 +160,11 @@ jobs:
shell: pwsh
run: python .github/scripts/test_run_bazel_with_buildbuddy.py
- name: Test Windows Bazel target selector
if: matrix.shard == 1
shell: pwsh
run: python -m unittest discover -s .github/scripts -p test_select_windows_bazel_targets.py
- name: Prepare Bazel CI
id: prepare_bazel
uses: ./.github/actions/prepare-bazel-ci
@@ -180,29 +185,19 @@ jobs:
set -euo pipefail
bazel_test_query='tests(//...) except tests(//third_party/v8:all) except attr(tags, "manual", tests(//...))'
mapfile -t bazel_targets < <(
./.github/scripts/run-bazel-query-ci.sh --output=label -- "${bazel_test_query}" \
| LC_ALL=C sort
)
bazel_targets_file="$(mktemp)"
selected_targets_file="$(mktemp)"
trap 'rm -f "${bazel_targets_file}" "${selected_targets_file}"' EXIT
selected_targets=()
for bazel_target in "${bazel_targets[@]}"; do
target_bucket="$(
printf '%s\n' "${bazel_target}" \
| cksum \
| awk -v shard_count="${BAZEL_TEST_SHARD_COUNT}" '{ print ($1 % shard_count) + 1 }'
)"
if [[ "${target_bucket}" == "${BAZEL_TEST_SHARD}" ]]; then
selected_targets+=("${bazel_target}")
fi
done
if [[ ${#selected_targets[@]} -eq 0 ]]; then
echo "No Bazel test targets selected for Windows shard ${BAZEL_TEST_SHARD}/${BAZEL_TEST_SHARD_COUNT}." >&2
exit 1
fi
echo "Selected ${#selected_targets[@]} of ${#bazel_targets[@]} Bazel test targets for Windows shard ${BAZEL_TEST_SHARD}/${BAZEL_TEST_SHARD_COUNT}."
# Keep query and selection in the foreground so failures cannot leave
# different jobs assigning different partial target lists.
./.github/scripts/run-bazel-query-ci.sh --output=label -- "${bazel_test_query}" \
| LC_ALL=C sort > "${bazel_targets_file}"
python .github/scripts/select_windows_bazel_targets.py \
--shard "${BAZEL_TEST_SHARD}" \
--shard-count "${BAZEL_TEST_SHARD_COUNT}" \
< "${bazel_targets_file}" > "${selected_targets_file}"
mapfile -t selected_targets < "${selected_targets_file}"
bazel_test_args=(
test