v1.0.3's heartbeat did its job: it showed exactly where the next failure was.
In monsoon run 56 the heartbeats stopped at 14m29s while the build was still
running, and the job was killed at 10:20:50 — five minutes after COPR had
reported success at 10:15:20.
The poll loop was blocked inside `$(build_state ...)`. copr-cli makes network
requests with no internal timeout, so `copr-cli status` hung for over twelve
minutes and took the loop with it. Same failure mode as watch-build in run 48,
just relocated.
Bound all of them: status (60s), submit (600s), download-build (600s), each
overridable. A timed-out status yields empty output, which reads as
non-terminal, so the loop simply heartbeats and retries on the next tick.
Test reproduces it with a stub whose first status call hangs; it deadlocks
against v1.0.3 and passes here.
rob thijssen
d690d5aaa8
