metrics: performance by build — recording rules, dashboard row, throttle reasons

Monitoring side of quantus/miner#9. The miner now exports build identity,
per-device and per-job series (quantus/miner#14); this makes them readable.

- asset/prometheus/quantus-rules.yml: recording rules joining the miner's
  series with the GPU series on a derived `host` and device index —
  hashrate by device and commit, hashrate per watt, batch time off the GPU,
  stale work, seal latency p95, idle waiting for the node.
- script/infra-setup.sh (metrics role): prometheus.yml gains `rule_files:`
  for /etc/prometheus/rules.d, guarded like scrape_config_files; the dir;
  a sudoers grant for the rules file.
- deploy.yaml deploy-metrics: pushes the rules (reload is already gated on
  content change); validate asserts the quantus-miner rule group loaded.
- asset/grafana/quantus.json: "Performance by build" row, eight panels, and
  a deploy annotation from changes in miner_build_info. No credentials
  involved; Prometheus is the annotation source.
- asset/gpu/nvidia-textfile.sh (gpu role, operator-shipped): five
  nvidia_gpu_event_reason_active{reason=...} gauges from nvidia-smi's clock
  event reasons, so a regression that coincides with thermal or power-brake
  capping can be told from a code regression.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CBgs2nSi4H2mdh8kD8vMX5

script/infra-setup.sh

@@ -239,6 +239,7 @@ provision_metrics() {
     usermod -aG systemd-journal gitea_ci
 
     install -d -m 0755 /etc/prometheus/scrape_configs.d
+    install -d -m 0755 /etc/prometheus/rules.d
     install -d -m 0755 /etc/grafana/provisioning/dashboards/quantus
 
     # 1. include drop-in scrape configs
@@ -258,6 +259,23 @@ PROM
       curl -fsS -X POST http://127.0.0.1:26559/-/reload && echo "  = prometheus reloaded"
     fi
 
+    # 1b. include drop-in recording rules (quantus/miner#9), same shape.
+    if grep -q '^rule_files:' /etc/prometheus/prometheus.yml; then
+      echo "  = prometheus.yml already includes rules.d"
+    else
+      cp -a /etc/prometheus/prometheus.yml \
+            "/etc/prometheus/prometheus.yml.bak.$(date +%s)"
+      cat >> /etc/prometheus/prometheus.yml <<'PROM'
+
+# Per-project recording and alerting rules. Added by lair/quantus
+# script/infra-setup.sh.
+rule_files:
+  - /etc/prometheus/rules.d/*.yml
+PROM
+      echo "  + prometheus.yml now includes rules.d"
+      curl -fsS -X POST http://127.0.0.1:26559/-/reload && echo "  = prometheus reloaded"
+    fi
+
     # 2. give Grafana somewhere to read provisioned dashboards from
     q=/etc/containers/systemd/grafana.container
     if grep -q 'provisioning/dashboards:' "$q"; then
@@ -275,6 +293,7 @@ PROM
     tmp=/etc/sudoers.d/.quantus-metrics_gitea_ci.tmp
     cat > "$tmp" <<'SUDO'
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/rsync * /etc/prometheus/scrape_configs.d/quantus.yml
+gitea_ci ALL=(root) NOPASSWD: /usr/bin/rsync * /etc/prometheus/rules.d/quantus.yml
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/rsync * /etc/grafana/provisioning/dashboards/quantus/quantus.json
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/rsync * /etc/grafana/provisioning/dashboards/quantus/gpu.json
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/rsync * /etc/grafana/provisioning/dashboards/quantus-dashboards.yaml