fix: ship the grafana dashboard provider, and assert it loaded

The dashboard JSON was deployed but its provider config never was, so
Grafana had no reason to read the directory -- the dashboard would have
been a file on disk and an empty folder in the UI.

Tracked separately from the dashboard because Grafana reads providers
only at container startup while dashboards are re-read on the
provisioning interval: a provider change restarts grafana, a dashboard
change does not.

validate now queries Grafana's search API for the dashboard uid rather
than testing that a file exists, since a provisioned dashboard that
fails to parse leaves the JSON looking perfectly fine on disk.

.gitea/workflows/deploy.yaml

@@ -62,6 +62,7 @@ env:
   METRICS_HOST: golgafrinchans.kosherinata.internal
   # Host-published port of the fleet Prometheus (port-allocations.md §5).
   PROM_PORT: "26559"
+  GRAFANA_PORT: "28767"
   # Scrape targets, rendered into the Prometheus scrape config. These must agree
   # with the deploy-node / deploy-miner matrices below; `validate metrics`
   # asserts Prometheus actually has every target UP, so drift fails loudly rather
@@ -685,6 +686,25 @@ jobs:
           push --mkpath --chmod=F0644 asset/grafana/quantus.json \
             gitea_ci@"$HOST":/etc/grafana/provisioning/dashboards/quantus/quantus.json
 
+          # The PROVIDER tells Grafana the dashboards directory exists at all;
+          # without it the JSON above is just a file nobody reads. Tracked
+          # separately from the dashboard because Grafana reads providers ONLY at
+          # startup, while dashboards are re-read on updateIntervalSeconds — so a
+          # provider change needs a restart and a dashboard change does not.
+          PROVIDER=0
+          pout=$(rsync -e "ssh $SSHOPTS" --rsync-path='sudo rsync' -ic --mkpath --chmod=F0644 \
+            asset/grafana/quantus-dashboards.yaml \
+            gitea_ci@"$HOST":/etc/grafana/provisioning/dashboards/quantus-dashboards.yaml)
+          if [ -n "$pout" ]; then
+            PROVIDER=1
+            printf '%s\n' "$pout" | sed 's/^/    changed: /'
+          fi
+
+          if [ "$PROVIDER" = 1 ]; then
+            echo "dashboard provider changed — restarting grafana (providers are read at startup only)"
+            run sudo systemctl restart grafana.service
+          fi
+
           if [ "$RELOAD" = 1 ]; then
             # Hot reload rather than a restart: Prometheus restarts drop nothing
             # permanent but do interrupt scraping, and --web.enable-lifecycle is
@@ -729,10 +749,16 @@ jobs:
           printf '%s\n' "$got" | grep -qv " up " && { echo "  FAIL not every quantus target is up" >&2; fail=1; } || true
 
           echo "--- grafana dashboard ---"
-          if run "test -s /etc/grafana/provisioning/dashboards/quantus/quantus.json"; then
-            echo "  ok   dashboard JSON present on the host"
+          # Assert Grafana actually LOADED it, not merely that a file is on disk.
+          # A provisioned dashboard that failed to parse leaves the JSON sitting
+          # there looking fine while the folder stays empty.
+          if run "curl -fsS 'http://127.0.0.1:${{ env.GRAFANA_PORT }}/api/search?query=Quantus'" \
+               | grep -q 'quantus-node-miner'; then
+            echo "  ok   grafana has loaded the dashboard (uid quantus-node-miner)"
           else
-            echo "  FAIL dashboard JSON missing" >&2; fail=1
+            echo "  FAIL grafana has not loaded the dashboard" >&2
+            echo "       provider present? check /etc/grafana/provisioning/dashboards/" >&2
+            fail=1
           fi
 
           exit $fail

script/infra-setup.sh

@@ -271,6 +271,8 @@ PROM
     cat > "$tmp" <<'SUDO'
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/rsync * /etc/prometheus/scrape_configs.d/quantus.yml
 gitea_ci ALL=(root) NOPASSWD: /usr/bin/rsync * /etc/grafana/provisioning/dashboards/quantus/quantus.json
+gitea_ci ALL=(root) NOPASSWD: /usr/bin/rsync * /etc/grafana/provisioning/dashboards/quantus-dashboards.yaml
+gitea_ci ALL=(root) NOPASSWD: /usr/bin/systemctl restart grafana.service
 SUDO
     chmod 0440 "$tmp"
     visudo -cf "$tmp"