node-exporter already runs on every host, is already scraped (job
'node', targets from cichlid's inventory, labelled by host), and already
has the textfile collector enabled with a spool at
/var/lib/prometheus/node-exporter. So a textfile drop needs no new
exporter, no port allocation, no firewalld service and no scrape config
-- which is why it is one-time provisioning rather than a deploy job.
DCGM was the alternative and is the wrong tool: it targets datacenter
cards and its GeForce support is partial. nvidia-smi reports everything
wanted.
The script writes to a temp file and renames atomically, because the
collector reads whole files and a half-written one surfaces as
node_textfile_scrape_error. When nvidia-smi fails it emits an explicit
nvidia_gpu_scrape_ok 0 rather than leaving a stale file, so a broken
driver reads as broken instead of the graphs quietly flatlining.
Verified on the dual-GPU host: both cards reporting temperature, power,
utilisation, memory, clocks and fan through node-exporter with
node_textfile_scrape_error 0.
Fleet-general rather than quantus-specific -- it lives here because
there is no monitoring repo yet, and the dashboard says so.
