grenadeandClaude Opus 5
6d8c3fe3ea
The proxy's shared access.log mixes every site on oolon and records neither Host nor Origin. Origin is the only field that separates our UI's WebSocket from another site's, because browsers send no Referer on the upgrade. The public vhost now writes its own JSON-lines log carrying both. Ten days of the shared log showed no third-party site using the API, and two headless pollers: an OVH host polling summary once a minute (88% of all no-referer REST) and a Hetzner host identifying as `stardust-devops`. The public vhost now refuses the hosting networks of both providers, from the prefixes RIPEstat reports as announced by AS24940, AS213230, AS212317 and AS16276, generated by script/deny-networks.py and committed so the list is reviewed. AS35540 is OVH's residential broadband and is left out. Against the same ten days, the list matches the two pollers and no visitor. Refusals are a plain-text 403 and are still logged with "denied":1, so a VPN user caught by it shows up rather than disappearing. Closes #24 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E4Ldjix2ofMdQ7dzqgnYA8
76 lines
2.9 KiB
Python
Executable File
76 lines
2.9 KiB
Python
Executable File
#!/usr/bin/env python3
|
|
#
|
|
# Regenerate asset/nginx/blackbeard-deny.conf: the networks the public vhost
|
|
# refuses, as an nginx `geo` map from the prefixes each ASN announces.
|
|
#
|
|
# ./script/deny-networks.py > asset/nginx/blackbeard-deny.conf
|
|
#
|
|
# Then review the diff and install it with `infra-setup.sh --role edge`. The
|
|
# output is committed rather than fetched at install time so that what the edge
|
|
# refuses is something a reviewer has read, and so a RIPEstat outage cannot turn
|
|
# an install into an empty list.
|
|
#
|
|
# Hosting networks only. AS35540 (OVH Telecom) is left out on purpose: it is
|
|
# OVH's residential broadband, and the point is headless pollers, not people.
|
|
# See blackbeard/observer#24 for the log analysis that justified each entry.
|
|
|
|
import datetime
|
|
import ipaddress
|
|
import json
|
|
import sys
|
|
import urllib.request
|
|
|
|
ASNS = {
|
|
24940: "Hetzner Online (dedicated)",
|
|
213230: "Hetzner Cloud",
|
|
212317: "Hetzner Cloud (US, Singapore)",
|
|
16276: "OVH SAS (hosting)",
|
|
}
|
|
|
|
SOURCE = "https://stat.ripe.net/data/announced-prefixes/data.json?resource=AS{}"
|
|
|
|
|
|
def announced(asn):
|
|
with urllib.request.urlopen(SOURCE.format(asn), timeout=60) as r:
|
|
body = json.load(r)
|
|
if body.get("status") != "ok":
|
|
# A failed query must not produce a file that quietly stops denying a
|
|
# provider — the diff would show a deletion nobody asked for.
|
|
sys.exit(f"RIPEstat answered {body.get('status')!r} for AS{asn}")
|
|
nets = [ipaddress.ip_network(p["prefix"]) for p in body["data"]["prefixes"]]
|
|
if not nets:
|
|
sys.exit(f"AS{asn} announces nothing; refusing to write an empty entry")
|
|
# Collapsing removes duplicates and more-specifics; nginx's geo warns on a
|
|
# duplicate network and the file is smaller for it.
|
|
v4 = ipaddress.collapse_addresses(n for n in nets if n.version == 4)
|
|
v6 = ipaddress.collapse_addresses(n for n in nets if n.version == 6)
|
|
return list(v4) + list(v6)
|
|
|
|
|
|
def main():
|
|
today = datetime.date.today().isoformat()
|
|
out = [
|
|
"# GENERATED by script/deny-networks.py — do not edit by hand; regenerate.",
|
|
f"# Announced prefixes per RIPEstat, {today}.",
|
|
"#",
|
|
"# Installed to /etc/nginx/conf.d/blackbeard-deny.conf. `$blackbeard_denied`",
|
|
"# is 1 for a client inside any network below; the public vhost refuses it.",
|
|
"# `geo` reads $remote_addr, which conf.d/proxy-protocol.conf has already",
|
|
"# restored to the client's address — without that every request would come",
|
|
"# from 127.0.0.1 and nothing would ever match.",
|
|
"",
|
|
"geo $blackbeard_denied {",
|
|
" default 0;",
|
|
]
|
|
for asn, name in ASNS.items():
|
|
nets = announced(asn)
|
|
out.append("")
|
|
out.append(f" # AS{asn} {name}: {len(nets)} networks")
|
|
out.extend(f" {n} 1;" for n in nets)
|
|
out.append("}")
|
|
print("\n".join(out))
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|