rob thijssen
243b17e9b3
Two fixes surfaced by the live feed. modelwatch: some repos report a bogus tiny `safetensors.total` (e.g. deepreinforce-ai/Ornith-1.0-35B came through as "1M params"), which both misrendered and — worse — let an oversized model slip past the size cap. Add `params_from_name` (parse the first <num>B|M token from the repo id, which by convention is the total size) and `effective_params` (trust safetensors.total only when it's >= 100M, else fall back to the name). Use it for the size filter and the summary. New unit tests cover the name parser, the fallback, and that a 70B reporting a 1M total is still rejected as too large. (Forward-looking: the idempotent ingest won't re-render items already in the feed.) fetch: feed requests now send browser-shaped `Accept` / `Accept-Language` headers. reqwest sends no Accept by default, which some anti-bot filters 403. NB this is hygiene, not a Reddit fix — Reddit fingerprints the TLS/HTTP client (rustls) and hard-403s it where curl only gets rate-limited (429); headers don't change the fingerprint. The control path (github releases.atom, etc.) is unaffected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP
34 lines
768 B
TOML
34 lines
768 B
TOML
[package]
|
|
name = "newsfeed-modelwatch"
|
|
version.workspace = true
|
|
edition.workspace = true
|
|
rust-version.workspace = true
|
|
license.workspace = true
|
|
authors.workspace = true
|
|
description = "Push producer: watches Hugging Face Hub for open-weight model releases and submits filtered candidates to the newsfeed ingest endpoint."
|
|
|
|
[lib]
|
|
name = "newsfeed_modelwatch"
|
|
path = "src/lib.rs"
|
|
|
|
[[bin]]
|
|
name = "newsfeed-modelwatch"
|
|
path = "src/main.rs"
|
|
|
|
[dependencies]
|
|
newsfeed-entities.workspace = true
|
|
|
|
tokio.workspace = true
|
|
reqwest.workspace = true
|
|
|
|
serde.workspace = true
|
|
serde_json.workspace = true
|
|
chrono.workspace = true
|
|
regex.workspace = true
|
|
|
|
anyhow.workspace = true
|
|
tracing.workspace = true
|
|
tracing-subscriber.workspace = true
|
|
figment.workspace = true
|
|
clap.workspace = true
|