Block a user
Per-account top-up overrides, so one workload's appetite doesn't set everyone's limits
S1 — sampling defaults from generation_config, plus top_k, seed and tunable repetition
[epic] helexa.ai/games — nightly generative benchmark scored by people playing the results
[epic] Sampling correctness: honour the caller's parameters and the model's own generation_config
Converted to an epic — children filed
Sized against the code: 8 LogitsProcessor construction sites, 4 places
extracting sampling fields from a request, 6 signatures threading them as
loose…
S3 — stop lying about unsupported parameters: reject, and advertise what we support
S2 — stop sequences, and the penalties that need per-token logit work
S1 — sampling defaults from generation_config, plus top_k, seed and tunable repetition
Sampling correctness: standard inference parameters are silently ignored, and the model's own generation_config is never read
A streamed turn logs nothing when it ends, so it cannot be diagnosed afterwards
Prefix cache savings are invisible:
cached_tokens is never reported, so every client shows 0% cache hit