Per-account top-up overrides, so one workload's appetite doesn't set everyone's limits #276

New Issue

grenade · 2026-08-19T17:09:30Z

grenade commented

2026-08-19 17:09:30 +00:00

Self-service top-up limits live in app_config and are global:
topup.auto.grant_tokens, .cooldown_secs, .max_per_account,
.threshold_pct. Tuning them for one account tunes them for every
account.

That was acceptable on 2026-08-19 — lairball needed ~3M tokens/day
(measured: 1.46M 14-day mean, 2.80M peak) and the shipped values allowed
three top-ups ever at 1M each with a 24-hour gap, so it would have
stopped inside a week. The global values were raised to grant_tokens = 3000000, cooldown_secs = 3600, max_per_account = 1000, which fits
lairball comfortably.

The cost is that every account now gets 3M-token grants on a 1-hour
cooldown. Nothing is contending for allocation today, so this is a
deliberate, cheap trade rather than a problem. It stops being cheap the
moment there is a second account with a different profile — an
interactive user who should not be able to mint 3M tokens hourly, or a
batch workload that needs more than lairball.

Scope

Per-account overrides for the four top-up settings, falling back to the
global app_config value when unset. Shape roughly:

ALTER TABLE accounts
  ADD COLUMN topup_grant_tokens   BIGINT,   -- NULL = use global
  ADD COLUMN topup_cooldown_secs  BIGINT,
  ADD COLUMN topup_max_per_account BIGINT,
  ADD COLUMN topup_threshold_pct  BIGINT;

topup.rs already reads all four through config_store::get_i64 in two
places (can_auto_top_up and the grant path), so the resolution point is
small and singular: read the account's override, fall back to global.
Worth mirroring config_store's clamping so an override cannot exceed
the schema bounds either.

Two things to get right

Clamping is silent. config_store::get_i64 clamps to the row's
declared min_value/max_value and says nothing. Setting
max_per_account = 1000000 today yields 1000 with no error and no log
— it looks applied and is not. Per-account overrides should either log
when they clamp, or reject out-of-bounds values at write time. Silent
clamping on a budget control is the same class of defect as #271's
silently-dropped sampling parameters.

max_per_account cannot express "unlimited". Its bound is 1000 and
0 already means "no top-ups", so there is no sentinel. At lairball's
3M/day with 3M grants that is one grant/day and therefore ~2.7 years
before it needs an operator-minted code — fine, but finite, and it will
expire silently on a workload nobody is watching. Either widen the bound
with an explicit -1 = unlimited sentinel (widening min_value to -1),
or accept the ceiling and add an alert when an account approaches it.

Priority

Not urgent — deferred until there is contention. Filed so the global
values raised today are a recorded trade rather than an accident, and so
whoever adds the second heavy account finds this instead of rediscovering
it.

Related: #47 (governance epic), #257 (KV budget — the other place a
per-account dimension will eventually be wanted).

Self-service top-up limits live in `app_config` and are **global**: `topup.auto.grant_tokens`, `.cooldown_secs`, `.max_per_account`, `.threshold_pct`. Tuning them for one account tunes them for every account. That was acceptable on 2026-08-19 — lairball needed ~3M tokens/day (measured: 1.46M 14-day mean, 2.80M peak) and the shipped values allowed three top-ups ever at 1M each with a 24-hour gap, so it would have stopped inside a week. The global values were raised to `grant_tokens = 3000000`, `cooldown_secs = 3600`, `max_per_account = 1000`, which fits lairball comfortably. The cost is that **every** account now gets 3M-token grants on a 1-hour cooldown. Nothing is contending for allocation today, so this is a deliberate, cheap trade rather than a problem. It stops being cheap the moment there is a second account with a different profile — an interactive user who should not be able to mint 3M tokens hourly, or a batch workload that needs more than lairball. ## Scope Per-account overrides for the four top-up settings, falling back to the global `app_config` value when unset. Shape roughly: ```sql ALTER TABLE accounts ADD COLUMN topup_grant_tokens BIGINT, -- NULL = use global ADD COLUMN topup_cooldown_secs BIGINT, ADD COLUMN topup_max_per_account BIGINT, ADD COLUMN topup_threshold_pct BIGINT; ``` `topup.rs` already reads all four through `config_store::get_i64` in two places (`can_auto_top_up` and the grant path), so the resolution point is small and singular: read the account's override, fall back to global. Worth mirroring `config_store`'s clamping so an override cannot exceed the schema bounds either. ## Two things to get right **Clamping is silent.** `config_store::get_i64` clamps to the row's declared `min_value`/`max_value` and says nothing. Setting `max_per_account = 1000000` today yields `1000` with no error and no log — it looks applied and is not. Per-account overrides should either log when they clamp, or reject out-of-bounds values at write time. Silent clamping on a budget control is the same class of defect as #271's silently-dropped sampling parameters. **`max_per_account` cannot express "unlimited".** Its bound is 1000 and `0` already means "no top-ups", so there is no sentinel. At lairball's 3M/day with 3M grants that is one grant/day and therefore ~2.7 years before it needs an operator-minted code — fine, but finite, and it will expire silently on a workload nobody is watching. Either widen the bound with an explicit `-1 = unlimited` sentinel (widening `min_value` to -1), or accept the ceiling and add an alert when an account approaches it. ## Priority **Not urgent — deferred until there is contention.** Filed so the global values raised today are a recorded trade rather than an accident, and so whoever adds the second heavy account finds this instead of rediscovering it. Related: #47 (governance epic), #257 (KV budget — the other place a per-account dimension will eventually be wanted).

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: helexa/helexa#276