The Throttling Valve
Granular, tiered rate limiting that stops abuse without punishing legitimate users.
The incident
A client app shipped with a retry loop missing its delay. One buggy build turned every user's phone into a tiny DDoS cannon. The WAF's answer was to block IP ranges — which took out an office building's worth of legitimate customers along with the bug.
Why the obvious approach fails
Infrastructure-level rate limiting sees IPs, not identities — too blunt for a world of NATs, corporate proxies, and mobile carriers where one IP is a thousand users. And a hard global limit fails the other way: it can't distinguish a paying enterprise tenant's burst of legitimate traffic from a scraper. Rate limiting is an application-layer decision that needs application-layer keys — and it needs to speak a protocol clients can cooperate with, or well-behaved clients just look like retrying attackers.
The standard
- Token-bucket limiting keyed on the right dimension per route: user ID, API key, tenant ID — IP is the fallback, never the primary key.
- Tiered policies as configuration, not code: guest / authenticated / premium / internal each carry declared budgets.
- Responses carry X-RateLimit-Remaining and Retry-After — clients are told how to behave, and SDKs honor it.
- Limit events are observable per key and per tier; a spike in 429s is a signal to inspect, not just a wall.
How Cortex validates it
At plan time
Every public route in the blueprint declares its limiting key and tier budgets; an unprotected state-changing route fails plan review.
At validation
assert: public_routes.rate_limit != nullassert: 429_responses.include(Retry-After)Assertion syntax is illustrative.
At review
The approver sees the abuse simulation: burst traffic per tier, and proof legitimate-tier traffic was unaffected.