Availability depends on the work accepted before rejection
Budget CPU, retained state, and downstream work under untrusted load while distinguishing bounded admission from unlimited queueing.
On this page
The question for today
What you’ll understand
- Calculate a CPU demand budget with consistent resource-time units.
- Explain why small unauthenticated requests can trigger larger service costs.
- Design measurements for admission limits without confusing rejection with free work.
A service can reject every malicious request and still become unavailable if rejection consumes the scarce resource. Connection state, parsing, cryptographic work, database access, logging, and response generation can all occur before the final denial. Denial-of-service analysis therefore asks how much work a party can force the system to accept relative to its own effort. Model the expensive stage, define a finite admission boundary, and verify what legitimate users experience while the boundary is under pressure.
Count work in the resource that saturates
A byte-rate graph cannot reveal every availability bottleneck. A small handshake attempt can consume CPU; a partial request can retain memory; an authenticated request can trigger expensive downstream fan-out. For CPU, multiply accepted request rate by CPU-seconds per request and compare the result with available CPU-seconds per wall-clock second. For memory, count retained objects and their actual size and lifetime. Keep legitimate and rejected work in the accounting, since a fast error path is still a path with cost.
Resource asymmetry does not require a large response packet. Parsing complexity, verification, cache misses, and dependency calls can amplify small inputs into substantial internal work. Conversely, a cheap precheck can sometimes avoid a more expensive stage, provided the precheck itself has bounded cost and does not become an unchecked state allocation. Measure stages separately under an explicitly authorized synthetic workload. The examples here are capacity arithmetic, not instructions for generating harmful traffic against someone else's service.
Admission and concurrency controls bound different things
A rate limit bounds admissions over a stated time policy; a concurrency limit bounds simultaneous in-flight work. They answer different questions when request durations vary. An unbounded queue can convert a short overload into long tail latency and memory pressure. A finite queue, deadlines, cancellation, and load shedding can limit retained work, but each needs a defined response and retry policy. Retrying denied work immediately can increase offered load and make a local protective action worsen the wider system.
Identity-based quotas can separate customers better than a single source-IP bucket when many legitimate clients share a NAT, but verified identity may be unavailable before an expensive handshake or authentication stage. Use layered limits appropriate to what is known at each boundary. Also bound log volume and dependency fan-out on rejected requests. A rejection counter rising while CPU and downstream traffic continue to scale unchecked is evidence that the intended cost boundary has not yet been achieved.
Test useful service, not only refusal volume
A protective mechanism succeeds only relative to an availability objective. Track successful legitimate throughput, latency, fairness, rejected work cost, resource saturation, and recovery after the synthetic load ends. Use bounded tests and explicit stop conditions in an environment authorized for the experiment. A system that returns many errors quickly may still starve legitimate requests sharing a worker pool. Inspect whether admission checks, cryptographic workers, and application work share the same exhausted resource.
The queue lab provides a small fixed-rate comparison. Reset, set both λ=4 and μ=3, predict that the four initial work items grow by one per second, then Play or Step. Reset again, set both λ=2 and μ=3, and predict drainage to zero at four seconds before playing. These are separate runs: parameter edits retain elapsed time and recompute from the initial four. The fluid model has no attacker, finite buffer, CPU scheduler, or admission limiter, and its M/M/1 readout requires additional stochastic assumptions not established by this exercise.
Work it through
Budget a fictional verification stage
An isolated service has four ideal CPU cores for a verification stage. It admits 1,000 requests per second, each costing 2 ms of CPU time. Estimate utilization, then compare 2,500 admitted requests per second.
Convert the per-request cost: 2 ms is 0.002 CPU-seconds. At 1,000 requests/s, demand is 1,000 × 0.002 = 2 CPU-seconds per wall-clock second.
Divide by the four available CPU-seconds per second: modeled utilization is 2/4 = 0.5, or 50%. At 2,500 requests/s, demand becomes five CPU-seconds per second, exceeding the four-core budget.
The second case has modeled utilization 1.25, so sustained processing cannot keep up under these assumptions. A real limit needs room for legitimate work, variability, and other stages; the arithmetic is not a recommended operating threshold.
The base stage demand uses half the idealized budget, while the larger admission rate demands 125%. Denying requests only after this full cost would not make overload disappear.
The living notebook
Explore the model
Interactive lab · queue
Bound accepted work before backlog dominates
Compare fresh Reset → set both λ=4,μ=3 and Reset → set both λ=2,μ=3 cases. Predict slope and four-item drain time, then Play or Step. The fluid model has no real admission policy, attacker traffic, or finite-buffer behavior.
- Fluid backlog
- 4
- Net rate · λ − μ
- −1 /s
- M/M/1 mean residence
- 1 s
Fluid backlog starts at 4 and follows max(0, 4 + (λ − μ)t). The separate M/M/1 residence-time formula assumes Poisson arrivals, exponential service and λ < μ. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forAvailability controls must bound expensive admitted work and preserve useful service, not merely produce error responses.
Pause & reason
Check your understanding
Your turn
Try it yourself
An authorized synthetic test admits 200 requests per second, with average admission-to-release residence time 0.3 seconds. Estimate average in-flight work, then explain why that number alone cannot set a safe hard concurrency cap.
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Use a consistent Little's-law boundary and distinguish an average from the upper tail of a variable workload.
Compare with the solution
The stable average is 200 × 0.3 = 60 in-flight requests. A hard cap of sixty would not automatically preserve the target service level because arrivals and durations vary, other resource demands may dominate, and legitimate fairness matters. Measure the occupancy distribution, deadlines, per-request resource cost, and useful throughput under the intended admission policy before selecting a finite cap and retry behavior.
Keep the idea
Take it with you
- Rejected requests still consume whatever work precedes the rejection boundary.
- CPU demand, concurrent state, queue length, and byte rate need distinct budgets.
- Protection is evaluated by bounded resource use and legitimate service under load.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.