Utilization: name the useful work and the available capacity
Model committed capacity and useful resource-time while separating allocation, busy hardware, and service-feasible utilization.
On this page
The question for today
What you’ll understand
- Define available, allocated, busy, and productive resource-time separately.
- Calculate fictional cost per productive resource-hour.
- Explain why utilization and latency constraints must be considered together.
A device can be allocated, busy, and still produce no accepted progress. Retried training work, invalid inference outputs, and padding can raise hardware activity without increasing the useful denominator. All prices here are explicitly fictional stable teaching inputs in fictional credits, FC. Our model prices a fixed quantity of available device-hours over a defined period, without assuming that a real discount product guarantees physical capacity. Reservation, commitment, scheduling request, and actual availability are different concepts whose specific guarantees must be read from the relevant system and contract.
Define a denominator that matches the question
Available capacity is the resource-time that the chosen pool could supply over the measurement horizon under the stated availability boundary. Allocated time is what workloads hold; busy time measures selected hardware activity; productive time credits work that satisfies a specified acceptance rule. These quantities can differ substantially. For training, a replayed failed interval can count as busy while adding no new durable progress. For serving, an output outside the quality or deadline requirement can consume capacity without qualifying as successful work.
State whether maintenance, failures, and quarantined devices remain in the available denominator. A pool that excludes all downtime can look highly utilized while delivering less total capacity. Compare like definitions and retain the raw counts. Kubernetes resource requests and limits provide scheduling and enforcement inputs, not automatic measurements of useful work or invoice quantities. Similarly, a commercial billing commitment and a scheduler reservation are not interchangeable promises. Record the actual capacity guarantee separately from the rate or allocation policy used for accounting.
Separate a lower unit cost from lower total spending
Suppose the fictional pool costs a fixed rate per available device-hour throughout the period. More productive work on that unchanged pool reduces cost per productive hour because the same numerator is divided by a larger accepted denominator. It does not reduce the period’s pool bill. To claim actual savings, identify avoided capacity, a shorter billed lifecycle, or another changed charge. To claim increased capability, show that additional useful work still meets quality, isolation, and deadline requirements on the unchanged resource budget.
High average utilization also leaves less room for bursts, failures, and heterogeneous requests. A queue can become slow before a resource appears continuously busy, and a short measurement interval can hide demand peaks. Resource vectors matter: a job can exhaust memory or bandwidth while arithmetic units remain underused. Original multi-resource allocation research treats these constraints explicitly; a single percentage is not a complete capacity plan. Keep headroom policy and service objectives visible rather than make utilization maximization the sole operational target.
Test useful utilization under the service constraint
Instrument workload classes, accepted work, allocation events, and resource bottlenecks together. Report whether a utilization increase came from better packing, more demand, longer queues, changed acceptance, or slower individual work. A system can become busier because an inefficient kernel takes longer; that is not an efficiency gain. Evaluate matched output and service objectives, including rejected traffic and degraded work. For a reservation model, include what happens when borrowed capacity is reclaimed and whether the guaranteed workload can still be served.
The queue lab illustrates one cost of approaching a service ceiling. Reset, explicitly set arrival rate two and service rate three, predict, then Play or Step. Compare a separate fresh Reset with arrival rate 2.5 and service rate three explicitly set. Its displayed M/M/1 mean residence times are one and two seconds under Poisson arrivals and exponential service. Utilizations in that separate stochastic formula are two thirds and five sixths. The animated backlog is a deterministic fluid toy starting at four, not an M/M/1 simulation, billing model, reservation scheduler, or measurement of a real device.
Work it through
Higher useful utilization on an unchanged pool
Eight devices are available for ten hours each. The fictional rate is 1.2 FC per available device-hour, with no other charges. Compare forty and sixty productive device-hours under the same acceptance rule.
Available capacity is 8×10=80 device-hours and the period costs 80×1.2=96 FC. Forty productive device-hours give u=40/80=0.5; seventy busy device-hours would not change that useful denominator.
At forty productive hours the allocated pool cost is 96/40=2.4 FC per productive device-hour. At sixty productive hours utilization is 0.75 and that ratio is 96/60=1.6 FC per productive device-hour.
Both cases still cost 96 FC. The second delivers more useful work from the same fictional pool; calling the ratio reduction a thirty-two-FC bill reduction would invent a spending change that the model does not contain.
Useful unit cost falls while total pool cost stays fixed. A real comparison must also preserve service quality and feasible headroom.
The living notebook
Explore the model
Interactive lab · queue
Higher offered load can consume latency headroom
For each independent run use Reset → explicitly set BOTH λ and μ → predict → Play/Step: (2,3), then (2.5,3). Displayed M/M/1 mean residence is one versus two seconds under its assumptions; animated q is a separate fluid toy with initial four. No reservation cost or useful-work acceptance is modeled.
- Fluid backlog
- 4
- Net rate · λ − μ
- −1 /s
- M/M/1 mean residence
- 1 s
Fluid backlog starts at 4 and follows max(0, 4 + (λ − μ)t). The separate M/M/1 residence-time formula assumes Poisson arrivals, exponential service and λ < μ. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forEvaluate utilization together with accepted output and latency constraints.
Pause & reason
Check your understanding
Your turn
Try it yourself
Four devices are available twelve hours each at a fictional rate two FC per available device-hour. Compute cost per productive hour for twelve and twenty-four productive hours.
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
The same forty-eight available device-hours appear in both numerators.
Compare with the solution
The pool costs 4×12×2=96 FC. Twelve productive hours give utilization 0.25 and cost eight FC per productive hour. Twenty-four give utilization 0.5 and cost four FC per productive hour. The total remains 96 FC, so no bill reduction has been demonstrated. Confirm that the extra useful work meets the same acceptance and latency rules and that unavailable time was handled consistently in both denominators.
Keep the idea
Take it with you
- Keep allocated, busy, and productive resource-time distinct.
- More useful work can lower unit cost without lowering the bill.
- Preserve latency, headroom, and multi-resource feasibility.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.