Optimize completed work rather than the smallest wattage

The pinned NVML R535 command interface exposes power-management limits on supported hardware, with allowed constraints and permissions that must be checked. A requested limit is not a guarantee that every instantaneous sample equals that value or that the board always operates at the limit. Read back the relevant configuration, check return codes, and measure actual consumption. Record the device identity, driver, permitted range, and policy; otherwise a failed or overridden request can masquerade as an experimental setting.

Operating response depends on the workload and the hardware’s control mechanisms. Reducing a limit can change clocks or voltage and can interact with temperature, memory traffic, and contention. A compute-heavy phase and a bandwidth-heavy phase need not respond alike. Run enough completed work to observe the relevant phases, and inspect throttling or thermal conditions rather than assuming a universal percentage relationship between the requested cap and throughput. Controlled settings should not require invented device specifications.

For fixed work and a fixed boundary, the ratio of new to old energy is the power ratio multiplied by the runtime ratio. A one-quarter power reduction accompanied by a runtime increase greater than one-third raises energy under that simple comparison. Use actual averages rather than nameplate limits. Including a steady host load can reduce or reverse an accelerator-only benefit because every additional second incurs that host load, even if the device itself becomes more efficient.

A synthetic performance ceiling can explain why some resource changes fail to help without predicting watts. A memory bottleneck may keep throughput flat while additional compute capacity becomes available. Conversely, a cap can hurt a critical phase disproportionately even if average utilization looks modest. These mechanisms motivate measurement; they do not identify an energy optimum from a utilization chart. Search supported settings with the same representative workload and report where a bottleneck or service constraint changes.

An energy minimum that violates a deadline is not a feasible setting for that service. Track tail latency, completed throughput, output quality, and failures alongside gross energy. Holding a fixed duration while allowing fewer completed requests changes the denominator and may reward dropping work. Holding fixed output count but allowing different model quality changes the service. Declare which dimensions are matched and which are constraints before looking at the energy results.

Repeat matched runs with controlled warmup and ordering so thermal drift, background jobs, or cache state do not become confounded with policy. A short low-limit run and a long already-hot baseline are not an informative pair. Preserve raw timings and energy windows as well as aggregates. When several settings have similar measured energy, uncertainty and operational robustness can matter more than a nominal minimum whose tiny advantage disappears under another representative input mix.