Count useful sparse work and the metadata around it

Compressed sparse formats store values together with indices, offsets, or masks. For small values or very short rows, this metadata can rival the useful numeric payload. The right representation depends on sparsity structure, dimensions, and available decoder hardware. Structured blocks may reduce metadata and improve vectorization while retaining some zero arithmetic. Unstructured sparsity may skip more mathematical work but require irregular gather, accumulation, and routing. Include conversion and packing costs if the application does not already produce that format.

Sparse work can also be distributed unevenly. Equal numbers of rows do not imply equal nonzeros or equal accumulation cost, so one processing element can determine completion while others wait. SCNN and EIE demonstrate specific compressed execution mechanisms and must be read with their operator and precision scope attached. Report actual executed operations separately from dense-equivalent operation counts. Calling skipped operations executed throughput can make an architecture appear to exceed a physical limit without explaining the counting convention.

For a fixed collection of candidate multiply pairs, the fraction with both operands nonzero is a joint probability. Multiplying weight and activation densities is valid only with an appropriate independence assumption for those pairs. Correlated masks can change useful work substantially. Record measured joint patterns by layer and input regime, not just global density. The scheduler may need extra capacity for bursts, and a static average can understate the worst hot-spot load.

An event-driven spiking system has several distinct counters: input spikes, routed packets and hops, destination updates, and the decision’s integration window. One spike can fan out to many targets. Background state updates, synchronization, and idle power can remain between spikes, so inactivity is not automatically free. The Loihi architecture paper describes a particular event-based design; its per-operation and simulation-window quantities cannot be read as whole-system inference energy or wall-clock decision latency without the relevant execution boundary.

Sparsity changes produced through pruning, thresholds, quantization, or event encoding can alter model quality. Evaluate that change using the same dataset and task requirement as the dense or conventional baseline. Include any retraining and conversion process in the released recipe. A sparse kernel benchmark at a fixed mask is useful for mechanism analysis, while an application claim needs representative input-dependent masks and complete inference behavior.

Measure the costs that survive zero skipping: index decoding, memory transactions, irregular reductions, communication, and load imbalance. Report latency distributions when sparse patterns vary per request, and separate low-load efficiency from saturated throughput. For event systems, state the decision window and count all messages and updates needed for that decision. A matched comparison may reveal a useful sparse mapping, but density alone cannot choose the winner without the surrounding representation and scheduling evidence.