Load skew: name the population behind every expert counter

For exactly k requested experts per eligible token, requested expert counts sum to Tk before capacity admission. Dividing by Tk gives each expert's share of assignments. Dividing by T gives assignments per token and produces fractions that sum to k; these are not interchangeable denominators. After dropping, accepted counts sum to a smaller total. Normalizing only by accepted work can hide how much demand was rejected. Record the original requested denominator alongside acceptance and overflow, with padding or masked tokens explicitly excluded or named.

Unique-token counters add information that expert counts cannot provide. A token may lose one branch, all branches, or none. Shared experts may execute regardless of routed admission and therefore need separate path counters. Scheduled rows can include alignment or capacity padding, whereas accepted rows represent useful assigned work. Collect these stages from the actual implementation rather than infer them from one returned tensor called expert_count. DeepSpeed 0.19.7's routing source, for example, exposes counts whose placement before capacity masking matters for interpretation.

A max-to-mean count ratio reveals a busiest expert relative to the average over the stated expert set. Include zero-count experts if they are eligible members; removing them makes apparent balance improve without changing the workload. The population coefficient of variation adds information about spread. Both require a positive mean, so an empty routing batch needs a separate not-applicable result. These statistics remain count summaries, not measures of specialization, fairness, hardware occupancy, or convergence quality.

A whole-step average can hide a badly skewed microbatch that caused an allocation spike or stalled a communication phase. Report distributions over layers and microbatches as well as totals, and identify whether reduction combines replicas, expert-parallel groups, or independent data shards. Per-rank totals add the loads of colocated experts and can expose a placement problem invisible to per-expert averages. Aggregate percentages should be computed from compatible summed numerators and denominators rather than averaging rates from batches with unequal populations.

Compare expert counts with measured expert durations, scheduled matrix rows, peer bytes, allocation peaks, and exposed layer time. Equal counts need not mean equal costs when widths, kernels, tensor-parallel layouts, or remote paths differ. Conversely, a count imbalance may be harmless if the relevant work overlaps and remains outside the critical path. Use a fixed routing trace to separate placement and kernel changes from a changing router. Evaluate data-group quality and admission rates if an optimization systematically affects certain token populations.

The probability lab gives an exact marginal under deliberately simple assumptions. If twenty independent top-one tokens choose a particular expert with probability 0.25, its count is binomial with mean five and variance 3.75. Reset, set p=0.25 and n=20, predict those values, then Play or Step. Separately Reset, set p=0.50 and n=20, predict mean ten and variance five, then Play or Step. Playback highlights exact bars rather than sampling training batches. Real tokens can be correlated, and expert counts in one multinomial batch are not mutually independent.