Expert capacity: distinguish a dropped branch from a rerouted token

For T eligible tokens with exactly k distinct requested experts each, there are Tk requested assignments before overflow. Dividing by E experts gives the mean requested load. A teaching fixed-capacity rule can multiply that mean by a capacity factor and round upward. Real implementations may impose a minimum, alignment, per-rank scope, or a different normalization; inspect those details. Megatron-Core 0.15.0 documents a capacity expression using tokens per rank and top-k, whereas an aggregate expert-parallel batch would use a different T.

A total allocation of E times capacity does not ensure every expert fits. Load can concentrate on one destination while other slots stay empty. Padding reserves or computes space for missing assignments; it does not redistribute a busy expert's work. Capacity-factor comparisons must report the population used in the denominator and whether capacity is applied before or after exchange. A change in local batch size, group partition, or microbatch accumulation can alter overflow behavior without changing the global training-token count.

In a simple branch-dropping policy, an overflowing token-expert assignment contributes no expert result. A residual path may still carry the token representation, as in the original Switch architecture; the whole token is not necessarily removed from the sequence. Under top-two routing, a token can retain one branch while losing the other. Whether remaining weights are renormalized changes its output. Admission can prioritize token position, routing score, or randomized selection, producing different survivors even when accepted counts are identical.

Rerouting attempts a different expert and therefore changes expert identity, gate interpretation, communication, and potentially the learned function. It is not equivalent to padding spare capacity elsewhere. GShard's original top-two procedure also includes a stochastic second-route mechanism; do not label every top-two implementation deterministic selection of the two highest probabilities. Dropless methods instead allow requested work to be represented without token dropping, using variable work or suitable sparse kernels. MegaBlocks is one such design, but dropless execution still has finite memory and can suffer skew.

Record requested and accepted counts, partial-token drops, complete-token branch loss, and padding separately. Slice those counters by sequence position or data group to detect a positional admission bias. Compare training quality at matched token budgets and meaningful compute boundaries; a faster step that skips more requested computation is not automatically a better sparse model. Reproduce the exact train and evaluation capacity policies because some APIs expose separate factors. A no-drop evaluation path can execute a function different from a dropping training path.

The queue lab represents one constrained work stream, not a routing batch. Reset, set λ=4 and μ=3, predict backlog six at t=2, then use four Steps to reach that time. Separately Reset, set λ=2 and μ=4, predict the initial backlog reaches zero at t=2, then use four Steps. The initial backlog is four, and these are fluid work units rather than token assignments. The instrument has no capacity cutoff, admission priority, or expert reassignment; calculate the discrete overflow on paper instead of interpreting a falling curve as rerouting.