Choose between learning an effect and allocating reward

In a stationary stochastic bandit, each arm has an unknown expected reward, and a standard objective compares the learner’s cumulative reward with always selecting the best arm. Exploration can incur immediate cost but reveal which arm is better. Original finite-time bandit results analyze that tradeoff under specified reward and stationarity assumptions. They do not automatically provide a conventional A/B confidence interval or protect arbitrary operational guardrails.

A fixed experiment can instead hold assignment probabilities stable and estimate a prespecified contrast over a defined population and horizon. A best-arm identification study has another objective: select an arm with a controlled error or information budget. These tasks should not be collapsed into the assertion that a bandit is always a faster A/B test. Changing the objective changes the relevant guarantee, sample allocation, stopping rule, and decision evidence.

Record the probability of the selected action conditional on the information available at that decision, including prior history, context, eligible action set, and algorithm version. An overall fraction of A actions is generally not that conditional probability. Maintaining exploration supports later evaluation, while greedy zero-probability decisions can remove support for alternatives. Delayed outcomes complicate adaptation because the learner may preferentially react to fast-arriving rewards rather than completed horizon outcomes.

Unweighted arm averages can compare different calendar or context mixtures when allocation changes. Even randomized choices at each time do not make the eventual A and B observations identically distributed if their sampling times differ and the baseline drifts. Appropriate weighting, model adjustment, and adaptive inference can address specific designs under assumptions, but a fixed-trial standard error pasted onto the adaptive log does not inherit those guarantees.

If assignment is user-sticky, an adaptive enrollment policy may change probabilities for newly enrolled users while keeping existing users in their original arm. A per-request bandit implements a different intervention with potential carryover. Log those distinctions instead of calling both a traffic ramp. Preserve minimum exploration, guardrail logic, and the full propensity trajectory when they are part of the design, and evaluate outcome-maturity handling before allowing updates.

Bandit results should report cumulative reward or regret under the stated model, deployment-policy quality, and uncertainty from a suitable adaptive method when causal claims are made. A high observed mean for the heavily favored arm is not proof it would win under a common population distribution. Retain a fixed randomized holdout when the product question requires one, or use an explicitly justified alternative design; a changing allocation strategy is a choice with a different evidence contract.