Discovery significance and look-elsewhere effects

Choose a test statistic for which larger values indicate a more signal-like result. The background p-value is the probability, assuming the background model and the declared procedure, of obtaining a statistic at least as large as observed. It is not the probability that background is true and not the probability that this particular excess was caused by noise. A posterior hypothesis probability would require additional model and prior specifications, including alternatives.

For a nonnegative signal-strength parameter, a common discovery statistic compares the best fit with the fit constrained to zero signal, and sets the discovery statistic to zero for a downward unconstrained signal estimate. The physical boundary changes its null distribution. Under suitable large-sample conditions, the one-sided local significance can be approximated by the square root of that statistic. This is a calibrated asymptotic result, not an algebraic identity for every likelihood or every event count.

A local probability treats a specified mass or signal model as fixed before observing the excess. A global probability instead asks whether the complete search would produce at least one excess of comparable strength anywhere in its declared domain. For K independent tests with the same local tail probability p, the probability that none crosses the threshold is the Kth power of one minus p. Taking its complement gives the global probability. This independence model is a transparent example, not a realistic description of adjacent highly correlated mass points.

Without independence, the union bound still limits the global probability by the sum of individual tail probabilities. A continuous scan requires its correlations and boundaries to be modeled; the number of grid points is not automatically the number of independent trials. Gross–Vitells methods use properties of the scanned test-statistic process under specified asymptotic assumptions. Pseudoexperiments can calibrate the maximum statistic directly when they repeat the full scan and nuisance procedure.

Background functional forms, event categories, mass windows, and cut choices can all create additional opportunities for a fluctuation to become the selected result. If choices were made after looking at the candidate signal, calibrating only the final fixed test understates the selection process. Predeclared procedures, blinded analyses, or independent development samples help make those choices explicit. They do not remove the need to validate background tails and nuisance models where the excess appears.

A significance describes incompatibility with a stated model, while discovery of a physical particle additionally requires scrutiny of instrumental effects, alternative backgrounds, reproducibility, and consistency across observables. Large data samples do not protect against a systematically wrong tail model. Conversely, a non-significant result does not establish the absence of every possible signal. Confidence limits, model comparison, expected sensitivity, and discovery p-values answer different questions and should be reported with their corresponding assumptions and search domains.