Marginal likelihood and Bayes factors

For model $M_k$, define a sampling density on a common observed-data space and a proper parameter prior. Integrate their product to obtain $m_k(y)$. Distinct models may have different parameter dimensions, but their marginal densities must refer to the same observed event or density convention. A model with no free parameters simply supplies its sampling density. Integration averages over the prior rather than taking the maximum; a parameter region with excellent fit may contribute little if it receives very little prior mass.

Use positive finite marginal densities when forming a finite Bayes factor. Continuous-data densities can exceed one and carry units, while a ratio between densities for the same observations and base measure is dimensionless. A common transformation of the data introduces a common Jacobian that cancels; unrelated density conventions do not. For a binomial experiment, the probability of an ordered sequence differs from the probability of its success count. A combinatorial factor cancels only if it is genuinely the same factor in both models.

Assign prior probabilities to the candidate models separately from their within-model parameter priors. Bayes' rule then shows that posterior model odds equal prior model odds times the Bayes factor. A factor greater than one favors the numerator model relative to the denominator, but does not specify its posterior probability without prior model odds. With more than two candidate models, normalize all prior-probability times marginal-likelihood products. The resulting probabilities are conditional on the candidate list; omitted possibilities do not receive probability automatically.

Prior scale can have a large effect because the marginal likelihood averages over all prior predictions, including predictions far from the observed data. Widening a proper prior may reduce this average even when the posterior near a well-supported mode barely changes. This is a reason to examine predictive implications and sensitivity, rather than to search for a universal diffuse prior. An improper prior has an arbitrary multiplicative constant, which generally remains in its marginal likelihood. Proper posterior estimation alone does not make an ordinary cross-model Bayes factor well-defined.

Terms constant in a model's parameter can be dropped for some posterior computations but cannot generally be dropped for marginal likelihoods. The same caution applies to normalized priors: their constants depend on prior scales. Numerical integration, specialized bridge methods, or carefully validated importance sampling may estimate marginal likelihoods. A stable-looking posterior chain is insufficient evidence that such an estimate is accurate, since marginal likelihood methods can depend strongly on regions that ordinary posterior summaries barely inspect.

A Bayes factor is not a posterior probability that a scientific theory is true, nor a measure of future predictive quality independent of the chosen prior. Predictive evaluation on held-out data answers a different conditioning question and may be preferable for a specific forecasting objective. For scientific comparison, report the data model, prior specification, direction of the ratio, prior odds if posterior probabilities are quoted, and sensitivity to reasonable alternatives. If both models fail predictive checks, strong relative evidence can still select an inadequate model.