Bayes' rule and prior predictive models

Write the unknown parameter as $\theta$ and observations as $y$. A proper prior integrates to one over its declared support, and the sampling distribution integrates or sums to one over possible observations for each parameter. Their product defines the joint distribution. A likelihood holds the observed data fixed and varies the parameter; it generally does not integrate to one over that parameter. Multiplying by a constant independent of the parameter preserves likelihood ratios within this model but may destroy its interpretation as a normalized data distribution.

Conditional independence must be stated rather than inferred from a convenient product. Repeated trials may be independent given one shared $\theta$ while remaining dependent after that shared parameter is integrated out. This explains why uncertainty about a common failure rate can induce dependence between future failures. A model with a new independent rate for every trial has different predictive implications. Decide which experiment you mean before evaluating a likelihood, and record selection or censoring rules when they affect what observations can appear.

For observed data with $0<m(y)<\infty$, division by the marginal density produces a proper posterior. With continuous observations, $m(y)$ is a density, not the probability of observing one exact real number. Conditional densities are defined up to appropriate null sets, so pointwise formulas are understood where this denominator is legitimate. A posterior probability is conditional on both the observed information and the model assumptions. It is not automatically a statement about the long-run coverage of a confidence procedure at each fixed parameter.

A prior that assigns an event zero probability cannot acquire positive posterior probability through a finite, positive likelihood update. That is a support choice, not strong evidence manufactured by the data. Improper priors require separate proof that the posterior normalization is finite; they do not themselves define normalized prior predictive distributions. Parameter changes also matter: a density on a transformed scale requires its Jacobian. A flat density in one coordinate system generally becomes nonflat in another, so the word uninformative does not remove modeling decisions.

Prior predictive checking asks what data the entire prior and sampling model jointly expect. Generate a parameter from the prior, then a complete data set conditionally on that parameter; inspect meaningful summaries such as extreme counts or differences between groups. A broad prior on a coefficient can imply implausibly enormous observations after a nonlinear transformation. Looking only at the parameter scale can miss this. The purpose is to reveal implications, rather than to tune a prior until it guarantees the desired posterior answer.

After observing data, repeat the same generative idea with the posterior to predict future observations. Keep parameter uncertainty separate from observation noise: even exact knowledge of a failure probability does not determine the next binary outcome. Prior predictive checks also do not prove that the model is correct. A plausible distribution of total counts can conceal implausible time ordering or dependence. Choose summaries tied to the application, document any revisions prompted by existing data, and examine sensitivity to reasonable alternative priors.