Maximum likelihood and score equations

Define an MLE as a parameter value attaining the supremum of the likelihood over the permitted parameter space. Where the likelihood is positive, maximizing its logarithm is equivalent because the logarithm is increasing. An interior differentiable maximum has zero score, but that necessary condition is not sufficient. Examine concavity, compare candidate values, and check boundaries. A closed parameter space can contain a boundary maximum even when no score root exists in its interior; an open parameter space may have only an unattained supremum.

For Bernoulli data with both successes and failures, the log likelihood is strictly concave on the open probability interval and its score root is the sample proportion. If every observation is a success, the likelihood increases toward one. On $[0,1]$, the MLE is one; on $(0,1)$, no maximizer exists. This difference is a parameter-space decision, not a numerical convergence failure. Likewise, the uniform upper-endpoint MLE is the sample maximum, obtained by respecting the support rather than solving an interior score equation.

For positive exponential waiting times with rate $\lambda$, the log likelihood is $n\log\lambda-\lambda S$ plus a data-only term, where $S$ is the sum. When $S>0$, the unique maximizing rate is $n/S$: the second derivative is negative everywhere, and the log likelihood tends to minus infinity at either end of the positive rate domain. The reciprocal of the sample mean is a rate, so its physical units are inverse time. Confusing a rate with a mean waiting time reverses the interpretation.

MLE construction does not impose unbiasedness. For iid exponential observations and $n>1$, the sum has gamma shape $n$ and rate $\lambda$, yielding $E[1/S]=\lambda/(n-1)$. Thus the rate MLE has expectation $n\lambda/(n-1)$, larger than its target. Multiplying it by $(n-1)/n$ removes that bias. This correction is a finite-sample calculation and depends on the exponential model. Whether to use it depends on the chosen loss and other requirements, not solely on the label maximum likelihood.

Here is a precise route to the usual scalar limit, stated as sufficient assumptions rather than a universal theorem about MLEs. Suppose observations are iid from a correctly specified identifiable model at an interior true value. Assume a consistent sequence of interior score roots exists, log densities are twice continuously differentiable on a common neighborhood with fixed support, the true score has mean zero and positive finite variance $I$, and the normalized negative Hessian converges uniformly in probability to $I$ on shrinking neighborhoods containing the roots and truth. Assume the iid score central limit theorem applies.

Taylor's theorem then writes the estimation error as the average score divided by a nearby normalized negative Hessian. The score central limit theorem and convergence of that denominator give a centered normal limit with variance $I^{-1}$ after multiplication by $\sqrt n$. This is an asymptotic statement for a fixed regular true parameter, not exact normality at a chosen sample size or uniform validity near boundaries. Misspecification, weak identification, or nonexisting finite maximizers require other arguments; maximizing software cannot establish those hypotheses on its own.