Maximum Likelihood Beta Estimates

Using the argument from section 1.1.1, we can see that the likelihood equation of a binomial model has a Beta distribution. The method used in the last section began with the statistics $\hat p$and $\hat\sigma_p^2$ and came up with estimates a and b for a (Beta) posterior distribution of the parameter p.

The method is ostensibly contrived because instead of beginning with an explicit Binomial distribution $\rm {Bin}(n,\hat p)$ of the data, the parameter n has to be estimated with the constraint that the subsequent beta likelihood has a variance equal to $\hat\sigma_p^2$.

A more direct solution to this problem is to find the maximum likelihood equations for parameters a and b and use those to estimate the distribution of the parameter p. This approach is difficult to implement, and it suffers from the problem that it requires the sufficient statistics $\sum{\ln \hat p_i}$ and $\sum{\ln (1-\hat p_i)}$. The only statistics we have to work with, however, are $\bar p = \sum \hat p_i$ and $\hat\sigma_p^2 = \hat {\rm {Var}} (p).$ This makes maximum likelihood impractical. However, it can be usefull in evaluating other approaches to this problem.

Johnson & Kotz[#!kotz!#] state the maximum likelihood estimators of a and b as the pair of equations

$$\Psi(\hat a) - \Psi(\hat a + \hat b) = {1 \over n} \sum \ln p_i$$ (12)

$$\Psi(\hat b) - \Psi(\hat a + \hat b) = {1 \over n} \sum \ln (1-p_i)$$ (13)

where $\Psi(\cdot)$ is the digamma function

$$\Psi(x) = { {\Gamma^\prime (x)} \over {\Gamma(x)}}$$ (14)

The solution of these equations must be done numerically. (And presently, I've depended on Mathematica to find them.)