0

In many accounts of Bayesian inference the posterior is written as being proportional to the product of the likelihood and the prior:

$$ P(H \mid D) \propto P(D \mid H) P(H) $$

Why write it as a proportionality ? How does that impact actual estimation procedures? Why not work directly with Bayes' formula ?:

$$ P(H \mid D) = \frac{P(D \mid H) P(H)}{P(D)} $$

StubbornAtom
  • 18,135
JKM
  • 449
  • The precise form of $P(D)$ may depend on your choice of $D$ when specifying the likelihood (for example, did you use the sum of the sample or the mean). But since $P(D)$ or $\frac{1}{P(D)}$ is what is called a normalising constant, you can save finding it until the end if ever and then merely to make the whole posterior distribution be $1$. – Henry Nov 13 '20 at 00:40

1 Answers1

1

In some situations, computing the denominator $P(D)$ can be computationally prohibitive, since it can be a sum (e.g. $P(D) = \sum_h P(D \mid H=h) P(H=h)$) or an integral (for continuous random variables).

If the information you need from the posterior probability doesn't require this normalizing constant (e.g. all you care about is how the posterior probabilities for different values of $H$ compare relative to each other), then you don't need to compute the normalizing factor $P(D)$ at all.

angryavian
  • 94,324
  • 7
  • 77
  • 147