MAP, ML & the Cancer Example

Source: Unit 3 §5

Bayes' theorem for hypotheses

$$P(h \mid D) = \frac{P(D \mid h) \cdot P(h)}{P(D)}$$

Term	Name	Meaning
P(h|D)	Posterior	probability of the hypothesis after seeing the data - this is what we want
P(h)	Prior	probability of the hypothesis before any data
P(D|h)	Likelihood	probability of the data given the hypothesis
P(D)	Evidence	prior probability of the data, a normalising constant

MAP: maximum a posteriori

Find the most probable hypothesis given the data:

$$h_{\text{MAP}} = \arg\max_{h \in H} P(h \mid D) = \arg\max_{h \in H} P(D \mid h) \cdot P(h)$$

$P(D)$ is dropped because it is a constant independent of $h$, and a constant cannot change which $h$ wins the argmax.

Best hypothesis means most probable hypothesis. The whole goal of Bayesian learning is to locate the hypothesis that best explains the observed data. Linear and logistic regression can both be framed as MAP estimation.

ML: maximum likelihood

If every hypothesis is equally probable a priori, so that $P(h_i) = P(h_j)$ for all $i, j$, the prior factors out of the argmax and MAP collapses to:

$$h_{\text{ML}} = \arg\max_{h \in H} P(D \mid h)$$

◎Exam cueThe one-line relationship

If the prior is uniform, then $h_{\text{MAP}} = h_{\text{ML}}$. ML is not a different philosophy, it is MAP with the prior switched off.

Worked example: does the patient have cancer?

#NumbersThe setup

• $P(\text{cancer}) = 0.008$, so $P(\neg\text{cancer}) = 0.992$.
• Test accuracy given cancer: $P(+ \mid \text{cancer}) = 0.98$, $P(- \mid \text{cancer}) = 0.02$.
• Test accuracy given no cancer: $P(+ \mid \neg\text{cancer}) = 0.03$, $P(- \mid \neg\text{cancer}) = 0.97$.

Test \ Actual	Cancer	¬Cancer
+	0.98	0.03
−	0.02	0.97

A patient tests positive. Cancer or not? Compare the two unnormalised posteriors, which is all MAP needs.

→StepsWorking the posterior

1. Hypothesis cancer: $P(+ \mid \text{cancer}) \cdot P(\text{cancer}) = 0.98 \times 0.008 = \mathbf{0.00784}$.
2. Hypothesis ¬cancer: $P(+ \mid \neg\text{cancer}) \cdot P(\neg\text{cancer}) = 0.03 \times 0.992 = \mathbf{0.02976}$.
3. Since $0.02976 > 0.00784$, the winner is $h_{\text{MAP}} = \neg\text{cancer}$. The patient most likely does not have cancer.
4. Normalise to recover real probabilities. The sum is $0.00784 + 0.02976 = 0.0376$.
5. $P(\text{cancer} \mid +) = 0.00784 / 0.0376 = \mathbf{0.21}$.
6. $P(\neg\text{cancer} \mid +) = 0.02976 / 0.0376 = \mathbf{0.79}$.

The test is excellent and the patient tested positive, yet the no-cancer bar is still nearly four times longer. The prior of 0.008 is doing all the work.

⚠GotchaA test that catches 98% of cancers, and a positive still means 21%

Even after a positive test, cancer is only 21% likely, because the prior is only 0.008. There are simply far more healthy people generating false positives than sick people generating true positives. Bayesian inference depends strongly on the priors, and this example exists to make that impossible to forget.