Maximium A Posteriori (MAP) and Maximum Likelihood (ML) are both approaches for making decisions from some observation or evidence.

MAP takes into account the prior probability of the considered hypotheses. ML does not. This set of probabilities, known as "a priori" probabilities or simply "priors", is often known imperfectly, but even rough approximations are often better than nothing.

The approaches are the same in the case where the prior probability is truly uniform. For example in the roll of a single fair die, or deciding a bit in a random binary message.

MAP and ML can be quite different. For an anecdote on how the obvious answer can be exactly wrong, search for "Abraham Wald and the Missing Bullet Holes" from the excellent book "How Not to be Wrong" by Jordan Ellenberg.

A simpler case of imbalanced priors is found in the old saying,

When you hear hooves, think 'horses' not 'zebras'

One can think of MAP decision/estimation as a rigorous version of Occam's razor.

Let's say we observed event

• $A$ := Audible hooves

goal: Decide what animal is thundering towards us.

and we live in world with only 3 ungulate (hooved) species:

• $Z$ebras,
• $H$orses,
• $C$hevrotain

(such worlds are common in post-apocalyptic fiction and math problems)

Let's say we know when each of these animals is present, their chance of making $A$udible hoof sounds is as follows

• P(A | Z) = .91 ( read as probability of event A, given that Z is true)
• P(A | H) = .9
• P(A | C) = .2

These conditional probabilities are known as likelihoods on $Z,H,C$ when $A$ is true.

ML = An easy answer. Find the maximum of the likelihoods.

The above is all the info we need to make the ML decision = Zebras. The zebras have a higher likelihood of generating the observation when they are present.

MAP = a better answer. Find the maximum posterior probability.

What we really want to decide is: which is highest posterior probability {P(Z|A),P(H|A),P(C|A)} i.e. what is the chance a particular animal is present, given that we observed $A$?

To decide that, we need to know more about the abundance of the different animals. In the state where you live there are 100 wild chevrotain , 100 zebras, and 800 horses. These are the prior probabilities (scaled by a constant).

We can find our posterior probabilities via Bayes' Rule ($$ P(Z|Y) = P(Y|Z)P(Z)/P(Y)$$). We can take a short cut and settle for proportionality ($$\propto$$), i.e. omit the common denominators.

Those posterior probabilities are (proportionally)

$$ P(H|A) \propto P(A|H)P(H) = .9 \times 800 = 720 $$ $$ P(Z|A) \propto P(A|Z)P(Z) = .91 \times 100 = 91 $$ $$ P(C|A) \propto P(A|C)P(C) = .2 \times 100 = 20 $$

So the MAP decision is overwhelmingly $H$orses. In fact, by using the law of total probability, we can say that there is an 86% chance it is horses (= 720/(720+91+20))

Of course, by the time you've done all this math, you've probably been trampled.
Sometimes an easy answer gives you what you need to act.

Maximium A Posteriori (MAP) and Maximum Likelihood (ML) are both approaches for making decisions from some observation or evidence.

MAP takes into account the prior probability of the considered hypotheses. ML does not. This set of probabilities, known as "a priori" probabilities or simply "priors", is often known imperfectly, but even rough approximations are often better than nothing.

The approaches are the same in the case where the prior probability is truly uniform. For example in the roll of a single fair die, or deciding a bit in a random binary message.

MAP and ML can be quite different. For an anecdote on how the obvious answer can be exactly wrong, search for "Abraham Wald and the Missing Bullet Holes" from the excellent book "How Not to be Wrong" by Jordan Ellenberg.

A simpler case of imbalanced priors is found in the old saying,

When you hear hooves, think 'horses' not 'zebras'

One can think of MAP decision/estimation as a rigorous version of Occam's razor.

Let's say we observed event

• $A$ := Audible hooves

goal: Decide what animal is thundering towards us.

and we live in world with only 3 ungulate (hooved) species:

• $Z$ebras,
• $H$orses,
• $C$hevrotain

(such worlds are common in post-apocalyptic fiction and math problems)

Let's say we know when each of these animals is present, their chance of making $A$udible hoof sounds is as follows

• P(A | Z) = .91 ( read as probability of event A, given that Z is true)
• P(A | H) = .9
• P(A | C) = .2

These conditional probabilities are known as likelihoods on $Z,H,C$ when $A$ is true.

ML = An easy answer. Find the maximum of the likelihoods.

The above is all the info we need to make the ML decision = Zebras. The zebras have a higher likelihood of generating the observation when they are present.

MAP = a better answer. Find the maximum posterior probability.

What we really want to decide is: which is highest posterior probability {P(Z|A),P(H|A),P(C|A)} i.e. what is the chance a particular animal is present, given that we observed $A$?

To decide that, we need to know more about the abundance of the different animals. In the state where you live there are 100 wild chevrotain , 100 zebras, and 800 horses. These are the prior probabilities (scaled by a constant).

We find our posterior probabilities via Bayes' Rule: e.g. $P(H|A) = P(A|H)P(H)/P(A)$. In this case, we can take a shortcut and settle for proportionality ($\propto$), omitting the common denominator $P(A)$.

The posterior probabilities of hearing the various animals, given that we heard hooves are (proportionally)

$$ P(H|A) \propto P(A|H)P(H) = .9 \times 800 = 720 $$ $$ P(Z|A) \propto P(A|Z)P(Z) = .91 \times 100 = 91 $$ $$ P(C|A) \propto P(A|C)P(C) = .2 \times 100 = 20 $$

So the MAP decision is overwhelmingly $H$orses. In fact, by using the law of total probability, we can say that there is an 86% chance it is horses (= 720/(720+91+20))

Of course, by the time you've done all this math, you've probably been trampled.
Sometimes an easy answer gives you what you need to act.

Maximium A Posteriori (MAP) and Maximum Likelihood (ML) are both approaches for making decisions from some observation or evidence.

MAP takes into account the prior probability of the considered hypotheses. ML does not. This set of probabilities, known as "a priori" probabilities or simply "priors", is often known imperfectly, but even rough approximations are often better than nothing.

The approaches are the same in the case where the prior probability is truly uniform. For example in the roll of a single fair die, or deciding a bit in a random binary message.

MAP and ML can be quite different. For an anecdote on how the obvious answer can be exactly wrong, search for "Abraham Wald and the Missing Bullet Holes" from the excellent book "How Not to be Wrong" by Jordan Ellenberg.

A simpler case of imbalanced priors is found in the old saying,

When you hear hooves, think 'horses' not 'zebras'

One can think of MAP decision/estimation as a rigorous version of Occam's razor.

Let's say we observed event

• $A$ := Audible hooves

goal: Decide what animal is thundering towards us.

and we live in world with only 3 ungulate (hooved) species:

• $Z$ebras,
• $H$orses,
• $C$hevrotain

(such worlds are common in post-apocalyptic fiction and math problems)

Let's say we know when each of these animals is present, their chance of making $A$udible hoof sounds is as follows

• P(A | Z) = .91 ( read as probability of event A, given that Z is true)
• P(A | H) = .9
• P(A | C) = .2

These are known as likelihoods on $Z,H,C$ when $A$ is true.

ML: an easy answer

The above is all the info we need to make the ML decision = Zebras. The zebras have a higher likelihood of generating the observation when they are present.

MAP: a better answer

What we really want to decide is: which is highest in {P(Z|A),P(H|A),P(C|A)} i.e. what is the chance a particular animal is present, given that we observed $A$?

To decide that, we need to know more about the abundance of the different animals. In the state where you live there are 100 wild chevrotain , 100 zebras, and 800 horses. These are the prior probabilities (scaled by a constant).

We can order our posterior probabilities (Omitting common factors in the posterior probabilities found via Bayes' Rule) in the following order

$$ P(H|A) \propto P(A|H)P(H) = .9 \times 800 = 720 $$ $$ P(Z|A) \propto P(A|Z)P(Z) = .91 \times 100 = 91 $$ $$ P(C|A) \propto P(A|C)P(C) = .2 \times 100 = 20 $$

So the MAP decision is overwhelmingly $H$orses. In fact, by using the law of total probability, we can say that there is an 86% chance it is horses (= 720/(720+91+20))

Of course, by the time you've done all this math, you've probably been trampled.
Sometimes an easy answer gives you what you need to act.

Maximium A Posteriori (MAP) and Maximum Likelihood (ML) are both approaches for making decisions from some observation or evidence.

MAP takes into account the prior probability of the considered hypotheses. ML does not. This set of probabilities, known as "a priori" probabilities or simply "priors", is often known imperfectly, but even rough approximations are often better than nothing.

The approaches are the same in the case where the prior probability is truly uniform. For example in the roll of a single fair die, or deciding a bit in a random binary message.

MAP and ML can be quite different. For an anecdote on how the obvious answer can be exactly wrong, search for "Abraham Wald and the Missing Bullet Holes" from the excellent book "How Not to be Wrong" by Jordan Ellenberg.

A simpler case of imbalanced priors is found in the old saying,

When you hear hooves, think 'horses' not 'zebras'

One can think of MAP decision/estimation as a rigorous version of Occam's razor.

Let's say we observed event

• $A$ := Audible hooves

goal: Decide what animal is thundering towards us.

and we live in world with only 3 ungulate (hooved) species:

• $Z$ebras,
• $H$orses,
• $C$hevrotain

(such worlds are common in post-apocalyptic fiction and math problems)

Let's say we know when each of these animals is present, their chance of making $A$udible hoof sounds is as follows

• P(A | Z) = .91 ( read as probability of event A, given that Z is true)
• P(A | H) = .9
• P(A | C) = .2

These conditional probabilities are known as likelihoods on $Z,H,C$ when $A$ is true.

ML = An easy answer. Find the maximum of the likelihoods.

The above is all the info we need to make the ML decision = Zebras. The zebras have a higher likelihood of generating the observation when they are present.

MAP = a better answer. Find the maximum posterior probability.

What we really want to decide is: which is highest posterior probability {P(Z|A),P(H|A),P(C|A)} i.e. what is the chance a particular animal is present, given that we observed $A$?

To decide that, we need to know more about the abundance of the different animals. In the state where you live there are 100 wild chevrotain , 100 zebras, and 800 horses. These are the prior probabilities (scaled by a constant).

We can find our posterior probabilities via Bayes' Rule ($$ P(Z|Y) = P(Y|Z)P(Z)/P(Y)$$). We can take a short cut and settle for proportionality ($$\propto$$), i.e. omit the common denominators.

Those posterior probabilities are (proportionally)

$$ P(H|A) \propto P(A|H)P(H) = .9 \times 800 = 720 $$ $$ P(Z|A) \propto P(A|Z)P(Z) = .91 \times 100 = 91 $$ $$ P(C|A) \propto P(A|C)P(C) = .2 \times 100 = 20 $$

So the MAP decision is overwhelmingly $H$orses. In fact, by using the law of total probability, we can say that there is an 86% chance it is horses (= 720/(720+91+20))

Of course, by the time you've done all this math, you've probably been trampled.
Sometimes an easy answer gives you what you need to act.