Skip to main content
Signal Processing

Return to Answer

Updated the correct link
Source Link
edit approved Jan 18, 2020 at 14:08
David
edit approved Jan 18, 2020 at 14:08
David
  • 144
  • 1
  • 3
  • 18

You can think of MAP as a regularization of the ML.
Just like you have regularization for Least Squares Problem (They can be built, mostly, as MAP problem).

The nice thing is that, as always, the best regularization is more data, namely, in most case when there is a lot of data they collide (Namely, low sensitivity fir the Posterior PDF).

So they differ mainly when there is (Relatively) low amount of data.
Now, when you have low amount of data and you know nothing about the parameters you're trying to estimate, ML is the way to go.
If you have low amount of data yet some prior knowledge about it, or reasonable assumption, make those assumption as regularization.
It even good to make the effort and describe this knowledge in Posterior PDF form.

Update

Found really nice tutorial about the topic - The Truth About Priors and OverfittingThe Truth About Priors and Overfitting.

You can think of MAP as a regularization of the ML.
Just like you have regularization for Least Squares Problem (They can be built, mostly, as MAP problem).

The nice thing is that, as always, the best regularization is more data, namely, in most case when there is a lot of data they collide (Namely, low sensitivity fir the Posterior PDF).

So they differ mainly when there is (Relatively) low amount of data.
Now, when you have low amount of data and you know nothing about the parameters you're trying to estimate, ML is the way to go.
If you have low amount of data yet some prior knowledge about it, or reasonable assumption, make those assumption as regularization.
It even good to make the effort and describe this knowledge in Posterior PDF form.

Update

Found really nice tutorial about the topic - The Truth About Priors and Overfitting.

You can think of MAP as a regularization of the ML.
Just like you have regularization for Least Squares Problem (They can be built, mostly, as MAP problem).

The nice thing is that, as always, the best regularization is more data, namely, in most case when there is a lot of data they collide (Namely, low sensitivity fir the Posterior PDF).

So they differ mainly when there is (Relatively) low amount of data.
Now, when you have low amount of data and you know nothing about the parameters you're trying to estimate, ML is the way to go.
If you have low amount of data yet some prior knowledge about it, or reasonable assumption, make those assumption as regularization.
It even good to make the effort and describe this knowledge in Posterior PDF form.

Update

Found really nice tutorial about the topic - The Truth About Priors and Overfitting.

Source Link
Royi
Royi
  • 20.5k
  • 4
  • 199
  • 240

You can think of MAP as a regularization of the ML.
Just like you have regularization for Least Squares Problem (They can be built, mostly, as MAP problem).

The nice thing is that, as always, the best regularization is more data, namely, in most case when there is a lot of data they collide (Namely, low sensitivity fir the Posterior PDF).

So they differ mainly when there is (Relatively) low amount of data.
Now, when you have low amount of data and you know nothing about the parameters you're trying to estimate, ML is the way to go.
If you have low amount of data yet some prior knowledge about it, or reasonable assumption, make those assumption as regularization.
It even good to make the effort and describe this knowledge in Posterior PDF form.

Update

Found really nice tutorial about the topic - The Truth About Priors and Overfitting.