Terminologies: Adaptive, recursive and iterative

Question

1. Considering non-linear filtering technique like Extended Kalman filtering with Expectation Maximization (EM). EM is an iterative technique but what is Kalman filtering? Is Kalman Filtering called iterative approach?

2. Adaptive signal processing algorithms like Least Mean Square and Recursive Least square (RLS) estimate weights at each time instant and then update them. Since, RLS has a recursive nature and it falls under adaptive technique, then does this mean that adaptive is another name for recursive?

3. What is the difference between adaptive and iterative terminologies? Are Kalman filtering and its nonlinear versions adaptive?

Q1: In short, is adaptive=recursive=iterative?

Q2: Where is EM and Kalman Filtering approaches categorized under and why?

Answer 1

Q1: In short, is adaptive=recursive=iterative?

Filtering is applying a filter $f$ to an input signal $x$ to get an output signal $y$:

$$ y(t) = f(x(t), t) $$

The filter $f$ is called iterative if its next value can be calculated from other time-dependent values of either $x$ or $y$ or both.

The filter $f$ is called recursive if it depends on $y$ for some time $s \le t$ (where $t$ is the current time).

If the way $f$ uses past values of $y$ to filter $x$ changes, then the filter $f$ is also adaptive.

For example, the filter: $$ y(t) = x(t) $$ is iterative, but not recursive nor adaptive.

For example, the filter: $$ y(t) = x(t) + \alpha y(t-1) $$ is both iterative and recursive, but not adaptive.

For example, the filter: $$ y(t) = x(t) + \alpha(y,x) y(t-1) $$ is both recursive and adaptive (the filter coefficient $\alpha$ depends on some function of past inputs $x$ and past outputs $y$).

Q2: Where is EM and Kalman Filtering approaches categorized under and why?

Kalman filtering is definitely iterative and recursive as the new updates will depend on past values of the input and output (well state estimates).

The standard implementation of the Kalman filter is not adaptive. However, there are simple extensions that make it adaptive.