Estimating a Signal Given a Noisy Measurement of the Signal and Its Derivative (Denoising)

I have a signal and its derivative simultaneously measured, both including additive noise. The measurement is completed before the analysis, so it can be looked ahead. Now I want to reconstruct a less noisy version of the signal. I'm looking for pointers to algorithms I should look into.

Kalman filter seems to be on the right track, but the implementations I see so far are trying to estimate based on previous measurements only, while I probably should use both previous and coming measurements for optimal results at each point.

Ideas?

This is a really nice problem.

Problem Formulation

I will formulate it as following:

Let $$x \in \mathbb{R}^{n}$$ be a signal. Given $$y \in \mathbb{R}^{n}$$ which is a noisy measurement of $$x$$ such that $$y = x + v$$ and $$z$$ be a noisy measurement of the derivative of $$x$$ such that $$z = F x + w$$ where $$F$$ is the finite differences operator. It is known that both $$v$$ and $$w$$ are White Noise which is independent form each other. Find a reasonable estimator of $$x$$.

OK, actually, I did the hard part by formulating the problem the way I did (Formulating connection between $$x$$ and its derivative).
I find this formulation to be reasonable. It doesn't assume too much but gives us enough to work with.

So my first of choice would be something like:

$$\arg \min_{x} \frac{1}{2} {\left\| x - y \right\|}_{2}^{2} + \frac{\lambda}{2} {\left\| F x - z \right\|}_{2}^{2}$$

This is really easy to solve.
The tricky part is $$\lambda$$. Well, I put it there to allow playing with the ratio between the noise level of the measurements. If the $$\sigma$$ of both noises is the same, set $$\lambda = 1$$. Else, set it as $$\lambda = \frac{ {\sigma}_{v}^{2} }{ {\sigma}_{w}^{2} }$$. The intuition is straight forward, if the noise level of $$w$$ is smaller then $$v$$ we want $$\lambda$$ to be bigger than $$1$$ in order to maximize the knowledge coming from $$z$$.

Remark
The model above is actually the Maximum Likelihood Estimator of the model above if you assume the noises are AWGN.
I don't want to get into the Math of deriving it, but Least Squares is almost (Always) the ML for the AWGN model.

Solution to the Model

As mentioned above:

$$\hat{x} = \arg \min_{x} \frac{1}{2} {\left\| x - y \right\|}_{2}^{2} + \frac{\lambda}{2} {\left\| F x - z \right\|}_{2}^{2}$$

This is a strictly convex problem hence the solution is given in the stationary point.

\begin{align*} \frac{\mathrm{d}}{\mathrm{d} \hat{x}} \frac{1}{2} {\left\| \hat{x} - y \right\|}_{2}^{2} + \frac{\lambda}{2} {\left\| F \hat{x} - z \right\|}_{2}^{2} & = 0 && \text{Definition of Stationary Point} \\ & = \hat{x} - y + \lambda {F}^{T} \left( F x - z \right) && \text{} \\ & \Leftrightarrow \left( I + \lambda {F}^{T} F \right) = y + \lambda {F}^{T} z && \text{} \\ & \Leftrightarrow \hat{x} = {\left( I + \lambda {F}^{T} F \right)}^{-1} \left( y + \lambda {F}^{T} z \right) \end{align*}

MATLAB Simulation

The model I chose is using an harmonic signal - sine.
We use Finite Differences as the Derivative Model.

Here are the signals and the estimation using $$\lambda = \frac{ {\sigma}_{v}^{2} }{ {\sigma}_{w}^{2} }$$:

It can be seen that the estimation is much better then any of the given signals.

Let's verify the optimality of $$\lambda$$:

It is hard to see 2 circles at the bottom but they are close to verify the optimality of $$\lambda$$.

The code is available on my StackExchange Signal Processing Q52150 GitHub Repository.

One alternative track could be that, when sampling at the same time a band-limited analog signal and its derivative, it can be sampled at half the standard Nyquist sampling rate. I have learn quite recently that it was used in geophysics when recording at the same location with a vibration and and pressure sensor.

Hence there is room to improve your signal's quality (which I am still looking for). Meanwhile, a few references to start from: