11 votes

Covariance vs Autocorrelation

According to your definition of autocorrelation, the autocorrelation is simply the covariance of the two random variables $Z(n)$ and $Z(n+\tau)$. This function is also called autocovariance. As an ...
  • 82.2k
9 votes
Accepted

Quadratic Programming with Linear Equality Constraints

Let's solve a more general problem (Least Squares with Linear Equality Constraints): $$ \begin{alignat*}{3} \arg \min_{x} & \quad & \frac{1}{2} \left\| A x - b \right\|_{2}^{2} \\ \text{...
  • 44k
7 votes
Accepted

What Is the Difference Between PCA and Karhunen Loeve (KL) Transform?

For discrete data both are the same - Finding set of orthogonal directions which maximizes the Variance (Energy) of data along them. Sometimes those are called the natural axis of the data (Inferred ...
  • 44k
7 votes
Accepted

Kalman Filter State Covariance Matrix for Non Constant Process Noise Matrix in PyKalman

For classic Kalman Filter, where $ {Q}_{k} = Q $ and $ {R}_{k} = R $, namely the process noise covariance and the measurement noise covariance (I'm using Wikipedia - Kalman Filter notations) the ...
  • 44k
6 votes

Difference between $\mathbb{E}[\mathbf{x} \mathbf{x}^{\rm{H}}]$ and $\mathbb{E}[(\mathbf{x}-\boldsymbol{\mu}) (\mathbf{x}-\boldsymbol{\mu})^{\rm{H}}]$

For the first case, as you wrote, it means the elements are not correlated. Since this is a Gaussian Random Vector it means the elements are independent. It means that at most only one element of $ \...
  • 44k
6 votes
Accepted

Show That the Power Spectrum Density Matrix Is Positive Semi Definite (PSD) Matrix

Pay attention that for a Scalar Random Process the Power Spectrum Density is non negative. Namely, let $ y \left[ n \right] \in \mathbb{R} $ be a WSS Random process with its Auto Correlation function ...
  • 44k
6 votes
Accepted

Covariance matrix of an adaptive filter input

The Covariance Matrix is commonly defined as $$\mathbf Q = E\left[ (\mathbf x -\mathbf\mu_{x})(\mathbf x -\mathbf\mu_{x})^*\right]$$ with $\mu$ denoting the mean value, i.e. $\mu_{x}=E\left[\mathbf ...
5 votes
Accepted

Cross-correlation or cross-covariance of non-zero mean signals

What are reasons to choose for cross-correlation or cross-covariance when comparing signals with non-zero mean? Well, part of the issue is that cross-correlation as defined in your equation: $$(f \...
  • 23.4k
5 votes
Accepted

Intuition for $\mathbf{P} = \mathbf{0}$ in steady-state when $\mathbf{Q} = \mathbf{0}$ (Kalman filter)

We each have different life experiences to fuel our intuition, but try this one out: Let $\mathbf A = 1$ and $\mathbf Q = 0$, and $\mathbf C = 1$ -- i.e., the actual state variable just doesn't change,...
  • 10.1k
4 votes
Accepted

PSD and $\lim_{T\rightarrow \infty} \frac 1 {2T} \int_{-T}^T x(t)\bar y(t)\,dt$

For power signals $x(t)$ and $y(t)$, the function $$R_{xy}(\tau)=\lim_{T\rightarrow\infty}\frac{1}{2T}\int_{-T}^{T}x(t)\bar{y}(t+\tau)dt\tag{1}$$ is the cross-correlation of $x(t)$ and $y(t)$. So ...
  • 82.2k
4 votes
Accepted

Generate signals with a particular variance and SNR

It depends on what you mean by SNR. It's a common joke in the DSP community to spell it out as "something to noise ratio", referring to the fact that there is no unique definition of SNR, so the term ...
  • 2,323
3 votes

Difference between $\mathbb{E}[\mathbf{x} \mathbf{x}^{\rm{H}}]$ and $\mathbb{E}[(\mathbf{x}-\boldsymbol{\mu}) (\mathbf{x}-\boldsymbol{\mu})^{\rm{H}}]$

One must be careful when asking questions about the relationships between the elements of a complex random vector. The short answer to your question is that you cannot say much for either cases ...
3 votes
Accepted

Help in understanding from book expression of variance of an estimator : PRBS vs real valued input

A PRN sequence is a Pseudo-Random Noise sequence, often generated by using an Linear Feedback Shift Register (LFSR) with the feedback taps done by using a primitive irreducible polynomial in GF{2}, ...
  • 39.1k
3 votes

PSD and $\lim_{T\rightarrow \infty} \frac 1 {2T} \int_{-T}^T x(t)\bar y(t)\,dt$

Usually, for power signals, we define the inner product to be \begin{align} \left<x\,,\,y\right> &= \lim_\limits{T\rightarrow \infty} \frac 1 {2T} \int\limits_{-T}^T x(t)\bar y(t)\,dt \end{...
3 votes
Accepted

What is the Technique to Find Variance of Estimation Error

I don't understand the subscript $n$ notation, however, in the least squares problem that is given by: \begin{equation} {\bf{y}}={\bf{H}}{\theta}+\bf{n}, \end{equation} where ${\bf{n}}\sim\mathcal{N}(...
  • 600
3 votes

Calculating covariance matrix for MVDR beamforming

MVDR is a narrowband beamformer. For broadband signals it is usually applied for each frequency bin. That means that $\mathbf{R}_{xx}$ is frequency dependent. In other words, for each time you should ...
  • 1,450
3 votes
Accepted

Generalized correlation coefficients

This question belongs more on stats.SE (where many similar questions have been thrashed out in detail) but nonetheless here goes. Let's take the simplest case of $N=1$. Just because $X$ and $Y$ are ...
2 votes

Variance of a filtered signal

This answer is not fundamentally different from the others; it's more of a complement and addendum. If $n(t)$ is zero-mean white Gaussian noise, then its variance is actually infinite; its power ...
  • 13.9k
2 votes
Accepted

Covariance matrix $P$ and $Q$ for complex valued data

In general, the equations from your first code block work equally for real and complex values. Note that the ()' operation is the Hermitian Conjugate (i.e. transpose + conjugate). If you are in the ...
2 votes

Problem with Covariance matrix using diagonal loading involved in calculation of eigenvalues

You can write $$ R=YY^H $$ where $Y$ is a matrix of size $N\times N_f$ and $N$ is the dimension of $y_k$. $Y$ contains all the measured $y_k$ as its columns. Then, the rank of $R$ is upper bounded ...
2 votes
Accepted

Variance of a filtered signal

Your question (as expanded in the comments) is asking if we start with $$ x(t) = x_{\rm true}(t) + n_1(t) $$ and filtering it using a filter $H(\omega)$ to get $$ x_{\rm hp}(t) = h(t) * x(t) = x_{\rm ...
  • 23.4k
2 votes
Accepted

Is it possible to estimate variance of noise for a step answer signal?

Hi: In order to estimate the variance, you need to have an underlying model for your signal. So, suppose that the model is $y_{t+1} = y_t + \epsilon_t$ $~\forall ~ t = 1,\ldots n $. assuming that $E(...
  • 1,102
2 votes

How to calculate the Diagonal loading factor evaluate calculate the inversion of a covariance matrix

Short answer: just use $\sigma = 10^{-8}$. Covariance matrices have eigenvalues $\geq 0$ (theoretically), so $Ri + 10^{-8} \, I$ will have eigenvalues $\geq 10^{-8}$, safely non-singular. A longer ...
  • 608
2 votes

Should I pass Kalman Filter absolute or offset-from-mean sensor values?

As mentioned in another answer, the variance of the measurement noise is a property of the model which is baked in to the Kalman filter. I think you're running into a different problem, which is that ...
  • 10.1k
2 votes
Accepted

Variance of function of random variable

Assume $Y = g(X)$ be the function of RV $X$, then by using the following $$E\{ g(X) \} = \int g(x) f_X(x) dx $$ variance of $Y$ can be computed without the computation of pdf $f_Y(y)$ as: $$ \begin{...
  • 27.2k
2 votes
Accepted

Covariance Matrix Polar to Cartesian

The covariance matrix is given by $$C_{X,Y}=\begin{bmatrix}E(XX)& E(XY) \\ E(YX )& E(YY) \end{bmatrix}$$ This can be written as below: $$C_{X,Y}=\begin{bmatrix}E(R^2cos^2(\Theta) )& E(...
  • 2,552
2 votes
Accepted

Matched filter for "amplitude SNR" vs power SNR

Power is proportional to the square of amplitude, so there's really no difference (since SNR is always a positive number) when maximizing: A filter that maximizes "amplitude SNR" maximizes "power SNR"....

Only top scored, non community-wiki answers of a minimum length are eligible