We’re rewarding the question askers & reputations are being recalculated! Read more.
11

According to your definition of autocorrelation, the autocorrelation is simply the covariance of the two random variables $Z(n)$ and $Z(n+\tau)$. This function is also called autocovariance. As an aside, in signal processing, the autocorrelation is usually defined as $$R_{XX}(t_1,t_2)=E\{X(t_1)X^*(t_2)\}$$ i.e., without subtracting the mean. The ...


7

You can think of linear-least squares in single dimension. The cost function is something like $a^{2}$. The first derivative (Jacobian) is then $2a$, hence linear in $a$. The second derivative (Hessian) is $2$ - a constant. Since the second derivative is positive, you are dealing with convex cost function. This is eqivalent to positive definite Hessian ...


6

If you have two signal vectors $x_1[n]$ and $x_2[n]$ each of $N$ elements, then there are two different things we can consider. How do the quantities $\displaystyle\sum_{n=1}^N x_i[n]x_j[n],~ i, j \in \{1,2\}$ compare? In particular, when the signals are noisy and the noises can be considered to be jointly stationary (or jointly wide-sense stationary), ...


5

What are reasons to choose for cross-correlation or cross-covariance when comparing signals with non-zero mean? Well, part of the issue is that cross-correlation as defined in your equation: $$(f \star g)[n]\ \stackrel{\mathrm{def}}{=} \sum_{m=-\infty}^{\infty} f^*[m]\ g[m+n].$$ will not exist (or be infinite) if $f$ and $g$ have non-zero mean. So, in ...


4

For power signals $x(t)$ and $y(t)$, the function $$R_{xy}(\tau)=\lim_{T\rightarrow\infty}\frac{1}{2T}\int_{-T}^{T}x(t)\bar{y}(t+\tau)dt\tag{1}$$ is the cross-correlation of $x(t)$ and $y(t)$. So the expression you're asking about is the cross-correlation of $x(t)$ and $y(t)$ evaluated at lag $\tau=0$: $$R_{xy}(0)=\lim_{T\rightarrow\infty}\frac{1}{2T}\...


4

The Covariance Matrix is commonly defined as $$\mathbf Q = E\left[ (\mathbf x -\mathbf\mu_{x})(\mathbf x -\mathbf\mu_{x})^*\right]$$ with $\mu$ denoting the mean value, i.e. $\mu_{x}=E\left[\mathbf x\right]$, and $\mathbf x$ being column vectors. The fact that you define the covariance matrix as $$\mathbf{R}_i = E\left[\textbf{u}_i^*\textbf{u}_i \right]$$ ...


3

MVDR is a narrowband beamformer. For broadband signals it is usually applied for each frequency bin. That means that $\mathbf{R}_{xx}$ is frequency dependent. In other words, for each time you should have $M$ matrices, each one is $3\times 3$. Now, since you usually cannot compute $\mathbf{R}_{xx}$ exactly, you perform covariance estimation $\tilde{\mathbf{...


3

A PRN sequence is a Pseudo-Random Noise sequence, often generated by using an Linear Feedback Shift Register (LFSR) with the feedback taps done by using a primitive irreducible polynomial in GF{2}, which is the Golois Field of 2 elements. When a primitive and irreducible polynomial in GF{2} is used, the LFSR will produce a "maximum length sequence", meaning ...


3

I had worked on an array processing problem where I had used diagonal loading of the measurement covariance matrix. But I had used diagonal loading as a solution to what I thought was a numerical issue with my eigen values being too small. Since I had to invert the covariance matrix with small eigen values, I had numerical issues with the result. Diagonal ...


3

You can't really have a covariance of a matrix. What you can have, is a covariance matrix of a set of vectors. So, if you think of the rows of your matrix A as two vectors in 3D: [2 3 4] and [5 5 6], then the covariance matrix of this set of two vectors is C = A' * A (A transpose times A). Note that if you shuffle the rows of A in a different order, C ...


3

Usually, for power signals, we define the inner product to be \begin{align} \left<x\,,\,y\right> &= \lim_\limits{T\rightarrow \infty} \frac 1 {2T} \int\limits_{-T}^T x(t)\bar y(t)\,dt \end{align} which induces the vector norm \begin{align} ||x||^2 &= \left<x\,,\,x\right>\\ &=\lim_{T\rightarrow \infty} \frac 1 {2T} \int\limits_{-T}^...


3

I don't understand the subscript $n$ notation, however, in the least squares problem that is given by: \begin{equation} {\bf{y}}={\bf{H}}{\theta}+\bf{n}, \end{equation} where ${\bf{n}}\sim\mathcal{N}(\bf{0}, \sigma^2I_N)$ is a zero mean additive white Gaussian noise and $I_N$ is the $N \times N$ identity matrix, the maximum likelihood and the least squares ...


3

It's the key point of array signal processing, I suppose. Say $x$ is the input vector of $[N,1]$ dimension collected from $N$ array sensors. $x(k)$ is its realization at the $k$ moment of time. By its definition covariance matrix (sometimes it's called autocorrelation matrix): $R = E[x\cdot x^H]$ , where $E[]$ is expectation operator and $x^H$ is Hermitian ...


3

For discrete data both are the same - Finding set of orthogonal directions which maximizes the Variance (Energy) of data along them. Sometimes those are called the natural axis of the. Since we're dealing with variance it is only natural both are calculated from the covariance matrix of data. You may encounter places where KL might be even defined on the ...


3

The IEEE signal processing literature deviates somewhat from the terminology used in statistics and actually from itself so your confusion is justified. If $\mathbf{x_k}$ is a vector signal the quantity at sample time $\mathbf{k}$, $$ \mathbf{R}= \sum_k \mathbf{x_k \, x_k^H} $$ is often called either. The SP transactions historically focused on ...


2

There is something that is not clear of what you have done with the data, and that is who do you form the random vectors to perform de SVD (or EVD) on the covariance matrix. 1 -The KLT can be succesfully used on a one dimensional signal (only one Geophone), taking frames of $M$ samples and estimating a covariance matrix from it, and the performing ...


2

I would think that it is indeed the "normal" variance of the image. You look at the distribution of the pixel values (i. e. gray levels), and compute its variance.


2

What you want to do is dimension reduction. The most basic, yet very powerful and commonly used, technique to do it is principal component analysis (PCA). PCA operates on the covariance matrix. You can look it up on Wikipedia for a throughout tutorial. PCA decomposes your data $y$, having dimension $d$, into $d$ principal components. The first principal ...


2

Variance is defined as $V(x)=\frac{\sum_{i=1}^n(x_i-\mu)^2}{n}$. Just in case for you, mean $\mu$ is defined as $\mu=\frac{\sum_{i=1}^nx}{n}$. Covariance between two random variables $x$ and $y$ (or columns of a matrix) is defined as $Cov(x,y)=\frac{\sum_{i=1}^n[(x_i-\mu_x)(y_i-\mu_y)]}{n}$ and $Cov(x,x)=V(x)$. The term covariance matrix may be misleading ...


2

This answer is not fundamentally different from the others; it's more of a complement and addendum. If $n(t)$ is zero-mean white Gaussian noise, then its variance is actually infinite; its power spectral density is constant and often denoted $\mathcal P_n(f)=N_0/2$. If this noise is input into a filter with impulse response $h(t)$, then the power spectral ...


2

Let me give a clue $$ E\{ y \} = E\{ x g \} = E\{ x \} g $$


2

Your question (as expanded in the comments) is asking if we start with $$ x(t) = x_{\rm true}(t) + n_1(t) $$ and filtering it using a filter $H(\omega)$ to get $$ x_{\rm hp}(t) = h(t) * x(t) = x_{\rm true}(t) + n_2(t) $$ where the variance of $n_1$ is $v_1$, then what is $v_2$, the variance of $n_2$? That seems ill-posed because the filter $H$ will change $...


2

In general, the equations from your first code block work equally for real and complex values. Note that the ()' operation is the Hermitian Conjugate (i.e. transpose + conjugate). If you are in the real domain, it only becomes transpose (because conjugation on reals doesn't change them). Hence, you would not need to change anything in the code. Regarding ...


2

One must be careful when asking questions about the relationships between the elements of a complex random vector. The short answer to your question is that you cannot say much for either cases simply by considering the covariance (or correlation) matrix. Actually, the covariance (correlation) matrix is not enough to capture all the relationships that ...


1

You can write $$ R=YY^H $$ where $Y$ is a matrix of size $N\times N_f$ and $N$ is the dimension of $y_k$. $Y$ contains all the measured $y_k$ as its columns. Then, the rank of $R$ is upper bounded by $N_f$. In particular, if $N_f<N$, $R$ will always be a singular matrix. So, if you have too few measurements, you will likely run into the problem you ...


1

I'd recommend looking into the relation of correlation and covariance; the one is just the "bias-corrected" version of the other. Then, use the Wiener-Khinchin¹ theorem: If, and only if, the signal is weak-sense stationary, the Fourier transform of the autocorrelation of that signal is the same as Expectation value of the magnitude-squared Fourier transform ...


1

To make the Online variance algorithm work for vectors, change one line: M2 += np.outer( delta, delta2 ) # deltas 3-vecs, M2 3 x 3 # outer( x, y ) is pairs x[i] * y[j] -- # [[ x0 y0, x0 y1, x0 y2 ], # [ x1 y0, x1 y1, x1 y2 ], # [ x2 y0, x2 y1, x2 y2 ]] A small python class for this is under gist.github.com/denis-bz . It can covariance points with ...


1

I'm not totally sure I understand your question: could you give a concrete example? For instance, what's the randomness? Anyway, here are some relations between the quantities you're interested in. The energy of the signal is $$\mid \mid Y \mid \mid_2^2 = \sum_{i=0}^n y_i^2$$ The Auto-correllation matrix $R$ is formed as $YY^T$ (outer product between the ...


1

There is no "best method", it will vary according to application. There are many trains of thought regarding this subject. I've done something in that area using Cepstral Analysis. Take a look at this: https://8b14d84720d60d831e70ec440417feb774595227-www.googledrive.com/host/0B4-fFSwiy1mGdS00SHB2Nlo3bkU/imreg_translation/imreg_translation.html Note: I am ...


1

I think in general you have to consider your signal as a stochastic process, so you not only have a variance $\sigma_x^2$ , but you have an entire autocorrelation sequence for $x(n)$, $r_x(t) = E[x(n)x(n-t)]$. So what you need to think about is to obtain the autocorrelation sequence of the output of your signal passed through the filter. This is not hard to ...


Only top voted, non community-wiki answers of a minimum length are eligible