1
$\begingroup$

I am new to DSP.

So I have a noisy signal with some high-frequency components. I was able to characterize the statistical properties of this signal through a gaussian distribution. My aim is to run a Singular Spectrum Analysis in order to remove some components at certain frequencies and see if they impact the Gaussian distribution (I assume they will not but have to prove this). However, the signal is at a considerably high sampling rate (256 Hz) and for a larger period of time (256 s) and I would have to do this for more than one signal. Hence, this process requires a large computational time, which I can't really afford right now.

I attempted to just use a smaller segment of the signal but noticed that the FFT properties change considerably. So is there a way to reduce the length of the signal while maintaining the FFT properties and the statistical distribution for faster analysis (Keeping in mind that I am new to DSP)?

Thank You.

I have attached the SSA code below.

Tp = 1/freq_intrest;
tendIn=size(y,2)/256;
tendIn=tendIn-mod(tendIn,Tp);
tendindex=(tendFFT*256)-mod(tendFFT*256,2);
M = round(tendFFTindex/2 - mod(tendFFTindex/2,Tp*256)); %Window Length

N = length(y);
t = (1:N)';

Y=zeros(N-M+1,M);
for m=1:M
    Y(:,m) = y((1:N-M+1)+m-1);
end
Cemb=Y'*Y / (N-M+1);
C=Cemb;

[RHO,LAMBDA] = eig(C);
LAMBDA = diag(LAMBDA);               % extract the diagonal elements
[LAMBDA,ind]=sort(LAMBDA,'descend'); % sort eigenvalues
RHO = RHO(:,ind);                    % and eigenvectors
PC = Y*RHO;

RC=zeros(N,M);
for m=1:30                  % m upto 30 to save comptuational time. Already noticed only PC(1:8) are important.
    buf=PC(:,m)*RHO(:,m)';  % invert projection
    buf=buf(end:-1:1,:);
    for n=1:N               % anti-diagonal averaging
        RC(n,m)=mean( diag(buf,-(N-M+1)+n) );
    end
end
$\endgroup$
14
  • $\begingroup$ 256 Hz … 256 s … Hence, this process requires a large computational time, which I can't really afford right now. seriously, people do this live, on millions of samples per second. You do not have to shorten anything, this is nearly no data at all. If this takes a long time, you're probably doing something strange algorithmically, to be completely honest! $\endgroup$ Commented Aug 11, 2021 at 16:41
  • $\begingroup$ @MarcusMüller Well time is relative. I need to finish all this in a day. At the current rate, each signal requires some 2 hours to complete two SSA (One with components I'm interested in and one without). The thing is, SSA requires large computational effort as it requires the storage of multiple matrices. I don't have the computational power or time so its simply not feasible. $\endgroup$
    – user244717
    Commented Aug 11, 2021 at 16:51
  • 1
    $\begingroup$ Again, if your SSA takes a day, your software is broken and you need to fix this, not shorten your signal. "Multiple matrices": Unless these matrices are several GB in size each, I doubt this is any problem. I'm not quite sure whether I should apologize: On one hand I'm telling you "for all I can tell, you're barking up the wrong tree", on the other hand "for all I can tell, your problem is probably easier to fix than you think". $\endgroup$ Commented Aug 11, 2021 at 16:52
  • 2
    $\begingroup$ yes 65536 samples is nothing scary to an FFT which will slam-bam-wam that in about a millisecond. $\endgroup$ Commented Aug 11, 2021 at 17:00
  • 1
    $\begingroup$ @robertbristow-johnson The issue isn't the FFT rather it is the SSA. $\endgroup$
    – user244717
    Commented Aug 11, 2021 at 17:07

1 Answer 1

0
$\begingroup$

So, after reading our discussion, and understanding your performance problems a bit better, it'd seem to me that indeed, there's a way that can do something very similar to SSA on the full correlation matrix.

However it is not SSA in the strict sense of the word, but works by assuming a finite-sized noise subspace, and a finite-sized signal subspace, which you find through the eigenvalue decomposition from a smaller autocorrelation matrix estimate $\mathbf C$, and projecting "test functions" (in this cases, sinusoidal sample vectors) into that to check for presence of a frequency component.

Therefore, this can greatly reduce the time spent on decomposing the matrix, but it also doesn't give you the same result as SSA – unless you're very clear about presence of discrete tones in your signal.

Anyway, the method I describe above is called MUSIC, and it would work for your purpose:

My aim is to run a Singular Spectrum Analysis in order to remove some components at certain frequencies and see if they impact the Gaussian distribution

I assume you don't "know" certain frequencies beforehand (otherwise, you wouldn't need any spectral estimation to remove them), but need to estimate them precisely. You could do that using MUSIC / Root-MUSIC or ESPRIT (which is kind of similar in many ways), then synthesize the component (incl. correct phase and amplitude), subtract it and continue your statistical analysis.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.