0
$\begingroup$

I have this data that need to be downsampled so that it can be fed into a classifier. The issue is that I need to downsample from 1000Hz into 20Hz. If you see the graph below, the blue line is the original data, and the red line is my attempt at downsampling... which looks awful.

Is there any way I can downsample this correctly? If it matters, I'm mostly concerned about maximizing correlation. Graph of original vs. downsampled

Note: I'm working in MATLAB. I used the decimate() command. I have also tried spline, downsample, and resample.

Improve this question
$\endgroup$
2
  • $\begingroup$ What is the sampling rate? $\endgroup$
    – Dsp guy sam
    Commented Apr 16, 2020 at 8:46
  • $\begingroup$ @J Doe, I'd probably go about doing this using one of the generative, statistical models (e.g GMM). Creating a model means defining the model parameters (mean and variance for a 1D GMM) and after you've done that, it should be easy to sample from that model. By sampling, I mean extracting an arbitrary number of samples from the model. While your data doesn't match a normal distribution perfectly you can still use it. $\endgroup$
    – dsp_user
    Commented Apr 16, 2020 at 15:01

2 Answers 2

Reset to default
0
$\begingroup$

As said by Marcus Müller, I doubt you cannot contains enough information you desire via any traditional down sample.

Take the blue line for example, there are 10 impulsive peaks in 50 sample data with 1000 Hz sampling frequency (i.e., sampling period is 10^-3 second ). So the frequency of the impulsive peaks are around 10/(50*10^-3)=200Hz.

According to Nyqust Theorem, the minimum (ideal) sampling frequency of your signal is 400 Hz. So, it's impossible to recover after you down sample it to 50 Hz.

As you mentioned in the comment, you were given specific instructions to downsample it to 1/50 the size. This instructions look like problem of a features extraction/dimension reduction from periodic signals.

As suggest by Marcus Müller, a neural net is a king of down sample tool, but some-how we cannot explain what happened in that dark block. There are some other options for features extractions.

Since features extraction problems are very commonly seen in the field of Natural Language Processing, some techniques have been developed. One classic example is Mel-Frequency Cepstrum Coefficients (MFCCs) [1]. Typical sampling rate for audio is 48KHz, but MFCC of a signal are a small set of features (usually about 10-20 elements).

However, you probably cannot obtain result good enough if you apply simply MFCCs directly. Because the band of frequency components of your signal seems to be different from that of audio. In the field of signal processing, the more understanding about target signal, the more suitable/powerful tool/filters you can design.

If you want a fast and workable (maybe) solution without utilizing knowledge about signal properties. Try blind sour separation like Independent Components Analysis (ICA), (Principal Component Analysis)PCA, and NMF (Nonnegative Matrix Factorization).

[1] https://musicinformationretrieval.com/mfcc.html

Improve this answer
$\endgroup$
0
$\begingroup$

he issue is that I need to downsample from 1000Hz into 20Hz

Well, who says your data can be represented by 1/50 of the original amount of samples?

I don't think it is – the bandwidth of your signal very clearly exceeds half of the resulting sample rate; in fact, your peaks seem to have a single-sample duration, so the bandwidth currently have (1000 Hz / 2 = 500 Hz) would be merely sufficient to represent the signal as is!

So, classical linear, aliasing-free approaches that keep all the signal in the target bandwidth untouched and free of aliases don't help – since that target bandwidth doesn't (and can't) contain the info you want from that original signal.

So, you'll need to come up with something other than trying to reduce a signal below its bandwidth. Sadly, we know nothing about your signal, but it stands to reason that the PCM representation (as sequence of samples) is simply not suited to your problem.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thank you for this response. As you suggested, I want to refrain from downsampling this data because it doesn't make sense to me to do so, but I was given specific instructions to downsample it to 1/50 the size. The signal is the labels for a dataset, and it needs to be downsampled because the features for the dataset was calculated using a running window. (So if the original dataset had 500 labels, the feature matrix has only 10 rows. I was told to train by downsampling the given labels; then, I need up upsample the predictions back up to 500.) $\endgroup$
    – J. Doe
    Commented Apr 16, 2020 at 8:44
  • $\begingroup$ someone might have meant "reduce the data size by a factor of 50", not necessarily the kind of decimation you do here. That would be the classical thing that you'd train a neural net for, by the way. $\endgroup$
    – Marcus Müller
    Commented Apr 16, 2020 at 8:47
  • $\begingroup$ So to reduce the dataset labels by a factor of 50, I need to train a neural net (did I understand that properly?) $\endgroup$
    – J. Doe
    Commented Apr 16, 2020 at 8:50
  • $\begingroup$ @J.Doe No, the kind of downsampling you need is random division of your data into training(1/50) and testing(49/50) data. If that is not what you want, then the other thing which i can think of is projecting your 500 dimensional feature vector onto 10 dimensional space in order to reduce computational complexity and other things. This can be done via different set of techniques which should be asked at Data Science StackExchange website. $\endgroup$
    – DSP Rookie
    Commented Apr 16, 2020 at 10:01
  • $\begingroup$ I doubt this is referring to splitting up the dataset, it would be a weird training/test split to suggest. It is typical to do a 75/25 training/testing split for example, or 50/25/25 training/validation/test $\endgroup$
    – Engineer
    Commented Apr 16, 2020 at 12:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.