0
$\begingroup$

I already asked a similar question, but did not receive an answer and did not find a solution, I want to make another attempt to get to the bottom of the truth.

There is a signal consisting of rectangular pulses at different frequencies and other noises. The amplitude of the rectangular pulses may vary. The duration of the pulses can be either 100 μs, or vary by 150-200 pulses per 200 μs.

This is a real signal, which consists of pulses at a frequency of 1 kHz, pulses at a frequency of 1.4 kHz and other noises. enter image description here

How can one distinguish the amplitude of these pulses and find out the duration? If more details, I need to find:

  • The amplitude of the pulses is 1 kHz. (the pulses themselves are not important, I just need the average amplitude of each pulse)
  • The pulse amplitude of 1.4 kHz. (the pulses themselves are not important, I just need the average amplitude of each pulse)
  • The duration of 1 kHz impulses (the exact time is not required, I just need “yes” if the duration is 100 µs, “no” if the duration is 200 µs.)
  • The duration of 1.4 kHz impulses (the exact time is not needed, I need “yes” if the duration is 100 µs, “no” if the duration is 200 µs.)

I thought that you can use two band IIR filters, for example 6 orders of magnitude at a frequency of 1 kHz and a frequency of 1.4 kHz, respectively. But after the filter, the amplitude and duration of the initial rectangular signal are lost, and I cannot calculate the initial amplitude and duration of the pulses.

I saw a device that somehow solves this problem on a simple microcontroller, when applying such a signal to the input, at the output I see the amplitudes of both signals, so I concluded that somehow it can be done programmatically. For a month I have been trying to find a solution to this problem on the Internet in both my native language and internationally, but my searches have come to nothing. Perhaps someone solved a similar problem and simply did not share the solution, or no one came across a similar task ... I re-read a lot of pages of “stackexchange” in search of a solution to the problem, but I couldn’t find the answer. I will be glad of any help in solving this problem!

Here is an attempt to simulate this signal in a matlab:

Fs = 100000;
t = 0:1/Fs:0.01;
F = 1.5;
f0 = 1000;
tau = 0.0001;
s = (square(2*pi*t*f0, f0*tau*100) + 1)* F/2;
s = s+randn(size(t))/50;

F2 = 2;
f02 = 1400;
tau2 = 0.0001;
s2 = (square(2*pi*t*f02, f02*tau2*100) + 1)* F2/2;
s2 = s2+randn(size(t))/50;

a=s+s2;

enter image description here

Next, I created mask “b” and tried to use circular cross correlation for verification:

b=0;
for i=1:10
    b(i)=1;
end
for i=11:100
    b(i)=0;
end
b=[b,b,b,b,b];

xcorr = fft(a).*conj(fft(b,length(a)));

At the output, I got this picture.

enter image description here

Here is the result of circular cross correlation with no bias of the original signal, with bias of 30: enter image description here enter image description here

Next, on the threshold, I find the beginning of the impulse, and did the integration over 100 μs and a dump:

temp=0;
i=1;
corrf=[];
for g=1:1000
   if (corr(g)>42) || (temp==1)
      temp=1;
      corrf(i)=corr(g);
      i=i+1;
    end
end
corrf=corrf(1:900);
intdump(corrf,10);

enter image description here

$\endgroup$
  • 1
    $\begingroup$ I think this is a straightforward signal identification as was already answered previously- create a mask of the pulses and compare minimum distance to make decision - is there any further constraining information as to the rate of amplitude change from pulse to pulse or the number of pulses possible? This would further improve the performance under noise conditions $\endgroup$ – Dan Boschen Dec 31 '19 at 15:17
  • $\begingroup$ So your output looks good to me: your test signal has zero offset in time so is aligned with your mask, which has a maximum correlation in the first bin as expected (or so it appears from the picture). The other peaks are the positions in time where a partial number of pulses overlap with your mask (as to be expected) but the largest occurs where they all overlap. Try shifting your test signal to be later in time so that you can see how your cross correlation changes accordingly, and see how you can predict where the signals are from your cross correlation alone. $\endgroup$ – Dan Boschen Jan 3 at 5:31
  • $\begingroup$ (Also try making your 1.4 KHz mask to confirm you can locate either pulse separately as you change the time offset of each.) If the pulses were always present, as you show then there is no reason to make your mask longer than 5 ms. $\endgroup$ – Dan Boschen Jan 3 at 5:33
  • $\begingroup$ Well, I’ll try to experiment with this simulation of the signal. But I still do not fully understand how "xcorr = fft (a). * Conj (fft (b, length (a)))" does this formula work? $\endgroup$ – red15530 Jan 3 at 17:53
  • $\begingroup$ multiplication in frequency is convolution in time. Complex conjugation in frequency is time reversal in time. Convolution is Correlation of the time reversed signal, so the complex conjugate multiplication in the frequency domain is the correlation in the time domain. If done with FFT, FFT are periodic in frequency so the result in this case is a circular correlation in time. You can do correlation in time as I described to get the same result and you may want to confirm that for yourself. $\endgroup$ – Dan Boschen Jan 3 at 17:59
1
$\begingroup$

The following is what I believe to be an optimized approach for performance in the presence of additive white noise when no other information is known about quantity of pulses or their amplitude distributions, beyond that they are 100 or 200 us long rectangular pulses and repeat at the 1KHz and 1.4KHz rates. This can be even further improved if any other further information is known (including learning from the data to see if anything else about the waveform is repetitive and therefore can be predicted once identified).

Note that the composite waveform consisting of the two pulse types combined (pulse type meaning 1KHz and 1.4KHz pulses) repeats every 5 ms.

Create a mask to establish the timing of the pulse types. If the starting time between the two pulses is fixed and known then you can create a single reference waveform that is 0 everywhere except where a pulse could be located, and 1 in those positions using 100 us assumed durations. The longer the mask the better the result up to the maximum expected duration of a burst of pulses but keep it in 5 ms durations to take advantage of the cyclical property for using circular cross correlation.

If the 100 and 200 us pulses are equiprobable then in would actually be slightly better to use 200 us pulses in the mask (the break-even is when the ratio of 200 us pulses to 100 us pulses is $\sqrt{2}$, since 200 us masks would have 3 dB more noise which is offset by the signal increase if the mask allows the accumulation of $\sqrt{2}$ more pulses.)

Perform a circular cross correlation of the mask with a capture of the data. The result will have cyclical peaks showing the starting position of the 5ms composite pattern.

The circular cross correlation is given by the following which could be used as a quick test of the procedure:

$$XCORR = ifft(fft(b)conj(fft(a))$$

Where fft(a) is the fft of your mask and fft(b) is the fft of your data.

Here is a very simple example of that to demonstrate how this is done properly and the expected result since the OP is having trouble with this step:

>> mask = [1 0 0 0 0 1 0 0 0 0 1 0 0 0 0];
>> // create three cases of test data for example
>> dataAlligned = mask;
>> dataShiftOne = circshift(mask,1,2);
>> dataShiftTwo = circshift(mask,2,2);

Thus each would be:

mask = [1 0 0 0 0 1 0 0 0 0 1 0 0 0 0];

dataAlligned = [1 0 0 0 0 1 0 0 0 0 1 0 0 0 0];

dataShiftOne = [0 1 0 0 0 0 1 0 0 0 0 1 0 0 0];

dataShiftTwo = [0 0 1 0 0 0 0 1 0 0 0 0 1 0 0];

The circular cross correlation is done for each case above as:

>> corr = @(data,mask) ifft(fft(data).*conj(fft(mask)));

Below shows the result of this for each case:

>> corr(dataAllgined, mask)
ans = 
   3   0   0   0   0   3   0   0   0   0   3   0   0   0   0
>> corr(dataShiftOne, mask)
ans =
   0   3  -0   0   0   0   3  -0   0   0   0   3  -0   0   0
>> corr(dataShiftTwo, mask)
ans = 
   0   0   3   0  -0   0   0   3   0  -0   0   0   3   0  -0

Given the sparse nature of the mask, the implementation for the circular cross correlation would likely be more efficient in the time domain by repeated multiply and sums at all circular rotations of the mask. Given the mask only has values of 0 and 1 this would be quite simple to compute even in a microcontroller, as the multiply operation would just be a selection of which samples to sum.

If the starting delay between the two pulse types is not known, then do the above with two masks; one for the 1KHz pulses and one for the 1.4KHz pulses.

Once timing is established for the above techniques, then we know when a pulse of each type should appear if a pulse is to be present in that interval. So at this point to detect each pulse use an integrate and dump that would integrate over 100us in two successive durations at all the expected start times. (This can be implemented with a moving average filter that has a width equal to the pulse duration, and taking the samples of the moving average output at each expected end of pulse location.) A decision metric is established using a noise optimized threshold on the integrate and dump output to decide pulse present=1 or pulse not present = 0. The use of two successive durations is to determine if the pulse was 100us long or 200 is long (decoding 1,1 or 1,0 for that interval).

The magnitude of each pulse is the magnitude of the integrate and dump output scaled by the number of samples—- keep a running average of the detected magnitude for each pulse type and adjust the threshold based on the averaged magnitude to optimize the decision for pulse present or not present.

$\endgroup$
  • $\begingroup$ Thank you very much for your answer! I will try to explain in more detail: 1) I do not know the exact start time of 1 kHz and 1.4 kHz pulses. That is, impulses always run (endlessly), and I can start processing at any moment. $\endgroup$ – red15530 Jan 1 at 12:59
  • $\begingroup$ 2) As for the impulses of 200 μs, I will try to explain this: If 1kHz: 100-900-100-900-100-900 ... 200-800-200-800-200-800 (and so on 200 packs) ... 100-900-100-900-100-900. All values are in micro seconds. If 1.4kHz: 100-614-100-614-100-614 ... 200-514-200-514-200-514 (and so on 200 packs) ... 100-614-100-614-100-614. All values are in micro seconds. 3) In this case, as it was in the waveform above, the pulses can partially or completely overlap each other. $\endgroup$ – red15530 Jan 1 at 13:00
  • $\begingroup$ If I understand correctly, you suggest passing pulses through a 5ms mask. But here a problem arises, because I do not know the exact time of the start of the impulse, and I also do not understand what to do at the moment of partial or complete overlap of the impulses. $\endgroup$ – red15530 Jan 1 at 13:00
  • $\begingroup$ For example, I have a sampling frequency of 100kHz. I have two masks for 1 kHz this is an array of 500 samples (5ms) (1-10..0-90 ... 1-10..0-90 ... 1-10..0-90 ... 1 -10..0-90), where ten 1, then ninety 0 and so five times. For 1.4kHz, this is an array of 355 samples (1-10..0-61 ... 1-10..0-61 ... 1-10..0-61 ... 1-10..0-61 ... 1-10..0-61), where ten 1, then sixty one 0, and so five times. Next, you need to perform a "rolling circular correlation." Did I understand this part correctly? $\endgroup$ – red15530 Jan 1 at 18:56
  • $\begingroup$ I wanted to try to simulate these signals in a Matlab first. While I can’t do anything, my experience with the Matlab is very small. Can you give any real example with any signal in the Matlab? $\endgroup$ – red15530 Jan 1 at 19:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.