1
$\begingroup$

I have a single "pulse" signal (looks like a gaussian but it is not exactly), followed by several bounces, with a constant period, that probably come from the electronics used (I don't have access to this).

Specifically, it is a light pulse (I don't have access to measurement of it, I assumed Gaussian for simplicity), reflected on a surface, and measured by a light sensor, at 100 MHz.

Measured Data

I need to study the tail of the main "pulse" (the right side of the main peak, i.e. the right side of the gaussian pulse), but this is precisely where bounces happen, and overlap my signal of interest !

Is there a way to remove these bounces?

I don't have a model of the bounces, do you think it is reasonable to think that the measured signal y(t) = (x * h)(t) where * is a convolution, x is the input signal, and h captures the bounces process?

I can reproduce this experience many times with many pulses: bounces always come the same way, with the same period, and seem proportional to the input pulse's amplitude.

What algorithm can I use to remove these bounces?

Example signal:

import base64, zlib, numpy as np
x = np.frombuffer(zlib.decompress(base64.b64decode(b'eJwdUAd4FNUWpobwEVDKYyewt82du3dmdnZ22pJkFxFRMVRRHiBgEIK0oPRexEAgUhKqRIo06VUeoEQjfOADFKTE8EKRUAWCJlICCITy7uf3f+c/Z075z7mTLeVLpVKmVOq76qsuPfel+Y43TvY50rdSjjRB2ie9lbg6MSPxK2mgNFFKkwp9tXz5jYkvxefz/dk4w5cqXZXuS5ekWomzEwsT5yWWi7l10l5pmDRSGpDYtWlLkIsyyFE6gy3kCVqObhtvhKaZN8MF1lI7x5nvdvIueEMjcc1WNYslnU2KJs9MLk4mKYNSdqRUpDjRIdF10XPR+Fgk9n4sM7YqVhAril2P3YtVxqo0r9L8Sexu7GrsVCxfVKbG0mMtYr5YWXR/dG40LapF76f8kJKd8k6KP+Vm8u7krOSuycHkasn5SaOSkpOqJJ0QG8c169zMbdao2cPI+ciByMbIvMj4yIeRdyItIsFIYqRW5JF30zvjHfG+9TZ5S73Z3mRvqJfudfXaei08z9M92Uv0XvJqeVW9h+4fbon7q3vELXB3uF+5X7i57nR3vPux28ft7nZyU91X3STXdoMucSW3nhvnPnMqnFLnknPGOekccr53djnbnA3OSudzZ46T5Ux0hjjpTlenrfOKYzrIaeDEOZV2uX3ZLrJ/sgvs7fZKe76dZY+1B9jd7DZ2c9u0kV3frmE/ssqsi9ZJ64C1y1pv5VnZ1gRrqNXX6mKlWq9YnqVb0Gpk1bGqWffCJeHD4d3hdeHF4azw8HB6uEu4dTgS5mF/uH44Plxp3jF/Ny+Yp82jZoG5y9xqrjeXm/PNbHOqOdkcbWaY/c0+Zk+zk/mmmWLaJjebmnXMKubfoduhK6Gi0KHQd6FtoRWhnFBmaHRocCgt1CnUKpQUMkIwlBB6YVQYt4yzxmFjj7HRWGLMMsYaHxm9jW5GqvGK4RiGIRs+I954GrwZPB/cF1wVzA5OCg4J9gqmBlsEI0EnyIJNgrWDNYOP9Ft6kX5MP6If1n/Qd+gb9S36ZsGr9fX6On2DvlxfpOfoC/T5+mz9U32KsBn6ZH2k/rE+Qh+mD9b76O/r3QRS9ahu6kE9oDfV4/Vq+jOtUivVTms/avu1ndpqbaY2XvtY66u9rbXQLC2oYa2B9kgtU6+pF9Sj6k51jbpC/VzNVseoQ9R+am+1o/qamqQ6alhV1IZqglpHravWUO/xUn6fP+C3+G+8hF/kl3khP8j38WPCH+P5/AAvFh1X+SV+jT/mNdTqaiV/zquqNdWq6mP+kFcXGs/4DVH/nd8VPSf4f8VEMT/Ed/KtfDvfwufyMbwv789781TOeWPekCfw24GTgQKBnYG8wITAgECvwLsBN9Ag8IJVsgp2luWzHexrtpOtZNPZODaeDWFp7DWWzCLMZSrzM8hMlvRPrLMoe52lMMow00TdFjmbvcH+zbqz91i6mM5iU9hINkrwHDaTTRRqs9kStlDwXJbHlrIcUZnIPmXZggexPqw/yxD8OnuVtWItmcHqsjhWjT1TypUSpVA5qfyi7FYWKzMEMpWhyttKW6WjsIiClUZKA+Ul5Qk9Sb+h++k+uoJOpMPpXJpH59AMOpJOp7k0k46hY+kiupBm02Eil0mX0M/oAFHPpivpMjqLfknz6a/0F3qYFtG79Al9SP8SHC+0awvtSlpPaaI0VmoK1FN8ystKFaWSxilxyn1aTAvpFVpOT9NNdA3dSjfSHJpGO9IOtDX10yfyVfmGfFneL+fJ0wSmyr1kR5ZkJhtyY7mEnCXXyS1ylOwiO0kBOUH2kdVkCVkr4rVkHplOviR7yBYyi+SSTeQI2S6y60g+OUC+IRvI1+RnUk6ukfPkComXdZnK9eQGckTuIEeFflhuLb8ntxVxRO4k95Dby57syu3kLsJk2S/2u/LL8mNymzwiT8hNcojsFXecFvuyyQQyhUwj/YhLNGISg9Qh5fiywEV8DG/FG3A+/lHwVDwaf4pn40m4G34XD8Qf4e64HW6D++AM3B93wT3wcJyNx+IBuB/+BC/DS/EEnIWX4z14JV6Al+BdeD/+Dm/De3ERLsN/4P/hQnwPJ5BapAyX4r9xdXJf+FL8TKAMF+NTYn8ZPiP6t+Cf8AlcgHPxdJyH14otg8XuXuKOVljGEm6CFRyPC9H36AoqRafQJDQKLULbUBZqhdqg8eL7Q+RHAdQWDUJdkSbijmga6oEaIY7aockiZ6EIGoo+EX0dUH/0mVDagRaiuegsqoGr4kOoBD1DCfgR+hPdEJm6uB6+g8pRXUxwffxcRA3FBX4cJ9AQI1wN30LX0AvReR2tQQvQBpSPctGryESdUW/UHP0Fr8HbMAEdh7vgVrgfFsNlcBL8CA6E82E6TIIB2A6Og31hfVgXavBNqMCqsDrk0IYh+DeoBU3YGbaEMpRgRzgWDhL9QTgMzoMjYHf4AZwDt8A8mA1zhXI5PAv3/OProxfwKPwZVkA/qopuwFuwjvgPL8TuYvgYxqELcDNcAw/BK/A4nAaHi+nNQjsNpgrVoeIGABtCD3aCEJ4D+eA2qA3vgE1gPfgJFIMNYAYYAXLAHME9wQdgGJgPBoEgCIAPwQSQDjiIgTSQBfqBNqAzyAVfgKVgCBgNVojp7WIqUyjeAMfBYrBRqD0FheAgKAAPwL9gDXgVlIJG4rWNYCk4A16GFqQi9wDUh03hU3AOlIDnoC68K5QOgivgnvCLwWSwBPwH5IF2oD3IAGNAD9ASeOKWnsJrQAG6qDQERf5j/gp/APhBif+y/6FfBtXBJf9dfwORqQF+81/yP/VDUAWc8Z/3J4KIUKjwx4vpVBED0AS0FnrtxSs9oToGDABtgSleOR1MER3/B4HIB+o=')), dtype=np.float16)
$\endgroup$
12
  • $\begingroup$ Removing the completely unrelated example; it's just going to confuse people. We know that we can convolve an impulse response with a a Gauss, we need an actual plot of your actual data! And the question you asked in your edit is a good one, but one completely separate from the rest of your question, so you should ask it in a separate question (it would be a duplicate, however, because "how do I de-convolve" has been asked before). $\endgroup$ Commented Mar 26 at 13:34
  • $\begingroup$ @MarcusMüller My example (I re-edited to re-include it) is really very close to the real signal indeed. The question in this example is the same as my initial question: how do I recover the original signal without knowing the impulse response (h in my example). So it's not just a simple deconvolution... $\endgroup$
    – g6kxjv1ozn
    Commented Mar 26 at 14:36
  • $\begingroup$ You say it's close to the real signal. Cool, so you can already tell what is relevant, meaning you know your system well enough! So, why ask us? $\endgroup$ Commented Mar 26 at 16:10
  • 1
    $\begingroup$ @MarcusMüller Thank you for your comment. It is a light pulse (I don't have access to measurement of it, I assumed Gaussian for simplicity), reflected on a surface, and measured by a light sensor, at 100 Mhz. I wonder if we can do a deconvolution if we don't have h... Any idea is welcome! $\endgroup$
    – g6kxjv1ozn
    Commented Mar 27 at 11:32
  • 1
    $\begingroup$ could you check whether the plot of your signal that I added to your question is correct? $\endgroup$ Commented Mar 27 at 14:27

2 Answers 2

2
$\begingroup$

No, probably not.

If your convolutional model was based on the system being excited by your input being a linear one, it is almost certain you'd see negative amplitudes on the output.

Your graph doesn't show these, so that model is not sufficient. Probably, at least. And if the system is non-linear, there's very many ways in which it can be ambiguous! (linear systems are non-invertible, too, but their uncertainty is additive, and of isolated frequencies.)

It would make way more sense to talk about an actual signal plot, ideally with the data uploaded somewhere; this looks pretty hand-drawn to me! And by hand-drawing it, you unconsciously pre-selected features of the actual trace to keep, because you considered them important to draw, and added, involuntarily, features that might or might not have been there.

So, the step to actually identify the system you're dealing with is setting up a good test fixture – something that should give you very similar outputs, as far as possible. Actually compare these! are they always having the same distance between local extrema? What about the height of maxima and minima? Do some statistics! A 2D heatmap might help. If the distance between the maximas varies wildly across different measurements, but you assume the stimulus has a relatively fixed duration, for example, then that would indicate your system is not time-invariant, and hence not a convolution.

If you want to know whether your system is convolutional, you should by starting to tests individual aspects of that asssumption. A relatively easy one to test for (given good SNR) is linearity; if you can modulate the amplitude of the stimulus, does the output amplitude vary by the same amount?

You can't see that from looking at a single plot. You need a lot of them. Even if you can't modulate anything, you might still have a stochastic model of the input amplitude – for example, the amplitude of the stimulus being drawn from a normal distribution. Since linearity should but scale that, you should see a normal distribution of output maxima, as well.

So, do statistics. You'll have to still make a lot of assumptions (especially this), but that can be totally fine – after all, you're noting them down, and would be able to show that your model is good enough to predict the output you're getting.

Sadly, this all means that we're not able to help you infer model properties at this point; we are simply not given enough data, and you're (probably unintentionally) unhelpfully secretive about what it is you are looking at: "from the electronics" really helps nobody. Telling us something like "this is from the sonar receiver amplifier", and giving us a properly labeled graph including time scales might actually give domain experts something to work with.

$\endgroup$
4
  • $\begingroup$ Thank you. you'd see negative amplitudes on the output: why? See my example in the edit of the question: x is positive, h is [1 0 0 ... 0 0.2 0 0 ... 0 0.1 0 0 ... 0 0.05 0 ...] and y is positive too. Which argument tells us there should absolutely be some negative parts of the signal? $\endgroup$
    – g6kxjv1ozn
    Commented Mar 26 at 13:27
  • $\begingroup$ @g6kxjv1ozn as said, it's very unlikely for a physically occurring impulse response to only have positive values, unless you know very drastic things about your system that you forget to tell us! In that case, please do tell us, by editing your question. $\endgroup$ Commented Mar 26 at 13:37
  • $\begingroup$ Thank you again. I will think about including some real data. About your questions: yes there is some linearity: if the input signal is * 2 in amplitude, then the bounces are also * 2 in amplitude. Also, yes, the distance between the local maximas is constant, always the same. $\endgroup$
    – g6kxjv1ozn
    Commented Mar 26 at 14:39
  • $\begingroup$ sorry, I'm occupied otherwise right now. I'll have another look once you've added real data, please notify me in the comments as soon as you do!" $\endgroup$ Commented Mar 26 at 16:11
1
$\begingroup$

With the uncertainties involved, no, deconvolution is not something you could apply here.

two factors:

  1. mismatch of physics and model
  2. uncertainty larger than information

to the first factor:

your system gives you an amplitude, but that's a nonlinear function of the result of convolving the field amplitude of a light pulse with the impulse response of a system. So, not LTI, not convolutional. Light can interfere with itself, so that two delayed and phase-shifted components don't have their sum amplitude.

However, for your observational bandwidth, that might not matter much; it's very low. I've been trying to get information on the experimental setup, especially expected delay spread, out of you, but it's been impossible, and I'm tired of asking; I don't think I should blame you, because I don't think you can tell me:

don't have more precise information @MarcusMüller, I'm not the person who does the experimental design, I only usually work on the data.

So, this is a problem for someone who understands the physical problem domain, not for you. "Needing to know what one is looking at" sadly makes a difference when looking at data! The rest of the questions I'd be asking would all be of physical nature (coherence time of your light source, especially), and it really seems you're not the right person to ask! So, the honest answer here is that in any case, you can't recover the information you want from the observed signal; it would need to be someone with an in-depth knowledge of the physical system.

To the second point:

Think about it: you would need to deconvolve with your prototype input pulse that you assume is in the order of a few µs; however, the main pulse width is narrower than even 1 µs, so the system we are looking at must be shortening the impulse. That is thoroughly possible – just not physically possible in a system that just observes intensity and is fed with non-coherent light (which is the only way we could salvage the option that you can model this as convolutional), unless it is a high-pass system. But looking at the spectrum of your recording reveals no indication of such behaviour. SO, your system does something to the light which is much finer resolved than your initial search space for possible widths of the input Gaussian are, you get ambiguity through the nonlinear behaviour of intensity observation, and you thus won't be able to resolve this to a reasonable degree of certainty.

By all means, try: you can deconvolve your observation with Gaussians of different width (variance); in absence of physical boundary conditions, you get equally likely results for the impulse response, which, again, is not guaranteed to be purely positive (even if you like to assume that in your question).

So, concluding, there's two factors that make it impossible to extract the impulse response from your observation:

  1. The system from all we know is not LTI, hence no impulse response exists, and
  2. finding alternative models for describing the system mathematically are impossible, because the uncertainty about the properties of the system by far exceed what you know about it.

I'm afraid that if all you told us about what you're looking at is true, then 1. is impossible to solve. However, 2. might very well be solvable by sitting down with the people running the experiments, learning about the physical modelling, and then looking at the dispersive non-linear models that people employ for IM-DD (intensity modulation, direct detection – exactly what your photsensor does!) in fiber-optical communications. Good luck!

$\endgroup$
7
  • $\begingroup$ 1/2 Thank you for the answer! Are you sure about "mismatch of physics and model"? I have seen in the past that bounces can appear from electronics parts (impedance mismatch causing ripples, bounces, etc.). So couldn't the reason be purely electronic between sensor and data collection? With this hypothesis, can we assume LTI? $\endgroup$
    – g6kxjv1ozn
    Commented Mar 27 at 16:55
  • $\begingroup$ 2/2 If we assume LTI (even if it is finally wrong, I don't lose anything to test anyway...), what would be the next steps to try? $\endgroup$
    – g6kxjv1ozn
    Commented Mar 27 at 16:58
  • $\begingroup$ @g6kxjv1ozn 1/1 yes. I wrote this answer after very carefully considering what you were certain about. I also don't understand how you jump from "I have little insight into the physics of this system" to "I want to argue based on what I perceive could be physics of the system, even if the answer says that based on my information, the physics of the system are not like that". $\endgroup$ Commented Mar 28 at 10:58
  • $\begingroup$ @g6kxjv1ozn 2/2 I already said that in my answer. Nothing. Assuming it's LTI still gives you not enough useful information to decide what the most likely result of deconvolution is. That is the whole thing about "To the second point" in my answer. I even gave you a "by all means, try" section for you to do after reading this answer. Your approach is insufficient, your model is insufficient, and the data you have on the stimulus is insufficient. You basically have nothing based on which you can continue the analysis, I'm afraid! You need to refine your model. That means understanding the… $\endgroup$ Commented Mar 28 at 11:00
  • $\begingroup$ … experiment, on a physical level! Saying "I only deal with the data, I don't look at the thing about which the data is" doesn't exactly lead you to success, anywhere. $\endgroup$ Commented Mar 28 at 11:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.