I have a distance sensor (VL53L0X), which measures the distance to an object. It is placed outside. I currently do 100 measurements in the best available range mode, then I take the median and insert it to a database.

The current distance is about 1m.

This is how it looks like during a period af about 12 hours. As can be seen there is a light dependent error in the data.

I also have a LDR (light dependent resistor) output (log scale on y-axis), which shows this more. At "peak" time the error is about 15cm.


Green: distance in meters, yellow, Voltage from LDR (light dependent resistor). The darker it gets, the higher the voltage of the LDR. At night it's about 2.5V.

Now, the distance hasn't changed in those 12 hours. My question is: how can I compensate this error? I am looking for a term on how this procedure is called.

My first guess was to calculate the difference to 1m for all distance points, then maybe fit the LDR output to that error points.

Data: https://pastebin.com/2fXcSUQB

Column 1: distance in meters, Column 2: voltage from LDR

  • 1
    $\begingroup$ Is it possible to post the data of the two curves? $\endgroup$ – A_A Jan 24 '19 at 17:27
  • $\begingroup$ thanks for the reply. yes, I will upload them in a few minutes. $\endgroup$ – fsp Jan 24 '19 at 17:30
  • $\begingroup$ sorry, took longer. It's uploaded now. $\endgroup$ – fsp Jan 24 '19 at 19:02
  • 1
    $\begingroup$ Could you explain what we see? I don't under stand what is the data you are showing. I can tell it is a range measurement but I don't understand the yellow line (What's LDR)? Please read again your question and try make it clearer. $\endgroup$ – Royi Jan 24 '19 at 19:32
  • $\begingroup$ clarified it a bit $\endgroup$ – fsp Jan 24 '19 at 19:34

Ideally, you would like to explain the LIDAR signal as a combination of one or more other variables, here LDR and/or Temperature and then remove the effect of those variables from the LIDAR signal.

The first part can be achieved with linear (hopefully) regression and specifically Multiple Linear Regression where you would be fitting a model like LIDAR = Constant + LDR_Coef * LDR + Temp_Coef * Temp.

With the availability of just the LDR data (code in GNU Octave but easily transferable to other platforms too):

%Load the data
D = dlmread("myfile.txt",";"); %myfile.txt is the file you provided which is delimited by semicolon.
% Adjust the LDR signal
D(:,3) = log(D(:,3));
% Fit the model using regress.
% For more information see: https://octave.sourceforge.io/statistics/function/regress.html
[B, BINT, R, RINT, STATS] = regress(D(:,2), [ones(size(D(:,3))), D(:,3)]);

After you do this, you get your model coefficients in B and a few other things (more about this later on).

It is now possible to express LIDAR as a function of LDR:

% Plot LIDAR and LIDAR as a function of LDR
xlabel('Time');ylabel('LIDAR / LDR');legend('LIDAR','LDR');grid on;

This now looks like:

enter image description here

This concludes with the fitting part. Now we want to remove the variation due to LDR, which is the "easy" part:

LIDAR_adjusted = D(:,2) - B(2).*D(:,3);
xlabel('Time');ylabel('LIDAR');legend('LIDAR','LIDAR_adjusted');grid on;

Which results in:

enter image description here

The STATS vector contains a number of coefficients that inform you about how good is this fit. In this case 7.6591e-01 1.4952e+03 0.0000e+00 7.8229e-04. So, the $R^2$ is not ideal but not too bad too (that is the first number, it can be between $0$ and $1$ and it is ideal at $1$), the rest of the numbers are not bad too (the error is very small), but if you look at the residuals (that is the difference between the LIDAR and the LIDAR as a function of LDR) it is not exactly noise. Here is the residual (R):

plot(R);grid on; 

Which results in:

enter image description here

There are clearly trends in the residual which means that LDR on its own cannot explain the variation in the LIDAR signal. You can add 'r' as a regressor in there, which would return three coefficients for the model LIDAR = B(1) + B(2)*D(:,3) + B(3)*r and that would "take away" that linear upwards trend as a function of time and improve the error but you still get those two humps because the sensor responses are non-linear.

To conclude with this, if you wanted to add the temperature to the model too, just change the regress line to:

[B, BINT, R, RINT, STATS] = regress(D(:,2), [ones(size(D(:,3)), D(:,3), D(:,4)]);

Assuming that you will add another column for the temperature. The rest of the process remains the same (with reasonable adjustments for removing the components further below).

The other thing that you can do of course is a Discrete Fourier Transform (just on the LIDAR) and retain the first few harmonics from it for detrending. The latter assumes that there is a periodic well defined effect of light (and / or temperature and other parameters) on LIDAR which can be approximated with a set of periodic components. And in that case, the adjustment comes as a function of the time of day "for free".

Hope this helps.

|improve this answer|||||
  • $\begingroup$ wow, thanks for this answer. just a small question: I get a syntax error on the regress = ... line in octave 4.4.1 $\endgroup$ – fsp Jan 25 '19 at 4:31
  • 1
    $\begingroup$ Ah, I missed closing the bracket on 'ones' in regress :/ edited. $\endgroup$ – A_A Jan 25 '19 at 6:42
  • $\begingroup$ Thanks a lot for your detailed post. I took a look at the data today, and to my surprise, it's changed (while the weather hasn't) so I assume this might be some "accllimation issues" with the sensor it self. See this screenshot: i.imgur.com/bh7Wi00.png while it looks like much due to yrange, the values now vary much less. I will let it run for a few days to see what is happening. After the first day I was convinced that it had a very strong dependence on sunlight, that's why I made this post. $\endgroup$ – fsp Jan 26 '19 at 0:07
  • $\begingroup$ @fsp No worries, thanks for letting me know. Do you think you could add a black tube of a few cm in front of the sensor to focus the returns to be coming from directly below it? This will smooth out those two humps at the beginning and end and will limit the angle at which the sunlight causes errors to your signal. If you still get errors after that, then temperature might have an involvement too (?). All the best. $\endgroup$ – A_A Jan 27 '19 at 20:57
  • $\begingroup$ So after a few days, the old pattern started again. So I read the datasheet again. Previously I was confused by their "Ref calibration", I thought I needed to enter the current temperature as parameter, but it turns out it's just some internal mechanism. They recommend recalibrating if the temperature difference is more then 8 °C. I haven't done that, because the difference hasn't been that big. Now I recalibrate every few minutes. If the difference between current and the last value inserted is greater then 5 cm, I also discard that value and recalibrate again. $\endgroup$ – fsp Jan 30 '19 at 19:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.