PID With Prediction

Question

First, I just want to start off by saying that I am a programmer, and am no expert in complex algorithms, and will probably not be able to apply complex pseudo-code or other descriptions of a solution. I have already asked a similar question on one of the programming SE forums in hope of finding a code sample, but no one there had a solution.

I have a stream of data that I get at around 10 snapshots per second. I wrote a (C#) controller that takes this data and adjusts some data-gathering parameters based on the distance away from the expected result. Currently, I just do a linear scale operation on my data so that the farther away I am from the expected result the more it corrects. The problem is that the incoming data stream is delayed by somewhere between 0.5 and 2 seconds (I can calculate that at runtime). Because of this delay, it is correcting for results from a while ago and is constantly overcorrecting and even sometimes correcting in the wrong direction. I am looking for an algorithm to do the following:

• Implement a correction algorithm (I'd prefer PID) that will attempt
  to hone-in on the optimal value
• Predict a certain amount of time (or number of datasets) in advance based on the history of corrections and results; the values won't be exactly linear, but they will probably be close

What options do I have in terms of algorithms that can accomplish this? As I said earlier, I am a programmer and therefore code samples would be appreciated; I am not great at converting complex pseudo-code in to a working implementation.

EDIT: To address some questions in the comments...

• I did make a small abstraction in my question. What's actually happening is I get a stream of data, calculate 3 (separate) numbers based on that data, and try to optimize those separately; each of them having their own output parameters. What I need from this algorithm is to be able to operate on a single number and optimize that. I would give it the history and a result and it would modify it's own output parameter. I can then just use the algorithm 3 times for each parameter.
• The rate does change slightly (usually at a slow pace), though I should be able to calculate the change if needed. But I don't need exact enough numbers to require variable prediction times; I can just pick an arbitrary number of snapshots to predict in advance and it should get me close enough to my result.
• It is a physical process and is noisy and generally not very accurate. I am taking my data stream and doing a bunch of estimations and tracking and things like that, so there are often outliers. There are many factors that make a basic model relatively easy, but anything more accurate would be completely off. Basically, I can't really model it to the extent of being useful. Just a note, the output values that this algorithm has to control are the 'drift', or rate of change. Currently I just scale the values so that the farther away the input is from the ideal the higher the output is (pretty simple).
• Are you asking for me to provide a sample of my data? I should be able to at some point, but it may take a few days for me to get the chance. To test it, I was (and am still) planning to just put the algorithm in a test program with a slider and a noise generator and see what happens.

Answer 1

This is just a start, because I am not sure I fully understand the problem yet.

Here is a plot of what I think is happening. The blue curve is the "truth" of your quantity and the red curve is a delayed, noisy, measured version of it.

Does this look like your problem? If not, how should it be changed to make it look more like it?

I suspect that a modified Kalman filter (a Kalman predictor) will be able to give you something, but I want to understand what your data looks like first.

Another question: How do you know that there's a delay between the "truth" and your measurement? I'm trying to get a handle on what numbers we have to play with to correct for the time delay.

---

Scilab code to generate the figure.

// Q 12924
N = 1000;
process_noise = rand(1,N,'norm');
mid = filter(ones(1,20)/20,1,process_noise);
alpha = 0.99;
truth = filter(1-alpha,[1 -alpha], mid);

clf;
plot(truth);

max_delay = 50;
true_delay = floor(rand(1,1,'uniform')*max_delay);

sigma = 0.01;
measurement_noise = rand(1,N,'norm')*sigma;

measurements = truth(max(1,[1:N]-true_delay)) + measurement_noise;

plot(measurements,'r:')