In a healthcare application, I need to calculate urine flow by differentiating the mass of urine emitted by a person over time. The measuring instrument consists of a load-cell under a fluid container, with its signal being sampled at 2kHz.

My original dataset is here.

The signals are noisy due to the load cell, and drippings over the fluid container, with an FFT showing an exponential-like decay in the magnitude of higher frequencies. Due to this noise, when I try to differentiate the volume signal to obtain the flow signal, I get a very rough signal.

I have already tried smoothing the volume signal in many ways (savatski-golay, gaussian convolution, butterworth low-pass), but by differentiating the smoothed signal I get either a still very wavy signal, or one that is so smooth that it no longer contains the expected characteristics of the flow profile.

I studied a bit about differentiation of noisy signals so as to estimate, say, velocity and acceleration directly from the position signal, and the main suggestion is to use a Kalman filter, which I never got to the point of making work, or even understanding conceptually.

There are some factors influencing on the signal characteristics, some of them due to the physiological model being represented, others not:

  1. Physiological component: the flow is expected to be determined by muscle contractility (mostly involuntary bladder muscles) and urethral shape (affected by static structures like prosthate, and active structures like pelvic floor muscles);
  2. Principle of measurement component: the fluid mechanics of the urine falling into the container, especially "turbulent" elements like drippings;
  3. Instrumentation component: The load-cell/strain-gauge noise, with a much higher frequency than the two previous factors.

So my goal would be to implement a Kalman filter algorithm (I can use Python or C#) that gives me estimates for both Volume and Flow, based on the measured Volume, and that models the physiological component while "removing" the turbulence of the impact on the container (e.g. drippings) and the instrumentation noise.

I don't expect to get a ready answer, but since this is a bit beyond my skill level, I'm quite lost regarding where to start.

  • $\begingroup$ have you tried, polyfit or spline fit to your data ? $\endgroup$ – Fat32 Jul 12 at 20:29
  • 1
    $\begingroup$ You will first need to do some sort of system identification to get a model of your system, which can then be used in a Kalman filter. $\endgroup$ – fibonatic Jul 13 at 16:34
  • $\begingroup$ 1) It seems your "data set" is already differentiated. If so, could you supply the raw data (which I would expect to be monotonically increasing). 2) I second the "system identification" 3) If you want to classify the "flow rate" into contractions, drips, etc you need to formulate the criteria and perhaps use wavelets to isolate significant events 3) If your load sensor is introducing inband noise you have a physical problem that probably means you have to examine the mechanics and understand it. 4) I really don't think you are geting off by simple filtering: Kalman or not. 5) Think a lot. $\endgroup$ – rrogers Jul 16 at 17:47

I ended up with a quite satisfactory solution, not by using a Kalman filter, but by using the Savitzky-Golay differentiating filter.

The algorithm is described more or less like this:

  1. In a for loop, apply a running window to get a segment of the unfiltered volume signal around a range of given time instants, as measured by the load cell;
  2. For each segment, perform a polynomial interpolation. I am using quadratic;
  3. Differentiate the found polynomial;
  4. Evaluate the derivative at the given time instant;

This gives me a very natural-looking flow curve, based on the premise that the flow cannot physically contain high frequencies. So the "model" ended up being a well-chosen low-pass filter, with the bonus that I can differentiate a continuous function (the smoothing polynomial) instead of the discrete differentiation that tends to amplify the noise.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.