5
$\begingroup$

I am new to signal processing and would really appreciate your help. happy to provide more context/details.

In my research, I have a system that has many sensors $X_1, \ldots, X_N$ and using this information, the system makes a prediction on a value, $A$. For simplicity, let's assume that $A$ is binary, and please keep in mind that this is a real system I am trying to improve so that means that I can tell whether $A$ was right or wrong (the system can be checked against "ground truth").

Now each sensor makes measurement sporadically (not necessarily randomly, but definitely not some obvious pattern) for reasons that are too complicated to go into right now. This means that $X_1$ for example can resemble this: $[\textrm{2, 5, 1, na, na, 1, na, na, 89, $\dots$}]$ where each index of this vector is a time step, and $\textrm{'na'}$ represents times when the sensor did not take a measurement. This means that the vector actually contains two sets of information: the value of each individual measurement, and when the measurements were done. There is a third value (but maybe that's not important for now) which is about how much effor each sensor put into collecting this information.

Now my problem is that I want to measure the dependence between $X$'s. i.e. given a specific vector of measurement from, say, $X_4$, what is the probability that $X_{10}$ was influenced by $X_4$. Specifically, I am looking for a number (could be a probability, or a correlation coefficient) that would say that $X_{10}$ was influenced by $X_4$, and I want this number for all pairs of $X$'s.

What I am trying to do is to factor in these influences into the system that is trying to make a decision. Since I know the ground truth, I can tell how good my "dependence measure" is.

I have zero background in signal analysis and I thought that you all might know something here. I think this is an interesting general problem and I would really appreciate some help.

$\endgroup$
  • 2
    $\begingroup$ Uncertain Time-Series Similarity: Return to the Basics [PDF]. Follow the references or search for "uncertain time series similarity", rather than "sparse time series" for more leads. $\endgroup$ – Emre Apr 15 '16 at 22:52
  • 1
    $\begingroup$ I have found that the way to do it is using a geometrical method (I will write more later, but basically I calculate the difference between consecutive non-NA values). Other methods I came across to check out are Point processes (en.wikipedia.org/wiki/Point_process). Will write more later $\endgroup$ – pipo17171 Jun 15 '16 at 2:50
0
$\begingroup$

If you have enough sample measurement data, you could reconstruct a time series for each sensor pair that eliminates any time step where either one or both of the sensor pairs are na. Then you are left with a time series for that pair without any na's and can proceed with standard time series approaches for calculating correlation. Then repeat those steps for each sensor pair.

For example:

X_1 = [2, 5, 1, na, na, 1, na, na, 89]
X_2 = [na, 4, 9, 8, na, na, na, 2, 9]

after reconstruction becomes:

X_1 = [5, 1, 89]
X_2 = [4, 9, 9]
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.