2
$\begingroup$

Suppose I have several groups of signal measurements, each containing multiple replicates, and I know that within each group the signal "shape" is approximately the same but with variance/error that depends on the signal. E.g. for each group I would have a set of vectors of the same length, which are strongly correlated, but which exhibit irregular variance across the length of the signal.

Ultimately I would like to ask if some test signal X is more likely to belong to group 1, 2, etc. but while taking this variance into account.

Is it then reasonable to treat each group as a multi-variate normal distribution? At each time point group values are approximately normal, although correlations will obviously be high between time points, and the variance across the signal will not be independent at each point, but related to its neighbors. If this is an appropriate modeling then one could simply compare the likelihood of the test vector belonging to the different groups' multivariate distributions.

Why or why not would this be an appropriate single-signal to group-of-signals comparison method? Are there alternative methodologies already in place for asking if some signal belongs to one group or another? Thank you!

Edit for clarification: the data in question are actually nucleosome coverage profiles in specific regions of the genome, which are known to show consistent signals within disease types. The question is then whether a single test signal can be grouped with one or another groups of signals, say a set of healthy or diseased replicates. The measurements can be considered simultaneous, not a true time signal, and variance will be highly correlated with neighboring points. The reason I wish to include the noise/variance when asking this question is because it varies considerably along the signal and between groups, so simply taking the mean signal, for instance, would not capture some of the expected range. I have attached a picture of three such "groups" and their uncertainties.enter image description here

$\endgroup$
2
  • $\begingroup$ Hi Dennie- Thanks for your interesting question. Did you take simultaneous measurements, and thus the noise in each group would be correlated (with an additional uncorrelated component)? Is the test item you are using also known to have been captured together with one of the groups and it is for this reason that it does make sense to include the noise component in the metric? Could you further clarify this bigger picture part of what you are doing and why? $\endgroup$ Commented Mar 28, 2023 at 11:59
  • $\begingroup$ The measurement is exactly as you say - I will update the question to make it more clear! However the test item(s) have been captured separately, I will expand on that as well. $\endgroup$
    – Dennie
    Commented Mar 28, 2023 at 17:06

1 Answer 1

1
$\begingroup$

To summarize my assumptions: The OP is making multiple concurrent measurements at multiple nodes such that both the signal and the noise will have high correlation within the group. This measurement is repeated in time such that for subsequent groups the noise would be independent (and perhaps to some degree the signal as well). Further as depicted in the updated plot by the OP, the SNR is time variant. The user would like to select a measurement from an individual node and make the maximum likelihood estimate which group the node measurement came from.

The OP shows in the plots the dark lines in each group, which I assume was computed from a simple mean for that group, and then the lighter shaded area shows the (time varying) distribution given by the noise for that group. If that is the case, then we see that the noise within the group is NOT itself correlated over each run within that group, as otherwise the mean would converge to a noisy signal itself.

With that, an optimized maximum likelihood correlation metric can be obtained by performing a weighted normalized correlation rather than a standard normalized correlation coefficient of the test signal with the mean signal, using the instantaneous variance of each group as a weighting factor. With that the samples where the noise is lower is given more weight to the correlation result, resulting in an optimized maximum ratio combining with the weight as the inverse of the SNR for each sample.

Specifically this is accomplished as follows:

Starting with the normalized correlation coefficient given as follows with $x$ representing the test waveform (and $x_n$ each sample), and $y$ representing the mean waveform for the set (and $y_n$ each sample within the mean):

$$\rho_{xy}=\frac{cov(x,y)}{std(x)std(y)}$$

Where:

$$cov(x,y)= \frac{1}{N}\sum_N (x_ny_n)$$ $$std(x) = \sqrt{\frac{1}{N}\sum_N (x_nx_n)}$$

$$std(y) = \sqrt{\frac{1}{N}\sum_N (y_ny_n)}$$

The weighted correlation coefficient is similar to above with each of the products weighted with the inverse of the instantaneous variance for the group as $\sigma^2$:

$$\rho_{xyw}=\frac{cov(x,y,w)}{std(x,w)std(y,w)}$$

Where:

$$cov(x,y,w)= \frac{1}{N}\sum_N (x_ny_n/\sigma^2)$$ $$std(x,w) = \sqrt{\frac{1}{N}\sum_N (x_nx_n/\sigma^2)}$$

$$std(y,w) = \sqrt{\frac{1}{N}\sum_N (y_ny_n/\sigma^2)}$$

Thus with each of the terms in the numerator and denominator using the same sample by sample weighting appropriately, the resulting correlation coefficient will still normalize to a maximum value of 1 under noise free conditions (thus whichever result when a test sample is correlated to each set that has the maximum correlation would be the most likely set for that test sample), but this way with the weighting as used, the areas of the signal that are most likely to deviate from the expected mean (the noisiest portions of the signal) will contribute the least to the resulting metric.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.