Timeline for Examples of Independent and uncorrelated data in real-life, and ways to measure/detect them
Current License: CC BY-SA 3.0
16 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Mar 11, 2012 at 22:41 | comment | added | Rachel | @pichenettes I have tried to calculate the Mutual Information, as suggested, and used Kernel Density Estimators for estimating the required PDFs. Unfortunately it seems I have a bug in my code =/ You could have a look at it over here, if you like, perhaps you have a couple of useful suggestions.. | |
| Mar 9, 2012 at 17:40 | comment | added | Rachel | @pichenettes In Matlab, you can compute the 1D histograms (let's call them N1 and N2) using hist, and the 2D histogram using hist3 (let's call this Z). Now (assuming you use 10 bins - the default), N1 and N2 are 10-element vectors, and Z is a 10x10 matrix. How do you compare the product of the vectors to the matrix? Should the outer product be calculated? And in this case, do you do N1' * N2, or N1 * N2'? | |
| Mar 8, 2012 at 16:40 | comment | added | Spacey | @Rachel I have not been able to find anything to do that yet, but in my explorations I have found something called a Kernel Density Estimator. (Scroll to example and download the .m file it has a very nice example code). Anyway, apparently this can help in generating a good 1-D smooth PDF that might be more useful and robust that a histogram approach. I will let you know if I find anything for 2-D. | |
| Mar 7, 2012 at 20:32 | comment | added | Rachel | @Mohammad By any chance, did you manage to compute the 2D histogram? I know this would be sort of estimating the joint pdf, but I have no idea how to do this! | |
| Feb 29, 2012 at 19:43 | history | bounty ended | CommunityBot | ||
| Feb 22, 2012 at 1:23 | comment | added | pichenettes | Yes, if count pairs you can "collapse" the counts on both axes to get the histograms of the marginals. | |
| Feb 21, 2012 at 23:44 | comment | added | Spacey | 1) I see, radial basis functions - I will research those more. 2) It also think if we indeed have the 2-D histogram (joint pdf), then I can also just sum down the rows to get the marginal pdf of x, sum down the columns to get the marginal pdf of y, and THEN take those two vectors' outer products. Should this make a difference in this regard? I do not think so myself... | |
| Feb 21, 2012 at 23:37 | comment | added | pichenettes | 1) A kernel typically used to estimate a continuous density from iid samples is the gaussian kernel (also called RBF). Triangular kernel are sometimes also used. 2) This approach is correct. If you are using Kullback-Leibler divergence to compare the output of your 2D histogramming to your outer product of 1D histograms, you are effectively computing the Mutual Information between a discretized version of your two distributions. | |
| Feb 21, 2012 at 23:26 | comment | added | Spacey | Ah! thanks for that. 1) Regarding the kernel method - what kernel does one use for this? (I recently learnt about kernels as of 1 month ago). Are there particular ones that one uses for this? 2) Regarding the joint pdf histogram method: Perhaps one way to test for independence would be the following: Compute the 2-D histogram as described. Then, compute the histogram of x and y separately, but then, take both histogram's vectors' outer product, (to make a grid). Compare this grid to the 2-D grid. If x and y are indep, they should be the same. Agree? | |
| Feb 21, 2012 at 21:18 | comment | added | pichenettes | Another example : two sines wave at an integer multiple of the same frequency. Null correlation (Fourier basis is orthonormal) ; but if you know the value of one there is only a finite set of values that the other one can take (think of a Lissajous plot). | |
| Feb 21, 2012 at 21:12 | comment | added | pichenettes | "2 physical signals that would be dependent, but not correlated": Let's say we hack the GPS of a NY cab to record a (latitude, longitude) history of its position. There's a good chance the lat., long. data will be uncorrelated - there's no privileged "orientation" of the point cloud. But it'll hardly be independent, since, if you were asked to guess the latitude of the cab, you would provide a much better a guess if you knew the longitude (you could then look at a map and rule out the [lat, long] pairs occupied by buildings). | |
| Feb 21, 2012 at 20:08 | comment | added | pichenettes | Let's say you have two signals $x_n$ and $y_n$ represented as vectors of $N$ elements. You can get an estimate of $p(x, y)$ using, for example, a Kernel density estimator: $p^*(x, y) = \sum_i \frac{1}{N}K(x - x_i, y - y_i)$ where $K$ is a Kernel function. Or you can use the same technique as for building an histogram, but in 2D. Build a rectangular grid, count how many pairs $(x_n, y_n)$ fall in each cell of the grid, and use $p^*(x, y) = \frac{C}{N}$ where N is the size of your signals and $C$ is the number of elements in the cell associated with point $(x, y)$. | |
| Feb 21, 2012 at 19:52 | comment | added | Spacey | (contd) 2) So to summarize: If the covariance matrix of x, and y is diagonal, then they are uncorrelated, but NOT necessarily independent correct? To test for independence was the issue with follow up question (1). However, if we show they are indep, then of course their covariance matrix HAS to be diagonal. Have I understood right? What is an example of 2 physical signals I can measure in real life that would be dependent, but not correlated? Thanks again. | |
| Feb 21, 2012 at 19:50 | comment | added | Spacey | Thank you pichenettes: 1) Can you please elaborate on your first point - I am really having a hard time understanding just how, from two data vectors, x[n] and y[n], I can possibly come up with their JOINT PDF, $p(x,y)$. I can understand how taking a histogram of x[n] will give me pdf of X, ($p(x}$), and the same with Y, but how on earth does one come up with a joint given two vectors?? I am asking concretely - exact concrete mapping of a PDF from observed samples. This is what is confusing me the most. (contd) | |
| Feb 21, 2012 at 19:19 | history | edited | pichenettes | CC BY-SA 3.0 |
added 89 characters in body
|
| Feb 21, 2012 at 19:14 | history | answered | pichenettes | CC BY-SA 3.0 |