A few elements...

Q1

To check whether two distributions are independent, you need to measure how similar their joint distribution $p(x,y)$ is to the product of their marginal distribution $p(x) \times p(y)$. To this purpose, you can use any distance between distributions. If you use the Kullback-Leibler divergence to compare those distributions, you will consider the quantity:

$\int_x \int_y p(x, y) \log \frac{p(x, y)}{p(x) p(y)} dx dy$

And you will have recognized... the Mutual Information! The lower it is, the more independent the variables are.

More practically, to compute this quantity from your observations, you can either estimate the densities $p(x)$, $p(y)$, $p(x, y)$ from your data using a Kernel density estimator and do a numerical integration on a fine grid ; or just quantify your data into $N$ bins and use the expression of the Mutual Information for discrete distributions.

Q2

From the Wikipedia page on statistical independence and correlation:

At the exception of the last example, these 2D distributions $p(x, y)$ have uncorrelated (diagonal covariance matrix), but not independent, marginal distributions $p(x)$ and $p(y)$.

Q3

There are indeed situations in which you might look at all the values of the cross-correlation functions. They are arise, for example, in audio signal processing. Consider two microphones capturing the same source, but distant from a few meters. The cross-correlation of the two signals will have a strong-peak at the lag corresponding to the distance between microphones divided by the speed of sound. If you just look at the cross-correlation at lag 0, you won't see that one signal is a time-shifted version of the other one.

A few elements... (I know that this is not exhaustive, a more complete answer should probably mention moments)

Q1

To check whether two distributions are independent, you need to measure how similar their joint distribution $p(x,y)$ is to the product of their marginal distribution $p(x) \times p(y)$. To this purpose, you can use any distance between distributions. If you use the Kullback-Leibler divergence to compare those distributions, you will consider the quantity:

$\int_x \int_y p(x, y) \log \frac{p(x, y)}{p(x) p(y)} dx dy$

And you will have recognized... the Mutual Information! The lower it is, the more independent the variables are.

More practically, to compute this quantity from your observations, you can either estimate the densities $p(x)$, $p(y)$, $p(x, y)$ from your data using a Kernel density estimator and do a numerical integration on a fine grid ; or just quantify your data into $N$ bins and use the expression of the Mutual Information for discrete distributions.

Q2

From the Wikipedia page on statistical independence and correlation:

At the exception of the last example, these 2D distributions $p(x, y)$ have uncorrelated (diagonal covariance matrix), but not independent, marginal distributions $p(x)$ and $p(y)$.

Q3

There are indeed situations in which you might look at all the values of the cross-correlation functions. They arise, for example, in audio signal processing. Consider two microphones capturing the same source, but distant from a few meters. The cross-correlation of the two signals will have a strong-peak at the lag corresponding to the distance between microphones divided by the speed of sound. If you just look at the cross-correlation at lag 0, you won't see that one signal is a time-shifted version of the other one!