Consider I am given two functions of one random variable each for example x=cos(at),y=rect(bt) where a and b are random variables.And I am given Probability density function for a and b then if I am asked if the two functions are independent or not so, I want to confirm that before proceeding I will have to convert the pdf of a and b to pdf of x and y or can I directly proceed with pdf of a and b by proceeding I mean checking if joint probability density is equal to product of marginal probability density.

  • $\begingroup$ If $A$ and $B$ are independent random variables, then $g(A)$ and $h(B)$ also are independent; no need to check anything or do any pdf conversions etc. If $A$ and $B$ are not independent, then in general $g(A)$ and $h(B)$ are not independent either. $\endgroup$ – Dilip Sarwate Jul 5 '19 at 11:30

The canonical definition of independence of two random variables $X$ and $Y$ is

$X$ and $Y$ are called independent random variables if for every choice of Borel sets $B_1, B_2$, the events $\{X \in B_1\}$ and $\{Y \in B_2\}$ are independent events, that is, $$P\{X \in B_1, Y \in B_2\} = P\{X \in B_1\}P\{Y \in B_2\} \tag{1}$$

If you don't know what Borel sets are, rest assured that every set of real numbers you have encountered (and many more that you have never even dreamt of) is a Borel set. Choosing $B_1 = \{x\colon x \leq u\}$ and $B_2 = \{y\colon y \leq u\}$, Eq. $(1)$ tells us that $$P\{X\leq u, Y \leq v\} {=} P\{X\leq u\}P\{Y \leq v\}\tag{2}$$ which can also be expressed as $$F_{X,Y}(u,v) = F_X(u)F_Y(v).\tag{3}$$ It can be proved that if $(3)$ holds for all real numbers $u$ and $v$, then $(1)$ also holds and so $(3)$ is usually taken as the operational definition of independence of $X$ and $Y$:

$X$ and $Y$ are called independent random variables if $$F_{X,Y}(u,v) = F_X(u)F_Y(v) ~\text{for all}~u, v \in \mathbb R.\tag{4}$$

If $g(\cdot)$ and $h(\cdot)$ are real-valued (measurable) functions, then the random variables $W = g(X)$ and $Z = h(Y)$ are independent whenever $X$ and $Y$ are independent. This is because the events $\{W \leq u\}$ and $\{Z \leq v\}$ are the same as the events $\{X \in B_1\}$ and $\{Y \in B_2\}$ respectively where $B_1$ and $B_2$ are the pre-images of the sets $\{x\colon x \leq u\}$ and $\{y\colon y \leq v\}$ respectively under the maps $g(\cdot)$ and $h(\cdot)$. That is, \begin{align} B_1 &= \{w\colon g(w) \leq u\},\\ B_2 &= \{z\colon h(z) \leq v\}, \end{align} and from $(1)$, we know that $\{X \in B_1\}$ and $\{Y \in B_2\}$ are independent events. Thus, we have that $$P\{W \leq u, Z \leq v\} = F_{W,Z}(u,v) = P\{W \leq u\}P\{Z \leq v\} = F_{W}(u)F_{Z}(v)~\text{for all}~u, v \in \mathbb R $$ and so $g(X)$ and $h(Y)$ are also independent random variables. In summary,

Functions of independent random variables are independent random variables.

  • $\begingroup$ In this case W(X,Y) and Q(X,Y) should also be independent isn't it. $\endgroup$ – Buzz bee Jul 5 '19 at 18:27
  • 1
    $\begingroup$ @Buzzbee No, your notion is incorrect. See Fat32's answer. $\endgroup$ – Dilip Sarwate Jul 5 '19 at 19:57
  • 2
    $\begingroup$ @Buzz bee: It's best to develop some inution for the concept of independence rather than just worrying about formulae. In the case of W(X,Y) and Q(X, Y), the random variables X and Y are in both functions, so clearly the value of W(X,Y) is going to related to the value of Q(X,Y) and vice versa. So, they ( W and Q) can't be independent because independence of two random variables ( W and Q can still be thought of as RV's ) says that knowing the value of Q doesn't provide any information about the value of the W and vice versa. Think of dependence as being related to providing information $\endgroup$ – mark leeds Jul 5 '19 at 20:58
  • 1
    $\begingroup$ @markleeds +1 But while you are correct in pointing out that W and Q depend on both X and Y and so cannot be claimed to be independent, there are instances when for specific cases, independence does hold. For example, if $X$ and $Y$ are independent $N(0,\sigma^2)$ random variables, then $R=\sqrt{X^2+Y^2}$ and $\Theta=\arctan\left(\frac YX\right)$ are independent random variables even though both $R$ and $\Theta$ are functions of both $X$ and $Y$. I tell you this against my better judgment because BuzzBee will forget the caveats that this is a very special case and assert that ..... $\endgroup$ – Dilip Sarwate Jul 5 '19 at 21:30
  • 1
    $\begingroup$ .... $g(X,Y)$ and $h(X,Y)$ are independent random variables for all choices of functions $g$ and $h$ and for all possible random variables $X$ and $Y$. Just watch for his next question on dsp.SE..... $\endgroup$ – Dilip Sarwate Jul 5 '19 at 21:34

It happens that if $X$ and $Y$ are independent then so will their functions $g(X)$ and $h(Y)$ be; but not $g(X,Y)$ and $h(X,Y)$.

  • $\begingroup$ @Dilip Sarwate Plz can you mention how can I tell by inspection that where it went wrong after E[ZW]=E[Z]E[W] step in above answer. $\endgroup$ – Buzz bee Jul 5 '19 at 11:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.