# Discrete entropies

I've been given a problem where I need to find the entropy of two random variables. I can find part of the answer, but not all of it.

I am given the following:

• $X$ is a uniformly distributed random variable between 1-8 (so with probability $\frac{1}{8}$)

• $Y$ is distributed as such: $\textrm{Pr}(Y = k) = 2^{-k}$, where $k = 1,2,3.....$

I need to find $H(X)$, $H(Y)$, and $H(X +Y, X-Y)$.

I'm able to find the first two:

\begin{align} H(X) &= -\sum_{k=1}^8\left(\frac{1}{8}\cdot \log_2\left(\frac{1}{8}\right)\right)=3\\ H(Y) &= -\sum_{k=1}^\infty \left(2^{-k}\cdot \log_2\left(2^{-k}\right)\right) = \sum k2^{-k} = \frac{\frac{1}{2}}{\left(1-\frac{1}{2}\right)^2}=2 \end{align}

The part I'm not really sure how to do is the $H(X+Y, X-Y)$. I'm just not sure how you could combine these two. Could anyone point me in the right direction?

Using the chain rule: $$H(A,B)=H(A)+H(B|A)$$ we have $$H(X+Y,X-Y)=H(X+Y)+H(X-Y|X+Y)$$
Now let's assume $X+Y$. Since $X\in[1,8]$ and $Y\in(0,0.5]$, for any given $X+Y$, one can find out the actual $X$ and $Y$ and therefore, the value of $X-Y$. For this reason, we have $$H(X-Y|X+Y)=0$$
$$H(X+Y,X-Y)=H(X+Y)$$
Again, based on the above explanation there is a one-one relationship between $X+Y$and $(X,Y)$ and we conclude $$H(X+Y,X-Y)=H(X,Y)=H(X)+H(Y|X)$$
You didn't mention if $X$ and $Y$ are independet or not. If so, then $H(Y|X)=H(Y)$ and $$H(X+Y,X-Y)=H(X)+H(Y)=5$$