Skip to main content
just a teeny bit of copy edit
Source Link

I need to prove that mutual information given by$$I(X;Y)=\int_{x,y}f(x,y) log_2 \frac{\left (f(x,y)\right)}{f(x) f(y)}$$

$$I(X;Y)=\int_{x,y}f(x,y) \log_2 \left( \frac{\left (f(x,y)\right)}{f(x) f(y)}\right) \, dx \, dy$$

is equivalent to $I(X;Y)=H(Y) - H(Y|X)$ I maam proceeding like this $$I(X;Y)=\int_{x,y} f(x) \frac{ f\left(x,y \right)}{f(x)} log_2 (f(Y|X)) dx dy - \int_{x,y}f(x,y) log_2f(y) dx dy$$ where

$$I(X;Y)=\int_{x,y} f(x) \frac{ f\left(x,y \right)}{f(x)} \log_2 (f(Y|X)) \, dx \, dy - \int_{x,y}f(x,y) \log_2(f(y)) \, dx \, dy$$

where first term gives me $H(X|Y)$ and second term gives me $$\int_{x,y}f(y) f(x|y) log_2f(y) dx dy $$

$$\int_{x,y}f(y) f(x|y) \log_2(f(y)) \, dx \, dy $$

and I am taking   

$$\int_{x} f\left (x|y \right) dx =1$$ 

I mean to say if this equation is right what does this physically symbolisesymbolize that given y$y$ there must be some x$x$ that means given output there must be some input with probability 1$1$? Am I doing the derivation right ?

I need to prove that mutual information given by$$I(X;Y)=\int_{x,y}f(x,y) log_2 \frac{\left (f(x,y)\right)}{f(x) f(y)}$$ is equivalent to $I(X;Y)=H(Y) - H(Y|X)$ I ma proceeding like this $$I(X;Y)=\int_{x,y} f(x) \frac{ f\left(x,y \right)}{f(x)} log_2 (f(Y|X)) dx dy - \int_{x,y}f(x,y) log_2f(y) dx dy$$ where first term gives me $H(X|Y)$ and second term gives me $$\int_{x,y}f(y) f(x|y) log_2f(y) dx dy $$ and I am taking  $$\int_{x} f\left (x|y \right) dx =1$$ I mean to say if this equation is right what does this physically symbolise that given y there must be some x that means given output there must be some input with probability 1? Am I doing the derivation right ?

I need to prove that mutual information given by

$$I(X;Y)=\int_{x,y}f(x,y) \log_2 \left( \frac{\left (f(x,y)\right)}{f(x) f(y)}\right) \, dx \, dy$$

is equivalent to $I(X;Y)=H(Y) - H(Y|X)$ I am proceeding like this

$$I(X;Y)=\int_{x,y} f(x) \frac{ f\left(x,y \right)}{f(x)} \log_2 (f(Y|X)) \, dx \, dy - \int_{x,y}f(x,y) \log_2(f(y)) \, dx \, dy$$

where first term gives me $H(X|Y)$ and second term gives me

$$\int_{x,y}f(y) f(x|y) \log_2(f(y)) \, dx \, dy $$

and I am taking 

$$\int_{x} f\left (x|y \right) dx =1$$ 

I mean to say if this equation is right what does this physically symbolize that given $y$ there must be some $x$ that means given output there must be some input with probability $1$? Am I doing the derivation right ?

Source Link

How to prove relationship between mutual information and differential entropy?

I need to prove that mutual information given by$$I(X;Y)=\int_{x,y}f(x,y) log_2 \frac{\left (f(x,y)\right)}{f(x) f(y)}$$ is equivalent to $I(X;Y)=H(Y) - H(Y|X)$ I ma proceeding like this $$I(X;Y)=\int_{x,y} f(x) \frac{ f\left(x,y \right)}{f(x)} log_2 (f(Y|X)) dx dy - \int_{x,y}f(x,y) log_2f(y) dx dy$$ where first term gives me $H(X|Y)$ and second term gives me $$\int_{x,y}f(y) f(x|y) log_2f(y) dx dy $$ and I am taking $$\int_{x} f\left (x|y \right) dx =1$$ I mean to say if this equation is right what does this physically symbolise that given y there must be some x that means given output there must be some input with probability 1? Am I doing the derivation right ?