# Mutual Information, is this calculation correct or should I contact the author about fixing it?

Background information

According to Cover's text book on information theory Mutual Information is calculated as:

$$(1) I(W;C) = \Sigma_k\Sigma_i P(C_k,W_i)log(\frac{P(C_k,W_i)}{P(C_k)P(W_i)})$$

Now if we were interested in a single Word W_i we would just calculate this with one sum:

$$(2) I(W_i;C) = \Sigma_k P(C_k,W_i)log(\frac{P(C_k,W_i)}{P(C_k)P(W_i)})$$

OR we can rewrite this as

$$(3) I(W_i;C) = \Sigma_k P(C_k,W_i)log(\frac{P(W_i|C_k)}{P(W_i)})$$

because $$(4) P(W_i|C_k) = \frac{P(W_i,C_k)}{P(C_k)}$$ (unless i am incorrect)

Question While reading a paper on feature selection using information theory measurements I came across A segment of the text: Should the equation be more like (3), and I should contact the Author, or have I mis-understood/interpreted mutual information?

• I don't see any difference between Prof. Cover's equation $(3)$ and the Author's equation $(2)$ from the paper. – Sudarsan Sep 12 '13 at 23:46