Timeline for What is Shannon's source entropy
Current License: CC BY-SA 3.0
6 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jul 31, 2012 at 17:52 | comment | added | Jim Clay | Yes, that is correct. K implies the number of discrete values that the random variables can take, not the sample number. | |
| Jul 31, 2012 at 17:27 | comment | added | user1214586 | Thank you once again. So to sum up, formulae for source entropy and shannon's entropy is the same however, osurce entropy denotes the entropy of the sender's signal which is communicating through the channel. Whereas, Shannon's entropy in particulary does not imply either source or receive, it can be the entropy of either. Am I correct? How about part (C) in whichI ask about a confusion about the varaible k | |
| Jul 31, 2012 at 17:20 | vote | accept | user1214586 | ||
| Jul 31, 2012 at 13:48 | comment | added | Jim Clay | If $X$ is the source signal and Y is the signal that a receiver gets after $X$ travels through a channel, then $H_k$ is the source entropy, simply because $X$ is the source. When you ask about joint probabilities, are you talking about the joint probabilities of $X$ and $Y$. If so, that is not correct. When calculating entropy you just calculate it for $X$ or $Y$. You consider the joint probabilities when calculating the mutual information. | |
| Jul 31, 2012 at 2:53 | comment | added | user1214586 | Thank you for the eye opener. I have few more queries based on ur reply.(A)So, if Y is a discretized version, discretized into n bins, of X then source entropy would mean entropy of X or Y? (B)Also, when calculating & implementing entropy, do we consider the joint probability or the probability of each occurence of variable?(C)In the formula k would imply the number of discretization levels/bins or the total number of sample(ie data length)? | |
| Jul 31, 2012 at 2:12 | history | answered | Jim Clay | CC BY-SA 3.0 |