This question already has an answer here:
By entropy I understand the uncertainty or randomness. But if the uncertainty is high then what is its implication and advantage in transmitting information or as an information source or is high entropy preferred for processing information?
Then there is entropy rate, which I cannot understand due to heavy mathematics. All that I can follow is that entropy rate = entropy/ number of unique symbols.
Please forgive if these questions seem trivial, but it is quite hard to filter the answers from textbooks which are mainly focused on theoretical definition and not on how one can use entropy in real applications. Below are my confusions.
If the entropy of an information generating source is high, that is considered to be good. But what is the role of entropy rate?
Does entropy have a lower and upper bound or a range? Based on equi-probable symbols, we say then 0.5 is the max entropy as equi-probable symbols convey maximum uncertainty about the source. Is the range of entropy is 0--1?
What information is gained or loss when entropy is discussed from information theory viewpoint?