Timeline for Is concept of "bit" in computer programming similar to the concept of "bit" in information theory?
Current License: CC BY-SA 3.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 15, 2015 at 18:42 | vote | accept | benjamin | ||
| Oct 15, 2015 at 0:52 | comment | added | MBaz | @benjamin, I think you're on the right track here. Think about it this way, when the text (or any computer file or date) is perfectly compressed, every computer bit in it will carry one information bit. | |
| Oct 15, 2015 at 0:09 | comment | added | benjamin | One quantization level is 1 binary digit bit. So now we can use lesser quantization levels or less binary digit bits. So aren't we considering binary digit bits and quantization bits as basically the same thing ? Back then, we used 8 bits to transmit a single symbol. But now we know the symbols are worth only 4 bits of information on average. So we are sending them using 4 quantization levels or 4 binary digit bits, because they carry 4 bits of information. In fact I tried to learn a little about Hoffman coding on my own, but here is where I got really stuck. | |
| Oct 15, 2015 at 0:05 | comment | added | benjamin | Hi, I posted the comment in the answer section in detail actually as it was getting too long. Must have forgotten to delete it. btw, that basically means that text symbols contain 3 to 4 bits worth of information whereas we use 8 bits to transfer them, right? So they contain useless bits or redundancy. So, for efficient data transfer, we can encode them using less bits and thus compress them. That means we can create lesser quantization levels to encode them. Previously we created 8 quantization levels, but now 4 quantization levels would be enough. | |
| Oct 14, 2015 at 23:53 | history | edited | MBaz | CC BY-SA 3.0 |
Further example.
|
| Oct 14, 2015 at 23:44 | comment | added | MBaz | @benjamin, that is right, and that's the reason text can be compressed so much. It contains much less information than the number of (computer) bits used to represent it. | |
| Oct 14, 2015 at 22:42 | comment | added | benjamin | Thanks MBaz . But here is one thing I am finding trouble understanding this topic. See, in data transfer of English alphabets, if we use ASCII code, we basically represent each symbol with 8 bits. Suppose that's 00000000 for a, 00000001 for b etc. So we are essentially allocating 8 quantization levels for each symbol. But when the information theory comes into play, we take the probability of each symbol into account. 'E' has the highest frequency, where 'Z' has the lowest. So average information content comes down to 3 or 4 bits, right ? | |
| Oct 14, 2015 at 21:47 | history | answered | MBaz | CC BY-SA 3.0 |