In example take into consideration 11-bit
<11,5> signed fixed-point arithmetic.
000000.11111 - 0.96875
000001.00000 - 1
so the error magnitude will be:
(1 - 0.96875)/2 = 0.015625 ?
Signal Processing Stack Exchange is a question and answer site for practitioners of the art and science of signal, image and video processing. It only takes a minute to sign up.Sign up to join this community