# More Simultaneous Dynamic Range with Fixed Point or Floating Point?

What data path would have more simultaneous dynamic range: 32 bit fixed-point or 32 bit floating-point? Why?

To clarify "simultaneous dynamic range": This is the ability to discern strong and weak signals at the same time, while dynamic range is the total range between between the weakest signal (sensitivity to still meet a performance metric) and strongest signal (saturation or some other decrease in a performance metric such as 1 dB compression, clipping, etc). One example where we would be concerned with simultaneous dynamic range is in an FFT where we have both weak and strong tones simultaneously at two different frequencies and we want to be able to discern both in the same FFT result).

Our vision and hearing are good examples of the difference between simultaneous dynamic range and dynamic range: see https://en.wikipedia.org/wiki/Dynamic_range

• Dan, what do you mean by "instantaneous" dynamic range? – MBaz Mar 31 '17 at 3:03
• I'm sure everybody would say floating point and therefore is looking forward to seeing your revelation of the surprising answer ;-)) – Fat32 Mar 31 '17 at 9:25
• @Fat32 I am going to wait a week; usually MattL waits 3 days to give others a chance, and then has the answer--so I am confident we will see a good answer before I reveal it. – Dan Boschen Mar 31 '17 at 15:20
• Dan, thanks for clarifying. Just to make sure I understand: the data path includes the ADC and all digital processing afterwards, right? Can wechoose any fixed point format as long as it's 32 bits? Can we assume there is no noise? – MBaz Mar 31 '17 at 15:47
• @MBaz I am not sure I completely understand your question about noise, but will say that we are not limited by noise in the signal itself but we are limited by the noise of the numbering format we choose. That noise limits are bottom end and saturation/overflow would limit our upper end. Does this clear everything up? – Dan Boschen Mar 31 '17 at 21:42

For the same total bit width, a fixed-point representation has a higher instantaneous dynamic range than a floating-point representation.

For a value to be simultaneously including information about a strong and a weak signal (where the information is being carried by the sum of the strong and weak signal components), the weak signal has to have a value at least as large as the stronger signal's numerical error floor, given by the format's epsilon. In a floating-point representation, every bit allocated to the exponent similarly reduces the number of bits allocated to the significand, and thus increases the format's epsilon. This correspondingly decreases the ratio between the strong and weak signal components that can be included in a single value.

For example in a standard 32-bit IEEE-754 single-precision floating point representation, 23 bits are allocated to the significand. As a result the smallest number that can be added to a given number $x$ is of the order of $x/2^{23}$ or approximately $8.6\times 10^6$ times smaller. Note that this is consistent with the C programming language's FLT_EPSILON which is approximately $1.19\times 10^{-7}$. In comparison, using a 32-bit fixed-point representation the smallest number that can be added to a large number using near full scale (e.g. 0xffffff00) is of the order of a single unit (e.g. 0x00000001) which is approximately $2^{32} \approx 4.3\times10^9$ times smaller.

• same issue as with Dan. your IEEE-754 32-bit floats can represent every 25-bit signed integer exactly. you gotta count both the sign bit and the "hidden 1" bit in with the mantissa. a 25-bit fixed-point and a 32-bit IEEE float will sound as good as each other if the fixed-point signal amplitude is saturated to the rail. – robert bristow-johnson Apr 1 '17 at 7:17
• Great comment @robertbristow-johnson Thanks for pointing that out! – Dan Boschen Apr 1 '17 at 7:47
• It seems that @DanBoschen asks the following floating point phenomenon, by codifying it as a dynamic range issue. Consider FPU implementation of (a+b), of the format $a = 2^{exp} \times 1.a_1a_2...a_N$ where $N$ is the number of significand bits and $exp$ is the exponent. In the usual math, you will first shift the numbers appropriately to left or right so that their exponents be the same. Then you can add the significands to get the result. Since you can at most shift such a number by $N$ bits left or right before losing its significand, the dynamic range will be $2^N$ – Fat32 Apr 1 '17 at 21:57
• @Fat32 yes I think you described that well. Note I don't know where else this would be significant but is a dominant concern in my field of wireless communications where we have a concern with instantaneous dynamic range in the design of receivers; for instance the ability to continue to receive weak signals in the presence of strong interference (jammers etc.). – Dan Boschen Apr 2 '17 at 1:06
• @SleuthEye please see Robert's explanation in the various comments and then my confirmation in my response by computing the effective number of bits for both cases that the actual equivalent dynamic range for a 32 bit floating point is 25 bits. If you agree, can you please update your response to reflect this as 25 bits (instead of 23 bits as you currently have)? – Dan Boschen Apr 2 '17 at 13:32

i have my own definition of dynamic range, at least in the audio application:

Dynamic range in dB is the sum of the signal-to-noise ratio in dB plus the headroom in dB.

now to be fair, if we compare fixed to floating point, we should use the same word width.

let's say 32-bit words with an 8-bit exponent for the floats. then it depends on the headroom you require. if you need more than 40 dB of headroom, then the 32-bit floats (with 8 bits of exponent) will give you better S/N. but if you don't need 40 dB of headroom, then the 32-bit fixed-point values will give you better S/N.

looks like you're looking for headroom. 32-bit floats with 8-bit exponents have something like 1523 dB of headroom for the smallest normalized floats. that plus the 150 dB S/N + headroom for the 25-bit mantissa (counting the sign bit and the "hidden 1" bit) gives you 1673 dB dynamic range.

is that enough for your application?

• Your dynamic range definition is consistent with mine for "dynamic range" from communications applications. So even in audio we would have the need for "instantaneous dynamic range"; consider a situation where you want to process a loud note and weak note at the same time (at different frequencies). I would imagine our human ability to discern loud and weak sounds simultaneously would be similar; we can make out really loud sounds, and when it is quiet we can hear very weak sounds (that is dynamic range)- but we cannot here those weak sounds in the presences of the loud sounds- less IDR – Dan Boschen Apr 1 '17 at 6:19
• that's a perception or psycho-acoustic issue, @DanBoschen. to really explore it, i would think you need AB testing. (i like AB testing over ABX testing. all you ask the blind subject is "Are the two sounds the same or are they different?" and you put in as many pairs with sounds identical to detect a tendency for false positives.) but it would be a function of both frequencies and the dB difference. – robert bristow-johnson Apr 1 '17 at 7:06
• Yes was not convinced human hearing was the best example, thanks! Vision might be a better one with bright lights vs seeing in the dark. – Dan Boschen Apr 1 '17 at 7:20
• but the point is, your perceptual discrimination issue really is not an issue of word size (except, of course, you need a large enough word to faithfully realize the two-tone test). – robert bristow-johnson Apr 1 '17 at 7:25
• @DanBoschen I'm afraid Robert is right (don't know why I'm afraid?) IEEE 32bit single precision floating point format employs an 8 bit biased exponent with 23 bit significand. This format's largest number is about $2^{128}$ and the smallest (yet positive) value is about $2^{-127-23}$ and therefore the dynamic range is about max/min = $2^{256+23}$ which is about 1500+ dB's as Robert points in detail. But I can see that you want to refer to the smallest number that can be added to a given number x, which requires the exponents being same and therefore depends on the size of the significand. – Fat32 Apr 1 '17 at 21:29

Here are some additional graphics to add to the correct answer posted that will hopefully make this important point immediately clear. Floating point adds an automatic "gain control" in the exponent, shown below as an "elevator" that can bring its instantaneous dynamic range capabilities up and down to where it is needed. This takes up bits in each sample but does not add anything to the "instantaneous dynamic range": it can handle extremely large numbers and small numbers, but not at the same time. Fixed point DSP systems can also do this through gain scaling and bit shifting where needed with extended precision arithmetic; it just takes more work on the designer's part (or compiler) to ensure there are no overflow or underflow conditions. In the end this is the reason why fixed point systems will tend to be lower cost and lower power (you don't have to carry this "automatic gain control" with every sample), while floating point systems will get you to design completion faster. So for most cases involving high volume production with concern for lowest cost or power, floating point is great for initial prototyping and proof of concept but fixed point is the way to go for product release (unless the DSP/FPGA processing is an insignificant part of the whole solution in terms of cost and power, or required for solving certain ill-conditioned matrices). That has been my experience but will be very interesting to learn from the greater community if that is typically the case, or situations where it is not - so please comment on that cost/power vs fixed/floating trade from your own experience! This example with the FFT in Matlab comparing single and double precision floats (both floating point one as 32 bit and the other as 64 bit) demonstrate well the significance of "instantaneous dynamic range". While the total dynamic range for a 32 bit floating point is is 1520 dB (and to convert to dB I am using 20 Log of the largest number/ smallest number), the instantaneous dynamic range is only 150.5 dB; and we can see that in the range between maximum signals in any given bin and the noise floor caused by quantization. With 32 bit floating point we can easily see a tone that is 120 dB lower than a strong signal, but we certainly cannot see one that is 1400 dB lower! That is the significance of instantaneous dynamic range. [UPDATE As Robert Bistrow-Johnson has pointed out in the comments, there is actually 1 more bit of precision with floating point than I initially showed in these figures; while I was previously showing 23 bits + 1 sign bit = 24 bits, we actually get 25 bits total as the control of the exponent adds 1 more bit. I had to check myself by backing out the effective number of bits based on SNR using the well known relationship SNR =6.02dB/bit + 1.76dB for a full scale sine wave and indeed got what Robert had stated (lesson learned: never doubt the Robert!) Here are the results for 32 bit floating point along with a know 12 bit quantization example to validate my approach (in the floating point calcuations I used 64 bit floating point numbers in order to be able to compute quantization noise). The spikes are input test frequencies where the assumption of quantization noise being uncorrelated to the input signal breaks down, but overall we get a result of the effective number of bits (ENOB) for 32 bit floating point is 25.26 bits.  ]

• gotta count both the sign bit and the "hidden 1" bit with the 23 mantissa bits. you get 25 bits and 150 dB. i.e. IEEE-754 32-bit floats can represent every 25-bit signed integer exactly. (they can represent only the even-valued 26-bit signed integers exactly.) – robert bristow-johnson Apr 1 '17 at 7:09
• I was using sign+ 23 = 24 bits, what is the hidden 1 bit? There is a 1 that is always assumed (the mantissa range is between 1 and 2) but since we cannot set that to 0 it does not add anything-- is that what you are referring to? – Dan Boschen Apr 1 '17 at 7:18
• yes. and it counts. to compare: an IEEE-754 32-bit float can exactly represent every 25-bit signed integer. you have 25 $\times$ 6.02 dB of dynamic range. want me to dig up my ppt i presented at this AES 9 years ago? – robert bristow-johnson Apr 1 '17 at 7:22
• I will take your word for it, is there an easy explanation as to why? I have trouble seeing (without trying it) how the 1 would add binary information if we can't change it--- or am I missing something there and it can be toggled somehow? Very interesting! Thanks for pointing this out – Dan Boschen Apr 1 '17 at 7:26
• you can mathematically "change it" by changing the exponent. the easy explanation is, again, an IEEE 32-bit float, that has 1 sign bit, 8 exponent bits, a *"hidden 1" bit, and 23 mantissa bits can represent the same set of $2^{25}$ numbers that a 25-bit fixed-point format can represent. therefore they gotta sound the same at the same, appropriate scaling. – robert bristow-johnson Apr 1 '17 at 7:30