More Simultaneous Dynamic Range with Fixed Point or Floating Point?

What data path would have more simultaneous dynamic range: 32 bit fixed-point or 32 bit floating-point? Why?

To clarify "simultaneous dynamic range": This is the ability to discern strong and weak signals at the same time, while dynamic range is the total range between between the weakest signal (sensitivity to still meet a performance metric) and strongest signal (saturation or some other decrease in a performance metric such as 1 dB compression, clipping, etc). One example where we would be concerned with simultaneous dynamic range is in an FFT where we have both weak and strong tones simultaneously at two different frequencies and we want to be able to discern both in the same FFT result.

Our vision and hearing are good examples of the difference between simultaneous dynamic range and dynamic range: see https://en.wikipedia.org/wiki/Dynamic_range

• Dan, what do you mean by "instantaneous" dynamic range?
– MBaz
Mar 31 '17 at 3:03
• I'm sure everybody would say floating point and therefore is looking forward to seeing your revelation of the surprising answer ;-)) Mar 31 '17 at 9:25
• @Fat32 I am going to wait a week; usually MattL waits 3 days to give others a chance, and then has the answer--so I am confident we will see a good answer before I reveal it. Mar 31 '17 at 15:20
• Dan, thanks for clarifying. Just to make sure I understand: the data path includes the ADC and all digital processing afterwards, right? Can wechoose any fixed point format as long as it's 32 bits? Can we assume there is no noise?
– MBaz
Mar 31 '17 at 15:47
• @MBaz I am not sure I completely understand your question about noise, but will say that we are not limited by noise in the signal itself but we are limited by the noise of the numbering format we choose. That noise limits are bottom end and saturation/overflow would limit our upper end. Does this clear everything up? Mar 31 '17 at 21:42

For the same total bit width, a fixed-point representation has a higher instantaneous dynamic range than a floating-point representation.

For a value to be simultaneously including information about a strong and a weak signal (where the information is being carried by the sum of the strong and weak signal components), the weak signal has to have a value at least as large as the stronger signal's numerical error floor, given by the format's epsilon. In a floating-point representation, every bit allocated to the exponent similarly reduces the number of bits allocated to the significand, and thus increases the format's epsilon. This correspondingly decreases the ratio between the strong and weak signal components that can be included in a single value.

For example in a standard 32-bit IEEE-754 single-precision floating point representation, 23 bits are allocated to the significand. As a result the smallest number that can be added to a given number $x$ is of the order of $x/2^{23}$ or approximately $8.6\times 10^6$ times smaller. Note that this is consistent with the C programming language's FLT_EPSILON which is approximately $1.19\times 10^{-7}$. In comparison, using a 32-bit fixed-point representation the smallest number that can be added to a large number using near full scale (e.g. 0xffffff00) is of the order of a single unit (e.g. 0x00000001) which is approximately $2^{32} \approx 4.3\times10^9$ times smaller.

• same issue as with Dan. your IEEE-754 32-bit floats can represent every 25-bit signed integer exactly. you gotta count both the sign bit and the "hidden 1" bit in with the mantissa. a 25-bit fixed-point and a 32-bit IEEE float will sound as good as each other if the fixed-point signal amplitude is saturated to the rail. Apr 1 '17 at 7:17
• Great comment @robertbristow-johnson Thanks for pointing that out! Apr 1 '17 at 7:47
• It seems that @DanBoschen asks the following floating point phenomenon, by codifying it as a dynamic range issue. Consider FPU implementation of (a+b), of the format $a = 2^{exp} \times 1.a_1a_2...a_N$ where $N$ is the number of significand bits and $exp$ is the exponent. In the usual math, you will first shift the numbers appropriately to left or right so that their exponents be the same. Then you can add the significands to get the result. Since you can at most shift such a number by $N$ bits left or right before losing its significand, the dynamic range will be $2^N$ Apr 1 '17 at 21:57
• @Fat32 yes I think you described that well. Note I don't know where else this would be significant but is a dominant concern in my field of wireless communications where we have a concern with instantaneous dynamic range in the design of receivers; for instance the ability to continue to receive weak signals in the presence of strong interference (jammers etc.). Apr 2 '17 at 1:06
• @SleuthEye please see Robert's explanation in the various comments and then my confirmation in my response by computing the effective number of bits for both cases that the actual equivalent dynamic range for a 32 bit floating point is 25 bits. If you agree, can you please update your response to reflect this as 25 bits (instead of 23 bits as you currently have)? Apr 2 '17 at 13:32

i have my own definition of dynamic range, at least in the audio application:

Dynamic range in dB is the sum of the signal-to-noise ratio in dB plus the headroom in dB.

now to be fair, if we compare fixed to floating point, we should use the same word width.

let's say 32-bit words with an 8-bit exponent for the floats. then it depends on the headroom you require. if you need more than 40 dB of headroom, then the 32-bit floats (with 8 bits of exponent) will give you better S/N. but if you don't need 40 dB of headroom, then the 32-bit fixed-point values will give you better S/N.

looks like you're looking for headroom. 32-bit floats with 8-bit exponents have something like 1523 dB of headroom for the smallest normalized floats. that plus the 150 dB S/N + headroom for the 25-bit mantissa (counting the sign bit and the "hidden 1" bit) gives you 1673 dB dynamic range.

is that enough for your application?

• Your dynamic range definition is consistent with mine for "dynamic range" from communications applications. So even in audio we would have the need for "instantaneous dynamic range"; consider a situation where you want to process a loud note and weak note at the same time (at different frequencies). I would imagine our human ability to discern loud and weak sounds simultaneously would be similar; we can make out really loud sounds, and when it is quiet we can hear very weak sounds (that is dynamic range)- but we cannot here those weak sounds in the presences of the loud sounds- less IDR Apr 1 '17 at 6:19
• that's a perception or psycho-acoustic issue, @DanBoschen. to really explore it, i would think you need AB testing. (i like AB testing over ABX testing. all you ask the blind subject is "Are the two sounds the same or are they different?" and you put in as many pairs with sounds identical to detect a tendency for false positives.) but it would be a function of both frequencies and the dB difference. Apr 1 '17 at 7:06
• Yes was not convinced human hearing was the best example, thanks! Vision might be a better one with bright lights vs seeing in the dark. Apr 1 '17 at 7:20
• but the point is, your perceptual discrimination issue really is not an issue of word size (except, of course, you need a large enough word to faithfully realize the two-tone test). Apr 1 '17 at 7:25
• @DanBoschen I'm afraid Robert is right (don't know why I'm afraid?) IEEE 32bit single precision floating point format employs an 8 bit biased exponent with 23 bit significand. This format's largest number is about $2^{128}$ and the smallest (yet positive) value is about $2^{-127-23}$ and therefore the dynamic range is about max/min = $2^{256+23}$ which is about 1500+ dB's as Robert points in detail. But I can see that you want to refer to the smallest number that can be added to a given number x, which requires the exponents being same and therefore depends on the size of the significand. Apr 1 '17 at 21:29

• yes. and it counts. to compare: an IEEE-754 32-bit float can exactly represent every 25-bit signed integer. you have 25 $\times$ 6.02 dB of dynamic range. want me to dig up my ppt i presented at this AES 9 years ago? Apr 1 '17 at 7:22
• you can mathematically "change it" by changing the exponent. the easy explanation is, again, an IEEE 32-bit float, that has 1 sign bit, 8 exponent bits, a *"hidden 1" bit, and 23 mantissa bits can represent the same set of $2^{25}$ numbers that a 25-bit fixed-point format can represent. therefore they gotta sound the same at the same, appropriate scaling. Apr 1 '17 at 7:30