More Simultaneous Dynamic Range with Fixed Point or Floating Point?

Question

What data path would have more simultaneous dynamic range: 32 bit fixed-point or 32 bit floating-point? Why?

To clarify "simultaneous dynamic range": This is the ability to discern strong and weak signals at the same time, while dynamic range is the total range between between the weakest signal (sensitivity to still meet a performance metric) and strongest signal (saturation or some other decrease in a performance metric such as 1 dB compression, clipping, etc). One example where we would be concerned with simultaneous dynamic range is in an FFT where we have both weak and strong tones simultaneously at two different frequencies and we want to be able to discern both in the same FFT result.

Our vision and hearing are good examples of the difference between simultaneous dynamic range and dynamic range: see https://en.wikipedia.org/wiki/Dynamic_range

Please preface your answer with spoiler notation by typing the following two characters first ">!"

Answer 1

For the same total bit width, a fixed-point representation has a higher instantaneous dynamic range than a floating-point representation.

For a value to be simultaneously including information about a strong and a weak signal (where the information is being carried by the sum of the strong and weak signal components), the weak signal has to have a value at least as large as the stronger signal's numerical error floor, given by the format's epsilon. In a floating-point representation, every bit allocated to the exponent similarly reduces the number of bits allocated to the significand, and thus increases the format's epsilon. This correspondingly decreases the ratio between the strong and weak signal components that can be included in a single value.

For example in a standard 32-bit IEEE-754 single-precision floating point representation, 23 bits are allocated to the significand. As a result the smallest number that can be added to a given number $x$ is of the order of $x/2^{23}$ or approximately $8.6\times 10^6$ times smaller. Note that this is consistent with the C programming language's FLT_EPSILON which is approximately $1.19\times 10^{-7}$. In comparison, using a 32-bit fixed-point representation the smallest number that can be added to a large number using near full scale (e.g. 0xffffff00) is of the order of a single unit (e.g. 0x00000001) which is approximately $2^{32} \approx 4.3\times10^9$ times smaller.

Answer 2

i have my own definition of dynamic range, at least in the audio application:

Dynamic range in dB is the sum of the signal-to-noise ratio in dB plus the headroom in dB.

now to be fair, if we compare fixed to floating point, we should use the same word width.

let's say 32-bit words with an 8-bit exponent for the floats. then it depends on the headroom you require. if you need more than 40 dB of headroom, then the 32-bit floats (with 8 bits of exponent) will give you better S/N. but if you don't need 40 dB of headroom, then the 32-bit fixed-point values will give you better S/N.

looks like you're looking for headroom. 32-bit floats with 8-bit exponents have something like 1523 dB of headroom for the smallest normalized floats. that plus the 150 dB S/N + headroom for the 25-bit mantissa (counting the sign bit and the "hidden 1" bit) gives you 1673 dB dynamic range.

is that enough for your application?

Answer 3

Here are some additional graphics to add to the correct answer posted that will hopefully make this important point immediately clear. Floating point adds an automatic "gain control" in the exponent, shown below as an "elevator" that can bring its instantaneous dynamic range capabilities up and down to where it is needed. This takes up bits in each sample but does not add anything to the "instantaneous dynamic range": it can handle extremely large numbers and small numbers, but not at the same time. Fixed point DSP systems can also do this through gain scaling and bit shifting where needed with extended precision arithmetic; it just takes more work on the designer's part (or compiler) to ensure there are no overflow or underflow conditions. In the end this is the reason why fixed point systems will tend to be lower cost and lower power (you don't have to carry this "automatic gain control" with every sample), while floating point systems will get you to design completion faster. So for most cases involving high volume production with concern for lowest cost or power, floating point is great for initial prototyping and proof of concept but fixed point is the way to go for product release (unless the DSP/FPGA processing is an insignificant part of the whole solution in terms of cost and power, or required for solving certain ill-conditioned matrices). That has been my experience but will be very interesting to learn from the greater community if that is typically the case, or situations where it is not - so please comment on that cost/power vs fixed/floating trade from your own experience! This example with the FFT in Matlab comparing single and double precision floats (both floating point one as 32 bit and the other as 64 bit) demonstrate well the significance of "instantaneous dynamic range". While the total dynamic range for a 32 bit floating point is is 1520 dB (and to convert to dB I am using 20 Log of the largest number/ smallest number), the instantaneous dynamic range is only 150.5 dB; and we can see that in the range between maximum signals in any given bin and the noise floor caused by quantization. With 32 bit floating point we can easily see a tone that is 120 dB lower than a strong signal, but we certainly cannot see one that is 1400 dB lower! That is the significance of instantaneous dynamic range. [UPDATE As Robert Bistrow-Johnson has pointed out in the comments, there is actually 1 more bit of precision with floating point than I initially showed in these figures; while I was previously showing 23 bits + 1 sign bit = 24 bits, we actually get 25 bits total as the control of the exponent adds 1 more bit. I had to check myself by backing out the effective number of bits based on SNR using the well known relationship SNR =6.02dB/bit + 1.76dB for a full scale sine wave and indeed got what Robert had stated (lesson learned: never doubt the Robert!) Here are the results for 32 bit floating point along with a know 12 bit quantization example to validate my approach (in the floating point calcuations I used 64 bit floating point numbers in order to be able to compute quantization noise). The spikes are input test frequencies where the assumption of quantization noise being uncorrelated to the input signal breaks down, but overall we get a result of the effective number of bits (ENOB) for 32 bit floating point is 25.26 bits. ]