10
$\begingroup$

I have a digital signal processing system that operates on a fast x86 machine using double precision floating point numbers. It occurred to me that I am not really using the huge dynamic range of the floating point representation -- all quantities fit easily in the range ±32768.

My question: is it possible that switching to fixed point computations would provide a benefit in numerical accuracy (high priority) or computation time (low priority)?

Of course, the answer depends on how many bits are available for a fixed-point calculation. How many bits of precision do typical fixed-point systems utilize? Is it possible to efficiently do fixed-point calculations, with, say, 64 bits (16 bits integer part, 48 bits fractional part) on x86-64?

I had always thought that fixed-point calculations were used only in situations where CPU power is limited--does it make sense to use fixed-point calculations when CPU power is no concern?

Improve this question
$\endgroup$
1
  • $\begingroup$ Do you really need more than the ~15 significant figures that a double-precision floating-point value gives you? While broad generalizations are bad, I would say that if you looked at the aggregate of all fixed-point DSP systems, 16-bit integers are likely to be the most common format. $\endgroup$
    – Jason R
    Commented Oct 29, 2012 at 13:20

5 Answers 5

Reset to default
7
$\begingroup$

The numerical accuracy of integers will only be better than the numerical accuracy of floats if the integer resolution is better. Doubles have 52 fractional bits, so double-precision floats have a resolution worse than integers at around $2^{52}$, which is much larger than 32768 ($2^{15}$). So, no, the numerical accuracy will not be better if you go to integers.

The second issue is speed. The answer is: it depends on the hardware. If you are running your program on a digital signal processor that has multiple fixed point multiply/accumulate cores then yes, it will be far faster in fixed point. On an x86 chip, on the other hand, it will actually probably be slower in fixed point. I did exactly what you are talking about once and saw my execution times increase.

After doing some searches on the internet I found out that that is common. The reason is because it has a dedicated floating-point processor that isn't doing anything when you transition to fixed point, while the fixed-point hardware is shared with the regular fixed-point action, such as pointer arithmetic.

If you want to speed up the processing, the way to do it is to change from double-precision floats to single-precision floats. That should produce a significant increase in speed. That would, of course, reduce your numerical accuracy though.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I was meaning what this answer says when I wrote mine. This one is better. If I'm not mistaken, also, I read somewhere that on some computers (64 bits maybe?) the native hardware floating point type is double, so using single precision (four byte) floats could actualy be slower. This is something to take into consideration, anyway. $\endgroup$
    – heltonbiker
    Commented Oct 29, 2012 at 15:24
  • $\begingroup$ Single precision floats have 23 bit mantissae, doubles have 52 bits. $\endgroup$
    – Paul R
    Commented Oct 29, 2012 at 16:38
  • $\begingroup$ I'm suggesting 16 bits integer + 48 bits fraction as an alternative to double precision floating point. I mentioned 32768 to indicate that my values will easily fit into this range. Given the restriction to these values, I think Q16.48 would provide greater numerical accuracy than double precision floating point. $\endgroup$
    – nibot
    Commented Oct 29, 2012 at 20:04
  • 1
    $\begingroup$ @nibot Okay. The doubles would have better precision from -16 to +16, and the fractional integers would have better precision elsewhere up to -32769 and +32768. They could not, of course, represent anything beyond that. They would also be slower than doubles. For me the limited range and slow speed would be deal breakers, but YMMV. $\endgroup$
    – Jim Clay
    Commented Oct 29, 2012 at 21:18
6
$\begingroup$

The merits of fixed-point are mostly in terms of power (such as when you have a choice of processor hardware, or the processor is good at shutting down unused functional units). That is because fixed-point units are commonly smaller (less transistors, shorter wires, less capacitance to overcome per MAC) for a given technology and operation issue rate, than floating point.

However a vast amount of common contemporary processors (server, PC and even mobile), have more and faster FPUs (especially single-precision FP units) than integer multipliers, and most of the system power is not from using the FPU, so using fixed-point will have little or no advantages for typical DSP computation on these products, and can likely be a disadvantage in terms of pure performance. Using current technology, any advantage to fixed-point will mostly accrue mostly in tiny embedded products, such as button-sized devices.

However also consider memory and processor cache footprints. Clever use of smaller data types (short int and float) to fit a large computation completely in data cache may offset any pure FPU bandwidth advantages.

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ +1 for the mention of the importance of cache issues with respect to performance. On modern x86 processors, designing your algorithm with cache in mind can have a huge effect on performance. $\endgroup$
    – Jason R
    Commented Oct 30, 2012 at 1:09
5
$\begingroup$

Prefer single precision floats to doubles - this will halve your memory bandwidth, cache footprint and storage requirements, and make some mathematical operations faster. It also opens up the possibility of 4 way SIMD if further optimization is needed.

Fixed point is only really worthwhile when you don't have an FPU - most modern x86 CPUs have two FPUs so there is nothing to be gained from using fixed point, and performance may even be significantly worse with fixed point. (Note also that fixed point requires additional instructions compared to floating point for operations such as multiplication.)

Improve this answer
$\endgroup$
4
  • $\begingroup$ I'm interested in increasing numerical accuracy, not reducing it. $\endgroup$
    – nibot
    Commented Oct 29, 2012 at 19:57
  • $\begingroup$ How do you see fixed point improving numerical accuracy relative to a double, which has 52 bits of precision and a huge dynamic range already ? $\endgroup$
    – Paul R
    Commented Oct 30, 2012 at 22:18
  • $\begingroup$ Well, I could use a fixed-point format with more than 52 bits. $\endgroup$
    – nibot
    Commented Oct 31, 2012 at 9:19
  • $\begingroup$ Since you apparently need at least 16 bits for the integer part of your fixed point representation this will take you well over 64 bits, so you're probably then looking at a format for which your CPU does not even have any native integer instructions. In which case you might as well just use an existing big integer library or similar. The most important question to answer though is: how much precision do you really need ? $\endgroup$
    – Paul R
    Commented Oct 31, 2012 at 10:12
3
$\begingroup$

In addition to the very good answers provided here, a few things worth adding:

  • There are situations in which even if you have very basic requirements on the dynamic range of the data you process, you will still need a very good precision for some of the operations performed on it - for example you will want to apply an IIR filter which requires relatively small coefficients; and truncating them would cause instabilities. As soon as your system has feedback, there's a good chance quantization/truncation problems will bite you back when using fixed point - you have to be much more careful about things like filter topology and truncation/fraction-saving schemes.
  • Unlike many DSP/DSC architectures, the x86 doesn't have saturated integer operations (well, it's there in SSE, not on standard scalar code). This means that in case of overflow, bad things can happen - values changing signs and "wrapping". You have to be extra cautious with overflows and dynamic range, or sprinkle tests on operands ranges all over your code. This can seriously hurt performance. By comparison, floating point is more resilient to these problems, because the large dynamic range gives you more "headroom", and overflows will not lead to catastrophic failures. Most audio signal processing code running on desktop computers is using the -1.0 .. 1.0 range, single or double precision; so this gives more than hundreds of dB of headroom. I have written audio signal processing code with both approaches, and when using floating point there are only a few places when I have to explicitly clip/saturate the signal - usually just at the end of the signal processing chain or in places where feedback occur.
Improve this answer
$\endgroup$
1
$\begingroup$

Some points to consider:

  • Most modern processors have been optimizing floating-point numbercrunching for many years, and even GPUs are being used for that already, very successfully;
  • Fixed-point calculations hurt your data and can cause serious problems when arithmetic operations are not well-conditioned (that's why fixed-point numbers were replaced by floating point ones);
  • Even if you use signed shorts to CONTAIN your data (lots dataloggers use 16bit precision), the CALCULATIONS should be done in floating point then converted back to integers, otherwise there could be artifacts such as quantization and aliasing.

As a final word, I think our realworld data is precious and the computer's blind numbercrunching is a humble menial work. The computer must be put to do the heavy-lifting work for your data and for you, and not be treated as if it was the real star in the show.

Improve this answer
$\endgroup$
3
  • $\begingroup$ I didn't mean to imply that I would use 16-bit shorts to contain my quantities, but rather something like a 64-bit fixed point format with a 16-bit integer part and a 48-bit fractional part. The motivation is that, if I am not using most of the exponent bits in the floating point format anyway, would my numerical accuracy improve if I instead used those bits to provide additional significant digits? $\endgroup$
    – nibot
    Commented Oct 29, 2012 at 12:48
  • $\begingroup$ You should add your 16-bit integer + 48-bit fractional to the original question. It looks like $2^{15}$ is causing confusion. $\endgroup$
    – Christopher Felton
    Commented Oct 29, 2012 at 15:12
  • $\begingroup$ One more thing: It seems to me that StackOverflow (instead of DSP.SE, here) would be the ideal place to get deeper reasons about pros and cons of one format over the other. $\endgroup$
    – heltonbiker
    Commented Oct 29, 2012 at 15:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.