# Relationship of ADC resolution and 'lossless' float data type storage

For a given digitised voltage trace, I want to avoid any quantization error introduced via an insufficient precision of the float data type – and at the same time reduce the memory cost. However, I am not sure that I have understood all the concepts correctly. The digitised signal is given as a 32-bit integer which needs to be multiplied by a constant ($$12958 \times 10^{-12}$$) to get the voltage value...

The signal originates from an ADC with an input range of ±10V and 24-bit resolution, the quantisation step size should be $$\frac{20}{2^{24}} \approx 1.19e{-06}$$. Therefore it should not be possible to store the resulting (int*const) float value as float32, since the float32 resolution on my machine (using numpy -> np.finfo(np.float32).resolution) is $$1e{-06}$$.

Does this make sense or am I confusing some concepts?

EDIT: I want to process the signal (e.g. filtering) therefore further use of the integers is not possible. And as I work in Python I will work with floating point integers.
I want to point out, that I do not want to map the integers to floats. As far as I understand the integers are stored for memory efficiency and after multiplication with a constant get the 'real' float signal of the sensor. My concern is, that the floating numbers does lose some information because the step size of the 24-bit integers multiplied with the constant is less than the minimal step size of the float (in that region, accounting for relative error of float). The scaling constant for the integer is $$12958 \times 10^{-12}$$ (for Volt)- However, since I want to work with µV I usually normalise with $$12958 \times 10^{-6}$$. The usual range of the signal is about ±200µV.

This is the machine epsilon of my machine for the relevant signal interval.

Does this mean, that it should be 'save' to use the float32 since the step sizes in the relevant signal range are smaller than my scaling constant $$12958 \times 10^{-6}$$?

I would highly appreciate some literature suggestion to understand the concepts better.

• A 32-bit IEEE-754 float can represent every signed 25-bit integer exactly. If the output of the ADC has 24-bit resolution, that is every value is one of $2^{24}$ values, that can be represented exactly, with no loss of data, by a 32-bit float. Commented Jun 5, 2023 at 18:11
• Thanks! I am not sure if you mean this, but I see the point that it is possible to map any 24-bit integer to a 32-bit float without loss, since the latter has a higher cardinality. However, I now realise that my question is whether the 32-bit float can also hold the integer values transformed by the scaling constant without loss. I am afraid that in some regions the step size of the float is not sufficient to account for this. See the edit for more information. Does your argument still hold? And if so, can you explain/suggest some mathematical/computational intuition/literature? Commented Jun 6, 2023 at 5:56
• C'mon, if you can save your 24-bit sample value as a 24-bit integer that can be represented exactly in a 32-bit IEEE float, then you can store it without loss. Now, if you choose to store the same fixed-point number, but scaled by a power of 2 so that the range of the number fits a specified range (usually it's from -1 to +1), then the mantissa is exactly the same bits. Nothing is lost. Commented Jun 6, 2023 at 7:28
• hmm ok, although this might disqualify myself. Why is it obvious that the 24bit can be lossless expressed by a float32? I read this at wikipedia as well but it is not that obvious for me. I see the point for everything in the exponent (7bit) and adding the mantisse another bit. How is it for bit 8 to 24? Commented Jun 6, 2023 at 14:57
• Helmut, it's the number of bits in the mantissa that matter the most. There needs to be enough range in the exponent value (and 8 bits in the exponent easily give us enough range, 5 bits would be enough). But this comes simply from the explicit definition of the value of the floating-point number with 23 mantissa bits, one *"hidden 1" bit, and one sign bit. And from the definition of the 25-bit signed integer using the sign-magnitude format. Write the definitions down side-by-side. Commented Jun 6, 2023 at 16:33