Start simple: that calls for any Run-Length coding (RLE).
Also, floating point makes very little sense for image data, so the first step that will probably save you 50% of space is conversion to unsigned fixed point (multiply every value with 2¹⁶, then divide through the largest value of the original image, round to nearest integer, keep the result as uint16).
5% mean absolute error means: you don't need floating point; rounding small values to zero is absolutely¹ acceptable to you. So, drop the float idea and convert to integer, as described above.
If for some (somewhat untypical for image data) reason you want to keep it floating point, convert it to a format where you store the mantissa and exponent bits separately and the latter at all only for non-zero mantissa values. Since pixel data typically only has a positive sign, you don't even need to store the sign bit.
After reducing the number of exponents you've stored, maybe any further compression isn't necessary to achieve your goal (don't overengineer!).
Then, look at your data: It's also quite likely that your exponents don't need the full 2⁻¹²⁶ to 2¹²⁷ range, so dropping bits on your exponents makes a lot of sense, too. Make sure you're not totally killing it with the number of mantissa bits, too. Wild guess: half as many as in 32 bit IEEE754 will do.
If after doing these thing it still is necessary to compress further, RLE or Lempel-Ziv would be appropriate solutions.
(LZ4 is kind of exotic; the more established LZMA algorithm as e.g. in XZ should work sufficiently well, and it really doesn't sound like you're CPU bound here...)
In all seriousness, though, sounds like a job PNG, high-quality JPEG, or if you have access to that, JPEG-XL, could do out of the box, so I'd simply look into converting my data to a format that makes sense to readily available lossless image codecs instead of trying to roll my own.
¹ hurr durr, a pun!