I can't put tiffs directly into a TFRecords format...
You can. TFRecords does not know what an "image" is, it simply stores a string of bytes. What this means to higher level applications is up to them.
When I imported a sample GeoTIFF file into python using rasterio I found that the range for pixel values was between [282, 1560], which seemed rather high.
Depending on the kind of data they contain, GeoTIFF images can have higher colour depth per pixel. For example, the SRTM data provide a digital elevation model of the Earth. Each pixel value is associated with the height of a "square" that is 30 by 30 meters (along the equator). The variation of that height over the whole planet is much larger than 256. SRTM's GeoTIFFs have a 16bit colour depth.
Now PNG or JPEG files seem to have pixel ranges of between [0, 255] or [0,1], from what I gather.
PNG and JPEG can save 16bit per pixel colour depth images and you can still work with them with libraries such as osgeo or GIMP.
Depending on the content of the GeoTIFFs though, the data in PNG format (lossless encoding) might not compress as much as you might expect and with JPEG (lossy encoding) you are going to have to tune the compression to avoid losing any necessary features. In the end, it might be better to just leave the data in GeoTIFF or read them in a numpy array and then serialise that straight into TFRecords.
This seems like a simple rescaling of the values to my naive eyes.
So my question is whether anyone could provide the basic algorithm to convert TIFF file pixels into PNG or JPEG pixels. I don't need to worry about the spatial coordinates in this case, so I can treat this as a straight TIFF --> PNG or JPEG conversion.
Indeed, it has to be a simple linear scaling but this is not a good idea for a number of reasons.
If the absolute values are of importance (for example) then the 8bit dynamic range is simply not adequate.
If your subsequent processing is about learning gradients (for another example), then you are going to have to normalise to the global extrema (not the maximum / minimum of each individual file) so that the relative slopes do not change. The danger in doing so is again dynamic range. Although it is possible to squash any numerical range in the interval $0 \dots 1$, when this interval is quantised in 8 bits it is only divided into 256 "steps". If two pixel values differ by less than $\frac{1}{256}$, then they will be assigned to the same "bin". Also, if the dataset contains "spikes", that is the majority of the values are between 20-100 (for example) but there is one value at 10000, then these spikes will "kill" the contrast when you normalise at 8bits. What you will get is a mostly black image with 1 white pixel. Again, it might be possible to remove those spikes but if you have to work with them (e.g. for a multispectral application), then normalising to an 8bit colour depth is not a good idea.
The other thing you would be losing if you went with this option is the transform data. That is, the transform that converts coordinates from image (pixel positions and values) to world coordinates (longitude, latitude, height).
Suppose that your GeoTIFF image ($I$) describes a digital elevation model that has both negative and positive values. The total range of the GeoTIFF would be $max(I)-min(I)$. If $min(I)$ is not zero then your $0$ point in the converted data will be "floating". It might be 20 on one image, 42 on another 100 on yet another and so on.
The transformation between two ranges is straigthforward: $O_{x,y} = O_{min} + \frac{I_{x,y}-min(I)}{max(I)+min(I)} \times range(O)$, where $O$ is the output image, $O_{min}$ is the new desired minimum and $range(O)$ is the desired range.
While the transformation is straightforward, the impact it has on the data and subsequent models can be severe, therefore my recommendation would be to preserve the original numbers as much as possible and only intervene if you are absolutely sure that the transformation will not affect your objectives.
Hope this helps.
