TL/DR: I'd like to take the median of lots of images to make a single "better" image. The problem is, the camera is slowly moving, and there is significant lens distortion. On the plus side the motion is constant, I've got lots of images, timestamps on each image, and I've been able to track points across the images to help align them.
I have a large 2000 frame series of night sky images taken from a GoPro Hero3. Which is in itself silly because GoPros have bad low light performance. I've done some light math on the images, and identified star position "arcs" over the 2 hour period.
Is there a way, using the star movement, to solve for the distortion of the lens AND the star rotation? Perhaps using OpenCV, or barrel distortion correction formulas? Star rotation alone is easy - 1 rotation every 24 hours. But generating the formula of (x1, y1) = f(x0, y0, num_minutes) is hard given I don't have the north star identified, the camera has a fish-eye effect, etc.
I feel like there should be a generic way to solve the pixel offset equation, regardless of the interaction between the lens distortion and star rotation. Any ideas? I believe I have all the info needed, about 30k data points across 250 stars of: (star_id, time, x, y) with good coverage, see the fast graph image below. (I'd chop off the left side where some trees stayed still.)