Video stitching for static scene (infinite loop video)

Question

I have a video that is a panoramic video of a static scene. Just to be clear, I am not talking about a spherical 360° degree video but a video that starts in some direction and then does a full 360 horizontal tour either clock or counterclock wise (direction is known). The video then cuts approximately off when a complete tour is completed (~10° error). The video is most likely between 10 to 20 seconds long. I would like to modify the beginning and end of the video of in such a way that if I play the video in a loop the user has the impression he sees a never ending video (end and beginning should have a seamless transition). The main problem is that beginning and end might be slightly vertically shifted and tilted with respect to one another. Otherwise I would only need to find a frame that matches.

My question is therefore I do best solve this problem. Should I look into image stitching or are there more specific algorithms for this type of video stitching. When I search for video stitching I mostly come across 360° video stitching. Pointers to the appropriate literature are also very welcome.

EDIT: You might be wondering why I don't just create a panorama picture as the scene is static. I want to take advantage of the camera that adapts to the lightning changes.

Answer 1

Lets assume your camera axis has polar and azimuthal angles and you want to align your frames only regarding the azimuthal angle and set the polar angle free.

First you have to find the geometric transform between sequential frames of your video, then by multiplying these sequential geometric transforms between your reference frame and the current frame you could obtain the geometric transform between your current frame and the reference frame. To obtain geometric transform between sequential frames you could use automatic point matching algorithms like SIFT or SURF (considering all your frames don't have any intersections but sequential frames have intersection I suggested to use it sequentially).

After finding the geometric transform you have to modify the geometric transform to set the polar direction of camera free. To do this You have to decompose the geometric transform to find external camera parameters which contain the position and the pose of camera. Pose of camera is a Rotation matrix which rotate the axis of camera between your reference and current frame. After obtaining rotation matrix you have to decompose the rotation matrix to obtain the rotation angles which are yaw, pitch and roll and set Yaw angle to zero (I'm not sure which one is the polar angle of your camera but any of these angles which is polar angle of camera must be zero to set the polar direction free), then you have to obtain the modified geometric transform from the modified rotation matrix.

Now you could align all your frames using the inverse geometric transform.

All parts of suggested method have intrinsic MATLAB functions, also if your camera has nonlinear distortion like fish-eye you have to take care of that before point matching part.