Making static 3D image is very simple - we just take image pair with something like this:
then we can see 3D image by putting the left and right eye image side by side and crossing eyes or overlapping the images and using color filters for each eye. So far so good.
VR headsets (e.g. Oculus Rift) now allows seeing the 3D image directly.
However, I would like to incorporate slight movements and tilting of the head. Of course, this would immediately spoil the 3D effect.
First idea for better capture is using more cameras (e.g. 2x2 or 3x3 array) to acquire necessary viewpoints (e.g. top and bottom camera is necessary to reconstruct 3D when viewed with head leaned on one side). But how to interpolate between these views based on shifted and rotated head pose? Is depth/3D reconstruction necessary or are there any computational photography techniques to generate appropriate left/right eye images (for a given viewpoint/pose)?
I found various articles regarding depth reconstruction, 3D reconstruction and light fields, but nothing leads me toward the goal of interpolating the views.
Ultimately, I am planning to create a panorama viewer for VR glasses (something like krpano's web viewer supporting Oculus Rift). However, the viewer can account for limited head movements (pan, tilt) and the 3D impression does not hold for leaning, slight head shifts and correct for different eye-to-eye distance.
Any pointing to appropriate articles will be appreciated, thanks!