1
$\begingroup$

Consider an image sensor that is moving with some velocity $\mathbf{v}(t) \in \mathbb{R^3}$, and an orientation $\mathbf{R}(t) \in SO(3)$. Treat those vectors as fixed. At a specific time $t$ the projection of the scene on the sensor is: $$u_{\mathbf{v},\mathbf{R}}(t)$$ The sensor pixel data is now sampled in a series fashion. This means that each pixel will be sampled at a different time $t$.

Now suppose that you want to calculate the Fourier transform of a frame. How would you do it? Can you do it without interpolation? I'm after accuracy here.

Furthermore, what is the effect of the lack of sample synchronization on the FFT i.e. If you just take the FFT without first synchronizing the pixels. Is there a rigorous mathematical treatment of this subject? What is the frequency representation of the sampled data?

Edit

This is for a rectangular image sensor in an interferometry application. This means that although the velocity is small $v \approx 1 \mathrm{m/s}$, the image changes rapidly.

diagram

I thought about it some more and I guess what you would have is aliasing for each pixel equal to the frame rate, and each pixel will have a different amount of phase shift equal to the pixel clock.

Improve this question
$\endgroup$
9
  • $\begingroup$ Is this a linear image sensor that scans the scene or is it a rectangular sensor? In any case, pixels are sampled at different times anyway, the point is that the acquisition time is much quicker than the frame rate. If not, you will get phase distortions and these cannot be corrected. You can derive the spectrum of the composite by spectrums of components but if you can create the composite first, why not FFT that (like it was a single image) anyway? Can you please add a bit more information? $\endgroup$
    – A_A
    Jun 28, 2016 at 15:02
  • $\begingroup$ @A_A Added some more information. $\endgroup$
    – user110971
    Jun 28, 2016 at 15:09
  • $\begingroup$ Thank you. Can you please add a bit more information? I am aware of interferometry but can't quite imagine the moving sensor setup. Is there overlap? Is the interferometry pattern stable but very large or very large AND dynamic? So we need to understand what u is "looking at" to understand how to counteract the sampling. I am thinking here of course about the FFT shifting properties. There must be something predictable in u, to "exploit" for correcting for the unknown parts hapenning outside each imaged frame. $\endgroup$
    – A_A
    Jun 28, 2016 at 15:38
  • $\begingroup$ It generates an interference pattern of repeating bright bars. The system undergoes linear motion, and you know what the pattern will be at each position. The problem is that you use the spacing between the bright bars(fringes) to determine the position. The spacing changes as you move. The pattern repeats every wavelength, which is about $632.8 \mathrm{nm}$. $\endgroup$
    – user110971
    Jun 28, 2016 at 18:08
  • $\begingroup$ Thank you, I am sorry but I assumed that you had a 2D pattern. $\endgroup$
    – A_A
    Jun 28, 2016 at 20:02

1 Answer 1

Reset to default
1
$\begingroup$

I feel that the question needs a little bit more clarifying but there probably is enough material here to provide some sort of response and adjust it later if needed.

To cut a long story short: Rolling Shutter distortion. This concept is key here (irrespectively of whether or not you have a rolling or global shutter camera that is unsynchronised to the interferometer).

Now, I will treat $v, R$ as independent from time here for a minute. Also, $F_{FR}$ will stand for frame rate.

Your "problem" seems to be that due to $v$, each pixel row might actually be sampling spectral lines from two radically different wavelengths. If this is true then each one of your frames will acquire a slant whose angle will be proportional to $\frac{v}{F_{FR}}$.

In the time dependent version, the frames are capturing a "curve" whose curvature is proportional to $\frac{v(t)}{F_{FR}}$ and...good luck with that.

Since you are dealing with interferometry, you are only really imaging ONE single focal plane, therefore, you could apply a simple (anti)-skew transform to counteract the skew caused by the rolling shutter. This, of course, is within limits. In any case, the rolling shutter effect of the camera inserts a phase error in the image which, to an extent, can be counteracted but in the end it all comes down to controlling $v$.

Assuming that your camera can maintain a constant $F_{FR}$ for the duration of the scan, then essentially, you don't have a 2D camera.

I would like us to imagine it as one long linear $M \times N$ camera (because of $v$ and interferometry) only it is wrapped around with a "wrap" factor that depends on $v$.

So, theoretically, you could find a $v$ which scans a spectrum range in one frame. Maybe $v$ is such that the spectral range is contained in two frames or three frames or four...Obviously, the more, the better. Therefore, each pixel row now has a $\lambda_{start}$ and a $\lambda_{end}$. So, I want you to imagine a chirp that is wrapped across the camera sensor or in fact across many different frames.

Therefore, you could convert your single $M \times N$ image, into one long vector and then do a spectrogram on this vector. This vector is essentially encoding wavelength to space (across the camera sensor).

Does this have any distortions? Of course it has, but it would be like trying to obtain the frequency response of a system by feeding it a chirp. Within limits it is perfectly possible. If you scan it too fast, you might skip a spectral line

Perhaps it is easier to see why $v(t)$ would insert a phase error to the "chirp" when considering the camera as one long image sensor (?).

Hope this helps.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.