Is it possible to do single vehicle tracking using Fourier transform?

Question

I am working on a project in image processing which is based on importance of phase only reconstruction of a signal obtained using Fourier transform.For more information about phase only reconstruction,you can see the answer given by geometrikal in the link.

Now ,I have detected moving objects from the video of Traffic on road taken using stationary camera ( Please download the 1.47 MB video for testing MATLAB Code by ( step1) click on the play button then (step2) right clicking on video then ( step3 ) click on save as option )

Algorithm No. 1 The proposed approach

Requirement: An input image sequence I(x, y, n) (where x and y are image dimensions and n represent frame number in a video) which is extracted from video.

Outcome: The segmentation mask of moving object for each frame

1. For each frame in a input video perform step 2, append step 2 result in resultant array ‘I(x, y, n)’

2. Smoothen the current frame using 2D Gaussian filter

3. Perform 3D FFT for the whole sequence I(x, y, n) using (Eq.4.1)

4. Calculate the phase spectrum using the real and imaginary parts of 3D DFT

5. Calculate the reconstructed sequence Î(x, y, n) using (Eq.4.2)

6. For each frame in a input video perform step 7 to step 10 to get segmentation mask for each frame and append step 10 result in resultant segmentation mask array BW(x,y,n)’

7. Smooth the reconstructed frame of Î(x, y, n) using the averaging filter.

8. Compute the mean value of the current frame

9. Convert the current frame into binary image using mean value as the threshold

10. Perform morphological processing, i.e., filling and closing, to obtain segmented mask of moving object for the current frame

11. End algorithm.

If you run my MATLAB code, you can observe that I am quite successful in detecting all the moving objects in each video frames. But now I want to detect only one moving object at a time from current frame and avoid other moving objects by making changes in the same code or algorithm .But don't understand how it can be done.

so can anybody tell me

Is it possible to do single vehicle tracking using Fourier transform?

    tic
clc;
clear all;
close all;
  
%read video file
video = VideoReader('D:\dvd\Matlab code\test videos\5.mp4');

T= video.NumberOfFrames  ;           %number of frames%

frameHeight = video.Height;          %frame height

frameWidth = video.Width ;           %frameWidth

get(video);                          %return graphics properties of video


i=1;

for t=300:15:550  %select frames between 300 to 550 with interval of 15 from the video  
    frame_x(:,:,:,i)= read(video, t); 
    frame_y=frame_x(:,:,:,i);

    %figure,
    %imshow(f1),title(['test frames :' num2str(i)]);
    frame_z=rgb2gray(frame_y);                 %convert each colour frame into gray
    
    frame_m(:,:,:,i)=frame_y; %Store colour frames in the frame_m array 
     
    %Perform Gaussian Filtering
    h1=(1/8)*(1/8)*[1 3 3 1]'*[1 3 3 1]  ;   % 4*4 Gaussian Kernel  
    convn=conv2(frame_z,h1,'same');
        
    g1=uint8(convn);
    
                    
    Filtered_Image_Array(:,:,i)=g1; %Store filtered images into an array
    i=i+1;
end

%Apply 3-D Fourier Transform on video sequences
f_transform=fftn(Filtered_Image_Array);

%Compute phase spectrum array from f_transform
phase_spectrum_array =exp(1j*angle(f_transform));

%Apply 3-D Inverse Fourier Transform on phase spectrum array and
%reconstruct the frames
reconstructed_frame_array=(ifftn(phase_spectrum_array));


k=i;

i=1;
for t=1:k-1
    
    %Smooth the reconstructed frame of Î(x, y, n) using the averaging filter.
    Reconstructed_frame_magnitude=abs(reconstructed_frame_array(:,:,t));  
    H = fspecial('disk',4);
    circular_avg(:,:,t) = imfilter(Reconstructed_frame_magnitude,H);
        
    
    %Convert the current frame into binary image using mean value as the threshold
    mean_value=mean2(circular_avg(:,:,t));  
    binary_frame = im2bw(circular_avg(:,:,t),1.6*mean_value);
    
    
    %Perform Morphological operations
    se = strel('square',3);
    morphological_closing = imclose(binary_frame,se); 
    morphological_closing=imclearborder(morphological_closing); %clear noise present at the borders of the frames
    
    
    %Superimpose segmented masks on it's respective frames to obtain moving
    %objects
    moving_object_frame = frame_m(:,:,:,i);
    moving_object_frame(morphological_closing) = 255;  
    figure,
    imshow(moving_object_frame,[]), title(['Moving objects in Frame :' num2str(i)]);
    
 i=i+1;
end
toc

Answer 1

From my understanding of the linked answer which you base your algorithm on I would conclude that the FT will detect all the edges in the frame domain, so all the moving objects.

If you want to localize the transform information, I suggest you use a wavelet transform with a complex wavelet. Instead of correlating the signal with $e^{i2\pi fx}$ as FT does it would be correlated with oscillations localized by some window. A commonly used wavelet is the complex Morlet (a.k.a. Gabor) wavelet $e^{i2\pi fx - (xf_p)^2}$ which uses a Gaussian window to localize the oscillation (I left out the normalization constants). Commonly $f=f_p$ is used to get a so called constant Q transform. Note that this is just an example of a wavelet that I use for analyzing evolution of spectra of some simple 1D signals, you may find that a different choice of a wavelet suits your needs better.

The MATLAB wavelet toolbox is designed for 1D or 2D signals. Because you need to localize the transform only in the x and y dimensions, so you might be able to apply the 2D cwt on each slice and then compute the FFT only accross the frame dimension. You equation (4.1) would then have the terms $e^{-(xu)^2 - (yv)^2}$ added in the case of the complex Morlet wavelet. Note that the resulting transform matrix will have also dimensions for the (x,y) coordinates. All in all I think it will be really hard to interpret the result.

Perhaps a simpler way would be to use your existing algorithm and select a mask (region) in the image corresponding to the object of interest and then perform a correlation of that region with the next frame. Assuming the object won't greatly change in the next frame (which in this head-on view of the highway might work, but may fail if it becomes larger in the objective of the camera), the correlation should give you a maximum in the next frame in the region where it moved to. Then you perform the correlation of this new region with the following frame. These correlation maximums then create a trail in the frame domain.

You could also cut down on the computation time by computing the correlation only in the regions that are moving. If the regions don't overlap and move slowly, you could just select the region in the next frame that is the closest to the region location in the current frame. This would be more robust in terms of the object becoming larger, but would require slow movement and that the trails don't suddenly cross.

Edit: If you insist on using STFT (but I really think the correlation/detecting region movement would be a lot simpler), it would essentially require the algorithm to partition the FFT computation into several windows, so the whole algorithm beginning with the line %Apply 3-D ... would run in nested for loops (one for each dimension), each iteration representing a single x,y window

Complete_Filtered_Image_Array = Filtered_Image_Array;  % store the whole array
x_window_len = 20;  % you will have to determine these values empirically
x_window_overlap = 10;  % half overlap is a reasonable default
y_window_len = 10;
y_window_overlap = 5;
for x = 0:x_window_overlap:frameWidth-x_window_len
    for y = 0:y_window_overlap:frameHeight-y_window_len
        Filtered_Image_Array = Complete_Filtered_Image_Array(x:x+x_window_len, y:y+y_window_len, :);
        % continue the previous algorithm with this sliced Filtered_Image_Array
        %Apply 3-D ...
        f_transform=fftn(Filtered_Image_Array);
        ...
     end
end

You'd also have to create a way to select the initial window with the object of interest and track which window it moves to, so in the end I think you'll be using the correlation/region movement technique anyways, so perhaps you don't need the STFT step at all.