Signal Processing Stack Exchange is a question and answer site for practitioners of the art and science of signal, image and video processing. It only takes a minute to sign up.
Sign up to join this community
I read in a paper http://tcts.fpms.ac.be/publications/regpapers/2011/nrmmbgtd_icvs_LNCS.pdf about spatio-temporal filters, which is applied to frames of the videos. I could not understand these terms in the paper if anyone can help.
3D Low Pass Filtering
Temporal Neighborhood
Spatio Temporal Filters
In the context of a video the terms mean the following:
3D Low Pass Filtering: The three dimensions (3D) are x (horizontal), y (vertical), and time. Hopefully you know what a low pass filter is.
Temporal Neighborhood: I did not find this term in the paper. Given the context, though, I would guess that they mean for a video frame at time $t$, the temporal (time) neighborhood would be all data (i.e. frames) at nearby times. Thus, if the length your spatio-temporal filter along the time dimension was $T$, then the time neighborhood would be all frames from time $t-T/2$ to $t+T/2$.
Spatio-Temporal Filters: Filters that filter in both space and time. In this case you have two dimensions of space- x and y. Think of your data as being a three-dimensional matrix, where one dimension of the matrix is the x index, one dimension is the y index, and one dimension is the time index. The time index would be the frame numbers. The x and y indices would be the pixel positions. The spatio-temporal filter would simply be a three-dimensional filter that would operate on your data.
The temporal neighborhood refers to data nearby in time. That is; data located in the frames just before and just after the current frame.
The spatial neighborhood on the other side refers to the data nearby in space. In this case this is the pixels nearby in the same frame.
Spatio-temporal filters (which is 3D filters) takes both nearby pixels in the current frame and nearby pixels in nearby frames into account. In this way you can take advantage of information from more than one frame, which is often a good idea since most video sequences does not change much from frame to frame.
The temporal neighborhood might include points at the same X,Y coordinate, but in the preceding or subsequent video frame. The spatial neighborhood might include points in the same frame, but offset a small displacement from an X,Y coordinate. The spatio-temporal neighborhood might include points a small displacement in terms of (X,Y,T) coordinates (where T is some scaled delta frame time).
