Actually, I am new to this field and would like to work on a project in Video processing. I have a little background in Image processing but would like to study further in video processing. So, If anyone could suggest some readings as pre-requisites for video processing.
I recommend Video Tracking: Theory and Practice as comprehensive overview of what have been done in this area for last 15 years. Also, 3D Computer Vision: Efficient Methods and Applications for quick understanding useful algorithms.
This books will be a good start, but when you have enough knowledge, you should start looking into research papers, because information there is more updated than in books.