How can I extract the count of particular macro block types(I, P and S) from the frames in the video (H.264 and MP4)? Also, I would like to extract the prediction residual of each of the P and B frames (computed from its corresponding reference frame) in the video from the encoded version itself while the package is decoding it.
There are several tools to analyze video streams which which can provide slice and macroblock level information:
- Intel Video Pro Analyzer
- Tektronix Video Analyzer
- ffmpeg (page Analyzing Macroblock Types)
From How extract Intra-Prediction modes out of h264 stream, it seems that ffmpeg or ffprobe can do a lot of jobs.