Low MIPS Video Encoder

Question

I am looking for a low MIPS, low compression video encoder. This is for a 10fps, VGA type quality compression. What are my options? I need to be able to do this compression using a 150MHz ARM M4 CPU with Floating point support. (STM32F4)..

My idea is to push this compressed data out of the CPU with a parallel bus, there will be no processing done on the data. Compression rate wise, as much as possible, I just want to see the boundaries. This is for a low cost CCTV application, I like to see what I can achieve with a 5USD cpu and lots of transmission band with vs. a 30USD encoder with low data trasnmission bw.

10fps, VGA will generate roughly 25Mbit/sec data. This is fairly high data rate for anything out there. If I can bring this down to 5Mbit/sec, I think I can build a very low cost CCTV system. Once I get the data to the base, I can re-encode the data, therefore it doesn't matter what the compression mechanism is, as long as it is not very lossy.

Monochromatic video is what is necessary at moment over color.

Update

• This CPU has 120MHz allocated to this task.
• Memory interface is 16 bit, therefore external memory write/read is slower compared to internal memory.
• Internal memory is 120KByte and it has 32 bit access fast access. Both cases the memory is accessed via AHB bus which we should assume 60MHz as the clock frequency.
• Following data flow is expected:
  1. Camera ->DMA ->External Memory (no cpu involvement)
  2. External Memory -> CPU ->Compression ->Internal Memory
  3. Internal Memory -> DMA -> Data Bus -> External Device

CPU will only read a chunk of data compress and write to its internal memory (the compressed data), later start a DMA transfer.

Answer 1

1. Classify whether you need low MIPS or low complexity overall
Let me take a small liberty to split this problem in two parts.

1. Low complexity encoding - which allows lower resources (specially in memory) to make it quick responding encoding in the given system.
2. Low explicit in computation (MIPS). - which only concerned with minimum possible CPU cycles.

There is third criteria - where in people talk about "Low latency" - for application like Video conferencing where computational resource might not be of concern -but overall delay introduced by coded

What is important to note that in lower complexity systems - access to memory (memory bus speed and width) and sometimes IO are generally extremely slower which is why MPEG class of algorithms suffer even though the computationally algorithm could be simple.

Before you make judgement about the requirement for codec - you should try to make budget in terms of -

a. CPU cycles per second.
b. Maximum memory through put.
c. IO latency

2. You should customize MPEG rather than re-invent
In general - MPEG class of codecs gives you a very flexible mechanisms to do this. In that sense, you don't quite have to re-invent the codec as much as you would rather customize MPEG 2 or MPEG 4 to get your work done.

First off, let's say what all elements make compression possible and arrange them in the order of complexity:

1. Motion Estimation and Motion Compensation. 1.a. Forward prediction (P frames) 1.b. Double prediction (B frames) 1.c. high resolution motion vectors
2. Intra coding - DCT (+ IDCT)
3. Qunatization - and encoder mode selection
4. VLC - variable length coding. (CABAC in H.264)
5. Co-efficient prediction [ In mpeg2 only DC prediction- MPEG4 has more]

In MPEG class of algorithms, the DCT based encoding and VLC becomes pretty much compulsory without much choice - but rest of all mechanisms are very essential

For example, Motion estimation and motion compensation is one of the highest MIPS consuming element. If you don't have resources to do this - you can simply encode all frames as I frames (which makes it pretty much like MJPEG - but standard MPEG decoder can decode this). If you can afford slightly more resources - you can make trivial motion compensation using Frame difference - instead of sending every frame as Intra, you can subtract every blcok from it's previous frame block; if the difference is higher than original signal send it as Intra.

Of course - all this would mean that you will loose some efficiency promised by above encoder - but i think you are willing to give up that!

---

EDIT:
You can look at the following codecs as good reference points:

1. MSSG: http://www.mpeg.org/MPEG/video/mssg-free-mpeg-software.html
   Good for understanding but might be next to slowest in the world.

2. FFMPEG: http://ffmpeg.org/ Probably the fastest on the earth. Good to start as black box but don't try to change the code from inside. Might give you good options to control stuff when you use library API. It is already ported in many platforms - but doing it on a new one might be some task.

3. FAME: http://fame.sourceforge.net/ This was originally started with purpose same you described. Though, i am a bit out of touch of this - but you can try this.

4. Xvid: http://www.xvid.org/ This is MPEG-4. This is one of the best balance between clean code and reasonable speed. Should be easiest to work with if you endup dwelling inside of the encoder.

5. JPEG: http://www.ijg.org/ This is JPEG. This is one of the best libraries to port it across platforms. Also, JPEG is inherently simpler than some aspects of MPEG - so may be you should try this first. Most camera's in the world has probably used this library as is - rather than creating something its own!

May be i am wrong in using MPEG! But that's a kind of risk worth taking.

Probably best measure to check if that will work or not is - just try to take a standard 8x8 DCT with quantization on your image; optimize just this. If you are able to reach close to your real time requirement, than i think you are good to do about all All JPEG frames or all I frame MPEG codec. If you are way off the target - than it is not worth it.

Answer 2

Your first problem is memory - even the largest version of that processor doesn't have that much internal SRAM (in video terms - it's a stonking embedded processor compared to many though!) - a VGA frame is 300kB, vs the 100ishkB in the processor. That means you'll have to process it as it comes in - say in 8 or 16 line chunks, which makes it much more of a "hard-realtime" problem as the deadlines have less slack in them.

It's not clear how you'll be capturing data, but I assume that you'll have some sort of DMA from either the ADC or some digital port, otherwise you'll be spending half your time shuffling data around!

Aiming for lowish compression ratios, you might just want to look at encoding the deltas between pixels along a line or a 3x3 square and then arithmetic code them.

See also Simple, streaming, lossless image compression - although I was asking for explicitly lossless compression, there may be some ideas for you there.

As a datapoint, one of my colleagues implemented a 320x240 8-bit mono JPG compression in a 60MHz 32-bit micro with floating point unit and a small cache. He used some reference code (ie optimised for readability, not performance) and got 5 fps fairly easily. Compression ratios were of the order of 10x IIRC. The image capture was done by external hardware bus-mastering the data into the micro's external RAM.