7
$\begingroup$

I am looking for a low MIPS, low compression video encoder. This is for a 10fps, VGA type quality compression. What are my options? I need to be able to do this compression using a 150MHz ARM M4 CPU with Floating point support. (STM32F4)..

My idea is to push this compressed data out of the CPU with a parallel bus, there will be no processing done on the data. Compression rate wise, as much as possible, I just want to see the boundaries. This is for a low cost CCTV application, I like to see what I can achieve with a 5USD cpu and lots of transmission band with vs. a 30USD encoder with low data trasnmission bw.

10fps, VGA will generate roughly 25Mbit/sec data. This is fairly high data rate for anything out there. If I can bring this down to 5Mbit/sec, I think I can build a very low cost CCTV system. Once I get the data to the base, I can re-encode the data, therefore it doesn't matter what the compression mechanism is, as long as it is not very lossy.

Monochromatic video is what is necessary at moment over color.

Update

  • This CPU has 120MHz allocated to this task.
  • Memory interface is 16 bit, therefore external memory write/read is slower compared to internal memory.
  • Internal memory is 120KByte and it has 32 bit access fast access. Both cases the memory is accessed via AHB bus which we should assume 60MHz as the clock frequency.
  • Following data flow is expected:
    1. Camera ->DMA ->External Memory (no cpu involvement)
    2. External Memory -> CPU ->Compression ->Internal Memory
    3. Internal Memory -> DMA -> Data Bus -> External Device

CPU will only read a chunk of data compress and write to its internal memory (the compressed data), later start a DMA transfer.

$\endgroup$
  • $\begingroup$ how much of your processor can you afford to devote to this? And what's an acceptable compression rate; 5x, 2x, less than that? $\endgroup$ – Martin Thompson Apr 9 '12 at 12:45
  • $\begingroup$ I can dedicate 80-90%. My idea is to push this compressed data out of the CPU with a parallel bus, there will be no processing done on the data. Compression rate wise, as much as possible, I just want to see the boundaries. This is for a low cost CCTV application, I like to see what I can achieve with a 5USD cpu and lots of transmission band with vs. a 30USD encoder with low data trasnmission bw. $\endgroup$ – Ktuncer Apr 9 '12 at 13:51
  • $\begingroup$ Just to add: 10fps, VGA will generate roughly 25Mbit/sec data. This is fairly high data rate for anything out there. If I can bring this down to 5Mbit/sec, I think I can build a very low cost CCTV system. Once I get the data to the base, I can re-encode the data, therefore it doesn't matter what the compression mechanism is, as long as it is not very lossy. $\endgroup$ – Ktuncer Apr 9 '12 at 13:55
  • $\begingroup$ as you mention security... Colour or mono? $\endgroup$ – Martin Thompson Apr 9 '12 at 15:33
  • $\begingroup$ Mono is what is necessary at moment. Does that make a difference? $\endgroup$ – Ktuncer Apr 9 '12 at 16:28
2
+50
$\begingroup$

1. Classify whether you need low MIPS or low complexity overall
Let me take a small liberty to split this problem in two parts.

  1. Low complexity encoding - which allows lower resources (specially in memory) to make it quick responding encoding in the given system.
  2. Low explicit in computation (MIPS). - which only concerned with minimum possible CPU cycles.

There is third criteria - where in people talk about "Low latency" - for application like Video conferencing where computational resource might not be of concern -but overall delay introduced by coded

What is important to note that in lower complexity systems - access to memory (memory bus speed and width) and sometimes IO are generally extremely slower which is why MPEG class of algorithms suffer even though the computationally algorithm could be simple.

Before you make judgement about the requirement for codec - you should try to make budget in terms of -

a. CPU cycles per second.
b. Maximum memory through put.
c. IO latency

2. You should customize MPEG rather than re-invent
In general - MPEG class of codecs gives you a very flexible mechanisms to do this. In that sense, you don't quite have to re-invent the codec as much as you would rather customize MPEG 2 or MPEG 4 to get your work done.

First off, let's say what all elements make compression possible and arrange them in the order of complexity:

  1. Motion Estimation and Motion Compensation. 1.a. Forward prediction (P frames) 1.b. Double prediction (B frames) 1.c. high resolution motion vectors
  2. Intra coding - DCT (+ IDCT)
  3. Qunatization - and encoder mode selection
  4. VLC - variable length coding. (CABAC in H.264)
  5. Co-efficient prediction [ In mpeg2 only DC prediction- MPEG4 has more]

In MPEG class of algorithms, the DCT based encoding and VLC becomes pretty much compulsory without much choice - but rest of all mechanisms are very essential

For example, Motion estimation and motion compensation is one of the highest MIPS consuming element. If you don't have resources to do this - you can simply encode all frames as I frames (which makes it pretty much like MJPEG - but standard MPEG decoder can decode this). If you can afford slightly more resources - you can make trivial motion compensation using Frame difference - instead of sending every frame as Intra, you can subtract every blcok from it's previous frame block; if the difference is higher than original signal send it as Intra.

Of course - all this would mean that you will loose some efficiency promised by above encoder - but i think you are willing to give up that!


EDIT:
You can look at the following codecs as good reference points:

  1. MSSG: http://www.mpeg.org/MPEG/video/mssg-free-mpeg-software.html
    Good for understanding but might be next to slowest in the world.

  2. FFMPEG: http://ffmpeg.org/ Probably the fastest on the earth. Good to start as black box but don't try to change the code from inside. Might give you good options to control stuff when you use library API. It is already ported in many platforms - but doing it on a new one might be some task.

  3. FAME: http://fame.sourceforge.net/ This was originally started with purpose same you described. Though, i am a bit out of touch of this - but you can try this.

  4. Xvid: http://www.xvid.org/ This is MPEG-4. This is one of the best balance between clean code and reasonable speed. Should be easiest to work with if you endup dwelling inside of the encoder.

  5. JPEG: http://www.ijg.org/ This is JPEG. This is one of the best libraries to port it across platforms. Also, JPEG is inherently simpler than some aspects of MPEG - so may be you should try this first. Most camera's in the world has probably used this library as is - rather than creating something its own!

May be i am wrong in using MPEG! But that's a kind of risk worth taking.

Probably best measure to check if that will work or not is - just try to take a standard 8x8 DCT with quantization on your image; optimize just this. If you are able to reach close to your real time requirement, than i think you are good to do about all All JPEG frames or all I frame MPEG codec. If you are way off the target - than it is not worth it.

$\endgroup$
  • $\begingroup$ Dipan, this is a good answer however I am looking for something a little bit more quantified. You are saying, it depends. We know that, I am looking for an educated guess. An experienced person on these stuff could say, "don't bother with x, focus on Y and this is the critical bit you need to watch out. If you do all correctly, you will get this type of compression, this much mips will be used". Based on my experience once you find the right guy, the answer is usually 20% in the vicinity. $\endgroup$ – Ktuncer Apr 12 '12 at 8:55
  • $\begingroup$ In fact i also tried to first craft some detailed - profiling information, but i thought that might not reflect directly because yours is very specific hardware where this needs to get reflected. So i was limited to give you answer which was more of directional than quantitative. Also, probably best take away from answer could be, which i wanted to emphasis that stick to the well known MPEG and tune it rather than re-invent. $\endgroup$ – Dipan Mehta Apr 12 '12 at 9:01
  • $\begingroup$ For being quantitatively perfect, my suggestion would be that you take a good (reference) MPEG encoder and start profiling it. Based on those actual profiling results, i can guide you till the end. what will exactly work toward real time encoding will all depend on those specific things you can achieve, on your hardware platform to specifically tailor the encoder to meet real time constraints. (of course, that would need you to ask many more questions!) $\endgroup$ – Dipan Mehta Apr 12 '12 at 9:03
  • $\begingroup$ I appreciate your MPEG idea, all iFrames etc. This is a good approach, however if I start with this info, I may find in three months that it is impossible, since MIPS required (even for watered down) is beyond the CPU or compression is only 10% or something like that. I need some numbers, even if it is not for this platform. $\endgroup$ – Ktuncer Apr 12 '12 at 9:04
  • $\begingroup$ What is a good open source MPEG encoder? Something written in ANSI C, I can take and start playing with. $\endgroup$ – Ktuncer Apr 12 '12 at 9:07
4
$\begingroup$

Your first problem is memory - even the largest version of that processor doesn't have that much internal SRAM (in video terms - it's a stonking embedded processor compared to many though!) - a VGA frame is 300kB, vs the 100ishkB in the processor. That means you'll have to process it as it comes in - say in 8 or 16 line chunks, which makes it much more of a "hard-realtime" problem as the deadlines have less slack in them.

It's not clear how you'll be capturing data, but I assume that you'll have some sort of DMA from either the ADC or some digital port, otherwise you'll be spending half your time shuffling data around!

Aiming for lowish compression ratios, you might just want to look at encoding the deltas between pixels along a line or a 3x3 square and then arithmetic code them.

See also Simple, streaming, lossless image compression - although I was asking for explicitly lossless compression, there may be some ideas for you there.

As a datapoint, one of my colleagues implemented a 320x240 8-bit mono JPG compression in a 60MHz 32-bit micro with floating point unit and a small cache. He used some reference code (ie optimised for readability, not performance) and got 5 fps fairly easily. Compression ratios were of the order of 10x IIRC. The image capture was done by external hardware bus-mastering the data into the micro's external RAM.

$\endgroup$
  • $\begingroup$ Martin, thanks for the answer.. I can and will add a 4Mbit memory to this processor. So, I can keep the whole frame in memory. The data will flow from camera interface to the external memory via DMA, I have zero (or close to zero) involvement. I can optimize that portion of the code fairly easily. If I must, I can also add more memory. The idea is to use the internal RAM as a cache and use external RAM as storage while I am processing the blocks. The way I see it is I have 50msec to encode and shift the data out, I have 300KB data. If I can reach 20fps, amazing. $\endgroup$ – Ktuncer Apr 10 '12 at 12:13
  • 1
    $\begingroup$ @Ktuncer: how much time will you use pulling pixels in from external ram and pushing it back again? Is there a DMA between ext&int memory? $\endgroup$ – Martin Thompson Apr 10 '12 at 12:31
  • $\begingroup$ Yes.. I will use DMA to from Camera -> external memory, another DMA to external memory->internal memory. $\endgroup$ – Ktuncer Apr 10 '12 at 14:24
  • $\begingroup$ The open source JPG library that Martin mentioned is called IJG. Link: ijg.org $\endgroup$ – user1276 Apr 22 '12 at 6:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.