22
$\begingroup$

This question is on dsp.SE as I'm mostly interested in the signal processing part.

There is an Indian movie Mughal-e-Azam which was released in 1960 in black & white which has been reproduced in color in 2004.

  • How did they color each pixel perfectly?
  • What technique have they used to identify the color placement on each pixel?

Look at one of the screenshots from the movie:

Movie Screenshot

I've got an Einstein black-and-white photo which I want to colorize. How is it possible to do so without knowing what he was wearing back then and what was the actual color of his clothes, background etc.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Firstly, its impossible to perfectly reconstruct the original colors. Secondly, They most likely used a bit of guessing of not colored each image painstakingly in by hand. $\endgroup$
    – CyberMen
    Jun 15, 2012 at 17:39
  • 1
    $\begingroup$ With regard to the choice of colors, as I remember it, one reel of the movie was shot and released in color, and the colors of the clothing from those sequences might have influenced the choice of colors in the parts that were colorized later, e.g. a prince would be expected to have clothes of different colors, a slave girl might be expected to have just one or two outfits. On the other hand, inconsistency and lack of continuity, thy name is Bollywood. $\endgroup$
    – Dilip Sarwate
    Jun 18, 2012 at 13:27
  • 1
    $\begingroup$ You will notice, in those shots above, that a lot can be accomplished by simply separating the actors from the background, coloring the background bluish, and coloring the actors and their clothes a brownish color, with the intensely white or dark areas being colored differently, plus maybe a few carefully selected details being separately colored. $\endgroup$
    – Daniel R Hicks
    Jun 18, 2012 at 15:53
  • $\begingroup$ movies.stackexchange.com/questions/88387/… $\endgroup$
    – Muze
    May 10, 2018 at 18:57

3 Answers 3

Reset to default
26
$\begingroup$

There is no way of recovering the original color information from a black and white photo, so whether Einstein (resp. Waheeda Rehman) was wearing a pink or green sweater (resp. Dupatta) is up to your imagination.

Historically, this has been done by hand, by painting over the film. The first digital techniques to automate the process consisted in "painting" a few dots of color on each frame, at the center of each uniformly colored region, and using something like a voronoi partition + some blurring to get a color map for each frame (see for example US patent 4606625).

Today, this can be done relatively easily (though manually) with video editing software, by using vector masks to indicate regions of uniform color on a few keyframes, and interpolating between them. Then a color transform is applied to each mask. See it in action here.

Standard image segmentation and region tracking techniques can be used to automate the task of segmentation and the marking of regions on each keyframe - for example by propagating manual annotations to similar/adjacent pixels in space/time, or by detecting uniformly textured regions. Texture and gray level similarity can be used to propagate color cues from a color image to a greyscale image depicting a similar subject - in this case the manual process only consists in finding a template color image - this later task can be itself automated using content-based image retrieval techniques.

Improve this answer
$\endgroup$
6
$\begingroup$

I've only read about this, and it's been at least 15 years since I saw anything in print, but I understand there are a number of algorithms used.

First, as stated, some films have been colorized by painstakingly tinting each frame by hand. (The first few minutes of Gone with the Wind were done this way, after the decision was made to film the rest in color.)

After that, several techniques are used, all based on computer image processing. The simplest approach is to break the film into individual scenes, each where the camera doesn't change, and then have a computer algorithm recognize regions of differing intensity. Someone on a computer monitor looks at the first few frames of the scene and sets the color of each region, then the computer carries that coloring through the rest of the scene, assuming that the identified regions will not morph so rapidly that the connection with their assigned colors will be lost. This technique is usually limited to colorizing the main actors and the basic background, since scenes change so rapidly in most movies,

More sophisticated techniques use image recognition algorithms to recognize (eg) a person, separate the background from the foreground, etc. Then colors can be assigned on a more global basis and some degree of shading can be accomplished as actors move from shadow to light, etc. And, since this scheme can work beyond a single scene, more effort can be put into having a human initially set the colors, so a more varied and vivid palette can be used.

I would guess (though I've never read) that technique resembling computer animation can now be used, based on techniques used for animating action movies. A sort of motion capture could be used on the original film, and then a computer animation of the action would be produced. With some careful "blending" the original film's detail could be retained while using the colors generated by the animation.

Improve this answer
$\endgroup$
2
$\begingroup$

BBC discovered the way by decoding colour dots to restore a formerly colour movie to which only the black and white copy is available. They are able to restore the colour back to the original colour.

Source: http://www.youtube.com/watch?v=CjK-b4x9ZmQ

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ I declined to approve your edit on @pichenettes answer because it was in error. The reason this BBC approach worked is because there was, effectively, colour information encoded in the television recording. In general (as address in the other answer) this information is completely lost. BTW: +1: an interesting technique, if the source material is recorded this way! $\endgroup$
    – Peter K. ♦
    Dec 12, 2013 at 17:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.