There is already a succesful question How did they color old black and white movies? on this site. Very general and correspondingly in general answered. Let me please narrow it down to just the photograpy (i.e. no time dimension) and more specified starting point:
What is the algorithm to color the greyscale HD picture given the colors at various points on greyscale original?
Note: Yes, I am aware that the original colors can't be restored any more.