12
$\begingroup$

How would would one approximately reconstruct the branches in a photo of a tree, where portions of the branches are covered by leaves?

I can extract the visible portions of the branches by threshholding then opening with line segments at a few different angles. But how would I then (approximately) fill in the spaces that were covered by leaves?

Suppose we start with this drawing of a tree:

color tree

load the image as grayscale

uu = mmreadgray('tree2.jpg')

tree

threshold the darker colors; convert to binary; remove connected regions under 200px area

vv = mmareaopen(mmbinary(mmthreshad(mmneg(uu), 200)), 20)

tree

open vv with 10px line segments at 90 degrees, 180 degrees, 135 degrees and 45 degrees; add these images together to form a marker image zz

zz = mmaddm(mmaddm(mmaddm(mmopen(vv, mmseline(10, 90)), mmopen(vv, mmseline(10, 180))), mmopen(vv, mmseline(10, 135))), mmopen(vv, mmseline(10, 45)))

tree

reconstruct the visible limb portions in vv from marker image zz

aa = mminfrec(zz, vv)

tree

display result aa on top of original image uu; a reasonable extraction of limbs is achieved

mmshow(uu, aa) tree

I would appreciate any thoughts on how to now approximately reconstruct the entire limb. (Or improvements to the steps given above). Thank you!

Improve this question
$\endgroup$
4
  • 3
    $\begingroup$ Two people have upvoted your question - you should be able to upload images now. $\endgroup$
    – Niki Estner
    Mar 11 '13 at 8:16
  • 1
    $\begingroup$ @Martin I believe the technique known as 'inpainting' might be worth looking at. $\endgroup$
    – Spacey
    Mar 11 '13 at 14:50
  • 1
    $\begingroup$ Wouldn't it be more effective to isolate the branches by hue rather than by brightness? That is, pick the brown parts of the picture rather than the dark parts. As it is you are mixing in a lot of leaf shadow with your branches, and losing a lot of branches. $\endgroup$
    – travisbartley
    Jun 7 '13 at 4:41
  • $\begingroup$ I'd try to use lighting gradients on the branches to help determine their directions, or at least to eliminate bad proposed fill-ins. $\endgroup$
    – DarenW
    Aug 22 '13 at 20:19
1
$\begingroup$

The masked area hidden by leaves is very large and while the border and shading seems to imply what is hidden to a human being it is unsuited to "inpainting" which relies on pretty consistent boundary, such as where a foreground object temporarily obscures something continuing behind.

The branch orientation is a start but not enough data on its own to imply how the twigs extend without having some external concept of an oak tree versus a willow, poplar or pine. That is more about 3d fractal modeling than about image restoration.

The least obviously visible algorithms for in-painting involve texture synthesis called "Healing" in Photoshop. G'MIC and Resynth are popular open-source tools for that available as plug-ins for GIMP. While visual continuity is quite good they have no concept of a tree or depth, just 2-d patch statistics so would need input from an artist as well in the example given. The mask is white background on the outer edges and thick branches on the inside .. so texture interpolation gives a weird unnatural result.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.