Is Mask R-CNN a good fit for what I want to do?

Question

I'm looking at isolating the foreground from a set of photos, and I think I want to use instance segmentation so that I can try to gauge how well-focused the main subject of each photo is.

I have rudimentary experience of Yolo/darknet (but want to move beyond simple bounding boxes). I've never used python, and am mainly a PHP developer. Happy to learn whatever's needed though, but I don't want to spend a day messing around only to find I've picked the wrong tool for the job, or that it's far too complex and I can't get my head around it!

I have a server with a 1080 GPU in it. Speed is kind of important, but so long as it's less than a couple of seconds per photo, that should be ok. Is that feasible? The photos are 10MP+ in size, but I'm guessing it's standard practice to size down to (say) 1MP first, or whatever.

I have millions of photos that I want to run this against, to help weed out poor-quality shots.

Where should I be looking to get started? I see a lot of mentions of Mask R-CNN, but I don't know if that's a good fit or if there's better/faster/easier options out there right now?

Furthermore, should I use existing yolo/darknet bounding boxes, and somehow import them into Mask R-CNN et al, or is it better to start afresh and let Mask R-CNN deal with all of it?

Thank you!

Answer 1

I'm looking at isolating the foreground from a set of photos

This is called semantic segmentation. Instance segmentation is where the network actually can tell the difference between different instances of the same type. For example, a few apples on a table will be instance segmented and labels the three apples individually as apple 1, 2, and 3. Semantic segmentation is an easier problem than instance segmentation and they are the same if there is only one instance of the object.

Where should I be looking to get started?

I do not have experience with Yolo/darknet but have used Mask RCNN. Mask RCNN is written in Python using Tensorflow/Keras and the base version of the code can be found here: https://github.com/matterport/Mask_RCNN. It is fairly straight forward to get running out-of-the-box with little changes needed. Mask RCNN does instance segmentation but I recall it was straightforward to add a background class.

I'd suggest choosing a tool that is meant for the problem instead of hacking a tool to match your problem. You can look into the semantic segmentation network called DeepLab, https://github.com/sthalles/deeplab_v3. This also provides a base version of the code written in Python using Tensorflow/Keras. If you pick either of these tools, I assure you that you will not waste time and believe the setup to be minimal as they both work as is.