I see in many reviews on Autonomous Car how they segment the images with person, cars, etc...
How is it achieved in Deep Learning?
Could anyone give an example of that?
How it is done?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ Are you after the case of pre defined classes or object segmentation? $\endgroup$– MarkCommented Nov 29, 2021 at 8:36
-
$\begingroup$ Example of either will be great. $\endgroup$– DavidCommented Nov 29, 2021 at 18:11
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Well, I think the best way to tackle this question is a little background and a code as an example. I chose MATLAB for this example though PyTorch / Keras would probably be as easy.
This task requires the output to have the same dimensions as the input with per pixel classification (For other masks it could be regression as well).
A modern CNN architecture to tackle such task is the U-Net.
Utilizing skip connections allows is having a simple architecture while being very efficient in learning.
I created a function to build such net in MATLAB:
We basically have 2 nets with one begin encoder like and the other being decoder. The skip connections between the encoder and the decoder are the special sauce of this kind of nets.
In my net the merging of data from the skip connections is done by concatenation. One could chose other options as well (Add, Multiply, etc...).
To show how easy is to build and train such net I chose the Oxford Pets Data Set:
This simple dataset has enough samples with good masks.
We try to classify each pixel as either the pet, the background or the border between the two.
I resized all images into 128x128 (One could also employ data augmentation which I skipped to keep things simple).
I ran the net for 15 Epochs:
As can be seen, It is easy to get almost 90% accuracy.
While it is nice, it is not a good metric for this kind of task for many reasons. One of them is that the data isn't well balanced so while it is easy to get the background pixels and most of the pet pixels, the border, which is the important part, might be badly missed.
Here is the result on one of the images of the set at the end of training:
We can see improvement over time but still far from very good. Data augmentation, label weight balancing, more epochs and more intelligent Learning Rate policy would get much better results.
The code is available at my StackExchange Codes Signal Processing Q79314 GitHub Repository (Look at the SignalProcessing\Q79314 folder).
