I am reading the paper Faster R-CNN available here. In section 3.2, page 6 bottom left, the authors describe what they refer to as a 4-step Alternating Training between the Region Proposal Network, used to obtain region proposals used in the Fast R-CNN method for object detection. They write:
In this paper, we adopt a pragmatic 4-step training algorithm to learn shared features via alternating optimization. In the first step, we train the RPN as described in Section 3.1.3. This network is initialized with an ImageNet-pre-trained model and fine-tuned end-to-end for the region proposal task. In the second step, we train a separate detection network by Fast R-CNN using the proposals generated by the step-1 RPN. This detection network is also initialized by the ImageNet-pre-trained model. At this point the two networks do not share convolutional layers. In the third step, we use the detector network to initialize RPN training, but we fix the shared convolutional layers and only fine-tune the layers unique to RPN. Now the two networks share convolutional layers. Finally, keeping the shared convolutional layers fixed, we fine-tune the unique layers of Fast R-CNN. As such, both networks share the same convolutional layers and form a unified network.
What I do not understand is the third step. They say they use the detector network to initalize RPN training. I am not sure what this means. The detector network outputs bounding boxes and class labels for objects within a given image.
Consider the layers specific to RPN. We have a certain number of convolutional layers, followed by sliding windows with anchor boxes being fed to regression and classification (class label layers). Are the authors saying that the initially trained convolutional layers are now discarded for RPN and replaced with those trained by Fast R-CNN in step 2? Or do they mean something else.