Detection of Parking Slots Based on Mask R-CNN

Round 1

Reviewer 1 Report

In this paper, the authors propose a parking-slot-marking detection system based on deep learning and image processing techniques. The approach consists of the following steps:

• Generation of Around-View Monitoring (AVM) images from system composed by four fisheye cameras.
• Prediction of masks of the marking-points by using a Mask R-CNN network, pretrained on Coco dataset and using a ResNet101 as backbone;
• Extraction of parking guidelines and parallel lines from the previous masks, by using a line segment detection (LSD) to determine the candidate parking-slots.

Their method is compared with other state-of-the-art approaches on a public dataset and a newly collected dataset. Their approach also makes possible to detect different types of parking slots: vertical, parallel and slanted slots.

Coming to the evaluation of the paper, the approach is very interesting and the results seem good and promising. 

The most serious problems are the following. Firstly, English, which makes difficult to understand some sentences of the paper and can confuse the reader. The paper definitely needs a proof-reading phase. The second one concerns the formatting of the whole paper. There are several errors in the sentences, including punctuation errors, uppercase and lowercase formatting. There are formatting problems in the Sections’ titles. There are also errors in some technical terms related to deep learning. I would also suggest to remove the word “Segmentation” from the title of the paper, because there is no real segmentation of parking slots, but only the markers are segmented. For all these reasons I suggest to Accept the paper with Major revisions, that I will describe on the following section:

1. Introduction

• The contribution section must be totally rewritten: rewrite it as if it was a list of contributions.

2. Research Status

• Authors should format the use of references. Sometimes only the author's name is mentioned and sometimes "Name et al." is mentioned (for example, Wang et al [30]; Zhang [35]). Authors should use the correct method described in the Applied Science guidelines.

4．Method for detecting parking-slots based on Mask R-CNN

• “The orientation of marking-point is divided into two categories, R belongs (-π/2, π/2], L belongs (π/2, -π/2] as shown in Figure 10. This step is helpful for subsequent determination of the orientation of the parallel lines.” In this section of the paper it is not clear how this phase will be carried out. Only at the end the reader will understand that this phase is performed automatically after using the LSD algorithm. I think it is better to add a preview in this section.
• “Labelme [45] is employed for sample marking, Figure 11 shows six typical training samples.” The paper could be improved by adding more informations about this method. The caption of Figure 11 should also be improved becuase the masks (c,f) are not explained very well.
• I think the Mask RCNN is explained too much in detail, without actually changing the architecture. Instead it would be better to give more emphasis to the changes made than state-of-the-art methods.

5. Experimental Result

• “we use the migration learning method…” I think this term is a bad translation of "transfer learning."
• “As to the distribution of training, validation, and test, we refer to other common datasets, by the ratio of 6: 1: 3.”  The paper could be improved by adding references of these common datasets.
• “Mask R-CNN was compared with several classical methods of target detection, To save human  resources and maintain the consistency of training samples.”. No-sense sentence.
• “This paper writes Python code..:” No-sense sentence.
• Figure 17 shows two types of evaluation metrics: mAP and detection speed. I suggest to study these metrics in two different figures. Comparing them together can be confusing, as they are metrics that are not connected to each other.
• Equations 5 and 6 are wrong: there are the “ture positives” word on both formulas.
• “the test database” The correct term is “dataset”.
• The public dataset[49] is used for the experiments but it was never explained. The paper could be improved by adding more informations about these dataset.

TYPOS:

• 1. Introduction: “then Using the LSD algorithm [21] to detect the mask…”
• 2. Research Status: “Forinstance, Jae-kyu et al”
• 2. Research Status: “and in terms of the ability to extract image features. these approaches extracted”
• 2. Research Status: “Mask-RCNN to detect parking slots under daynamic environments”

Author Response

please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

The paper presents a parking-slot-marking detection approach based on deep learning. The procedure consists of:
- Generation of a mask of the marking-points by using the Mask R-CNN algorithm
- Extracting parking guidelines and parallel lines on the mask using the line segment detection (LSD)
- Determining the candidate parking-slots.
The method works well under the condition of complex illumination and around-view images from different sources, with a precision of 94.5% and a recall of 92.7%, for different slots types, including vertical, parallel, and slanted slots.

Comments:
1. Address the advantages and shortcomings of the methods for free-spaces detection based on the short-range radars, lidars, and ultrasonic radars.
2. I found several typos with coma before the full stop “,.” Correct “Section6“, „Forinstance”, “. these approaches”, missing space before 2(“, the sentence ending with “,”. Capitalize the title of Section 2.2. Namely, there is a lot of typos of this type in the paper, and this needs to be thoroughly revised.
3. Introduction lacks an overview of the Around-View Monitoring (AVM) methods.
4. Provide the references for the low-level vision algorithms based on the Fast detector, Harris detector, Hough transform, and Ransac transform in the Introduction section (letter referred to in Section 2.1).
5. Provide clear information in Section 3 on how many fisheye cameras the proposed method uses.
6. Ln 135 states that „the corresponding feature points are selected manually in the undistorted image”. Is there a way to automatize this? This may be a significant shortcoming of the proposed pipeline of methods. Was it used just for calibration, or this is a necessary step each time method is used?
7. Define all abbreviations when first used, e.g., FPN, FCN, RCNN etc.
8. How is the preprocessing of images for ROI detection done?
9. Provide more details for Labelme used for sample marking.
10. Provide also results for the accuracy and F1 score in Table 1. Also, space is missing in Table caption.
11. Something is missing in the sentence “largely because If contrast with the ground”.
12. Rewrite the section ln 321-331. It looks like something is missing.
13. Discuss the reasons for M2 providing lower Recall than M1 and M3.
14. I find a discussion of the obtained results to be rather brief. Also, extend the comparison of the proposed approaches to competitive methods found in ref. 39, 38, 25, and 29.
15. Provide details on the precision and recall in the paper abstract and conclusion.
16. Also, refer to the method’s shortcomings and directions for its improvement in paper Conclusion.

Author Response

please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The paper was strongly improved and is fine for publication.