3.2. Analysis
Both SIFT and SIFT-OCT are used to extract CPs in the master image (‘C’) and slave image (‘G’). However, the spatial distributions of the points from the SIFT and SIFT-OCT are quite similar, and presenting both results is not necessary. To clarify the proposed matching scheme is promising. The results of the original SIFT, which has been proven unsuitable for SAR, are presented and compared with the one-step scheme. The results of the different algorithms for the traditional one-step matching and proposed two-step matching are presented in
Table 1. It can be seen from
Table 1 that even for the original SIFT algorithm, which has been proven unsuitable for SAR image registration, the registration accuracy was substantially improved after the two-step matching method proposed in this paper, which is not much different from the registration accuracies of the other algorithms specifically proposed for SAR image registration. Thus, without specific instruction, only the results of the SIFT are presented in the following figures. The extracted keypoints of the SIFT are shown in
Figure 4. In the master image, 25,096 and 7303 keypoints from the SIFT and SIFT-OCT, respectively, were extracted. In the slave image, 21,294 and 6589 keypoints from the SIFT and SIFT-OCT, respectively, were extracted. As the SIFT-OCT skipped the first octave of a scale-space pyramid, the number of keypoints from the SIFT-OCT is much smaller than that of the SIFT. However, the keypoints from the SIFT-OCT are more robust to multiplicative speckle noise than those of the SIFT.
The value of the threshold (
) may affect the final registration results; thus, this paper examines the effect of the
on the matching results. The relationship between the
and the accuracy of the matching points is shown in
Figure 5a, and the relationship between the
and false-alarm rate is shown in
Figure 5b. With the increase in the
, the accuracy of the matching points of the matching rate of vegetation decreased. To achieve the matching accuracy of both buildings and vegetation, the value of the
was selected as 0.7. In the global dual-matching procedure, the threshold (
) was set to 0.7, and 218 and 204 pairs of CPs from the SIFT and SIFT-OCT, respectively, were selected. The SIFT results are shown in
Figure 6a,b with the blue and red points, respectively. The global dual-matching cost is 273.35 s for the SIFT and 28.82 s for the SIFT-OCT. After the RANSAC step, only 21 CPs for the SIFT and 26 CPs for the SIFT-OCT were selected. The yellow circled points in
Figure 6a,b are the selected global CPs of the SIFT. They cost 6.59 s and 3.71 s, respectively. From
Figure 6a,b, we can see that some of the global dual-matched CPs were mismatched. The mismatched CPs were filtered by the RANSAC procedure. However, some correct CPs were also filtered. The global affine transformation parameters were estimated by the selected 21 or 26 CPs. By transforming the keypoint location of the slave image into the master image coordinate system, the location information can be used to confine the searching scope. In the local-matching procedure, CPs of the other image within a radius of 100 pixels are searched, and the threshold (
) of the local dual match is also set to 0.7. After the local-matching procedure, 437 and 457 CPs for the SIFT and SIFT-OCT were selected, respectively (see the yellow crosses in
Figure 6c,d). Compared with the global-matching processing times (273.35 s and 28.82 s, respectively), the local-matching processing times were only 74.35 s and 5.15 s, respectively. As many similar CPs outside of the search area are already eliminated, the local-matched CPs are more robust than the global dual-matched CPs. Details of
Figure 6c,d can be seen in
Figure 6e–h. An interesting phenomenon appears in both the SIFT and SIFT-OCT methods: all the points selected by global matching after the RANSAC (21 for the SIFT and 26 for the SIFT-OCT) are also selected by local matching. The local matching also kept many other high-quality CPs, which could not be matched by global matching. The larger number of CPs (i.e., 437 − 21 = 416 and 457 − 26 = 431 for the SIFT and SIFT-OCT, respectively), which could solve the problem of complex distortion, is a significant improvement compared with one-step matching.
The normal method to evaluate the quality of matched CP pairs is to use match error, (i.e., root mean square error). However, the correct tie point, which cannot be determined in VHR images, even by manual selection, must be known. Thus, other methods, such as similarity measurements, are alternatively used to perform the evaluation. The high similarity of the extractor features are ensured in the matching scheme. Thus, some other similarity measurement functions are used to evaluate the performance of the CPs: the alignment metric (AM) [
59], invariant moment (IM) [
60], matching correlation surface (MCS) [
61], and mutual information (MI) [
62]. For these measurements, higher values correspond to the higher similarity of two image patches. For each CP, a pair of self-centered image patches are selected to calculate the similarities, and in these experiments, the patch size is 31 × 31. The cumulative distribution functions (CDFs) of these four similarities from both the SIFT and SIFT-OCT are shown in
Figure 7, and some of their corresponding statistical parameters, including maximums, minimums, and means, are presented in
Table 2. Even though the numbers of CPs of the different methods are not the same, CDF curves can show the similarity distribution of extracted CPs, which can be used to evaluate their overall quality from one method. In
Figure 7, we can see that the curves of the local matching from the SIFT and SIFT-OCT are quite close, which reflect that the local matching of both the SIFT and SIFT-OCT had similar performances. In
Figure 7a,d, the local curves of the AM and MI are almost below the global curves, which reflect the better similarity of the local-matched CPs. In
Figure 7c, two local curves are in the middle of two global curves, which means that the local-matched CPs have moderate similarity measured by the MCS. The local curves in
Figure 7b are slightly above both global curves, which should mean that the global-matched CPs have better similarities measured by the IM. However, in
Figure 7b, all four of these curves are very steep in small values, which means that the differences in these four curves are very small. We can conclude that the similarity of the local-matched CPs is not lower than that of the global-matched CPs. Thus, compared with traditional one-step matching, the proposed two-step matching increases the number of CPs while improving the matching accuracy.
To show that the simple model cannot describe complex geometric distortion, the SIFT algorithm locally matched the 437 control points, and the least-squares method was used to estimate the parameters of multiple global transformation models. The probability accumulation function (CDF) of the root square error (RSE) of the control point after registration was calculated. The experimental results are shown in
Figure 8. It can be seen from the figure that the best registration accuracy can be achieved by using the third-order polynomial; however, the mean square errors of most control points are still very high. Despite the large number of control points used in the experiment, the global-distortion model still did not achieve high registration accuracy.
Due to the inaccessible point-to-point correspondence between the two images in the real case, two methods were used to evaluate the registration accuracy in the experiments presented in this paper. One uses the pseudocolor method to synthesize the main image and registered auxiliary image into pseudocolor to allow for direct analysis with the human eye. The other calculates the correlation coefficient of each point in the image; the higher the absolute correlation coefficient, the higher the registration accuracy. The comparative results of this experiment are shown in
Figure 9 and
Figure 10.
Figure 9a and
Figure 10a show the evaluation results of the original data. It can be seen that the main images and auxiliary images are completely unaligned, there are many overlapping images, and the correlation coefficient is small.
Figure 9b,c and
Figure 10b,c are the coarse registration results. It can be seen that the two images basically match; however, there are still some overlapping images in some strong scattering regions, and the overall correlation coefficient is not high. Because both used global affine transformation as the distortion model, although the number of control points in the two groups varies greatly, the results are not different. In the case of the distortion model, the accuracy is not high, even if a large number of control points are used, and the overall registration accuracy cannot be improved. The registration results of the model parameter estimation using triangular net, spline interpolation, and local affine transformations as models, and 437 control points obtained using SIFT, are shown in
Figure 9d–f and
Figure 10d–f. It appears from the figure that the registration results obtained by using the local-distortion model are better than those obtained with the global-distortion model. In the yellow-box areas, the local affine model and spline value are better than the affine transformation. Because no control points are found in the image boundary region, the triangle net affine transformation results are poor in this region. The local affine transformation model alleviates this problem by increasing the number of control points slightly farther away. In the areas marked by the green boxes, all the methods have poor results, which is mainly because the area is mountainous with a large number of trees, and the SAR images, taken from different angles, are different without a substantial difference in the large number of control points obtained. A quantitative analysis of the results is presented in
Table 3.
To compare the robustness of the three local distortion models to the mismatching points, three control points were added for the pixel offset of the mismatching in the image. The local results after matching are shown in
Figure 11. From the results, we can see that the local affine model has a higher stability relative to the triangular net affine transformation and spline interpolation. The reason is that both spline interpolation and the triangle network transform the coordinates of all the control points to consistency, and if mismatched, the control points can distort the image of that region. However, the local affine transformation model uses more control points to estimate the transformation parameters, and it has a certain tolerance to the matching error. A quantitative analysis of the results is presented in
Table 4.
3.3. Registration Result
Using TPS [
30] as a model, the experiment images were registered by cubic interpolation. The parameters of the model were estimated by CPs extracted from our proposed two-step matching scheme. The registration result of the experiment dataset is shown in
Figure 12. The figure only retained the overlapping areas by putting the registered master image in the red channel and the slave image in the cyan channel. If the two images are exactly the same, then the overlapped synthetic image should be a grayscale image. Moreover, if two overlapped images are displaced, the synthetic image should have clear red and cyan shadow pairs. Thus, this synthetic image is suitable for the evaluation of the registration result. In
Figure 12, the synthetic image has almost no shadow pairs, which means that the two images are precisely registered. In addition, the grayscale images are not identical, which indicates that the intensities of the two image areas are quite different in various locations. The grayscale levels of the different range gates from the same image are not identical because the antenna pattern weight was not entirely corrected (which caused the top of
Figure 12), has some stronger red component, and the bottom has stronger cyan component. The experiment demonstrates that, with a larger quantity of good-quality CPs from SAR images suffering from geometrical deformation and intensity differences, the proposed method can obtain high-precision registration results.
In order to verify the registration performance of the proposed algorithm, two excellent methods in SAR image registration were selected as comparative experimental methods. SAR-SIFT [
24] is the method that has been used to achieve good SAR image registration performances in recent years. It is proposed in the framework of SIFT, and it adopts the mean proportion operator to overcome the spot noise to extract multiscale stable angle points. However, BFSIFT [
22] extracts local extreme points in the nonlinear scale space for SAR image registration. The experimental data were registered by different registration methods to verify the superiority of the proposed registration method. The first set of data is the airborne SAR images from the DLR Microwave Radar Research Institute. Images were taken near a small city, in southern Bavaria, Germany, and they contain different objects, such as forests, farmland, water bodies, and houses. Data were recorded from two images at different times, in different bands, and in different polarization situations, and the resulting images are significantly different, as shown in
Figure 13.
Figure 13a is the reference image, and
Figure 13b is the image to be registered. The reference image and registered image of the second set of data were taken in June 2008 and July 2008, respectively, in Iowa, the United States. The reference map and registered map are shown in
Figure 13c,d, respectively.
The matching points obtained for two sets of experimental data using BFSIFT, SAR-SIFT, and the method presented here are shown in
Figure 14. The matching point pairs obtained with the BFSIFT and SAR-SIFT methods are shown in
Figure 14a–d, respectively. The red circle is the matching point in the reference image, the green cross is the matching point in the graph to be registered, and the yellow line connects the reference map with the matching point pair in the graph to be registered.
Figure 14e,f shows the matching point pairs obtained by the proposed method. The yellow line in
Figure 14f is the line with more feature points than
Figure 14b,d, and the red line is the line with the same number of feature points as
Figure 14b,d. It can be seen from the figure that the method in this paper can identify more pairs of matching points, making the distribution of matching points more uniform.
Figure 15 and
Figure 16 show the resulting pseudocolor plots of the registration to the two sets of experimental data using BFSIFT, SAR-SIFT, and the method proposed here. In the pseudocolor diagram, green indicates the reference image, and purple represents the image to be registered. The two images are overlapped so that the registration effect of each method can be observed more clearly. The red rectangle box in the left image is the position of the right zoom-in in the original image. The SAR image registration obtained by the BFSIFT method and SAR-SIFT method did not accurately align the image to be registered to the reference image. The two overlapping images on the pseudocolor map have an obvious alignment error. The registration results of the proposed method have the best registration effect, and the reference image overlaps the image to be registered well.
As can be seen from
Table 5, in two sets of experiments, BFSIFT and SAR-SIFT can only extract a single feature for registration, the number of matching feature points is small, and the insufficient number of feature points are not effectively evenly distributed in the image, which affects the accuracy of the final registration results. The matching method proposed here adopts a two-step matching strategy under a SIFT-like SAR image registration scheme to obtain more control points and thus obtain accurate registration results. It also can be seen from the registration result graph and registration data table that the registration algorithm proposed in this paper is better than the BFSIFT and SAR-SIFT methods under equivalent conditions.
Through the above experiment, we can see that the two-step matching strategy proposed in this paper can obtain a large number of control points and, at the same time, greatly improve the image matching accuracy; however, the increase in the number of CPs must lead to the subsequent feature calculation and matching calculation. Therefore, in the two-step-matching strategy framework, the RANSAC [
34] method is added to remove false matching, eliminate some of the wrong matching points, and improve the calculation rate. To verify that the proposed algorithm does not increase the computation too much while improving the matching accuracy, and that is has a good registration performance, two sets of images were selected for comparison experiments. The first set of data is the airborne SAR image of a certain area of Serbia. In the image are a plain area and river. There is a large translation transformation and a certain amount of rotation transformation and scale transformation between the two images, and the overlap between them is small. The reference image and image to be registered are shown in
Figure 17a,b, respectively. The second set of data is the airborne SAR image of a certain area in Germany, which contains substantial farmland and a lake in the plain area. The overlap between the two images is relatively large. The reference image and image to be registered are shown in
Figure 17c,d, respectively.
The matching points obtained for two sets of experimental data using SIFT, SIFT-OCT, SAR-SIFT, and the method presented here are shown in
Figure 18. In
Table 6, the SIFT algorithm, SIFT-OCT algorithm, SAR-SIFT algorithm, and proposed algorithm were applied for two sets of images to compare the matching effects of the SAR image pairs in the experimental data, and the evaluation indexes of the matching effects were calculated.
Judging from the results of the matching index calculation, the proposed two-step matching algorithm increases the number of correct matching points. Because it extracts a large number of feature points for matching and introduces the RANSAC method to remove false matching, it also saves the running time of matching under the premise of ensuring the correct matching rate. Using the classic SIFT algorithm to match the SAR images, a large number of wrong matching pairs appeared, which finally caused registration failure. When the image has fewer overlapping areas and matching points, the registration accuracy of the SIFT-OCT algorithm is higher than that of the SAR-SIFT algorithm. This is because the SAR-SIFT algorithm retains a large number of wrong matching pairs, which results in a decrease in the matching accuracy. When the overlap of the registration image is large, the matching accuracy of the SAR-SIFT algorithm is greatly improved. The two-step matching algorithm proposed in this paper has higher matching accuracy than those of several other algorithms in these two sets of SAR image matching, and there is little difference in the matching time. The matching results of the two sets of images are shown in
Figure 19.
Next, we tested the robustness of the registration method proposed here to high-speckle noise. For the test purpose, spot noise with different noise variances was manually added to a set of multitemporal-phase-measured SAR images, and the noise variance was set to 0, 0.1, 0.2, 0.3, and 0.4. The test images used were acquired from the C-band images by the US AIRSAR system. Four SAR image registration methods were used: the SIFT-OCT method, BFSIFT method, NDSS-SIFT method, and proposed registration method, for the registration of the above four sets with different noise levels of SAR images. The results of the obtained registration evaluation are shown in
Figure 20. According to the performance evaluation results, we can see that the SIFT-OCT method registration failed when the noise variance was greater than 0.3, but the proposed registration method still maintained a good registration performance. It can be seen from the values of the RMSE and RMS
LOO that the registration accuracy proposed here is much higher than those of the SIFT-OCT method, BFSIFT method, and NDSS-SIFT method.
Figure 21 shows the registration results of the image when the noise variance was equal to 0.4. The obtained registration results basically match the edges and textures in the reference images, which also indicates that the registration method in this chapter is robust to spot noise.