Luojia-1 Nightlight Image Registration Based on Sparse Lights

Abstract

When mosaicking adjacent nightlight images of a large area that lacks human activities, traditional registration methods have difficulty realizing the tie point registrations due to the lack of structural information. In order to address this issue, this study devises an easy-to-implement engineering solution that allows for the registration of sparse light areas with high efficiency while guaranteeing accuracy in non-sparse light areas. The proposed method first extracts the sparsely distributed light point positions through use of roundness detection and the centroid method. Then, geometric positioning forward and backward algorithms and the random consistency sampling detection algorithm (RANSAC) are used in order to achieve a rough registration of the nightlight images and the remaining tie points are expanded through the affine model. Through experimentation it was found that, compared with traditional registration methods, the proposed method is more reliable and has a wider distribution in sparse light areas. Finally, through the registration test of 275 scenes of nightlight images of China from Luojia-1, the coverage ratio of the tie points was increased from 59.3% from the traditional method to 95.3% in this study and the accuracy of the block adjustment was 0.63 pixels, which verifies the effectiveness of the method. The proposed method provides a basis for the registration, block adjustment, and mosaicking of nightlight images.

1. Introduction

Nightlight remote sensing images provide evaluations of human activities [1], urban changes [2], socioeconomics [3], and even the impact of war [4]. Researchers typically select a regional or national area to evaluate the above indicators through various nightlight images, such as those that are acquired from the 1 km resolution operational line-scan system (OLS) data of the defense meteorological satellite program (DMSP) or the 750 m resolution’s day/night band (DNB) data that are collected by the NASA/NOAA Visible Infrared Imaging Radiometer Suite (VIIRS) [5]. With the launch of Luojia-1, the resolution of nightlight images has greatly improved to 129 m, allowing for more refined indicator evaluations [6,7,8,9]. There are also some high-resolution nighttime images that were acquired by the International Space Station (ISS), EROS-B, and JL1-3B [10] and more satellites (and CubeSats/nanosats) that will engage in nighttime Earth observation in the future. However, the increase in resolution has resulted in reduced image swath widths and the evaluation of a region or country now requires the mosaicking of multiple nightlight images. Hence, the registration problem of the tie points of these nightlight images has become prominent.

The pixels of nightlight images have high values in lighted areas, yet their values are almost zero in non-lighted ones (regardless of how richly textured they are during the day). Further, when influenced by background noise, some pixels appear to be lights. These issues limit many traditional registration methods. For areas lacking human activities, the lack of this structural information (which is composed of large areas of urban lights and road lights) creates difficulty for traditional registration methods to be able to register adjacent images in order to obtain the tie points of nightlight images.

Currently, the most commonly used registration methods in the engineering application field include grayscale-based registration and feature-based registration [11]. Grayscale-based registration (also called template registration) mainly compares the similarity of the grayscale of two images and it has a high degree of dependence on this grayscale information. Direct grayscale registration is very sensitive to lighting changes such as those which arise from changes to light angles and shadows; thus, grayscale normalized cross-correlation (NCC) is generally used as the registration criterion in order to enhance the robustness of this method under grayscale changes [12]. For nightlight images such as those that are produced by Luojia-1, grayscale registration usually fails because of the noise and changes in the brightness of the images that are captured in different periods. In addition, grayscale registration requires textural information. For places with few lights, the images mainly have a black background, which has a greater impact on the selection of grayscale templates and the registration’s accuracy.

Feature-based registration extracts feature descriptors in the affine invariant region, calculates their feature vectors, and then calculates the Euclidean distance between the feature vectors in order to determine whether two feature points are homonymous. The representative feature descriptors are scale-invariant feature transform (SIFT), SURF, KAZE, ORB, and BRISK [13]. Among these, feature registration that is based on the SIFT operator is the most representative. The SIFT operator has brightness, scale, and rotation invariance and is, therefore, not sensitive to the light intensity of the nightlight image and it can also be stable to noise [14,15,16]. SURF is based on SIFT and is reputed to be faster. In actual registration, more tie points can be obtained for nightlight images in areas with more texture, such as cities. However, for villages and towns in Western China with fewer inhabitants, as well as other sparsely illuminated areas, it is difficult to extract them as feature points in the feature point extraction stage because these areas appear as bright spots occupying only a few pixels in the image. Therefore, feature-based registration cannot be applied to all areas of nightlight remote sensing images.

The Luojia-1 nightlight remote sensing satellite has a sub-satellite point resolution of 129 m and the absolute geolocation accuracy of its images is approximately 650 m (1σ) [6]. This accuracy is sufficient for DMSP/OLS and VIIR/DNB data, which means that the dislocation between adjacent images is less than one pixel. However, it has a significant dislocation of five pixels when mosaicking images that are from Luojia-1. To further improve the positioning accuracy of nightlight images in China and finally realize the creation of complete nightlight images of China through image mosaicking, it is necessary to make block adjustments for all of the images in the region. The acquisition of the tie points between adjacent images during block adjustment is particularly critical.

Considering that the sparse light spots of villages and towns in nightlight images are usually isolated, their presence in the image is similar to that of a star point. This study devises an easy-to-implement engineering solution that allows for the registration of sparse light areas with high efficiency while guaranteeing accuracy in non-sparse light areas. Based on the idea of star point extraction [17], the coordinates of the sparse light points can be obtained through the use of a roundness threshold and the centroid method. The method can then take advantage of the existing geometric information of the image and use the geometric positioning forward and backward algorithms and the random consistency detection algorithm (RANSAC) in order to achieve the preliminary registration of the tie points [18]. Finally, through analysis, the affine model is used to realize the deletion and expansion of the tie points.

2. Materials and Methods

This section first introduces the method that was used to extract sparse lights from nightlight images [19], after which the registration method of Luojia-1 nightlight images is analyzed from the perspective of the geometric positioning model and the image transformation model. Finally, the specific process of the sparse light registration algorithm is elaborated upon.

Table 1. Luojia-1 satellite parameters.

Parameter	Value/Description
Orbit height	645 km
Orbit type	Sun-synchronous
Revisit period	15 days
Ground sampling distance	129 m (sub-satellite point)
Ground swath	264 km × 264 km
Camera focal length	55 mm
Camera pixel size	2048 × 2048
Camera detector size	11 μm × 11 μm
Camera field of view (FOV)	32.3°
Geolocation accuracy	650 m/5 pixels
Attitude accuracy	0.05°

shows the parameters of the Luojia-1 nightlight remote sensing satellite that was used to provide the data for the following analysis.

2.1. Sparse Light Extraction Method

2.1.1. Connected Domain Segmentation

The nightlight images of Luojia-1 are single-channel optical images. The digital number (DN) value of the image is theoretically zero in areas with no human activity. Generally, the greater the DN value, the greater the intensity of human activities. Therefore, the human activity area in the picture can be segmented from a black background using background threshold segmentation. Assuming that the gray value of the image containing the light points is expressed as $g\left(x,y\right)$ and $t$ is the background threshold, the image binarization is as follows:

$$f\left(x,y\right)=\{\begin{array}{c}1, g\left(x,y\right)\ge t\\ 0, g\left(x,y\right)<t\end{array}$$

(1)

which means that if the pixel value is larger than the background threshold $t$, the pixel value is set to one, otherwise it is set to zero. Each pixel in $f\left(x,y\right)$ has a value of one and those which are adjacent to one another are marked as one connected domain. In this manner, $k$ groups of connected domains ${\mathsf{\Omega }}_i, i=\left\{1,2,\dots k\right\}$ are obtained. By the process of construction there is no intersection between each connected domain:

$$\forall {\mathsf{\Omega }}_{i\in C}{{\displaystyle \cap }}^{ }\forall {\mathsf{\Omega }}_{j\in C}=0, i\ne j,C=\left\{1,2,\dots ,k\right\}$$

(2)

2.1.2. Roundness Detection

In order to improve the accuracy of the coordinate extraction of sparse lights, this research filtered the registration points by calculating the roundness of the sparse lights and selecting the lights with higher roundness. The calculation equation for the roundness of the connected domain is as follows:

$$P=\frac{4\pi S}{L^2}$$

(3)

where $P$ represents the roundness, $S$ represents the area of the connected domain, and $L$ represents the perimeter of the connected domain. From the relationship between the area of the circle and the perimeter, the closer $P$ is to one, the rounder the sparse light.

In the process of the selection of light points that are to be registered, the area $S\left({\mathsf{\Omega }}_i\right)$ is obtained by calculating the number of pixels of the connected domain and the perimeter $L\left({\mathsf{\Omega }}_i\right)$ by calculating the number of pixels in the outer layer of the connected domain. Then, the appropriate connected domains are selected according to the area and roundness. The light points, after filtering, are expressed as follows:

$$\mathsf{\Omega }={\mathsf{\Omega }}_i if \left(S_{min}<S\left({\mathsf{\Omega }}_i\right)\langle S_{max} and P\left({\mathsf{\Omega }}_i\right)>e\right)$$

(4)

where $S_{min}$ and $S_{max}$ represent the minimum and maximum numbers of pixels; $e$ represents the threshold of the roundness; and the value range is (0, 1). In the specific selection strategy of sparse light points, for areas with few urban buildings, $e$ is appropriately reduced in order to extract more points; whereas, in areas with more urban buildings, $e$ is increased in order to improve the position accuracy of the light point extraction.

2.1.3. Centroid Extraction

When the light points are filtered out, the centroid can be used to extract the center coordinates of the light points and the gray square weighted centroid method can be used to obtain the centroid sub-pixel coordinates $\left(\overline{x},\overline{y}\right)$ of the light points. The calculation equation is as follows:

$$\{\begin{array}{c}\overline{x}=\frac{{\sum }_{\left(x,y\right)\in \mathsf{\Omega }}x·g{\left(x,y\right)}^2}{{\sum }_{\left(x,y\right)\in \mathsf{\Omega }}g{\left(x,y\right)}^2}\\ \\ \overline{y}=\frac{{\sum }_{\left(x,y\right)\in \mathsf{\Omega }}y·g{\left(x,y\right)}^2}{{\sum }_{\left(x,y\right)\in \mathsf{\Omega }}g{\left(x,y\right)}^2}\end{array}$$

(5)

It can be seen from the above equation that the higher the roundness of the light point is, the higher the accuracy of the calculation of the center point will be.

2.2. Sparse Light Registration Method

The following are the methods for automatically detecting and selecting the homonymous light point:

2.2.1. Geometric Positioning Model Forward and Backward Algorithms

The nightlight image that is used for registration is usually a sensor calibration product [20]. Owing to the different satellite positions and attitudes at the time of imaging, there are errors in translation, scale, and rotation between the adjacent images that need to be registered. If there is already a rigorous geometric model within the image, the geometric positioning forward and backward algorithms can be used to preliminarily solve the problem [21]. Generally, the process of obtaining the object point coordinates according to the image point coordinates is called the forward algorithm and the process of obtaining the image point coordinates according to the object point coordinates is called the backward algorithm [22]. As Figure 1 shows, assuming that there are two images to be registered, named left and right, the forward algorithm, combined with the global digital elevation model (DEM) data, is used to project the left image point $p$ to the ground and obtain the $P$ coordinates. Then, the backward algorithm is used to retro-project $P$ to the right image and get the $p$′ coordinates.

According to satellite imaging parameters, the classic rigorous geometric model of optical remote sensing satellites can be constructed as follows [23,24]:

$$\left[\begin{array}{c}X\\ Y\\ Z\end{array}\right]=\left[\begin{array}{c}{X}_S\\ X_S\\ X_S\end{array}\right]+m·R·\left[\begin{array}{c}x-x_0\\ y-y_0\\ f\end{array}\right]$$

(6)

where

$$R=R\left(\phi \right)R\left(\omega \right)R\left(\kappa \right)=\left[\begin{array}{ccc}{a}_1& a_2& a_3\\ b_1& b_2& b_3\\ c_1& c_2& c_3\end{array}\right]$$

(7)

In the above equation, ${\left[\begin{array}{ccc}X& Y& Z\end{array}\right]}^T$ represents the coordinates of the ground point in the World Geodetic System 1984 (WGS84) coordinate system, ${\left[\begin{array}{ccc}{X}_s& Y_s& Z_s\end{array}\right]}^T$ represents the satellite’s orbital position in the WGS84 coordinate system, $m$ represents the scale factor, and $R$ represents the satellite’s attitude of the body coordinate system relative to the WGS84 coordinate system, where $R$ can be expressed as a combination of the satellite’s pitch angle $\phi$, roll angle $\omega$, and yaw angle $\kappa$. The variable $\left(x,y\right)$ represents the coordinates of the corresponding image point in the image coordinate system, $(x_0,y_0)$represents the coordinates of the principal point of the camera, and $f$ represents the focal length of the camera. For the geometric forward algorithm, according to the orbit and attitude interpolation model, the orbit and attitude parameters at the imaging time can be obtained [25]. Combining the known camera principal point and the principal distance parameters, as well as the Earth’s ellipsoid parameters and global DEM parameters, the object coordinates corresponding to any point on the image can be calculated according to the following ellipsoid equation [26]:

$$\frac{X^2+Y^2}{{\left(ma+h\right)}^2}+\frac{Z^2}{{\left(mi+h\right)}^2}=1$$

(8)

where $ma$ and $mi$ are the semi-major axis and semi-minor axis parameters of the Earth ellipsoid in the WGS84 coordinate system and $h$ represents the elevation of the object point. The solution of the object point coordinates is an iterative process. The above process is the forward algorithm of the geometric positioning model. For the backward algorithm process, Equation (6) can be expressed as follows [27]:

$$\{\begin{array}{c}x-x_0=f\frac{a_1\left(X-X_S\right)+b_1\left(Y-Y_S\right)+c_1\left(Z-Z_S\right)}{a_3\left(X-X_S\right)+b_3\left(Y-Y_S\right)+c_3\left(Z-Z_S\right)}\\ y-y_0=f\frac{a_2\left(X-X_S\right)+b_2\left(Y-Y_S\right)+c_2\left(Z-Z_S\right)}{a_3\left(X-X_S\right)+b_3\left(Y-Y_S\right)+c_3\left(Z-Z_S\right)}\end{array}$$

(9)

That is, according to the object coordinates of the point, the orbit parameters at the imaging time, the attitude parameters, and the orientation elements in the camera, the corresponding image point coordinates can be obtained. All of the points in the left image are reprojected onto the right image through the use of the forward and backward algorithms. In this way, most of the translation, rotation, and scale errors between the homonymous points on the left and right images are eliminated.

2.2.2. Image Transformation Model Analysis

The parameters that are introduced into the image geometric positioning model have measurement errors and the DEM data have elevation errors. Therefore, after use of the forward and backward algorithms of the geometric positioning model, there is an offset between the homonymous points. If the transformation model between the homonymous points in the left and right images can be established, registration between the homonymous points can be realized. Because the orientation accuracy of the interior parameters of Luojia-1 is better than 0.3 pixels after geometric correction [6], the translation model can be used with a larger threshold in order to achieve rapid preliminary alignment between the homonymous points. In the subsequent process of deleting misregistered points, the overall constraints of the affine model can be used with a smaller threshold so as to eliminate the misregistered points. The parameter error of the satellite and the elevation error are discussed in Appendix A. This analysis indicates the process of the selection of the transformation model between the two sets of homonymous points.

2.2.3. Sparse Light Registration Algorithm

Combined with the selection of the image model transformation, the complete registration process is as shown in Figure 2. The specifics are as follows:

• Extract the sparse lights of which the roundness is less than a certain threshold on the left and right adjacent images. Judging by the number of points that are extracted, appropriately increase the roundness threshold if the image is of an urban area. Conversely, appropriately decreased it if the image is of a sparsely lit area.
• Use the forward algorithm to calculate the latitudes and longitudes of all of the sparse lights in the left image and then use the backward algorithm to calculate the position of these sparse lights in the right image. Set a certain threshold to delete the light points that exceed the range of the right image. Then, perform the geometric positioning forward and backward algorithms and delete the lights that exceed the range of the left image.
• If the number of remaining lights in the final left and right images is zero, reduce the roundness threshold $e$ and extract the lights again. If the number of light points is one or two, the corresponding registration point pair can be directly found through the geometric positioning model and positioning error threshold. Use the isolated point principle in order to determine whether the two points are uniquely paired. The principle of isolated points is as follows: taking any point on the left image, through the geometric positioning forward and backward algorithms only one point can be found within a certain distance threshold on the right image. Calculate this point on the left image. If only one point can be found within a certain distance threshold, then called this pair of points an isolated point.
• If the number of remaining light points in the left and right images is greater than three, use the RANSAC algorithm and translation model to achieve registration through voting [18,28], as follows.
  • Randomly select a point on the left image and project it onto the right image through the forward and backward algorithms. Calculate the distance between this point with the homonymous point on the right image. Count all of the distances and the votes for these distances. Based on the RANSAC algorithm, the distance with the most occurrences is the translation parameter of the right image relative to the left image.
  • If the translation parameters are unique, use the translation model to translate the light point of the left image onto the right image. Then find the light point of the right image within a larger threshold range. If the isolated point principle is satisfied, the right image’s point is a homonymous point. Because of the sparse distribution of light points, usually when the left and right images have a certain number of light points, the translation value is unique and takes the highest number of votes according the RANSAC algorithm.
  • If the translation value is not unique, then adopt the method that is described in Step 3.
• After judging all of the points, establish the affine transformation model of the registration point pair and calculate the residual error of each point. Starting with the largest residual, delete the larger error values in succession and recalculate the affine model transformation parameters until the residuals of all of the points meet a certain threshold.
• The isolated point principle will inevitably lead to the omission of certain light points. Based on the registered point pairs, establish an affine model again and calculate the position of the unregistered light point in the left and right images. If it is less than a certain threshold, the point can be expanded.
• Determine the number of registered homonymous points. When it is greater than one, the tie points of the two images have been registered successfully. However, when it is zero you must reduce the roundness threshold $e$ and start the process over from the first step. The registration has failed if the threshold $e$ is less than 0.1.

3. Experiment and Results

This experiment used Luojia-1 nightlight data from 275 scenes covering China. The imaging time range of the data was within the second half of 2018. The background noise in the nightlight image is generally one pixel and the connected domain that is composed of more than a certain number of pixels is usually irregular in shape. So, the minimum and maximum number of pixels were respectively set to $S_{min}=2\ast 2$ and $S_{max}=20\ast 20$. The initial threshold of the roundness was $e=0.3$. When the number of light points that could be extracted in the image was less than four, the $e$ threshold was lowered by 0.1 in order to continue the extraction process. The registration parameters were the same for all of the image settings and the tie points were automatically extracted and registered.

The following sections present the results and analyses of the reliability, distribution, comparison, and accuracy of the tie point registrations of the Luojia-1 nightlight images.

3.1. Reliability of Tie Points

Figure 3 shows the registration result of the tie points of the adjacent images of two scenes in Zhengzhou, Henan. A total of 71 sets of tie points were obtained from the registration process of the two images. The tie points’ numbers were sorted by their roundness and the top eight sets were selected with numbers A1–A8 and B1–B8 as shown. By comparing the positions of the centers of the light points, the registration accuracy was found to be very high; moreover, most of the automatically selected light points existed in the image, with a high roundness, and the position of the center point changed less in the different images.

Figure 4 shows the registration details of the first two pairs of tie points that were selected from the Zhengzhou registration points. The distribution of the light points was extracted according to the established roundness threshold.

3.1.1. Roundness Threshold Limiting Effect

For the light point that is numbered 340 in Figure 4a, the light information at the corresponding position in Figure 4b was weakened and the light point could not be extracted. Similarly, for the light point that is numbered 713 in Figure 4c, no light point was extracted at the corresponding position, as shown in Figure 4d. Therefore, for the registration of nightlight images, owing to the imaging angle, date, noise, etc., the shape of the light points of the left and right images changed. The actual light points that were extracted from the left image do not necessarily appear in the right image when the light points are extracted using the roundness threshold.

3.1.2. Isolated Point Principle Effect

After obtaining their translation parameters by using the RANSAC algorithm, point 1 of Figure 4a and point 799 of Figure 4b could be registered; however, point 514 in the upper-left corner of Figure 4a was relatively close to point 75. When determining the isolated point, it was impossible to determine which point corresponded to point 360 in Figure 4b. These three points were discarded in the initial registration stage in order to avoid affecting the calculation of the translation parameter. However, during subsequent expansion of the registration process, through the affine transformation parameters that were obtained from other registration points, it could be determined that point 514 in Figure 4a and point 360 in Figure 4b were homonymous points. Similarly, the isolation principle between point 2 of Figure 4c and point 366 of Figure 4d was not satisfied; however, after the point was finally expanded by the affine model, point 2 of Figure 4c and point 199 of Figure 4d were registered.

Based on the above situation, it can be seen that, for sparse lights with fewer pixels, the roundness and area size of the lights guarantee the extraction accuracy of the center position of the lights. In addition, the position error of the homonymous points that were extracted from the two images using the centroid method is small. Even if there is an error in the position, it is usually less than one pixel, owing to the small range of the light area itself.

3.2. Distribution of Tie Points

3.2.1. Distribution of Sparsely Populated Areas

As Figure 5 shows, the Tibet area of China is sparsely populated and the lights that are there are weak. Using the proposed method, the tie points that are shown in Figure 5 were automatically extracted and registered. According to points 1 through 6, it is evident that the homonymous points are small and not round, bright spots. Because there are very few lights in the western region, the program automatically reduces the roundness threshold when the light points are insufficient. Therefore, the extraction points are not all circular, but the center coordinate of the homonymous point is at the center of the light areas. For the slightly larger light areas numbered 7–9, they were not selected by the algorithm as tie points. Obviously, if these larger lights were selected as the tie points, there is no guarantee that the center coordinate would be calculated at the same location in the city, which would cause a greater error in the positional accuracy of the registration point.

3.2.2. Distribution of Densely Populated Areas

As shown in Figure 6 and Figure 7, in densely populated areas, such as the Yangtze River Delta and Beijing–Zhangjiakou, the tie points that were automatically extracted in this study effectively avoided the dense light areas inside the city. At the same time, the algorithm selected various isolated areas around the city with a small light area. The original intention of this algorithm is to extract the tie points that are in areas with less light information. In fact, where there are more small light areas, more effective tie points can be obtained.

Conventional algorithms cannot register nightlight images in sparsely populated areas in Western China. The algorithm that is proposed in this paper does not only compensate for such limitations, but also ensures the appropriate registration of the densely populated areas in eastern China.

3.3. Registration Method Comparison

3.3.1. Comparison with the SIFT Algorithm

This section compares the proposed method with the SIFT algorithm, which is a representative algorithm that is based on feature registration. The SIFT algorithm sets the window size to 256 pixels, the grid size to five pixels, and the search area size to 30 pixels. In fact, the parameters here only determine the number of light points that can be extracted from an urban area. The lights in sparse light areas cannot be effectively extracted as feature points by the SIFT method (as shown in Figure 8).

Figure 8 shows the registration results of the tie point in Kuche City, Xinjiang, Western China. Figure 8a,b shows the schematic diagrams of the tie points of the two images, which obtained by the SIFT algorithm’s registration, and Figure 8c,d shows the schematic diagrams of the tie points of the two images that were obtained by the proposed registration method. Figure 8e,f shows the enlarged displays corresponding to the red frame area of the SIFT algorithm and Figure 8g,h shows the enlarged displays corresponding to the area of the proposed method. Comparing Figure 8a,b with Figure 8c,d, it is evident that, by using the proposed method, the registration points are not concentrated in one area but rather they are widely distributed in the image. Specific comparisons between Figure 8e,f and Figure 8g,h show that the traditional SIFT algorithm tends to find characteristic points in urban lights and road lights with structural information, while the proposed method is more inclined to use sparse light areas. In fact, for the adjacent areas of Figure 5, there are several homonymous points that were acquired through the proposed method, but no corresponding points were found through the use of the SIFT algorithm. So, for the traditional SIFT algorithm, there are no homonymous points between adjacent images when registered in the sparse light areas of Western China.

3.3.2. Comparison with the NCC Algorithm

In this section, the proposed algorithm is compared with the representative algorithm, NCC, which is based on grayscale correlation. Figure 9 shows the results of the obtained tie points based on grayscale correlation registration in the same area.

The accuracy of the points is within one pixel, except for point 38 in Figure 9a,b and point 33 in Figure 9c,d. Therefore, for nightlight images in areas with more lights, such as cities, it is also possible to use algorithms that are based on grayscale correlation in order to register the tie points. However, for sparse light areas, such as those in Figure 9e–h, which are affected by noise and the imaging angle of view, the shape of the light changed slightly across different periods. Combined with the lack of textural structure, when registration is based on grayscale-related algorithms the tie point usually does not correspond to the same position of the light point. Therefore, even if the registration of nightlight images in urban areas can be achieved based on grayscale correlation, the NCC method is similar to the SIFT algorithm and still cannot solve the registration of tie points in sparse light areas.

3.4. Accuracy of Tie Points

3.4.1. Tie Point Registration Results in China

The traditional SIFT registration method and the method that was used in this study were applied to register 275 scenes of Luojia-1 nightlight images of China. The final distribution of the tie points is shown in Figure 10a and the mosaic result of nightlight images is shown in Figure 10b.

By using the traditional SIFT algorithm, it is evident from the distribution of the red points in Figure 10a that the tie points are mainly distributed in the densely populated eastern region; indeed, in China’s vast western region, as well as parts of China’s northeastern and southwestern regions, it is impossible to use this method to obtain effective tie points between images. Using the proposed sparse light registration method, not only are the tie points more evenly registered in the eastern areas, but, more importantly, most of the vacancies in the sparse light areas are filled and the tie points of almost all of the areas in China were obtained. The proposed algorithm is also unable to obtain tie points in the zone that corresponds to large uninhabited areas and desert areas such as the Coco Siri region and Taklimakan Desert in Western China, which do not have human-made nightlights. When these areas were photographed by Luojia-1, they were completely black. The statistics for the above calculation process can be found in

Table 2. Comparison of registration methods for Luojia-1 in China.

Registration Method	Registration Time	Tie Points Account	Covered Images/ Total Images	Tie Points Coverage Ratio
SIFT-GPU	31 min	5413	163/275	59.3%
This article	17 min	14588	262/275	95.3%

.

As shown in Table 2, compared to the SIFT-GPU algorithm, which is optimized and accelerated by GPU, the efficiency of the proposed registering algorithm is nearly twice that of the traditional SIFT method. The number of tie points that was obtained by SIFT is less than that which was obtained by the proposed method, but this may be because the SIFT algorithm will thin out the dense area when registering, so the number of points is less; however, it can be seen from Figure 10a that the distribution of the tie points that were extracted by the proposed method is more uniform. It should be noted that the points of the SIFT algorithm are concentrated in big cities such as Beijing and Shanghai, but the tie points that were extracted in this study are scattered across different regions.

The number of images with tie points was more important when performing the mosaicking of the images. The traditional algorithm had only 163 images from which to obtain the tie points, whereas the proposed algorithm had 262 images that cover the tie points. The coverage ratio of the tie points increased from 59.3% to 95.3%, an increase which is extremely large, and the proposed method increased the degree of connection between the images, making the subsequent overall block adjustment results of the nightlight images in China more reasonable.

3.4.2. Block Adjustment Accuracy of Free Network

This study also evaluated the block adjustment accuracy of 275 images for the entire China area. Using the global 1 km grid DEM to assist the block adjustment, the results that are shown in Figure 10 and Figure 11 were obtained.

The detailed mosaic effect of the adjacent images is shown in Figure 11a,c; two adjacent urban area images have a certain offset due to a positioning error before block adjustment. By using the proposed sparse light method, the two images can be better stitched, as shown in Figure 11b,d. The overlay effect is also given for before and after the block adjustment in the sparse light area. As shown in Figure 12, for two images, img1 and img2, the red contour denotes the edge of img1’s light (Figure 12a). Before block adjustment, img2’s light was not in the red contour (Figure 12b). After block adjustment, img2 could be better overlaid with the red contour (Figure 12c).

The statistics for the block adjustment accuracy of the whole China region are shown in

Table 3. Comparison of registration accuracy of Luojia-1 in China.

Method	Error Direction	min	max	rms
SIFT-GPU	x	0	4.335	0.429
	y	0	6.174	0.354
	plane	0	6.235	0.556
Proposed	x	0	6.258	0.448
	y	0	7.917	0.445
	plane	0	7.936	0.632
Combination of both tie points	x	0	5.883	0.453
	y	0	5.764	0.403
	plane	0	6.179	0.606

. As shown, using the traditional SIFT-GPU algorithm, the plane error was 0.556 pixels after the block adjustment and the maximum error was 6.235 pixels. Using the proposed algorithm, the plane error was 0.632 pixels after the block adjustment and the maximum error was 7.936 pixels. The accuracy of the proposed algorithm was slightly worse than that of the SIFT algorithm, but the area and number of images that were involved in the block adjustment of the proposed method were much greater than those that were involved in the SIFT algorithm. Furthermore, many of the available images were not participating in the block adjustment of the SIFT algorithm. Hence, the decrease in accuracy is reasonable. When comparing the root mean square (RMS) error in the x and y directions, it can be observed that the RMS error of the SIFT algorithm was 0.429 pixels in the x direction and 0.354 pixels in the y direction and that the error was not uniform. The median error of the proposed algorithm was 0.448 in the x direction and 0.445 in the y direction, error values which are equivalent. By combining the tie points that were extracted by the proposed method and the SIFT method, the block adjustment accuracy was slightly improved compared to the results of the proposed method alone and the plane accuracy was approximately 0.6 pixels.

4. Discussion

According to the above experimental results, the tie points that were extracted by the sparse light method were usually based on isolated light information, so they are unique. Assuming that in a sparsely populated area with sparse light there are two light points in the overlapping part of the two nightlight images and that the two light points are far apart, they are clearly distinguishable. Using the proposed method, the correspondence between the two points was easy to find. Additionally, because the area of an isolated light point is usually small, when selecting sparse light points in this study, the light points were selected through the use of a roundness threshold. Therefore, the center positions of the two scene images that were extracted by the centroid method were used. In the actual registration relationship, this is also a pair of homonymous points with a smaller positioning error. A larger light area usually has obvious roads or other textural information. At this time, the proposed algorithm rarely extracted the points in this area but obtained data regarding sparse light with a level of high roundness nearby. Using the proposed algorithm to register tie points, tie points that are more reasonably distributed around big cities can now be obtained.

5. Conclusions

This paper proposes a method of using sparse lights to obtain the tie points of nightlight images. Through the use of a roundness threshold and centroid extraction, the light points satisfying the conditions were selected, after which the geometric positioning forward and backward algorithms were used in order to initially eliminate the position error between the points that are to be registered. Subsequently, the preliminary registration of the light points was realized according to the random consistency detection and translation model. Based on the affine model, the points with larger errors were deleted and the tie points were expanded. The experimental results show that the proposed method has a high registration accuracy between homonymous points. The larger extraction range of the light points solves the problem of the difficulty of obtaining tie points in sparsely lit areas. For densely lit areas, smaller light points around the area can also be used to register the tie points. Compared with the traditional registration algorithm, the proposed method has good accuracy and improved distribution performance for the registration of nightlight image tie points. Through a registration test of 275 images of China from Luojia-1, the coverage ratio of the tie points increased from 59.3% to 95.3%. The proposed registration algorithm is thus applicable to the nightlight images of various areas.