Superpixel-Based Grain Segmentation in Sandstone Thin-Section

Abstract

Mineral segmentation is an equally important and difficult task in the quantification of mineral composition. Difficulties come from the process of determining boundaries of distinctive mineral grains necessary for further analysis and mineral identification. Done by hand, the task is very time-consuming and higher accuracies are burdened with the possible human fatigue factor. The presented method is a fully automated solution to the problem that uses a superpixel approach and feature-based merging. The method is validated by comparison with the manual approach. Analyzed data consist of photos taken by a Nikon Eclipse LV100N POL polarizing microscope at 200× magnification, in transmitted light, with crossed polarizers. Images are first prepared by Gaussian filter and meanshift operations, then the initial segmentation is provided by the superpixel algorithm. Oversegmentation is resolved by feature-based merging. The last step consists of counting the individual grain boundaries and preparing the results as easily readable visual data.

1. Introduction

The process of mineral recognition and classification is of high importance to mineralogical research. The classification of rocks is performed during laboratory study, after extraction from the earth depth or being described in a rock outcrop. The identification of the distinctive minerals making up the rock is based on a set of possible features: size, shape, arrangement in the space, and intrinsic properties, such as refractive index, birefringence, cleavage, and twins. Such a task is important in many fields: petrology, deposit geology, rock mechanics, mining engineering, construction, and material engineering, see [1,2,3,4,5].

1.1. Mineral Analysis

Optical properties of minerals make the identification possible due to the visibility of distinctive minerals and therefore already mentioned size, shape, and distribution of their components. To estimate the quantitative composition of the rock by optical microscopy, it is important to accurately determine the boundaries between all components of the rock [6,7]. Sandstones are composed of mineral grains and rock fragments, which form the so-called framework grain of the rock. The free spaces between the grains of the framework are filled to varying degrees by a mass of the so-called matrix and cement [8]. The matrix is made up of detrital components, mainly clay minerals and fine grains of quartz and feldspar under 30 mm in diameter. Due to their size, individual matrix components are often difficult to identify under an optical microscope.

The cement is chemically precipitated in empty spaces in the rock and binds the sandstone grains together, but not all the spaces are always filled, so sandstones usually show porosity. During this process, the surfaces of the grains are often partially etched by the cement and become uneven [9]. The presence of matrix, cement, and rock pores in the intergranular spaces and partially over-digested framework grains with indistinct boundaries is a common feature of sandstones [10]. This often causes, in contrast to magmatic rocks and high-grade metamorphic rocks, difficulties in determining the exact boundaries between individual rock components by optical microscopy. This negatively affects the quantitative composition determinations of sandstones.

In geology, it is important to know the porosity and open porosity as these determine the flow of fluids through the rock medium. Thus, the importance of reservoir parameters of rocks and their characterization in sandstone layers where water, oil, and gas accumulate [11,12].

A quantitative analysis of porosity can be performed with an optical microscope by creating a set (more than a dozen) of thin section samples. The composition expressed in percentages by volume is then measured (usually) by manual counting of points, with one of the specified methods: planimetric, linear, or point.

In the point counting method, the image is covered in a planimetric grid of points that cover the entire surface of the prepared thin section. These points are then counted separately inside each of the previously identified elements. In the case of sandstones, these are framework grains, matrix and cement, and pore space (Figure 1a). Individual elements are identified using microscopic methods. The boundaries between them are hand drawn (Figure 1b) and a number is given to identify the components (Figure 1c). The percentage share of individual areas is calculated after hand-drawing the boundaries between them. This non-automated method is a very time-consuming process. Based on the complexity of the rock structure and density of the measurement grid, it can take approximately 3 to 8 h of manual work. The smaller-sized the mineral grains in a rock thin section image, the higher the density of the grid necessary to obtain correct results. In the case of bigger grains size, grid density (and therefore time) can be reduced with the downside of lesser accuracy.

The mineral composition is another element used in the rock reservoir mineral description. Individual minerals, their forms of occurrence, inclusions, and boundaries between mineral grains are identified [13,14]. The disintegration of the primary rocks into smaller particles is a result of weathering and erosion, which is later transported to the sites of final sedimentation.

1.2. Device-Based Methods

Qualitative and quantitative analyses of the minerals form the base for basic rock classification. Standard analysis is complemented by a range of different, more advanced techniques that might require the presence of specialized personnel. Among these, we can mention electron microscopy (SEM, scanning electron microscopy; TEM, transmission electron microscopy), microarea chemical analysis (EPMA, electron probe microanalyzer; SEM-EDS, SEM with energy-dispersive X-ray spectroscopy), and X-ray diffraction (XRD) [15].

SEM is one of the most extensively researched methods that find application in geo and material sciences. More commonly used in practice, SEM-AM (scanning electron microscopy-based automated mineralogy) can be described as a combination of hardware platform, processing software, and image analysis algorithms. The image analysis component of this system is based on the assesment of the backscattered electron image [16]. Examples of its possible application can be seen in [17], where mineralogy was quantified with the use of SEM meaning generating fast and automated profiling of soil samples in huge quantities, that later allow us to determine if the sample indicates an unnatural origin.

SEM is often combined with energy-dispersive X-ray spectrometry (EDS), which allows for high-resolution measurement and, thanks to the very comprehensive data, can be supported with a variety of clustering algorithms, such as in [18]. Simultaneously with the applications, are developed solutions to the problems existing internally to the method. An example can be seen in [19], where the authors propose a multilayered approach to make SAM-EDS available for investigation of submicrometer-sized grains. Additionally, due to the increasing amount of obtainable data, there is an increased importance in studying the associated statistical topics. The importance of uncertainties in quantitative mineralogy and the approach towards a reduction of the statistical errors is discussed in [20]. Authors based the research on a case of study of the Bolcana Au-Cu porphyry deposit in the South Apuseni Mountains in Romania.

1.3. Image Analysis-Based Methods

To obtain fast and repeatable results in a large number of experiments, manual parts of the process are consistently replaced with more automatic approaches. The counting of points is supported by mechanical measuring devices and more advanced methods based on computer analysis [15].

The problem of segmentation in image analysis can be approached in different ways. The first important distinction is whether neural networks (NN) were used. Methods that do not rely on NN usually focus on feature extraction such as color edge detection or gradient filtering [21,22]. In [23], the authors proposed an ore image segmentation algorithm based on a histogram accumulation moment. In [24], authors proposed froth segmentation based on implementing an improved Harris corner detection algorithm, after which the images are segmented by automatically choosing the corresponding parameters for identifying bubble edge points through extracting the local gray value minima.

Some of the image processing methods suitable for mineral segmentation should also be mentioned, such as image segmentation through the MATLAB environment. These methods can be divided into several categories: treshold-based segmentation, edge-based segmentation, watershed segmentation, and region-based segmentation [25]. The effectiveness of the individual methods is often determined by its application. As an example, watershed segmentation is especially useful for dividing objects that are in contact with each other, while simple edge-based segmentation would provide comparable, but much faster results in less complicated tasks. At the same time, all the mentioned methods have significantly less accuracy in comparison to the methods supported by machine learning.

Machine learning, especially in the form of neural networks, is currently the most popular method of solving any segmentation task due to its easy implementation and high performance. At the same time, solutions based on neural networks are very task specific and often need a significant number of available training data. Some possible approaches consist of using convolutional neural networks [26,27,28,29] and superpixels [30,31].

In this article, we present a method for fully automatic grain border recognition of a sandstone thin section. The method is based on the application of superpixels. Meanshift operations were used as initial image preparation with color and histogram-based merging of superpixels before the final steps. The end result consists of a set of contours each with a calculated percentage of the whole image area.

2. Experiment Description

Research begins with the preparation of a thin section from the collected sandstone sample. This process consisted of a few consecutive steps: A small cuboid-shaped chip of sandstone rock was cut from a drill core (Figure 2a,b) and then a thin slice of rock was cut using a diamond saw to fit onto a standard petrographic microscope slide (28 × 48 mm). This slice of the rock was glued to the glass slide using epoxy (Figure 2c) and ground until it was optically flat using progressively finer abrasive grit. This means that light passes through the slice of rock sliver of material. The standard thickness for a thin section in an optical microscope study is 30 microns. Finally (Figure 2d), a coverslip was glued onto the surface of the thin section when only optical microscope studies were planned. On the other hand, if further chemical analysis of minerals was planned, the final step was polishing the surface of the grind with diamond pastes until the surface was smooth, flat, and free of deformation and scratches, which is bright, shiny, and has a mirror-like appearance with sharp edges and good distinction between mineral grains. When observing a sandstone thin section in an optical microscope under polarized light, optical features of minerals, including pleochroism, light extinction, cleavage, and interference colors are visible. Because individual minerals vary in their optical properties, most rock-forming minerals can be easily identified by polarized light in optical microscopy.

Photos of a sandstone thin section were taken using a Nikon Eclipse LV100N POL polarizing microscope (Nikon Corporation, Tokyo, Japan), at 200× magnification in transmitted light with crossed polarizers. All of the rock-forming minerals were identified.

Then, to determine the borders of distinctive grains, the fully automated algorithm described in the next section was applied.

3. Methodology

The presented methodology is best described by dividing into four sections: initial image preparation, implementation of SLIC (simple linear iterative clustering) algorithm, superpixel merging, and detection of the final borders with area calculation afterward. Consecutive steps of the presented algorithm can be seen in Figure 3.

3.1. Initial Image Preparation

To obtain the best possible results, some initial operations had to be conducted. First, an image was blurred by a Gaussian filter with a small kernel size. Then, the image was filtered with color gradients and texture flattened by using the meanshift iterations on every pixel (x, y). The pixel neighborhood (X, Y) in the joint space–color hyperspace is considered:

$$(x,y):X-sp<=x<=X+sp,Y-sp<=y<=Y+sp,\left|\right|(R,G,B)-(r,g,b)\left|\right|<=sr$$

(1)

where sp is the spatial window radius, sr is the color window radius, (R, G, B) and (r, g, b) are the vectors of color components at (X, Y) and (x, y). Over the neighborhood, the average spatial value and average color vector were found and act as the neighborhood center on the next iteration.

After the iterations, the color components of the initial pixel (that is, the pixel from where the iterations started) were set to the final value (average color at the last iteration):

$$I(X,Y)<-(R\ast ,G\ast ,B\ast )$$

(2)

Although it is a time-consuming process, the described filtering is helpful in regard to removing the small particle influence during later steps of the algorithm together with creating sharper boundaries on the color region edges.

3.2. Superpixel Implementation

Superpixels can be described as a collection of points (pixels) that represent a unified region of the image. They are defined by a set of features, including Gestalt cues of contour, texture, brightness, and good continuation [32].

SLIC is one of many algorithms designed to create superpixels, which can be described as adaptation of k-means for superpixel generation [33].

The SLIC algorithm is widely used in tasks of visual recognition, with examples such as tracking [34,35], prostate segmentation on magnetic resonance images [36], diseased leaf segmentation, and recognition [37].

The SLIC implementation in the algorithm presented in this paper is used for segmentation purposes. Its task is to create initial borders on the edges of the mineral grains, while creating regions with similar properties that can be further modified or merged with each other, based on the features described in Section 3.3.

The basic parameter of the SLIC algorithm can be either the number of superpixels with relatively similar or the same sizes, as they directly describe the same property of the region and can be deduced alternately.

The clustering procedure begins with an initialization step where k initial cluster centers are sampled on a regular grid spaced S pixels apart. The grid interval can be described as:

$$S=\sqrt{N/k}$$

(3)

with N being the overall number of pixels in the image. The centers are moved to seed locations corresponding to the lowest gradient position in a defined neighborhood. Then, SLIC computes distances from each cluster center to pixels within a 2S × 2S region. Once each pixel has been associated with the nearest cluster center, an update step adjusts the cluster centers to the mean vector of all the pixels belonging to the cluster.

To represent the position of the clusters, the distance between the pixel and cluster center must be measured, in both the spatial and color spaces.

$$d_c=\sqrt{{\left(l_j-l_i\right)}^2+{\left(a_j-a_i\right)}^2+{\left(b_j-b_i\right)}^2}$$

(4)

$$d_s=\sqrt{{\left(x_j-x_i\right)}^2+{\left(y_j-y_i\right)}^2}$$

(5)

where l, a, b are parameters of CIELAB color space, and x, y represent the coordinates of spatial position.

Then, after simplification, the maximum distance between the pixel members of the cluster can be described by:

$$D=\sqrt{d_c^2+{\left(\frac{d_s}{S}\right)}^2{m}^2}$$

(6)

where S stands for spatial sampling interval, m represents a fixed constant of maximum color distance, d_c and d_s are understood, respectively, as normalized color and spatial proximity.

The final step of the SLIC algorithm consists of assigning the label of a nearest cluster to the pixels that did not fit anywhere during the segmentation process. The end result of the algorithm comes in the form of an oversegmented image that needs further processing.

3.3. Superpixel Merging

To merge the superpixels into larger clusters resembling the exact borders of the distinctive objects (different minerals in the case of this paper), some properties must be obtained. Adjacency is the first, as it allows for a reduction of computational time by extracting and comparing only the data from the superpixels that have common borders.

To determine the superpixels adjacency, a simple algorithm was used. First, one superpixel from each possible pair was dilated with the use of the smallest kernel. Then, the superpixels in the checked pair are compared bitwise. With enough overlapping pixels inside the masks, the pair is written into an adjacency matrix with a non-zero value.

The matrix size is determined by the number of generated superpixels and can be written as N × N, where each coordinate stands for a separate pair of pixels. Only half of the matrix is filled and taken into consideration in later steps since the other half is mirrored with the same superpixels paired in reverse order.

In the next step, determined by the pair of superpixels, two separate features are compared: mean color value and color histogram. Only one of the two needs to check positively for the superpixels to be merged.

The mean color value is determined based on the mean value of each HSV (hue, saturation, value)channel thresholded separately instead of thresholding its values compressed into single-channel grayscale. This way, saturation can be given a bigger window of acceptable difference. A similar approach was taken in the case of the histograms. For the comparison metric, a correlation described by the below equation was used.

$$d(H1,H2)=\frac{{\sum }_I(H_1\left(I\right)-\overline{H_1})(H_2\left(I\right)-\overline{H_2})}{\sqrt{{\sum }_I{(H_1\left(I\right)-\overline{H_1})}^2{\sum }_I{(H_2\left(I\right)-\overline{H_2})}^2}}$$

(7)

$$\overline{H_k}=\frac{1}{N}\sum _J^{}{H}_k\left(J\right)$$

(8)

where N is the number of histogram bins.

Histograms were created with three dimensions, each one based on the HSV image ranges (hue {0–180}, value and saturation {0–255}), and 15 bins each (except the hue that was given 10 bins). Bins can be understood as a subdivision that decide the separation thresholds inside a defined range.

Labeled superpixels that were identified as having similiar properties were merged afterwards by pushing them into a set of vectors with their numerical identifiers. Each superpixel from the vector was then printed on the grayscale matrix with edge identification afterwards. That result was stored on a separate matrix with other contours added by each consecutive vector of IDs. At the end of the process, the matrix of all detected mineral borders was obtained.

3.4. Area Detection and Calculation

The last step consists of simply detecting and calculating distinctive areas. Before contour detection, a simple morphological closing was performed to eliminate potential holes in the edges detected in previous steps. The found contour topologies were stored in another set of vectors storing the hierarchy and points. Based on the information about the number of points inside each contour, their percentage share of the image area can be easily calculated and printed afterward over the original image. The centers of each contour (necessary to display the results in a readable way) are based on the calculation of contour moments. This way each label, together with the percentage value, is displayed inside the described area and can be easily used for later comparisons. This information can be easily obtained (and therefore integrated into) during the superpixel merging step of the algorithm, but the presented approach allows for better control over the data.

4. Results

To maintain clarity of the consecutive steps, only four images from the set are presented with the step-by-step algorithm, with the last step of the algorithm compiling the results of the grain borders information gained from the whole set.

For the purposes of presentation, four images were picked with a consecutive angle difference of 50°. The picked set of original images can be seen in Figure 4. A change in color of distinctive minerals is clearly visible witha change of the light angle.

The set of the images after the applied filters can be seen in Figure 5. It mostly accomplished a more coherent surface (both in terms of color and edge clarity) of each mineral. Impact of dirt from mineral, dirt from the camera, small dots, and other impurities were also minimized.

In Figure 6, the results of the SLIC algorithm with 1147 created superpixels, which is based on the pixel size of each of the distinctive areas and the maximum image size.

Examples of the obtained adjacency matrixes can be seen in Figure 7. A lesser number of superpixels results in faster computing time but with a significant sacrifice in accuracy. The higher number of superpixels would be even more necessary in cases with smaller mineral grains.

The result for the first image (labeled in previous figures as 0 angle) can be seen in Figure 8 with the labeled contour (which is automatic and therefore with a degree of randomness in the order) and the percentage of its area is relative to the overall image size. Connected areas with less than 0.5 percent of the image size were not considered in the final results, as these are hard to evaluate and can be discoloured fragments of the bigger grains. Connectivity was forced only to a certain degree.

In terms of time, a single photo takes up to one-half of a minute, based on the image resolution, to perform the full task with the end result in the form presented in Figure 8. The algorithm was not optimized timewise, therefore, time consumption can be significantly reduced both inside the written code and by running it with the use of graphic processors and software dedicated for that purpose (such as NVIDIA’s CUDA).

5. Validation

The obtained results were validated in two ways. First, they can be compared directly to the by-hand method, both in terms of accuracy and time consumption. The problem with that comparison is the by-hand method’s small precision, especially for lesser resolution (necessary to reduce the time-consumption to bearable levels). Therefore, although it gives good comparison for the time required to obtain the results, in terms of precision, another benchmark is necessary.

In the case of this paper, the benchmark in the form of a single image from the same set is discussed in [38]. The method presented there is based on a semi-automatic approach, drawing the actual borders on the graphic application with simple program assessing the grain’s area percentage afterward. Due to the nature of this method, it’s precision corresponds to the precision of the person drawing the borders with all other calculations performed afterwards. The validation is done on a single image from the set, not an end result of the set of processed images. The result of that experiment can be summarized in Figure 9.

The same calculations were made using the manual method described at the beginning of the article. The individual elements were marked out (Figure 1b) and given individual identifiers (Figure 1c). This process takes up to 30%–40% of the total measurement time. The planimetric grid was then applied and the points in the individual fields were manually counted. The choice of the density of the planimetric grid influences the accuracy of the calculation of the occupied area. Four planimetric grid densities were adopted for the calculations: case I n = 100, case II n = 210, case III n = 474, case IV n = 930 (Figure 10).

After counting the points in each field, the surface area covered (in %) was calculated for each field highlighted at an earlier stage. Due to the sparse grid, no calculation point (F = 0) was found in the six smallest fields (minerals). The lowest value for Case I was obtained for areas K1, K2, K6, K8, K9, K10, K14, K18, and K24 of 1.01%, the highest 11.11% for K11. The low density of the planimetric grid results in a low resolution of the measured parameter and the existence of the same values for several areas. As the grid became denser, this problem was eliminated. The difference in calculated area (%) was compared between the different densities of the planimetric grid. As the grid became denser, the percentage of large grains relative to smaller grains decreased. Differences in a few cases between the least compacted grid and the most reached up to 3% (Figure 11). Differences in the calculated mineral area from the ratio using the algorithm to manual calculations were up to 1.8% for the lowest density grid (case I), and up to 0.5% for the densest calculation grid (case IV).

The timing of the measurements remains important. As mentioned earlier, this can take several hours. For the described thin section, it took about 2 h to recognize the components that build up the sandstone, to classify them petrographically and to identify the boundaries. The next step was to count the planimetric grid points in each area. This took between a few and several minutes depending on the case. Assuming a time of 1 (t = 1) for case 1, a graph was plotted of the increment of counting time to the increment of planimetric grid points (Figure 12). It was shown that a 9.4-fold increase in points results in a 4.7-fold increase in the time required to perform the count. In addition, this procedure was repeated for a less complex division (20 areas/minerals) (Figure 13). The complexity of the internal structure (number of calculation fields) also significantly affects the length of the calculation process. The disadvantages of manual counting are that with complex thin grinds, with a large number of calculation fields, there is a human fatigue factor (e.g., work stoppage) and problems with the subjective assignment of planimetric grid points lying on the boundary of the fields. These problems are not present in computerized methods. In addition, other parameters, e.g., the ratio of the longest to shortest grain axis, which are indicative of the processes of the sedimentary rock formation environment, can be read from the analysis of the fine grind image.

6. Conclusions

A set of sandstone photos was taken and analyzed to extract the individual grain boundaries. To streamline the process and decrease the time required in the manual method, a fully automated algorithm was proposed, focused on detecting the boundaries of individual sandstone elements. This consists of the following tasks: individual grain boundary detection, area calculation, and various output possibilities, such as images with applied data or tabularized results.

Obtained results were compared on the same dataset with a manual method and semi-automatic methods based on borders drawn by hand over the existing image with use of graphic software. The semi-automated method can be taken as a precision benchmark since it omits problems with boundary detection by allowing the human to draw over the image with the results coming right after. In both cases (semi- and fully-automated methods), the precision of the area calculation is based on the image resolution, instead of the manually applied grid as in the case of manual counting.

In direct comparison with manual counting, differences were most visible in the case of the lowest grid density (up to 1.8% for 100 points) and decreased with growing density (down to 0.5% for 930 points). Although higher density allows for more accurate results, it also significantly extends the time necessary to perform the task; therefore, time can be optimized by lowering the density, while still achieving sufficient results. It is also important to notice the fatigue error that, in the case of multiple sample analyses, can influence the results, especially in the case of higher density grids that require constant focus during manual counting. In some cases, fully automated algorithms can separate a single big element into separate areas, based on its shape. That can be corrected manually by adding the results of the appropriate contours during the supervision and can be a part of the future improvements of the algorithm.

The time required for the non-optimized program presented in this paper to perform the described task takes up to one-half of a minute with unoptimized code, hardware, and software, which is a significant reduction compared to the manual method. Multiple samples can also be analyzed simultaneously, which further improves the time factor.

Other advantages of the fully automated method are: easily stored and represented data, readiness for further analysis (such as determining the mean color of each distinctive grain), easily changed calculation parameters, and possibility for the integration with other steps of the mineral analysis process. Additional improvements of the algorithm could be made in terms of additional feature detection during superpixel merging and improving the run-time by preparing the code to be used with graphic processors.

Future work consists of proceeding with the analysis after correct clusterization. Part of it is tracking the variation of parameters (change of color, intensity during light extinction, etc.) for a given cluster, as this allows for preliminary classification according to specific groups or processes. Further steps would focus on the use of methods related to machine learning to recognize particular groups of clusters and then assign them to the elements identified during the petrographic description of the sandstone studies.