Systematic Review of Aggregation Functions Applied to Image Edge Detection

Abstract

Edge detection is a crucial process in numerous stages of computer vision. This field of study has recently gained momentum due to its importance in various applications. The uncertainty, among other characteristics of images, makes it difficult to accurately determine the edge of objects. Furthermore, even the definition of an edge is vague as an edge can be considered as the maximum boundary between two regions with different properties. Given the advancement of research in image discontinuity detection, especially using aggregation and pre-aggregation functions, and the lack of systematic literature reviews on this topic, this paper aims to gather and synthesize the current state of the art of this topic. To achieve this, this paper presents a systematic review of the literature, which selected 24 papers filtered from 428 articles found in computer databases in the last seven years. It was possible to synthesize important related information, which was grouped into three approaches: (i) based on both multiple descriptor extraction and data aggregation, (ii) based on both the aggregation of distance functions and fuzzy C-means, and (iii) based on fuzzy theory, namely type-2 fuzzy and neutrosophic sets. As a conclusion, this review provides interesting gaps that can be explored in future work.

1. Introduction

One of the most common approaches for detecting discontinuities in images is edge detection [1]. An edge is defined as the maximum boundary between two regions with different properties; that is, the boundary between two objects or object faces in an image [2]. This research field represents an important task in several steps of computer vision and image processing, such as [3,4,5,6]: object detection [7], pattern recognition [8,9,10,11,12], photo-sketching [13], image retrieval [14,15,16,17], face recognition [18,19,20], corner detection [21,22,23], road detection [24], and target tracking [25,26,27,28].

In recent years, the study of edge detection has gained momentum due to its importance in various applications, such as autonomous cars [29,30]; augmented reality [31,32]; image colorization [33,34,35]; and medical image processing [36,37,38]. Either because of the discretization intrinsic to the digital capture process or because of some subsequent quantization process, the edge or faces of the objects show a small smoothing around the actual boundary of the regions.

The uncertainty, among other characteristics of images, makes it difficult to determine the edge of objects accurately. To accomplish this goal, different methods have been proposed throughout history, from methods based on partial derivatives, such as Sobel [39], Log [40], and Canny [41] in the 1970s and 1980s, to methods based on convolutional neural networks in recent years [42].

Considering that the human visual system has been a source of inspiration for computer vision researchers, recent studies considering the usage of aggregation and pre-aggregation [43] functions have been applied to simulate the behavior of the information flow in the visual cortex in the task of edge detection and image segmentation, as well as to assist in the decision making of other algorithms with the same objective (see, e.g., [44,45,46,47,48,49,50,51,52]).

Aggregation functions are a type of function with certain special properties, which aim to combine or fuse inputs into a value that is able to represent the set of inputs. Precisely, this function must respect two main properties, namely (i) boundary conditions and (ii) monotonicity. The relaxation of the monotonicity condition has been the focus of recent investigations. The notion of weak monotonicity was introduced by Beliakov et al. [53] and extended to directional monotonicity by Bustince et al. [54]. After that, the concept of pre-aggregation functions [55] was introduced, which are functions that respect the same boundary conditions as an aggregation function; however they are just directional increasing.

Given the advancement of research in image discontinuity detection, especially using clustering, aggregation, and pre-aggregation functions, and since there are no systematic literature reviews on this topic, this paper aims to gather and synthesize the current state of the art of this research field. In this sense, the question that is intended to be answered is:

What edge detection methods are based on aggregation and pre-aggregation functions?

The main objectives of the study are to:

1. 

Fill this gap in the literature of systematic reviews on the topic;

2. 

Summarize the existing technology regarding methods that make use of those functions in digital image processing, more specifically regarding edge detection;

3. 

Identify the gaps in the detection approach using aggregation or pre-aggregation functions, proposing themes for future work.

This paper is organized, in addition to this introduction, as follows. In Section 2 (Preliminaries), the most widespread edge detection methods to date are presented chronologically. Section 3 (Materials and Methods) presents the search terms, where inclusion, delimitation, and exclusion criteria are discussed, as well as the number of papers found and admitted for review. Then, in Section 4 (Results and Discussion), we present the aforementioned research question, summarizing the methods reviewed and performing a qualitative assessment that compares the main techniques and approaches. Finally, Section 5 is the Conclusions.

2. Preliminaries

In order to provide a better understanding of the techniques that will be presented in this review, this section shows a rigorous presentation regarding aggregation and pre-aggregation functions (Section 2.1) and a summary of the classical and most widespread edge detection techniques in recent years. This rationale is based on four major review articles published in the last three years [42,49,56,57]. The following is a summary grouped by approach: gradient-based methods (Section 2.2), region-segmentation-based methods (Section 2.3), methods based on machine learning (Section 2.4) and fuzzy-logic-based methods (Section 2.5).

2.1. Aggregation and Pre-Aggregation Functions

An aggregation function [58] is an n-ary function $f:{[0,1]}^n\to [0,1]$ with the following properties: (A1) boundary conditions: $f(0,0,...,0)=0$ and $f(1,1,...,1)=1$; (A2) increasingness: $X\le Y$ implies that $f\left(X\right)\le f\left(Y\right)$, for all $X,Y\in {[0,1]}^n$.

An n-ary function $P:{[0,1]}^n\to [0,1]$ is said to be a pre-aggregation function [55] if it satisfies the following conditions: (PAI) boundary conditions as defined in (A1); (PA2) directional increasingness [55]: there is a real vector $\overrightarrow{r}\in {\mathbb{R}}^n(\overrightarrow{r}\ne \overrightarrow{0})$ for which P is $\overrightarrow{r}$-increasing; that is, for all $\overrightarrow{x}\in {[0,1]}^n$ and $c>0$ such that $\overrightarrow{x}+c\overrightarrow{r}\in {[0,1]}^n$: $P(\overrightarrow{x}+c\overrightarrow{r})\ge P\left(\overrightarrow{x}\right)$.

2.2. Gradient-Based Methods

A digital image is a discrete representation of the variation in light in the real world. Thus, each numerical value carried by the pixel represents the intensity of light or color at that point. The contours of objects in the image can be interpreted as a transition zone between these intensities. Very intense transitions have a much higher chance of being an edge than too smooth transitions. It should be noted that the first edge detectors that appeared in the literature took this point into account based on the gradient of the image [59].

There are two approaches toward gradient-based detection: (i) based on first-order derivatives and (ii) based on second-order derivatives. The most well-known gradient-based detection methods are: (i) first-order fixed operations; (ii) first-order oriented operations that use the maximum energy of the orientation; and (iii) two-direction operations.

Table 1. The gradient magnitude and orientation calculation for a scalar image I, where $I_{\theta }$ represents the derivative of the image using the first-order filter in orientation $\theta$ in radians [60].

Operator Type	Fixed Operator	Filter-Oriented	Half-Gaussian Kernels
Magnitude of gradient	$|\nabla I|=\sqrt{I_0^2+I_{\pi /2}^2}$	$|\nabla I|={max}_{\theta \in [0,\pi [}\left|I_{\theta }\right|$	$|\nabla I|={max}_{\theta \in [0,2\pi [}|I_{\theta }|-{min}_{\theta \in [0,2\pi [}\left|I_{\theta }\right|$
Gradient direction	$\eta =arctan\frac{I_{\pi /2}}{I_0}$	$\eta ={arg\; max}_{\theta \in [0,\pi [}\left|I_{\theta }\right|+\frac{\pi }{2}$	$\eta =\left({arg\; max}_{\theta \in [0,2\pi [}{I}_{\theta }+{arg\; min}_{\theta \in [0,2\pi [}{I}_{\theta }\right)/2$

summarizes the first-order methods, which consist of three fundamental steps:

(i) 

Computation of the magnitude of the gradient $|\nabla I|$ and its orientation $\eta$, using a kernel $3\times 3$, (Table 1), with a first-order, vertical and horizontal derivative filter such as steerable Gaussian, oriented anisotropic Gaussian kernels, or a combination of two half-Gaussian kernels;

(ii) 

Non-maximum suppression operation for thinner edges: selection of pixels with a local maximum gradient magnitude along the direction $\eta$ of the gradient, which is perpendicular to the orientation edge;

(iii) 

Determination of the thresholds of the fine contours to obtain the contour map.

These steps are described visually in Figure 1.

Considering the gradient-based methods [49], in Figure 2, we provide a timeline showing the main related works and their order of publication. Taking this chronological order into account, in the following, we discuss the main articles.

The Canny detector [41], which is one of the most widely used methods, was proposed considering a Gaussian smoothing followed by a gradient operation and finally thresholding. In [42], the authors discussed other first-order derivative-based methods inspired by Canny, such as D-ISEF [61], color-boundary [62], and FDAG [63].

The detector proposed by Sobel et al. [39] calculates the gradient value at each pixel position in the image using a fixed operator. In turn, Prewitt et al. [64] has a similar technique to Sobel, which, however, has no adjustment coefficients, varies by a constant value, and calculates the magnitude of the gradient with the image orientations. In the study by Roberts [65], the derivative is calculated using the root of the difference between diagonally adjacent pixels as follows:

$$Z_{i,j}=\sqrt{{(y_{i,j}-y_{i+1,j+1})}^2+{(y_{i+1,j}-y_{i,j+1})}^2}$$

(1)

where

$$y_{i,j}=\sqrt{x_{i,j}},$$

(2)

focusing on invariant properties that the edges exhibit.

A log filter [40], or Gaussian Laplacian, is based on Gaussian filtering followed by a Laplacian operation as follows:

$${\nabla }^{2}f(x,y)=\frac{{\delta }^{2}f(x,y)}{\delta x^2}+\frac{{\delta }^{2}f(x,y)}{\delta y^2}.$$

(3)

This approach considers a second-order operator, which seeks to identify the maximum and minimum points of the variation in intensities so that, by finding these points that cross to zero, the edge candidates in the image are found. The Gaussian filter step is important because it is a second-order method and therefore extremely sensitive to noise.

2.3. Region-Based Segmentation Methods

In these methods, regions are segmented in various forms, such as clustering methods and automated thresholding. The edges are detected as the border of these regions. It is possible to leverage the regions formed by the textures in high-complexity images as an enabler for the process of detecting the edges between them [56].

In general, the region-based segmentation approach outperforms edge-based methods. Other methods, such as those based on frequency domain filtering and statistical methods, are used both in region-based segmentation methods and for direct edge detection. Through the emergence of texture descriptors and other local information, such as the brightness, texture, and color gradient, the probabilistic contour (Pb) method [42,56,66] has been one of the most representative approaches in this type of detector.

2.4. Methods Based on Machine Learning

By combining the Pb method with a logistic regression, it was possible to develop another kind of edge detection model [42]. The method has been extended over the years in many ways, bringing, for example, the multi-scale probabilistic contour method and multi-scale spectral clustering.

Currently, new techniques based on machine learning have emerged, particularly those based on Convolutional Neural Networks (CNNs). In [42], the authors present other machine-learning-based methods in their research, such as Holistically Nested Edge Detection, better known as HED [67], proposed to improve the performance of convolutional neural networks, which then inspired the methods: CEDN [68], RCF [69], and LPCB [70], among others. Recently, the BDCN [71] method has emerged, proposing detection at different scales. Currently, techniques such as Fined [72], EDTER [73], and PiDiNet [74] have been proposed to deal with edge detection without the need for a voluminous database for model training.

2.5. Fuzzy-Logic-Based Methods

A fuzzy inference system modeling for efficiently extracting edges in noisy images, called the Russo method, was proposed in [75]. From this point, the application of fuzzy set theory [76] in edge detection increased, justified by the fuzzy nature of object edges in a digital image, which makes fuzzy set theory suitable for solving such problems.

Detectors based on the junction of techniques such as divergence and Fuzzy Entropy minimization (FED) [77], and detectors based on morphological gradient and type-2 fuzzy logic (Type-2 Color) [78] have appeared in the literature. Hybrid techniques and fuzzy versions of neural networks have been created, improving the performance of other approaches and, in some cases, decreasing the computational complexity.

In the studies of hybrid methods [57] that use neural networks, fuzzy logic type-1, type-2, and type-3 (hybrid approach of neuro-fuzzy-1, neuro-fuzzy-2, and neuro-fuzzy-3, respectively), which are extensions of fuzzy logic with more degrees of associated uncertainty, can also be identified in the literature, as well as the use of fuzzy logic to improve classical methods, such as Canny (C&I-TYPE2), and a hybrid method that uses Sobel, type-1 fuzzy logic, and a type-2 fuzzy interval system (T2FLS). For a better understanding of fuzzy-logic-based methods, Figure 3 shows a timeline presenting, in chronological order, the papers previously discussed. Figure 4 presents an attempt to broadly represent a schema for fuzzy-logic-based methods.

3. Materials and Methods

This section presents the methodology used in this research. We first introduce the concept of a systematic literature review and the methodology used to answer the research problems.

3.1. Systematic Literature Review Scope

A synthesized literature review is a form of secondary study that aims to identify, evaluate, and interpret all available relevant research on a specific research problem, topic, or phenomenon of interest [79]. Among the many reasons to engage in a systematic literature review, the most common are: (i) to summarize the existing evidence regarding treatment or technology; (ii) to identify gaps in current research with the goal of suggesting areas of future investigation; (iii) to provide a body of information that appropriately positions new research activities; (iv) to reduce or attempt to eliminate research bias.

In this context, this research seeks to summarize the existing technology regarding methods that make use of aggregation functions for digital image processing, more specifically regarding edge detection. Another important contribution was the identification of gaps in the state of the art.

3.2. Definition of Criteria, Search in Indexing Databases, and Obtaining Primary Research

In order to answer the research problems, searches were conducted in two large databases widely used in the computer science field: Scopus (SC) (available at www.scopus.com, accessed on 1 November 2022) and Web Of Science (WS) (available at www.webofscience.com, accessed on 2 November 2022). The inclusion terms used to answer which edge detection methods use clustering and pre-aggregation functions were the terms:

(i) 

“preaggregation” OR “pre-aggregation" OR “aggregation” OR “fusion” OR “Sugeno” OR “Type-2” that appear in the abstract OR;

(ii) 

The terms of the item (i) joined to the terms “Fuzzy” and “Logic” concatenated with “Type-2” with the exception of “Sugeno” as terms that appear in the title;

(iii) 

The terms of the items (i) and (ii) in conjunction with the terms “image” AND “segmentation” OR “image” AND “processing” OR “edge” AND “detection” as terms that appear in the abstract.

To conduct the search, delimitation criteria were used, such as articles written in English, published from 2016 onward, that have referenced the work of “Canny” or authors who have worked with “canny” and that were articles or reviews in the area of computing. This search returned 428 results, which were reduced by the exclusion criteria.

Initially, all of the papers that had a title that demonstrated a lack of relevance of that study for this review work were eliminated, all of the papers that were flagged as retracted by the journal were eliminated, and an analysis of the abstract was performed, observing the objectives of that research in particular, thus eliminating all of the articles that were outside the scope of this review. Finally, an analysis of the methodology of the papers was performed in order to check whether they present any serious technical problem that would justify discarding them even before their full reading.

At the end of the process, 17 articles remained. These remaining articles were used to find other relevant works. We analyzed the sections of related works, also searching using the keywords of such papers, whether from the same research groups or not, to find papers that were somehow composed of the development of the concept or the history of what was developed in the remaining articles, thus considering the presentation of what had already been completed and had somehow been missed in the initial search.

In total, 7 works were added and are marked in the second column of Table 2. An overview of the systematic review performed is available in Figure 5. It is possible to observe that, given the search question, the inclusion terms are considered, where ABS() and TITLE() represent searches of the abstracts and titles of the papers, respectively.

In delimitation terms, PUBYEAR, REFAUTH, and REF are, respectively, the year of publication of the paper, papers that cite a specific author, and papers that present a specific term in their references. In a number of occurrences, one has in INC the number of papers found through the inclusion and delimitation terms. Then, filtering was performed by title, abstract, and the exclusion of articles retrieved and included after, where EXC represents the number of articles excluded and REM the remaining.

4. Results and Discussion

After filtering the papers, using the inclusion and exclusion criteria already discussed, 24 papers remained to be reviewed, listed in Table 2. Among them, 19 papers were found that are specifically dedicated to edge detection, either using multiple descriptors extraction and aggregation or based on fuzzy set theory: type-2 fuzzy and neutrosophic sets [44,47,80,81,82,83,84,85,86,87,88,89,90] or works that use clustering and pre-aggregation functions, which are dedicated to region segmentation, but consider that their characteristics can be extended to the task of edge detection [46,48,51,52,91,92].

Table 2. Found and included papers.

Found Papers	Papers Included	Total Number of Reviewed Papers
[44,51,80,81,82,83,84,86,87,89,90,93,94,95,96,97,98]	[45,46,47,48,85,88,91]	24

Table 2. Found and included papers.

Found Papers	Papers Included	Total Number of Reviewed Papers
[44,51,80,81,82,83,84,86,87,89,90,93,94,95,96,97,98]	[45,46,47,48,85,88,91]	24

4.1. Summary of Methods

The found methods of segmentation or edge detection, based on aggregation and pre-aggregation functions, can be divided into three groups: (i) based on the aggregation of distance functions and FCM, (ii) multiple descriptors extraction and aggregation, and (iii) based on fuzzy set theory: type-2 fuzzy and neutrosophic sets.

Table 3. Summary of papers by method.

Multiple Descriptors Extraction and Aggregation	Based on Aggregation of Distance Functions and FCM	Based on Fuzzy Theory: Type-2 Fuzzy and Neutrosophic Set
[44,45,47,80,81,82,83,84,85,86,87,93,95,96,97,98]	[46,48,51,52,91]	[88,89,90,94]

lists the papers included in each group. Next, for each approach, we present a discussion of the related papers.

4.1.1. Multiple Descriptors Extraction and Aggregation

In [44,80,81,82,93], the authors present an edge detection method inspired by the way that the human visual system works. The central idea of this approach lies in the integration of several types of global or local information features, such as brightness, color, and the relationship between these descriptors. In these works, the authors model some descriptors from the color and luminance channels, which travel through the image pixel by pixel, which, in turn, generate auxiliary images that mimic the way information is processed in the retina and visual cortex, respectively. At the end, the images are aggregated by a function that considers, besides the objective responses of each descriptor, a direction vector that is obtained by the difference between the direction of each pixel in each of the channels. In this sense, each descriptor would have its own capacity to represent the visual clues or primitive shapes, which, when combined, delineate the location of the positive points of the edges of the objects in the image with more confidence.

This same principle was explored in [44], where the image feature extraction step aims to simulate the responses of the ganglion cells present in the eye, and the fusion of the information is the process that occurs in the visual cortex. In this case, the feature extraction step was carried out through the discrete difference between pixels in various directions, which, in short, is a type of first-order gradient detection. The aggregation of the different extracted features was carried out by using a function known as the Choquet integral [99] and its generalizations [43], which is a kind of pre-aggregation function [55].

The authors point out that these functions are best suited to the nonlinear profile of modeling how the prefrontal cortex acts in fusing visual stimuli. The Choquet integral generalized by a t-norm T [100], which is called the $C_T$-integral [55], is defined below. Note that, if the t-norm used is the product, the standard Choquet integral is obtained.

Definition 1 

($C_T$-integral). Let $\mathfrak{m}:2^N\to [0,1]$ be a fuzzy measure and $T:{[0,1]}^2\to [0,1]$ be a t-norm. Taking as a basis the Choquet integral, the $C_T$-integral is defined as the function ${\mathfrak{C}}_{\mathfrak{m}}^T:{[0,1]}^n\to [0,n]$, for all $x\in {[0,1]}^n$, given by:

$${\mathfrak{C}}_{\mathfrak{m}}^T\left(x\right)=\sum _{i=1}^{n}T(x_{\left(i\right)}-x_{(i-1)},m\left(A_{\left(i\right)}\right))$$

(4)

where $(x_{\left(1\right)},\cdots ,x_{\left(n\right)})$ is an increasing permutation on the input xl that is, $x_{\left(1\right)}\le \cdots \le x_{\left(n\right)}$, with the convention that $x_{\left(0\right)}=0$, and $A_{\left(i\right)}=\left(i\right),\cdots ,\left(n\right)$ is the subset of indices of the $n-i+1$ largest components of x.

The researchers sought to simulate the behavior of X (visual information for the near plane) and Y (visual information for the focus plane) ganglion cells; however, according to studies on the physiology of the visual system, there are Y and W (visual information from the far plane) ganglion cells in the visual system, which play essential roles in processing visual information.

Taking into account the importance of the w information, an edge detection method was proposed, which additionally considers modeling for the W cells, called DXYW [82]. In this model, three separate sub-models simulating the X, Y, and W channels are designed to process grayscale images and disparity maps, the latter responsible for the W visual information. Furthermore, a depth-guided multi-channel fusion method is introduced to integrate the visual response from different channels. The responses from each channel are fused by a weighted average function so that the product between each contribution from each channel is operated by a Hadamard product. The authors understand that DXYW achieves a competitive advantage detection performance that better maintains the integrity of object contours and suppresses background texture.

The use of depth information was also explored by [83] for the detection of region boundaries. According to the authors, detecting the border of salient regions of an image in an accurate way and distinguishing between objects is an extremely difficult task, especially when one has complex backgrounds or when the objects in the main plane and in the background have low contrast between each other. Therefore, depth information can provide visual clues to locate and distinguish these objects.

In both works [83,84], the model was obtained through a convolutional neural network, which, in the case of the latter, was justified by the attempt to simulate the human subjective visual attention guidance. Similarly, in [45], authors used the Choquet integral, embedded in the form of a neural network, to integrate the results of the segmentations generated by multiple deep neural network models that used the same basis, but considering, in the training stage, the manual segmentation of different experts. Manual annotations in medical image segmentation tasks usually present a certain inter-observer variability, even when performed by experts.

To increase the robustness of the segmentation system, the Choquet integral is proposed as a replacement of the fusion by majority vote. This same process could be used to segment only the border region, or edges of the image.

The importance of contrast for describing edges in the image becomes even clearer in the work of [85], where they proposed a local contrast function based on aggregation functions and fuzzy implications—what they call salient region detection—which is essentially a detection, or at least partial segmentation, of the front object of the image. The concept of local contrast as a measure of the variation with respect to the membership degrees of the elements in a specified region of a fuzzy relation was presented, along with a new construction method using the concept of consensus degrees among the input values. The concept of total contrast was also discussed as the aggregation of the local contrast values.

The segmentations of the front object regions of the image consist of thresholding from the detection of the highest contrast regions. To achieve this, they converted the image from RGB color space to CIELAB space and then each individual channel was fuzzified. After that, through a defined $m\times n$ window, the image was subjected to the local contrast process. Finally, all of the channels were merged and the processed image was automatically thresholded using Otsu’s method.

In addition to the information already presented as important for characterizing image edges, such as contrast and depth information, in [96], a method that was inspired by the focus-selective nature of the human visual system was introduced, which, when faced with a progressive blurring of a scene, and naturally having the sharpness of its tonal transitions diminished, deals with this problem by automatically compensating for the relevance of some edges compared to the rest of the image.

For that, two classes of self-adaptive vector fusion operators and the parameter variation of a Gaussian smoothing filter, followed by a gradient extraction on each of the images generated from the previous process, were proposed. Then, these intermediate images were fused into one image, which was then binarized to obtain the final edge image. One of the fusion functions developed was the ordered weighted gradient fusion operator given, for all gradient vector $\mathbf{D}\in {\left({\mathbb{R}}^2\right)}^n$, by:

$${\varphi }_h\left(\mathbf{D}\right)=(\sum _{i=1}^n{h}_i{\tilde{D}}_{i,x},\sum _{i=1}^n{h}_i{\tilde{D}}_{i,y}),$$

(5)

where $h=(h_1,\cdots ,h_n)$, with ${\sum }_{i=1}^n{h}_i=1$, is a weighing vector, and ${\tilde{D}}_i$ is the vector in $\mathbf{D}$ with the i-th greatest magnitude, whose horizontal and vertical components are ${\tilde{D}}_{i,x}$ and ${\tilde{D}}_{i,y}$, respectively. The use of a weighing vector h allows for the variation in the importance of the gradients to be fused. This operator is strongly inspired by Ordered Weighted Averaging (OWA) operators [101]. To be clear about the importance of varying blur using a Gaussian approach and how this relates to the way our visual system compensates for relevance and edges, it is enough to understand that a high level of detail should force the detection of small or not very significant objects, whereas a low one should lead to the extraction of (only) the most relevant silhouettes. In other words, a decrease in the level of detail should force an edge detection method to avoid objects that are too small or are part of a background texture. The aggregation of these variations in detail should strengthen the most relevant transitions and negatively strengthen the less relevant information by offsetting the boundaries between regions, thus improving the performance of edge detections.

An interesting result on the performance of applying information fusion methods for edge detection can be found in [80], where the authors discussed why the edge detection approach in combination with aggregation methods, or information fusion, is so important, and demonstrated that it performs better than other detection methods using information theory.

According to their research, fusion techniques aim to reduce information uncertainty and to minimize redundancy, improving reliability and maximizing information relevant to an objective task. The same image can arrive at different fusion results because of the fusion objectives and the priority of relevant information for a given task. The authors presented an unsupervised edge detection model as an optimization problem. In this sense, the final detection result was produced by the consensus between results provided by different detectors or by the same algorithm with varying parameters.

Considering that information aggregation has the fundamental role of reducing uncertainty or improving reliability and maximizing relevant information for the edge detection task, in [87], the authors proposed a methodology for aggregating the HSV (hue saturation and value) channels in order to improve the detection of the edges given by the Sobel and Canny methods.

The idea of this method is to select features through clustering that contribute to the desired detection. In [97], one has an edge detection algorithm based on a Bonferroni Mean-type pre-aggregation operator (BM), with more emphasis given to feature image extraction. A comprehensive comparative study was conducted to evaluate the results obtained using the proposed edge detection algorithm with some other well-known and widely used edge detectors in the literature, demonstrating the superiority of the proposed method regarding the aspects of accuracy and edge continuity. Additionally, it was shown that the correct selection of channels to fuse makes all of the difference in the result of edge detection.

In [95], the authors proposed two methods. The first consists of separating the RGB channels, followed by a directional edge detection using the Sobel method, one horizontal and one vertical, and then the concatenation of the layers to obtain a first edge detection, which then undergoes a transformation to YCbCr space. Then, this is followed by the application of the Canny detector to the R and Y channels, and, after, the application of their fusion. The second method simply consists of applying Canny to the G and B channels, followed by a fusion of these processed channels to generate the final edge detection. Then, the two fused detection results were compared and it was concluded that the second method produced visually better results for the base used. Despite this, the authors neither explained what type of fusion function they used nor provided metrics for a quantitative evaluation of their results.

The authors in [81] presented an algorithm called Improved Wavelet Modulus Maxima Degree of Polarization (IWMMDP), which is based on the fusion between two edge detection results, namely (i) that obtained from the light intensity and (ii) that obtained from the degree of polarization. Initially, the image was pre-processed using a noise suppression filter called Non-Local Mean denoising (NLM), and then the modulus of the gradient vector and amplitude of the variable angle of the intensities were calculated, followed by a Suppress Non-Maxima (SNM) and a thresholding process. Finally, edges were selected and connected through another thresholding process, and then aggregated with those edges obtained by the degree of polarization threshold. After the experiment, this technique presented a better-defined trace and better connectivity in relation to the classical methods. Assuming the pixel referring to the edge of the image produced by light intensity detection as $EA(i,j)$ and the one referring to the polarization degree image that considers the four directions $0^{\circ }$,${45}^{\circ }$, ${90}^{\circ }$, and ${135}^{\circ }$ as $EB(i,j)$, the edge fusion is given by:

$$\left\{\begin{array}{cc}E(i,j)={\alpha }_{1}EA(i,j)+{\alpha }_{2}EB(i,j),& \\ {\alpha }_1={\alpha }_2=0.5\end{array}\right.$$

(6)

where ${\alpha }_1$ and ${\alpha }_2$ are arbitrary weights.

The construction of an image feature extraction function was proposed in [47] using an Ordered Weighted Average (ODM) clustering function. The proposed method consists of building a feature map of the image based on the neighborhood information. The information taken from the neighborhood values is fused using the ordered aggregation function, which has, as a main feature, the ability to consider different directions of the variation in the intensity growth. Finally, this method was used in combination with different edge detectors, such as Canny, gravitational, and fuzzy morphology. The experiments showed that the use of MDGs performs competitively with respect to classical methods. On the other hand, the use of the approach that aimed to determine a consensus image when applying different methods resulted in a superior performance with respect to those tested.

Another method using local and neighborhood descriptors can be found in [98]. The method consists of first converting the image to grayscale, and then having a Gaussian smoothing followed by edge detection by the VLEP descriptor, which is a flexible circular edge detection descriptor that characterizes the local spatial structure of the different direction information. The direction of the edge is measured along the straight line passing through two zero points of the VLEP descriptor. This descriptor varies in all directions but depends on the radius and neighborhood parameters. Then, the final gradient is given by the root of the sum of the detections in all directions squared. Finally, a binarization is performed to obtain the final image. The fusion function is given in cases where it has descriptors with different parameters. The experimental results show that the method obtained a better performance in some aspects, such as continuity, smoothness, refinement, and localization accuracy.

A new method for multimodal medical image fusion based on Salient features for Fusion Discrimination (SFD) and fuzzy logic was proposed in [86]. First, the SFD method is employed to extract two saliency features for fusion discrimination. Then, two new fusion decision maps, called an incomplete fusion map and supplementary fusion map, are constructed from salient features. In this step, the supplementary map is constructed by two different fuzzy logic systems. The supplementary and incomplete maps are then combined to build an initial fusion map. The final fusion map is obtained by processing the initial fusion map with a Gaussian filter. Finally, a weighted average approach is adopted to create the final fused image.

4.1.2. Based on the Aggregation of Distance Functions and FCM

The methods based on the aggregation of distance functions and Fuzzy C-Means (FCM) [46,48,51,52,91] are established on the importance of distance functions as a criterion for making decisions about the belonging of an element to a certain group or segment in clustering algorithms.

All of the works presented here used the same base algorithm (FCM) in their experimental tests, this being one of the main limitations. The FCM segmentation method is a data clustering technique that allows each of the data points to belong to multiple clusters at the same time at certain degrees of pertinence. The algorithm is based on minimizing the following objective function [102]:

$$J_m=\sum _{i=1}^D\sum _{j=1}^N{\mu }_{i,j}^m{d}_{i,j}^2,$$

(7)

where

• D is the number of data points;
• N is the number of clusters;
• m is a fuzzy partition matrix exponent for controlling the degree of fuzzy overlap, with m > 1. Fuzzy overlap refers to how fuzzy the boundaries between clusters are; that is, the number of data points that have significant membership in more than one cluster;
• ${\mu }_{i,j}$ is the degree of membership of $x_i$ in the jth cluster. For a given data point $x_i$, the sum of the membership values for all clusters is one;
• $d_{i,j}^2$ is a measure of distance, which, in general, can vary according to the proposed approach, but is classically given by $\left|\right|x_i-c_j{\left|\right|}_A^2$, where ${\left|\right|.\left|\right|}_A$ is a norm.

A summary of methods based on the aggregation of distance functions or fuzzy measurements can be found in

Table 4. Edge detection using clustering and distance functions or fuzzy measurements.

Ref.	Aggregation	Measures		Clustering Algorithm
[91]	OWA	$d_S(p_1,p_2)=\frac{1}{255}|s_2-s_1|$ e $d_N(p_1,p_2)=\frac{1}{255}|n_2-n_1|$	1
[48]	GQ-AM	$C_{\omega }={\tau }^{\omega }.t^{\omega }$	2
[51]	EPP and EWAMP	$d_{\lambda ,\omega }(p_1,p_2)$	3	FCM
[46]	OWA and PP and WAMP	$d_{\alpha }{\omega }_{\left[5\right]},{\lambda }_{\left[5\right]}(p_{i,j},p_{k,n})$	3
[52]	AOOCC	$d_{{\alpha }_r,{\alpha }_g,{\alpha }_b}((r_1,g_1,b_1),(r_2,g_2,b_2))$; $d_r((r_1,g_1,b_1),(r_2,g_2,b_2))$; $d_g((r_1,g_1,b_1),(r_2,g_2,b_2))$ and, $d_b((r_1,g_1,b_1),(r_2,g_2,b_2))$	4

¹S_i,j represents the gray level of the pixel and ni,j the average gray levels in the eight-connected neighborhood; ² τ and t described in Equations (25) and (26), where F_i are the normalized color components, D_i is a similarity descriptor, and K a coefficient of adjustment; ³ Equation (16), where ω and λ are adjustment parameters; ⁴ Equations (12)–(15).

. The first column contains the reference, the second the name of the aggregation function, then the used fuzzy measure, and finally the adopted clustering method.

In general, the aggregation of different distance functions in combination with FCM allows for the segmentation of important regions of the image, which, in turn, can be used for edge determination (see Section 2.3). Next, for each of the papers, considering the distance-based approach, the main differences between the studies are pointed out.

In [91], the authors presented a review of the Ordered Weighted Average function (OWA), which is a type of aggregation operator. OWA is used to initially aggregate two absolute distance functions that consider:

(i) 

The $d_S$ intensity of pixels $p_1$ and $p_1$, given by:

$$d_S(p_1,p_2)=\frac{1}{255}|s_2-s_1|$$

(8)

and

(ii) 

The average pixel in an eight-connected neighborhood, calculated as

$$d_N(p_1,p_2)=\frac{1}{255}|n_2-n_1|.$$

(9)

As a result, they obtained the distance function, defined below, where $\alpha$ is an adjustment parameter:

$$d_{S,N;\alpha }(p_1,p_2)=\alpha d_S(p_1,p_2)+(1-\alpha )d_N(p_1,p_2).$$

(10)

Three other functions analogous to Equation (8) were proposed for the intensity of each channel of the RGB color space, resulting in the following distance function:

$$d(p_1,p_2)=\frac{\alpha }{255}|c_2^1-c_1^1|+\frac{\beta }{255}|c_2^2-c_1^2|+\frac{1-\alpha -\beta }{255}|c_2^3-c_1^3|$$

(11)

where $|c_2^1-c_1^1|$, $|c_2^2-c_1^2|$ and $|c_2^3-c_1^3|$ are, respectively, the minimum, average, and maximum difference in each individual RGB channel of the pixels $p_1$ and $p_2$, and $\alpha$ and $\beta$ are arbitrary parameters.

Using the difference between the gray levels in each color channel as descriptors and varying a parameter $\alpha$ to fit the results, also using FCM as the clustering method, the authors in [52] used the Aggregation Operator Of Convex Combination (AOOCC) to achieve the results. The aggregated distance is defined as:

$$d_{{\alpha }_r,{\alpha }_g,{\alpha }_b}((r_1,g_1,b_1),(r_2,g_2,b_2))=\sum _{i\in \{r,g,b\}}{\alpha }_i{d}_i((r_1,g_1,b_1),(r_2,g_2,b_2))$$

(12)

where

$$d_r((r_1,g_1,b_1),(r_2,g_2,b_2))=\frac{1}{255}|r_1-r_2|$$

(13)

$$d_g((r_1,g_1,b_1),(r_2,g_2,b_2))=\frac{1}{255}|g_1-g_2|$$

(14)

$$d_b((r_1,g_1,b_1),(r_2,g_2,b_2))=\frac{1}{255}|b_1-b_2|$$

(15)

A descriptor, motivated by the Local Binary Patterns descriptor family (LBPs), widely used in the literature for texture analysis, or its variation, called a Shift Local Binary Pattern (SLBP), are other methods for constructing distance functions [51]. For that, the authors introduced two extended aggregation functions: Extended Powers Product (EPP) and Extended Weighted Arithmetic Mean of Powers (EWAMP), which were used to construct the proposed distance functions. Then, the aggregated distance function is defined by:

$$\begin{array}{c}\hfill d_{\lambda ,\omega }(p_1,p_2)={\lambda }_1{d}_1^{{\omega }_1}(r_1,r_2)+{\lambda }_2{d}_1^{{\omega }_2}(g_1,g_2)+\\ \hfill {\lambda }_3{d}_1^{{\omega }_3}(b_1,b_2)+{\lambda }_4{d}_2^{{\omega }_4}(s_1,s_2)+{\lambda }_5{d}_2^{{\omega }_5}(q_1,q_2)\end{array}$$

(16)

where $\omega$ and $\lambda$ are fitting parameters, and $d_1$ and $d_2$ are distance functions given by:

$$d_1(c_1,c_2)=\frac{1}{255}|c_1-c_2|$$

(17)

$$d_2(t_1,t_2)=|t_1-t_2|$$

(18)

The authors came to the conclusion that, when using the appropriate selection of parameters, the aggregated distance function produces good results in the application in images, and the similarity descriptor between the pixels and their neighbors also contributes significantly to the quality of the segmentation.

The application of three aggregation functions, OWA, Powers Product (PP), and Weighted Arithmetic Mean of Powers (WAMP), was proposed in [46]. The constructed distance function is given by:

$$\begin{array}{c}\hfill d_{\alpha }{\omega }_{\left[5\right]},{\lambda }_{\left[5\right]}(p_{i,j},p_{k,n})={\omega }_{5,1}{d}_1^{{\lambda }_{5,1}}(r_{i,j},r_{k,n})+{\omega }_{5,2}{d}_1^{{\lambda }_{5,2}}(g_{i,j},g_{k,n})\\ \hfill +{\omega }_{5,3}{d}_1^{{\lambda }_{5,3}}(b_{i,j},b_{k,n})+{\omega }_{5,4}{d}_2^{{\lambda }_{5,4}}(S\left(p_{i,j}\right),S\left(p_{k,n}\right))+{\omega }_{5,5}{d}_2^{{\lambda }_{5,5}}(I{C}_{\alpha }\left(p_{i,j}\right),I{C}_{\alpha }\left(p_{k,n}\right))\end{array}$$

(19)

where, as descriptors, one has the pixel saturation S given by

$$S\left(P_{i,j}\right)=\frac{1}{255\sqrt{3}}\sqrt{r_{i,j}^2+g_{i,j}^2+b_{i,j}^2}\in [0,1],$$

(20)

the neighborhood average $NAC$ given by

$$NAC\left(p_{i,j}\right)=\frac{1}{3}(NA{C}^{\left(R\right)}\left(p_{i,j}\right)+NA{C}^{\left(G\right)}\left(p_{i,j}\right)+NA{C}^{\left(B\right)}\left(p_{i,j}\right))$$

(21)

where

$$NA{C}^{\left(c\right)}\left(p_{i,j}\right)=\frac{1}{8}(\sum _{k=j-1}^{j+1}{c}_{i-1,j}+c_{i,j-1}+c_{i,j+1}+\sum _{k=j-1}^{j+1}{c}_{i+1,j})$$

(22)

and a neighborhood-oriented similarity descriptor $I{C}_{\alpha }^{\left(c\right)}$ given by

$$I{C}_{\alpha }^{\left(c\right)}\left(p_{i,j}\right)=\sum _{m=1}^8{I}_{p_{i,j};\alpha }^{\left(c\right)}\left(m\right)$$

(23)

where

$$I_{p_{i,j};\alpha }^{\left(c\right)}\left(m\right)=\{\begin{array}{ccc}1\hfill & \hfill |c_{i,j}-c_{i,j,m}|\le \alpha & \\ \hfill & \hfill & ,withm\in \{1,\cdots 8\}\hfill \\ 0\hfill & \hfill |c_{i,j}-c_{i,j,m}|>\alpha \end{array}$$

(24)

and $\alpha$ is a chosen threshold.

Considering as attributes, for the determination of the measure, the difference in the measures of the components in the color space and the value of a descriptor of the pixels that incorporates the calculations of the spatial relations with each neighbor, in [48], the authors used as an aggregation function an example of the Generalized Quasi-Arithmetic Mean (GQ-AM).

The composition of the measurements was performed by the product t-norm and the metrics generated for the experiments were incorporated into the FCM as a difference in the measure for every two observed pixels. The measurements took into account the local binary pattern descriptor LBP, $C_{\omega }={\tau }^{\omega }·t^{\omega }$, where $\tau$ and t are given by:

$$\tau (F_i,F_j,K)=\prod _{l\in (R,G,B)}\frac{\frac{F_i^{\left(l\right)}+F_j^{\left(l\right)}}{2}+K}{max{F}_i^{\left(l\right)},F_j^{\left(l\right)}+K}$$

(25)

$$t(D_i,D_j,K)=\frac{K}{K+|D_i-D_j|},$$

(26)

where $F_i$ are the normalized color components, $D_i$ is a similarity descriptor, and K is a fitting coefficient.

4.1.3. Based on Fuzzy Set Theory: Type-2 Fuzzy and Neutrosophic Set

The representation of different feature classes of an image, pixel level, local level, and global level, through triangular interval type-2 fuzzy numbers was performed in [88]. The operations on the images and edge detection were performed by aggregating these numbers by a type of interval fuzzy triangular set aggregation function. The membership functions, upper ($UM{F}_{\overline{M}}\left(x\right)$) and lower ($LM{F}_{\overline{M}}\left(x\right)$), of a type-2 triangular interval fuzzy set, is represented by a triangular fuzzy number $\overline{M}=\langle [\underline{l_M},\overline{l_M}],C_M,[\underline{r_M},\overline{r_M}]\rangle$, called TIT2FS, where:

$$LM{F}_{\overline{M}}\left(x\right)=\left\{\begin{array}{cc}\frac{x-\overline{l_M}}{C_M-\overline{l_M}}\hfill & \mathrm{if}\overline{l_M}\le x<C_M\hfill \\ 1\hfill & \mathrm{if}x=C_M\hfill \\ \frac{x-\underline{r_M}}{C_M-\underline{r_M}}\hfill & \mathrm{if}{C}_M\le x<\underline{r_M}\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(27)

$$UM{F}_{\overline{M}}\left(x\right)=\left\{\begin{array}{cc}\frac{x-\underline{l_M}}{C_M-\underline{l_M}}\hfill & \mathrm{if}\underline{l_M}\le x<C_M\hfill \\ 1\hfill & \mathrm{if}x=C_M\hfill \\ \frac{x-\overline{r_M}}{C_M-\overline{r_M}}\hfill & \mathrm{if}{C}_M\le x<\overline{r_M}\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(28)

The authors developed an edge detection application of a three-dimensional image of a patient’s brain, reconstructed through a sequence of DICOM images, whose pre-processing is filtered by two directional gradients, and, after the aggregation of the triangular numbers, the method performs the edge detection. However, the authors did not make it totally clear how the fuzzy triangular numbers are generated from the image information to finally perform the aggregation and edge detection.

A very similar contribution of a fuzzy edge detection method was presented in a later work [89], where the computation of the image gradient was performed using a t-norm. In short, the minimum and maximum were chosen for the purpose of morphological operations. The mathematical properties of the aggregation operations for representing all morphological operations using the Triangular Interval Type-2 Fuzzy Yager Weighted Geometric (TIT2FYWG) and the Triangular Interval Type-2 Fuzzy Yager Weighted Arithmetic (TIT2FYWA) were deduced. These properties represent the components of the image processing.

Edge detection was performed for a DICOM image by converting it into a two-dimensional grayscale image. The architecture of the method consists of defining the triconsistsangular norms, defining the properties of the clustering operation using type-2 interval fuzzy triangular numbers, converting the DICOM image into a two-dimensional grayscale image, applying the gradient using the triangular norms, obtaining the image segmentation, and applying the edge detection using type-2 fuzzy logic.

Another type-2-fuzzy-logic-based edge detection method was presented in [90]. The general process consists of obtaining the image gradients in four directions: horizontal, vertical, and the two diagonals. After this, a generalization of the type-2 fuzzy Sugeno Integral is used to integrate the four image gradients. In this second step, the integral establishes the criteria for determining which level of gradient obtained from the image belongs to an edge during the process; this is calculated by assigning different fuzzy densities of type-2, and these fuzzy gradients are aggregated using the meet and join operations.

Gradient integration using the Sugeno integral provides a methodology for achieving more robust edge detection, especially when one is working with blurred images. The experimental analyses were performed using real and synthetic images, and the accuracy was quantified using the Pratt’s Figure of Merit. The evaluated results show that the proposed edge detection method outperforms existing methods.

Adopting the neutrosophic set theory, the paper [94] developed a generalization of clustering operators. In this study, the grayscale images were first transformed into the Linguistic Neutrosophic Cubic Set domain using three degrees of pertinence, fuzzy on each channel, and then this information was aggregated using the proposed operators, which were: the Generalized Linguistic Neutrosophic Cubic Weighted Average (GLNCWA) and Generalized Linguistic Neutrosophic Cubic Weighted Geometric (GLNCWG). The variation in the clustering operators enables different processing operations on the image, including edge detection.

According to the authors, these methods clarified the noise in the tested images, which would justify the use of this approach. To demonstrate their claim, they considered several types of noise, such as Gaussian, Poisson, and speckle. The authors compared the computational efficiency of the proposed method with existing methods, and although the results show that the approach consumes less memory and is faster than the compared methods, the research did not consider a data set that presented a variability of conditions of light, shadow, shapes, and textures in order to reach a sufficiently adequate conclusion regarding the applicability of the method.

In summary, we can divide the methods presented here into three groups:

(i) 

Multiple descriptors extraction and aggregation;

(ii) 

Based on the aggregation of distance functions and FCM;

(iii) 

Based on fuzzy set theory: type-2 fuzzy and neutrosophic sets.

The first group considers methods inspired by the way that the human visual system works. The idea is the integration of different global or local features, where each descriptor would have its own capacity to represent the visual clues or primitive shapes that compose the edges of the composite shapes of the objects present in the image. In this sense, the aggregation functions model the behavior of the information processed in the cortex, and the descriptors play the role of the stimuli generated by the ganglion cells.

Each theorist proposes a different type of aggregation or pre-aggregation function to simulate the behavior of information fusion, and there is no consensus on this. Although there are no studies demonstrating which type or family of clustering or pre-clustering functions would be best suited for modeling feature fusion for edge detection purposes, some researchers directly argue that Choquet integrals would serve this purpose better, since they would better model the non-linear behavior of the problem. Another clue to the Choquet integral performance is possibly the use of a weighing vector that allows for the variation in the importance of the information to be fused, as well as some other functions presented here. It is noteworthy that only average functions were used in all papers, whether arithmetic-weighted or geometric-weighted.

Regarding the descriptors, taking into account that most of the works considered modeling the X and Y-type cells, i.e., the visual information close to the image plane and the focal plane information, and the biological and performance justification that exists in the use of modeling for the visual information far from the plane, i.e., the modeling of W-type cells, it could be concluded that the use of depth data, such as disparity maps, laser sensor depth maps, and the like, is fundamental, especially when working with images with complex backgrounds or low contrast. The importance of contrast as a descriptor for image edges becomes clear when one observes the features used by the researchers.

The edge detection approach in combination with aggregation methods, or information fusion techniques, is important because it enables a reduction in information uncertainty and minimizes redundancy, improves reliability, and maximizes the information relevant to a task. This allows for the approach of consensus detection, via aggregation, between different detectors or the same detector by varying parameters. Beyond this, features can also be used as information extractors, so that the image features themselves are identified using aggregation. Works that link extraction and detection with aggregation functions have not been found.

In summary, we can compile the results of this approach:

(i) 

These are methods that take direct inspiration from the way that the human visual system works, seeking to simulate, through feature extraction or the determination of local or global descriptors, the visual stimuli and the processing of information for the recognition of some pattern through aggregation or pre-aggregation functions.

(ii) 

Functions that make use of weighing vectors and that enable a better modeling of nonlinear behavior, such as the Choquet integral, would perform better.

(iii) 

Depth information is essential, especially when working with images with complex backgrounds or low contrast, the latter being an important feature in the performance of the models.

(iv) 

Combining detection methods with aggregation methods or information fusion makes it possible to reduce information uncertainty and minimize redundancy, improve reliability, and maximize the information relevant to a task.

(v) 

Works that link extraction and detection via clustering functions have not been found.

The second set of techniques (based on the aggregation of distance functions and FCM) is based on the importance of distance functions as a criterion for decision making regarding the belonging of an element to a certain group or segment in clustering algorithms. Through a set of distance functions or fuzzy measures, using a set of aggregation or pre-aggregation functions, it is possible to build new distance functions that can be used as a parameter of clustering algorithms such as FCM. After this segmentation, it is possible to apply a morphological operation or classical detection to obtain the contours of the main objects of the image. One of the main limitations of this work is that only experiments using FCM were performed, since it theoretically allows for implementation in any clustering algorithm, or, in general, uses a distance metric as a stopping or starting criterion.

In summary, this approach has the following characteristics:

(i) 

They are based on the importance of distance functions as decision criteria in data clustering algorithms;

(ii) 

Aggregation and pre-aggregation functions are used to construct new distance functions from others that represent some dimension relevant to the problem;

(iii) 

The papers are limited to application to a single clustering algorithm and do not make the performance of other methods clear.

The third group of techniques (based on fuzzy set theory: type-2 fuzzy and neutrosophic sets) performs an interpretation of the image in terms of fuzzy logic. The gray levels of the image are treated as a variation on a fuzzy set. This allows for the direct use of aggregation functions. Overall, this technique has proven to be quite robust and superior to those compared in their respective works, particularly with respect to noise sensitivity.

5. Conclusions

Edge detection represents an important task in several steps of computer vision and image processing, ranging from object detection to image retrieval. Studies in this area of research have been gaining momentum in recent years due to its importance in applications such as autonomous cars and augmented reality. The difficulties inherent to the task have required a continuous improvement of techniques and modeling.

Some approaches applied to solve this task considered using aggregation and pre-aggregation functions. Due to the lack of works that summarize and show the gaps in the area of edge detection by functions of this type, this paper developed a systematic review of the literature, which aimed to answer:

“Which edge detection methods use aggregation and pre-aggregation functions?”

Considering a review of 24 papers filtered from 428 articles found in computer databases in the last seven years, it was possible to synthesize important information that was grouped into three approaches, which support the answering of the research question. Precisely, the groups are:

(i) 

Multiple descriptors extraction and aggregation;

(ii) 

Based on the aggregation of distance functions and FCM;

(iii) 

Based on fuzzy set theory: type-2 fuzzy and neutrosophic sets.

In summary, through this systematic literature review research, it was possible to:

(i)

Fill an existing gap in the literature of systematic reviews on edge detection using clustering functions;

(ii)

Summarize the existing technology regarding methods that make use of these functions in edge detection;

(iii)

Identify the gaps in the detection approach using aggregation or pre-aggregation functions, proposing topics for future work and fulfilling the initial objectives of the research.

As future works, some paths have emerged from this work. It is proposed to perform a quantitative analysis of the methods, comparing the metrics provided by researchers who used the same databases for validation, as well as replicating the methods that did not present metrics with the same datasets in order to answer which type of aggregation or pre-aggregation functions presents a better performance in the task of edge detection.

Another suggestion is the development of works that aim to fill the gaps presented here, such as:

(i) 

The application of the method of constructing distance functions by clustering functions in different data clustering techniques, such as DBscan, region growth, K-means, and others;

(ii) 

Applying non-average aggregation functions for edge detection;

(iii) 

The development of further work involving the modeling of W ganglion cells; in this sense, work with depth information, such as lidar sensors and others;

(iv) 

Exploring the combination of aggregation functions, both in feature extraction and in information fusion;

(v) 

The direct use of classical detectors in an ensemble fused by aggregation functions, also taking into account the fusion of descriptors and other visual cues, thus ensuring the participation of primitive shapes and the influence of contrast, color, and depth information, among others discussed here.