Information Extraction from Retinal Images with Agent-Based Technology

Abstract

The study of retinal vessels can provide information on a wide range of illnesses in the human body. Numerous works have already focused on this new field of research and several medical software programs have been proposed to facilitate the close examination of retinal vessels. Some allow for the automatic extraction of information and can be combined with other clinical tools for effective diagnosis and further medical studies. This article proposes an Agent-based Virtual Organizations (VO) System which applies a novel methodology for taking measurements from fundus images and extracting information on the retinal vessel caliber. A case study was conducted to evaluate the performance of the developed system, and the fundus images of different patients were used to extract information. Its performance was compared with that of similar tools.

1. Introduction

Fundus examination is a noninvasive technique which consists of the assessment of blood vessels in the eye from an image. The importance of these images lies in the information they contain, as different medical studies have related the caliber of retinal vessels with different pathologies. Fundus image analysis is challenging for several reasons, such as: (i) the intertwining of numerous blood vessels; (ii) image tone, which is affected by the amount of light present when the photo is taken and may result in variability between two images of the same eye; (iii) the difficulty in differentiating arteries from veins, even when they are evaluated by an expert; and (iv) the lack of validated procedures when measuring retinal blood vessels. However, there are also some advantages to consider; thanks to morphological relationships, it is possible to identify and analyze different components in the images. To be able to identify those relationships, it is first necessary to extract the information through the use of systems that measure different vessel factors. The present work proposes a supervised automatic software system as a support tool in the research on the relationship between diseases and the blood vessels of the eye. The software architecture is based on a Multi Agent System (MAS) running on the Platform for Automatic coNstruction of orGanizations of intElligent Agents (PANGEA) [1]. The case study, in which different fundus cameras were used to take 50 retinal images of several patients, has demonstrated that the system works correctly.

Although several previous studies have intended to develop tools or techniques that would determine the thickness of blood vessels, such as [2,3,4,5], human intervention is required in all of them. There are also different tools that attempt to deal with the analysis of retinal vessels, most of them, [6] or [7], focus on specific eye pathologies. In other research, the amount of extracted information is insufficient, such as [8,9,10,11,12], or the extracted measurements depend on the user who analyzes the images [5].

In addition, the tools used in previous works are not open source, they lack flexibility and therefore cannot be expanded or integrated with other types medical applications. As a result, it is necessary to design a new tool, which will not only be capable of following existing methodologies but will also allow medical researchers to extract new parameters and propose new methods.

This study presents a new methodology that integrates, modifies and adapts existing image analysis techniques. All of the algorithms are described so that they can be freely implemented and adapted to any other tool. In addition, thanks to its design based on Virtual Organizations (VO) of agents, the modules of the presented tool can be modified or extended for the inclusion of new techniques.

Artificial Intelligence (AI) has been used in the development of the present software tool in order to make it capable of performing analysis even if the images are very different from one another due to the large number of factors. Compared to traditional techniques, AI provides greater flexibility when analyzing images. The survey presented in [13] demonstrates increased precision thanks to the evolution of AI techniques.

In this work, an MAS-based VO was designed, where agents with specific knowledge about every analysis stage collaborate with each other [14]. We present a novel methodology focused on retinal analysis, where the agents do most of the work that is normally done by an expert user. Agents perform different image processing techniques, and detection and extraction of the different measurements from the blood vessels (associated with every parameter).

The information extracted by the platform from the fundus images has been validated by the staff of the Research Unit from Centro de Salud de La Alamedilla (Sacyl) [15], and a validation protocol was published in [16]. Therefore, the main contribution of this article is a validated methodology for semi-supervised and accurate extraction of different measurements.

The remainder of the article is structured as follows: the next section provides a comprehensive review of related state-of-the-art works and a review of VO of agents. Subsequently, the proposed architecture is presented; then, the agent platform and the algorithms and techniques that make up this methodology are all described one by one in great detail. Finally, the results obtained from the case study are outlined, conclusions are drawn from them and plans for valuable future work are provided.

2. Fundus Image Analysis

A fundus camera provides clear images of the retina. This noninvasive technique makes it possible to see blood vessels. Upon analyzing the image, a structure associated with the blood vessels can be achieved separately from the rest of the components so that their characteristics can be extracted. This vascular structure is composed of veins and arteries which must be identified when detected. Once this information is obtained, it can be associated with existing knowledge about certain pathologies [17]. More specifically, studies can relate the caliber of retinal blood vessels with arterial hypertension [18], metabolic syndrome [19], left ventricular hypertrophy [20], stroke [21] and coronary heart disease [22].

Evidently, this relationship is not a recent discovery. In fact, retinal images have been analyzed for decades, and were initially analyzed manually, as in [23,24].

As a result of the considerable advances in computer vision, there has been greater interest in creating a tool for automatic extraction of information from those images [25]. Tools such as [26] apply algorithms based on adaptive filtering techniques to highlight the difference between the background and the blood vessels structure.

Studies such as [27,28,29] propose the application of morphological methods, which are widely used when extracting features of images whose shape is known beforehand. Specifically, these methods have been very successful when used to obtain vasculature segmentation applied to the detection of micro-aneurisms. An image preprocessing algorithm which can automatically detect exudates in retinal images is proposed in [6]. The information extracted in these works is not sufficiently complete as the information that the present work is intended to extract.

In addition to the previously mentioned tools, there are others; the Singapore I Vessel Assessment (SIVA) software tool [11] which allows for semi-supervised measurement of many retinal vessel parameters, such as vessels mean width, length, branching coefficient and angles, simple and curvature tortuosity, or fractals. It is the most complete tool on the market, however, unlike the tool proposed in this work, it does not provide information about the arteriovenous index (AVindex [30]) by area, which, according to the medical team involved in this work, is one of the most important parameters for retinal vessel measurement. Moreover, neither the article which originally proposed the SIVA tool nor other works which also analyzed it [31]) provide any type of experimental results. In addition, the methodology and the tool are not open source, so it was not possible to to compare its performance with the platform proposed in this work.

Methods to detect diabetic retinopathy have also been proposed, for example in [7], where a four-block division of the analysis is presented, including ‘Image preprocessing’, ‘Shape estimation’, ‘Feature extraction’ and ‘Classification’. The authors in [8] present a set of techniques when measuring blood vessels, also focused on the task of detecting diabetes, in addition to other pathologies such as hypertension or premature retinopathy.

The medical team involved in this development was also involved in the development of the tool called ‘AVIndex’, introduced in [10], which obtained information similar to that desired with the present project; however, it is incomplete, uses a smaller analyzed area, and is less accurate in extracting the vasculature.

A previous version of this tool was presented in [32], which was used to validate that the applied methodology was able to extract the correct parameters. In that case, the analysis was time-consuming for the supervising user, whose participation and decisions were required at every step. Based on that application design and additional measurement techniques, this new methodology was designed and developed.

A review of existing tools that apply artificial vision to the analysis of fundus images is presented in [33], which divides the analysis process into three different stages: preprocessing, location of the optic disc, and segmentation of the vasculature. Techniques applied to each section are then analyzed. Contrast enhancement techniques are the most commonly applied in the preprocessing stage (7 articles of 19). In the location of the optic disk stage, algorithms analyzing the position of vessels are the most applied (6 articles of 38). Finally, for the segmentation of vasculature stage, vessel tracking algorithms are the most applied (17 articles of 62).

A more recent review is presented in [13], which in addition to a new methodology provides a comparative of 26 recent articles addressing the topic of blood vessel segmentation (it does not include the information extraction phase), where it is shown that the accuracy in the segmentation ranges between 87.73% and 95.36% of its proposal. The results of our methodology place it among the top three of those that had been evaluated.

3. Virtual Organizations

Software agents are able to adapt to the context of a problem at any given moment due to their specific characteristics. In the case of a fundus image analysis, in which there are differences among the images, despite components follow standard criteria, a decision-making process could be very useful. To this end, agents have to be specialized in one single functionality providing all their knowledge when required.

Since there are different stages and components that must be detected when analyzing the fundus images, the present study proposes an architecture based on an MAS where different agents cooperate to reach a common goal of extracting the hidden information in blood vessels. As a result, every agent is able to apply the same technique in a different way at the same stage, thus allowing the system to choose the proposal that provides the best results in every stage, and to change the input parameters by adapting them for every image.

As a result, every agent can apply the involved techniques to solve a stage in a different way or with different values than other agents trying to solve the same stage, so the system can choose the proposal with the best provided results in every stage and also is able to change the input parameters adapting them for every image.

In addition, agents are able to learn with the history of patients so they can refine that input parameters with every new image. In this way, results will be more efficient as the patient database increases [34].

This structure also allows the possibility of integrating the VO with others specialized in processing the extracted information in order to find relationships between this information and other diseases. This integration is the next phase of this project.

4. Automatic Image Analyzer to Assess Retinal Vessel Caliber (ALTAIR): A Virtual Organization Based Fundus Image Analyzer Platform

As mentioned, this paper uses the tool presented in [32] as its starting point and develops it further. The tool takes a series of steps to obtain the parameters form the fundus images that are associated with blood vessels. The clinical validity of the developed tool has been confirmed and its method of information extraction works; however, the image analysis results were not good enough and it was necessary for a human expert to intervene considerably in the process.

Once the parameters associated with the blood vessels (length, area and thickness) were identified, the methodology of analysis was improved with automatic detection (however, an unsupervised system would not be able to guarantee 100 accuracy rate).

MAS technology has been leveraged for the creation of this platform because of its ability to dynamically interpret the information hidden in every image; agents provide different values (for the required parameters) so that the best result can be found for every image, at each step of the methodology. Throughout this section, two aspects of the proposed system are described. The first one is focused on a VO-based solution for agents. The second one is focused on the algorithms, specifically designed for the proposed methodology. The algorithms only use the techniques that have the highest performance in image analysis.

4.1. Virtual Organizations

When defining the agents and the system structure, it is necessary to consider the platform on which they are going to be deployed. A new version of PANGEA [1] was used to develop the platform. Its main advantage is that it integrates a wide range of functionalities, such as self-managed database access or the ability to easily define new agents and VO within the system. Thanks to PANGEA, developers do not have to worry about basic functionalities and may fully focus their attention on more specific functionalities, necessary for resolving the problem.

Furthermore, it is a multi-language programming platform capable of running in Cloud Computing (CC) environments, which makes it possible to publish an online tool that provides enough resources for parallel image processing. This will result in a bigger database with even more data, thus improving the accuracy and the knowledge of the system.

One of the designed VOs is responsible for extracting information from the image. It is composed of several agents which fulfill 14 different roles. Specifically, the agent with the Coordinator role is responsible for coordinating the flow of the analysis and for establishing rules. The agent with the Image role represents the current status of an image and is updated in every step of the flow. One of the agents performs four roles related to the techniques applied at every step of the analysis; it decides the best set of techniques to achieve the best result. These four roles are: Location, Segmentation, Skeleton, and Identification. The agent with the Measurement role extracts the parameters obtained in the analysis. The remaining roles are related to different AI and image analysis techniques, which are implemented by different agents to provide values for each of the input parameters of the algorithms, thus allowing them to select options such as using the mean, mode, maximum values or minimum values. Several agents may perform this task simultaneously; each one will intend to solve the problem using different values; however, only the best results, provided by any of the agents, are considered.

The remainder of this section provides a description of all the agent roles that are performed every time a new image is processed, as seen in Figure 1.

Coordinator: this role is implemented by a single agent responsible for keeping track of the entire analysis process—from the moment the image is received until the extracted information is presented. It regulates the analysis process and the communication between the system’s agents and it is in charge of obtaining the inputs for the other agents of the system.

Image: this role is implemented by a single agent that keeps track of the status of the image at every moment. The agents associated with each step are responsible for updating the status of the image by communicating it through the Coordinator Agent. Initially, an image is uploaded and, if the image agent validates that the image is correct, it notifies the Coordinator agent that the analysis process should start. Subsequently, the Coordinator agent notifies the Location agent to start its process. When a new algorithm requires preprocessed images, the Image agent is in charge of providing images from previous stages that have already been preprocessed by other algorithms.

Location: this role is used by agents whose purpose is to locate: (i) the macula; (ii) both retina edges and the optic disc. In these cases, the analysis is not as complex as in the following steps; consequently, a single agent is responsible for executing the task of locating the macula and another agent is responsible for locating the optic disk. The macula detection process is described in Algorithm 1 (Section 4.2.1), the retina edge detection process is described in Algorithm 2 (Section 4.2.2), and the optic disc detection process is described in Algorithm 3 (Section 4.2.2).

Segmentation: this role is implemented by four agents whose input values are the mean, mode, maximum values and minimum values, as mentioned previously. Its goal is to present a structure that represents the segmentation of the vessels by dismissing the background and allowing the agents to process the vessels separately in subsequent steps. Every agent executes the step and applies the methodology using the required techniques and providing its results as output. Further description is presented in Section 4.2.3 and the blood vessel segmentation process is described in Algorithm 4.

Skeleton: this role is implemented by a single agent which is responsible for extracting the skeleton from the blood vessels structure obtained as output in the previous step. All the details are presented in Section 4.2.3 and the process performed to obtain the skeleton of the blood vessels is described in Algorithm 5.

Identification: this role is implemented by four agents whose inputs are also based on the mean, the mode, the maximum values and the minimum values. Their main goal is to identify every detected blood vessel by classifying them as either an artery or a vein. The only useful difference among the analyzed vessels is their color tone. However, identification is not an easy issue as the image illumination is not uniform and can vary according to the zone. To identify the vessels, the methodology (detailed in the next section) is applied in parallel by the agents, each using their own technique. The results are provided to the Coordinator Agent. All the details are presented in Section 4.2.4 and the blood vessel identification process is described in Algorithm 6.

Measurement: this role is implemented by the agent responsible for extracting the required parameters. These parameters are specified by the medical team. The thickness, length, and area of every vein and artery were identified as the data required for our analysis. The measurement process is fully detailed in Section 4.3.

Other roles representing techniques: there are other roles related to the technologies that can be applied at each stage of the analysis. Every agent receives a series of arguments representing which they use as the input parameters for the algorithm. For example, some of the agent roles are NoiseRemove, Contrast, Binarizing, GrayScale, MorphologyFilter, VesselMorphology or Blur.

When evaluating the results provided by the agents at every step, it is important to bear in mind that there are several characteristics of retinal images that must be satisfied. As highlighted in the introduction, one of the few advantages of fundus analysis is the morphological relationship between some of the components. For example, we know that the number of arteries should be slightly higher than the number of veins, or that there should be at least three or four veins in the image. One of the agents with the Identifier role will then show the most accurate results which will be used to update the image status.

Therefore, the main advantage of the VOs approach of the proposed software is that the all agents perform image processing simultaneously (each agent applies filters sequentially, so this process cannot be distributed in any other way). Thus, VOs allow for obtaining better results in a shorter period of time than if it were a single sequential execution.

4.2. Information Extraction Methodology

Having described the architecture of the system, we now proceed to the methodology used to generate the results at each step. The set of techniques and algorithms used in the methodology is detailed below.

To gain a better understating of this methodology, the process that image goes through from the time it enters the system has been divided into steps. Figure 2 shows a schema of those steps and the analysis techniques used to obtain the desired results at every step. The output of every step defines the status of the image, which is the input for the next step.

This section describes the steps involved in visual analysis that lead to the removal of information from blood vessels.

4.2.1. Image Selection (Eye Side Detection)

Once the patient’s profile has been created in the system, the system associates new images with that patient. The moment the user selects an image with a patient’s retina, the system initially starts with the first stage of analysis.

The first stage consists of identifying the eye (left or right); this task is performed by the Location Agent. It locates the macula for this purpose. In all the processed images, the optic disc is near the center of the image, so the macula must be located on either side, at a medium height. The macula can be detected as a darker area in the external part of the eye, so, when detected on the left side of the image, it corresponds to the right eye and vice versa. Algorithm 1 shows how the image provided by the software is processed, while Figure 3 shows the different parts of the image that Algorithm 1 uses, and the different parts of the eye.

Algorithm 1: Eye side detection

// w and h are the image I width and height  Input: $I,w,h$  Output: $S\subseteq I$  // $I_{ij}\in \mathbb{N}\times \mathbb{N}\times \mathbb{N}$  $I^1\leftarrow \{I_{ij}/i\ge w\times 0.2,i\le w\times 0.5;j\ge h\times 0.35,j\le h\times 0.65\}$;  $I^2\leftarrow \{I_{ij}/i\ge w\times 0.5,i\le w\times 0.8;j\ge h\times 0.35,j\le h\times 0.65\}$;  // $G^1\in \mathbb{N},G^2\in \mathbb{N}$  $G^1\leftarrow rgb2gray\left(I^1\right)$;  $G^2\leftarrow rgb2gray\left(I^2\right)$;  ${\displaystyle \overline{G^1}\leftarrow \frac{1}{i\times j}\sum _{\begin{array}{c}i,j\end{array}}{G}_{ij}^1}$;  ${\displaystyle \overline{G^2}\leftarrow \frac{1}{i\times j}\sum _{\begin{array}{c}i,j\end{array}}{G}_{ij}^2}$;  if $\overline{G^1}>\overline{G^2}$ then$\phantom{(}$  | $S\leftarrow I^1$;  else  | $S\leftarrow I^2$;  end

The example shown in Figure 3 would get values of 93.5 for the left image ($G^1$) and 114.5 for the right image ($G^2$). Bearing in mind that the values vary between 0 and 255, where 0 is black and 255 is white, the darkest image and where the macula is located, is on the left side ($G^1$) and therefore it is the right eye. The differences between $G^1$ and $G^2$ images usually range from 5% to 20%, which is enough to determine the location of the macula.

The proposed algorithm is simple because it was intended to have maximum efficiency in the shortest possible time. As demonstrated in Section 5, all the performed tests have correctly identified whether it was the right eye or the left eye. However, should it fail, the software tool always provides manual tools in the left side menu to correct the automatic analysis and change the eye being analyzed.

4.2.2. Bounds Detection

In the next step of the analysis, the edges of both the retinal and the optic disc (or papilla) are detected. Information about the size of the retina is provided by detecting its edge, which is very useful when applying morphological techniques to analyze the image. As detecting the edge of the retina is computationally easier than detecting the edge of the optic disc, and the size of the optic disc is directly related to the size of the retina, it is the first step taken when locating the optic disc. Algorithm 2 shows the methodology used to detect the edge of the retina.

The result can be seen in the example shown in Figure 4, where the diameter of the retina d in the image G, is determined by discriminating the black border and the noise (few individual or grouped pixels) that may be in it. It is therefore necessary for the background of the image to be uniform; most fundus cameras are designed to make the background uniform.

Once the retina edge has been detected, additional information about the optic disc is available and it can now be located. More specifically, the diameter of the optic disc diameter is known to be slightly less than one-sixth of the diameter of the retina, which is taken into consideration when locating it. The techniques used to locate the optic disc are described in Algorithm 3. Figure 4 shows both the relationship between the retina and the optic disc, and the result of different filters applied to the image in this step.

Algorithm 2: Retina edge detection

// w, h and d are the image I width, height and diameter  Input: $I,w,h$  Output: d  $G\leftarrow rgb2gray\left(I\right)$;  // c is the background color, $nc$ is the number of c color point  // $d_1$ is the initial diameter point, $d_2$ is the end diameter point  $c\leftarrow G_{00}$;  $nc\leftarrow 0$;  $d\leftarrow 0$;  $d_1\leftarrow 0$;  $d_2\leftarrow 0$;  for $i\leftarrow 1$ to w do  end

Algorithm 3: Optic disc location

// $crx$ is the x coord of the retina center, $cry$ is the y coord of the retina center and d is the diameter of the retina$\phantom{(}$  // $cp$ is the x,y coords of the papilla  Input: $I,crx,cry,d$  Output: $cp$  // $I_{ij}\in \mathbb{N}\times \mathbb{N}\times \mathbb{N},G_{ij}\in \mathbb{N}$  $G\leftarrow rgb2gray\left(I\right)$;  $S\leftarrow squareInscribed(G,d/2,crx,cry)$;  // Papilla size is always about 6 times lower than the the retina  $P\leftarrow \cup G_{ij}$;  $P^{\prime }\leftarrow shortToLower\left(P\right)$;  $threshold\leftarrow P_{\pi \times {(d/6)}^2}^{\prime }$;  $B\leftarrow binaryFilter(S,threshold)$;  // Open filter to erode and dilate the image B  $O\leftarrow openFilter\left(B\right)$;  // Fill gaps in the image O$\phantom{(}$  $C\leftarrow closeFilter\left(O\right)$;  $blob\leftarrow getBiggestBlob\left(C\right)$;  $rect\leftarrow getBoundingRect\left(blob\right)$;  $cp\leftarrow getCenter\left(rect\right)$;

Locating the optic disc is necessary due to the fact that it is used by the vessels to access the retina. Consequently, the analysis of the vessels begins with the edge of the optic disc.

4.2.3. Segmentation

Once the edges of the retina and optic disc have been detected, it is necessary to analyze the area between them to segment the blood vessels from the background of the eye.

More specifically, the analysis is focused on the area between the edge of the optic disc and the concentric circle with a radius three times greater than the radius of the optic disc edge. This area is analyzed and Figure 5 shows the images obtained at every step of the detection methodology described in Algorithm 4 which are provided by the agent with the best results; that is, the one that provides the largest, morphologically correct vascular area and, therefore, a greater number of blood vessels.

Gaussian filters were used to segment the blood vessels at the bottom of the retina. The decision to adopt this approach was determined by the morphological characteristics of the retina, where the blood vessels (darker tonality) would become indistinguishable from the background (lighter tonality) because they are narrow and elongated.

In the literature, Gaussian filters is the most used technique in segmentation. However, our proposal presents novelties with respect to existing techniques. For example, when applying Gaussian filters in the area of the optic disc, where the tonality is much clearer than in the background, error occurs in the area of analysis that lies next to the optical disk. To avoid this, the optical disc is filled with the mean tonality of its closest outer contour, so that the Gaussian filter does not introduce error ($P\leftarrow \overline{S^{\prime }}$).

Algorithm 4: Vessels detection

// $cpx$ is the x coord of the papilla center, $cpy$ is the y coord of the papilla center and $rp$ is the radio of the papilla  // $D^{\prime }$ is image with the detected vessels in white and the other points in black  Input: $I,cpx,cpy,rp$  Output: $D^{\prime }$  // $I_{ij}\in \mathbb{N}\times \mathbb{N}\times \mathbb{N},I{r}_{ij}\in \mathbb{N}$  $Ir\leftarrow rgb2gray(I,infraredFilter)$;  // P contains all the papilla points  $P\leftarrow \{I{r}_{ij}/{(i-cpx)}^2+{(j-cpy)}^2\le r{p}^2\}$;  $S\leftarrow \{I{r}_{ij}/i\ge cpx-rp\times 1.5,i\le cpx+rp\times 1.5;j\ge cpy-rp\times 1.5,j\le cpy+rp\times 1.5\}$;  $S^{\prime }\leftarrow S_{ij}/S\cup P=\varnothing$;  // Change papilla color with the closest background mean color to avoid noise when applying gaussian filters  $P\leftarrow \overline{S^{\prime }}$;  // Analysis area goes from the papilla limit to 3 times its radio size  $A\leftarrow \{I{r}_{ij}/i\ge cpx-rp\times 4,i\le cpx+rp\times 4;j\ge cpy-rp\times 4,j\le cpy+rp\times 4\}$;  // $G{a}_{ij}\in \mathbb{N}$  $Ga\leftarrow gaussianFilter\left(A\right)$;  $I{r}^{\prime }\leftarrow \{I{r}_{ij}/i<cpx-rp\times 4,i>cpx+rp\times 4;j<cpy-rp\times 4,jcpy+rp\times 4\}$;  $I{r}^{\prime }\leftarrow \{A_{ij}/i\ge cpx-rp\times 4,i\le cpx+rp\times 4;j\ge cpy-rp\times 4,j\le cpy+rp\times 4\}$;  // $D_{ij}\in \mathbb{N}$  $D\leftarrow Ir$;  // Grayscale: 0 black; 255 white$\phantom{(}$  $threshold\leftarrow I{r}^{\prime }-I{r}^{\prime }/100$;  $D_{ij}\leftarrow 255/D_{ij}\ge threshold$ $\forall ij$;  $D_{ij}\leftarrow 0/D_{ij}<threshold$ $\forall ij$;  $D_{ij}\leftarrow 0/P_{ij}\in P$;  // Remove noise: blobs whose area is lower than papilla diameter (all vessels must be bigger)  $B\leftarrow getBlobs\left(D\right)$;  $B^{\prime }\leftarrow \{B_i/area\left(B_i\right)<(rp\times 2)\}$;  $D^{\prime }\leftarrow D$;  $D_{ij}^{\prime }\leftarrow 0/ij\in B_k^{\prime }$;

As the Gaussian filter is a technique that consumes a considerable amount of computational capacity, these are only applied in the area of analysis, which is not the whole retina, but goes from the edge of the optical disc to the bounding box (A) of a concentric circle with three times the radius of the optical disc. This limitation has been specified by the medical team.

Next, Gaussian filters are applied and any possible noise is filtered (small blobs that morphologically do not meet the characteristics of the blood vessels) and the result is later compared with the $Ir$ image, in which the blood vessels appear more marked with respect to the background than in the original grayscale image (G).

At this point, each agent applies a different threshold that will provide a vascular area. The detected vascular area is analyzed morphologically (branching pattern and vessel tracking) to ensure that blood vessels and noise are eliminated. The performance of an agent is measured by the area and number of blood vessels it has identified, the agent that identifies the largest number of blood vessels is classified as the best performing agent. In the performed tests, the threshold that gave the best results was $I{r}^{\prime }-I{r}^{\prime }/$100, so it is the one described in Algorithm 4.

After the segmentation of the vascular structure, the vessels must be identified. However, in order to identify them correctly, an intermediate process is required to avoid the problem of one vessel crossing another. This intermediate process determines whether a point of the structure belongs to a given vessel. To do this, the methodology proposed in [35] is followed. As a result, the skeleton of the structure is obtained.

Thanks to the analysis and classification of each of the points of the skeleton, useful information can be extracted for the identification process. Thus, the advantage of our proposal over existing techniques is that it effectively prevents crossing and branching. As shown in Figure 6, four points can be found according to the number of neighbors:

(a)

normal (2 neighbors, Figure 6a),

(b)

branch (3 neighbors, Figure 6b),

(c)

final (1 neighbor, Figure 6c),

(d)

cross (4 or more neighbors, Figure 6d).

This way, every detected vessel is fragmented into a series of segments which are analyzed individually. Every segment is a set of all the points between the following types of pairs: final–final, branch–final, branch–branch, branch–cross, cross–cross and cross–final. Additionally, full vessels (both veins and arteries) are points between a final–final pair with one or more chained segments. Algorithm 5 describes the process followed to obtain all the segments.

Algorithm 5: Skeleton and segment extraction

// $\phantom{(}$$cpx$ is the x coord of the papilla center, $cpy$ is the y coord of the papilla center and $rp$ is the radio of the papilla$\phantom{(}$  // $\phantom{(}$$Ir$ is the infrared-grayscaled image $D^{\prime }$ is a binary image with the detected vessels in white and the other points in black$\phantom{(}$  // $\phantom{(}$$Sk$ is a binary image with the vessels skeleton and $S{k}^{\prime }$ is the image with the vessels skeleton represented by the $Ir$ image background points$\phantom{(}$  // $\phantom{(}$$TS$ is a subset with the skeleton of all the segments$\phantom{(}$  Input: $Ir,D^{\prime },cpx,cpy,rp$  Output: $Sk,S{k}^{\prime },TS$  // $\phantom{(}$$I{r}_{ij}\in \mathbb{N},D_{ij}^{\prime }\in \{0,255\}$$\phantom{(}$  // $\phantom{(}$A contains all the points of the analyzed area$\phantom{(}$  $A\leftarrow \{(i,j)/{(i-cpx)}^2+{(j-cpy)}^2>r{p}^2,{(i-cpx)}^2+{(j-cpy)}^2\le {(rp\times 4)}^2\}$;  // $\phantom{(}$Get skeleton image ($Sk$) and a new one from it ($S{k}^{\prime }$) with the closest (distance of $rp/2$) background mean instead of white$\phantom{(}$  // $\phantom{(}$$S{k}_{ij}\in \{0,255\},S{k}_{ij}^{\prime }\in \mathbb{N}$$\phantom{(}$  $Sk\leftarrow getSkeleton\left(D^{\prime }\right)$;  // $\phantom{(}$$P_{ij}$ contains the set of background points of the retina at a distance of $rp/2$ $\phantom{(}$  $P_{ij}\leftarrow \{(x,y)/{(x-i)}^2+{(y-j)}^2={(rp/2)}^2,S{k}_{ij}=255,D_{xy}^{\prime }=0,(x,y)\in A\}$;  $S{k}^{\prime }\leftarrow Sk$;  $S{k}_{ij}^{\prime }\leftarrow \frac{1}{|P_{ij}|}\sum P_{ij}\forall (i,j)/S{k}_{ij}=255$;  // $\phantom{(}$Get all white points type from $Sk$ image (based on the number of neighbors)$\phantom{(}$  // $\phantom{(}$$S{P}_{ij}\in \{final,regular,branch,intersection\}$$\phantom{(}$  $SP\leftarrow \{(i,j)/S{k}_{ij}=255\}$;  $S{P}_{ij}\leftarrow getPointsType\left(S{k}_{ij}\right)$; // $\phantom{(}$See Figure 5$\phantom{(}$  // $\phantom{(}$Get all segments $T{S}_i\subseteq A$$\phantom{(}$  $TS\leftarrow getSegments\left(SP\right)$;

4.2.4. Identification

Once each point is associated with its blood vessel within the vascular structure under examination, they can be individually classified as a vein or an artery.

The algorithm designed to identify these vessels takes the skeleton tonalities for each blood vessel and its closest background tonalities (ensuring that there is no other vessel point considered as a background point).

In addition to the tonality, and in order to facilitate the identification of the blood vessels, there are other features, such as the veins and arteries, which must be present in each of the images. The agent with the Coordinator role is responsible for selecting the most appropriate results. This helps to automatically determine the used threshold values. Algorithm 6 describes this process in more detail.

4.3. Diagnosis

Once the position of veins and arteries is known, the next step of the systems consists of extracting the encapsulated information. Different techniques are applied by the Measurement Agent to obtain parameters such as the length, the thickness and the area of every vessel classified by type (vein or artery) and by proximity to the optic disc in three concentric circles.

Area is obtained automatically by counting the number of pixels inside every vessel. Length is also easy to calculate; the number of pixels of every vessel skeleton is counted. Finally, the relationship between the area and the length of every single vessel determines its mean thickness.

Figure 7 shows an example in which an area of 3919 pixels is detected (1); 308 pixels represent both its skeleton and its length (2); and the mean thickness is 12.72 pixels; it is the result of dividing area by length, this can be verified in the zoomed section of the vessel (3).

Algorithm 6: Vessels and segments identification

// $\phantom{(}$A is the set of segments which have been identified as arteries, V is the set of segments which have been identified as arteries$\phantom{(}$  Input: $Ir,S{k}^{\prime },TS,$  Output: $A,V$  $avThr\leftarrow 15;mxThr\leftarrow 8;count\leftarrow 0;A\leftarrow \left\{\right\};V\leftarrow \left\{\right\}$;  while $count<7$ do    end

The system works with pixels when these parameters are calculated, but the results are provided in real millimeters by applying the scale provided by the fundus camera manufacturer, which can be easily configured with the software tool.

5. Results

When evaluating the results provided by every stage of the software, the only images used were those in which an expert user could easily identify the blood vessels. In addition, images with any type of errors were been considered in the next steps, as their results depend on the output from the previous steps.

Table 1. Success rate for every step and comparative in blood vessel segmentation stage.

Success (%) /Stage	Eye Side	Retina	Papilla	Segment.	Ident.
Proposed System	100%	100%	99%	95%	91%
GeethaRaman et al. (2016) [13]	-	-	-	95.36%	-
Franklin et al. (2014) [36]	-	-	-	95.03%	-
Lam et al. (2010) [37]	-	-	-	94.72%	-
Chaudhuri et al. (1989) [38]	-	-	-	87.73%	-

shows the success rate for every step from a sample of 150 retinal images.

The works found in the state-of-the-art all focus on specific analyses and extract specific information, thus there are great differences between them. One common component that is shared between those works is the blood vessel segmentation stage, whose results in other proposals could be compared with ours. According to [13], a comparative should be made of the techniques related to this stage. In this study, as indicated in Section 2, the authors present a comparison of 26 articles. Out of all of them, only the methodologies presented in [36] (95.03%) and the proposal of [13] (95.36%) achieve a percentage higher than 95%. Equally high percentage has been obtained in the present work. The percentage of segmentation precision in the other 24 articles varies from 94.72% of precision obtained in [37] to 87.73% obtained in the 1980s in the work presented in [38].

However, the proposed system has better computational performance than existing tools. The tool presented in [13] has a 95.36% efficiency in segmentation (the highest one); however, it takes this tool “around 20 mins” to process an image on a PC with i5 processor at 2.53 GHz and 4 GB RAM. In comparison, our system does the same task in less than 2 min (1 min and 54 s) on a PC with the same characteristics.

Thanks to the use of an MAS as the basis of the platform, the techniques and algorithms were easily applied, facilitating the choice of the best methodologies and their best input parameters. In addition, the system can be auto-adapted to the requirements of every single image and multiple agents ensure that the best possible results are obtained at each step of image processing and analysis.

Another advantage of the MAS is that it allows the platform to run in a distributed environment. This makes it possible to decouple the technology applied from the analysis process.

The outcome of this research is a multiplatform software tool (Figure 8) whose main features include: compatibility with different fundus cameras, and the ability to share results on a common server. This makes it possible for any medical center to process images, and the data from medical centers can be sent to the database.

The main parameters measured in each image are: total vessel thickness (vThck), total artery thickness (aThck), total vein area (vArea), total artery area (aArea), total vein length (vLength), total artery length (aLength) and the relationship between artery and vein thickness (AVIndex). An example is shown in

Table 2. Main exported parameters for the image Figure 8, right.

vThck	aThck	vArea	aArea	vLength	aLength	AVIndex
0.119359	0.0804724	1.62384	1.83216	13.6046	22.7675	0.674203

.

Other parameters are also exported, such as the eye (right or left), the image name, patient identifier, other clinical variables of the patient, or the main parameters classified by quadrants and proximity.

6. Conclusions

The system allows for the analysis of fundus images using different algorithms that are adjusted automatically according to the characteristics of the image. During this process, filtering, analysis and data mining techniques are applied in combination with the knowledge of experts to carry out the analysis and the extraction of parameters.

Thanks to the flexibility provided by the VO based system, the software can be integrated with other clinical software in order to launch new studies on the relationship between the extracted information and other fields of medicine, such as blood test results. Furthermore, the use of virtual organizations simplifies the inclusion of new algorithms due to the open characteristics of this technology.

The medical research team involved in this research is currently finishing the validation phase; they are aiming to find connections between the extracted values and different pathologies. Knowledge of the parameters that are indicative of certain pathologies is very valuable to medicine. For this reason, it is important to develop tools that simplify the parameter analysis process and allow for obtaining reliable results.