Large Geographical Area Aerial Surveillance Systems Data Network Infrastructure Managed by Artificial Intelligence and Certified over Blockchain: A Review

Abstract

This paper proposes an aerial data network infrastructure for Large Geographical Area Surveillance Systems. The work presents a review of previous works from the authors, existing technologies in the market, and other scientific work, with the goal of creating a data network supported by Autonomous Tethered Aerostat Airships used for sensor fixing, a drones deployment base, and meshed data network nodes installation. The proposed approach for data network infrastructure supports several independent and heterogeneous services from independent, private, and public companies. The presented solution employs Edge Artificial Intelligence (AI) systems for autonomous infrastructure management. The Edge AI used in the presented solution enables the AI management solution to work without the need for a permanent connection to cloud services and is constantly fed by the locally generated sensor data. These systems interact with other network AI services to accomplish coordinated tasks. Blockchain technology services are deployed to ensure secure and auditable decisions and operations, which are validated by the different involved ledgers.

1. Introduction

Surveillance systems have been evolving from simple local close-circuit television systems, aeronautical/aerospace systems [1,2,3], and integrated cities infrastructure systems [4] to complex military systems [5]. With the evolution of technology, the surveillance systems have become more powerful, with complex autonomous tasks delivered by Artificial Intelligence (AI) platforms [6].

Surveillance systems are sometimes wrongly associated to people behavior control protocols [7,8]. The power of surveillance systems for fire control, infrastructure management, traffic control, assets managements, and others is indubitably valuable.

A surveillance system over a large geographical area needs the support of a large data network infrastructure to interconnect several ground, sea, and aerial sensors to services that analyze its data.

The network infrastructures are fundamental services of a society that are dependent on the continuous and proper functioning of a critical infrastructure network such as transportation, telecommunications, electric power, natural gas and oil, water distribution, economy, security, and quality of life [9]. These infrastructures are expensive, and the obstacles to its implementation limit its performance, functionality, and maintenance. By leveraging several Heterogeneous Smart Solutions (HSS) from several independent entities in the same infrastructure, private and public companies can share its costs.

The same infrastructure can support several independent HSS as long as it guarantees its security, operability, independence, and performance. The goal and contribution of the presented Large Geographical Area Aerial Surveillance Systems Data Network Infrastructure is to offer such a system. Through employing AI integrated services, a reduction in the operability and maintenance costs is achieved.

The AI growth in technology and solutions has been supported by investments from market leader corporations and governments. This is a solid indicator of its disruptive potential in the market. In 2019, the United States Government invested almost $5 billion in AI projects, including the $2 billion DARPA program [10]. China is showing a higher investment in AI R&D, with cities such as Shanghai and Tianjin investing $15 billion and $16 billion, respectively, in their AI industries over 10 years [10]. Other Chinese cities are following the investment in the AI innovation area. In 2017, China accounted for 48% of the global AI R&D funding [10].

Governments see a big strategic importance in the AI technology, leveraging large investments in its research. The European Commission is investing $1.7 billion for research and development, China alone is investing $47 billion, and Canada is investing $125 million in its AI strategy [10]. In 2016, €3.2 billion were invested in AI in Europe, and in the last 3 years, the European Union has funded €1.5 billion in AI research and innovation projects [11].

By 2025, the world will see the shipment of 40 billion personal smart devices, and 90% of device users will have a smart digital assistant. Data utilization will reach 86%, and AI services will be readily available [12]. According to Huawei, AI becomes a new general-purpose technology and will change all industries and organizations [13].

AI has the power to change almost everything about the way businesses operate and will contribute up to $15.7 trillion to the global economy by 2030 [14]. Of this, $6.6 trillion is likely to come from increased productivity and $9.1 trillion is likely to come from consumption side effects [14]. The global corporate investment in AI in 2015 was $12.8 trillion increasing to $67.9 trillion in 2020, including private investments, public offerings, M&A, and minority stakes [15]. Today, AI achieves 98.8% image recognition accuracy where humans only achieve 94.9%, as seen in the ImageNet Challenge Top-5 Accuracy created by Stanford University and Princeton University [15]. The ImageNet training cost has dropped from $1100 in 2017 to $7.43 in 2020 for 93% accuracy. In English language understanding benchmarks, in the SuperGLUE single-metric benchmark, the AI reached a score of 90.3 compared to a score of 89.8 for the human performance [15].

Blockchain technology is an emerging technology with wide-ranging application in the industry and services, namely transportation, aeronautics, and aerospace industries [16,17,18]. NASA and other space agencies have also expressed the intention to launch Blockchain-based distributed space missions [19] and internal operations.

The Blockchain concept is becoming a valuable tool to record transactional data using a distributed ledger, leveraging efficient transactions validations in a verifiable permanent way and assuring transactional security [20]. The aviation industry is very sensitive to records; in fact, the record keeping is one of the main bases imposed by the regulators for the entire aviation ecosystem. Aviation would benefit largely from Blockchain-based record keeping, since it is impossible to forge, lose, or manipulate aviation records. Aircraft or engine lessors will specially benefit from this type of technology. When an aircraft changes owners, a very bureaucratic process is unleashed involving a few dozen persons and grounding the aircraft for weeks or months. With a Blockchain-based system, the processing time is reduced, optimizing all processes. When an aviation operator ceases its operations, it is frequent to see that the lessors have difficulty retrieving all records from an aircraft or engine, which in some situations can lead to an entire recertification process, which can cost millions to achieve. If a Blockchain-based service was available, the airworthiness information is never lost, becoming accessible in those situations. In [21], the authors explained how the aviation world can improve in the area of aircrafts security with the implementation of Blockchain. The authors state that the Blockchain can help to secure the GPS satellite signal sent to the aircraft; in an event of GPS spoofing, the system can use the Blockchain-based digital ledger instead of the jammed GPS signal, which helps to secure the digital signal received by the aircraft.

The Blockchain technology is a secure platform to register and make visible a series of transactional data called blocks that are linked using cryptography, which asserts an immutable, change-resistant timeline of events and its associated data that brings transparency, immutability, security, traceability, and interoperability to the stored data [18,22]. The Blockchain is open and distributed over different involved parties called ledgers. The ledgers work over the Blockchain platform and are composed of several parties that record and certify transactions, which are recorded in a verifiable and permanent way [23].

An integrated Blockchain platform, based on common regulated principles, can be used to register and assert the historic data of network activity, AI decision making, HSS interconnection activity, and others [24,25,26]. Half (50%) of the companies in life science, consumer products, tech, healthcare, and media have already deployed Blockchain solutions, being $1.4 billion invested in new projects between 2014 and 2017 [27]. The deployment of a Blockchain ecosystem in the proposed Large Geographical Area Aerial Surveillance Systems Data Network Infrastructure will guarantee the various HSS, which is a verifiable activity of its operations and managing services.

The paper is structured as follows. Section 2 presents the proposed Data Network Infrastructure for Heterogeneous Smart Solutions. Section 3 presents an overview of the proposed Edge AI infrastructure to automate the operation and maintenance of the Data Network Infrastructure services. An Edge AI service operation example is presented to give the reader a view over several Edge AI Agents cooperation. To give some technology context to the reader, several hardware and software solutions to support AI services are presented. Section 4 outlines the conclusion.

2. Data Network Infrastructure for HSS

Following the acquired knowledge on previous works on a services enabler architecture for smart grid and smart living services providers under industry 4.0 [28], an added step into Industry x.0 [29,30] is proposed with an innovative Data Network Infrastructure for HSS (DNIHSS) for Large Geographical Area Aerial Surveillance Systems. The DNIHSS supports several HSS from independent private and public companies in the same infrastructure.

For large geographical area surveillance systems, the DNIHSS is supported by several autonomous tethered aerostat airships (ATAA). The ATAA work autonomously, supported by AI services, offering an aerial platform for the integration of wireless data network nodes called Data Network Infrastructure Blocks (DNIB), which are linked in a meshed fashion way to provide a wireless data network to support several sensors, actuators, and services.

Several large wing gliders or aerostat airships designs exist in the market [31,32,33], and different layouts are to be tested in future works. Two examples of such shapes are shown in Figure 1 and Figure 2.

Although all ATAA rely on buoyant gas to float, in some environments, the use of lift forces of a wing and the control of a tail to govern the ATAA position may be necessary. So, several shapes are to be considered depending on the installation environment or portability needs.

The ATAA wireless data network nodes mesh is connected to one or more Ground Connection Points (GCP) installed in one or more geographical areas or over a satellite, which provide a point of connection to the DNIHSS Network Managing Services (NMS). The NMS are responsible for managing the DNIHSS infrastructure’s operation and security. The NMS also interconnects the HSS modules in its network to its corresponding managing systems in the cloud.

Several goals were determined for the proposed DNIHSS and its components in order to ensure independence and security to the supported HSS:

• Service type agnostic;
• Hardware and software agnostic;
• Support for several HSS at the same time;
• Security and independence of HSS services;
• Separate operability from maintenance/operation and HSS services;
• Automation of maintenance and operation tasks;
• Simplicity of components and services implementation, maintenance, and substitution;
• Separation of concerns on services and components;
• Visibility and transparency over maintenance/operation to assure the HSS services security.

To ensure visibility and transparency over operation, maintenance, and security, a Blockchain system is implemented. The Blockchain is an architectural concept that offers open, secure, and distributed ledger transactional validation and verification. The entire ledger can record transactions between parties efficiently and in a verifiable and permanent way.

In the DNIHSS, the Blockchain is a multi-party collaboration of software, hardware vendors, service operators, and network operators. The interoperability of these independent ledgers in transactional verification guarantees the transparency of all the operations, maintenance, and governability of the DNIHSS. It guarantees the transactions’ immutability, security, transparency, and auditability. This foundation for building transactional applications that establish trust and transparency while streamlining the DNIHSS processes and services offers security to services operator supported by the DNIHSS.

There are two types of DNIHSS: Permanent Installation (PI) and Temporary Installation (TI). In a PI, the ATAA are anchored in a static anchor platform. In a TI, the anchor platform is mobile and distributed as needed in geographical areas that only need the temporary installation of a DNIHSS.

Two examples of the offered DNIHSS using ATAA are presented in Figure 3 and Figure 4.

Figure 3 shows an example of a PI of the offered DNIHSS using ATAA over the sea for marine traffic control and security, marine species monitoring, water quality control and weather monitoring, and others. Figure 4 presents an example of a Temporary Installation, deployed as needed, to oversee a fire evolution and for fire prevention monitoring.

With the goal of contributing to a flexible, secure, easy to maintain, and capable to evolve infrastructure, the DNIHSS proposes a data network developed over the Industry X.0 paradigm [30,34]. The architecture targets the development of an infrastructure capable of enabling the creation of several independent HSS over the same infrastructure under the Industry X.0 design principles. The DNIHSS is aware of all the main technologies, products, protocols, and services in order to be an enabler of integrated services, empowering the coexistence of heterogeneous ecosystems in the same data network [35].

DNIHSS Communication Components

The DNIHSS services are built in blocks having restrictive services, actions, and tasks [36,37]. This block modularity and restriction of responsibility offers the following advantages:

• Independence of software solution implementation: the blocks can be implemented with different technology and software solutions;
• Increased security of the architecture by simplification of concerns: the reduction of responsibilities of each block confines the impact of the action of hackers in case of a compromised module. Usages of different technology solutions in the individual blocks increase the difficulty of hacking activity;
• Responsibility division: the assignment of well-defined responsibilities to the blocks allows a simplified substitution of them by other ones with different technology and implementation solutions. The well-defined responsibility facilitates the security and activity supervision, detecting malicious activity faster;
• Redundancy existence: the blocks can be implemented alongside other blocks with the same responsibilities to add in redundancy and performance.

This modularity increases the resilience of the architecture to failures and attacks.

The different HSS served by the DNIHSS have their own components and local data networks infrastructure distributed in the geographical area, interconnecting sensor, actuators, and local services to an HSS module. This HSS module functions as a gateway between the HSS service local components and the DNIHSS. Over the DNIHSS, these HSS modules communicate to its corresponding HSS cloud managing services and to other modules from the same HSS.

All the communication between the DNIHSS services, its actions, interactions, and tasks are registered in Blockchains managed by independent Blockchain services. It guarantees the transactions’ registration immutability, security, transparency, and auditability, establishing trust and transparency over its transactions, actions and operations.

As long as it is contracted between two different HSS managing services and authorized by the NMS, one HSS module can communicate to another module from a different HSS. This enables the sharing of resources between two different service modules, without compromising security and service independence. These communications are always made over the DNIHSS and managed by the NMS. An HSS module cannot communicate to another HSS managing service except its own.

The medium for the data exchange between the different blocks is call commands [28]. The commands encapsulate the type of the needed interaction. The main interaction activities can be characterized as status request, action order, and data transport [28]. An HSS module data exchange command frame [28] is illustrated in Figure 5.

The data exchange command frames are delivered and received by the DNIB. The DNIB are the point of connection for the heterogeneous services modules to the DNIHSS. The DNIB are responsible for receiving the encrypted data from the HSS modules, creating the data exchange command frame, and delivering the command to its destination. The majority of the HSS modules data exchange is delivered in asynchronous data exchange command frames, but when necessary, the DNIB in collaboration to other DNIB can create temporary Virtual Private Networks (VPN) between the HSS modules and their corresponding cloud services.

A representation of the DNIB within the DNIHSS is presented in Figure 6. Each HSS module is connected to its components over a local private data network.

The HSS modules are connected to the DNIB by a Block Interface (DNIBI). The DNIBI does not have the knowledge about the business processes or data structure of the DNIB or the HSS module. Its relation is restricted to previously contracted command types. The restricted command types delivered by a certain DNIB are granted and managed by the NMS. The DNIBI functions as a proxy between the HSS module and the DNIHSS.

Figure 7 presents a representation of the DNIBI components and Figure 8 shows the flow of the asynchronous data exchange commands.

The DNIBI is divided in two areas: one responsible for communicating with the HSS module and other responsible for the interface with the DNIHSS. When a HSS module needs to send a data exchange command, it connects to the Communication Interface unit, delivering the data and destination address. The Communication Interface creates a message frame and sends it to the Data Encryption unit. At the same time, if the data exchange command type is flagged by the NMS for activity control, a communication is initiated by the Communication Interface with the Blockchain Service unit and a Blockchain is initiated. In the Data Encryption unit, the data are encrypted and delivered to the Message Framework Creator, where a data exchange command frame is created. Then, the command frame is delivered to the DNIB Interface unit, and if needed, an activity is added to the previous created Blockchain by the DNIB Blockchain Service. The DNIBI connects to the closest DNIB and sends the data exchange command frame.

When a data exchange command frame is delivered from the DNIHSS to the HSS module, the data flow follows the opposite path with only two exceptions: an activity is added to the Blockchain by the Message Framework Creator unit when it delivers the data to the Data Encryption unit and by the Data Encryption unit when it receives the data.

3. Edge Artificial Intelligence

The DNIHSS and its components are managed by several Edge AI Agents (EAIA) working in collaboration. The main drivers for extending the AI to the Edge are data volume and bandwidth capacity, intermittent connectivity, fast decision making on the edge, and scalability. Edge AI means that AI inference is processed locally on a hardware device using locally generated data, usually a gateway where sensors and actuators are connected. The EAIA does not need to be connected with cloud services for inference.

Each EAIA in the proposed DNIHSS has as simplified goals and tasks as possible, implementing the goal for the separation of concerns about services and components. The reduction in the complexity of each EAIA decreases the time and complexity in the data preparation and AI training steps. However, this means that an increase in the number of EAIA and collaboration interconnections must be orchestrated.

By applying the design principle of separation of concerns commonly used in computer science, the simplification of the EAIA tasks means that not only the data preparation and AI training tasks are simplified, but each agent capabilities upgrade can be accelerated during its operation and even its substitution can be easily achieved, even without the substitution of other agents with direct connection.

With the separation of concerns design principle applied to the DNIHSS EAIA implementation, a more reliable, resilient, and adaptable system is obtained, quickly adapting its operation to the ever-changing needs of the DNIHSS and technology evolution, as observed in previous works on a services enabler architecture for smart grid and smart living services providers under industry 4.0 [28].

The Edge AI is experiencing a fast adoption and implementation supported by the increased investment of key technology AI leaders. These technology leaders offer specialized AI hardware and software solutions. This new Edge AI trend is increasing in interest and implementation, with several specialized chips being presented for edge inference.

Specialized chips are being offered for several stages of the AI implementation. For example, Intel offers specialized chips, for instance the Intel Nervana Neural Network Processor-L 1000, to train deep learning, whose high bandwidth memory and local static random-access memory are closer to the data processing; another, the Intel Nervana Neural Network Processor-i 1000, is designed specifically for the growing complexity and scale of inference applications. Huawei, on the other hand, has the Ascend generation chips comprising five chips: Ascend Nano for components such as earphones; Ascend-Tiny for always-on components; Ascend Lite for smartphones; Ascend 310 for personal computers and servers due to its extremely efficient AI SoC for low power computing and inference scenarios; and Ascend 910 for cloud data centers with high computing density in a single chip, mainly for large scale distributed training.

The inference chips offer the ability to accelerate the inference processing and the capacity to work offline if necessary. Although some chips are AI multipurpose, some inference chips can be more specialized to the task at hand such as the Vision Processing Unit (VPU) chips for image inference [38,39] or be specialized for the use of AI frameworks such as the Tensor Processing Unit (TPU) design by Google, especially for the TensorFlow Machine Learning inference at the edge [40].

The market leader’s hardware provider companies also offer open-source development platforms alongside ready-to-use AI computing platforms. Huawei alongside its open source development platform [41] offers the Atlas AI Computing Platform [42], comprising the products: Atlas 200 AI Accelerator Module for real-time HD video analytics for cameras, drones, and robots [43]; Atlas 300 AI Accelerator Card [44] for high-density video inference; Atlas 500 AI Edge Station [43] for smart city and smart transportation and unattended retail services; Atlas 600 AI Inference Application [43] smart city, transportation, and finance services; Atlas 800 AI Appliance [45] for deep learning and model training; Atlas G2500 Smart Video Analytics Server [46] for video intelligent analytics services.

There are several development kits in the market that accelerate the development of new AI solutions via prototyping: Google Coral Development Board to quickly prototype on-device ML products [40]; Google Coral USB Accelerator that brings machine learning inferencing to existing systems [40]; SparkFun Edge Apollo3 Blue development board [47]; Qualcomm DragonBoard 820c Development Board [48]; Intel Movidiu Neural Compute Stick [49]; Orange Pi AI 2801 stick [50]; and others. There are also increasing offerings of software frameworks and libraries:

• TensorFlow [51,52,53,54,55,56,57,58] is an open-source tool developed by Google’s AI department that is perfectly suited for complex numerical computations of high volumes and used in a vast number of fields. It has been used by several tech giants such as Google, SAP, Intel, NVIDIA, AMD, and others. It uses the programing languages C++ and Python;
• Microsoft Cognitive Toolkit [59] is an open-source tool suitable for a variety of AI applications. It allows the distributed training and supports C++, C#, Java, and Python programming languages;
• Theano [60,61] is a Python library that allows the definition, optimization, and evaluation of mathematical expressions involving multi-dimensional arrays efficiently. Appearing in 2007, it had its last version in 2017;
• Keras [62,63,64] is a high-level neural networks API, written in Python, that is capable of running on top of TensorFlow, Microsoft Cognitive Toolkit, or Theano. It was developed with a focus on enabling fast experimentation;
• Caffe [65,66,67,68] a very popular deep learning framework made with expression, speed, and modularity in mind. It supports the C++ programming language, and bindings for MATLAB are available.

These frameworks and libraries have proven to have quality and efficiency over time with market leader companies such as SAP, Google, Facebook, Microsoft, Yahoo, and Apple implementations in deep learning and machine learning projects and contributing to further frameworks and libraries developments.

Services can be implemented by leveraging them over several existing AI Cloud-based services to achieve a faster AI infrastructure development. Along with a functional AI infrastructure with different services that support all the phases of the AI infrastructure implementation, these services offer several trained solutions in areas such as image, speech, and text inference in different industries and lines of business. Some of the most recognized offers for AI platform-as-a-service (PaaS) in the market are SAP Leonardo ML [34,69], Amazon AWS Machine Learning [70,71,72,73], and Microsoft Azure Cognitive Services [74,75], to name a few.

3.1. Edge AI Proposed Infrastructure

The AI local inference in the edge is an important tool for the autonomy of the DNIHSS. The DNIHSS EAIA is able to work offline and make decisions upon locally generated data.

The DNIHSS EAIA will have simplified goals and tasks to inference. The simplification of the EAIA tasks means that the data preparation, AI training, and model deployment tasks are simplified and accelerated. For the realization of complex tasks, several EAIA work in cooperation, communicating with each other over data exchange commands. Although cooperating with each other, each EAIA does not know how the other EAIA work. This simplification, autonomy, and cooperation lead to an increase in EAIA orchestration complexity, during the DNIHSS design phase, but there is a decrease in the DNIHSS maintenance time and complexity over the medium and long term.

A simplified example of the EAIA cooperation diagram is presented in Figure 9.

As seen in Figure 9, several levels of EAIA compose the orchestration of the ATAA energy management. An energy management EAIA manages the ATAA energy needs, and during its operation, it delivers and receives higher level commands to its children EAIA, which manage the energy storage and generation. The energy management EAIA itself is also connected to a parent EAIA.

The energy storage EAIA manages two children EAIA that in turn manage the lithium battery storage and the flow battery storage. Depending on the state of the two different types of storage and the needs of the ATAA, the energy flow may be redirected from and to one of the two types of storage. This management is different from the local battery management needs, simplifying the supervision and the training of the different management task that are distributed to different EAIA.

The energy generation EAIA has two children EAIA that manage the local ATAA energy generation with solar panels or wind turbines. In turn, these EAIA are connected to children EAIA that manage a certain group of solar panels or a certain group of wind turbines.

The Edge AI services running on the EAIA are managed by the Edge Cloud Management Platform. The Edge Cloud Management Platform infrastructure overview is presented in Figure 10.

The Edge Cloud Management Platform is divided in three main services:

• Big Data Manager—for data preparation, analysis, storage, and distribution;
• AI Manager—for data collection and preparation, model training, and model deployment;
• Data Streams Manager—for Edge AI data streams management, from rules definition to data manipulation.

These services manage the behavior of the Edge AI services in the EAIA. The behavior of the EAIA is controlled by several local services managed by the Edge Cloud Management Platform. The EAIA local services are presented in Figure 11.

The Big Data Manager service in the Edge Cloud Management Platform is responsible for the Big Data and Data Intelligence services offering insights over the behavior of the EAIA. It also processes the data to be delivered for AI model training. The Big Data Manager has four modules: Data Preparation, Data Insights, Data Curator, and Data Deployer.

In the Data Preparation module, the collected data are prepared, processed, and delivered to the Data Insights module and Data Curator module. In the Data Insights module, several data intelligence services provide insights over the data streams in the EAIA to support the HSS operations, services, and decision making. In the Data Curator module, the data are collected, saved, and prepared for distribution over the Data Deployer module. The Data Deployer module aggregates the data in data containers that can be delivered to other services such as the AI Manager service.

The AI Manager service in the Edge Cloud Management Platform is responsible for the AI models’ training and deployment to the EAIA. This service is divided into three modules: Data Ingest, Model Training/Retraining, and Model Deployment/Redeployment.

The Data Ingest module makes requests to the Data Deployer for the data containers needed by AI Manager services and prepares them for its consumption by the AI models training. The data containers are closed aggregated data. Its content cannot be modified once it is declared closed and ready to be consumed. Although a data container cannot be modified during time, its data can be updated or enhanced by other data containers. Several data containers can be aggregated in one data containers batch.

The Model Training/Retraining module is responsible for the AI models training, using the data processed and prepared by the Data Ingest module in the form of data containers batch. The deployment of the trained AI models to the EAIA is managed by the Model Deployment/Redeployment module.

The Data Streams Manager service in the Edge Cloud Management Platform is responsible for the management of the data stream in the EAIA and its actions. It manages the EAIA activity, rules, and operations. The Data Streams Manager service is divided into four modules: Edge Streams Rules, Edge Data Harvesting, Edge Retrain Management, and Edge Predictive Analytics Management.

The Edge Streams Rules module defines the rules for real-time data manipulation in the EAIA Stream Manager module. All the incoming and outcome data in the EAIA go through the EAIA Stream Manager, where rules can be applied in real time to the data. These rules go from common mathematical calculations to complex algorithms. The rules can manipulate data or trigger events. These events can trigger other events in an event chain. It also can send data to other EAIA modules for local data storage or for predictive analytics. These events can also be dispatched to the DNIHSS as data exchange command frames for communication with other EAIA or for communication with the HSS cloud services.

The EAIA Stream Manager is an important tool when dealing with a large amount of data generated in the EAIA. A smart compression and purge of unnecessary of data is fundamental.

The Edge Data Harvesting module is responsible for collecting the relevant data in the EAIA for future processing, mainly in the Big Data Manager service. It interacts with the Edge Streams Rules module for the definition of rules that define the selected data to be locally stored in the EAIA Local Repository module. The data from the EAIA Local Repository module can be accessed from the EAIA Stream Manager module for time-related algorithm rules.

The Edge Retrain Management module manages the evolution needs of the existing EAIA AI inference. It also interacts with other modules and services in the Edge Cloud Management Platform such as the Edge Streams Rules module, Edge Data Harvesting module, Big Data Manager service, and others to orchestrate the retraining and redeployment of AI models to the EAIA.

The Edge Predictive Analytics Management module is responsible for the EAIA inference operations done in the EAIA Predictive Analytics module. The inference processes over the data stream can be done by applying simple mathematical operations or a recurring AI model inference. Not all inference processes need to be done by AI models, some rules can be applied by simple mathematical algorithms. The needs of the EAIA predictive analytics over its data stream and its complexity are managed by the Edge Predictive Analytics Management module.

3.2. Edge AI Services Operation Example

The AI local inference in the edge is an important tool for the autonomy of the DNIHSS and for the automation of the HSS that supports. The different HSS services can communicate with each other and share its resources. An HSS for marine traffic control and borders security management is illustrated as an example for the Edge AI services operation.

Figure 12 illustrates the moment when a container ship approaches an area covered by an ATAA sensors. Several EAIA will cooperate to achieve the level of understanding of what kind of object was caught by the EAIA; if the object is a ship, what kind of ship it is; if it is a cargo ship, the ship type; the identification of the ship and its freight company; and whether the container ship has authorization to enter the exclusive economic zone. If necessary, other resources can be deployed to assist the services such as flying or underwater drones.

Several EAIA services are involved in the evaluation of the container ship. An example is shown in Figure 13.

In the HSS for marine traffic control and borders security management, several sensors are installed in the ATAA. Although the ATAA’s main goal is to physically support the installation of the wireless data network nodes for the mesh type network of the DNIHSS, it can be used to support several HSS EAIA sensors, actuators, and local data networks.

In the example, an EAIA analyzes several images of the surrounding area (signaled in Figure 13 as (A). This EAIA’s simplified goal is to identify an anomaly in the image of the sea. If something was found (B), the EAIA Predictive Analytics module sends a message to the EAIA Stream Manager with the coordinates, and in turn, data are sent to its DNIBI. The DNIBI connects to the nearest DNIB that sends the data exchange command frame to the destination EAIA. At the same time, a Blockchain is initiated with this event.

The next EAIA is responsible for identifying what kind of object it is (C): a ship; a whale; a dolphin; or others. If a whale was found, an interaction with another HSS is initiated (C2) by sending a data exchange command frame to the corresponding Cloud Managing Service. If a ship was identified, a message is delivered to the next EAIA. The Blockchain initiated in the previous EAIA (B) is added by this new event and will be enhanced by the next events.

An event is sent to the EAIA responsible for identifying the type of ship (D). This EAIA cooperates with other EAIA in order to inference the identification of the object: type of ship (object inference) and identification (text feature extraction inference). If this is not possible with the ATAA’s local sensors, several air or water drones can be deployed to inference the object characteristics.

Once the object is identified, a message is sent to the HSS for marine traffic control and the borders security management Cloud Managing Service. If previously contracted, the Cloud Service can inform the respective container ship owner company about its approach. With the identification of the object, the Cloud Managing Service can deploy, if necessary, a warning message to the human controllers, which in turn can deploy air forces or the navy to intercept the ship. With the autonomous HSS services, the human assets are only required for higher decision-making tasks, since all the tedious and labor-intensive tasks are done by autonomous EAIA.

4. Conclusions

This work proposes a data network infrastructure for Large Geographical Area Aerial Surveillance Systems, supporting several Heterogeneous Smart Services (HSS) from several independent public and private entities. The proposed data network infrastructure and its services are the basis for future work following the authors’ previous developments. The intention is to offer a review of the scientific work and technologies currently offered commercially, which have been integrated in order to accomplish the presented data network infrastructure.

The Data Network Infrastructure for Heterogeneous Smart Services (DNIHSS) is supported by several autonomous tethered aerostat airships (ATAA) deployed to work as the physical support infrastructure of the wireless data network nodes. Although several HSS operate over the same network infrastructure, the DNIHSS guarantees their independence and security. The DNIHSS components are leveraged by the latest developments in Artificial Intelligence (AI), Edge AI, Blockchain, and Internet of Things (IoT).

Several goals are determined for the proposed DNIHSS in order to ensure independence and security to the supported HSS, such as service type agnostic; hardware and software agnostic; support for several HSS at the same time guarantying its security and independence; automation of maintenance and operations; simplification of components and services with separation of concerns; and visibility and transparency over maintenance and operation to assure the HSS services security.

An Edge AI infrastructure is proposed for the implementation of the DNIHSS Edge AI services. These services can execute autonomous and offline operation. The Edge AI Agents (EAIA) have as simplified goals and tasks as possible, implementing the proposed DNIHSS goal of separation of concerns on services and components. By reducing the EAIA responsibilities, a reduction and simplification of decision tasks are achievable. The several simplified EAIA collaborate with each other by means of high-level commands to accomplish complex autonomous tasks and decisions.

To ensure the visibility and transparency over operation, maintenance, and security, a Blockchain service is used. The Blockchain services is introduced in the center of the data message exchange modules of the architecture.

In order to better illustrate the operation and interaction of EAIA services, an example of surveillance operation is illustrated. This operation example illustrates the interaction of several EAIA to accomplish the complex task of autonomous sea traffic surveillance and identification. The steps of identifying objects in the sea and differentiating them are accomplished by different EAIA services that cooperate with each other.

5. Patents

There are no patents resulting from the work reported in this manuscript.