Machine Learning and Data Mining in Pattern Recognition

Networked systems control is a known problem complicated because of the need to work with large groups of elementary agents. In many applications, it is impossible (or difficult) to validate agent movement models and provide sufficiently reliable control actions at the elementary system components level. The evolution of agent subgroups (clusters) leads to additional uncertainty in the studied control systems. We focus on new decentralized control methods based on local communications in complex multiagent dynamical systems. The problem of intelligence in a complex world is considered in connection to multiagent network systems, including a system named airplane with feathers, load balancing, and the multisensor-multitarget tracking problem. Moreover, the new result concerning the emergency of intelligence in a group of robots is provided. All these methods follow the paradigm of the direct reaction of each element (agent) of the system to its sensory data of current situation observations and the corresponding data from a limited number of its neighbors (local communications). At the same time, these algorithms achieve a mutual goal at the macro level. All of the considered emergent intelligence appearances inspire the necessity to “rethink” the previously recognized concepts of computability and algorithm in computer science. Full article

Various Monte Carlo techniques for random point generation over sets of interest are widely used in many areas of computational mathematics, optimization, data processing, etc. Whereas for regularly shaped sets such sampling is immediate to arrange, for nontrivial, implicitly specified domains these techniques are not easy to implement. We consider the so-called Hit-and-Run algorithm, a representative of the class of Markov chain Monte Carlo methods, which became popular in recent years. To perform random sampling over a set, this method requires only the knowledge of the intersection of a line through a point inside the set with the boundary of this set. This component of the Hit-and-Run procedure, known as boundary oracle, has to be performed quickly when applied to economy point representation of many-dimensional sets within the randomized approach to data mining, image reconstruction, control, optimization, etc. In this paper, we consider several vector and matrix sets typically encountered in control and specified by linear matrix inequalities. Closed-form solutions are proposed for finding the respective points of intersection, leading to efficient boundary oracles; they are generalized to robust formulations where the system matrices contain norm-bounded uncertainty. Full article

Evaluating web traffic on a web server is highly critical for web service providers since, without a proper demand forecast, customers could have lengthy waiting times and abandon that website. However, this is a challenging task since it requires making reliable predictions based on the arbitrary nature of human behavior. We introduce an architecture that collects source data and in a supervised way performs the forecasting of the time series of the page views. Based on the Wikipedia page views dataset proposed in a competition by Kaggle in 2017, we created an updated version of it for the years 2018–2020. This dataset is processed and the features and hidden patterns in data are obtained for later designing an advanced version of a recurrent neural network called Long Short-Term Memory. This AI model is distributed training, according to the paradigm called data parallelism and using the Downpour training strategy. Predictions made for the seven dominant languages in the dataset are accurate with loss function and measurement error in reasonable ranges. Despite the fact that the analyzed time series have fairly bad patterns of seasonality and trend, the predictions have been quite good, evidencing that an analysis of the hidden patterns and the features extraction before the design of the AI model enhances the model accuracy. In addition, the improvement of the accuracy of the model with the distributed training is remarkable. Since the task of predicting web traffic in as precise quantities as possible requires large datasets, we designed a forecasting system to be accurate despite having limited data in the dataset. We tested the proposed model on the new Wikipedia page views dataset we created and obtained a highly accurate prediction; actually, the mean absolute error of predictions regarding the original one on average is below 30. This represents a significant step forward in the field of time series prediction for web traffic forecasting. Full article

Multiagent technologies provide a new way for studying and controlling complex systems. Local interactions between agents often lead to group synchronization, also known as clusterization (or clustering), which is usually a more rapid process in comparison with relatively slow changes in external environment. Usually, the goal of system control is defined by the behavior of a system on long time intervals. As is well known, a clustering procedure is generally much faster than the process of changing in the surrounding (system) environment. In this case, as a rule, the control objectives are determined by the behavior of the system at large time intervals. If the considered time interval is much larger than the time during which the clusters are formed, then the formed clusters can be considered to be “new variables” in the “slow” time model. Such variables are called “mesoscopic” because their scale is between the level of the entire system (macro-level) and the level of individual agents (micro-level). Detailed models of complex systems that consist of a large number of elementary components (miniature agents) are very difficult to control due to technological barriers and the colossal complexity of tasks due to their enormous dimension. At the level of elementary components of systems, in many applications it is impossible to verify the models of the agent dynamics with the traditionally high degree of accuracy, due to their miniaturization and high frequency of control actions. The use of new mesoscopic variables can make it possible to synthesize fewer different control inputs than when considering the system as a collection of a large number of agents, since such inputs will be common for entire clusters. In order to implement this idea, the “clusters flow” framework was formalized and used to analyze the Kuramoto model as an attracting example of a complex nonlinear networked system with the effects of opportunities for the emergence of clusters. It is shown that clustering leads to a sparse representation of the dynamic trajectories of the system, which makes it possible to apply the method of compressive sensing in order to obtain the dynamic characteristics of the formed clusters. The essence of the method is as follows. With the dimension N of the total state space of the entire system and the a priori assignment of the upper bound for the number of clusters s, only m integral randomized observations of the general state vector of the entire large system are formed, where m is proportional to the number s that is multiplied by logarithm $N/s$. A two-stage observation algorithm is proposed: first, the state space is limited and discretized, and compression then occurs directly, according to which reconstruction is then performed, which makes it possible to obtain the integral characteristics of the clusters. Based on these obtained characteristics, further, it is possible to synthesize mesocontrols for each cluster while using general model predictive control methods in a space of dimension no more than s for a given control goal, while taking the constraints obtained on the controls into account. In the current work, we focus on a centralized strategy of observations, leaving possible decentralized approaches for the future research. The performance of the new framework is illustrated with examples of simulation modeling. Full article

Artificial intelligence, deep learning, and machine learning are dominant sources to use in order to make a system smarter. Nowadays, the smart speech emotion recognition (SER) system is a basic necessity and an emerging research area of digital audio signal processing. However, SER plays an important role with many applications that are related to human–computer interactions (HCI). The existing state-of-the-art SER system has a quite low prediction performance, which needs improvement in order to make it feasible for the real-time commercial applications. The key reason for the low accuracy and the poor prediction rate is the scarceness of the data and a model configuration, which is the most challenging task to build a robust machine learning technique. In this paper, we addressed the limitations of the existing SER systems and proposed a unique artificial intelligence (AI) based system structure for the SER that utilizes the hierarchical blocks of the convolutional long short-term memory (ConvLSTM) with sequence learning. We designed four blocks of ConvLSTM, which is called the local features learning block (LFLB), in order to extract the local emotional features in a hierarchical correlation. The ConvLSTM layers are adopted for input-to-state and state-to-state transition in order to extract the spatial cues by utilizing the convolution operations. We placed four LFLBs in order to extract the spatiotemporal cues in the hierarchical correlational form speech signals using the residual learning strategy. Furthermore, we utilized a novel sequence learning strategy in order to extract the global information and adaptively adjust the relevant global feature weights according to the correlation of the input features. Finally, we used the center loss function with the softmax loss in order to produce the probability of the classes. The center loss increases the final classification results and ensures an accurate prediction as well as shows a conspicuous role in the whole proposed SER scheme. We tested the proposed system over two standard, interactive emotional dyadic motion capture (IEMOCAP) and ryerson audio visual database of emotional speech and song (RAVDESS) speech corpora, and obtained a 75% and an 80% recognition rate, respectively. Full article

This article presents an novel approach inspired by the modern exploration of short texts’ patterning to creations prescribed to the outstanding Islamic jurist, theologian, and mystical thinker Abu Hamid Al Ghazali. We treat the task with the general authorship attribution problematics and employ a Convolutional Neural Network (CNN), intended in combination with a balancing procedure to recognize short, concise templates in manuscripts. The proposed system suggests new attitudes make it possible to investigate medieval Arabic documents from a novel computational perspective. An evaluation of the results on a previously tagged collection of books ascribed to Al Ghazali demonstrates the method’s high reliability in recognizing the source authorship. Evaluations of two famous manuscripts, Mishakat al-Anwa and Tahafut al-Falasifa, questioningly attributed to Al Ghazali or co-authored by him, exhibit a significant difference in their overall stylistic style with one inherently assigned to Al Ghazali. This fact can serve as a substantial formal argument in the long-standing dispute about these manuscripts’ authorship. The proposed methodology suggests a new look on the perusal of medieval documents’ inner structures and possible authorship from the short-patterning and signal processing perspectives. Full article

Assessing nutritional content is very relevant for patients suffering from various diseases, professional athletes, and for health reasons is becoming part of everyday life for many. However, it is a very challenging task as it requires complete and reliable sources. We introduce a machine learning pipeline for predicting macronutrient values of foods using learned vector representations from short text descriptions of food products. On a dataset used from health specialists, containing short descriptions of foods and macronutrient values: we generate paragraph embeddings, introduce clustering in food groups, using graph-based vector representations, that include food domain knowledge information, and train regression models for each cluster. The predictions are for four macronutrients: carbohydrates, fat, protein and water. The highest accuracy was obtained for carbohydrate predictions – 86%, compared to the baseline – 27% and 36%. The protein predictions yielded the best results across all clusters, 53%–77% of the values fall in the tolerance-level range. These results were obtained using short descriptions, the embeddings can be improved if they are learned on longer descriptions, which would lead to better prediction results. Since the task of calculating macronutrients requires exact quantities of ingredients, these results obtained only from short description are a huge leap forward. Full article

Because of the lack of reliable information on the spread parameters of COVID-19, there is an increasing demand for new approaches to efficiently predict the dynamics of new virus spread under uncertainty. The study presented in this paper is based on the Case-Based Reasoning method used in statistical analysis, forecasting and decision making in the field of public health and epidemiology. A new mathematical Case-Based Rate Reasoning model (CBRR) has been built for the short-term forecasting of coronavirus spread dynamics under uncertainty. The model allows for predicting future values of the increase in the percentage of new cases for a period of 2–3 weeks. Information on the dynamics of the total number of infected people in previous periods in Italy, Spain, France, and the United Kingdom was used. Simulation results confirmed the possibility of using the proposed approach for constructing short-term forecasts of coronavirus spread dynamics. The main finding of this study is that using the proposed approach for Russia showed that the deviation of the predicted total number of confirmed cases from the actual one was within 0.3%. For the USA, the deviation was 0.23%. Full article

In recent years, deep learning models have been used successfully in almost every field including both industry and academia, especially for computer vision tasks. However, these models are huge in size, with millions (and billions) of parameters, and thus cannot be deployed on the systems and devices with limited resources (e.g., embedded systems and mobile phones). To tackle this, several techniques on model compression and acceleration have been proposed. As a representative type of them, knowledge distillation suggests a way to effectively learn a small student model from large teacher model(s). It has attracted increasing attention since it showed its promising performance. In the work, we propose an ensemble model that combines feature-based, response-based, and relation-based lightweight knowledge distillation models for simple image classification tasks. In our knowledge distillation framework, we use ResNet−20 as a student network and ResNet−110 as a teacher network. Experimental results demonstrate that our proposed ensemble model outperforms other knowledge distillation models as well as the large teacher model for image classification tasks, with less computational power than the teacher model. Full article

As of the end of 2019, the world suffered from a disease caused by the SARS-CoV-2 virus, which has become the pandemic COVID-19. This aggressive disease deteriorates the human respiratory system. Patients with COVID-19 can develop symptoms that belong to the common flu, pneumonia, and other respiratory diseases in the first four to ten days after they have been infected. As a result, it can cause misdiagnosis between patients with COVID-19 and typical pneumonia. Some deep-learning techniques can help physicians to obtain an effective pre-diagnosis. The content of this article consists of a deep-learning model, specifically a convolutional neural network with pre-trained weights, which allows us to use transfer learning to obtain new retrained models to classify COVID-19, pneumonia, and healthy patients. One of the main findings of this article is that the following relevant result was obtained in the dataset that we used for the experiments: all the patients infected with SARS-CoV-2 and all the patients infected with pneumonia were correctly classified. These results allow us to conclude that the proposed method in this article may be useful to help physicians decide the diagnoses related to COVID-19 and typical pneumonia. Full article

Connected and Autonomous Vehicle (CAV)-related initiatives have become some of the fastest expanding in recent years, and have started to affect the daily lives of people. More and more companies and research organizations have announced their initiatives, and some have started CAV road trials. Governments around the world have also introduced policies to support and accelerate the deployments of CAVs. Along these, issues such as CAV cyber security have become predominant, forming an essential part of the complications of CAV deployment. There is, however, no universally agreed upon or recognized framework for CAV cyber security. In this paper, following the UK CAV cyber security principles, we propose a UML (Unified Modeling Language)-based CAV cyber security framework, and based on which we classify the potential vulnerabilities of CAV systems. With this framework, a new CAV communication cyber-attack data set (named CAV-KDD) is generated based on the widely tested benchmark data set KDD99. This data set focuses on the communication-based CAV cyber-attacks. Two classification models are developed, using two machine learning algorithms, namely Decision Tree and Naive Bayes, based on the CAV-KDD training data set. The accuracy, precision and runtime of these two models when identifying each type of communication-based attacks are compared and analysed. It is found that the Decision Tree model requires a shorter runtime, and is more appropriate for CAV communication attack detection. Full article

We propose a new forecasting procedure that includes randomized hierarchical dynamic regression models with random parameters, measurement noises and random input. We developed the technology of entropy-randomized machine learning, which includes the estimation of characteristics of a dynamic regression model and its testing by generating ensembles of predicted trajectories through the sampling of the entropy-optimal probability density functions of the model parameters and measurement noises. The density functions are determined at the learning stage by solving the constrained maximization problem of an information entropy functional subject to the empirical balances with real data. The proposed procedure is applied to the randomized forecasting of the daily electrical load in a regional power system. We construct a two-layer dynamic model of the daily electrical load. One of the layers describes the dependence of electrical load on ambient temperature while the other simulates the stochastic quasi-fluctuating temperature dynamics. Full article

The paper is devoted to an overview of multi-agent principles, methods, and technologies intended to adaptive real-time data clustering. The proposed methods provide new principles of self-organization of records and clusters, represented by software agents, making it possible to increase the adaptability of different clustering processes significantly. The paper also presents a comparative review of the methods and results recently developed in this area and their industrial applications. An ability of self-organization of items and clusters suggests a new perspective to form groups in a bottom-up online fashion together with continuous adaption previously obtained decisions. Multi-agent technology allows implementing this methodology in a parallel and asynchronous multi-thread manner, providing highly flexible, scalable, and reliable solutions. Industrial applications of the intended for solving too complex engineering problems are discussed together with several practical examples of data clustering in manufacturing applications, such as the pre-analysis of customer datasets in the sales process, pattern discovery, and ongoing forecasting and consolidation of orders and resources in logistics, clustering semantic networks in insurance document processing. Future research is outlined in the areas such as capturing the semantics of problem domains and guided self-organization on the virtual market. Full article