Submit to Entropy Review for Entropy Propose a Special Issue

Journal Menu

Journal Browser

Selected Papers from the 26th International Conference on Artificial Neural Networks - ICANN 2017

Special Issue Editors
Special Issue Information
Published Papers

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".

Deadline for manuscript submissions: closed (28 February 2018) | Viewed by 48814

Share This Special Issue

Special Issue Editors

Prof. Arkadiusz Orlowski

E-Mail Website1 Website2
Guest Editor

Department of Informatics, Faculty of Applied Informatics and Mathematics, Warsaw University of Life Sciences, Warsaw, Poland
Interests: machine learning; image analysis and pattern recognition; artificial neural networks; quantum information and reversible computing; classical and quantum entropies; physics of information; distributed computing; modeling of complex systems
Special Issues, Collections and Topics in MDPI journals

Prof. Dr. Roseli S. Wedemann

E-Mail Website
Guest Editor

Instituto de Matemática e Estatística, Universidade do Estado do Rio de Janeiro, Rua São Francisco Xavier 524, Rio de Janeiro 20550-900, Brazil
Interests: theoretical and computational models of complex systems; biological and artificial neural networks; parallel and distributed computing models; mathematical and computational models of brain and mental processes; computational science; statistical mechanics and complex systems

Special Issue Information

Dear Colleagues,

Research in artificial neural networks involves investigations regarding the computational modeling of connectionist systems, inspired by the functioning of the mind and the brain. Ideas, concepts, methods, and techniques from diverse areas such as learning algorithms, graph theory, and information theory are applied to the study of these complex systems. Models inspired by the functioning of the dynamics of neuronal substrates that describe brain and mental processes, and illustrative simulations of these phenomena, are used both to understand the mind and brain, as well as to approach the issue of the development of artificially intelligent devices. Physical quantities such as entropies that reflect relevant properties of the topologies and dynamics of these complex networks are proposed, measured, and analyzed using theories and methods from statistical mechanics, physics of dynamical systems, information theory, and biological models. The connectionist approach, with artificial neural network models and models based on brain functioning, form the area currently called Computational Neuroscience. With this modeling perspective, enduring issues in the area of Artificial Intelligence, regarding the comprehension of the computability (the mechanics) of the human mind are investigated, and contribute to the current discussion regarding the description of basic mechanisms involved in consciousness and also to the development of intelligent machines.

The International Conference on Artificial Neural Networks (ICANN) is the annual flagship conference of the European Neural Network Society (ENNS). The ideal of ICANN is to bring together researchers from two worlds: information sciences and neurosciences. The scope is wide, ranging from machine learning algorithms to models of real nervous systems. The aim is to facilitate discussions and interactions in the effort towards developing more intelligent computational systems and increasing our understanding of neural and cognitive processes in the brain.

We encourage the authors who have presented an article at the 26th International Conference on Artificial Neural Networks (ICANN 2017) and who feel that their contribution is within the scope of interest of the journal Entropy to submit an original and essential extension of the ICANN paper to be considered for publication.

Prof. Dr. Arkadiusz Orłowski
Prof. Dr. Roseli S. Wedemann
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Published Papers (10 papers)

Download All Papers

Order results

Result details

Show export options Show export options

Select all

Export citation of selected articles as:

Research

18 pages, 1465 KiB

Open AccessArticle

Supervised and Unsupervised End-to-End Deep Learning for Gene Ontology Classification of Neural In Situ Hybridization Images

by Ido Cohen, Eli (Omid) David and Nathan S. Netanyahu

Entropy 2019, 21(3), 221; https://doi.org/10.3390/e21030221 - 26 Feb 2019

Cited by 3 | Viewed by 4244

Abstract

In recent years, large datasets of high-resolution mammalian neural images have become available, which has prompted active research on the analysis of gene expression data. Traditional image processing methods are typically applied for learning functional representations of genes, based on their expressions in these brain images. In this paper, we describe a novel end-to-end deep learning-based method for generating compact representations of in situ hybridization (ISH) images, which are invariant-to-translation. In contrast to traditional image processing methods, our method relies, instead, on deep convolutional denoising autoencoders (CDAE) for processing raw pixel inputs, and generating the desired compact image representations. We provide an in-depth description of our deep learning-based approach, and present extensive experimental results, demonstrating that representations extracted by CDAE can help learn features of functional gene ontology categories for their classification in a highly accurate manner. Our methods improve the previous state-of-the-art classification rate (Liscovitch, et al.) from an average AUC of 0.92 to 0.997, i.e., it achieves 96% reduction in error rate. Furthermore, the representation vectors generated due to our method are more compact in comparison to previous state-of-the-art methods, allowing for a more efficient high-level representation of images. These results are obtained with significantly downsampled images in comparison to the original high-resolution ones, further underscoring the robustness of our proposed method. Full article

(This article belongs to the Special Issue Selected Papers from the 26th International Conference on Artificial Neural Networks - ICANN 2017)

► Show Figures

Figure 1

25 pages, 1450 KiB

Open AccessArticle

Making the Coupled Gaussian Process Dynamical Model Modular and Scalable with Variational Approximations

by Dmytro Velychko, Benjamin Knopp and Dominik Endres

Entropy 2018, 20(10), 724; https://doi.org/10.3390/e20100724 - 21 Sep 2018

Cited by 19 | Viewed by 3947

Abstract

We describe a sparse, variational posterior approximation to the Coupled Gaussian Process Dynamical Model (CGPDM), which is a latent space coupled dynamical model in discrete time. The purpose of the approximation is threefold: first, to reduce training time of the model; second, to enable modular re-use of learned dynamics; and, third, to store these learned dynamics compactly. Our target applications here are human movement primitive (MP) models, where an MP is a reusable spatiotemporal component, or “module” of a human full-body movement. Besides re-usability of learned MPs, compactness is crucial, to allow for the storage of a large library of movements. We first derive the variational approximation, illustrate it on toy data, test its predictions against a range of other MP models and finally compare movements produced by the model against human perceptual expectations. We show that the variational CGPDM outperforms several other MP models on movement trajectory prediction. Furthermore, human observers find its movements nearly indistinguishable from replays of natural movement recordings for a very compact parameterization of the approximation. Full article

(This article belongs to the Special Issue Selected Papers from the 26th International Conference on Artificial Neural Networks - ICANN 2017)

► Show Figures

Figure 1

13 pages, 10460 KiB

Open AccessArticle

Approaching Retinal Ganglion Cell Modeling and FPGA Implementation for Robotics

by Alejandro Linares-Barranco, Hongjie Liu, Antonio Rios-Navarro, Francisco Gomez-Rodriguez, Diederik P. Moeys and Tobi Delbruck

Entropy 2018, 20(6), 475; https://doi.org/10.3390/e20060475 - 19 Jun 2018

Cited by 6 | Viewed by 5476

Abstract

Taking inspiration from biology to solve engineering problems using the organizing principles of biological neural computation is the aim of the field of neuromorphic engineering. This field has demonstrated success in sensor based applications (vision and audition) as well as in cognition and actuators. This paper is focused on mimicking the approaching detection functionality of the retina that is computed by one type of Retinal Ganglion Cell (RGC) and its application to robotics. These RGCs transmit action potentials when an expanding object is detected. In this work we compare the software and hardware logic FPGA implementations of this approaching function and the hardware latency when applied to robots, as an attention/reaction mechanism. The visual input for these cells comes from an asynchronous event-driven Dynamic Vision Sensor, which leads to an end-to-end event based processing system. The software model has been developed in Java, and computed with an average processing time per event of 370 ns on a NUC embedded computer. The output firing rate for an approaching object depends on the cell parameters that represent the needed number of input events to reach the firing threshold. For the hardware implementation, on a Spartan 6 FPGA, the processing time is reduced to 160 ns/event with the clock running at 50 MHz. The entropy has been calculated to demonstrate that the system is not totally deterministic in response to approaching objects because of several bioinspired characteristics. It has been measured that a Summit XL mobile robot can react to an approaching object in 90 ms, which can be used as an attentional mechanism. This is faster than similar event-based approaches in robotics and equivalent to human reaction latencies to visual stimulus. Full article

(This article belongs to the Special Issue Selected Papers from the 26th International Conference on Artificial Neural Networks - ICANN 2017)

► Show Figures

Figure 1

14 pages, 373 KiB

Open AccessArticle

A Novel Boolean Kernels Family for Categorical Data

by Mirko Polato, Ivano Lauriola and Fabio Aiolli

Entropy 2018, 20(6), 444; https://doi.org/10.3390/e20060444 - 06 Jun 2018

Cited by 10 | Viewed by 3153

Abstract

Kernel based classifiers, such as SVM, are considered state-of-the-art algorithms and are widely used on many classification tasks. However, this kind of methods are hardly interpretable and for this reason they are often considered as black-box models. In this paper, we propose a new family of Boolean kernels for categorical data where features correspond to propositional formulas applied to the input variables. The idea is to create human-readable features to ease the extraction of interpretation rules directly from the embedding space. Experiments on artificial and benchmark datasets show the effectiveness of the proposed family of kernels with respect to established ones, such as RBF, in terms of classification accuracy. Full article

(This article belongs to the Special Issue Selected Papers from the 26th International Conference on Artificial Neural Networks - ICANN 2017)

12 pages, 333 KiB

Open AccessArticle

End-to-End Deep Neural Networks and Transfer Learning for Automatic Analysis of Nation-State Malware

by Ishai Rosenberg, Guillaume Sicard and Eli (Omid) David

Entropy 2018, 20(5), 390; https://doi.org/10.3390/e20050390 - 22 May 2018

Cited by 12 | Viewed by 6319

Abstract

Malware allegedly developed by nation-states, also known as advanced persistent threats (APT), are becoming more common. The task of attributing an APT to a specific nation-state or classifying it to the correct APT family is challenging for several reasons. First, each nation-state has more than a single cyber unit that develops such malware, rendering traditional authorship attribution algorithms useless. Furthermore, the dataset of such available APTs is still extremely small. Finally, those APTs use state-of-the-art evasion techniques, making feature extraction challenging. In this paper, we use a deep neural network (DNN) as a classifier for nation-state APT attribution. We record the dynamic behavior of the APT when run in a sandbox and use it as raw input for the neural network, allowing the DNN to learn high level feature abstractions of the APTs itself. We also use the same raw features for APT family classification. Finally, we use the feature abstractions learned by the APT family classifier to solve the attribution problem. Using a test set of 1000 Chinese and Russian developed APTs, we achieved an accuracy rate of 98.6% Full article

(This article belongs to the Special Issue Selected Papers from the 26th International Conference on Artificial Neural Networks - ICANN 2017)

► Show Figures

Figure 1

12 pages, 1250 KiB

Open AccessArticle

Transfer Information Energy: A Quantitative Indicator of Information Transfer between Time Series

by Angel Caţaron and Răzvan Andonie

Entropy 2018, 20(5), 323; https://doi.org/10.3390/e20050323 - 27 Apr 2018

Cited by 8 | Viewed by 3473

Abstract

We introduce an information-theoretical approach for analyzing information transfer between time series. Rather than using the Transfer Entropy (TE), we define and apply the Transfer Information Energy (TIE), which is based on Onicescu’s Information Energy. Whereas the TE can be used as a measure of the reduction in uncertainty about one time series given another, the TIE may be viewed as a measure of the increase in certainty about one time series given another. We compare the TIE and the TE in two known time series prediction applications. First, we analyze stock market indexes from the Americas, Asia/Pacific and Europe, with the goal to infer the information transfer between them (i.e., how they influence each other). In the second application, we take a bivariate time series of the breath rate and instantaneous heart rate of a sleeping human suffering from sleep apnea, with the goal to determine the information transfer heart → breath vs. breath → heart. In both applications, the computed TE and TIE values are strongly correlated, meaning that the TIE can substitute the TE for such applications, even if they measure symmetric phenomena. The advantage of using the TIE is computational: we can obtain similar results, but faster. Full article

(This article belongs to the Special Issue Selected Papers from the 26th International Conference on Artificial Neural Networks - ICANN 2017)

► Show Figures

Figure 1

20 pages, 574 KiB

Open AccessArticle

On the Reduction of Computational Complexity of Deep Convolutional Neural Networks

by Partha Maji and Robert Mullins

Entropy 2018, 20(4), 305; https://doi.org/10.3390/e20040305 - 23 Apr 2018

Cited by 39 | Viewed by 8998

Abstract

Deep convolutional neural networks (ConvNets), which are at the heart of many new emerging applications, achieve remarkable performance in audio and visual recognition tasks. Unfortunately, achieving accuracy often implies significant computational costs, limiting deployability. In modern ConvNets it is typical for the convolution layers to consume the vast majority of computational resources during inference. This has made the acceleration of these layers an important research area in academia and industry. In this paper, we examine the effects of co-optimizing the internal structures of the convolutional layers and underlying implementation of fundamental convolution operation. We demonstrate that a combination of these methods can have a big impact on the overall speedup of a ConvNet, achieving a ten-fold increase over baseline. We also introduce a new class of fast one-dimensional (1D) convolutions for ConvNets using the Toom–Cook algorithm. We show that our proposed scheme is mathematically well-grounded, robust, and does not require any time-consuming retraining, while still achieving speedups solely from convolutional layers with no loss in baseline accuracy. Full article

(This article belongs to the Special Issue Selected Papers from the 26th International Conference on Artificial Neural Networks - ICANN 2017)

► Show Figures

Figure 1

10 pages, 8832 KiB

Open AccessArticle

Optimization of CNN through Novel Training Strategy for Visual Classification Problems

by Sadaqat Ur Rehman, Shanshan Tu, Obaid Ur Rehman, Yongfeng Huang, Chathura M. Sarathchandra Magurawalage and Chin-Chen Chang

Entropy 2018, 20(4), 290; https://doi.org/10.3390/e20040290 - 17 Apr 2018

Cited by 43 | Viewed by 5180

Abstract

The convolution neural network (CNN) has achieved state-of-the-art performance in many computer vision applications e.g., classification, recognition, detection, etc. However, the global optimization of CNN training is still a problem. Fast classification and training play a key role in the development of the CNN. We hypothesize that the smoother and optimized the training of a CNN goes, the more efficient the end result becomes. Therefore, in this paper, we implement a modified resilient backpropagation (MRPROP) algorithm to improve the convergence and efficiency of CNN training. Particularly, a tolerant band is introduced to avoid network overtraining, which is incorporated with the global best concept for weight updating criteria to allow the training algorithm of the CNN to optimize its weights more swiftly and precisely. For comparison, we present and analyze four different training algorithms for CNN along with MRPROP, i.e., resilient backpropagation (RPROP), Levenberg–Marquardt (LM), conjugate gradient (CG), and gradient descent with momentum (GDM). Experimental results showcase the merit of the proposed approach on a public face and skin dataset. Full article

(This article belongs to the Special Issue Selected Papers from the 26th International Conference on Artificial Neural Networks - ICANN 2017)

► Show Figures

Figure 1

17 pages, 5782 KiB

Open AccessArticle

Simulation Study on the Application of the Generalized Entropy Concept in Artificial Neural Networks

by Krzysztof Gajowniczek, Arkadiusz Orłowski and Tomasz Ząbkowski

Entropy 2018, 20(4), 249; https://doi.org/10.3390/e20040249 - 03 Apr 2018

Cited by 13 | Viewed by 3900

Abstract

Artificial neural networks are currently one of the most commonly used classifiers and over the recent years they have been successfully used in many practical applications, including banking and finance, health and medicine, engineering and manufacturing. A large number of error functions have been proposed in the literature to achieve a better predictive power. However, only a few works employ Tsallis statistics, although the method itself has been successfully applied in other machine learning techniques. This paper undertakes the effort to examine the

q

-generalized function based on Tsallis statistics as an alternative error measure in neural networks. In order to validate different performance aspects of the proposed function and to enable identification of its strengths and weaknesses the extensive simulation was prepared based on the artificial benchmarking dataset. The results indicate that Tsallis entropy error function can be successfully introduced in the neural networks yielding satisfactory results and handling with class imbalance, noise in data or use of non-informative predictors. Full article

(This article belongs to the Special Issue Selected Papers from the 26th International Conference on Artificial Neural Networks - ICANN 2017)

► Show Figures

Figure 1

19 pages, 1161 KiB

Open AccessArticle

An Adaptive Learning Based Network Selection Approach for 5G Dynamic Environments

by Xiaohong Li, Ru Cao and Jianye Hao

Entropy 2018, 20(4), 236; https://doi.org/10.3390/e20040236 - 29 Mar 2018

Cited by 3 | Viewed by 3406

Abstract

Networks will continue to become increasingly heterogeneous as we move toward 5G. Meanwhile, the intelligent programming of the core network makes the available radio resource be more changeable rather than static. In such a dynamic and heterogeneous network environment, how to help terminal users select optimal networks to access is challenging. Prior implementations of network selection are usually applicable for the environment with static radio resources, while they cannot handle the unpredictable dynamics in 5G network environments. To this end, this paper considers both the fluctuation of radio resources and the variation of user demand. We model the access network selection scenario as a multiagent coordination problem, in which a bunch of rationally terminal users compete to maximize their benefits with incomplete information about the environment (no prior knowledge of network resource and other users’ choices). Then, an adaptive learning based strategy is proposed, which enables users to adaptively adjust their selections in response to the gradually or abruptly changing environment. The system is experimentally shown to converge to Nash equilibrium, which also turns out to be both Pareto optimal and socially optimal. Extensive simulation results show that our approach achieves significantly better performance compared with two learning and non-learning based approaches in terms of load balancing, user payoff and the overall bandwidth utilization efficiency. In addition, the system has a good robustness performance under the condition with non-compliant terminal users. Full article

(This article belongs to the Special Issue Selected Papers from the 26th International Conference on Artificial Neural Networks - ICANN 2017)

► Show Figures

Journal Menu

Journal Browser

Selected Papers from the 26th International Conference on Artificial Neural Networks - ICANN 2017

Share This Special Issue

Special Issue Editors

Special Issue Information

Published Papers (10 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI