Special Issue “Natural Language Engineering: Methods, Tasks and Applications”

Natural language engineering includes a continuously enlarging variety of methods for solving natural language processing (NLP) tasks within a pervasive number of applications. In this field, impressive achievements have been reached recently, by means of systems using deep learning or different approaches, which allowed AI to advance toward human levels in NLP tasks such as translation [1], reading comprehension [2,3], information retrieval [4], and sentiment analysis [5,6,7], and to build systems for question answering [8,9,10,11], conversational systems [12,13], and recommender systems (https://developers.google.com/machine-learning/recommendation, accessed on 24 March 2022).

However, despite the remarkable successes in different NLP tasks, natural language engineering is nowadays a field of research of increasing interest due to the remaining difficulties associated with its comprehension and generation, which are capabilities of humans still not well understood by computer systems from a cognitive perspective. Current difficulties include the complexity of deep learning models, growing in directions chosen empirically [14], the difficulty of scaling them down for implementation on the edge, the scarcity of datasets for some languages (https://www.aclweb.org/portal/content/emnlp-workshop-deep-learning-low-resource-nlp, accessed on 24 March 2022), and the lack of explainability of the models [15].

This Special Issue highlights the most recent research being carried out in the field of NLP methods, to face these open issues, with particular emphasis on emerging approaches for learning interactively or autonomously from data, single and multiple language understanding and grounding for analysis and generation, as well as potential or real applications in different domains and everyday devices.

To this aim, this Special Issue gathers original contributions by researchers with broad expertise in various fields—natural language processing, cognitive science and psychology, artificial intelligence and neural networks, computational modeling and neuroscience—discussing their cutting-edge work as well as perspectives on future directions in the whole range of theoretical and practical aspects, technologies, and systems in this research area.

There are six contributions selected for this Special Issue, representing progress and potential applications in the following NLP areas specifically addressed:

1. 

Low-resource natural language processing. Yimam et al. state that the available pre-trained models do not fit well with the need for low-resource languages; thus, they introduce different semantic models for Amharic and fine-tune two pre-trained models and train seven new models. Moreover, they employ these models for different NLP tasks and study their impact.

2. 

Natural language understanding, generation and grounding. Agafonova et al. revisit the receptive theory in the context of computational creativity; they present a fully autonomous text generation engine with raw output simulating the narrative of a mad digital person and discuss the impact of receptive theory, chance discovery, and simulation of fringe mental state on the understanding of computational creativity.

3. 

Neuroscience-inspired cognitive architectures. Onorati et al. propose a model to control a specific class of syntax-oriented neural networks by adding declarative rules, by exploiting parse trees and subtrees, to include human control in NLP systems, and they show that declarative rules representing human knowledge can be effective for some NLP tasks.

4. 

Search and information retrieval. Yu et al. underline that classification of resource can help the filtering of massive resources, and they propose for this scope an Association Content Graph Attention Network, which is based on association features and content attributes of academic resources, considering both semantic relevance and academic relevance, to improve the accuracy of academic resource classification.

5. 

Text de-identification. Libbi et al. consider the lack of large, annotated Electronic Health Records datasets due to privacy concerns and annotation costs, thus they propose the use of language models for generating artificial data jointly with annotations that can be effectively used, alone or in combination with real data, to train supervised named-entity recognition models for de-identification.

6. 

Applications in science, engineering, medicine, healthcare, finance, business, law, education, industry, transportation, retailing, telecommunication and multimedia. Song and Huang propose to use the massive amount of data generated by social media for disaster analysis, and in particular to use Twitter to track disaster events to make a speedy rescue plan, and for this scope, they propose a sentiment-aware contextual model, consisting of a layer that can generate sentimental contextual embeddings from tweets, a BiLSTM layer with attention, and a 1D convolutional layer for local feature extraction, demonstrating superior performance in Tweets-based disaster analysis.