Dear Colleagues,

This Special Issue spotlights the advancements in and applications of Transformer-based deep learning architectures. Transformers have significantly influenced artificial intelligence (AI), particularly natural language processing (NLP), with their innovative approach to handling sequential data. This Special Issue explores the core components of these architectures, including their self-attention mechanism and positional encoding, and discusses recent developments that enhance efficiency, interpretability, and scalability.

The Special Issue also delves into the broad spectrum of applications of Transformers, ranging from traditional tasks such as text summarization, machine translation, and sentiment analysis, to innovative utilizations in language generation and conversational AI, including chatbots and dialogue systems like ChatGPT. Beyond these conventional domains, the Special Issue also highlights breakthrough applications in emerging fields such as computer vision, bioinformatics, health informatics and climate modeling. It provides insight into how models such as BERT and GPT are changing paradigms across various sectors.

Moreover, this Special Issue tackles the existing challenges in utilizing Transformer models, giving readers a well-rounded view of this field. It outlines potential future directions, providing a roadmap for continued innovation. This comprehensive guide offers invaluable insights to researchers, students, and practitioners interested in the cutting edge of deep learning technology.

Dr. Ting Xiao
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Applied Sciences is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

• AI
• NLP
• transformer
• self-attention
• ChatGPT
• BERT
• GPT
• deep learning

Title: Predictive Analysis of X Post Privatization Using ChatGPT and Deep Learning Techniques
Author: Kumar
Highlights: Developed a unique approach using LLMs and NLC for estimating the value of "X" post privatization. Combined traditional financial valuation with news sentiment analysis for precision. Utilized ChatGPT for creating custom transformers tailored to financial predictions. Showcased deep learning's role in modernizing financial forecasting.

Title: Scaling Implicit Bias Analysis across Transformer-based Language Models through Embedding Association Test and Prompt Engineering
Authors: Ravi Varma Kumar Bevara; Nishith Reddy Mannuru; Sai Pranathi Karedla; Ting Xiao
Affiliation: Department of Information Science, Department of Computer Science and Engineering, University of North Texas
Abstract: In the evolving field of machine learning, the deployment of fair and transparent models remains a challenge. This study builds on earlier research showing that neural architectures have built-in biases by looking at a wide range of transformer-based language models, from basic to extra-large configurations, for analyzing movie genre-based bias in reviews. Utilizing the Word Embedding Association Test, it was found that scaling models up tends to mitigate bias, with larger models showing up to a 29% reduction in prejudice. Remarkably, custom prompting emerges as a superior strategy, reducing bias by more than 37% on average. A bias mitigation framework is proposed that harmonizes scale-adapted prompts with model biases, advocating for the refinement of development practices. This study highlights the potential of prompt engineering in shaping model outputs, akin to neural conditioning in biological systems, and paves the way for interdisciplinary approaches to embed fairness into the fabric of transformer models. Despite computational limitations, these findings illuminate the path toward equitable machine intelligence by leveraging the dynamic nature of neural networks and structured prompts.