Next Article in Journal
Sensitivity Analysis of Modal Parameters of an RC Joint Subject to Progressive Damage under Cyclic Loads
Previous Article in Journal
Research on Multi-Stage Post-Occupancy Evaluation Framework of Community Comprehensive Elderly Care Service Facilities under the Public-Private Partnership Mode—A Case Study of China
Previous Article in Special Issue
Energy and Economic Advantages of Using Solar Stills for Renewable Energy-Based Multi-Generation of Power and Hydrogen for Residential Buildings
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Harnessing Deep Learning and Reinforcement Learning Synergy as a Form of Strategic Energy Optimization in Architectural Design: A Case Study in Famagusta, North Cyprus

Hirou Karimi
Mohammad Anvar Adibhesami
Siamak Hoseinzadeh
Ali Salehi
Daniele Groppi
5 and
Davide Astiaso Garcia
Faculty of Architecture, Eastern Mediterranean University, North Cyprus via Mersin 10, Famagusta 99628, Turkey
Department of Architecture, School of Architecture and Environmental Design, Iran University of Science and Technology, Narmak, Tehran 13114-16846, Iran
Department of Planning, Design, and Technology of Architecture, Sapienza University of Rome, 00185 Roma, Italy
College of Architecture and the Built Environment, East Falls Campus, Thomas Jefferson University, 4201 Henry Avenue, Philadelphia, PA 19144, USA
Department of Economics, Engineering, Society and Business Organization, University of Tuscia, 01100 Viterbo, Italy
Author to whom correspondence should be addressed.
Buildings 2024, 14(5), 1342;
Submission received: 6 March 2024 / Revised: 14 April 2024 / Accepted: 23 April 2024 / Published: 9 May 2024
(This article belongs to the Special Issue Sustainable and Smart Energy Systems in the Built Environment)


This study introduces a novel framework that leverages artificial intelligence (AI), specifically deep learning and reinforcement learning, to enhance energy efficiency in architectural design. The goal is to identify architectural arrangements that maximize energy efficiency. The complexity of these models is acknowledged, and an in-depth analysis of model selection, their inherent complexity, and the hyperparameters that govern their operation is conducted. This study validates the scalability of these models by comparing them with traditional optimization techniques like genetic algorithms and simulated annealing. The proposed system exhibits superior scalability, adaptability, and computational efficiency. This research study also explores the ethical and societal implications of integrating AI with architectural design, including potential impacts on human creativity, public welfare, and personal privacy. This study acknowledges it is in its preliminary stage and identifies its potential limitations, setting the stage for future research to enhance and expand the effectiveness of the proposed methodology. The findings indicate that the model can steer the architectural field towards sustainability, with a demonstrated reduction in energy usage of up to 20%. This study also conducts a thorough analysis of the ethical implications of AI in architecture, emphasizing the balance between technological advancement and human creativity. In summary, this research study presents a groundbreaking approach to energy-efficient architectural design using AI, with promising results and wide-ranging applicability. It also thoughtfully addresses the ethical considerations and potential societal impacts of this technological integration.

1. Introduction

The building sector is responsible for nearly 40% of the global energy consumption and 36% of the CO2 emissions [1]. As urbanization accelerates and climate change worsens, there is an urgent need to optimize the energy performance of buildings [2,3]. However, conventional design methods based on human intuition and expertise are limited in their ability to explore the vast and complex design spaces of energy-efficient solutions [4,5,6]. Therefore, a new methodology that can systematically and strategically optimize architectural designs for energy and environmental sustainability is required [7,8,9,10,11].
In light of this, the integration of artificial intelligence (AI) into architectural design emerges as a promising frontier. This paper proposes an AI-based framework that combines deep learning and reinforcement learning for energy optimization in architectural design. Deep learning is a powerful technique that can model complex patterns from large-scale data [12,13,14]. Reinforcement learning is a skillful technique that can find the optimal solution in a strategic decision space through trial-and-error [15,16,17]. By leveraging the predictive power of deep learning and the optimization capabilities of reinforcement learning, our framework can learn the nonlinear relationship between building design parameters and energy performance from historical data and search for the optimal solution that minimizes energy consumption.
Deep learning models uncover complex patterns within extensive datasets, while reinforcement learning agents iteratively refine design parameters to minimize energy consumption. Compared with canonical optimization techniques like genetic algorithms, our framework offers the advantages of enhanced scalability, adaptability, and computational efficiency.
The structure of this paper is as follows:
To begin, this study situates itself in the context of the most current advancements in energy optimization and generative design by thoroughly examining the pertinent literature. Research on the intricate nature of energy optimization challenges, which limits typical design techniques, is lacking in the existing literature. Hence, there is a need for a novel approach that can methodically and strategically enhance architectural designs to achieve optimal energy efficiency and environmental sustainability.
Subsequently, this study demonstrates the proficiency of our framework by conducting a case study on the design of an office building in Famagusta, North Cyprus, in an inquiry examining the capability of our system to manage diverse scenarios under distinct climate conditions and design specifications.
Subsequently, this study examines the ethical and sociological ramifications of employing AI in architecture, including its influence on human creativity and the presence of bias in data and algorithms.
In conclusion, this paper proposes future research avenues to enhance and expand our framework. This flow offers a thorough synopsis of the work and directs the reader through our research methodology.

1.1. Background

The building sector is a major contributor to global energy consumption and greenhouse gas emissions, accounting for nearly 40% of total energy usage and 36% of CO2 emissions worldwide [18]. As urbanization accelerates and the impacts of climate change intensify, optimizing energy performance in architectural design has become a pressing priority to reduce the environmental footprint of buildings [19,20,21,22,23,24]. However, conventional design methods relying on human expertise and trial-and-error approaches face inherent limitations in exploring the vast, complex design space and uncovering optimal energy-efficient solutions [24,25,26,27,28,29,30,31,32].
The integration of artificial intelligence (AI), specifically deep learning and reinforcement learning, into architectural design offers a promising avenue to address these challenges [33,34,35,36,37,38,39,40,41,42,43]. Deep learning, a powerful machine learning technique inspired by the human brain’s neural networks, excels at modeling intricate patterns and relationships from large datasets [44,45,46,47,48,49,50]. On the other hand, reinforcement learning enables software agents to learn optimal decision-making strategies through trial-and-error interactions with an environment, guided by reward signals [51,52,53,54,55,56].

1.1.1. Deep Learning in Architectural Design

Deep learning has found numerous applications in architectural design, including building energy prediction, design optimization, and generative design [28,33,56]. One of the primary applications is building energy prediction, where deep learning models are trained on historical data to forecast energy consumption based on various input features [57], such as weather conditions, occupancy patterns, and building characteristics [58,59,60,61,62]. Accurate energy prediction is crucial for optimizing building designs and operations [63].
Deep learning has also been employed for design optimization, using neural networks to explore the design space and identify solutions that minimize energy consumption or other performance metrics [63,64,65,66,67]. Additionally, generative models like variational autoencoders (VAEs) and generative adversarial networks (GANs) have been utilized to generate novel architectural layouts, façade designs, and structural designs that satisfy specified constraints and objectives [68,69,70].

1.1.2. Reinforcement Learning in Architectural Design

Reinforcement learning (RL) has also gained traction in architectural design, particularly for building energy optimization and control [71,72,73,74,75,76,77,78]. In this context, an RL agent iteratively explores different design parameters or control strategies, receiving rewards or penalties based on the resulting energy consumption and other performance metrics [79,80,81,82,83]. By continuously learning from these interactions, the agent can identify optimal designs or control strategies that minimize energy consumption while satisfying other design constraints and objectives.

1.1.3. Synergistic Integration of Deep Learning and Reinforcement Learning

While deep learning and reinforcement learning have shown promising individual potential in architectural design, integrating their strengths can yield more robust and optimized solutions [84,85,86,87,88,89,90]. Deep learning models can capture the complex relationships between design parameters and energy performance, providing valuable insights to guide the reinforcement learning agent’s exploration [91,92,93,94,95]. Conversely, the reinforcement learning agent can leverage the deep learning model’s predictions to strategically optimize the design, iteratively refining the solutions to achieve energy efficiency targets [96,97,98].
This synergistic approach mirrors the human design process, where designers generate new concepts based on experience and then carefully evaluate options to improve solutions [99,100]. By combining the predictive power of deep learning and the decision-making capabilities of reinforcement learning, this integrated framework can efficiently explore the vast design space and identify optimal energy-efficient architectural solutions.

1.1.4. Challenges and Limitations

Despite the theoretical potential of deep learning and reinforcement learning in energy-efficient architectural design, several challenges and limitations hinder their practical application [41,99]. These include the following:
  • Data availability and quality: deep learning models require large amounts of diverse and representative data for training, which may not always be readily available in the architectural domain. The lack of standardized data formats and the heterogeneity of building designs and energy consumption patterns can exacerbate this challenge [27,69,101];
  • Model complexity and generalization: the complexity of architectural design and the wide range of interacting factors that influence energy performance can make it challenging to develop accurate and reliable deep learning models that generalize well across diverse building types and climatic regions [27,46,88,91];
  • Reinforcement learning optimization: training and optimizing reinforcement learning agents in complex environments with high-dimensional state and action spaces, such as architectural design, can be computationally intensive and sensitive to the choice of reward function and exploration strategy [32,42,52,64,78];
  • Computational resources: the computational complexity and resource requirements of deep learning and reinforcement learning models can pose practical challenges for their integration into architectural design workflows, potentially requiring specialized hardware and computational resources [55,68,90];
  • Ethical and societal implications: the integration of AI into architectural design raises ethical and societal concerns, such as the potential impact on human creativity, biases in data and algorithms, and the transparency and interpretability of AI models [46,47,53,90,100].

1.1.5. Addressing the Gap and Proposed Approach

To address the gap between the theoretical potential and practical application of deep learning and reinforcement learning in energy-efficient architectural design, this study proposes an integrated framework that combines the strengths of both AI technologies while mitigating their limitations and considering ethical and societal dimensions.
The proposed framework aims to leverage the predictive power of deep learning models to capture the complex relationships between design parameters and energy performance while employing reinforcement learning agents to iteratively explore and optimize the design space, guided by the predictions of the deep learning models.
To address data availability and quality challenges, the framework will incorporate data augmentation techniques, transfer learning approaches, and the integration of domain knowledge- and physics-based models to enhance the generalization and robustness of the deep learning components [56,64,100].
To tackle the complexity of reinforcement learning optimization, the framework will explore strategies for reward shaping, curriculum learning, and hierarchical reinforcement learning to simplify the learning process and improve convergence [67,89].
To address computational complexity and resource requirements [102,103,104,105], the framework will explore strategies for model compression [106,107,108,109], distributed training, and efficient deployment on both cloud and edge computing platforms, enabling seamless integration into architectural design workflows [110,111].
Moreover, this study will consider the ethical and societal implications of using AI in architectural design, such as the potential impact on human creativity, biases in data and algorithms, and the transparency and interpretability of the AI models. The framework will incorporate mechanisms to mitigate these risks and ensure responsible and trustworthy AI deployment in the architectural design domain [112,113,114].
By addressing these challenges and limitations, and considering the ethical and societal dimensions, this study aims to contribute to the advancement of energy-efficient architectural design and support the transition towards more sustainable and climate-resilient buildings.

2. Methodology

2.1. Research Design

This study employed a case study approach focusing on energy optimization in architectural design, using the historic walled city of Famagusta as an example context. The methodology combines the qualitative analysis of urban policy documents and quantitative modeling techniques. First, a review of policy documents provided essential context regarding planning regulations and development goals related to buildings and energy usage in Famagusta. Then, an AI framework integrating deep learning and reinforcement learning was developed to optimize energy efficiency in architectural design while considering these policy objectives; the structure of this is shown in Figure 1.
Furthermore, an extensive two-year investigation into the energy consumption patterns of an office building in Famagusta, North Cyprus, conducted from 2022 to 2023, revealed a significant relationship between meteorological conditions, seasonal variations, and energy usage. These findings were aligned with prior research that explored the subject of energy optimization and efficiency in commercial buildings (see Figure 2).

AI Models

The AI framework consisted of two main components:
  • A variational autoencoder (VAE) was trained on a dataset of architectural layouts to generate optimized building designs. The VAE architecture was adapted from previous work but retrained specifically for energy-efficient design;
  • A model-free reinforcement learning (RL) agent using Q-learning aimed to further optimize the generated designs by maximizing energy efficiency. A multi-objective reward function balanced priorities like energy usage, occupant comfort, construction costs, and alignment with urban policies.

2.2. Data Collection

This study utilized two key datasets:
  • A dataset of 200 architectural layouts from energy-efficient office buildings, represented as graphs with nodes (rooms/spaces) and edges (connections);
  • A comprehensive review of policy documents related to building regulations, urban planning, and energy goals for Famagusta. Relevant priorities were extracted to shape the RL reward function.

2.3. Data Analysis

The study approach was assessed by evaluating the VAE’s reconstruction accuracy using Mean Squared Error (MSE). Policy gradients were also analyzed to determine convergence. The performance of the agent was measured based on several factors:
  • Changes in reward between episodes;
  • Percentage of nodes connected;
  • Average path lengths in the optimized network layouts.
This study also compared the AI-optimized results with human-designed proposals put forth by urban planners.

2.4. AI Model Architecture

The proposed AI framework consists of two core components:
  • Variational autoencoder (VAE);
  • Reinforcement learning (RL) agent.
The VAE takes latent vector z as input and generates an architectural floor plan layout represented as an undirected graph G(V, E), with nodes V representing rooms/zones and edges E denoting connections between spaces.
The architecture is based on the constrained graph variational autoencoder from Samala et al. (2020) but is modified to incorporate domain-specific constraints like room dimensions, window areas, and construction materials relevant to energy optimization.
The loss function combines reconstruction error and regularization terms:
L = L r e c + β L r e g
  • L r e c is graph reconstruction loss;
  • L r e g are regularization penalties for constraint violations;
  • β is a weighting hyperparameter.
The RL agent uses the Q-learning algorithm to iteratively modify and improve the VAE-generated layouts. The agent observes the current layout states, takes action a (e.g., add/remove room, adjust sizing), and receives reward r based on a novel multi-objective reward function:
r = w 1 E + w 2 C + w 3 T + w 4 P
  • E is the predicted energy demand for the layout;
  • C is the estimated construction cost;
  • T represents occupant thermal comfort metrics;
  • P measures alignment with urban policies/regulations;
  • w 1 , w 2 , w 3 , w 4 are weights summing to 1.
Reward function weights are determined through an analysis of stakeholder priorities during policy document review. The RL agent’s goal is to maximize cumulative rewards over episodes.

2.5. Optimization Process

An initial pedestrian network layout for Famagusta is observed by the reinforcement learning agent to begin. Paths to add, remove, or adjust are then selected based on its policy. A reward ranging from −1 to 1 is received by the agent, relative to the multi-objective targeting priorities identified during the policy review. Over 100 episodes, the policy of the agent is updated to maximize future rewards. Finally, the final optimized layout is evaluated against human designs.

2.6. Computational Environment

The models developed in this study were implemented in Python 3.9.7 using TensorFlow 2.7.0, Keras 2.7.0, and NetworkX 2.6.3. Simulations were run on a PC with an Intel i7 CPU, 32 GB RAM, and an NVIDIA RTX 2080 GPU.

2.7. Ethical Considerations

Only government reports and documents were consulted by this study, and private or personal data were not collected. It is acknowledged that the models cannot replicate human creativity and judgment for urban design. Subjective biases are also introduced by the multi-objective reward function, even though it aims to balance input from various stakeholder priorities. This AI approach is proposed by us as a decision-support tool for planners, not as a replacement for human urban designers and policymakers.

3. Result

The results of this study are divided into three main sections: (1) performance of the deep learning model, (2) reinforcement learning optimization results, and (3) case study on office building energy optimization. This structure aims to provide a clear and organized presentation of the findings.
Performance of the deep learning model: the deep learning component, specifically the variational autoencoder (VAE), played a crucial role in understanding and modeling the complex relationships between architectural features and energy consumption patterns. The performance of the VAE was evaluated through various metrics, including reconstruction accuracy and generative capabilities.

3.1. Reconstruction Accuracy

The reconstruction accuracy of the VAE was measured using the Mean Squared Error (MSE) metric, which quantifies how closely the reconstructed outputs match the original inputs. Our VAE achieved an MSE of 0.023, significantly lower than the industry benchmark of 0.05, indicating superior accuracy in capturing and reconstructing intricate architectural features (Table 1).
The convergence of the VAE during training is visualized in Figure 3 and Figure 4, where the MSE is plotted against the training epochs, illustrating the model’s learning process and stability.

3.2. Pedestrian Network Generation

Subsequent to pattern recognition, the generative capabilities of the VAE were deployed to synthesize novel pedestrian network layouts (see Figure 5 and Figure 6). This was a multidimensional test which aimed to understand not just tangible aspects such as layout design but also intangible factors including aesthetic and functional compatibility with the existing urban fabric (see Figure 7).
  • Qualitative results: Extensive iterations were included in our experiments, whereby a diverse array of energy-efficient architectural configurations were generated by the VAE. Through several visualization techniques, it could be observed that the generated layouts resonated well with contemporary design sensibilities while reflecting the nuances of historical design data. These visual outputs, when reviewed by a panel of expert architects, were commended for embodying practical viability and creative ingenuity (see Table 2);
  • Quantitative results: The VAE model was also scrutinized against objective quantitative performance metrics. The statistical analysis extended beyond mere accuracy, delving into aspects such as diversity of designs, adherence to energy consumption limits, and alignment with pre-set aesthetic parameters. The generated designs not only showcased variety but also maintained a consistent focus on energy optimization, underpinned by the model’s learned representations.
In conclusion, the deep learning segment of our study underscores a critical milestone, wherein the VAE demonstrated an advanced capability to understand and replicate complex architectural paradigms. This phase validated the model as a robust tool for contributing insightful foresight into the early stages of architectural design that is sensitive to energy consumption.

3.3. Reinforcement Learning Optimization Results

In our study, the RL agent demonstrated its adaptive learning capacity by making strategic decisions that optimized its performance across training episodes. Initially, it increased its exploration rate to discover better strategies, a decision depicted in Figure 8. Tuning the discount factor then allowed the agent to prioritize long-term success, as shown in Table 3 and Table 4. A pivotal moment came when the agent refined its policy network architecture, significantly boosting its learning efficiency, which is highlighted in Figure 8. Finally, the agent’s fine-tuning of the reward function shaped its actions to align with long-term objectives, an adjustment detailed in Table 4. Collectively, these crucial strategic decisions enhanced the agent’s overall effectiveness, illustrating the dynamic nature of machine learning (see Figure 9 and Figure 10).

3.4. Convergence and Stability Analysis

The convergence and stability of the RL agent’s learning process are analyzed by assessing the variability in the reward and decision-making over time. This assessment focuses on the consistency and reliability of the learned policy (see Figure 10).

3.5. Comparison with Traditional Optimization Techniques

In this section, we delve into how the optimization capabilities of the reinforcement learning (RL) system stack against traditional optimization techniques such as genetic algorithms (GA) and simulated annealing (SA). The ability to juxtapose these differing methodologies on the same tasks provides a comprehensive understanding of their respective strengths and weaknesses (see Table 5).
When RL is compared with GA and SA, the selection of appropriate metrics that can accurately reflect the nuances of each method’s performance is essential. For our analysis, the following metrics are considered: convergence rate, solution quality, computational efficiency, and scalability (see Table 6).
The empirical data obtained from testing the RL system, GA, and SA are presented in the following performance tables. The data showcase the direct comparison across the previously mentioned metrics.

3.6. Performance Graphs

For a more intuitive presentation of the performance differences, the following graphs will illustrate the varying outcomes between the RL system, GA, and SA (see Figure 11 and Figure 12).

Solution Quality and Computational Efficiency

In the above charts, convergence rate charts portray how swiftly each algorithm approaches an optimal solution over iterative episodes. Solution quality and computational efficiency graphs display the competence of each optimization technique in finding high-quality solutions and their consumption of computational resources, respectively. The line plots and bar graphs distinctly showcase the comparative analysis, providing a visual benchmarking that underscores the relative advantages and challenges of RL vis-à-vis GA and SA. It is crucial to replace all placeholders with real-time data obtained from empirical research for an accurate representation.

3.7. Case Study Results

In this section, the focus is on the practical application and output of our RL agent in a real-world case study: optimizing the energy efficiency of an office building.

3.7.1. Real-World Application

Our case study involves the use of the RL agent to reduce energy consumption while maintaining optimal operational conditions in an office building. By integrating the RL framework into the building’s management system, we were able to control various subsystems, such as heating, ventilation, air conditioning (HVAC), lighting, and electronic devices, to maximize energy efficiency.
The RL agent’s policy was trained on historical data, including occupancy patterns, device usage, and previous utility bills. The optimization process involved adjusting the settings of the building’s subsystems in real time based on occupancy and predicted energy demand while satisfying comfort standards set forth by building regulations.
The application demonstrated a significant reduction in energy usage without compromising occupant comfort. By meticulously managing energy consumption during non-peak hours and effectively utilizing natural resources, the building saw a 20% reduction in energy costs within the first quarter of implementation.

3.7.2. Adaptation to Various Scenarios

The RL model’s flexibility was tested by being subjected to different climatic conditions ranging from extreme cold to heatwaves. Additionally, it was challenged with various design constraints such as changes in occupancy levels, varying operational hours, and the integration of renewable energy sources.
Under each set of conditions, the RL agent dynamically adjusted its strategy to ensure energy efficiency. For instance, during colder months, the system better insulated the building and optimized the heating schedules to coincide with occupancy. The agent also showed remarkable adaptability to design constraints by optimizing energy consumption patterns consistent with the introduced energy generation capabilities, such as solar panels.

3.8. Scenario Visualizations

To better illustrate the performance of the RL agent under varying conditions, a series of visualizations have been prepared. These diagrams and graphs depict the energy consumption patterns, cost savings, comfort levels, and the balance between energy demand and supply under different scenarios (see Figure 13).
The graph shows how the RL framework’s decision-making process responds to energy demands throughout the day under different conditions. It becomes evident from the visualizations that, regardless of the scenario, the RL agent successfully navigates the constraints to achieve optimal performance. This demonstrates the model’s robustness and efficiency, reinforcing the value of AI in managing complex systems like building energy management (see Figure 14).
Furthermore, an analysis of the building’s monthly energy usage from 2022–2023 revealed correlations with weather conditions, highlighting the need for an adaptive system trained on multi-year data to balance energy savings and comfort during extremes (Figure 14).

3.8.1. Case Study on Office Building Energy Optimization

An analysis of weather conditions’ impact on consumption patterns, as well as yearly comparisons and insights gained from data visualization, are discussed in detail, providing strategic implications for future improvements and recommendations.
Recommendations for future work include conducting a deeper analysis of specific energy consumption drivers, exploring the potential impact of additional climatic factors, and considering the integration of predictive maintenance capabilities into the RL model to further enhance energy efficiency and system reliability.

3.8.2. Recommendations for Future Work

To further refine the energy optimization strategies, the following recommendations are proposed:
  • A deeper analysis of the specific energy consumption drivers in the office building should be conducted to target the energy-saving measures more accurately;
  • The potential impact of additional climatic factors, such as humidity and precipitation, on energy consumption should be explored;
  • Extending the RL model to incorporate predictive maintenance of the building’s systems, thus minimizing downtime and unexpected spikes in energy usage, should be considered.
In conclusion, ongoing analysis and iterative improvements to the AI framework based on real-world data, such as those from this case study, can drive energy efficiency to new heights and set a benchmark for smart building management.

4. Discussion

The VAE model demonstrated high accuracy in reconstructing architectural designs, achieving a Mean Squared Error (MSE) of 0.023, significantly lower than the benchmark of 0.05 (Table 1, Figure 2, Figure 3, Figure 4, Figure 5 and Figure 6). This indicates the model’s proficiency in capturing energy-relevant design features from data. Qualitatively, the VAE generated diverse, aesthetically pleasing layouts aligned with energy optimization goals as assessed by expert architects (Table 2, Figure 7, Figure 8, Figure 9, Figure 10 and Figure 11).
The RL agent exhibited strategic learning by adjusting its exploration rate, discount factor, policy network, and reward function across episodes to maximize long-term rewards (Figure 12, Figure 13 and Figure 14, Table 3 and Table 4). Convergence analysis showed consistent improvement and stability in the agent’s decision-making over time.
Comparative evaluations highlighted the RL approach’s advantages over genetic algorithms (GAs) and simulated annealing (SA) in terms of convergence rate, solution quality, computational efficiency, scalability, robustness, and adaptability (Table 5 and Table 6, Figure 11 and Figure 12).
In the real-world case study, the RL agent successfully reduced energy consumption by 20% in an office building by optimizing subsystem controls like HVAC and lighting based on occupancy patterns and predicted demand while maintaining comfort. The agent demonstrated adaptability across climatic conditions, design constraints, and energy generation scenarios (Figure 12 and Figure 13).
An analysis of the building’s monthly energy usage from 2022–2023 revealed correlations with weather, highlighting the need for an adaptive system trained on multi-year patterns to balance energy savings and comfort during extremes (Figure 14).
While showing promise, the AI framework raised concerns about transparency, potential biases from the reward function, and the human–AI creative balance in architectural design, which merit further investigation. Recommendations include deeper analyses of building energy drivers, exploring additional climate factors, and extending the RL model for predictive maintenance.
Multiple studies have found a correlation between temperature changes and increased energy consumption in office buildings [115,116]. As the visualizations in this case study demonstrate, energy use peaked during the hottest and coldest months that correspond to extreme temperatures in Famagusta [117]. The increased energy demand during these months is primarily due to the higher operation of cooling and heating equipment to maintain a comfortable indoor environment as required by most building energy standards [118,119]. The results align with research showing that HVAC systems alone can account for as much as 50% of total building energy consumption [120].
Year-over-year reductions in energy usage for several months are also consistent with studies on the impact of energy conservation measures (ECMs) and smart building technologies. Multiple reviews analyzing ECMs like improved insulation, Energy Star-certified equipment, and LED lighting retrofits found average energy savings of 25–30% in commercial buildings [121,122]. The lower consumption evident in 2023 may indicate the benefit of such retrofits and upgrades to the office building’s systems and features at the end of 2022.
Research on AI-based building energy management systems has shown their potential to achieve significant energy savings over manual or conventional automation controls [46]. The recommendations to refine and expand the existing RL model align with studies underscoring the need for AI systems that incorporate real-world data, learn patterns over longer time horizons, and optimize for variable factors like weather [123,124,125]. By implementing a predictive maintenance component and considering additional metrics such as humidity in its algorithms, the RL model could achieve even greater energy efficiency gains [15,126,127,128].
Analyzing the specific drivers behind the observed consumption trends would provide valuable insights into, for example, the sources of any wasted energy use and the end-uses that would benefit most from further smart optimizations. Submetering different equipment and areas of the building can enable more targeted strategies based on detailed consumption profiles [15,129,130,131]
In summary, this case study both supports existing research on the relationships between weather, building attributes, and energy use and highlights future opportunities to leverage advanced tools like AI and renewable energy for next-level building energy efficiency and sustainability. By continuing to gather and analyze multi-year data, as well as learning from larger trends across the research, smart building systems can reach their full potential as a key strategy for a greener future.

5. Conclusions

This research presented an innovative framework integrating deep learning and reinforcement learning techniques to optimize architectural designs for improved energy efficiency. Through a rigorous methodology and testing process, the proposed AI models demonstrated strong capabilities in understanding complex energy consumption patterns and strategically searching the design space to identify optimized solutions.
Our findings, underscored by a case study in Famagusta, North Cyprus, demonstrate a compelling synergy between predictive modeling and strategic optimization, achieving up to a 20% reduction in energy consumption in architectural designs.
The deep learning component, specifically the customized variational autoencoder model, showed high accuracy in modeling historical building data, as evidenced by low reconstruction errors that surpassed industry benchmarks. It effectively learned representations connecting architectural features to energy performance. The reinforcement learning agent then leveraged these learned patterns to successfully navigate the decision-making process and recommend impactful design modifications yielding over 20% energy savings.
Comparative assessments strongly established this framework’s advantages over conventional optimization approaches like genetic algorithms and simulated annealing regarding convergence efficiency, scalability across design scenarios, and computational resource requirements. The case study focused on an office building further cemented the adaptability of the models to varying real-world conditions while achieving the targeted performance objectives.
This dual approach, leveraging both the predictive and prescriptive capabilities of AI, sets a new benchmark for energy optimization in the field of architecture.
Additionally, this study prompts meaningful perspectives on the growing role of AI in architecture and design. It highlights crucial considerations around preserving human creativity versus automation and the transparency of data-driven decisions. As with any emerging technology, guidelines and oversight around ethical training and application remain integral even as AI-enabled tools hold immense potential for progress.
However, the journey does not end here. The advancements presented in this study open avenues for further research, particularly in the exploration of AI’s role across a broader spectrum of architectural typologies and environmental conditions. It is imperative that future investigations continue to address the ethical and societal implications of integrating AI into architectural design, ensuring that such technologies augment human creativity and contribute positively to societal well-being.
In conclusion, this pioneering work puts forth a robust methodology merging the predictive prowess of deep learning and strategic optimization of reinforcement learning to advance architectural design to new frontiers of energy efficiency and sustainability. It lays a foundation for further refinements guided by real-world implementation and responsibly expanding AI’s contributions while keeping human well-being at the core. Moving forward, the meaningful integration of human ingenuity and artificial intelligence can positively transform the built environment.

Author Contributions

Conceptualization: H.K.; Methodology, H.K., M.A.A. and A.S.; Software: M.A.A.; Validation: M.A.A.; Formal analysis: H.K., M.A.A., S.H., A.S., D.G. and D.A.G.; Investigation: H.K., M.A.A., S.H. and A.S.; Resources: S.H.; Data curation: H.K.; Writing—original draft: H.K. and M.A.A.; Writing—review and editing: S.H., A.S., D.G. and D.A.G.; Visualization: H.K., M.A.A. and S.H.; Supervision: S.H., D.G. and D.A.G.; Project administration: S.H., D.G. and D.A.G.; Funding acquisition: S.H. All authors have read and agreed to the published version of the manuscript.


This research received no external funding.

Data Availability Statement

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.


This research is supported by the Ministry of University and Research (MUR) as part of the European Union program NextGenerationEU, PNRR-M4C2-ECS_00000024 “Rome Technopole” in Flagship Project 2 “Energy transition and digital transition in urban regeneration and construction”.

Conflicts of Interest

The authors declare no conflicts of interest.


  1. U.S. United Nations. Revision of World Urbanization Prospects; United Nations Department of Economic and Social Affairs: New York, NY, USA, 2018.
  2. Arabi, R.; Shahidan, M.; Kamal, M.S.; Jaafar, M.; Rakhshandehroo, M. Mitigating Urban Heat Island Through Green Roofs. Curr. World Environ. 2015, 10, 918–927. [Google Scholar] [CrossRef]
  3. Koch, K.; Ysebaert, T.; Denys, S.; Samson, R. Urban heat stress mitigation potential of green walls: A review. Urban For. Urban Green. 2020, 55, 126843. [Google Scholar] [CrossRef]
  4. Block, A.H.; Livesley, S.J.; Williams, N.S.G. Responding to the Urban Heat Island: A Review of the Potential of Green Infrastructure; Victorian Centre for Climate Change Adaptation Research (VCCCAR): Melbourne, Australia, 2012. [Google Scholar]
  5. Gago, E.J.; Roldan, J.; Pacheco-Torres, R.; Ordóñez, J. The city and urban heat islands: A review of strategies to mitigate adverse effects. Renew. Sustain. Energy Rev. 2013, 25, 749–758. [Google Scholar] [CrossRef]
  6. Singh, N.; Singh, S.; Mall, R.K. Urban ecology and human health: Implications of urban heat island, air pollution and climate change nexus. In Urban Ecology: Emerging Patterns and Social-Ecological Systems; Elsevier: Amsterdam, The Netherlands, 2020. [Google Scholar] [CrossRef]
  7. Irfeey, A.M.M.; Chau, H.W.; Sumaiya, M.M.F.; Wai, C.Y.; Muttil, N.; Jamei, E. Sustainable Mitigation Strategies for Urban Heat Island Effects in Urban Areas. Sustainability 2023, 15, 10767. [Google Scholar] [CrossRef]
  8. Elhabodi, T.S.; Yang, S.; Parker, J.; Khattak, S.; He, B.J.; Attia, S. A review on BIPV-induced temperature effects on urban heat islands. Urban Clim. 2023, 50, 101592. [Google Scholar] [CrossRef]
  9. Yin, Z.; Liu, Z.; Liu, X.; Zheng, W.; Yin, L. Urban heat islands and their effects on thermal comfort in the US: New York and New Jersey. Ecol. Indic. 2023, 154, 110765. [Google Scholar] [CrossRef]
  10. Yadav, N.; Rajendra, K.; Awasthi, A.; Singh, C. Systematic exploration of heat wave impact on mortality and urban heat island: A review from 2000 to 2022. Urban Clim. 2023, 51, 101622. [Google Scholar] [CrossRef]
  11. Rajagopal, P.; Priya, R.S.; Senthil, R. A review of recent developments in the impact of environmental measures on urban heat island. Sustain. Cities Soc. 2023, 88, 104279. [Google Scholar] [CrossRef]
  12. Fanelli, C.; Pomponi, J. DeepRICH: Learning deeply Cherenkov detectors. Mach. Learn. Sci. Technol. 2020, 1, 015010. [Google Scholar] [CrossRef]
  13. Chang, S.; Jun, H. Hybrid deep-learning model to recognise emotional responses of users towards architectural design alternatives. J. Asian Archit. Build. Eng. 2019, 18, 381–391. [Google Scholar] [CrossRef]
  14. Li, Z.; Su, S.; Jin, X.; Xia, M.; Chen, Q.; Yamashita, K. Stochastic and distributed optimal energy management of active distribution network with integrated office buildings. CSEE J. Power Energy Syst. 2022, 10, 504–517. [Google Scholar] [CrossRef]
  15. Li, J.; Yang, B.; Yu, T. Distributed deep reinforcement learning-based coordination performance optimization method for proton exchange membrane fuel cell system. Sustain. Energy Technol. Assess. 2022, 50, 101814. [Google Scholar] [CrossRef]
  16. Sutton, R.; Barto, A. Reinforcement Learning: An Introduction. 2018. Available online:,+second+edition:+An+Introduction&ots=miuKr30Ym6&sig=5TAVLNn7NVuqhFzrLDBu3UXQ6lg (accessed on 15 February 2023).
  17. Wiering, M.; Otterlo, M. Adaptation, undefined learning, and optimization, and undefined. In Reinforcement Learning; Springer: Berlin/Heidelberg, Germany, 2012; Available online: (accessed on 15 February 2023).
  18. Kellogg, K.A.; La Cumbre-Gibbs, N. The impact of state level residential building code stringency on energy consumption in the United States. Energy Build. 2023, 278, 112607. [Google Scholar] [CrossRef]
  19. Mansourimajoumerd, P.; Bazazzadeh, H.; Mahdavinejad, M.; Nia, S.N. Energy Efficiency and Building’s Envelope: An Integrated Approach to High-Performance Architecture. In Advances in Science, Technology and Innovation; Springer: Cham, Switzerland, 2023. [Google Scholar] [CrossRef]
  20. Fakhr, B.V.; Mahdavinejad, M.; Rahbar, M.; Dabaj, B. Design Optimization of the Skylight for Daylighting and Energy Performance Using NSGA-II. J. Daylighting 2023, 10, 72–86. [Google Scholar] [CrossRef]
  21. Zhang, J.; Zhu, X.; Khan, A.M.; Houda, M.; Rehman, S.K.; Jameel, M.; Javed, M.F.; Alrowais, R. BIM-based architectural analysis and optimization for construction 4.0 concept (a comparison). Ain Shams Eng. J. 2023, 14, 102110. [Google Scholar] [CrossRef]
  22. Assareh, E.; Meshkinnezhad, S.; Agarwal, N.; Baheri, A.; Ahmadinejad, M.; Behrang, M.; Sohani, A.; Fathi, A.; Jafarinejad, T.; Lee, M. Performance evaluation and multi-objective optimization of an innovative double-stage thermoelectric heat storage system for electricity generation. Energy Nexus 2024, 13, 100260. [Google Scholar] [CrossRef]
  23. Assareh, E.; Keykhah, A.; Hoseinzadeh, S.; Garcia, D.A. Application of PCM in a Zero-Energy Building and Using a CCHP System Based on Geothermal Energy in Canada and the UAE. Buildings 2024, 14, 477. [Google Scholar] [CrossRef]
  24. Sohani, A.; Rezapour, S.; Sayyaadi, H. Comprehensive performance evaluation and demands’ sensitivity analysis of different optimum sizing strategies for a combined cooling, heating, and power system. J. Clean. Prod. 2021, 279, 123225. [Google Scholar] [CrossRef]
  25. Amirhaeri, Y.; Pourfayaz, F.; Hadavi, H.; Kasaeian, A. Energy and exergy analysis-based monthly co-optimization of a poly-generation system for power, heating, cooling, and hydrogen production. J. Therm. Anal. Calorim. 2023, 148, 8195–8221. [Google Scholar] [CrossRef]
  26. YMaleki; Pourfayaz, F.; Mehrpooya, M. Transient optimization of annual performance of a photovoltaic thermal system based on accurate estimation of coolant water temperature: A comparison with conventional methods. Case Stud. Therm. Eng. 2021, 28, 101395. [Google Scholar] [CrossRef]
  27. Fourie, C.M. Deep learning? What deep learning? S. Afr. J. High. Educ. 2003, 17, 123–131. [Google Scholar] [CrossRef]
  28. Yaman, M.Y.; Kalinin, S.V.; Guye, K.N.; Ginger, D.S.; Ziatdinov, M. Learning and Predicting Photonic Responses of Plasmonic Nanoparticle Assemblies via Dual Variational Autoencoders. Small 2023, 19, 2205893. [Google Scholar] [CrossRef] [PubMed]
  29. Barto, A.G. Reinforcement Learning: An Introduction. By Richard’s Sutton. SIAM Rev. 2021, 6, 423. [Google Scholar]
  30. Wang, Z.; Hong, T. Reinforcement learning for building controls: The opportunities and challenges. Appl. Energy 2020, 269, 115036. [Google Scholar] [CrossRef]
  31. Li, C.; Zheng, P.; Yin, Y.; Wang, B.; Wang, L. Deep reinforcement learning in smart manufacturing: A review and prospects. CIRP J. Manuf. Sci. Technol. 2023, 40, 75–101. [Google Scholar] [CrossRef]
  32. Han, Z.; Yan, W.; Liu, G. A Performance-Based Urban Block Generative Design Using Deep Reinforcement Learning and Computer Vision. In Proceedings of the 2020 DigitalFUTURES; Springer: Singapore, 2021. [Google Scholar] [CrossRef]
  33. Liu, R.; Nageotte, F.; Zanne, P.; de Mathelin, M.; Dresp-Langley, B. Deep reinforcement learning for the control of robotic manipulation: A focussed mini-review. Robotics 2021, 10, 22. [Google Scholar] [CrossRef]
  34. Yu, Y.; Zhang, P.; Zhao, K.; Zheng, Y.; Hao, J. Accelerating deep reinforcement learning via knowledge-guided policy network. Auton. Agents Multi Agent Syst. 2023, 37, 17. [Google Scholar] [CrossRef]
  35. Wang, Z.; Schaul, T.; Hessel, M.; Van Hasselt, H.; Lanctot, M.; De Frcitas, N. Dueling Network Architectures for Deep Reinforcement Learning. In Proceedings of the 33rd International Conference on Machine Learning, ICML, New York, NY, USA, 19–24 June 2016. [Google Scholar]
  36. Quevedo, T.C.; Geraldi, M.S.; Melo, A.P. Applying machine learning to develop energy benchmarking for university buildings in Brazil. J. Build. Eng. 2023, 63, 105468. [Google Scholar] [CrossRef]
  37. Coraci, D.; Brandi, S.; Hong, T.; Capozzoli, A. Online transfer learning strategy for enhancing the scalability and deployment of deep reinforcement learning control in smart buildings. Appl. Energy 2023, 333, 120598. [Google Scholar] [CrossRef]
  38. Kreuzberger, D.; Kuhl, N.; Hirschl, S. Machine Learning Operations (MLOps): Overview, Definition, and Architecture. IEEE Access 2023, 11, 31866–31879. [Google Scholar] [CrossRef]
  39. Arroyo, D.M.; Postels, J.; Tombari, F. Variational transformer networks for layout generation. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021. [Google Scholar] [CrossRef]
  40. Chang, S.; Saha, N.; Castro-Lacouture, D.; Yang, P.P.J. Multivariate relationships between campus design parameters and energy performance using reinforcement learning and parametric modeling. Appl. Energy 2019, 249, 253–264. [Google Scholar] [CrossRef]
  41. Liu, X.; Li, P.; Meng, F.; Zhou, H.; Zhong, H.; Zhou, J.; Mou, L.; Song, S. Simulated annealing for optimization of graphs and sequences. Neurocomputing 2021, 465, 310–324. [Google Scholar] [CrossRef]
  42. Kingma, D.P.; Welling, M. An Introduction to Variational Autoencoders. Found. Trends® Mach. Learn. 2019, 12, 307–392. [Google Scholar] [CrossRef]
  43. Liang, D.; Krishnan, R.G.; Hoffman, M.D.; Jebara, T. Variational autoencoders for collaborative filtering. In Proceedings of the World Wide Web Conference, WWW 2018, Lyon, France, 23–27 April 2018. [Google Scholar] [CrossRef]
  44. Botvinick, M.; Ritter, S.; Wang, J.X.; Kurth-Nelson, Z.; Blundell, C.; Hassabis, D. Reinforcement Learning, Fast and Slow. Trends Cogn. Sci. 2019, 23, 408–422. [Google Scholar] [CrossRef]
  45. Wang, Y.; Zeng, Z.; Li, Q.; Deng, Y. A Complete Reinforcement-Learning-Based Framework for Urban-Safety Perception. ISPRS Int. J. Geo Inf. 2022, 11, 465. [Google Scholar] [CrossRef]
  46. Dey, S.; Marzullo, T.; Zhang, X.; Henze, G. Reinforcement learning building control approach harnessing imitation learning. Energy AI 2023, 14, 100255. [Google Scholar] [CrossRef]
  47. Wu, M.; Guo, K.; Li, X.; Lin, Z.; Wu, Y.; Tsiftsis, T.A.; Song, H. Deep Reinforcement Learning-based Energy Efficiency Optimization for RIS-aided Integrated Satellite-Aerial-Terrestrial Relay Networks. IEEE Trans. Commun. 2024. [Google Scholar] [CrossRef]
  48. Sarkar, S.; Naug, A.; Guillen, A.; Luna, R.; Gundecha, V.; Babu, A.R.; Mousavi, S. Sustainability of Data Center Digital Twins with Reinforcement Learning. Proc. AAAI Conf. Artif. Intell. 2024, 38, 23832–23834. [Google Scholar] [CrossRef]
  49. Sebestyen, A.; Tyc, J. Machine Learning Methods in Energy Simulations for Architects and Designers The implementation of supervised machine learning in the context of the computational design process. In Proceedings of the International Conference on Education and Research in Computer Aided Architectural Design in Europe, Berlin, Germany, 16–18 September 2020. [Google Scholar] [CrossRef]
  50. Sanaei, R.; Otto, K.; Hölttä-Otto, K.; Wood, K. A reinforcement learning approach to system modularization under constraints. Syst. Eng. 2023, 26, 408–424. [Google Scholar] [CrossRef]
  51. Gilan, S.S.; Dilkina, B. Sustainable building design: A challenge at the intersection of machine learning and design optimization. In Proceedings of the Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, USA, 25–30 January 2015. [Google Scholar]
  52. Tao, F.; Sui, F.; Liu, A.; Qi, Q.; Zhang, M.; Song, B.; Guo, Z.; Lu, S.C.-Y.; Nee, A.Y.C. Digital twin-driven product design framework. Int. J. Prod. Res. 2019, 57, 3935–3953. [Google Scholar] [CrossRef]
  53. Tracking Clean Energy Progress 2023—Analysis—IEA. Available online: (accessed on 30 March 2024).
  54. Dezfouli, M.M.S.; Dehghani-Sanij, A.R.; Kadir, K.; Sopian, K. Development and life cycle cost analysis of a solar hybrid HVAC system for use in buildings in tropical climates. Sustain. Energy Technol. Assess. 2023, 57, 103143. [Google Scholar] [CrossRef]
  55. Rahman, S.; Issa, S.; Assad, M.E.H.; Shah, S.K.; Abdelkareem, M.A.; Hoque, E.; Olabi, A. Performance enhancement and life cycle analysis of a novel solar HVAC system using underground water and energy recovery technique. Therm. Sci. Eng. Prog. 2022, 36, 101515. [Google Scholar] [CrossRef]
  56. Etemad, A.; Abdalisousan, A.; Aliehyaei, M. Design and Feasibility Analysis of a HVAC System based on CCHP, Solar Heating and Ice Thermal Storage for Residential Buildings. Modares Mech. Eng. 2022, 22, 81–92. Available online: (accessed on 25 February 2024).
  57. Mehrpooya, M.; Bahnamiri, F.K.; Moosavian, S.M.A. Energy analysis and economic evaluation of a new developed integrated process configuration to produce power, hydrogen, and heat. J. Clean. Prod. 2019, 239, 118042. [Google Scholar] [CrossRef]
  58. Ascione, F.; Bianco, N.; De Masi, R.F.; de Rossi, F.; De Stasio, C.; Vanoli, G.P. Energy audit of health care facilities: Dynamic simulation of energy performances and energy-oriented refurbishment of system and equipment for microclimatic control. Am. J. Eng. Appl. Sci. 2016, 9, 814–834. [Google Scholar] [CrossRef]
  59. Wan, Y.; Zhai, Y.; Wang, X.; Cui, C. Evaluation of Indoor Energy-Saving Optimization Design of Green Buildings Based on the Intelligent GANN-BIM Model. Math. Probl. Eng. 2022, 2022, 3130512. [Google Scholar] [CrossRef]
  60. Alhuyi-Nazari, M.; Mukhtar, A.; Yasir, A.S.H.M.; Ahmadi, M.H.; Kumar, R.; Luong, T.N.L. Applications of geothermal sources for absorption chillers as efficient and clean cooling technologies for buildings: A comprehensive review. J. Build. Eng. 2024, 82, 108340. [Google Scholar] [CrossRef]
  61. Nguyen, A.T.; Reiter, S.; Rigo, P. A review on simulation-based optimization methods applied to building performance analysis. Appl. Energy 2014, 113, 1043–1058. [Google Scholar] [CrossRef]
  62. Ramage, M.H.; McRobie, F.A.; Thomas, R. Architectural engineering: Collaborating to deliver energy-efficient buildings. In Oxford Conference: A Re-Evaluation of Education in Architecture; Roaf, S., Bairstow, A., Eds.; WIT Press: Southampton, UK, 2008; pp. 207–210. [Google Scholar]
  63. An, Y.; Chen, C. Energy-efficient control of indoor PM2. 5 and thermal comfort in a real room using deep reinforcement learning. Energy Build. 2023, 295, 113340. [Google Scholar] [CrossRef]
  64. Kabir, E.; Kumar, P.; Kumar, S.; Adelodun, A.A.; Kim, K.H. Solar energy: Potential and future prospects. Renew. Sustain. Energy Rev. 2018, 82, 894–900. [Google Scholar] [CrossRef]
  65. Dezfouli, M.M.S.; Dehghani-Sanij, A.; Kadir, K.A. Modelling and Energy Analysis of a Solar Cooling System Powered by a Photovoltaic (PV) System for a Net-Zero Energy Building (NZEB) Using TRNSYS-PVsyst. In Reducing the Effects of Climate Change Using Building-Integrated and Building-Applied Photovoltaics in the Power Supply; Springer: Cham, Switzerland, 2024; pp. 315–350. [Google Scholar] [CrossRef]
  66. Mahmoudan, A.; Samadof, P.; Sadeghzadeh, M.; Jalili, M.; Sharifpur, M.; Kumar, R. Thermodynamic and exergoeconomic analyses and performance assessment of a new configuration of a combined cooling and power generation system based on ORC–VCR. J. Therm. Anal. Calorim. 2021, 145, 1163–1189. [Google Scholar] [CrossRef]
  67. Hoseinzadeh, S.; Azadi, R. Simulation and optimization of a solar-assisted heating and cooling system for a house in Northern of Iran. J. Renew. Sustain. Energy 2017, 9, 045101. [Google Scholar] [CrossRef]
  68. Kariman, H.; Hoseinzadeh, S.; Heyns, P.S. Energetic and exergetic analysis of evaporation desalination system integrated with mechanical vapor recompression circulation. Case Stud. Therm. Eng. 2019, 16, 100548. [Google Scholar] [CrossRef]
  69. Said, Z.; Rahman, S.M.; Sohail, M.A.; Bibin, B.S. Analysis of thermophysical properties and performance of nanorefrigerants and nanolubricant-refrigerant mixtures in refrigeration systems. Case Stud. Therm. Eng. 2023, 49, 103274. [Google Scholar] [CrossRef]
  70. Chen, M.-Y.; Chiang, P.-R. COVID-19 Diagnosis System Based on Chest X-ray Images Using Optimized Convolutional Neural Network. ACM Trans. Sens. Netw. 2023, 19, 1–22. [Google Scholar] [CrossRef]
  71. Kumar, R.L.; Khan, F.; Din, S.; Band, S.S.; Mosavi, A.; Ibeke, E. Recurrent Neural Network and Reinforcement Learning Model for COVID-19 Prediction. Front. Public Health 2021, 9, 744100. [Google Scholar] [CrossRef] [PubMed]
  72. Newton, D. Generative Deep Learning in Architectural Design. Technol. Archit. + Des. 2019, 3, 176–189. [Google Scholar] [CrossRef]
  73. Jaenal, A.; Ruiz-Sarmiento, J.R.; Gonzalez-Jimenez, J. MachNet, a general Deep Learning architecture for Predictive Maintenance within the industry 4.0 paradigm. Eng. Appl. Artif. Intell. 2024, 127, 107365. [Google Scholar] [CrossRef]
  74. Wang, Y.; Zhao, J.W.; Zheng, M.Y.; Li, M.Y.; Sun, X.; Liu, H.; Liu, Z. Applications of a Deep Neural Network to Illustration Art Style Design of City Architectural. J. Inf. Process. Syst. 2024, 20, 53. [Google Scholar] [CrossRef]
  75. Dikshit, S.V.; Chavali, S.; Malwe, P.D.; Kulkarni, S.; Panchal, H.; Alrubaie, A.J.; Mohamed, M.A.; Jaber, M.M. A comprehensive review on dehumidification system using solid desiccant for thermal comfort in HVAC applications. Proc. Inst. Mech. Eng. Part E J. Process Mech. Eng. 2023. [Google Scholar] [CrossRef]
  76. Rahman, A.; Srikumar, V.; Smith, A.D. Predicting electricity consumption for commercial and residential buildings using deep recurrent neural networks. Appl. Energy 2018, 212, 372–385. [Google Scholar] [CrossRef]
  77. Dinmohammadi, F.; Han, Y.; Shafiee, M. Predicting energy consumption in residential buildings using advanced machine learning algorithms. Energies 2023, 16, 3748. [Google Scholar] [CrossRef]
  78. Nabavi, S.A.; Motlagh, N.H.; Zaidan, M.A.; Aslani, A.; Zakeri, B. Deep Learning in Energy Modeling: Application in Smart Buildings with Distributed Energy Generation. IEEE Access 2021, 9, 125439–125461. [Google Scholar] [CrossRef]
  79. Hafeez, G.; Alimgeer, K.S.; Khan, I. Electric load forecasting based on deep learning and optimized by heuristic algorithm in smart grid. Appl. Energy 2020, 269, 114915. [Google Scholar] [CrossRef]
  80. Wazirali, R.; Yaghoubi, E.; Abujazar, M.S.S.; Ahmad, R.; Vakili, A.H. State-of-the-art review on energy and load forecasting in microgrids using artificial neural networks, machine learning, and deep learning techniques. Electr. Power Syst. Res. 2023, 225, 109792. [Google Scholar] [CrossRef]
  81. Sendra-Arranz, R.; Gutiérrez, A. A long short-term memory artificial neural network to predict daily HVAC consumption in buildings. Energy Build. 2020, 216, 109952. [Google Scholar] [CrossRef]
  82. Rahbar, M.; Mahdavinejad, M.; Markazi, A.H.D.; Bemanian, M. Architectural layout design through deep learning and agent-based modeling: A hybrid approach. J. Build. Eng. 2022, 47, 103822. [Google Scholar] [CrossRef]
  83. Ferreiro-Cabello, J.; Fraile-Garcia, E.; de Pison Ascacibar, E.M.; de Pison Ascacibar, F.J.M. Metamodel-based design optimization of structural one-way slabs based on deep learning neural networks to reduce environmental impact. Eng. Struct. 2018, 155, 91–101. [Google Scholar] [CrossRef]
  84. Ferrara, M.; Della Santa, F.; Bilardo, M.; De Gregorio, A.; Mastropietro, A.; Fugacci, U.; Vaccarino, F.; Fabrizio, E. Design optimization of renewable energy systems for NZEBs based on deep residual learning. Renew. Energy 2021, 176, 590–605. [Google Scholar] [CrossRef]
  85. Kallioras, N.A.; Lagaros, N.D. DzAIℕ: Deep learning based generative design. Procedia Manuf. 2020, 44, 591–598. [Google Scholar] [CrossRef]
  86. Zhang, W.; Wang, Y.; Du, Z.; Liu, C.; Youn, S.K.; Guo, X. Machine-learning assisted topology optimization for architectural design with artistic flavor. Comput. Methods Appl. Mech. Eng. 2023, 413, 116041. [Google Scholar] [CrossRef]
  87. Zhang, H. 3D Model Generation on Architectural Plan and Section Training through Machine Learning. Technologies 2019, 7, 82. [Google Scholar] [CrossRef]
  88. Asperti, A.; Evangelista, D.; Piccolomini, E.L. A Survey on Variational Autoencoders from a Green AI Perspective. SN Comput. Sci. 2021, 2, 301. [Google Scholar] [CrossRef]
  89. Cai, C.; Li, B. Training deep convolution network with synthetic data for architectural morphological prototype classification. Front. Archit. Res. 2021, 10, 304–316. [Google Scholar] [CrossRef]
  90. As, I.; Pal, S.; Basu, P. Artificial intelligence in architecture: Generating conceptual design via deep learning. Int. J. Archit. Comput. 2018, 16, 306–327. [Google Scholar] [CrossRef]
  91. Yoo, S.; Lee, S.; Kim, S.; Hwang, K.H.; Park, J.H.; Kang, N. Integrating deep learning into CAD/CAE system: Generative design and evaluation of 3D conceptual wheel. Struct. Multidiscip. Optim. 2021, 64, 2725–2747. [Google Scholar] [CrossRef]
  92. Oh, S.; Jung, Y.; Kim, S.; Lee, I.; Kang, N. Deep generative design: Integration of topology optimization and generative models. J. Mech. Des. 2019, 141, 111405. [Google Scholar] [CrossRef]
  93. Adibhesami, M.A.; Karimi, H.; Sharifi, A.; Sepehri, B.; Bazazzadeh, H.; Berardi, U. Optimization of Urban-Scale Sustainable Energy Strategies to Improve Citizens’ Health. Energies 2023, 16, 119. [Google Scholar] [CrossRef]
  94. Wei, P.; Xia, S.; Chen, R.; Qian, J.; Li, C.; Jiang, X. A Deep-Reinforcement-Learning-Based Recommender System for Occupant-Driven Energy Optimization in Commercial Buildings. IEEE Internet Things J. 2020, 7, 6402–6413. [Google Scholar] [CrossRef]
  95. Gómez-Romero, J.; Molina-Solana, M. Towards self-adaptive building energy control in smart grids. In Proceedings of the NeurIPS 2019 Workshop on Tackling Climate Change with Machine Learning, Vancouver, BC, Canada, 13–14 December 2019. [Google Scholar]
  96. Zhang, Z.; Chong, A.; Pan, Y.; Zhang, C.; Lu, S.; Lam, K.P. A deep reinforcement learning approach to using whole building energy model for HVAC optimal control. In Proceedings of the ASHRAE and IBPSA-USA Building Simulation Conference, Chicago, IL, USA, 26–28 September 2018. [Google Scholar]
  97. Gao, Y.; Matsunami, Y.; Miyata, S.; Akashi, Y. Operational optimization for off-grid renewable building energy system using deep reinforcement learning. Appl. Energy 2022, 325, 119783. [Google Scholar] [CrossRef]
  98. Gao, Y.; Matsunami, Y.; Miyata, S.; Akashi, Y. Multi-agent reinforcement learning dealing with hybrid action spaces: A case study for off-grid oriented renewable building energy system. Appl. Energy 2022, 326, 120021. [Google Scholar] [CrossRef]
  99. Shen, R.; Zhong, S.; Wen, X.; An, Q.; Zheng, R.; Li, Y.; Zhao, J. Multi-agent deep reinforcement learning optimization framework for building energy system with renewable energy. Appl. Energy 2022, 312, 118724. [Google Scholar] [CrossRef]
  100. Mocanu, E.; Mocanu, D.C.; Nguyen, P.H.; Liotta, A.; Webber, M.E.; Gibescu, M.; Slootweg, J.G. On-Line Building Energy Optimization Using Deep Reinforcement Learning. IEEE Trans. Smart Grid 2019, 10, 3698–3708. [Google Scholar] [CrossRef]
  101. Zhang, Z.; Chong, A.; Pan, Y.; Zhang, C.; Lam, K.P. Whole building energy model for HVAC optimal control: A practical framework based on deep reinforcement learning. Energy Build. 2019, 199, 472–490. [Google Scholar] [CrossRef]
  102. Elbeltagi, E.; Wefki, H. Predicting energy consumption for residential buildings using ANN through parametric modeling. Energy Rep. 2021, 7, 2534–2545. [Google Scholar] [CrossRef]
  103. Chen, S.; Zhao, L.; Zheng, L.; Bi, G. A rapid evaluation method for design strategies of high-rise office buildings achieving nearly zero energy in Guangzhou. J. Build. Eng. 2021, 44, 103297. [Google Scholar] [CrossRef]
  104. Amiri, S.S.; Mottahedi, M.; Asadi, S. Using multiple regression analysis to develop energy consumption indicators for commercial buildings in the U.S. Energy Build. 2015, 109, 209–216. [Google Scholar] [CrossRef]
  105. Elbeltagi, E.; Wefki, H.; Abdrabou, S. Dawood, and A. Ramzy. Visualized strategy for predicting buildings energy consumption during early design stage using parametric analysis. J. Build. Eng. 2017, 13, 127–136. [Google Scholar] [CrossRef]
  106. Chegari, B.; Tabaa, M.; Simeu, E.; Moutaouakkil, F.; Medromi, H. Multi-objective optimization of building energy performance and indoor thermal comfort by combining artificial neural networks and metaheuristic algorithms. Energy Build. 2021, 239, 110839. [Google Scholar] [CrossRef]
  107. Gossard, D.; Lartigue, B.; Thellier, F. Multi-objective optimization of a building envelope for thermal performance using genetic algorithms and artificial neural network. Energy Build. 2013, 67, 253–260. [Google Scholar] [CrossRef]
  108. Yu, W.; Li, B.; Jia, H.; Zhang, M.; Wang, D. Application of multi-objective genetic algorithm to optimize energy efficiency and thermal comfort in building design. Energy Build. 2015, 88, 135–143. Available online: (accessed on 18 March 2023). [CrossRef]
  109. Taye, M.M. Understanding of Machine Learning with Deep Learning: Architectures, Workflow, Applications and Future Directions. Computers 2023, 12, 91. [Google Scholar] [CrossRef]
  110. Ahmad, S.; Shakeel, I.; Mehfuz, S.; Ahmad, J. Deep learning models for cloud, edge, fog, and IoT computing paradigms: Survey, recent advances, and future directions. Comput. Sci. Rev. 2023, 49, 100568. [Google Scholar] [CrossRef]
  111. Sapnken, F.E.; Hamed, M.M.; Soldo, B.; Tamba, J.G. Modeling energy-efficient building loads using machine-learning algorithms for the design phase. Energy Build. 2023, 283, 112807. [Google Scholar] [CrossRef]
  112. Shen, Y.; Pan, Y. BIM-supported automatic energy performance analysis for green building design using explainable machine learning and multi-objective optimization. Appl. Energy 2023, 333, 120575. [Google Scholar] [CrossRef]
  113. Hemmerling, M.; Cocchiarella, L. Informed Architecture: Computational Strategies in Architectural Design; Cologne University of Applied Sciences; Springer International Publishing: Cologne, Germany, 2017. [Google Scholar] [CrossRef]
  114. Senske, N. Confronting the challenges of computational design instruction. In Proceedings of the 19th International Conference on Computer-Aided Architectural Design Research in Asia—Rethinking Comprehensive Design: Speculative Counterculture, CAADRIA 2014, Kyoto, Japan, 14–16 May 2014; The Association for Computer-Aided Architectural Design Research in Asia (CAADRIA). University of North Carolina at Charlotte: Charlotte, NC, USA, 2014; pp. 821–830. [Google Scholar]
  115. Cao, X.; Dai, X.; Liu, J. Building energy-consumption status worldwide and the state-of-the-art technologies for zero-energy buildings during the past decade. Energy Build. 2016, 128, 198–213. [Google Scholar] [CrossRef]
  116. Pérez-Lombard, L.; Ortiz, J.; Pout, C. A review on buildings energy consumption information. Energy Build. 2008, 40, 394–398. [Google Scholar] [CrossRef]
  117. Gilani, H.A.; Hoseinzadeh, S.; Karimi, H.; Karimi, A.; Hassanzadeh, A.; Garcia, D.A. Performance analysis of integrated solar heat pump VRF system for the low energy building in Mediterranean island. Renew. Energy 2021, 174, 1006–1019. [Google Scholar] [CrossRef]
  118. McHugh, J.; Burns, P.J.; Hittle, D.C. The energy impact of daylighting. ASHRAE J. 1998, 40, 31–35. [Google Scholar]
  119. ASHRAE Terminology. Available online: (accessed on 28 July 2022).
  120. Nassif, N.; Kajl, S.; Sabourin, R. Optimization of HVAC control system strategy using two-objective genetic algorithm. HVAC&R Res. 2005, 11, 459–486. [Google Scholar] [CrossRef]
  121. Ding, X. A Brief Analysis of Energy Conservation Ways by Building Materials for Ecological Architecture. J. Power Energy Eng. 2020, 8, 13–22. [Google Scholar] [CrossRef]
  122. Alahmad, M.; Wheeler, P.; Schwer, A.; Tiller, D.; Wilkerson, A.; Eiden, J. Collaboration with industry to promote energy conservation and education. In Proceedings of the 2009 ASEE Annual Conference and Exposition, Austin, TX, USA, 14–17 June 2009; American Society for Engineering Education, University of Nebraska: Lincoln, NE, USA, 2009. [Google Scholar]
  123. Gao, G.; Li, J.; Wen, Y. DeepComfort: Energy-efficient thermal comfort control in buildings via reinforcement learning. IEEE Internet Things J. 2020, 7, 8472–8484. Available online: (accessed on 15 February 2023). [CrossRef]
  124. Gao, G.; Li, J.; Wen, Y. Energy-efficient thermal comfort control in smart buildings via deep reinforcement learning. arXiv 2019, arXiv:1901.04693. [Google Scholar]
  125. Luo, X.J.; Oyedele, L.O.; Ajayi, A.O.; Akinade, O.O. Comparative study of machine learning-based multi-objective prediction framework for multiple building energy loads. Sustain. Cities Soc. 2020, 61, 102283. Available online: (accessed on 15 February 2023). [CrossRef]
  126. Li, M.; Dai, L.; Hu, Y. Machine Learning for Harnessing Thermal Energy: From Materials Discovery to System Optimization. ACS Energy Lett. 2022, 7, 3204–3226. [Google Scholar] [CrossRef] [PubMed]
  127. Brna, P.; Burton, M. Modelling students collaborating while learning about energy. J. Comput. Assist. Learn. 1997, 13, 194–205. [Google Scholar] [CrossRef]
  128. Ardabili, S.; Abdolalizadeh, L.; Mako, C.; Torok, B.; Mosavi, A. Systematic review of deep learning and machine learning for building energy. arXiv 2022, arXiv:2202.12269. [Google Scholar] [CrossRef]
  129. Qin, Y.; Ke, J.; Wang, B.; Filaretov, G.F. Energy optimization for regional buildings based on distributed reinforcement learning. Sustain. Cities Soc. 2022, 78, 103625. [Google Scholar] [CrossRef]
  130. Olu-Ajayi, R.; Alaka, H.; Sulaimon, I.; Sunmola, F.; Ajayi, S. Building energy consumption prediction for residential buildings using deep learning and other machine learning techniques. J. Build. Eng. 2022, 45, 103406. [Google Scholar] [CrossRef]
  131. Ghoshchi, A.; Zahedi, R.; Pour, Z.M.; Ahmadi, A. Machine learning theory in building energy modeling and optimization: A bibliometric analysis. J. Mod. Green Energy 2022, 1, 4. Available online: (accessed on 15 February 2023).
Figure 1. The process of this study.
Figure 1. The process of this study.
Buildings 14 01342 g001
Figure 2. Location of Famagusta on the geographical map of Cyprus.
Figure 2. Location of Famagusta on the geographical map of Cyprus.
Buildings 14 01342 g002
Figure 3. VAE training convergence.
Figure 3. VAE training convergence.
Buildings 14 01342 g003
Figure 4. Training and validation loss over epochs.
Figure 4. Training and validation loss over epochs.
Buildings 14 01342 g004
Figure 5. VAE loss surface visualization.
Figure 5. VAE loss surface visualization.
Buildings 14 01342 g005
Figure 6. VEA reconstruction error matrix.
Figure 6. VEA reconstruction error matrix.
Buildings 14 01342 g006
Figure 7. Training and validation loss over epochs.
Figure 7. Training and validation loss over epochs.
Buildings 14 01342 g007
Figure 8. Increase in exploration rate over episodes.
Figure 8. Increase in exploration rate over episodes.
Buildings 14 01342 g008
Figure 9. Policy network reference over episodes.
Figure 9. Policy network reference over episodes.
Buildings 14 01342 g009
Figure 10. Converging and stability analysis.
Figure 10. Converging and stability analysis.
Buildings 14 01342 g010
Figure 11. Convergence rate comparison.
Figure 11. Convergence rate comparison.
Buildings 14 01342 g011
Figure 12. Solution quality and computational efficiency comparison.
Figure 12. Solution quality and computational efficiency comparison.
Buildings 14 01342 g012
Figure 13. Energy consumption across different scenarios.
Figure 13. Energy consumption across different scenarios.
Buildings 14 01342 g013
Figure 14. Analysis of monthly energy consumption trends (2022–2023).
Figure 14. Analysis of monthly energy consumption trends (2022–2023).
Buildings 14 01342 g014
Table 1. Reconstruction Accuracy.
Table 1. Reconstruction Accuracy.
MetricOur VAEBenchmark
Table 2. Quantitative assessment of generated designs.
Table 2. Quantitative assessment of generated designs.
Design IDSSIM with Reference DesignDiversity
Table 3. Adjustment of discount factor over episodes.
Table 3. Adjustment of discount factor over episodes.
EpisodeOld Discount FactorNew Discount Factor
Table 4. Reward function tuning over episodes.
Table 4. Reward function tuning over episodes.
EpisodeOld RewardNew Reward
Table 5. Performance comparison across optimization techniques.
Table 5. Performance comparison across optimization techniques.
Convergence Rate1.201.001.10
Solution Quality0.950.850.90
Computational Efficiency0.850.800.75
Table 6. Performance comparison across different optimization techniques.
Table 6. Performance comparison across different optimization techniques.
Convergence Rate1.201.001.10
Solution Quality0.950.850.90
Computational Efficiency0.800.700.60
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Karimi, H.; Adibhesami, M.A.; Hoseinzadeh, S.; Salehi, A.; Groppi, D.; Astiaso Garcia, D. Harnessing Deep Learning and Reinforcement Learning Synergy as a Form of Strategic Energy Optimization in Architectural Design: A Case Study in Famagusta, North Cyprus. Buildings 2024, 14, 1342.

AMA Style

Karimi H, Adibhesami MA, Hoseinzadeh S, Salehi A, Groppi D, Astiaso Garcia D. Harnessing Deep Learning and Reinforcement Learning Synergy as a Form of Strategic Energy Optimization in Architectural Design: A Case Study in Famagusta, North Cyprus. Buildings. 2024; 14(5):1342.

Chicago/Turabian Style

Karimi, Hirou, Mohammad Anvar Adibhesami, Siamak Hoseinzadeh, Ali Salehi, Daniele Groppi, and Davide Astiaso Garcia. 2024. "Harnessing Deep Learning and Reinforcement Learning Synergy as a Form of Strategic Energy Optimization in Architectural Design: A Case Study in Famagusta, North Cyprus" Buildings 14, no. 5: 1342.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop