Machine Learning Models for the Prediction of Energy Consumption Based on Cooling and Heating Loads in Internet-of-Things-Based Smart Buildings

Round 1

Reviewer 1 Report

Review of the manuscript:

Machine Learning Models for the Prediction of Energy Consumption Based on Cooling and Heating Loads in IoT-Based Smart Buildings

1. The findings are sufficiently novel to warrant publication.

2. The conclusions are adequately supported by the data presented.

3. The article is clearly and logically written so that it can be understood by one who is not an expert in the specific field.

4. The work provides an important contribution to its field, consistent with the scope of the journal.

The paper is describing the actual problematics

 In this paper, authors propose novel predictive models for Cooling and Heating Loads in IoT-based smart buildings by applying the various machine learning techniques to the data and considering features to have an efficient energy consumption. It is the first study that uses both the Tri-Layered Neural Networks and Maximum Relevance Minimum Redundancy algorithms together to predict energy consumption in IoT-based smart buildings.

Comments:

Row 3: It is not suitable to use the short cut in the title (Iot)

Row 21: Please write the units of the quantities Heating Load and Cooling Load of MAE

Row 184: Please explain what does MRMR and RReliefF algorithms mean?

Row 190: Please explain the k-Fold cross-validation technique.

Row 304: Please describe also physical and not only numerical model of the Cooling and Heating.

Row 317: Please describe physical character of the original data.

Row 366: Table 2. Please write the equal decimal numbers in the columns, e. g. two or three for each number in the column. Please explain what Relative Compactness, Glazing Area and Glazing Area Distribution is.

Row 376: Sentence, the Orientation of buildings was the least important features in terms of energy consumption prediction in smart buildings is very problematics, because the Orientation of building on Nord or South influenced on the energy consumptions of the buildings. The data of the dataset are not consistent.

Row 379: Table 3- 14: Please write the physical units of the quantities. Please write the statistics critical values of F – test of each Tables with F-test parameter.

Row 482, 485: Please write the units of quantities for RMSE, MSE, and MAE, respectively.

Row 487: Tables 15 – 18: Please write the units of the quantities for RMSE, MSE, and MAE, respectively.

Row 509: Fig. 3: Please describe the short cuts.

Row 563: Please describe the state-of-the-563 art methods.

Row 569: Please describe your calculation.

Row 572: Table 19: Please write the units of the physical quantities.

Row 589: Please write the units of the physical quantities.

 

 

 

 

 

 

 

 

Comments for author File: Comments.pdf

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The manuscript reports on a machine learning model aimed at predicting energy consumption in IoT-based smart buildings. In terms of topic, the manuscript is well-suited to this special issue. However, in terms of methodology and presentation, the manuscript suffers from certain drawbacks. The authors are suggested to consider the possibility to address the following remarks.

Remarks:

- Section 3.2:

1. Please elaborate in more detail on why “the mentioned problem in this research is a regression problem” (p. 6, l. 289).

2. The architecture of the actual neural network applied in the study was not described. It would be useful if the authors provided more detail.

- Section 3.3:

3. Did the authors really deploy IoT-devices in smart building? If so, please provide more detail (including but not limiting to details on the number of buildings, number and types devices, produced dataset, etc.). If not, Section 3.3. (including the pseudocode) is not really necessary.

- Section 4.1:

4. In Experiments 1 and 3, the authors divide the Energy Efficiency dataset into the training set and the test set in the ratio 70/30, which is in line with the dataset splitting practice. However, in Experiments 2 and 4, the authors state that they use “the whole dataset for training” (p. 7, l. 317-318 and l. 320-321). It remains unclear on which data were the considered machine learning models tested in Experiments 2 and 4. This in turn makes the interpretation of the reported results unfeasible and raises the following question(s). If the test set in Experiments 2 and 4 was part of the training set, it introduces a bias (i.e., referred to as “training on the test set”) that artificially improves the results (which then cannot be considered valid). If so, Experiments 2 and 4 should be excluded from the study. Otherwise, if the models were tested on other dataset which has “no intersection” with the Energy Efficiency dataset, then the results obtained in Experiments 2 and 4 cannot be compared with results of other studies presented in Section 5.2 and Table 19, which were evaluated on the  Energy Efficiency dataset. With this in mind, I recall that Table 19 contains the result obtained in Experiment 4 (MAR=0.535, cf. the last row in Table 19). The authors should clarify this situation.

5. Eq. (1) representing the mean absolute error is not correct (absolute value brackets are missing).

- Section 4.3:

6. It is not clear on which subset of the eight reported features was the considered machine learning models trained. If the models were trained on all eight features available in the dataset, what is the purposes of comparing the Maximum Relevance and Minimum Redundancy algorithm to the F Test and RreliefF algorithms? Otherwise, if the models were trained on a proper subset of features, the applied feature subset(s) should be stated. (These details would also clarify some of the missing details about the neural network applied in this study, cf. Remark 2.)

- Section 5.1:

7. Related to the critical difference diagram provided in Fig. 3 which “illustrates the mean ranks of each model under the different rates for the test sets in four experiments” (p. 13, l. 495-496):
7.1. Please explain “the different rates” and “mean ranks” in the context of the applied test set(s).
7.2. Depending on the authors’ reply to remark 4, this diagram should maybe reflect only the results obtained in Experiments 1 and 3.

- Section 5.2:

8. The conclusion that “the proposed method outperformed the other methods with 80.8% and 71.7% improvements on average” is not relevant or useful – and should be removed from the manuscript. Why would it be important to compare the results of the proposed results to the average results obtained by other methods. In fact:
a) the mean absolute error obtained for heating load prediction (MAR=0.289, Experiment 1) slightly outperforms the results obtained in [17] (AdaBoost, MAR=0.292) and [34] (Bagging ANN, MAR=0.291),
b) the mean absolute error obtained for cooling load prediction (MAR=0.585, obtained in Experiment 3, cf. remark 4) is slightly outperformed by the results obtained in [24] (MAR=0.536, SRTE) and [34] (Bagging ANN, MAR=0.556).
(Please note that the insights in this remark do not make the obtained results less valuable  - I just think that they are more relevant and accurate for the reader than the aforementioned conclusion.)

Additional remarks:

9. Is the system described in Fig. 2 really implemented?

10. The statement “IoT overcomes the challenges in data mobility” (p. 1, l. 37) should be either supported by a citation or removed.

11. The second paragraph of Introduction (p. 1-2, l. 41-52) are too general, not necessary and may be removed or significantly shortened.

12. The authors state: “predicting and optimizing energy consumption in IoT-based buildings [...] is an essential humanitarian need, economic, and social development factor, significantly to save the Earth, which we focus on in this study” (p. 2, l. 64-67). While I agree that predicting and optimizing energy consumption is an important research question, this statement is probably too strong and should be reformulated– e.g., the manuscript neither relates the proposed prediction model to the optimization task nor supports a conclusion that “predicting and optimizing energy consumption in IoT-based buildings” exhibits the announced positive effect.

13. The manuscript reads redundant at places.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

The paper is titled – “Machine Learning Models for the Prediction of Energy Consumption Based on Cooling and Heating Loads in IoT-Based Smart Buildings”. It aligns with the scope of the special issue. The work seems novel which is supported by the results and associated discussions. However, the presentation of the paper needs improvement. It is suggested that the authors make the necessary changes/updates to their paper as per the following comments:

1. Missing references: Several fact-based sentences, especially in the introduction section are missing supporting references. For instance, this sentence – “According to the U.S. Building Sector, energy consumption for the buildings (39%) in comparison to industry (33%) and transportation (28%) is at a high level that can be related to residential and commercial heating, ventilating, and air-conditioning (HVAC), lighting, hot water, and IT equipment” should have a supporting reference.

2. The beginning of the introduction section needs improvement as several emerging application domains of IoT are not even mentioned. A couple of such emerging application domains include – healthcare (suggested reference: https://doi.org/10.3390/bdcc5030042) and sports (suggested reference: https://doi.org/10.1007/978-981-16-7610-9_62).

3. Please elaborate on why 70% training data and 30% test data were used in Section 4.1. Why was any other train:test ratio such as 80:20 not used? Why was cross-validation not used?

4. The dataset used was published 10 years ago. Please elaborate on why a 10-year-old dataset was used and why similar datasets published in the last 3-5 years were not used.

 

5. Tables 15-18 present the results of multiple machine learning approaches. The presentation of these tables is good. However, an explanation should be added on why certain other machine learning approaches such as Naïve Bayes or Random Tree were not used. 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

The authors have adequately addressed my remarks and I believe that the revised manuscript has been sufficiently improved to warrant publication in Symmetry. For the purpose of completeness, I state a minor remark which I do not consider critical, but might be beneficial to the manuscript.

Minor remark:

1. Please consider the possibility to reformulate the sentence part “The mentioned problem in this research is a regression problem since …” (l. 239-242), e.g., as follows: “The mentioned problem in this research is considered as a regression problem since …”. The observed problem could have been addressed by other statistical methods, and therefore I think that this reformulation states the authors’ position more clearly.

Author Response

Dear Reviewer, 

Thank you very much for giving us the opportunity to revise our paper again.

We are also very grateful that Reviewer #1 and Reviewer #3 recommended accepting for publication.

Please accept our sincere thanks for your valuable minor comment. 

We revised our manuscript according to your comment as follows:
Rows 239 - 244

"The mentioned problem in this research is considered as a regression problem since the output attributes (heating load and cooling load) contain continuous data. In machine learning, regression is concerned with the prediction of a continuous target variable based on the set of input variables. Therefore, as one of the most common statistical methods, regression analysis was performed in this study to determine the relationship between independent and dependent variables." 

We are very grateful to you for your time and effort in handling our manuscript.

Best regards.

Bita GHASEMKHANI
Reyat YILMAZ
Derya BIRANT
Recep Alp KUT

Reviewer 3 Report

The authors have revised the paper as per all my comments and feedback. I do not have any additional comments at this point. I recommend the publication of the paper in its current form. 

Author Response

Dear Reviewer, 

Thank you so much for your recommendation to publish our revised paper. 

We are very grateful to you for your time and effort in handling our manuscript.

Best regards.

Bita GHASEMKHANI
Reyat YILMAZ
Derya BIRANT
Recep Alp KUT