Towards Convergence in Federated Learning via Non-IID Analysis in a Distributed Solar Energy Grid

Round 1

Reviewer 1 Report

The paper presents a solid study for federated learning, with rigorous theoretical analysis and extensive empirical analysis. I do not have particular comments to the paper and felt that the paper can be accepted in the current form. One minor issue is that more recent works on federated learning should be covered, such as, FedFA: Federated Feature Augmentation.

Author Response

I really appreciate and wish to thank the reviewer for your valuable review. I have read the manuscript “FedFA: Federated Feature Augmentation” (ICLR, 2023; FedFA) and I agree that it dealt with a non-IID dataset and accommodated its diversity via optimal collaboration among the distributed network. However, FedFA focuses on training the classification problem, whereas this work differs in this regard, which focuses on conducting the regression-based prediction. As the performance metric based on the loss function differs, where it is not feasible to compare them to a quantitative extent. However, I quite agree that the FedFA manuscript will be helpful for the readers to interpret and digest the overall flow and aim of this research in a professional manner, therefore, I added this as a reference with more explanation in the related work section.

Again, I send my thanks to the reviewer for reading this manuscript and offering their valuable feedback.

Reviewer 2 Report

It is my pleasure to review the manuscript entitled “Towards Convergence in Federated Learning via non-IID Analysis in a Distributed Solar Energy Grid”. The author made good effort but there is some points to improve in the current version that should be resolved before the acceptance of the article. Here are some suggestions for the author to improve the quality of the manuscript.

-         - Line 12 ,energy datasets during 2017. Jan. ~ 2021. Aug , should be ,   ….energy datasets during 2017. January ~  2021 August.

-         - Please stated the novelty of this study at the end of the abstract.

-        -  The presentation of Introduction section and the related works section is not good, I do suggest to restructured both and start the introduction with a well described problem statement.

-        -  Delete that in the title of figure 5

-         - There is no discussion  section in the courant version.

-         - The conclusion can be improved and limitations should be added if any.

 Once these points are addressed, the manuscript will be considered for possible publication.

Good Luck.

Author Response

Thank you very much for your detailed comments. What you addressed is clear and I am more than welcome to make changes based on your suggestions. 
For the first matter, it has been dealt with, changing the dates with no abbreviations.
For the second matter, I added the parts that explained the novelty of this work: “Furthermore, we explore the periodical training effect and update the central global model in the real-world environment, presenting the optimal update period that converges, returning the lowest loss value among the suggested candidates of the update scheme. To the best of my knowledge, this is the initial study that offers the effective update period selection of the regression-based task in FL, also investigating the time-series non-IID property analysis with practically viable
Cases.” the paragraph right above the categorized contributions, is what our work presents and conveys to the community.

For the third comment, I did find some of the parts that are inconsistent, and parts vague about the high-level ideas that I wish to convey. For this part, I carefully went through the contents and revised the parts that seem to be substandard.

The fourth comment has been edited to “Heatmap of the PCC”, from “Heatmap that indicates the PCC”.
For the fifth comment, in general, the ‘discussion section’ in the typical paper elaborates on the background and addresses the current problem that author(s) is/are tackling, which guides the readers to the technical perspective from the high-level intuition. In that sense, the parts that do the work of the ‘discussion section’ is most similar to section 3. Section 3 starts off with the subsection explaining federated learning and its vanilla algorithm. Then it moves the agenda to the non-IID dataset and its negative effects of training them as well as its high possibility of having non-IID attributes in real life, which is identical to the problem definition (i.e., also briefly explained in Sections 1 and 2). Then, the paper analyzes the properties in the depth of that brought-up agenda. However, I carefully edited the parts for the readers to understand the direction that I wish to convey, such as changing the title of section 3 along with the other contents.
For the final comment, the conclusion section has been revised as well. As for the limitation, I have carefully pondered what to add. Since this is the application-oriented research literature, it will validate its value when it has been applied to the industry. In that sense, to fully evaluate the performance of this designed pipeline, we would need to see its actual performance in the field after a few years. As this research offers feasibility based on theoretical/experimental validations of its potential effect, I believe it already has value in the field of academia.

Thank you again for your review, and I wish you all the best.

Reviewer 3 Report

This paper presents the convergence analysis of federated learning with regard to the quantitative degree of IID and non-IID attributes. The authors proposed a practical approach to enhance the convergence rate of the global model which was evaluated through a multi-year real world solar energy dataset collected from three regions of South Korea. Experimental results show that the proposed scheme is valid and feasible. Overall, the paper is well-organized.

Figure 2 and Figure 7: increasing the size of the four sub-plots for better visualization. Similarly, the size of Figure 5 can be increased.

Minor writing issues exist, for example:

(1) line 62, "The FL that objective function is classification has been widely studied ...", change to "The FL with an objective function being classification has been widely studied ..."
(2) line 65, "applications, for it assumes ...", change to "applications since it assumes ..." or break the sentence "applications. It assumes ..."
(3) line 217, adjust the space between "case" and "B"
...

Proofreading is needed before resubmission.

Author Response

I would like to thank you for your valuable review.  

1. For the first comment, the commented figures have been enlarged as requested.
2. The suggested parts (1)~(3) have been revised. I really appreciate the given comments.
3. Multiple proofreading has been done by myself and from other Computer Science Ph.Ds.

Again, I wish to thank you for your detailed suggestions and will endeavor to improve the quality of this research.

Reviewer 4 Report

This paper proposes to use federated learning for the convergence analysis for smart grid. Generally, the paper is well written and the contribution is valid. I have the following comments. 1. Why the title emphasizes solar energy grid while the paper focus on smart grid? Furthermore, will solar energy has the characteristics of non-IID? 2. Please clarify why this is the first work. The contributions for FL lie in the algorithm or the application in smart grid. 3. Is the data open access for validation? 4. Some equations should be given enough explanations, such as (1)-(5),(8)- (11).

Author Response

Thank you for your review and interest in this research. 

 

1. Thank you for the comment. I myself have worked closely with the Smart Grid-related workers, and to the best of my knowledge, the overall purpose of the Smart Grid system is to efficiently operate the independent electric-grid system. Smart Grid has a variety of energy sources that produce the electricity, and as it is an independent and distributed system, renewable energy has been gaining attention for it is a natural resource that only requires a static facility to generate electricity, whereas other power plants require raw materials or any sort of input fuel that requires to build a supply chain, which dilutes the overall purpose of the independent system. In a high-level sense, renewable energy is a suitable scheme that offers the self-sustainable operation of those distributed systems such as Smart Grid and Micro Grid. Moreover, the portion of solar energy takes most of the renewable energy, due to its high accessibility that involves private business domains, and low-cost solar panel facility compared to other renewable energy sources. However, renewable energy is considered an unstable resource, where predicting the exact amount of energy generated is not guaranteed, which confuses the basic supply-and-demand principle. Also, as electricity is a fundamental resource that must be stably supplied, predicting the amount is crucial. In this regard, predicting energy generation via deep learning approaches has gained much attention in the field and Federated Learning is a suitable algorithm that accommodates the successful operation of the deep learning-based prediction model, via effectively aggregating the distributed internal properties. In section 4-1, the statistics show that the distribution is not stable, and the degree of non-IID is stated in the section, as well as the statistics and visualizations. 

 

2. Thank you for your review. Diverse research concerning Federated Learning and its implementation in the energy domain has been done. To the best of my knowledge, this is indeed the first work that analyzed the practically viable time-series non-IID cases and offered the effective update period selection of the regression-based task in FL. Most of the previous works have focused on implementing the classification-based problem, and regression models have rarely been studied. Furthermore, this work validates our claim, by experimenting with the genuine dataset to see if it converges with the smallest loss. As other reviewers have noted, I carefully reviewed and edited the manuscript so that the readers can fully comprehend the direction of this work.

 

3. Yes, the dataset is fully accessible to the public. The dataset is collected from the open portal of the South Korean government. The reference [34] has the URL where you can access the dataset.

 

4. I have carefully reviewed the equations, especially the ones that the reviewer has mentioned, and have been attentively edited. Additional explanations were added as well at the front or the back of the equations. 

Again, I would like to thank the reviewer for your valuable comments.

Round 2

Reviewer 2 Report

Dear Editor,

I am satisfied with the author's responses to my comments raised in my previous review.Thus , I recommend the paper to be published in your journal.

 

Reviewer 4 Report

The author has well addressed my previous comments.