Two-Stage Hybrid Network Clustering Using Multi-Agent Reinforcement Learning
Round 1
Reviewer 1 Report
Nice work, however I would kindly invite the authors to make improvements of the paper to increase its impact.
Abstract – what is obvious from the title makes no sense to be repeated in the abstract. Too much space allocated for problem positioning in the detriment of value added and findings. I expect more space allocated for findings. This is also encouraged as long as the first paragraph from Introduction is a kind of copy-paste of the abstract introduction.
Methodology – the paper treats an optimization problem. For such cases, evolutionary algorithms such as swarm etc. are also considered. I consider that the S-o-A is poorly treated and limited to one single reference approach. In order to adopt MARL, I would – as a reader – like to know if other optimization algorithms are not a suitable solution; or at least to have a benchmark with respect to the proposed approach with MARL. I give two examplea in this respect - Swarm intelligence-based algorithms within IoT-based systems: A review: in Journal of Parallel and Distributed Computing, 2018; A Survey of Using Swarm Intelligence Algorithms in IoT: in Sensors, 2020.
Results – there is no clue what technology was used to implement the proposed algorithm.
Discussions – there is nothing about limitations and future work. Findings still need more elaboration.
Author Response
Thank you for kind comments
Please see the attachment. We attach answers for reviewer's comments.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
In the manuscript, the authors have proposed a two-stage hybrid method, i.e., Delaunay triangulation and multi-agent reinforcement learning for broker assignment in pub/sub system. The idea is interesting and the logic is clear. There are some major issues for the authors to consider as follow:
- Delaunay triangulation and multi-agent reinforcement learning themselves are not new. It is unclear about the challenges when combing them. Especially in abstract, the authors should articulate the challenges and the corresponding solutions.
- Preprocessing and clustering are too general as keywords.
- The authors should summarize the main contributions explicitly.
- The authors should study the existing works of broker assignment as well.
- The quality (resolution) of the figures should be improved.
- Some important works are missing, e.g., "Pattern-RL: multi-robot cooperative pattern formation via deep reinforcement learning".
- The experiments currently are too simple. The authors should enrich this part.
Author Response
Thank you for kind comments
Please see the attachment. We attach answers for reviewer's comments.
Author Response File: Author Response.pdf
Reviewer 3 Report
Authors have presented a multi-stage for clustering algorithms using reinforcement learning. This is a lot of interest to the readers. However, I would recommend the authors to following methods as mentioned and observe the results or even suggest methods on how these can adopted for their study.
[1] Narayanan, B. N., Hardie, R. C., Kebede, T. M., & Sprague, M. J. (2019). Optimized feature selection-based clustering approach for computer-aided detection of lung nodules in different modalities. Pattern Analysis and Applications, 22(2), 559-571.
[2]Messay-Kebede, T., Narayanan, B. N., & Djaneye-Boundjou, O. (2018, July). Combination of traditional and deep learning based architectures to overcome class imbalance and its application to malware classification. In NAECON 2018-IEEE National Aerospace and Electronics Conference (pp. 73-77). IEEE.
Authors could implement optimization based clustering performance and compare the performance with their proposed approach.
Author Response
Thank you for kind comments
Please see the attachment. We attach answers for reviewer's comments.
Author Response File: Author Response.pdf
Reviewer 4 Report
In this work, the authors proposed a two-stage hybrid method for broker assignment, where the two-stage hybrid approach surpasses the single-agent reinforcement learning (SARL) strategy with the typical k-means clustering. However, first, the idea of the two-stage strategy is not new; second, the SARL approach can not adaptively select the key component k. Thirdly, the experiments are not clear, not only some important state-of-the-arts are not compared but also the results are not high. I hope you can carefully revise the work, from the structure, the novelty, and the experiments to improve this work.
Comments for author File: Comments.pdf
Author Response
Thank you for kind comments
Please see the attachment. We attach answers for reviewer's comments.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
I agree with the improvements.
Author Response
Thank you for your comments. The comments helped to improve our paper.
Reviewer 2 Report
The manuscript has been improved a lot after revision. There are still some minor issues as follows.
The main contributions should be summarized at the end of the introduction.
Between each section and subsection (for example between section 2 and section 2.1), there should be short sentences or paragraphs summarizing the section.
Algo. 1, Algo. 2, and Fig. 6 are displayed on two pages, which is not elegant.
Some important works are missing as references: https://ieeexplore.ieee.org/document/9300211
Author Response
Thank you for comments
Please see the attachment.
We answered all comments from the reviewer
Author Response File: Author Response.pdf
Reviewer 4 Report
To develop an effective management scheme for broker distributions and improve the overall performance of communication networks, the authors proposed a two-stage hybrid network clustering method to get a well-organized distribution of brokers. The first stage applies the Delaunay triangulation and deleting method to choose the candidate broker, where this selection process operates as a preprocessing of the MARL. The second stage uses the MARL to find the best combination of the broker nodes. Compared with k-means cluster algorithm with SARL and k-Firefly algorithm with SARL, the two-stage hybrid clustering requires a considerably shorter processing time. The algorithm and procedure are well descripted. The revised part discusses the contribution and the threats of validity. However, the method lacks novelty and the experiments are not sufficient.
- Content
- Adding more details about how to group nodes with broker with MARL in Sec.3.2
- The number of Sec.6 is wrong, it should be Sec. 5?
- Please re-arrange Sec.4 to clearly illustrate different algorithm3, i.e. Algorithm 1, Algorithm 2, Algorithm 3.
- Add more references about clustering algorithm such as:
[R1]Yang, Z.; Feng, L.; Chang, Z.; Lu, J.; Liu, R.; Kadoch, M.; Cheriet, M. Prioritized Uplink Resource Allocation in Smart Grid Backscatter Communication Networks via Deep Reinforcement Learning. Electronics 2020, 9, 622
[R2]Liu C A, Liu F., Liu C. Y., et al. Multi-Agent Reinforcement Learning Based on K-Means Clustering in Multi-Robot Cooperative Systems. Advanced Materials Research, 2011, 216:75-80.
[R3]Yunpeng Chang, Zhigang Tu, Wei Xie, Junsong Yuan. Clustering-driven Deep Autoencoder for Video Anomaly Detection. ECCV, 2020, pp.329-345.
- Experiments
- Lacking comparison experiments, such as the Delaunay triangulation preprocessing with SARL.
- Pease adding description about the experimental data in Sec.6.
- This paper only provides the processing time of different clustering approaches. Please conducting more comparisons of the clustering results’ effectiveness and performance.
Comments for author File: Comments.pdf
Author Response
Thank you for your comments.
Please see the attachment
We answered all comments from the reviewer
Author Response File: Author Response.pdf
