Blockchain-Based E-Voting Systems: A Technology Review
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThis paper provides a comprehensive review of blockchain-based electronic voting systems, illuminating their benefits, challenges, and future research directions. Valuable insights are offered, with suggested revisions to address identified issues during the review process.
-"Furthermore, while privacy, verifiability, efficiency, trustworthiness, and auditability are given significant consideration, accessibility, compatibility, availability, and usability are comparatively less noted."
Here, the term "noted" might be clarified for precision. Does it mean that these aspects are less discussed or less emphasized in the studies? Similarly, the term "consideration" might be clarified to convey whether these aspects were considered but not as extensively discussed.
-In Figure 2, it would be useful to explicitly mention the number of papers included (e.g., Included Studies (N=252)).
-3.3. Research Gap and Objective
To enhance the clarity and completeness of this section, you may consider explicitly stating the research objectives or goals. This could involve outlining specific aims that the systematic analysis aims to achieve or the questions it seeks to answer. Other information could be reduced as they are too general.
-Methodology
Including information about the initial retrieved results and the process of filtering out irrelevant search results can be valuable in a systematic review. This additional information provides transparency and allows readers to understand the scope of the initial search and the steps taken to refine and focus the search results.
-In several tables, the expression "Normalised in %" is used. The author stated that "We list the number of publications and a normalised value in order to indicate the magnitude relative to other techniques."
Although a brief explanation is provided, how "Normalised in %" is obtained is still not clear and not apparent in the table. Detailed explanations with examples are needed. Optionally, add a column that provides the frequency of the item and 100% so that how the % is obtained is more apparent.
-There is inconsistent use of the word 'normalized'; both 'normalized' versus 'normalised' can be found.
-Based on the definitions of the different perspectives, we would expect that
• suggested benefits to have been demonstrated and showing positive impact,
• but also that challenges would be re-iterated as future work.
Revise this sentence to maintain consistency and parallelism between the two bullet points.
-"Therefore, this study can illustrate potential and limitations of blockchain-based e-voting systems by combining properties with concrete technologies in the analysis and discussion."
There needs to be a stronger sense of closure.
-I recommend authors explore visualization tools like VOSViewer to gain insights from the compiled literature. By leveraging author keywords, they can uncover valuable insights within the collected papers. While it's not obligatory for this specific paper, utilizing such tools would enhance both the authors' understanding and the readers' experience.
Comments on the Quality of English LanguageWhile the manuscript exhibits robust content, refining minor language intricacies would enhance the clarity and coherence of the writing, aligning it more closely with this journal's rigorous academic standards.
Author Response
We thank you for your insightful comments and suggestions. Your input has greatly improved the quality and clarity of our manuscript. Furthermore, your comments have been instrumental in guiding our revisions. Below, we provide a point-by-point response to your comments.
COMMENT:
-"Furthermore, while privacy, verifiability, efficiency, trustworthiness, and auditability are given significant consideration, accessibility, compatibility, availability, and usability are comparatively less noted."
Here, the term "noted" might be clarified for precision. Does it mean that these aspects are less discussed or less emphasized in the studies? Similarly, the term "consideration" might be clarified to convey whether these aspects were considered but not as extensively discussed.
ANSWER:
We revised according to your comment in the abstract, avoiding the unclear terms as follows:
“In the selected 252 scientific papers, aspects such as security, transparency, and decentralization are frequently emphasized as the main benefits. In contrast, although aspects like privacy, verifiability, efficiency, trustworthiness, and auditability receive significant attention, they are not the primary focus. We observed a relative lack of emphasis on aspects such as accessibility, compatibility, availability, and usability in the reviewed literature. These aspects, while acknowledged, are not as thoroughly discussed as the aforementioned key benefits in the proposed solutions for blockchain-based e-voting systems.”
COMMENT:
-In Figure 2, it would be useful to explicitly mention the number of papers included (e.g., Included Studies (N=252)).
ANSWER:
We changed Fig. 2 according to the comment.
COMMENT:
-3.3. Research Gap and Objective
To enhance the clarity and completeness of this section, you may consider explicitly stating the research objectives or goals. This could involve outlining specific aims that the systematic analysis aims to achieve or the questions it seeks to answer. Other information could be reduced as they are too general.
ANSWER:
To enhance the clarity and completeness of the Research Gap and Objective section, we revised it as follows and included an explicit list of objectives to be covered:
“Our systematic analysis of blockchain-based e-voting systems is guided by identified gaps in current literature and specific objectives we aim to achieve. Despite ongoing research in this domain, existing studies often focus on the limitations of blockchain-based e-voting, lacking a comprehensive comparison with traditional and electronic voting systems in terms of benefits and challenges.
The primary objectives of this systematic analysis are therefore:
1- To conduct a comprehensive comparison of blockchain-based e-voting systems against traditional and e-voting systems, focusing on understanding their relative benefits and challenges.
2- To review and analyze the concrete implementation techniques of blockchain in e-voting systems, identifying how they address existing challenges.
3- To provide the potential implications of blockchain-based e-voting systems for addressing existing challenges in the blockchain based e-voting systems.
4- To establish an up-to-date roadmap for future research, emphasizing areas that require further investigation in the rapidly evolving landscape of blockchain-based e-voting.
This study aims to fill these gaps by offering a comprehensive and holistic analysis of blockchain-based e-voting systems. This involves an in-depth exploration of current challenges and potential areas for future research, thereby contributing to a more thorough understanding of blockchain technology's role in enhancing the integrity and efficiency of voting processes.”
COMMENT:
-Methodology
Including information about the initial retrieved results and the process of filtering out irrelevant search results can be valuable in a systematic review. This additional information provides transparency and allows readers to understand the scope of the initial search and the steps taken to refine and focus the search results.
-In several tables, the expression "Normalised in %" is used. The author stated that "We list the number of publications and a normalised value in order to indicate the magnitude relative to other techniques."
Although a brief explanation is provided, how "Normalised in %" is obtained is still not clear and not apparent in the table. Detailed explanations with examples are needed. Optionally, add a column that provides the frequency of the item and 100% so that how the % is obtained is more apparent.
ANSWER:
We provided definitions for each normalized value and added this as a footnote to each effected table as follows:
Table 2, 3, 4: Normalized Percentage = (Number of Papers in a Category / Total Number of Papers) * 100
Table 5,6,7,9 : Normalized Percentage = [(Number of Papers in a Category that mentioned - Min Number of Papers that mentioned one of the Categories)/ ( Max Number of Papers that mentioned one of the Categories - Min Number of Papers that mentioned one of the Categories)]*100
COMMENT:
-There is inconsistent use of the word 'normalized'; both 'normalized' versus 'normalised' can be found.
ANSWER:
We adopted American spelling as revised to “normalized” throughout the paper.
COMMENT:
-Based on the definitions of the different perspectives, we would expect that
- suggested benefits to have been demonstrated and showing positive impact,
- but also that challenges would be re-iterated as future work.
Revise this sentence to maintain consistency and parallelism between the two bullet points.
ANSWER:
We revised it according to your comment as follows:
"Based on the definitions of the different perspectives, we would expect that the suggested benefits have been demonstrated and shown to positively impact the field and that the challenges have been re-iterated as areas for future work."
COMMENT:
-"Therefore, this study can illustrate potential and limitations of blockchain-based e-voting systems by combining properties with concrete technologies in the analysis and discussion."
There needs to be a stronger sense of closure.
ANSWER:
We revised this statement according to your comment as follows to enhance clarity:
"Therefore, this study effectively clarifies both the potential and the limitations of blockchain-based e-voting systems. It achieves this by jointly integrating an analysis of fundamental properties with practical technological implementations and exploring the future roadmap, concluding in a comprehensive discussion that offers a holistic view of the topic."
COMMENT:
-I recommend authors explore visualization tools like VOSViewer to gain insights from the compiled literature. By leveraging author keywords, they can uncover valuable insights within the collected papers. While it's not obligatory for this specific paper, utilizing such tools would enhance both the authors' understanding and the readers' experience.
ANSWER:
We inspected VOSViewer and agree that it is very useful for bibliometric analyses. As we did not explore bibliographic networks and collaborations in this analysis, we did not use it in this particular case.
COMMENT:
Comments on the Quality of English Language
While the manuscript exhibits robust content, refining minor language intricacies would enhance the clarity and coherence of the writing, aligning it more closely with this journal's rigorous academic standards.
ANSWER:
Grammar and spelling have been revised throughout.
Reviewer 2 Report
Comments and Suggestions for AuthorsThis article aims to explore the application, challenges, and key focus of blockchain technology in electronic voting systems through literature review and quantitative analysis. The aim is to propose well structured solutions and future research prospects for blockchain based electronic voting systems. The paper has certain research significance and practical value, but there are still several points that need improvement:
(1) It is suggested that the author add more space in the literature review section, analyzing the current definition and research trends of blockchain technology from a qualitative perspective, and comparing the similarities and differences between blockchain technology in other applications and its application in electronic voting systems. Here are some references that can be cited: https://doi.org/10.1016/j.eswa.2022.117094; https://doi.org/10.1016/j.ipm.2021.102842; https://doi.org/10.1109/TEM.2021.3066090.
(2) The author needs to further improve and demonstrate the selection of search terms. Smart contracts, consensus algorithms, and others are all talking about blockchain. If an electronic voting system is built based on smart contracts, the current search terms may be difficult to retrieve.
(3) The current standards for quantitative analysis of retrieved literature are convincing. Is using only abstract and other text to find the application and challenges of blockchain in electronic voting systems one-sided, and is there a method for full-text search? Secondly, it is recommended to use some artificial intelligence technology for automatic acquisition.
(4) The paper needs to increase the use of analysis results and provide practical guidance and inspiration.
Comments on the Quality of English LanguageMinor editing of English language required
Author Response
We appreciate the time and effort you have invested in evaluating our manuscript. We have addressed each comment in a detailed and systematic manner, ensuring that the manuscript now aligns more closely with your suggestions and comments.
COMMENT:
(1) It is suggested that the author add more space in the literature review section, analyzing the current definition and research trends of blockchain technology from a qualitative perspective and comparing the similarities and differences between blockchain technology in other applications and its application in electronic voting systems. Here are some references that can be cited: https://doi.org/10.1016/j.eswa.2022.117094;
https://doi.org/10.1016/j.ipm.2021.102842;
https://doi.org/10.1109/TEM.2021.3066090.
ANSWER:
https://doi.org/10.1016/j.eswa.2022.117094 targets a patent registration and trading system.
https://doi.org/10.1016/j.ipm.2021.102842 provides a recommendation model for user sentiments in reviews.
https://doi.org/10.1109/TEM.2021.3066090 also covers issues similar to the first patent and copyright issues.
While in particular the first and third are very useful references in the wider context of our group’s work on blockchain technology and will prove to be good references, we have refrained from including them here in order to remain clearly in the defined focus on e-voting systems. Section "3.2. Blockchain Applications Across Domains" covers your concerns in this comment.
COMMENT:
(2) The author needs to further improve and demonstrate the selection of search terms. Smart contracts, consensus algorithms, and others are all talking about blockchain. If an electronic voting system is built based on smart contracts, the current search terms may be difficult to retrieve.
ANSWER:
Our approach focused mainly on broader terms using precise keywords and concepts related to electronic voting, such as e-voting, i-voting, evoting, ivoting, electronic voting, internet voting, and election. Additionally, the search approach encompasses blockchain-related terms such as blockchain, distributed ledger, and DLT.
We recognize that including a more specific term, as mentioned, could provide some more results for the application of blockchain technology in e-voting systems. However, regarding the objective of our study, we prioritized the search terms in this field rather than specific terms. Adding one specific term could have biased towards a specific approach, thus requiring a comprehensive list of these terms, which could be seen as subjective without further analysis.
According to other studies, we believed that our methodology and search terms were proper and adequate for providing a comprehensive view to answer the research questions that were designed.
COMMENT:
(3) The current standards for quantitative analysis of retrieved literature are convincing. Is using only abstract and other text to find the application and challenges of blockchain in electronic voting systems one-sided, and is there a method for full-text search? Secondly, it is recommended to use some artificial intelligence technology for automatic acquisition.
ANSWER:
This study is based on a thorough approach to data gathering and analysis, following common protocols. Specifically, we use a comprehensive review process that involves a full-text analysis of articles. This is detailed in the section discussing the methodology for selecting relevant literature.
The inclusion criteria for the literature review explicitly state that papers with a full-text version available were included to allow for comprehensive analysis and extraction of data​​, as we explain in each part. For example, in the challenges we mentioned, “These items are selected from various sections, primarily the abstract, introduction, and related works, applying a hybrid technique combining syntactic and semantic selection techniques.”
This approach resolves the concern about relying solely on abstracts or limited text and demonstrates a reliable method for analyzing the applications and challenges of blockchain in electronic voting systems.
Regarding the suggestion about using AI for automated data collection, it could be useful and enhance the methodology. As discussed above, we used a traditional review procedure, relying on multiple researchers being involved in different roles of analysis and validation, thus making the process transparent and removing bias in the process.
We have revised and clarified our methodology definition (see also the comments by Reviewer 4).
COMMENT:
(4) The paper needs to increase the use of analysis results and provide practical guidance and inspiration.
ANSWER:
We decided in the definition of our objectives to go beyond the procedure of a systematic literature review and analyze in depth the technology implications.
In order to clarify the contribution and value of the research, we revise, in particular, the Conclusions section (see also comments by Reviewer 1). In addition, we added the sections as follows:
- 5.4. In-Depth Analysis of Results
- 6.6. Analysis of Results
- 7.3. Insights and Implications from the Observations
Reviewer 3 Report
Comments and Suggestions for AuthorsThe paper is a systematic review that introduces the existing research on blockchain-based e-voting systems. The authors investigate the advantages, challenges, and impacts of these systems, as well as the technologies and implementations they used. The authors also identify future research directions in this domain.
Main contributions and advantages:
1. This paper provides a comprehensive and systematic review of the existing literature on the blockchain-based e-voting systems.
2. This paper introduces the history of the voting system and clearly analyzes the needs of the voting system
3. The paper presents a thorough analysis of the advantages, challenges, and impacts of blockchain-based e-voting systems based on the analysis of 252 scientific paper.
4. The paper is well-structured and introduces future research directions in this domain at the end of the review.
Problems and suggestions:
1. It is recommended that the authors enlarge the text in Fig8 appropriately, please pay attention to the standardization of tabulation.
2. As for the challenges faced by the current blockchain-based e-voting mentioned by the author in the paper, can you further elaborate on the better work done in the field to solve these challenges.
3. In order to enhance the convincing of the conclusion that blockchain-based e-voting technology has good potential, can the authors provide several current blockchain voting cases?
4. It is recommended that the authors compare the performance and efficiency of the blockchain-based e-voting method with existing e-voting systems or traditional voting mechanisms through experimental data.
5. Based on the years of papers collected in the article, it is hoped that the authors can logically introduce the history of the development of blockchain-based e-voting technology between 2017 and 2023. Moreover, the authors may discuss the related work to give more insightful prospectives. e.g. 1) FedAWR : An Interactive Federated Active Learning Framework for Air Writing Recognition. IEEE Transactions on Mobile Computing, 2023, DOI: 10.1109/TMC.2023.3320147 2) "Edge Computing for Internet of Everything: A Survey." IEEE Internet of Things Journal 9, no.(23), pp.23472–23485, 2022.
Comments on the Quality of English LanguageThe authors are suggested to double-check the writing.
Author Response
We express our gratitude for your comprehensive evaluation of our manuscript. Your insights and feedback have been fundamental in guiding the improvement of our work. In response, we have meticulously and systematically addressed each of your comments as follows:
COMMENT:
- It is recommended that the authors enlarge the text in Fig8 appropriately, please pay attention to the standardization of tabulation.
ANSWER:
We changed Table 8 accordingly.
COMMENT:
- As for the challenges faced by the current blockchain-based e-voting mentioned by the author in the paper, can you further elaborate on the better work done in the field to solve these challenges.
ANSWER:
We added a discussion on advancements addressing challenges in blockchain-based e-voting systems in Section 5.2, in which we elaborate more on selected challenges.
COMMENT:
- In order to enhance the convincing of the conclusion that blockchain-based e-voting technology has good potential, can the authors provide several current blockchain voting cases?
ANSWER:
We added Section 3.4. “Implementations of Blockchain-Based E-Voting Systems” in which we presented six different real-world implementations of blockchain-based e-voting systems.
COMMENT:
- It is recommended that the authors compare the performance and efficiency of the blockchain-based e-voting method with existing e-voting systems or traditional voting mechanisms through experimental data.
ANSWER:
While an experimental analysis of performance and other properties of these systems is of high value, our research focus and scope in this study focus on providing a systematic review and analysis of the benefits, challenges, impact, used technologies, and future roadmap of blockchain-based e-voting systems, thus being a secondary study.
While comparative studies are undoubtedly valuable, they fall outside the scope of this specific research study. We believe that such an in-depth analysis of the blockchain-based approach is essential for advancing our understanding of its potential in the context of e-voting. Comparative studies could be considered as future research directions to expand upon the findings of this work.
COMMENT:
- Based on the years of papers collected in the article, it is hoped that the authors can logically introduce the history of the development of blockchain-based e-voting technology between 2017 and 2023. Moreover, the authors may discuss the related work to give more insightful prospectives. e.g. 1) FedAWR : An Interactive Federated Active Learning Framework for Air Writing Recognition. IEEE Transactions on Mobile Computing, 2023, DOI: 10.1109/TMC.2023.3320147 2) "Edge Computing for Internet of Everything: A Survey." IEEE Internet of Things Journal 9, no.(23), pp.23472–23485, 2022.
ANSWER:
Firstly, we added the explanations below in the conclusion section to make a concluding reference to the relevant years 2017 to 2023:
“The evolution of blockchain-based e-voting systems from 2017 to 2023 has been marked by significant advancements, as evidenced by research papers from this period. Significant studies emerged, proposing a novel approach to utilizing blockchain technology for recording votes for different voting scenarios. These systems aimed to address common limitations in existing voting systems and involved a critical evaluation of popular blockchain frameworks suitable for e-voting applications. During the years, the primary research emphasis shifted towards enhancing security and developing robust frameworks for blockchain-based e-voting systems. In recent years, the other aspects of e-voting systems, scalability and cost-efficiency, have received more attention. Moreover, the importance of privacy-preserving protocols grew significantly, prompting the development of coercion-resistant and privacy-preserving e-voting protocols.”
Secondly, the first suggested reference focuses on federated learning and is somewhat outside the scope of this investigation. We, however, included the second reference as a pointer to blockchain application domains.
Reviewer 4 Report
Comments and Suggestions for AuthorsIn this paper, authors use the structured literature review (SLR) methods to provide a comprehensive overview of the extant literature on e-voting. However, the organization of the manuscript is not adequate, as it does not follow the usual requirements of the SLR method and presentation of the SLR results.
Namely, authors first delve into a traditional literature review (Section 3), based on their subjective evaluation of the literature. This is opposed to the very logic of SLR, which should study the entire indexed body of literature and uncover the logic of the field and structure of the existing research, by looking at the bibliographic/bibliometric evidence.
PRISMA protocol for SLRs is adequately applied, but the presentation of SLR results is uncovential. Authors seem to apply the subjectively applied methodology of paper classification and evaluation to the body of literature, discovered by using the SLR method and PRISMA guidelines. This is not acceptable in a SLR study, which should let the (bibliographic) data speak for itself.
It is customary to provide one (or several) table(s), with the most important (central) references, and the related work, and summarize the type of study, its content and contribution to the field. Structure of the field and its research can be inferred after the presentation of the described literature.
There are two ways out of this methodological problem:
a) You can simply disregard the SLR methodology, as you have missed the whole point of the SLR method. Further develop your paper as a traditional review paper, using a part of the SLR method (identification of relevant papers by using the bibliometric queries of the indexed literature). Do not try to convey the idea that you are producing a SLR study. It is NOT enough to identify the literature by using the bibliometric queries to qualify your literature review as a SLR.
b) Disregard your pre-existing subjective structure of the field and re-write your paper completely by following closely the SLR methodology.
I believe it would be much easier for you to follow the approach described under (a) and develop a traditional review article.
There are some technical issues to be corrected, as well:
- MIT Election Lab references in lines 63 and 77 are incorrect in terms of MDPI reference style and are not adequately referenced in the literature list
- References Elections and Technology and Verified Voting in lines 88 and 95 are not aligned to the MDPI reference style
- Section 2.2. refers to references 13-18, with a large descriptive part of the text, without any references. Please provide references relevant to each subsection, i.e. to each of the e-voting system requirement
- Fig. 1 on p. 6 is completely out of context - it should be introduced by a previous decription
- Reference to Ta¸s and Tanrıöver needs to point to item 19 in the reference list, instead of a question mark?! (line 237)
Author Response
We are thankful for your valuable insights and constructive feedback. Your analysis and suggestions have been invaluable in refining our manuscript. We have meticulously addressed each point raised, ensuring that our responses not only align with your concerns but also enhance the overall quality and coherence of our work. The following is our point-by-point response to your helpful comments:
COMMENT:
In this paper, authors use the structured literature review (SLR) methods to provide a comprehensive overview of the extant literature on e-voting. However, the organization of the manuscript is not adequate, as it does not follow the usual requirements of the SLR method and presentation of the SLR results.
Namely, authors first delve into a traditional literature review (Section 3), based on their subjective evaluation of the literature. This is opposed to the very logic of SLR, which should study the entire indexed body of literature and uncover the logic of the field and structure of the existing research, by looking at the bibliographic/bibliometric evidence.
PRISMA protocol for SLRs is adequately applied, but the presentation of SLR results is uncovential. Authors seem to apply the subjectively applied methodology of paper classification and evaluation to the body of literature, discovered by using the SLR method and PRISMA guidelines. This is not acceptable in a SLR study, which should let the (bibliographic) data speak for itself.
It is customary to provide one (or several) table(s), with the most important (central) references, and the related work, and summarize the type of study, its content and contribution to the field. Structure of the field and its research can be inferred after the presentation of the described literature.
There are two ways out of this methodological problem:
- a) You can simply disregard the SLR methodology, as you have missed the whole point of the SLR method. Further develop your paper as a traditional review paper, using a part of the SLR method (identification of relevant papers by using the bibliometric queries of the indexed literature). Do not try to convey the idea that you are producing a SLR study. It is NOT enough to identify the literature by using the bibliometric queries to qualify your literature review as a SLR.
- b) Disregard your pre-existing subjective structure of the field and re-write your paper completely by following closely the SLR methodology.
I believe it would be much easier for you to follow the approach described under (a) and develop a traditional review article.
ANSWER:
We agree that the method used by us was not sufficiently well defined and justified.
We agree that it is not a SLR throughout the paper, following the respective rules strictly, but applies a hybrid strategy.
In order to respond, we followed the proposal (a) by the reviewer and removed references to SLR from our methodology definition as the main approach. As said, we apply SLR principles to select papers and extract some information on bibliographic data, but we also conduct an analysis of technological aspects through our perspectives, such as benefits or challenges. These activities of this hybrid approach have, however, been conducted in a systematic way, with different roles assigned to the different authors to avoid bias and misrepresentations.
We believe that a deeper technology analysis is necessary in order to provide sufficient practical insights to move the field forward. Thus, we used SLR procedures to identify papers, but then applied a technology review to the paper base.
We have now introduced our methodology accordingly and have retained the PRISMA framework as a reference approach.
We have reworded this, where needed, in the abstract, introduction, definition of research objectives, and methodology sections.
COMMENT:
There are some technical issues to be corrected as well:
- MIT Election Lab references in lines 63 and 77 are incorrect in terms of MDPI reference style and are not adequately referenced in the literature list
- References Elections and Technology and Verified Voting in lines 88 and 95 are not aligned to the MDPI reference style
ANSWER:
We revised the references style throughout.
COMMENT:
- Section 2.2 refers to references 13-18, with a large descriptive part of the text without any references. Please provide references relevant to each subsection, i.e. to each of the e-voting system requirement
ANSWER:
The references mentioned in that section, specifically references 13–20, were intended to provide a foundational context for the e-voting system requirements discussed. Most of these papers cover a broad range of requirements. However, we understand the concern about the descriptive part of the text without direct references. We would like to clarify that the categorization and definitions presented in that section are based on our own inferences and analysis of the literature. While we did not explicitly cite each definition, almost all of them would appear for each of the following categories. Our intention was to provide an overall understanding of e-voting system requirements based on a wide range of sources, including the references cited earlier in the paper.
We have reworded the beginning of Section 2.2 to clarify our proposal.
COMMENT:
- Fig. 1 on p. 6 is completely out of context - it should be introduced by a previous description.
ANSWER:
We have added a corresponding explanation of Fig. 1 in Section 3.1, which introduces blockchain technology.
COMMENT:
- Reference to Ta¸s and Tanrıöver needs to point to item 19 in the reference list, instead of a question mark?! (line 237)
ANSWER:
The incorrect reference in the text has been fixed.
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsAuthors have made significant improvements. However, I have noticed some issues in the paper that need correction.
-Most importantly, the tables display incorrect percentages. The paper provides normalization formulas to illustrate the percentages of their papers relative to other papers (e.g., the formula shown in Table 5). This differs from the normalization formula shown in Table 4. According to the formula in Table 5, percentages should range from 0 to 100 or 0% to 100% due to normalization. In this case, the paper with the maximum percentage would be 100%, and the one with the minimum would be zero. However, it is unclear how the author adjusts this scheme.
-The text descriptions do not match the percentages presented in the table (e.g., Table 4).
-It is unclear why two different normalization formulas are necessary.
Author Response
Thank you for your meticulous review and for highlighting the discrepancies in the percentages displayed in our tables. We appreciate your attention to this important detail. We are grateful for your insightful feedback, which has significantly contributed to enhancing the accuracy and clarity of our research findings.
COMMENT:
-Most importantly, the tables display incorrect percentages. The paper provides normalization formulas to illustrate the percentages of their papers relative to other papers (e.g., the formula shown in Table 5). This differs from the normalization formula shown in Table 4. According to the formula in Table 5, percentages should range from 0 to 100 or 0% to 100% due to normalization. In this case, the paper with the maximum percentage would be 100%, and the one with the minimum would be zero. However, it is unclear how the author adjusts this scheme.
ANSWER:
In response to your observation, we have thoroughly revised and double-checked the normalization process used for all tables to ensure that the percentages accurately reflect the correct values. To clearly indicate the changes, we have highlighted the revised portions in cyan-colored text in the updated version of the paper.
COMMENT:
-The text descriptions do not match the percentages presented in the table (e.g., Table 4).
ANSWER:
We have updated the text to align coherently with the data and normalization process presented in Table 4.
COMMENT:
-It is unclear why two different normalization formulas are necessary.
ANSWER:
Two different normalization formulas are used for data analysis, as follows:
1- Normalization Based on the Maximum Number of Papers in Any Category: This approach normalizes the data by considering the maximum number of papers in any given category. This method is effective when comparing the distribution or significance of different categories within a single context. It allows for a relative comparison, showing how each category stands against the most represented one.2- Normalization Based on the Total Number of Papers: This method normalizes the data against the total number of papers considered in the study. It provides a more holistic view of the distribution of papers across different categories relative to the entire dataset. Since only some papers mention relevant data in some sections, using these two methods allows for both a relative comparison (to show significance against the most represented data) and an absolute comparison (to understand the overall distribution in the context of the entire dataset). This dual approach provides a more meaningful analysis for the readers, especially useful in fields where some aspects are less frequently studied.
Reviewer 3 Report
Comments and Suggestions for AuthorsAll my concerns have been addressed.
Comments on the Quality of English LanguageMinor editing of English language required.
Author Response
We would like to express our sincere gratitude for your insightful comments and the time you invested in reviewing my paper. Throughout, we revised the grammar and spelling.
Reviewer 4 Report
Comments and Suggestions for AuthorsAuthors have followed reviewer's recommendations. Since the SLR methodology is not used as a foundation for this review, there are two minor issues to be corrected in the revised manuscript:
- Please correct the title: there should not be a word 'systematic' in the title, as the SLR is not used any more. I suggest the final title to be formulated as follows: A Review of Blockchain-Based E-Voting Systems Technology.
- On p. 2. remove the sentence "This guideline aims to improve the quality of systematic review protocols and facilitate their registration in the PROSPERO database". It is now out of context. Please explain that you have followed PRISMA methodology to assist you in the review process and indicate how it was used.
Author Response
Thank you once again for your constructive feedback. We are committed to improving the manuscript based on your suggestions. We highlighted the changes in cyan-colored text in the new revision.
COMMENT:
- Please correct the title: there should not be a word 'systematic' in the title, as the SLR is not used any more. I suggest the final title to be formulated as follows: A Review of Blockchain-Based E-Voting Systems Technology.
ANSWER:
We dropped the word ‘systematic’ from the title and now call it ‘Blockchain-Based E-Voting Systems: A Technology Review’.
COMMENT:
- On p. 2. remove the sentence "This guideline aims to improve the quality of systematic review protocols and facilitate their registration in the PROSPERO database". It is now out of context. Please explain that you have followed PRISMA methodology to assist you in the review process and indicate how it was used.
ANSWER:
We removed the sentence as suggested. We also removed the footnote on PRISMA and included the corresponding sentence in the last paragraph in the Introduction section. More details on how PRISMA is applied follow in Section 4 Methodology.
