Raising Awareness of Smartphone Overuse among University Students: A Persuasive Systems Approach
Round 1
Reviewer 1 Report
The aim of the paper is stated to be an investigation of whether deployment of a conversational agent on university students’ cell phones can reduce their usage time by tracking and collecting usage data, and then sending messages to the user. The content is clearly significant for students undergoing stressful conditions as with the current global pandemic.
The paper begins with a clear introduction, setting the scene for the topic and study, and moves on to related work in which more detail is provided on prior work and existing apps, and what is lacking. The related work section however is rather short, and I feel more literature could be cited to show more research in this area. For example, What is the benefit of using a chatbot? Why not just have textual messages without the conversational aspect attached? Maybe the use of conversation for the purpose of motivating behaviour change is grounded in prior literature, but none of this is mentioned.
The research methods section is good, but is also missing information on for example, exactly what from the usage data is used to trigger the messages. Is it only time? And if so, how was this implemented and why? This question is seemingly answered in the Conclusion (lines 313-316) but needs to be mentioned earlier in the paper and in more detail. Similarly, there are example messages shown that the agent can send, but how were these designed – are they based on prior work in psychology? Did the authors come up with them themselves? Additionally, how was the conversation flow with the chatbot designed once users replied to the chatbot?
The procedure section is generally well written and easy to follow, however, the experimental design has clear limitations which are not discussed in the paper:
- in addition to the chatbot, there is also a visual representation of the usage time (this is a confound if the focus is on how the chatbot can influence behaviour change)
- users are aware their activity is being tracked and I assume also the goal of the study (i.e., screen time reduction) although this is not mentioned in the paper, and neither are its implications
All in all, the paper is well-written, but is lacking a lot of information. In addition, if the paper presented the study from the perspective of an app that gives users two ways of tracking their screen time, i.e., the widget and the chatbot, (instead of just a chatbot) the chosen study design would be more appropriate. The authors touch on this briefly in the Conclusion section of the paper (i.e., lines 290-294) and I believe a re-write to present their work in this way is needed before moving forward.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The authors present an Android App designed to raising awareness of smartphones overuse. The issue is very significant nowadays.
The quality of the presentation is high, and the paper is clear. Anyway, the paper must be improved in the following ways:
Nudge perspectives: the authors implemented a strategy to raise awareness towards smartphone overuse. Such strategies are framed within the concept of 'nudge' which has not been deeply discussed in the paper. Which kind of nudge they want to achieve? There are plenty of initiative in this regard, I suggest the following paper to better frame their strategy.
- https://doi.org/10.1080/1369118X.2016.1186713
- https://doi.org/10.1145/3054926
Related works : the paper needs a dedicated section to related works. The solution presented is not compared with other works in the literature, nor results "compared" with others obtained elsewhere, nor the authors places their work within the framework of studies that aimed to raise awareness to users concerning devices and technology and other related issues. Here below a brief and not exhaustive list of works that the authors should account/survey/overview:
- https://doi.org/10.1145/2517840.2517868
- https://doi.org/10.1016/j.comnet.2021.108614
- https://doi.org/10.1007/978-94-007-6314-2_4
- https://doi.org/10.3390/fi11070163
- https://doi.org/10.1145/3411764.3445516
- https://doi.org/10.1145/3357384.3357837
- https://doi.org/10.1038/s41598-021-03869-6
- https://doi.org/10.1109/CEAP.2019.8883454
- https://doi.org/10.1016/j.jobb.2019.09.004
- https://doi.org/10.1007/s00521-021-06343-6
Privacy and security perspectives: which kind of permissions are needed by the developed app? Is the app considered as a malware by VirusTotal? Which data are sent out of the users' device? Have authors surveyed privacy attitudes of students surveyed? It seems not but at the same time the authors declare that the app tracks the users.
Software and architecture are missing : the authors should provide a section with all the details about the implementation of the app. How the app track the users? Which APIs are used? Is there any server side of the app? Which computation is performed on the device and which one outside? What is the battery impact of such app on the devices?
A visual abstract is recommended to clarify the whole work to readers.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
The added content provides a much more detailed account of the study and the background literature on this topic. For the most part, the authors have addressed my initial concerns.
However, there are quite a few simple spelling and grammatical errors throughout the newly written content, for example, line 292 reads: “some conversational messages are presented in the revised section 3.1”. A quick pass through by the authors will fix these errors.
Additionally, although the content is there, Section 3.2 is difficult to read with sentences as in lines 149-150 which appear to be redundant. I would suggest a minor rewrite to better synthesize the newly referenced material and succinctly present the related work.
Lastly, Table 2 spans two pages and extends beyond the margins. I believe cleaning up the look of the table to fit better in the flow of the paper will help with readability.
Author Response
Point 1: "However, there are quite a few simple spelling and grammatical errors throughout the newly written content, for example, line 292 reads: “some conversational messages are presented in the revised section 3.1”. A quick pass through by the authors will fix these errors."
Response 1: Done, many thanks! (we used Grammarly and help from a native-speaker).
Point 2: "Additionally, although the content is there, Section 3.2 is difficult to read with sentences as in lines 149-150 which appear to be redundant. I would suggest a minor rewrite to better synthesize the newly referenced material and succinctly present the related work."
Response 2: Thank you, we made it more succinct and revised.
Point 3: "Lastly, Table 2 spans two pages and extends beyond the margins. I believe cleaning up the look of the table to fit better in the flow of the paper will help with readability."
Response 3: Done.
We cannot help to thank once again for the speedy and effortful review, which greatly helped us improve the work.
Reviewer 2 Report
The paper has greatly improved since the first submission. I thank the authors for their effort.
Anyway I suggest the authors the following upgrades:
Lines 263-264: "[...] In this application, only Google Sign-In was used, since Android smartphones have Google accounts associated with it"
The sentence does not end with a dot.
Line 272: typo at "[...] 7rganizing the app releases by version.[...]"
In the light of the above I strongly suggest a proofreading and grammar/spell check.
Lines 299-323: would be much nicer if the conversation is put in a more graphical/visual fashion
Table 2 has some rendering issue on the first column.
Battery impact: how do the authors checked the battery draining of the app? How do the authors state that is not significant? I guess that if the app drains too much battery then the users couldn't be interested in it even if it has beneficial outcomes.
Author Response
Point 1: "In the light of the above I strongly suggest a proofreading and grammar/spell check."
Response 1: Thank you helping to spot the typos, we have run a complete proofreading and grammar/spell check, mostly using Grammarly and also help from a native-speaking colleague.
Point 2: "Lines 299-323: would be much nicer if the conversation is put in a more graphical/visual fashion"
Response 2: We introduced an example of a complete conversation.
Point 3: Table 2 has some rendering issue on the first column.
Response 3: Indeed the PDF was generated from MS Word's track changes turned on. The PDF we now include comes from Word in normal mode, so the issue is now solved. Thank you once again!
Point 4: Battery impact: how do the authors checked the battery draining of the app? How do the authors state that is not significant? I guess that if the app drains too much battery then the users couldn't be interested in it even if it has beneficial outcomes.
Response 4: We use an Android tool called Battery Doctor to check it had a residual impact on battery life, but also by "not significant", we mean that no user complained about it.
On a final note, we wish to express our gratitude not just because of the review quality and effort, but also the quick feedback, which greatly helped us improve the manuscript.
