Sharing the Sidewalk: Observing Delivery Robot Interactions with Pedestrians during a Pilot in Pittsburgh, PA
Round 1
Reviewer 1 Report
This manuscript presents the results of a field deployment of a small-size mobile delivery robot in a city, focusing mainly on pedestrian and observer anecdotes. While the manuscript is easy to read, its presentation could be improved, as some parts are repetitive. Due to the characteristics of the study, a thorough literature review is necessary. However, this review was not well-performed, and the authors must improve it to differentiate their study from existing research on taller robots' interactions with pedestrians in public spaces, as there is already significant work in this area.
The authors' discussion of the distance between observers and the robot and its lack of publicity issue in the pilot study suggests poor study planning. I do not find this discussion valuable in section 6.1.1.
Section 6.1.2 also reveals weaknesses in the pilot study plan. There are many studies on efficient communication methods between robots and people, and while they may not be directly related to pedestrians, the authors should have considered them and applied their own hypotheses accordingly.
Although the authors cited study [27] in their discussion, it does not appear to have been considered when designing the pilot study. I found no additional insightful information in this study regarding interactions with children in pedestrian environments.
Some of the observation notes read more like a storybook than a description of the situation.
Minor comments:
Regarding cyclists, regulations may differ by country, with cycling on pedestrian pathways being illegal in some places. The authors should be more cautious when discussing the effects of robots on pedestrian pathways.
Author Response
We thank the reviewer for their comments.
1) Regarding the deployment, it seems that it was not clear enough that we did not deploy our own robot, but that this was a commercial robot deployment managed by the city. We were invited to observe the deployment and provide feedback on what we saw. We have clarified this point in section 3. The Pilot.
2) Since we did not control the deployment, we believe that the discussion of the challenges and limitations of the deployment and communication with residents is still important, especially for others who may wish to deploy robots publically.
3) We have added a section on human-robot communication in public settings. Our work is differentiated from prior work in that we observe a commercial deployment in a real-word setting. Our findings corroborate findings from priors works conducted in more controlled or online survey-type settings.
4) The observational notes are taken from memos which are a core research method in ethnographic studies. We include these to provide a rich and descriptive picture of what happened during the observations.
Reviewer 2 Report
Dear Editor and Dear Authors,
The topic of this paper is very interesting and thank you for your trust to send me the manuscript to review it.
The manuscript explains an observational study of a pilot project and aim to answer the question “How do people interact with the robots over the course of this public pilot?” Based on their observations, authors discussed recommendations for future pilots and questions to guide future design for robots in public spaces.
The comments and suggestions are below:
1. First of all, this manuscript is more a survey, than an article type of paper (or a kind of a report).
I don’t know does it fits to the scope of the journal?
I think this issue should be checked.
2. If the paper is intended to be a scientific research article, it should contain recently published valuable references from esteemed high quality journals about the topic. Further, the article should contain technical details related to the method and the robot design as well, supported with a brief mathematical background.
3. The use of personal pronounses (we, our, etc.) is unacceptable for scientific English. Please use Passive Voice instead.
The paper is interesting, however its topic should be checked and the correction to Passive Voice is needed.
Recommendation: need more information about the manuscript.
Author Response
1) Our paper is an observational study and uses ethnographic methods. We report out using memos from filed notes to provide readers with a rich understanding of what occurred. The Editor has informed us that the paper type is within the scope of the special issue.
2) We have included references to prior work within the human-robot interaction field.
3) We did not control the robot deployment as it was a commercial deployment, Thus, we cannot provide more details on the robot design than what we observed and is publically available.
4) Using personal pronouns is common in our field and suitable for a research publication in Human-Computer Interaction.
Reviewer 3 Report
Review on the Manuscript:
Paper ID: mti-2289046
Title: Sharing the Sidewalk: Observing Sidewalk Delivery Robot Interactions with Pedestrians During a Pilot in Pittsburgh, PA
I read the manuscript carefully. The purpose of the manuscript is to conduct an observational study, which will answer the question: "How do people interact with robots during this audience pilot?". After analyzing the manuscript, I recommend: 1. The questions for pedestrians must be well thought out and cover as many pedestrians as possible.
2. The analysis of the answers should be treated with statistical methods, so as to eliminate as many errors as possible.
Author Response
1) We have made clear our research questions about how people interact. We base out study on the real-world observations rather than on a sampled population.
2) As we have few respondents and not comparison group we do not believe there are statistical analyses that would be appropriate beyond counts.
Reviewer 4 Report
The article presents the results of a long-term in-the-wild study with delivery robots. As the authors argue, state-of-the-art research on public perception and interactions with delivery robots is mainly limited to studies run on universities grounds which may not be representative of the complexity of urban environments. The scope of understanding the actual implications of these robots on real sidewalks to inform policy-making is relevant and timely.
The approach is also appropriate overall.
The main limitations I see are somewhat lack of details regarding the observational study and the lack of an explicit discussion of results in relation to existing work. Regarding the observational study, it would be important to know how many observers there were, how were these instructed, what instances were deemed of particular importance for observation, and how they were instructed to 'be around the robot' without interfering with the situation.
Regarding the results, discussing these explicitly in relation to previous work would help clarify the actual contributions of this study, which right now remain somewhat implicit. For this, I would also suggest the authors to provide a summary of key take-aways at the end of the related works section, and to take these back as starting point for the discussion.
Minor suggestions for edits:
- the section starting at line 217 is overall repeating things that were described before;
- some references are not turned into numbers (e.g., lines 262 and 293)
- the aspect of care, people helping the robot, is really interesting and nicely connect with more conceptual works. This could be further reflected upon as a learning with important implications (see aspects of relationality and co-performance in Lupetti, M. L., Bendor, R., & Giaccardi, E. (2019). Robot citizenship: A design perspective. Design and Semantics of Form and Movement, 87.)
Author Response
We thank the reviewer for their comments. In our revision:
1) We have updated the method and data collection description to include more details about who and how the observations and interviews were conducted. We included details on how observers were trained and what they attended to while in the field. We have included the field observation guide as an Appendix item.
2) We have included a short summary at the end of the related work section highlighting key points that motivate our research and observational study.
3) We have revised and expanded our Discussion to weave in points from the related work. Specifically, we bring in how our work relates to prior work on robot communication and similarities with other observations of sidewalk robots. We have included Lupetti et al. as suggested by R4 and discussed this work in regards to how people might have (mis)treated the robots.
4) We have removed the repetition around lines 217
Round 2
Reviewer 3 Report
In my opinion, taking into account the author's answers and corrections, I recommend the acceptance of the manuscript for publication. However, the final decision is up to the editor-in-chief.
23.04.2023
