User Friendliness of a Wearable Visual Behavior Monitor for Cataract and Refractive Surgery

Round 1

Reviewer 1 Report

The aim of this study was to evaluate the user friendliness of a new tool.

That could be enough for a scientific article. The study is ambitious in the number of patients that were included.

The tool is the Vivior VBM, which measures the distance at which patient activities are performed, lighting conditions, the level of illumination during patient activities, and head angles to the three axes. It is explained in the background that this information, that show the patient’s real-world visual needs, is needed before a cataract or refractive surgery. However, the actual references and explanations for how this information will improve the new lens (there are not that many alternations to be considered, and most patients today still needs corrections afterwards) is missing.

Also in the usability of a measuring tool is also the question and quantification of its validity. A reference to a validation study or possibly, for each of the measures the validity of other tools using the same principles and hardware. This is also missing.

The method part shows example of the measurements that are included in the VBM. These are impressive! But again, their validity is not mentioned. A detail that I do not understand, is if the patients used glasses during night. As it looks from figure 5:

Measurements of head flexion and lateral bending over time. Further about that figure: Why is lateral bending good to know prior to a surgery? And I think that flexion and bending should be replaced with head position, since there does not seem to be a measurement of the torso posture.

For wow long did the patients use the VBM? As now it seems like they all used it at least those 36 hours, i.e. a compliance rate that is rare (do not people in that age have several different glasses, maybe an accelerometer at each pair would have been enough to reveal which pair they are using and what distance they are focusing?)

The company name is mentioned many times, too many times, as in a commercial folder. As now (see my comments above, this can possibly be improved) the manuscript looks too much as a product commercial folder.

I have comments on the authors’ conclusions:

“Prior to cataract/refractive surgery in addition to comprehensive eye exam in terms of overall health of the eyes, refraction measurements (amount of nearsightedness, farsightedness, astigmatism), determining the corneal curvature, as well as the length of the eye,...”  This is not new, but true.

“..it is necessary to have objective information about patient’s daily activities and real patient’s vision needs.”  This is not studied and cannot be conclusion of this study.

“All s information are essential to help surgeon select the proper power of intraocular lens and provide the best vision possible after surgery.” If s means such information, again, the surgeons’ usefulness of the VBM information is not studied, so nor this can be a conclusion of this study.

“Vivior Visual Behaviour Monitor (VBM) can measure useful parameters, such as working distances, viewing distances during the day, ambient light level or illuminance of the device’s surroundings, as well as head and neck rotation, while using its unique sensors.” Neither this has been studied. But as I write above, it should be studied (and if it has been studied it should be referred to). So again it cannot be a conclusion of this study.  

“To test the device usability among patients, the multi-center feasibility study was conducted. Results from this feasibility study demonstrate that the VBM is capable of objectively assessing vision in both cataract and refractive lens exchange patients. Patients found the device easy to use, with most reporting that the device was not intrusive. The patients, even elderly with significant cataracts, were able to comfortably use the VBM.”  This is OK and according to the results of the study (after my comments have been addressed).

“The next phase of clinical investigation, the Vivior Validation Study, will seek to objectively assess the impact of wearing the VBM on surgeons’ treatment decision making.”  This sounds interesting, but in this manuscript, assumptions of the importance for the decision has already been written (as highly important, e.g. a few lines above: "information are essential to help surgeon select the proper power of intraocular lens").

Possibly this whole manuscript could be accepted (or it will needed to be combined with the next steps), if it is more focused an the question of the title, and explain that if patients like to use the VBM, the next steps would involve testing of its measuring validity, and if it may influence the surgery.

Author Response

The aim of this study was to evaluate the user friendliness of a new tool.

That could be enough for a scientific article. The study is ambitious in the number of patients that were included.

The tool is the Vivior VBM, which measures the distance at which patient activities are performed, lighting conditions, the level of illumination during patient activities, and head angles to the three axes. It is explained in the background that this information, that show the patient’s real-world visual needs, is needed before a cataract or refractive surgery. However, the actual references and explanations for how this information will improve the new lens (there are not that many alternations to be considered, and most patients today still needs corrections afterwards) is missing.

Thank you for this important remark. We have implemented this aspect of how lens selection can be made from the VBM data in the paper.

Also in the usability of a measuring tool is also the question and quantification of its validity. A reference to a validation study or possibly, for each of the measures the validity of other tools using the same principles and hardware. This is also missing.

The usefulness of a measuring tool should be checked, and its validity evaluated. We absolutely agree with the reviewer’s comment. The following sentence was added in Discussion section:

Although the issue and quantification of the measurement tool validation are very important, they will not be discussed in present paper. The data collected is still being processed and surely be the subject of our future research.

 

The method part shows example of the measurements that are included in the VBM. These are impressive! But again, their validity is not mentioned. A detail that I do not understand, is if the patients used glasses during night. As it looks from figure 5:

Thank you very much for this excellent remark. Measurements during the night are automatically excluded because the patient does not wear his glasses during sleep. This is taken into account in the analysis. We absolutely agree with the reviewer that without further explanation the figure can be confusing. We have replaced figure 5 with a much better example (see paper).

Measurements of head flexion and lateral bending over time. Further about that figure: Why is lateral bending good to know prior to a surgery? And I think that flexion and bending should be replaced with head position, since there does not seem to be a measurement of the torso posture.

The head flexion and lateral bending are used by the algorithm to automatically recognize viewing activities (as now added in the description of the data analysis). The comment on head position/orientation vs. Head flexion is absolutely correct, since device measures head orientation in relation to gravitational field and torso orientation is not measured. However, we can assume the (semi-)vertical orientation of torso in most cases. Due to its very valuable remark, we have removed the corresponding figure, the more it is not essential for this study. Thank you very much.

For wow long did the patients use the VBM? As now it seems like they all used it at least those 36 hours,
i.e. a compliance rate that is rare (do not people in that age have several different glasses, maybe an
accelerometer at each pair would have been enough to reveal which pair they are using and what distance they are focusing?)

This is a very good and important point. Thank you. The length of wear obviously varies among patients and ranges from 12 to 100+ hours. Obviously, adherence to the required time is an important parameter, but it has not been systematically analysed in the current study. We have added explanations to the text on the handling of multiple spectacles: patients have been fitted with several adapters that are attached to the available spectacles and the device is configured to automatically detect the attachment to the spectacles and the start/pause measurements. Accordingly, we have made the extensions and explanations in the paper.

 

The company name is mentioned many times, too many times, as in a commercial folder. As now (see my comments above, this can possibly be improved) the manuscript looks too much as a product commercial folder.

We are in complete agreement with this. We only write VBM in the paper instead of "Vivior VBM". We have made this adjustment in the paper.

I have comments on the authors’ conclusions:

“Prior to cataract/refractive surgery in addition to comprehensive eye exam in terms of overall health of the eyes, refraction measurements (amount of nearsightedness, farsightedness, astigmatism), determining the corneal curvature, as well as the length of the eye,...” This is not new, but true.

Due to the valuable input from the reviewer, we have included details in the paper text that have their importance in relation to a surgery. Thank you very much.

 

“..it is necessary to have objective information about patient’s daily activities and real patient’s vision needs.” This is not studied and cannot be conclusion of this study.

“All s information are essential to help surgeon select the proper power of intraocular lens and provide the best vision possible after surgery.” If s means such information, again, the surgeons’ usefulness of the VBM information is not studied, so nor this can be a conclusion of this study.

Thank you very much for this important point. This content of the text was written much too explicitly by us. We have adapted the text accordingly in the paper.

 

“Vivior Visual Behaviour Monitor (VBM) can measure useful parameters, such as working distances, viewing distances during the day, ambient light level or illuminance of the device’s surroundings, as well as head and neck rotation, while using its unique sensors.” Neither this has been studied. But as I write above, it should be studied (and if it has been studied it should be referred to). So again it cannot be a conclusion of this study.

We are in complete agreement. We have removed that part accordingly.

“To test the device usability among patients, the multi-center feasibility study was conducted. Results from this feasibility study demonstrate that the VBM is capable of objectively assessing vision in both cataract and refractive lens exchange patients. Patients found the device easy to use, with most reporting that the device was not intrusive. The patients, even elderly with significant cataracts, were able to comfortably use the VBM.” This is OK and according to the results of the study (after my comments have been addressed).

Many thanks for this remark!

“The next phase of clinical investigation, the Vivior Validation Study, will seek to objectively assess the
impact of wearing the VBM on surgeons’ treatment decision making.” This sounds interesting, but in
this manuscript, assumptions of the importance for the decision has already been written (as highly
important, e.g. a few lines above: "information are essential to help surgeon select the proper power of
intraocular lens").

Many thanks! We have adapted the text in the paper.

Possibly this whole manuscript could be accepted (or it will needed to be combined with the next
steps), if it is more focused an the question of the title, and explain that if patients like to use the VBM,
the next steps would involve testing of its measuring validity, and if it may influence the surgery.

Thanks very much for the very valuable remarks and inputs. We have followed them all meticulously. This has made the paper much better.

Reviewer 2 Report

Objective measurement of visual behaviour is becoming more important since more sophisticated systems to replace age-related cataract lenses with multifoacal intraocular lenses are available. It is hoped that information about visual behaviour will allow surgeons to better understand individual patients´ needs and thus enable optimize the selection of the intraocular lens in cataract surgery. This study aims to understand patients´perspectives in regard to feasibility and ease of use of a visual behaviour monitoring system (Vivior AG). This is relevant, as the integration of patients´ perspectives into the evaluation of new medical devices is an essential part and contributes to patient-centered medicine (Leidy NK, Integrating the patient´s perspective into device evaluation trials. Value in Health 2006;9(6):394-401). The study aimed to collect and analyze data from patients by using a questionnaire.

Unfortunately, the study lacks significant rigor and the paper provides parsimonious information particularly in regard to the methodology, results and discussion. There are hardly any information about the patient selection criteria, enrollment of the selected patients, randomization process etc. There is no information about the age- an sex structure nor about any condition that might have had an impact on the application of the questionnaire. It is not defined how the criteria of "ease of use" and "feasibility" informed the study, and it seems as if the study tools had not been informed by a theoretical framework such as Davis´ framework about perceived usefulness, ease of use and user acceptance (see Davis, MIS Quarterly, 1989, 319-340). The section ´Materials and Methods`contains lots of technical information about the device and most of it does not answer the research question. Other basic information such as details about the questionnaire is lacking. The presentation of the results is again parsimonious, and any interesting aspect such as disaggregation of the findings is lacking. Other results do not answer the research question. The discussion and conclusions contain statements that are not supported by any data.

It might be helpful for the author to consider standards of evidence that were recommended for device evaluation, such as psychometrically soundness, validity, accurate analysis of bias etc. (the aforemntioned paper by Leidy et al might be useful).

Author Response

Objective measurement of visual behaviour is becoming more important since more sophisticated systems to replace age-related cataract lenses with multifoacal intraocular lenses are available. It is hoped that information about visual behaviour will allow surgeons to better understand individual patients´ needs and thus enable optimize the selection of the intraocular lens in cataract surgery. This study aims to understand patients´perspectives in regard to feasibility and ease of use of a visual behaviour monitoring system (Vivior AG). This is relevant, as the integration of patients´ perspectives into the evaluation of new medical devices is an essential part and contributes to patient-centered medicine (Leidy NK, Integrating the patient´s perspective into device evaluation trials. Value in Health 2006;9(6):394-401). The study aimed to collect and analyze data from patients by using a questionnaire.

Many thanks for this input. We have listed this important part in the discussion and cited the literature accordingly.

Unfortunately, the study lacks significant rigor and the paper provides parsimonious information particularly in regard to the methodology, results and discussion. There are hardly any information about the patient selection criteria, enrollment of the selected patients, randomization process etc. There is no information about the age- an sex structure nor about any condition that might have had an impact on the application of the questionnaire. It is not defined how the criteria of "ease of use" and "feasibility" informed the study, and it seems as if the study tools had not been informed by a theoretical framework such as Davis´ framework about perceived usefulness, ease of use and user acceptance (see Davis, MIS Quarterly, 1989, 319-340). The section ´Materials and Methods`contains lots of technical information about the device and most of it does not answer the research question. Other basic information such as details about the questionnaire is lacking. The presentation of the results is again parsimonious, and any interesting aspect such as disaggregation of the findings is lacking. Other results do not answer the research question. The discussion and conclusions contain statements that are not supported by any data.

This is a very important point. We have listed in detail the extensions and corrections in the paper with study data such as inclusion and exclusion criteria, demographics etc. Due to the Important Remarks, we have practically completely rewritten the section "Material and Methods". Thus, the paper has become much better! Thanks a lot!

We have informed the patient in great detail about the content of the paper, its usefulness and feasibility before starting the study. With this content we have expanded the paper, which was not described so explicitly. Thank you very much.

Thanks to the important input, we have implemented further results of the study, which illustrate the feasibility, usefulness and user-friendliness, in particular two additional figures were added. Thanks for the remark and the literature provided, which was very helpful. With this information the paper has gained a lot.

It might be helpful for the author to consider standards of evidence that were recommended for device evaluation, such as psychometrically soundness, validity, accurate analysis of bias etc. (the aforemntioned paper by Leidy et al might be useful).

Thank you very much for this important point. The cited paper was very helpful. We have made appropriate adjustments in the paper. Our work has thus become much more rounded off.

Reviewer 3 Report

Revision of paper applsci-6690222

The authors address a rather interesting topic concerning a method to improve cataract surgery planning and subsequent patient satisfaction by considering the patients visual needs. The paper focus on the results from a questionnaire reporting subject’s experience with a wearable device capable of measuring several visual related parameters during daily activities. Despite dedicating a large part of the paper describing the device functionalities, a link between these and the main outcome of this work (user friendliness assessment of the wearable device) is missing and, thus, the reader may feel that a conducting line clearly explaining the purpose of the paper is absent or fuzzy. Since the novelty of the paper is not placed on the wearable (if it was, the paper would have insufficient description of the proposed device), but rather on brief and very lightly described results of a questionnaire, it appears that this work in its present form is not suitable for publication, unless major improvement is performed. The following comments address some major and minor issues to be improved.

General Comments

Abstract:

The abstract does not give enough information about the time and characteristics of the questionnaire. It is not clear if the device was used pre-op or post-op.

Line 24: “questionnaire (on) about their experience in wearing the device” -> remove “(on)”.

 

Introduction:

Lines 34-35: The sentence mentions the past 10 years (this would be the 2010-2019 period) but reference [1] is from 2004.

Line 41: typo -> (and)

Lines 44-47: This analysis is concerned with distance vision correction. It would be relevant to analyze the intermediate and near vision problem as well, since “spectacle independence” is not achieved with every type of IOL.

Lines 55-72: This section presents a short review on cataract surgery outcome questionnaires. The authors argue that subjective may limit their application, while eye tracking technologies would provide a more objective approach. This needs further insight. A proper review on these emerging technologies and the relevant contextualization in ophthalmic applications would benefit this section. Moreover, a questionnaire on wearable comfort and ease of use is the main subject of the paper, but no review on this type of questionnaires is presented.

 

Materials and Methods:

This section should be better organized. Most of the section is devoted to the enumeration of the features of the wearable device but there are no details of the principles of operation, as well as the specifications (for instance, range of distance measurements and illuminance levels) of the instrument. Without explaining these details, the reader is not able to understand the advantages and limitations of the method. How is distance measured. How is gaze taken into account? How accurate is the procedure?

The Cat-Quest 9 questionnaire is performed but later in the Results section there is no mention to it. Exclusion criteria are not described. The validation of the Cat-Quest 9 questionnaire in different languages is not mentioned. Also, how are the results used? How do they relate to the proposed questionnaire (about patient experience in wearing the device)?

And why is the most important contribution of this paper not described in this section?

 

Results and Discussion

Lines 174-176: Why is post-operative data mentioned in this section? The previous section refers a total of 129 patients (no reference to the number of male and female) enrolled in the study. But in the Results section, a total of 178 eyes implanted with IOL is mentioned. How does this relate to the former information? Again, this is confusing, and one cannot understand at what stage of the process is the wearable device used.

Since a significant part of the paper is dedicated to device functionality, it would be interesting to cross the results with questionnaire results. Otherwise, there is not much point on dedication such a large part of the paper to it.

Insufficient statistical analysis of the Questionnaire results is presented.

The Discussion section are a repetition of what was previously said in the Introduction for the motivation of this work. It would be expected to discuss the Results. Subjectivity is referred as a limitation of current approaches. It is not relevant in this context, where a questionnaire regarding patient experience in wearing the device, prone to subjectivity as well, is presented.

Conclusion

Following the trends of the Discussion, the Conclusions are vague and barely relate to the main outcome of the paper as highlighted in the Abstract. When the authors state that “Results from this feasibility study demonstrate that the VBM is capable of objectively assessing vision in both cataract and refractive lens exchange patients.”, this is a rather questionable remark, since nowhere in the paper is this clearly demonstrated, nor is the methodology for achieving such demonstration is designed. If it was the case, the authors would have presented a stronger case for paper publication.

 

 

Author Response

The authors address a rather interesting topic concerning a method to improve cataract surgery planning and subsequent patient satisfaction by considering the patients visual needs. The paper focus on the results from a questionnaire reporting subject’s experience with a wearable device capable of measuring several visual related parameters during daily activities. Despite dedicating a large part of the paper describing the device functionalities, a link between these and the main outcome of this work (user friendliness assessment of the wearable device) is missing and, thus, the reader may feel that a conducting line clearly explaining the purpose of the paper is absent or fuzzy. Since the novelty of the paper is not placed on the wearable (if it was, the paper would have insufficient description of the proposed device), but rather on brief and very lightly described results of a questionnaire, it appears that this work in its present form is not suitable for publication, unless major improvement is performed. The following comments address some major and minor issues to be improved.

Thank you very much for the positive feedback on the importance of these data. At the same time, we thank the reviewer for giving us the opportunity to clarify all open points. Thus, the authors will work on each point in detail.  

General Comments
Abstract:
The abstract does not give enough information about the time and characteristics of the questionnaire. It is not clear if the device was used pre-op or post-op.

This is an important point. Thank you! A total of 129 were included in the study as part of a multicenter study. All measurements were performed before surgery. We have listed the text in the paper accordingly.

Line 24: “questionnaire (on) about their experience in wearing the device” -> remove “(on)”.

done

Introduction:
Lines 34-35: The sentence mentions the past 10 years (this would be the 2010-2019 period) but reference [1] is from 2004.

That is absolutely correct. Thank you very much. It should stand for 30 years. I have quoted the initial literature accordingly and corrected the text of the paper accordingly.

Line 41: typo -> (and)

done

Lines 44-47: This analysis is concerned with distance vision correction. It would be relevant to analyze the intermediate and near vision problem as well, since “spectacle independence” is not achieved with every type of IOL.

This is an excellent input. We have extended the text extensively and not only the distance but also the intermediate and near vision. The parts are marked accordingly in the paper.

Lines 55-72: This section presents a short review on cataract surgery outcome questionnaires. The authors argue that subjective may limit their application, while eye tracking technologies would provide a more objective approach. This needs further insight. A proper review on these emerging technologies and the relevant contextualization in ophthalmic applications would benefit this section. Moreover, a questionnaire on wearable comfort and ease of use is the main subject of the paper, but no review on this type of questionnaires is presented.

Thank you very much. This is an important point which we have dealt with in detail and have inserted at the appropriate place in the paper. A part of the methods we have extended with this question.

We have followed the reviewer's input by providing an overview of these new technologies and their contextualization in ophthalmological applications. We have extended the text of the paper accordingly. Thank you for the encouragement to include this aspect in our paper. As a result the paper has become better.

Materials and Methods:
This section should be better organized. Most of the section is devoted to the enumeration of the features of the wearable device but there are no details of the principles of operation, as well as the specifications (for instance, range of distance measurements and illuminance levels) of the instrument. Without explaining these details, the reader is not able to understand the advantages and limitations of the method. How is distance measured. How is gaze taken into account? How accurate is the procedure?

Thank you very much for this valuable remark. In the chapter “Methods” we have changed and structured in great detail. These remarks and text extensions are marked in the paper.

This has been addressed by restructuring device description and adding data analysis and considerations for the lens selection based on VBM data.

Specifically on the gaze tracking: device does include gaze tracking, instead it continuously samples the viewing scenery with the distance sensors and in combination with motion/orientation sensors reconstructs the 3D space, which is further used for automatic identification of visual activities. Based on the activity, the viewing distance is derived from the sampled space.

The Cat-Quest 9 questionnaire is performed but later in the Results section there is no mention to it. Exclusion criteria are not described. The validation of the Cat-Quest 9 questionnaire in different languages is not mentioned. Also, how are the results used? How do they relate to the proposed questionnaire (about patient experience in wearing the device)?

This is a very important point. We have added the inclusion and exclusion criteria in the methods, especially the primary endpoint. In particular, the questions of the questionnaire were listed in detail. These passages are marked accordingly in the paper. Many thanks for this input.

And why is the most important contribution of this paper not described in this section?

We absolutely agree with the reviewer. We have expanded detailed this part of the paper accordingly. Thanks a lot!

Results and Discussion
Lines 174-176: Why is post-operative data mentioned in this section? The previous section refers a total of 129 patients (no reference to the number of male and female) enrolled in the study. But in the Results section, a total of 178 eyes implanted with IOL is mentioned. How does this relate to the former information? Again, this is confusing, and one cannot understand at what stage of the process is the wearable device used.

In all patients the VBM measurement was performed before cataract or refractive surgery. From these 129 patients included in the study, 178 cataract surgeries ultimately resulted, i.e. both eyes were not operated on in all cases, but only those which had an indication for surgery. Thank you very much for this valuable input and for the opportunity to provide clarity here. We have listed and specified it accordingly in the paper.

Since a significant part of the paper is dedicated to device functionality, it would be interesting to cross the results with questionnaire results. Otherwise, there is not much point on dedication such a large part of the paper to it.

This is an important input. We have presented the relevant results according to the questionnaire, and in particular we have added figures. The result part fits so much better to the already extended methods part. The paper has become much better as a result. Many thanks for the encouragement to extend the result part significantly. The images and text are marked accordingly in the paper.

Insufficient statistical analysis of the Questionnaire results is presented.

Excellent remark and fits very well with the previous remark. We have edited this part and included it in the results. Thank you very much.

The Discussion section are a repetition of what was previously said in the Introduction for the motivation of this work. It would be expected to discuss the Results. Subjectivity is referred as a limitation of current approaches. It is not relevant in this context, where a questionnaire regarding patient experience in wearing the device, prone to subjectivity as well, is presented.

We took the reviewer's input very much to heart and expanded the discussion considerably, where our own results were discussed and built into the whole work. Thank you very much.

Conclusion
Following the trends of the Discussion, the Conclusions are vague and barely relate to the main outcome of the paper as highlighted in the Abstract. When the authors state that “Results from this feasibility study demonstrate that the VBM is capable of objectively assessing vision in both cataract and refractive lens exchange patients.”, this is a rather questionable remark, since nowhere in the paper is this clearly demonstrated, nor is the methodology for achieving such demonstration is designed. If it was the case, the authors would have presented a stronger case for paper publication.

Thanks for the remark. We have followed the input of the reviewer and have expanded the conclusion part in this sense. The corresponding parts were marked in the text.

We are convinced, and the results speak for themselves, that the objective measurement has a high impact in terms of patient consultation, documentation and thus also treatment quality. At this point we would like to thank the reviewer who took the time to provide very valuable input. As a result our paper has become significantly better.

Round 2

Reviewer 1 Report

Thank you for your improvements!

However, I still think that the conclusion are promising more than this study has shown:

Please delete the sentence:

”It is necessary to obtain objective information about patient¢s daily activities and real patient¢s vision needs.”.  How can that be necessary when very many cataract and refractive surgeries have been carried out without such information?

How does the sentence

”There is not the ideal correction for all distances, but through objective measurement using Visual Behaviour Monitor (VBM) regarding distance measurement, ambient lighting and head posture will potentially determine a much more accurate setting and support the choice of IOL.”  Justify the conclusion “It has been shown that VBM measurement...  ..will also have a major impact on a customize decision regarding surgical method and IOL choice.”?

I dislike wordings as “We are confident that…” in Conclusions.

To be clear, I think that the conclusions, which now includes text that may be written in the discussion, and unjustified conclusions, should be cut down to:

To test the VBM device usability among patients, this multi-center feasibility study was conducted. Patients found the device easy to use, with most reporting that the device was not intrusive. The patients, even elderly with significant cataracts, were able to comfortably use the VBM. Hence, the results of this feasibility study demonstrated that the VBM may be used correctly and reliably by the patient.

The VBM data may have the potential to improve treatment outcomes, and the next phase will seek to objectively assess the impact of the VBM on surgeons’ treatment decision making.

 

Author Response

We are indebted to the reviewer. His very valuable comments and suggestions have significantly improved our paper. Many Thanks!

Thank you for your improvements!

However, I still think that the conclusion are promising more than this study has shown:

Please delete the sentence:

”It is necessary to obtain objective information about patient¢s daily activities and real patient¢s vision needs.”.  How can that be necessary when very many cataract and refractive surgeries have been carried out without such information?

Many thanks for the important note. With this sentence the authors wanted to express that with the VBM measurement a further customize quality improvement is possible regarding the care of the patient. However, we absolutely agree that it is too explicit. We have removed this sentence in the conclusions. Many thanks!

How does the sentence

”There is not the ideal correction for all distances, but through objective measurement using Visual Behaviour Monitor (VBM) regarding distance measurement, ambient lighting and head posture will potentially determine a much more accurate setting and support the choice of IOL.”  Justify the conclusion “It has been shown that VBM measurement...  ..will also have a major impact on a customize decision regarding surgical method and IOL choice.”?

Many thanks for the very concrete input. We have replaced or adapted the corresponding sentence. The content is now even better emphasized and corresponds even more precisely to the study conditions.

I dislike wordings as “We are confident that…” in Conclusions.

Thank you very much for that remark. We have reworded or replaced the sentence.

To be clear, I think that the conclusions, which now includes text that may be written in the discussion, and unjustified conclusions, should be cut down to:

To test the VBM device usability among patients, this multi-center feasibility study was conducted. Patients found the device easy to use, with most reporting that the device was not intrusive. The patients, even elderly with significant cataracts, were able to comfortably use the VBM. Hence, the results of this feasibility study demonstrated that the VBM may be used correctly and reliably by the patient.

The VBM data may have the potential to improve treatment outcomes, and the next phase will seek to objectively assess the impact of the VBM on surgeons’ treatment decision making.

Many thanks for this excellent formulation on the part of the reviewer. We were very happy to adopt its content and incorporated it into the text accordingly. Parts of the conclusions were omitted. Thus, our paper has become much better.

Reviewer 2 Report

Thank you very much for the additional information that was provided. Although some of the issues were adressed there remain substantial weaknesses:

1) The study was designed to "evaulate easiness of use, as well as patient perception of wearing the device." Most of the information that was added to the section "Materials and Methods" do not contribute to answer your research question. For example, it is not comprehensive how the CATQ9-questionnaire was modified and even why it was selected as a tool. This specific questionnaire estimates patient reported difficulties with vision and can be used to compare pre- and postoperative situations, but how shall it provide information about "easiness of use of your device"? In how far do the results of this test (as shown in figure 8) enable the reader to draw conclusions about the research question? Consequently, the discussion suffers from a confusing concoction of patient´s satisfaction with "the surgery" (whatever that means) and satisfaction with the use of the device as well as claimed "reliability" of it. Most of the conclusions are not convincingly supported by the results that are still presented parsimoniously. 

2) The inclusion criteria yield a extremely diverse study group (still randomization is not clear), but results lack utter disaggregation. It would be interesting to learn if there were differences between a 20 year old patient undergoing refractive surgery and a 80 year old cataract blind patient.

3) Most of the additional information provided is confined to technical aspects of the device which does not inform the research question. If it was an issue that important parts of the study (clear definition of key terms such as "easiness of use", much more detailed data analysis, etc) had not been planned accordingly it might be worth wile to redesign the study.

Author Response

Thank you very much for the additional information that was provided. Although some of the issues were adressed there remain substantial weaknesses:

The study was designed to "evaulate easiness of use, as well as patient perception of wearing the device." Most of the information that was added to the section "Materials and Methods" do not contribute to answer your research question. For example, it is not comprehensive how the CATQ9-questionnaire was modified and even why it was selected as a tool. This specific questionnaire estimates patient reported difficulties with vision and can be used to compare pre- and postoperative situations, but how shall it provide information about "easiness of use of your device"? In how far do the results of this test (as shown in figure 8) enable the reader to draw conclusions about the research question? Consequently, the discussion suffers from a confusing concoction of patient´s satisfaction with "the surgery" (whatever that means) and satisfaction with the use of the device as well as claimed "reliability" of it. Most of the conclusions are not convincingly supported by the results that are still presented parsimoniously. 

Thank you for the opportunity to further clarify this point.

The CATQ9-questionnaire was initially designed to evaluate the benefit of cataract surgery. This seemed to us a very good basis for our study, however, it had to be adapted to our question, since not primarily the cataract operation is in the foreground but the application of the VMB. However, of course, the cataract operation parameters must also be taken into account. This led to a study specific question which looked like the following:

With this questionnaire we wanted to find out what difficulties cataract patients have in their everyday life and what expectations they have of our operation. We analysed how the patient currently perceives his or her vision with the following questions:

how much trouble does the patient have with vision in everyday life? how satisfied is the patient with your vision at the moment? how big problems does your vision cause you during the following activities: Reading on a computer or tablet Recognize the faces of the people the patient meets Recognising the prices when shopping Uneven ground, e.g. cobblestone Handicrafts, DIY, etc. Read the subtitles while watching TV With favourite occupation or hobbies

Each of these questions was evaluated by the patient using a 1-4 scor scale

 

At the end of this CRF each patient was asked the following open question:

What are the patient's expectations and hopes for his or her vision after the surgery.

This first part contains mainly questions about the patient's subjective expectations regarding his or her upcoming surgery.

After the VBM measurement had been carried out, a usability questionnaire was presented to the study patient, actually the main question of our study. The following points were asked of the study patient:

Is it easy to attach the magnetic clip to your frames?      

Is it easy to attach the wearable to the magnetic clip?      

Do you always feel comfortable using the wearable?      

Is it easy to use the wearable?

Is the connection between the station and the wearable always working?

Is the information provided on the station screen always clear to you?

The feasibility of a measurement in terms of reliability and simplicity has been eminently important for such an application to be applicable at all. These were illustrated in great detail in the results.

After the operation the patient was interviewed again. The third questionnaire was used to find out whether the patient's vision met the expectations he or she had before the operation and how vision had changed compared to the time before the operation. It was important to know if and how the patient's expectations were met.

Furthermore, the following question was evaluated analogous to the first questionnaire:

What difference did the patient notice in the following activities after his operation?

Recognizing the faces of the people the patient meets Recognising the prices when shopping Uneven ground, e.g. cobblestone Handicrafts, DIY, etc. Read the subtitles while watching TV With favourite occupation or hobbies

In this sense, several points were examined in the study, i.e. the feasibility of VMB measurement but also patient satisfaction with regard to his expectations and the outcome of the operation itself.

Thank you very much for giving us the opportunity to provide clarity. We have expanded and listed the missing points in the text accordingly. Many thanks to the reviewer.

 

The inclusion criteria yield a extremely diverse study group (still randomization is not clear), but results lack utter disaggregation. It would be interesting to learn if there were differences between a 20 year old patient undergoing refractive surgery and a 80 year old cataract blind patient.

This is an important point. Thank you. We have consecutively enrolled those patients who met the inclusion criteria. We've amended the text in the paper accordingly. The inclusion criteria were established according to international standards and the requirements of the ethics committee. We agree with the reviewer that it would be very interesting to evaluate the difference between a 20-year-old refractive surgery patient and a blind 82-year-old patient. Unfortunately, it is not possible in this study because as stated in chapter results the mean age was 65 with a range of 49 to 82 years old. As we have a very good medical care in Europe, we see practically no blind patients because of cataract. So, it was also in our study. But the question is very interesting, and we thank the reviewer very much. We will include this question in future studies. Many thanks.

3) Most of the additional information provided is confined to technical aspects of the device which does not inform the research question. If it was an issue that important parts of the study (clear definition of key terms such as "easiness of use", much more detailed data analysis, etc) had not been planned accordingly it might be worth wile to redesign the study.

Thank you very much for that remark. In this study, which is the first of its kind, we have seen many new aspects that we have incorporated in further studies. We have expanded the study design in many respects in a follow-up study. Of course, we had to take into account that the protocol should not be too large so that we could include all patient age groups. Thank you very much for these important points.

Reviewer 3 Report

I congratulate the authors for their effort in improving the manuscript contents.  It is now more pleasing to read and reveals a lot more interesting information. Some minor issues persist, and a small report on aspects to improve follows:

Some corrections:

Page 1, line 36: The range of years was updated but the number 400-fold referred to a time lapse between 1994 and 2004, so it is not correct again. Please use updated references and be more accurate in your statements. It should be “with over 3.6 million surgeries in the United

States alone”. (A CSR of 11000/million per million population in 1 year times 330 million) I suggest the reference should be updated to a more recent paper:

Wei Wang, William Yan, Kathy Fotis, Noela M. Prasad, Van Charles Lansingh, Hugh R. Taylor, Robert P. Finger, Damian Facciolo, Mingguang He; Cataract Surgical Rate and Socioeconomics: A Global Study. Invest. Ophthalmol. Vis. Sci. 2017;57(14):5872-5881. doi: https://doi.org/10.1167/iovs.16-19894.

Page 2, line 65: Instead of “There is not the ideal correction for all distances” should read “There is no ideal correction for all distances”.

Page 2, line 75: “Visual behaviours are developed through various technological approaches” should be perhaps “Visual behavior monitors (or trackers) are developed through various technological approaches”?

In the results section, it would be preferable to increase resolution in figures 6 and 7.  The inclusion of Figure 9 is very meaningful. It is now much easier to follow the rationale of the paper.

Page 13, line 340: a space is missing in “the wearableVBM”.

Page 13, lines 346-348: the sentence “Due to the detailed analysis and VBM measurement, not only the patient's behaviour was better understood, but also the patient satisfaction was very high or the result met the expectations” expresses the author’s optimistic perception on the outcome of their study. However, without a “control group” it is not possible to make this statement. Since it is not possible to go back and correct this problem, I would suggest more conservative considerations with this respect. As the authors refer in the Conclusions, this would pertain to the “Vivior Validation Study” that is yet to come.

 

Author Response

x

With the comments from the reviewer, the paper has become much better. Many thanks to the reviewer!

I congratulate the authors for their effort in improving the manuscript contents.  It is now more pleasing to read and reveals a lot more interesting information. Some minor issues persist, and a small report on aspects to improve follows:

Some corrections:

Page 1, line 36: The range of years was updated but the number 400-fold referred to a time lapse between 1994 and 2004, so it is not correct again. Please use updated references and be more accurate in your statements. It should be “with over 3.6 million surgeries in the United States alone”. (A CSR of 11000/million per million population in 1 year times 330 million) I suggest the reference should be updated to a more recent paper:

Wei Wang, William Yan, Kathy Fotis, Noela M. Prasad, Van Charles Lansingh, Hugh R. Taylor, Robert P. Finger, Damian Facciolo, Mingguang He; Cataract Surgical Rate and Socioeconomics: A Global Study. Invest. Ophthalmol. Vis. Sci. 2017;57(14):5872-5881. doi: https://doi.org/10.1167/iovs.16-19894.

Thank you very much for this excellent reference. We have now listed this part precisely and quoted the recommended literature. Many thanks!

Page 2, line 65: Instead of “There is not the ideal correction for all distances” should read “There is no ideal correction for all distances”.

Done. Many thanks

Page 2, line 75: “Visual behaviours are developed through various technological approaches” should be perhaps “Visual behavior monitors (or trackers) are developed through various technological approaches”?

Thanks for the remark. We have changed the text accordingly. Many Thanks!

In the results section, it would be preferable to increase resolution in figures 6 and 7.  The inclusion of Figure 9 is very meaningful. It is now much easier to follow the rationale of the paper.

This is a very important clue. We have worked through the figures accordingly. With the new figures the paper has become much better. Thank you very much for this input!

Page 13, line 340: a space is missing in “the wearableVBM”.

done

Page 13, lines 346-348: the sentence “Due to the detailed analysis and VBM measurement, not only the patient's behaviour was better understood, but also the patient satisfaction was very high or the result met the expectations” expresses the author’s optimistic perception on the outcome of their study. However, without a “control group” it is not possible to make this statement. Since it is not possible to go back and correct this problem, I would suggest more conservative considerations with this respect. As the authors refer in the Conclusions, this would pertain to the “Vivior Validation Study” that is yet to come.

We absolutely agree with the reviewer. We've removed that part from the discussion. In the Conclusions we have referred to the planned or ongoing Vivior Validation Study. Thank you very much for the input. This has made the content of the paper much more precise.

Round 3

Reviewer 2 Report

Thank you for forwarding a revised version of the article. As discussed already, most of the information provided in the different sections are not informing the research question. Also, the conclusions are still too vague (e.g. the VBM is claimed to be much more accurate but the study did not include any data about alternative measurement. It might be possible that "user friendliness" yields more accurate measurement yielding in better IOL selection, but this was not examined by the study).

Also, at least basic information about the visual acuity would have been important if no cataract blind patient had been included. According to the results of the LOC there were at least a few patients with grade 5 cataract, and some patients reported having "very great difficulty" recognizing faces and reading a computer screen. Disaggregated data for at least those patients is still missing. 

Author Response

We have reviewed the paper several times in detail and made appropriate adjustments, in particular two figures, figure 7 and 8, have been improved and table 2 has been added. The changes in the text have all been marked in the paper.

Thank you for forwarding a revised version of the article. As discussed already, most of the information provided in the different sections are not informing the research question. Also, the conclusions are still too vague (e.g. the VBM is claimed to be much more accurate but the study did not include any data about alternative measurement. It might be possible that "user friendliness" yields more accurate measurement yielding in better IOL selection, but this was not examined by the study).

We appreciate feedback of Reviewer 2. However, we would like to comment on the points mentioned by the Reviewer. Reviewer still believes that the conclusions are still too vague. In the previous iterations we have significantly reduced the scope of claims in the conclusions, we have essentially adopted the form suggested by Reviewer 1 on Round 2, which only relates to the questions addressed in the study. The claim cited by Review 3 for the VBM to be “to be much more accurate” is actually found at the end of Discussions section and actually speculates that “objective measurement … will potentially determine a much more accurate setting and support the choice of IOL”. Thus the future tense and word “potentially” is used to avoid misinterpretation. In the next sentence we clarify that the studies to address this question are ongoing. In order to make it clear that we are referring to the general method of objective measurements, we have modified this phrase to relate to all “devices as VBM” and not specifically to our solution. We have also rephrased the sentence to remove “much more”. We have tried everything to answer the questions of reviewer 2 in this way, without creating a new contradiction with reviewers 1 and 3. The corresponding passages are marked directly in the paper.

 

Also, at least basic information about the visual acuity would have been important if no cataract blind patient had been included. According to the results of the LOC there were at least a few patients with grade 5 cataract, and some patients reported having "very great difficulty" recognizing faces and reading a computer screen. Disaggregated data for at least those patients is still missing. 

We have reviewed the feedback on usability with regard to the difficulties reported in the tasks of daily life. Due to the small size of the groups that reported very great difficulties in daily life, high cataract density and difficulties in using the device, we could not establish a correlation and we did not observe any significant overlap of these groups. In fact, none of the patients who reported very great difficulties in daily life reported difficulties in handling the device, while some (30%) of the patients who reported difficulties in attaching the device to the glasses did not report any problems in activities close to work (crafts, etc.). We have also carried out the analysis of the patients' visual function based on the answers to the questionnaire on user-friendliness and have added Table 2 with the results. We found that the group reporting usability problems actually had a higher average visual acuity and a lower cataract score, which can be related to the higher demands of patients with clear lenses. We have therefore concluded that the difficulties are mainly related to non-visual problems, such as motor skills and variations in frame shapes and dimensions. Based on this feedback, the next generation of the system has been developed to address the implied deficits. We have added the corresponding changes or extensions in the paper, in particular in the discussion section.