A Pilot Randomised Control Trial of an Online Acceptance and Commitment Therapy (ACT) Resilience Training Program for People with Multiple Sclerosis
Round 1
Reviewer 1 Report
This manuscript describes a pilot study explored the effectiveness and feasibility of an online intervention for People with Multiple Sclerosis (PwMS), called e-READY for MS. This pilot study gives important insight on an efficacy protocol about a group ACT resilience training intervention, and the presented results provide an advancement of the current knowledge, demonstrating that participation in the e-READY program is associated with improvements in anxiety and stress, also found in the follow up evaluation. In spite of the conclusions are justified and supported by the results, future studies should increase the sample to strengthen the protocol.
The strength of the manuscript is that the topic is clear, relevant and innovative in the field of multiple sclerosis; good use of the online platform. The experimental design appears quite appropriate to test the hypothesis and reproducible based on the details given in the methods section.
The English Level is appropriate and understandable.
I reported below my main concerns, in particular regarding the size of Waitlist Control Group.
Comment 1. The experimental design appears good and strength. I would consider appropriate to choose a Waitlist Control Group of equal or greater size than the intervention group.
Comment 2. The introduction section is well written. However, please add evidence and references concerning a possible worsening about psychological distress and quality of life due to the recent pandemic situation for PwMS.
Comment 3. About Measures, the authors should consider to add, where available, some specific references on a previous use of the questionnaire on PwMS if is it present in the literature. Otherwise, give reasons why this scale was chosen instead of another.
Comment 4. Please endorse the bibliography. Indeed, in some references the journal is written in abbreviated form, in others in full.
Author Response
REVIEWER 1
This manuscript describes a pilot study explored the effectiveness and feasibility of an online intervention for People with Multiple Sclerosis (PwMS), called e-READY for MS. This pilot study gives important insight on an efficacy protocol about a group ACT resilience training intervention, and the presented results provide an advancement of the current knowledge, demonstrating that participation in the e-READY program is associated with improvements in anxiety and stress, also found in the follow up evaluation. In spite of the conclusions are justified and supported by the results, future studies should increase the sample to strengthen the protocol.
The strength of the manuscript is that the topic is clear, relevant and innovative in the field of multiple sclerosis; good use of the online platform. The experimental design appears quite appropriate to test the hypothesis and reproducible based on the details given in the methods section.
The English Level is appropriate and understandable.
RESPONSE: Thank you for recognizing the strengths of the study and its potential to contribute to the field.
I reported below my main concerns, in particular regarding the size of Waitlist Control Group.
Comment 1. The experimental design appears good and strength. I would consider appropriate to choose a Waitlist Control Group of equal or greater size than the intervention group.
RESPONSE: Thank you for recognizing the strengths of the experimental design. We have mentioned in sub-section 2.5. Data Analysis Approach that “A power analysis conducted before study commencement indicated that to detect an effect size of d=0.44, p <0.01 a total sample size of 90 (45 per group) was required.” Hence, although our intention was to obtain equal numbers in each group, unfortunately the total sample size achieved was 56 and the randomisation procedure resulted in the assignment of unequal numbers of participants assigned to each group (31 intervention and 25 WLC). This is explained in sub-section 2.5. In the manuscript, we have also explained the randomization “was generated with a process based on a custom algorithm based on a programmatic equivalent of a ‘coin-toss’. The algorithm generated a random fraction between 0 and 1, then rounded it to the nearest whole number where 1 = intervention allocation and 0 = WLC allocation. The allocation of each participant was completely independent of all previous allocations (i.e., each ‘coin-toss’ was done independently for each participant).” Finally, we used statistical analyses that were robust when dealing with unequal sample sizes.
Comment 2. The introduction section is well written. However, please add evidence and references concerning a possible worsening about psychological distress and quality of life due to the recent pandemic situation for PwMS.
RESPONSE: We have now mentioned in the Introduction the negative effects of the pandemic on the mental health and quality of life of PwMS as follows “for many PwMS, the emergence of the COVID-19 pandemic and associated strict public health social distancing measures have precluded participation in face-to-face psychosocial interventions, despite evidence suggesting that the pandemic has had adverse impacts on the mental health and QoL of PwMS (Motolese et al. 2020; Talaat et al. 2020).
We have also clarified in sub-section “2.2. Recruitment” that the study was conducted during the COVID-19 pandemic as follows – “Recruitment occurred between mid-May and mid-September 2022 during the COVID-19 pandemic.”
References
Talaat, F., Ramadan, I., Aly, S., & Hamdy, E. (2020). Are multiple sclerosis patients and their caregivers more anxious and more committed to following the basic preventive measures during the COVID-19 pandemic?. Multiple sclerosis and related disorders, 46, 102580.
Motolese, F., Rossi, M., Albergo, G., Stelitano, D., Villanova, M., Di Lazzaro, V., & Capone, F. (2020). The psychological impact of COVID-19 pandemic on people with multiple sclerosis. Frontiers in neurology, 11, 580507.
Comment 3. About Measures, the authors should consider to add, where available, some specific references on a previous use of the questionnaire on PwMS if is it present in the literature. Otherwise, give reasons why this scale was chosen instead of another.
RESPONSE: As requested, for each measure that was not developed for PwMS, we have now added a reference for research that has used the measure in MS samples.
Comment 4. Please endorse the bibliography. Indeed, in some references the journal is written in abbreviated form, in others in full.
RESPONSE: Thank you for pointing this out. We have reviewed the bibliography and made corrections where appropriate that are in accord with the Instructions for Authors document, which states that abbreviated journal names should be used.
Reviewer 2 Report
hank you very much for allowing me to review the article entitled “A Pilot Randomized Control Trial of an Online Acceptance and Commitment Therapy (ACT) Resilience Training Program for People with Multiple Sclerosis” (sclerosis-2316878). This is a pilot study that explored the effectiveness and feasibility of an online version of a group acceptance and commitment therapy (ACT) resilience training intervention for people with multiple sclerosis (MS) (PwMS), called e-READY for MS. In this study the authors explore the feasibility of an online version of the program by investigating recruitment, retention, program engagement, online program usability, and participant perceptions of intervention helpfulness and satisfaction. Comments: I recommend that the first time acronyms are used throughout the text, their meaning be indicated. I suggest that the recruitment of study participants be exposed with a flowchart for a better understanding and identification of participation (section 2.2). I consider that this is part of the methodology and not of the results (figure 1). As well as the informed consent that they have given to their participation, how has it been. The approval of the ethics committee that has approved this study must be indicated. In the intervention section they should indicate the time of each of the modules. I recommend that you expand the measurements collected and the way in which the collection has been carried out. Has the sample size been calculated for this study? In the discussion, one should begin with the main results of the study, not with a new approach to the objective, which is repetitive. Among the limitations of the study, the sample size must be taken into account. As the authors indicate the risk of type 1 error, concluding that there is a difference when in fact there is not is present in this study.
Author Response
REVIEWER 2
Thank you very much for allowing me to review the article entitled “A Pilot Randomized Control Trial of an Online Acceptance and Commitment Therapy (ACT) Resilience Training Program for People with Multiple Sclerosis” (sclerosis-2316878). This is a pilot study that explored the effectiveness and feasibility of an online version of a group acceptance and commitment therapy (ACT) resilience training intervention for people with multiple sclerosis (MS) (PwMS), called e-READY for MS. In this study the authors explore the feasibility of an online version of the program by investigating recruitment, retention, program engagement, online program usability, and participant perceptions of intervention helpfulness and satisfaction.
Comments:
Comment 1. I recommend that the first time acronyms are used throughout the text, their meaning be indicated.
RESPONSE: Thank you. We have reviewed the manuscript and realized that the acronym multiple sclerosis (MS) was not defined at first mention. We have corrected this and now the first mention of all acronyms is fully defined in text. After having introduced the acronyms, we consistently use them throughout the manuscript.
Comment 2. I suggest that the recruitment of study participants be exposed with a flowchart for a better understanding and identification of participation (section 2.2). I consider that this is part of the methodology and not of the results (figure 1).
RESPONSE: The recruitment of study participants is clearly reported in a flow chart in Figure 1. We prefer to leave the recruitment information in the Results section because it provides results pertinent to the feasibility aim of the study.
Comment 3. As well as the informed consent that they have given to their participation, how has it been.
RESPONSE: We have now clarified the consenting process as follows: “All participants read the study information sheet and the consent form and then gave informed consent by activating the “I agree” field on the screen, alternatively they could activate the “I disagree” field and not consent to study participation.”
Comment 4. The approval of the ethics committee that has approved this study must be indicated.
RESPONSE: We have now stated the name of the ethics committee as follows – “The University of Queensland’s Human Research Ethics Committee”.
Comment 5. In the intervention section they should indicate the time of each of the modules.
RESPONSE: Data on the mean number of minutes taken to complete each module are presented in Table 4.
Comment 6. I recommend that you expand the measurements collected and the way in which the collection has been carried out.
RESPONSE: We have now clarified in the first paragraph in the measures section that “All measures entail self-report and were completed online.”
Comment 7. Has the sample size been calculated for this study?
RESPONSE: We discuss the power calculation and required sample size in detail as follows:
“2.5. Data Analysis Approach
A power analysis conducted before study commencement indicated that to detect an effect size of d=0.44, p <0.01 a total sample size of 90 (45 per group) was required. However, the total sample size achieved for this pilot study was 56 (31 intervention and 25 WLC), therefore the analyses are severely underpowered due to the small sample. Because of the low power, a less stringent significance level of p<0.10 was employed corresponding to 90% confidence intervals (CIs) [49,50].”
Comment 8. In the discussion, one should begin with the main results of the study, not with a new approach to the objective, which is repetitive.
RESPONSE: In The Present Study 1.1 sub-section of the Introduction we state that “The primary aim of the present pilot RCT was to explore the effectiveness of the e-READY for MS program.” At the beginning of the Discussion section, we remind the reader of the primary aim as a way of refocussing the reader to the purpose of the study by repeating restating the primary aim in a very short sentence. In our view this is conventional; however, we are happy to take Editorial advice on this and delete the sentence from the Discussion if required by the Editor.
Comment 9. Among the limitations of the study, the sample size must be taken into account. As the authors indicate the risk of type 1 error, concluding that there is a difference when in fact there is not is present in this study.
RESPONSE: We agree that the less than optimal sample size is an important study limitation and we have discussed this in the Limitations sub-section of the Discussion as follows: “the 56 participants enrolled in the study was well below the target sample size of 90 established by a priori power analyses. Hence, the study was markedly underpowered to detect effect sizes of d = 0.40 and is likely to account for the lack of significant intervention effects in multivariate analyses. Third, because the study was underpowered a less stringent significance level was used for the primary analyses, which may increase Type I error rate.”
In addition, we have briefly mentioned at numerous points in the Discussion section the various implications of the smaller than expected sample size and consequently the study being underpowered.
We believe we have fully acknowledged the sample size and risk of Type I error issues. We have also distinguished between results that met the significance level set for this pilot study and those that were marginally significant and can legitimately be deemed “trends”. We emphasis the fact that this is a pilot or proof-of-concept study and hence, it is appropriate in this context to fully explore all data including trends (Kistin, & Silverstein, 2015; Freedland, 2020)
References
Kistin, C., & Silverstein, M. (2015). Pilot studies: a critical but potentially misused component of interventional research. Jama, 314(15), 1561-1562.
Round 2
Reviewer 2 Report
Thank you very much for allowing me to review again the article entitled “A Pilot Randomized Control Trial of an Online Acceptance and Commitment Therapy (ACT) Resilience Training Program for People with Multiple Sclerosis” (sclerosis-2316878), as well as the response from the authors to the recommendations made.
After reviewing the work, I still consider that the recruitment of the patients participating in the study and the controls should form part of the methodology and not of the results.
In relation to comment 6 in which we are informed that all the measurements were self-reported online, I believe that the role played by the fact that they have been collected in this way should be taken into account in the discussion.
Comment 8: I believe that it is not necessary to repeat the objective in the discussion to redirect the reader's attention, although as the authors indicate that it is the editor's decision, I am not going to comment further.
Author Response
REVIEWER 2
Thank you very much for allowing me to review again the article entitled “A Pilot Randomized Control Trial of an Online Acceptance and Commitment Therapy (ACT) Resilience Training Program for People with Multiple Sclerosis” (sclerosis-2316878), as well as the response from the authors to the recommendations made.
RESPONSE: We are very grateful for your commitment to reviewing our manuscript a second time. We appreciate the time and effort you have invested in this task.
After reviewing the work, I still consider that the recruitment of the patients participating in the study and the controls should form part of the methodology and not of the results.
RESPONSE: As requested, we have moved Figure 1 and the corresponding text that reports on the recruitment of patients participating in the study to the “recruitment” sub-section of the Methods section.
In relation to comment 6 in which we are informed that all the measurements were self-reported online, I believe that the role played by the fact that they have been collected in this way should be taken into account in the discussion.
RESPONSE: We have now reported in the Limitations section of the Discussion section that a limitation of the study is that all measures involved self-report online.
Comment 8: I believe that it is not necessary to repeat the objective in the discussion to redirect the reader’s attention, although as the authors indicate that it is the editor’s decision, I am not going to comment further.
RESPONSE: As a courtesy and respect for your second
