Association between Psychological Integration and Permanent Supportive Housing: An Exploratory Study with a Focus on Ethnicity

Round 1

Reviewer 1 Report

Generally, the manuscript is properly laid out. The authors presented a coherent piece of work, where different sections of the manuscript are interlinked.

However, there are a few areas in the manuscript that require the author(s) attention in order to improve on the quality of the paper.

1. Under procedure, Page 4 the authors need to provide justification for gifting the participants. This practice raises some ethical concerns in this paper.
2. There is no evidence of any ethical clearance or permission received from the organisation. The authors mentioned obtaining consent of the participants, even though evidence of permission from the participants may not be required but the authors have to provide evidence of any permission received from organisations. For example, the Reference number of such document will be sufficient without attaching the whole letter/document.
3. This paper is not foregrounded by any relevant theoretical orientation. Situating this present study within a relevant theoretical framework will add more value to this paper.
4. Even though full hierarchical mixed model of the fixed effects analysis was useful in providing the required data analysis in terms of fitness and purpose, a streamlined analytical tool would have been more useful.
5. The data interpretation is clear but quite a number of results mentioned by the authors in the course of the analysis are not presented anywhere in this manuscript. It would therefore be more acceptable for the authors to present all results before they are discussed.
6. This manuscript has no concluding and recombination section. In this case, the authors should add a conclusion and recommendation section as separate sections of this manuscript.
7. Citation of many authors should be done in a chronological order. From the earliest to the latest or vice-versa

Author Response

Responses to First Reviewer’s Comments

 

We would like to begin by thanking you for your insightful comments. We believe we have addressed all of them and in doing so improved our manuscript substantially.

 

1. Under procedure, Page 4 the authors need to provide justification for gifting the participants. This practice raises some ethical concerns in this paper.
   • We have added text in Section 2.3 that provides justification for gifting participants. The IRB was responsible for assessing the impact of incentives on study participants and determined that the amount of the gift cards was not considered coercive. Furthermore, it is good research practice to thank participants for their time and effort as long as the monetary gift is modest in amount.

 

2. There is no evidence of any ethical clearance or permission received from the organisation. The authors mentioned obtaining consent of the participants, even though evidence of permission from the participants may not be required but the authors have to provide evidence of any permission received from organisations. For example, the Reference number of such document will be sufficient without attaching the whole letter/document.
   • In section 2.3, we have added the IRB ID number and noted that the IRB also required letters of support from all collaborating agencies.

 

3. This paper is not foregrounded by any relevant theoretical orientation. Situating this present study within a relevant theoretical framework will add more value to this paper.
   • We have grounded our focus on psychological integration within two theoretical frameworks, including the Sense of Community Theory and the Sense of Belonging Theory. See Introduction for additional text.

 

4. Even though full hierarchical mixed model of the fixed effects analysis was useful in providing the required data analysis in terms of fitness and purpose, a streamlined analytical tool would have been more useful.
   • The decision to use hierarchical mixed modeling was very intentional. Our data consisted of nested, time-invariant variables, where time was nested in the individuals participating in the study, and individuals were nested within agencies. Our analysis used time-of-survey as a clustering unit, but also used the repeated measures of individuals over time. As a result, we had multiple measures of a single individual, violating the assumption of observation independence which most regression models require but hierarchical mixed models take this into consideration. Failing to consider this dependency structure of our nested data would have resulted in erroneous conclusions in hypothesis testing and inference (Bickel 2007). Bickel, R. (2007). Multilevel analysis for applied research: It's just regression! Guilford Press.

 

5. The data interpretation is clear but quite a number of results mentioned by the authors in the course of the analysis are not presented anywhere in this manuscript. It would therefore be more acceptable for the authors to present all results before they are discussed.
   • We are not certain about what missing results the reviewer is referencing. We have reviewed the manuscript for mention of analyses which were not reported but were unable to identify any. However, potentially confusing language in paragraph 3 of the Results section was modified to clarify that only one hierarchical mixed model (“full model”) analysis was conducted. We are wondering if perhaps the reviewer is talking about the post-hoc analysis referred to in the discussion section regarding county origin (we have highlighted this paragraph in yellow font in the manuscript). With it being a post-hoc analysis we believe that the appropriate place for this text is the discussion, however, we are willing to have it moved to the results section based on the reviewer and editor’s preference.

 

6. This manuscript has no concluding and recombination section. In this case, the authors should add a conclusion and recommendation section as separate sections of this manuscript.

• We have reformatted and added a conclusions/recommendations section to our discussion.

 

7. Citation of many authors should be done in a chronological order. From the earliest to the latest or vice-versa

• We are unclear what this comment means. Our citations are consistent with APA style which instructs that citation of multiple sources be listed in alphabetical order; any observed errors in APA formatting have been corrected.

 

 

Author Response File: Author Response.docx

Reviewer 2 Report

This is an innovative project that seeks to explore improvements over time in scores on a subscale of community integration (psychological integration) among people in a permanent supportive housing program (PSH) and factors that predict better psychological integration.  The authors also compare Hispanic versus non-Hispanic ethnicity.  While the paper has the potential to make a significant contribution to the literature, there are some shortcomings that must be addressed in the paper.

The authors do not do an adequate job of explaining at the outset why they are focusing on psychological integration and not the other aspects of community integration.  Toward the end, they argue that inconsistencies in results regarding community integration may be due to measurement problems in the Community Integration Scale (CIS).  However, it is not clear that any of the subscales have established validity, so this remains a question.  It also appears as if all three subscales were measured.  The authors state that "psychological integration has been identified as the defining feature of community life" is supported by one reference that is from the 1970s.  Stronger justification is needed.

Also in the introduction, the authors state that PSH follow a "housing first" model.  The authors are encouraged to read some recent scholarship on the variability of PSH models.  The office of Housing and Urban Development has promoted PSH but without initially giving agencies much guidance on what a housing first model looks like.  In fact, researchers have found considerable variability in PSH models and relatively low adherence to housing first principles.  

Dickson-Gomez, J. et al.  2017.  Identifying variability in permanent supportive housing:  A comparative effectiveness approach to measuring health outcomes.  American Journal of Orthopsychiatry 87(4)

Gilmer, T. P., Stefancic, A., Katz, M. L., Sklar, M., Tsemberis, S., &
Palinkas, L. A. (2014). Fidelity to the housing first model and effectiveness of permanent supported housing programs in California. Psychiatric Services, 65, 1311–1317. http://dx.doi.org/10.1176/appi.ps
.201300447
Gilmer, T. P., Stefancic, A., Sklar, M., & Tsemberis, S. (2013). Development and validation of a Housing First fidelity survey. Psychiatric
Services, 64, 911–914. http://dx.doi.org/10.1176/appi.ps.201200500
Watson, D. P., Orwat, J., Wagner, D. E., Shuman, V., & Tolliver, R.
(2013). The Housing First Model (HFM) fidelity index: Designing and
testing a tool for measuring integrity of housing programs that serve
active substance users. Substance Abuse Treatment, Prevention, and
Policy, 8, 16. http://dx.doi.org/10.1186/1747-597X-8-16
Watson, D. P., Wagner, D. E., & Rivers, M. (2013). Understanding the
critical ingredients for facilitating consumer change in housing first
programming: A case study approach. The Journal of Behavioral Health
Services & Research, 40, 169 –179. http://dx.doi.org/10.1007/s11414-
012-9312-0

Related, the authors mention that they used the PSH fidelity measure but do not describe what this tool measured, or who assessed the fidelity of the program.  The authors report outcomes at baseline, 6-months and at "discharge" from the program.  This would imply that they are using quite a different PSH model than that which is commonly considered a housing first model, or even an essential component of most PSH models.  The permanent in permanent supportive housing means at the very least, long-term housing, not discharge after 14 months.

The eligibility criteria also seem to differ from what is usually used to determine eligibility for PSH by HUD.  HUD defines "chronic homelessness" as 4 or more homeless episodes in the previous 3 years or a period of homelessness lasting longer than one year while the authors say that their criteria were having "considerable housing instability" over the past 3 years.  Similarly, the authors stated that to be eligible, participants had to have a mental illness, but make no mention of HIV or having a substance use disorder which can also be used for eligibility.  

The choice to study ethnicity and not race also needs better justification.  Although the NIH and US census define ethnicity as Hispanic/Latino or not Hispanic/Latino and race with a number of categories, the distinction between race and ethnicity that they use is arbitrary.  There is nothing more or less socially constructed about race or ethnicity and the distinction serves to imply that race is biological while ethnicity is social.  It is understandable, however, that the authors use the same designations as NIH and the US federal government.  They also acknowledge that discrimination does not only happen to Hispanics but is also directed toward other racial/ethnic minorities.  They do not do analysis according to these other categories as participants were able to check more than one category.  It should be possible to recode these; for example, the categories could be White, African American, Native American/Pacific Islander, mixed race or even White, non-White and mixed race depending on the distribution.

The pronouns who should be used to refer to people such as "participants who were housed."  This sentence, on line 438 is also confusing because I thought all participants were housed.  The pronoun that should be used with population.  

 

Author Response

 

Responses to Second Reviewer’s Comments

 

We would like to begin by thanking you for your insightful comments. We believe we have addressed all of them and in doing so improved our manuscript substantially.

 

1. The authors do not do an adequate job of explaining at the outset why they are focusing on psychological integration and not the other aspects of community integration.  Toward the end, they argue that inconsistencies in results regarding community integration may be due to measurement problems in the Community Integration Scale (CIS).  However, it is not clear that any of the subscales have established validity, so this remains a question.  It also appears as if all three subscales were measured.  The authors state that "psychological integration has been identified as the defining feature of community life" is supported by one reference that is from the 1970s.  Stronger justification is needed.

• We have grounded our focus on psychological integration within two theoretical frameworks, including the Sense of Community Theory and the Sense of Belonging Theory. See Introduction for additional text. We have also added a sentence in the discussion, under limitations, that indicates that data on the validity of CIS subscales is non-existent.

 

2. Also in the introduction, the authors state that PSH follow a "housing first" model.  The authors are encouraged to read some recent scholarship on the variability of PSH models.  The office of Housing and Urban Development has promoted PSH but without initially giving agencies much guidance on what a housing first model looks like.  In fact, researchers have found considerable variability in PSH models and relatively low adherence to housing first principles.  

• Thank you for highlighting this oversite. We clarified this sentenced and indicated in our methods section that the agencies involved in this study implemented PSH based on the Housing First model.

 

3. Related, the authors mention that they used the PSH fidelity measure but do not describe what this tool measured, or who assessed the fidelity of the program. 

• We have added information on the fidelity tool in section 2.4.

 

4. The authors report outcomes at baseline, 6-months and at "discharge" from the program.  This would imply that they are using quite a different PSH model than that which is commonly considered a housing first model, or even an essential component of most PSH models.  The permanent in permanent supportive housing means at the very least, long-term housing, not discharge after 14 months.

• We clarified the definition of “discharge” in section 2.3.

 

5. The eligibility criteria also seem to differ from what is usually used to determine eligibility for PSH by HUD.  HUD defines "chronic homelessness" as 4 or more homeless episodes in the previous 3 years or a period of homelessness lasting longer than one year while the authors say that their criteria were having "considerable housing instability" over the past 3 years.  Similarly, the authors stated that to be eligible, participants had to have a mental illness, but make no mention of HIV or having a substance use disorder which can also be used for eligibility.  

• Selection criteria for this program was chronic homelessness as well as a mental health and/or substance use diagnosis (as described in section 2.1). We clarified this in the methods section and throughout the rest of the manuscript.

 

6. The choice to study ethnicity and not race also needs better justification.  Although the NIH and US census define ethnicity as Hispanic/Latino or not Hispanic/Latino and race with a number of categories, the distinction between race and ethnicity that they use is arbitrary.  There is nothing more or less socially constructed about race or ethnicity and the distinction serves to imply that race is biological while ethnicity is social.  It is understandable, however, that the authors use the same designations as NIH and the US federal government.  They also acknowledge that discrimination does not only happen to Hispanics but is also directed toward other racial/ethnic minorities.  They do not do analysis according to these other categories as participants were able to check more than one category.  It should be possible to recode these; for example, the categories could be White, African American, Native American/Pacific Islander, mixed race or even White, non-White and mixed race depending on the distribution.

• Our choice to focus on ethnicity was based on two reasons. First, in the state in which this study was conducted, 49.3% of the individuals that our PSH programs serve is Hispanic. Therefore, it was especially important for our community-based partners to learn, if an association was found between PSH and psychological integration, whether it was equally observed among Hispanics and non-Hispanics. Second, the number of participants who identified with non-white race categories was small. We were concerned that if we included the race category in our model, the small sample sizes in the other minority categories would have resulted in a Type I error. We provided more information in the discussion section regarding our decision to focus on ethnicity vs. race and limitations to this approach.

 

7. The pronouns who should be used to refer to people such as "participants who were housed."  This sentence, on line 438 is also confusing because I thought all participants were housed.  The pronoun that should be used with population. 

• We have made these corrections.

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

The revised manuscript is much improved.  The inclusion of the Sense of Community and Theory of Belonging theories is appreciated.  However, a description of physical and social integration to contrast these with the psychological integration that was chosen for analysis would be helpful.

The description of "discharge" is also appreciated.  However, attrition appears to be high which is surprising given that the researchers attempted to interview everyone at the end of the three year study period.  What explanation is there for the attrition of nearly 60% of the sample?

Author Response

We have defined physical and social integration in the first paragraph of the introduction. 

In the limitations section part of the discussion, we have included three citations that address the challenge with attrition among populations that are homeless. We also explain, that despite extensive efforts to follow-up with participants, research assistants were unable to locate many participants for follow-up interviews. As noted in our original manuscript, an examination of differences, in the demographics of the study population overtime found no significant differences reducing concerns related to selection bias resulting from attrition.