Table 3 Illustrative multi-level partner feedback to inform revisions to a pragmatic measure and measure administration protocol for community opioid treatment programs
Pragmatic measure | |||||
Question Tested | Opioid Treatment Program Leader (001) | Clinical Supervisor (002) | Front-line Counselor (003) | Strategic Decision Made | Preliminary Item Revision |
1. Have you use opioids, sedatives, or cocaine in the past 3 months? a. Yes b. No 1a. If yes, how often have you used them? a. 5–6 times per week b. 3–4 times per week c. 2 times per week d. 1 time per week e. 1–3 times a month f. Less often | “I wonder about the word sedatives. I think people, our population, might need a greater explanation of what counts as a sedative. So like, do we mean benzos, tranquilizers, barbiturates, alcohol? You know, how broad are we going?” | “I’m not sure really what we’re trying to capture, if we’re trying to capture use in the last 12 months, 3, or 6. To me, if you’re trying to capture if somebody needs detox or if somebody needs a certain level of care, the 3, 6, and one year interval because 6 months in the addiction world, if you have not used then you’re in remission.” | “It is a relevant question and it’s an opioid treatment program, so obviously that’s the first question you want to ask. I ask that even during my sessions, so my sessions group, doing an intake, with any interaction, the first question I ask is ‘have you used in the past 30 days?” | -Focused question to only opioid use -Simplified response options to daily, weekly, or monthly use -Added skip logic on survey to display frequency question only if respondent selects “yes” to use | Have you used opioids in the past 3 months? a. Yes b. No If yes, how often have you used opioids? a. Daily b. Weekly c. Monthly |
2. Have you ever experienced problems as a result of the time you have spent to get opioids? a. Yes b. No | “That sounds a little confusing to me. I know what they mean, but the problems we’re trying to assess are specifically a result of the time spent trying to track down the substance. I just wonder about the wording of it.” | “Personally to me, my clients’ comprehension is way lower than that with that question, so rewording it and it basically: did you get in trouble.” | “That’s okay. Not bad.” | - Simplify question wording - Reconcile discrepant partner feedback (e.g., confusing question versus no revisions needed) | Have you gotten in trouble because of your opioid use? |
Measure administration protocol | |||||
Measure Administration Question | Opioid Treatment Program Leader (001) | Clinical Supervisor (002) | Front-line Counselor (003) | Strategic Decision Made | |
What are your thoughts about the measure/question content? • Any questions that should be added? Anything that should be deleted? | “I feel like the old sheets that we had them do in that old opioid cessation group were like ‘Did you do anything this week to try to use less or try to avoid using opioids?’ ‘What was it?’ ‘Did it work?’” | “So yeah, you really got to figure out what you’re trying to say in the first question [question 1] for the second one [question 1a] to make sense, but I would get rid of [response option] F [less often].” | “Maybe one question that I might want to add is ‘are you using both? Are you using opiates and cocaine at the same time?” | - Review existing measures at treatment programs to identify overlapping content and reduce measure length - Combine response options for measure simplification - Add questions about polysubstance use - Update measure administration protocol with a section on training - Pilot test revisions in additional cognitive interviews | |
What are your thoughts about measure administration procedures? • For example, how frequently should patients complete a measure like this? | “It depends a bit on the periodicity, I think. I’m not sure that all of these questions need to be administered weekly, for example. Or maybe having it set up in such a way that easily flows like ‘If no, skip.’ | “I don’t see it as a pre and post that all… I don’t really see it in any other way other than an intake.” | “I would like to administer something like this during their quarterly assessments. So, every three months we do an assessment.” | - Add skip logic so only relevant items are administered - Revise measure administration to occur at each completed counseling session, at program intake, and during required quarterly assessments - Pilot test revised measure administration plan in additional cognitive interviews | |
How do you feel a measure like this would fit into a typical group counseling session at your treatment program? | “The other thing I wonder about, in a group setting I wonder if patients would feel uncomfortable disclosing the answers…And I just wonder, in a group setting, how would we navigate all that data entry?” | “In a group? No, absolutely not. This is very individualistic, very personal. This is very shameful, this is– no. Absolutely not.” | “So, their answers will help you know what topics to treat in-group. As a group facilitator, because not all patients in your group are your patients, right? If you give this assessment in group to get the response, you are able to communicate with their individual clinicians. Maybe there is something that they’re telling their individual clinicians that you’re not aware of as a facilitator.” | - Revise measure administration plan to administer to patients prior to their counseling sessions - Revise measurement-based care protocol for use in individual instead of group counseling sessions given concerns regarding confidentiality and data entry burden - Pilot test revised measurement-based care protocol in additional cognitive interviews | |