Table 1 4-stage cognitive interviewing model and example verbal probes for implementation studies

Comprehension

Clarify understanding of constructs, define terms, paraphrase questions

-Does that make sense?

-What does “evidence-based practice” mean to you?

-What do you think is the difference between “sustainability” and “maintenance”?

-How would you ask this question in your own words?

-Are there other words a provider may use to describe “evidence-based practice”?

-How would you explain “organizational climate” to a clinician?

Memory Retrieval

Examine the utility of memory cues provided (e.g., time anchor, examples)

-Do you need more response options to help you answer the question?

-Is the “6-month” timeframe used in this question useful?

-Would having more examples of specific “evidence-based practices” in the instructions be helpful?

Judgement

Assess confidence judgements

-Are there other questions you would include on a clinician knowledge test about evidence-based practice?

-Do you think other clinicians would answer this question in the same way?

-How sure are you that you attended 5 days of training on evidence-based practice?

Response

Elaborate upon the response process

-Why did you choose “Agree” instead of “Strongly Agree”?

-What types of trainings were you thinking about when you said you went to “2 days of training on evidence-based practice”?

-Are you including time you spend in clinical supervision in your answer about evidence-based practice training?

Implementation Research

Collect partner feedback on administration procedures

-How often do you think your [staff/clinicians/patients] should complete this measure?

-Do you think it would be better for [staff/clinicians/patients] to complete this measure in private or in a group setting?

-Are there other supports (e.g., internet access, survey platform, additional training) your site needs to complete this measure?