Single Subject Randomization Design For Clinical Research

Ebbels tweet Intervention ResearchDuring the week April 23 – 29, 2017 Susan Ebbels is curated WeSpeechies on the topic Carrying Out Intervention Research in SLP/SLT Practice. Susan kicked off the week with a link to her excellent paper that discusses the strengths and limitations of various procedures for conducting intervention research in the clinical setting. As we would expect, a parallel groups randomized control design was deemed to provide the best level of experimental control. Many ways of studying treatment related change within individual clients, with increasing degrees of control were also discussed. However, all of the ‘within participant’ methods described were vulnerable to confounding by threats to internal validity such history, selection, practice, fatigue, maturation or placebo effects to varying degrees.

One design was missing from the list because it is only just now appearing in the speech-language pathology literature, specifically the Single Subject Randomization Design. The design (actually a group of designs in which treatment sessions are randomly allocated to treatment conditions) provides the superior internal validity of the parallel groups randomized control trial by controlling for extraneous confounds through randomization. As an added benefit the results of a single subject randomization design can be submitted to a statistical analysis, so that clear conclusions can be drawn about the efficacy of the experimental intervention. At the same time, the design can be feasibly implemented in the clinical setting and is perfect for answering the kinds of questions that come up in daily clinical practice. For example, randomized control trials have shown than speech perception training is an effective adjunct to speech articulation therapy on average when applied to groups of children but you may want to know if it is a necessary addition to your therapy program for a speciRomeiser Logan Levels of Evidence SCRfic child.

Furthermore,  randomized single subject experiments are now acceptable as a high level of research evidence by the Oxford Centre for Evidence Based Medicine. An evidence hierarchy has been created for rating single subject trials, putting the randomized single subject experiments at the top of the evidence hierarchy as shown in the following table, taken from Romeiser Logan et al. 2008.

 

Tanya Matthews and I have written a tutorial showing exactly how to implement and interpret two versions of the Single Subject Randomization Design, a phase design and an alternation design. The accepted manuscript is available but behind a paywall at the Journal of Communication Disorders. In another post I will provide a mini-tutorial showing how the alternation design could be used to answer a clinical question about a single client.

Further Reading

Ebbels, Susan H. 2017. ‘Intervention research: Appraising study designs, interpreting findings and creating research in clinical practice’, International Journal of Speech-Language Pathology: 1-14.

Kratochwill, Thomas R., and Joel R. Levin. 2010. ‘Enhancing the scientific credibility of single-case intervention research: Randomization to the rescue’, Psychological Methods, 15: 124-44.

Romeiser Logan, L., R. Hickman, R.R. Harris, S.R. Harris, and C. Heriza. 2008. ‘Single-subject research design: recommendations for levels of evidence and quality rating’, Developmental Medicine and Child Neuroloogy, 50: 99-103.

Rvachew, S. 1988. ‘Application of single subject randomization designs to communicative disorders research’, Human Communication Canada (now Canadian Journal of Speech-Language Pathology and Audiology), 12: 7-13. [open access]

Rvachew, S. 1994. ‘Speech perception training can facilitate sound production learning.’, Journal of Speech and Hearing Research, 37: 347-57.

Rvachew, Susan, and Tanya Matthews. in press. ‘Demonstrating Treatment Efficacy using the Single Subject Randomization Design: A Tutorial and Demonstration’, Journal of Communication Disorders.

 

Advertisements

How to choose a control condition for speech therapy research

This post is an addendum to a previous post “What is a control group?”, inspired by a recently published new paper (“Control conditions for randomized trials of behavioral interventions in psychiatry: a decision framework” Early View, Lancet Psychiatry, March 2017). Following a brief review of the literature on effect sizes associated with different types of control conditions, a framework for choosing an appropriate control condition in behavioral trials is offered. The types of control conditions discussed are as follows:

  • Active comparator
  • Minimal treatment control
  • Nonspecific factors control
  • No-treatment control
  • Patient choice
  • Pill placebo
  • Specific factors component control
  • Treatment as usual
  • Waitlist control

The considerations for choosing one of these control conditions for testing a behavioral intervention are (1) participant risk; (2) trial phase; and (3) available resources. With respect to participant risk, more active interventions should be provided as the control condition when the risk of withholding treatment (especially when known effective treatments are available) is high. Therefore, when making this decision characteristics of the participant population and characteristics of the available treatments will play a role in the decision making process.

Regarding trial phase, early stage exploratory trials should be concerned with the risk of Type II error; in other words the researcher will want to maximize the chances of finding a benefit of a potentially helpful new intervention. Therefore, a waitlist control group might be appropriate at this stage of the research process given that waitlist controls are associated with large effect sizes in behavioral trials. In the later stages of the research program, the researcher should strive to minimize Type I error; in other words it is important to guard against concluding that an ineffective treatment is helpful. In this case an active comparator would be a logical choice although the sample size would need to be large given that the effect size is likely to be small in this case.

Finally, the resources available to the researchers will influence the choice of control condition. For example, in a late stage trial an active comparator provided by trained and monitored study personnel would be the best choice in most circumstances; however, in this case the provision of the control may be at least as expensive as the provision of the experimental treatment. When sufficient resources are lacking, the cost effective alternative might be to ask the usual community provider to administer treatment as usual although every effort should be made to describe the control intervention in detail.

A very nice graphic is provided (Figure 2) to illustrate the decision framework and can be applied to speech therapy trials. There are a number of interventions that have been in use or are emerging in speech therapy practice with a minimal evidence base. We can consider the choice of appropriate control condition for the assessment of these interventions.

Ultrasound intervention for school aged children with residual speech errors has been examined in quite a number of single subject studies but is now overdue for a randomized control trial. Given that the exploratory work has been completed in single subject trials I would say that we could proceed to a phase 3 RCT. The risk to the participant population is more difficult to conceptualize. You could say that it is low because these children are not at particular risk for poor school outcomes or other harmful sequels of non-intervention and the likelihood of a good speech outcome will not change much after the age of nine. The cost of providing an active control will be high because these children are often low priority for intervention in the school setting. Therefore, according to Figure 2, a no-treatment control would be appropriate when you make this assumption. On the other hand, you could argue that the participant risk of NOT improving is very high-all the evidence demonstrates that the residual errors do not improve without treatment after this age. If you consider the participant risk to be higher, especially considering community participation and psychosocial factors, then the appropriate control condition would be something more vigorous: patient choice, an active comparator, a nonspecific factors component control or a specific factors component control. Given the relatively early days of this research, small trials utilizing these control conditions in order might be advisable.

Metaphon as a treatment for four-year-olds with severe phonological delay and associated difficulties with phonological processing has not, to my knowledge, been tested with a large scale RCT. The population would be high risk by definition due to the likelihood of experiencing delays in the acquisition of literacy skills if the speech delay is not resolved prior to school entry. Effective treatment options are known to exist. Therefore, the appropriate control condition would be an active comparator-in other words, another treatment that is known to be effective with this population. Another option would be a specific factors component control that examines the efficacy of specific components of the Metaphon approach. Therefore, the meaningful minimal pairs procedure could be compared directly to the full metaphon approach with speech and phonological processing skills as the outcome variables. Similar trials have been conducted by Anne Hesketh and in my own lab (although not involving Metaphon specifically).

PROMPT has still not been tested in good quality single subject or parallel groups research. If a Phase 2 trial were planned for three-year-olds with suspected apraxia of speech, treatment as usual would be the appropriate control condition according to Figure 2. The speech condition is too severe to ethically withhold treatment and the research program is not advanced enough for a specific factors components control although this would be the next step.

Finally, an RCT of the effectiveness of Speech Buddies to stimulate /s/ in 3-year-olds with speech delay could be implemented. In this case, the participant group would low risk due to the likelihood of spontaneous resolution of the speech delay. Given a phase 2 trial, either no treatment or waitlist control could be implemented.

The authors of this framework conclude by recommending that researchers justify their choice of control condition in every trial protocol. They further recommend that waitlist controls are only acceptable when it is the only ethical choice and state that “no behavioral treatment should be included in treatment guidelines if it is only supported by trials using a waitlist control group or meta-analytic evidence driven by such trials.” To me, this is eminently sensible advice for speech and language research as well.

And this I believe concludes my trilogy of posts on the control group!

Further Reading

What is a control group? Developmental Phonological Disorders blog post, February 5, 2017

Using effect sizes to choose a speech therapy approach, Developmental Phonological Disorders blog post, January 31, 2017

Gold, S. M., Enck, P., Hasselmann, H., Friede, T., Hegerl, U., Mohr, D. C., & Otte, C. Control conditions for randomised trials of behavioural interventions in psychiatry: a decision framework. The Lancet Psychiatry. doi:10.1016/S2215-0366(17)30153-0

Hesketh, A., Dima, E., & Nelson, V. (2007). Teaching phoneme awareness to pre-literate children with speech disorder: a randomized controlled trial. International Journal of Language and Communication Disorders, 42(3), 251-271.

Rvachew, S., & Brosseau-Lapré, F. (2015). A Randomized Trial of 12-Week Interventions for the Treatment of Developmental Phonological Disorder in Francophone Children. American Journal of Speech-Language Pathology, 24(4), 637-658. doi:10.1044/2015_AJSLP-14-0056

Who to refer for speech therapy?

Morgan et al. have recently published a very important paper: Who to refer for speech therapy at 4 years of age versus who to “watch and wait”? This longitudinal study reports speech outcomes at age 7 years for children who received GFTA and DEAP assessments at age 4. The children were recruited from an Australian community cohort study (the Early Language in Victoria study) that recruited almost 2000 infants between 7 and 10 months of age for long-term follow-up.

The data reported in Morgan et al. is interesting by itself, as follows:

  1. Eleven percent of 1496 children tested at age 4 had speech errors qualifying the child for repeat assessment at age 7 years (the 11% finding interested me because we settled on 11% as the best estimate for prevalence of developmental phonological disorders at school entry in the review that we reported in DPD).
  2. At age 7 years, approximately 40% of the children who had speech errors at age 4 still had speech errors.
  3. Children at age 4 who had speech delay (typical speech errors; 60% of the sample) were most likely to show resolution of the speech problem. Specifically 70% of these children were classed as “resolved” and 30% as “persistent” at age 7 years.
  4. Children at age 4 who had a speech disorder (atypical speech errors; 40% of the sample) were less likely to show resolution of the speech problem. Specifically, 40% of these children were classed as resolved and 60% as persistent.
  5. No other variables in the study predicted speech outcome but neither did these variables predict “delay” versus “disorder” group membership (sex, SES, family history, language skills, nonverbal IQ).
  6. Apparently, reliable data on receipt of SLP services and outcomes was not available but there was some suggestion that children with “speech delay” who received therapy were more likely to resolve than children with “speech disorder” who received therapy.

Therefore, in this paper that is published in a journal for pediatricians the conclusion was “our data call into question whether the “watch and wait” approach should be universally applied to all preschool children. Rather these data suggest an efficient model may guide children with disorder at age 4 years to be fast-tracked for speech therapy…”.

The data provided in this paper are exceptionally important for SLPs and the development of service delivery guidelines but I am a little uncomfortable with the conclusions that were drawn. The first assumption I suppose is that doctors are not referring any 4 year olds so if we could get them to refer some that would help. The second assumption seems to be that the reason we refer 4 year olds with speech errors to speech therapy is to eliminate the speech errors. This is only partially true. More importantly, we have the goal of preventing the sequels that are known to be associated with delayed/disordered speech at school entry. These are mostly in the area of literacy but also in the psychosocial domains. It is clear that children who show early speech delays are at-risk for persistent literacy difficulties regardless of whether the speech problem resolves before at age 7. The important age cut-off is resolution of the speech problem before school entry. The risk for literacy difficulties is predicted by direct measures of phonological processing and not by an examination of speech error types. Certain speech error types are associated with phonological processing difficulties and a heightened risk for literacy problems but they are poor predictors of this risk. I will come back to this point with some case histories below.

The second problem that I have with the conclusions is that they are delivered to pediatricians who are in no way qualified to differentiate typical from atypical speech errors. In fact, SLPs themselves find this hard enough to do reliably. The difference between speech delay and speech disorder is both qualitative and quantitative– in other words the dividing line between delay and disorder is a very large grey area. Family doctors should not attempt to make this differentiation. In the paper, Morgan et al. do point out that the real issue is intelligibility. When the child is unintelligible past the age of 3 or 4, the physician should refer to a SLP who should determine the best course of action. In our review of the literature for SAC, Susan Raffat and I proposed wait times recommendations for children who are “ producing so many speech sound errors that speech intelligibility falls below expectations given the speaker’s age and experience with the language being spoken.” All children in the 4 to 6 year age group were considered by us to be high priority for a rapid assessment by an SLP. Any child with speech intelligibility problems who is expected to start school in the year of referral and/or presenting with phonological processing difficulties would be considered a high priority for immediate intervention.

Now to some case studies that I draw directly from our DPD text (Rvachew and Brosseau-Lapré), showing only portions here to make a point about speech delay, speech disorder and literacy outcomes. The first example is a clear case of speech disorder (data shown from age 7;4 assessment, right).

Complete information is provided in DPD, showing that two years earlier this child also presented with a severe speech disorder and severely delayed phonolCase Study DPD 8-4.JPGogical processing skills. His error types were atypical and inconsistent throughout the longitudinal follow-up period, despite much speech therapy targeting motor aspects of his speech. At age 7 his nonword reading skills were slightly below normal limits and 14 points below his receptive vocabulary scores. We can predict that he will struggle with the acquisition of reading and spelling in addition to continuing to have highly unintelligible speech for some time. Interestingly, his mother reported that his speech accuracy finally started to improve after a systematic phonics program was instituted to help him with his reading in second grade. The outcomes reported at age 7 will not surprise anyone.

The interesting findings for me were associated with the children with milder speech delay. The second child shown here (age 6;9 assessment, left) had a mild speech delay at age 4 but a severe delay in phonological processing skills that was, fortunately for him, treated appropriately by the SLP program in the local children’s hospital. At age 7 his speech delay is more-or-less resolved. His nonword reading skills are borderline normal but there is a 28 point gap between his nonword reading score and his receptive vocabulary scoreCase Study DPD 8-1. I think that this child is essentially dyslexic. He is coping well because he is exceptionally bright with excellent inputs from his family and the community service providers. That does not mean that the outcome would have been as good without those services however. The 30% of kids with speech delay who don’t resolve by themselves? Someone has to watch out for those kids, especially since they are numerically the larger group of kids. As an SLP, I make it my job to worry about them.