How to choose a control condition for speech therapy research

This post is an addendum to a previous post “What is a control group?”, inspired by a recently published new paper (“Control conditions for randomized trials of behavioral interventions in psychiatry: a decision framework” Early View, Lancet Psychiatry, March 2017). Following a brief review of the literature on effect sizes associated with different types of control conditions, a framework for choosing an appropriate control condition in behavioral trials is offered. The types of control conditions discussed are as follows:

  • Active comparator
  • Minimal treatment control
  • Nonspecific factors control
  • No-treatment control
  • Patient choice
  • Pill placebo
  • Specific factors component control
  • Treatment as usual
  • Waitlist control

The considerations for choosing one of these control conditions for testing a behavioral intervention are (1) participant risk; (2) trial phase; and (3) available resources. With respect to participant risk, more active interventions should be provided as the control condition when the risk of withholding treatment (especially when known effective treatments are available) is high. Therefore, when making this decision characteristics of the participant population and characteristics of the available treatments will play a role in the decision making process.

Regarding trial phase, early stage exploratory trials should be concerned with the risk of Type II error; in other words the researcher will want to maximize the chances of finding a benefit of a potentially helpful new intervention. Therefore, a waitlist control group might be appropriate at this stage of the research process given that waitlist controls are associated with large effect sizes in behavioral trials. In the later stages of the research program, the researcher should strive to minimize Type I error; in other words it is important to guard against concluding that an ineffective treatment is helpful. In this case an active comparator would be a logical choice although the sample size would need to be large given that the effect size is likely to be small in this case.

Finally, the resources available to the researchers will influence the choice of control condition. For example, in a late stage trial an active comparator provided by trained and monitored study personnel would be the best choice in most circumstances; however, in this case the provision of the control may be at least as expensive as the provision of the experimental treatment. When sufficient resources are lacking, the cost effective alternative might be to ask the usual community provider to administer treatment as usual although every effort should be made to describe the control intervention in detail.

A very nice graphic is provided (Figure 2) to illustrate the decision framework and can be applied to speech therapy trials. There are a number of interventions that have been in use or are emerging in speech therapy practice with a minimal evidence base. We can consider the choice of appropriate control condition for the assessment of these interventions.

Ultrasound intervention for school aged children with residual speech errors has been examined in quite a number of single subject studies but is now overdue for a randomized control trial. Given that the exploratory work has been completed in single subject trials I would say that we could proceed to a phase 3 RCT. The risk to the participant population is more difficult to conceptualize. You could say that it is low because these children are not at particular risk for poor school outcomes or other harmful sequels of non-intervention and the likelihood of a good speech outcome will not change much after the age of nine. The cost of providing an active control will be high because these children are often low priority for intervention in the school setting. Therefore, according to Figure 2, a no-treatment control would be appropriate when you make this assumption. On the other hand, you could argue that the participant risk of NOT improving is very high-all the evidence demonstrates that the residual errors do not improve without treatment after this age. If you consider the participant risk to be higher, especially considering community participation and psychosocial factors, then the appropriate control condition would be something more vigorous: patient choice, an active comparator, a nonspecific factors component control or a specific factors component control. Given the relatively early days of this research, small trials utilizing these control conditions in order might be advisable.

Metaphon as a treatment for four-year-olds with severe phonological delay and associated difficulties with phonological processing has not, to my knowledge, been tested with a large scale RCT. The population would be high risk by definition due to the likelihood of experiencing delays in the acquisition of literacy skills if the speech delay is not resolved prior to school entry. Effective treatment options are known to exist. Therefore, the appropriate control condition would be an active comparator-in other words, another treatment that is known to be effective with this population. Another option would be a specific factors component control that examines the efficacy of specific components of the Metaphon approach. Therefore, the meaningful minimal pairs procedure could be compared directly to the full metaphon approach with speech and phonological processing skills as the outcome variables. Similar trials have been conducted by Anne Hesketh and in my own lab (although not involving Metaphon specifically).

PROMPT has still not been tested in good quality single subject or parallel groups research. If a Phase 2 trial were planned for three-year-olds with suspected apraxia of speech, treatment as usual would be the appropriate control condition according to Figure 2. The speech condition is too severe to ethically withhold treatment and the research program is not advanced enough for a specific factors components control although this would be the next step.

Finally, an RCT of the effectiveness of Speech Buddies to stimulate /s/ in 3-year-olds with speech delay could be implemented. In this case, the participant group would low risk due to the likelihood of spontaneous resolution of the speech delay. Given a phase 2 trial, either no treatment or waitlist control could be implemented.

The authors of this framework conclude by recommending that researchers justify their choice of control condition in every trial protocol. They further recommend that waitlist controls are only acceptable when it is the only ethical choice and state that “no behavioral treatment should be included in treatment guidelines if it is only supported by trials using a waitlist control group or meta-analytic evidence driven by such trials.” To me, this is eminently sensible advice for speech and language research as well.

And this I believe concludes my trilogy of posts on the control group!

Further Reading

What is a control group? Developmental Phonological Disorders blog post, February 5, 2017

Using effect sizes to choose a speech therapy approach, Developmental Phonological Disorders blog post, January 31, 2017

Gold, S. M., Enck, P., Hasselmann, H., Friede, T., Hegerl, U., Mohr, D. C., & Otte, C. Control conditions for randomised trials of behavioural interventions in psychiatry: a decision framework. The Lancet Psychiatry. doi:10.1016/S2215-0366(17)30153-0

Hesketh, A., Dima, E., & Nelson, V. (2007). Teaching phoneme awareness to pre-literate children with speech disorder: a randomized controlled trial. International Journal of Language and Communication Disorders, 42(3), 251-271.

Rvachew, S., & Brosseau-Lapré, F. (2015). A Randomized Trial of 12-Week Interventions for the Treatment of Developmental Phonological Disorder in Francophone Children. American Journal of Speech-Language Pathology, 24(4), 637-658. doi:10.1044/2015_AJSLP-14-0056

What is a control group?

I have a feeling that my blog might become less popular in the next little while because you may notice an emerging theme on research design and away from speech therapy procedures specifically! But it is important to know how to identify evidence based procedures and to do that requires knowledge of research design and it has come to my attention, as part of the process of publishing two randomized control trials (RCTs) this past year, that there are a lot of misperceptions about what an RCT is in the SLP and education communities, among both clinicians and researchers. Therefore, I am happy to draw your attention to this terrific blog by Edzard Ernst, and in particular to an especially useful post “How to differentiate good from bad research”. The writer points out that a proper treatment of this topic “must inevitably have the size of a book” because each of the indicators that he provides “is far too short to make real sense.” So I have taken it upon myself in this blog to expand upon one of his indicators of good research – one that I know causes some confusion, specifically:

  • Use of a placebo in the control group where possible.

Recently the reviewers (and editor) of one of my studies was convinced that my design was not an RCT because the children in both groups received an intervention. In the absence of a “no-treatment control” they said, the study could not be an RCT! I was mystified about the source of this strange idea until I read Ernst’s blog and realized that many people, recalling their research courses from university, must be mistaking “placebo control” for “no-treatment control.” However, a placebo control condition is not at all like the absence of treatment. Consider the classic example of a placebo control: in a drug trial, the patients randomized to the treatment arm will visit the nurse who hands him or her a white paper cup holding 2 pink pills containing active ingredient X and some other ingredients that do not impact the patient’s disease, i.e., inactive ingredients; the patients randomized to the control arm will also visit the nurse who hands him or her a white paper cup holding 2 pink pills containing only the inactive ingredients. In other words, the experiment is designed so that all patients are “treated” exactly the same except that only patients randomized to treatment receive (unknowingly) the active ingredient. Therefore, all changes in patient behavior that are due to those aspects of the treatment that are not the active treatment (visiting the nice nurse, expecting the pills to make a difference etc.) are equalized across arms of the study. These are called the “common factors” or “nonspecific factors”.

In the case of a behavioral treatment it is important to equalize the common factors across all arms of the study. Therefore in my own studies I deliberately avoid “no treatment” controls. In my very first RCT (Rvachew, 1994) for example the treatment conditions in the two arms of the study were as follows;

  • Experimental: 10 minutes of listening to sheet vs Xsheet recordings and judging correct vs incorrect “sheet” items (active ingredient) in a computer game format followed by 20 minutes of traditional “sh” articulation therapy, provided by a person blind to the computer game target.
  • Control: 10 minutes of listening to Pete vs meat recordings and judging correct vs incorrect “Pete” items in a computer game format followed by 20 minutes of traditional “sh” articulation therapy, provided by a person blind to the computer game target.

It can be seen that the study was designed to ensure that all participants experienced exactly the same treatment except for the active ingredient that was reserved for children who were randomly assigned to the experimental treatment arm, specifically exposure to the experience of listening to and making perceptual judgments about a variety of correct and incorrect versions of words beginning with “sh” or distorted versions of “sh”-the sound that the children misarticulated. Subsequently I have conducted all my randomized control studies in a similar manner. But, as I said earlier, I run across readers who vociferously assert that the studies are not RCTs because an RCT requires a “no treatment” control. In fact, a “no treatment” control is a very poor control indeed as argued in this blog that explains why the frequently used “wait list control group” is inappropriate. For example, a recent trial on the treatment of tinnitus claimed that a wait list control had merit because “While this comparison condition does not control for all potential placebo effects (e.g., positive expectation, therapeutic contact, the desire to please therapists), the wait-list control does account for the natural passing of time and spontaneous remission.” In fact, it is impossible to control for common factors when using a wait list control and it is unlikely that patients are actually “just waiting” when you randomize them to the “wait list control” condition; therefore Hesser et al.’s defense of the wait list control is  optimistic although their effort to establish how much change you get in this condition is worthwhile.

We had experience with a “wait list” comparison condition in a recent trial (Rvachew & Brosseau-Lapré, 2015). Most of the children were randomly assigned to one of four different treatment conditions, matched on all factors except the specific active ingredients of interest. However, we also had a nonexperimental wait list comparison group* to estimate change for children outside of the trial. We found that parents were savvy about maximizing the treatment that their children could receive in any given year. Our trial lasted six weeks, the public health system entitled them to six weeks of treatment and their private insurance entitled them to six to 12 weeks of therapy depending on the plan. Parents would agree to enrolled their child in the trial with randomization to a treatment arm if their child was waiting for the public service, OR they would agree to be assessed in the “wait list” arm if their child was currently enrolled in the public service. They would use their private insurance when all other options had been exhausted. Therefore the children in the “wait list” arm were actually being treated. Interestingly, we found that the parents expected their children to obtain better results from the public service because it was provided by a “real” SLP rather than the student SLPs who provided our experimental treatments even though the public service was considerably less intense! (As an aside, we were not surprised to find that the reverse was true). Similarly, as I have mentioned in previous blogs, Yoder et al. (2005) found that the children in their “no treatment” control accessed more treatment from other sources than did the children in their treatment arm. And parents randomized to the “watchful waiting” arm of the Glogowska et al. (2000) trial sometimes dropped out because parents will do what they must to meet their child’s needs.

In closing, a randomized control trial is simply a study in which participants are randomly assigned to an experimental treatment and a control condition (even in a cross-over design, in which all participants experience all conditions, as in Rvachew et al., in press). The nature of the control should be determined after careful thought about the factors that you are attempting to control, which can be many – placebo, Hawthorne, fatigue, practice, history, maturation and so on. These will vary from trial to trial obviously. Placebo control does not mean “no treatment” but rather, a treatment that excludes everything except the “active ingredient” that is the subject of your trial. As an SLP, when you are reading about studies that test the efficacy of a treatment, you need to pay attention to what happens to the control group as well as the treatment group. The trick is to think in every case – what is the active ingredient that explains the effect seen in the treatment group? what else might account for the effects seen in the treatment arm of this study? If I implement this treatment in my own practice, how likely am I to get a better result compared to the treatment that my caseload is currently receiving?

* A colleague sent me a paper (Mercer et al., 2007) in which a large number of researchers advocating for the acceptance of a broader array of research designs in order to focus more attention on external validity and translational research, got together to discuss the merits of various designs. During the symposium it arose that there was disagreement about the use of the terms “control” and “comparison” group. I use the terms in accordance with a minority of their attendees, as follows: control group means that the participants were randomly assigned to a group that did not experience the “active ingredient” of the experimental treatment; comparison group means that the participants were not randomly assigned to the group that did not experience the experimental intervention, a group that may or may not have received a treatment. This definition was ultimately not used by the attendees, I don’t know why – somehow they decided on a different definition that didn’t make any sense at all, I invite you to consult p. 141 and see if you can figure it out!

References

Glogowska, M., Roulstone, S., Enderby, P., & Peters, T. (2000). Randomised controlled trial of community based speech and language therapy in preschool children. British Medical Journal, 321, 923-928.

Hesser, H., Weise, C., Rief, W., & Andersson, G. (2011). The effect of waiting: A meta-analysis of wait-list control groups in trials for tinnitus distress. Journal of Psychosomatic Research, 70(4), 378-384. doi:http://dx.doi.org/10.1016/j.jpsychores.2010.12.006

Mercer, S. L., DeVinney, B. J., Fine, L. J., Green, L. W., & Dougherty, D. (2007). Study Designs for Effectiveness and Translation Research: Identifying Trade-offs. American Journal of Preventive Medicine, 33(2), 139-154.e132. doi:http://dx.doi.org/10.1016/j.amepre.2007.04.005

Rvachew, S. (1994). Speech perception training can facilitate sound production learning. Journal of Speech and Hearing Research, 37, 347-357.

Rvachew, S., & Brosseau-Lapré, F. (2015). A randomized trial of twelve week interventions for the treatment of developmental phonological disorder in francophone children. American Journal of Speech-Language Pathology, 24, 637-658. doi:10.1044/2015_AJSLP-14-0056

Rvachew, S., Rees, K., Carolan, E., & Nadig, A. (in press). Improving emergent literacy with school-based shared reading: Paper versus ebooks. International Journal of Child-Computer Interaction. doi:http://dx.doi.org/10.1016/j.ijcci.2017.01.002

Yoder, P. J., Camarata, S., & Gardner, E. (2005). Treatment effects on speech intelligibility and length of utterance in children with specific language and intelligibility impairments. Journal of Early Intervention, 28(1), 34-49.