Research and Psychotherapy
Let the Buyer Beware
by Jean Mercer, PhD,
Professor of Psychology
Richard Stockton College
Pomona, New Jersey
When mental health practitioners advertise today, they sometimes assert that research evidence supports their treatments. This is especially common among those whose practices are outside the professional mainstream. But parents seeking treatment for their children need to give careful thought to these claims.
Professionals doing research on the outcomes of psychotherapies often say, “This isn’t rocket science — it’s a lot more complicated than that!” Good research on psychotherapy is quite difficult to plan and to carry out, especially if the people being treated are children. Children are always changing and maturing, so it’s hard to know whether a change for the better was caused by psychotherapy, or whether it would have occurred anyway.
Ideally, we would study psychotherapies with randomized controlled trials. The RCT approach means that we assign people to one treatment or another, or to no treatment at all, by some genuinely chance approach like flipping a coin. The clients do not make the decision, and neither does the therapist. We compare the groups of people after a period of treatment, and if the clients getting a particular treatment have improved by a statistically significant amount over another group, we are justified in saying that the treatment was effective.
In the real world, of course, RCT studies are rare, because people want to choose their own treatments. If they do choose their treatments, we don’t know which factor made them get better— the treatment, the fact of making the choice, or the kinds of people they were to begin with.
When RCT research is impossible, the next best thing is controlled clinical trials. In CCT research, people do choose their own treatment, but they are compared to an other group of people who are getting a different treatment or not being treated at all. The two groups of people need to be matched as closely as possible. There should be equal numbers of males and females in each group, for instance. When children are being studied, it is critically important that children of similar ages be in the two groups. This helps us to know whether improvements were caused by treatment or whether they would occur naturally as the children got older.
It is really not useful to do “before and after” studies on child psychotherapy. These studies compare a child before treatment to the same child after treatment, but there is no way of knowing whether changes were caused by the treatment, the child’s natural development, or just variations in the disorder over time. Nevertheless, some researchers do these studies, and it might be in some cases that they are only a starting point for more adequate research approaches. However, potential clients should interpret them with care.
When researchers report “before and after” studies, it’s especially important to keep an eye on their statistical analysis. If someone says that over 50% of children improved, we need to keep in mind the nearly 50% who did not improve — and may even have gotten worse!
For their children’s sake, parents need to become sophisticated consumers of psychotherapy research. You may never make a more important purchase for your child than you do when you consult a mental health professional. Be sure you understand what research means before you accept claims about child psychotherapy.