Skip to main content

Visions Journal

A reminder that this article from our magazine Visions was published more than 1 year ago. It is here for reference only. Some information in it may no longer be current. It also represents the point of the view of the author only. See the author box at the bottom of the article for more about the contributor.

Evidence of Treatment Efficacy

How do we know what isn't so?

Paul Latimer, MD, PhD, FRCPC

Reprinted from the "Treatments: What Works?" issue of Visions Journal, 2015 15 (4), pp. 8-10

Ever since the Internet age began, there has been an explosion of health-related information available to anyone, anywhere, at any time. Just open your browser and type in any kind of medical issue—including mental health issues—and you’ll be inundated with thousands of pages of suggested treatments, advice and testimonials.

This great amount of information can be very helpful. It gives anyone with access to an Internet connection the ability to research and learn about their mental health issues and can be very useful for gaining some perspective and health education.

Access to all this information, however, can also be overwhelming and confusing when information conflicts. It can even be dangerous when it isn’t clear which information is accurate and evidence-based and which is not.

Is it enough to hear from friends or online advocates that a treatment has worked?

Although many people do make their decisions because of the testimony of others, this in itself is not evidence of a treatment’s effectiveness. But what exactly is evidence when it comes to medical information?

Testing treatments in scientifically valid clinical trials (as explained below) can tell us whether a treatment is truly effective.

Testing medications—a clear-cut process

Research for medications involves evaluating and comparing the potential new medication against non-active compounds (placebo). It also compares it to other approved medications for the specific disorder. In a trial of antidepressants, for example, a new antidepressant might be compared to a sugar pill (the placebo) and to an established antidepressant.

Before a study is begun, it must first be evaluated and approved by an independent ethics review board. Such review boards are typically made up of scientists, ethicists and lay people who have no vested interest in the outcome of the studies being reviewed.

The purpose of the review process is to assure, both in advance and periodically throughout the study, that the rights and welfare of the subjects (i.e., people taking part in the study) are protected. It also ensures the subjects are properly informed about the potential risks and benefits involved in their participation.

Additionally, the review will ensure that the study uses proper scientific methods so the results will be valid. Proper scientific methods include such things as randomization and double-blind procedures. In a trial of antidepressants, subjects who have depression would be randomly assigned to receive one of the new drug, placebo or established medication and neither they nor the researcher would know which treatment they were receiving. This is called a “double-blind” study.

Most trials are double-blind. This is done to minimize bias. Because, for example, when either the researcher or the subject knows who is getting the placebo, the occurrence of side-effects tends to be greater for the experimental treatment than for the placebo. However, in a double-blind study, side-effects are actually quite common in subjects taking the placebo.

It’s important to conduct the study so that placebo effects are minimized. A placebo effect occurs when a person perceives a positive effect from an inactive treatment. Many studies fail, not because the treatments are ineffective, but because the placebo effects are too large. As a hypothetical example, if the placebo was 100% effective, it would be impossible to show any other treatment was more effective. The higher the placebo response rate is, the more difficult it is to show the experimental treatment is more effective.

Proper scientific methods include adequate selection criteria for subjects. The goal is to choose subjects who have the disorder being studied but no other disorders that would complicate their treatment or put them at increased risk. For example, in a depression study, suicidal patients are usually excluded.

Proper scientific methods also include sufficient sample size (i.e., number of subjects) to show a statistically significant difference. The greater the difference between the effectiveness of two treatments, the smaller the sample size required to show a statistically significant difference. If differences between two treatments are likely to be small, a very large number of subjects is required to show that a difference between them is unlikely to occur by chance alone.

Preclinical research (i.e., research that takes place before the trials in humans) examines the chemical makeup of a new drug for general safety. Once the chemistry is understood, the compound is tested on laboratory animals to learn how it works in the lab animal’s body, to further ensure its safety. A new compound will only be tested on humans after promising results in these preclinical stages.

Human testing on healthy volunteers is typically referred to as Phase I. These subjects don’t have the disorder for which the treatment is intended. The goal of this phase is to determine a drug’s toxicity, absorption, distribution and metabolism. This usually involves about 20 to 80 subjects.

There may also be attempts in Phase I to see how individuals respond to various doses of the medication. Even when there’s good evidence a drug has the desired effects, it’s still necessary to work out what doses are required; side-effects often depend on the dose. An ideal dose will maximize the effectiveness while minimizing side-effects.

In Phase 2, subjects with the disorder being studied are used but they are selected in quite a narrow way to reduce risk and maximize chances of success. Phase 2 subjects are less like typical patients seen in a clinic than those in Phase 3. Phase 2 will involve 100 to 300 subjects, and Phase 3, 1,000 to 3,000 subjects.

After Phase 3, the results from all the phases can go to the regulatory authorities to consider whether to approve the drug or not. In the US, it’s the Federal Drug Administration (FDA), and in Canada, Health Canada has this mandate. It usually takes about 12 years to get to this submission stage.

The drug is approved for use only if it is shown to be both safe and effective and it will be approved only for the conditions that have been studied. Many potential new treatments don’t get approved.

If the drug is approved there are then Phase 4 or post-marketing studies that might compare the drug to a competitor, study adverse effects or explore some other aspect of the use of the medication.

Any medication treatment prescribed by a doctor (MD) in Canada will have gone through this process.

Non-med treatments—for most, the evidence isn’t there

Many therapies (e.g., naturopathic remedies) and products (e.g., vitamins or nutraceuticals) advertised and available in the community or on the internet have not met these standards. They are not subjected to rigorous testing and approval before being sold to consumers. This is why the medical profession may be skeptical of these products and therapies.

But how do we account for the many positive testimonies from people using unproven treatments? These could be accounted for by the placebo effect. Placebo effects are very common in most areas of medicine.

It’s estimated in psychiatric drug studies that between 20% and 50 % of participants taking an inactive medicine will report positive results.1 Their psychiatric symptoms really do improve while on the ‘fake’ medicine. They also have many side-effects. Things such as cost of the treatment, colour of the pill and what the subject is told to expect can also influence both the beneficial effects and the side-effects of the placebo. This is a complex topic about which whole books have been written.2

Medications are not the only source of placebo effects. This kind of effect can also occur from interaction with a practitioner, from counselling or psychotherapy or anything else that is offered as a treatment.

Unfortunately, the placebo effect is usually fleeting and cannot replace an actual treatment for a medical condition. For example, a depressed person may feel better for a time while taking an inactive medication, but will often soon experience a recurrence of their symptoms.

If you look online for medical advice, be sure to look at who is sponsoring the information. If it’s sponsored by a reputable source such as a provincial health board, professional organization or a university, it is likely to contain information that has been filtered with a scientific eye.

If you’re unsure of the source, bring your findings or questions with you to your doctor. They have been educated on scientific method and current treatments for mental health conditions. They should be willing to assist you in evaluating health information and working with you to find an option that will be effective in your situation.

About the author

Dr. Latimer is a general psychiatrist with more than 40 years of clinical and research experience. He has been involved in more than 150 clinical trials and the development of many psychiatric medications available today. He practises in Kelowna and is the Chief Executive Officer of Okanagan Clinical Trials

  1. Quitkin, F.M. (1999). Placebos, drug effects, and study design: A clinician’s guide. American Journal of Psychiatry, 156(6), 829-836.

  2. Shapiro, A.K. & Shapiro, E. (1997) The powerful placebo: From ancient priest to modern physician. Baltimore, MD: The Johns Hopkins University Press.

Stay Connected

Sign up for our various e-newsletters featuring mental health and substance use resources.