Experimental vs Quasi-Experimental Design: Which to Choose?

Here’s a table that summarizes the similarities and differences between an experimental and a quasi-experimental study design:

Experimental Study (a.k.a. Randomized Controlled Trial)	Quasi-Experimental Study
Objective	Evaluate the effect of an intervention or a treatment	Evaluate the effect of an intervention or a treatment
How participants get assigned to groups?	Random assignment	Non-random assignment (participants get assigned according to their choosing or that of the researcher)
Is there a control group?	Yes	Not always (although, if present, a control group will provide better evidence for the study results)
Is there any room for confounding?	No (although check Manson et al. for a detailed discussion on post-randomization confounding in randomized controlled trials)	Yes (however, statistical techniques can be used to study causal relationships in quasi-experiments)
Level of evidence	A randomized trial is at the highest level in the hierarchy of evidence	A quasi-experiment is one level below the experimental study in the hierarchy of evidence [source]
Advantages	Minimizes bias and confounding	– Can be used in situations where an experiment is not ethically or practically feasible – Can work with smaller sample sizes than randomized trials
Limitations	– High cost (as it generally requires a large sample size) – Ethical limitations – Generalizability issues – Sometimes practically infeasible	Lower ranking in the hierarchy of evidence as losing the power of randomization causes the study to be more susceptible to bias and confounding

What is a quasi-experimental design?

A quasi-experimental design is a non-randomized study design used to evaluate the effect of an intervention. The intervention can be a training program, a policy change or a medical treatment.

Unlike a true experiment, in a quasi-experimental study the choice of who gets the intervention and who doesn’t is not randomized. Instead, the intervention can be assigned to participants according to their choosing or that of the researcher, or by using any method other than randomness.

Having a control group is not required, but if present, it provides a higher level of evidence for the relationship between the intervention and the outcome.

Examples of quasi-experimental designs include:

One-Group Posttest Only Design
Static-Group Comparison Design
One-Group Pretest-Posttest Design
Separate-Sample Pretest-Posttest Design

What is an experimental design?

An experimental design is a randomized study design used to evaluate the effect of an intervention. In its simplest form, the participants will be randomly divided into 2 groups:

A treatment group: where participants receive the new intervention which effect we want to study.
A control or comparison group: where participants do not receive any intervention at all (or receive some standard intervention).

Randomization ensures that each participant has the same chance of receiving the intervention. Its objective is to equalize the 2 groups, and therefore, any observed difference in the study outcome afterwards will only be attributed to the intervention – i.e. it removes confounding.

(for more information, I recommend my other article: Purpose and Limitations of Random Assignment).

Examples of experimental designs include:

Posttest-Only Control Group Design
Pretest-Posttest Control Group Design
Solomon Four-Group Design
Matched Pairs Design
Randomized Block Design

When to choose an experimental design over a quasi-experimental design?

Although many statistical techniques can be used to deal with confounding in a quasi-experimental study, in practice, randomization is still the best tool we have to study causal relationships.

Another problem with quasi-experiments is the natural progression of the disease or the condition under study — When studying the effect of an intervention over time, one should consider natural changes because these can be mistaken with changes in outcome that are caused by the intervention. Having a well-chosen control group helps dealing with this issue.

So, if losing the element of randomness seems like an unwise step down in the hierarchy of evidence, why would we ever want to do it?

This is what we’re going to discuss next.

When to choose a quasi-experimental design over a true experiment?

The issue with randomness is that it cannot be always achievable.

So here are some cases where using a quasi-experimental design makes more sense than using an experimental one:

If being in one group is believed to be harmful for the participants, either because the intervention is harmful (ex. randomizing people to smoking), or the intervention has a questionable efficacy, or on the contrary it is believed to be so beneficial that it would be malevolent to put people in the control group (ex. randomizing people to receiving an operation).
In cases where interventions act on a group of people in a given location, it becomes difficult to adequately randomize subjects (ex. an intervention that reduces pollution in a given area).
When working with small sample sizes, as randomized controlled trials require a large sample size to account for heterogeneity among subjects (i.e. to evenly distribute confounding variables between the intervention and control groups).

About Me