In clinical research, there exist different types of studies which serve particular purposes. These studies are distinguished based on their experimental design (how the study is conducted) & the kind of data they produce– from the way a study is designed, we can draw certain expectations about the grade of evidence it produces.
Experimental designs can be described in several ways. A basic division of study designs can be made on how test subjects are enrolled, which significantly determines the study’s strength in describing a relationship between a cause (an experimental variable such as a drug to be tested) & an effect (an outcome such as cognitive performance). This particular way of classifying studies results in two main families of studies: observational studies & assignment studies.
Observational Studies
Observational studies are conducted in order to determine associations between certain prior exposures (e.g. a drug) & outcomes of interest (e.g. death).
Here, participants are selected based on exposure or outcome, depending on the type of observational study. These are more prevalent in research on nootropics as randomised controlled trials (RCTs) are generally conducted with larger samples requiring more funding. Small observational studies build up a body of evidence which provide the grounds for an RCT, a process called «hypothesis-generating» (as opposed to hypothesis-testing). Cohort studies & case-control studies are two major types of observational studies, both of which involve following a group of patients over a period of time.
Subjects in cohort studies are selected based on their having received a particular exposure, then they are followed prospectively (forward in time) until a certain outcome of interest (e.g. death) occurs. RCTs are also prospective.
In a case-control study, subjects are selected based on their exhibiting a certain outcome & tracing their history back (retrospectively) to find out whether they have had a certain exposure (e.g. used a particular drug). Retrospective case-control studies are especially useful when studying rare diseases.
Assignment Studies
Assignment studies enroll subjects to either test or control groups.
Assignment studies are subdivided based on (1) whether the allocation of subjects into test groups is randomised & (2) whether a control group is present.
Randomised controlled trials (RCTs) are generally considered to be the gold standard of clinical evidence for their strong internal validity, & are used to demonstrate causal relationships between experimental variables & outcomes. Randomly assigning patients to either treatment or control groups theoretically establishes equal groups, as any differences in age, race, comorbidity, or other features are equally distributed (eliminates sample selection bias). Prospective follow-up & the presence of a control group allows for comparison of the experimental variable (e.g. a new drug) against a standard treatment (to demonstrate a better treatment effect) or placebo (to demonstrate a treatment effect).
Synthetic Studies
Systematic reviews & meta-analyses critically evaluate the literature by consolidating the results of several studies focused on the same topic.
Synthetic studies are more recent study designs that have been developed out of a need to draw from the existing evidence on a topic. Prior to the rise of systematic reviews & meta-analyses, studies were selectively cited which led to bias (e.g. selecting only the studies which supported the use of a drug & either intentionally or unwittingly omitting the others which found significant side effects). Nowadays, both are considered the highest forms of clinical evidence, producing strong inferences of treatment effects. Synthetic studies are part of the trend of comparative efficacy analyses (CEAs): gathering data on several major drugs used for the same purpose & determining which are superior. Collecting findings from multiple RCTs & observational studies can produce a more complete picture of a drug’s safety & efficacy- in other words, considering the ‘big picture’. However, before drawing conclusions, one must be cognisant of differences between the studies that have been gathered (e.g. study protocol, different doses used, different sample characteristics).
Systematic reviews present findings from a pre-defined, reproducible search of the literature- that is, the authors exhaustively describe the methods they used to search databases & how they selected which studies to include in their systematic review, usually with a pre-defined criteria set. The importance of reproducibility is to reduce bias from selective inclusion of studies– this is a weakness of narrative reviews, in which the author performs a search & simply chooses which studies to include.
Meta-analyses are systematic reviews where the gathered data is then combined, producing an estimate of the true treatment effect from the pooled data. This is commonly expressed in what’s called a Forest plot, which shows the individual trials included in the systematic review as well as a diamond representing the estimate of the true efficacy or safety measure of the drug (how to read & interpret different tables & graphs will be covered later!).
Summary
As we have seen, the design of a clinical trial can provide a quick way to judge its findings.
- Observational studies produce evidence of associations by following patients over a period of time. Patients are selected based on a specific previous exposure in the case of prospective cohort studies or for a certain outcome in the case of retrospective case-control studies.
- Assignment studies produce evidence of causal relationships by assigning patients to multiple groups including a comparator arm. The most prominent example of an assignment study is the randomised control trial, which has become the standard for clinical data.
- Synthetic evaluations of the literature, such as systematic reviews & meta-analyses, draw on existing studies to better approximate treatment effects of drugs of interest.
- Other experimental designs which are less commonly relevant to the area of nootropics include cross-sectional observational studies & non-randomised controlled trials.