1

Statistics in clinical trials: Bias

Introduction

Statistical methods provide formal accounting for sources of variability in patients’ responses to treatment. The use of statistics allows the clinical researcher to form reasonable and accurate inferences from collected information, and sound decisions in the presence of uncertainty. Statistics are key to preventing errors and biases in medical research. This article covers the concept of bias in clinical trials.

What is bias?

Bias is the intentional or unintentional adjustment in the design and/or conduct of a clinical trial, and analysis and evaluation of the data that may affect the results.

Bias may affect the results of a clinical trial and cause them to be unreliable.

Bias can occur at any phase of research, e.g. during trial design, data collection, data analysis and publication.

Common types of bias include:

  • Selection bias
  • Measurement bias (this can be both the collection of measurements and their analysis and interpretation)
  • Publication bias

Selection bias (during patient recruitment)

If patients are selected differently according to their age or health status, the treatment outcomes may be more prominent in the group where the patients are younger and generally healthier. Therefore, any difference in outcome between the two treatment groups can no longer be attributed to the received treatment only.

Preventing selection bias during patient recruitment

Randomisation aims to ensure that two or more treatment groups (treatment arms) are comparable both in terms of known and unknown factors especially over a large number of patients.

This is done by allocating patients to treatment arms using random (by chance) allocation techniques.

Well-performed patient randomisation will allow the researcher to evaluate the observed treatment effects (response rate, survival, etc.) to be actually caused by the treatment and not by other factors (confounding factors).

Selection bias (at the time of analysis)

There are a few common problems that may arise during the course of a trial that relate to patient adherence (compliance), to the protocol (trial methodology) and to the described treatment schedule. For example:

  • Treatment may have been interrupted or modified but not according to the rules specified in the protocol
  • Disease assessments may have been delayed or not performed at all
  • A patient may decide to stop taking part in the trial, etc.
  • Patients may turn out to be ineligible after randomisation

Consider the setting of a clinical trial comparing a new experimental treatment to the standard of care. In this trial some patients taking the experimental treatment are too ill to go to the next visit within the allotted time. A possible approach would be to include only patients with complete follow-up in the analysis of results, so to exclude those patients unable to complete all visits from the analysis. However, by doing so, one selects a sub-group of patients who, by definition, will present an artificially positive picture of the treatment under evaluation.

Preventing bias at the time of analysis

One way to do this is to include every randomised patient in the analysis irrespective of whether they received the treatment or not, i.e. ‘once randomised, always analysed’. This is a statistical concept called an intent-to-treat (ITT) analysis.

ITT analyses maintain the balance of patients’ baseline characteristics between the different treatment arms obtained from the randomisation, therefore, data obtained from ITT analysis is considered to be more representative of the real life situation.

Measurement bias (during data collection)

Measurement bias can occur when the instruments, operations or systems to record data are flawed. Perhaps an instrument is incorrectly calibrated, or perhaps the schedule of hospital visits does not correctly capture the events that could not be observed by other means.

Preventing measurement bias (during data collection)

To take an example, if you are testing a medicine which may cause periodic high fever (indicating liver damage) it is only possible to detect this if the frequency of hospital visits captures the occurrence of the fever. So researchers need to make sure that an appropriate visit schedule allows for this and therefore can reduce measurement bias.

Researchers also need to ensure that all equipment used is calibrated to ensure they record accurate results (Good Laboratory Practice (GLP)), i.e. your thermometer should record the correct temperature.

Blinding

We can also prevent measurement bias by a process called blinding. Blinding is when the allocated treatment is unknown to the patients and/or the investigators. In double-blind trials both the patients and the investigators do not know who was allocated to the treatment. Double-blind trials are thought to produce objective results, since the expectations of the doctor and the participant do not affect the outcome. In a triple-blind trial, the patient, investigator and analyst are all unaware of who received the treatment.

Blinding is particularly relevant when the trial outcome is subjective, like the reduction in pain, or when an experimental treatment is being compared to a placebo. However, while a double-blind randomised trial is considered the gold standard of clinical trials, blinding may not always be feasible:

  • Treatments may cause specific adverse effects that make them easy to identify
  • Treatments may need different procedures for administration or different treatment schedules.

Measurement bias (during data analysis)

In a clinical trial, it is possible to find sub-groups of patients who respond better to treatment. If the sub-groups are identified and used for analysis after the data has been collected, bias is almost inevitable. Sub-group analyses involve splitting the trial participants into sub-groups. This could be based on:

  • Demographic characteristics (e.g. sex, age)
  • Baseline characteristics (e.g. a specific genomic profile)
  • Use of any other therapy in parallel.

Publication bias

Publication bias means that positive results from research are more likely to be published than negative results. Publication bias is harmful because it prevents access to negative research results, or in other words, researchers planning new experimentation may be misled by the information available in published results. Negative results may inform about the lack of efficacy of a treatment and the absence of justification for continuing with further development. In lay terms, if more negative research results were published it could prevent researchers making the same mistakes. Publication bias works in two ways: researchers may be reluctant to submit negative results for publication; and publishers, journals and article peer reviewers may also reject publishing of negative results. .

Preventing publication bias

Initiatives are ongoing for reducing publication bias. One of them is to promote the registration of clinical trials for medicines before implementation. For instance, the International Committee of Medical Journal Editors (ICMJE) will not publish trials that are not registered in public registries such as EU Clinical Trial Register (https://www.clinicaltrialsregister.eu), clinicaltrials.gov, United States. With such registries, researchers and patients know what the existing clinical trials are, even if their results were never published, and may contact the trial sponsor or researchers in order to gain access to the results.

A2-4.33.2-V1.2