Last update: 14 March 2016
Introduction
The development of technologies (medicine / treatment / device) relies heavily on research – the gathering of information or data to produce new knowledge. This is driven by the need to provide evidence for:
- regulators about the safety and potential benefit (efficacy) of a new therapy, and
- payers about the effectiveness of a new therapy in the real world along with information about the costs and expected use of the therapy.
Research that is used in medicines development or in any other disciplines (such as anthropology, sociology, astronomy, chemistry) can be classified as either ‘qualitative’ or ‘quantitative’.
What is quantitative research?
Quantitative research, as its name implies, is concerned with quantifying results of observations. Quantitative data is any data that is in numerical form such as statistics, percentages, etc. A familiar kind of quantitative research for most people is applied population statistics, where the proportions of certain types of people (for instance, 30% female; 10% unemployed, etc.) are used to help policymakers and others make decisions with regards to populations. Bio-statistical information (such as the prevalence, in percentage terms, of heart attacks) is used in medicine to help clinical care providers and administrators make decisions about health programmes.
Common types of quantitative research in health technology development are experiments, often in the form of randomised controlled trials, which seek to understand the effects of a new technology in comparison with other treatments or (rarely) no treatment. The characteristics of patients are measured and counted as well as the dose and frequency of the new therapy. Patients are observed and important data such as changes in measurable parameters of the targeted disease (endpoints), possible side effects (adverse events) and subjective data like pain scores are gathered. It is assumed that these observations are a fair reflection of reality and are predictive of the future. For example, if a new medicine reduces heart attacks compared to a comparator in repeated experiments, it is assumed that this will likely happen in similar patients with the same type of medicine in the real world situation (as opposed to the more controlled conditions in a clinical trial).
What is qualitative research?
Counting events that have occurred at a certain place and time (quantitative research) can help us to understand what might happen in the future. However, this provides very little information about feelings or motivations. Qualitative research can provide more information about how a patient reacts to a negative event (like a hospital admission) or manages a new treatment regime.
For example, quantitative research may provide very little information about factors such as:
- social or cultural values or arrangements,
- patient-physician relationships,
- stigma, or
- conflict with religious or cultural views.
A new contraceptive technology may avoid pregnancy, but this might not be desirable in populations who have strong cultural or religious motivations to have children. To understand how desirable the contraceptive technology is to patients and society, it must be studied using a different research approach. This is where qualitative research is valuable.
Qualitative research is important because it gives a more thorough and defensible understanding of how or why a population might use a new therapy or how they will feel about using it. Qualitative research is primarily exploratory research: it has been described as ‘a systematic, subjective approach to describe life experiences and give them meaning.’1
This kind of research is a way to gain understanding of underlying reasons and motivations and to uncover current thoughts or opinions of individuals. It provides insights into a problem or helps to develop ideas or hypotheses for potential quantitative research. While an individual patient may feel strongly that payers should provide access to a new medicine, payers must consider what society wants as a whole. It is possible that the beliefs, attitudes, or feelings of that individual patient do not reflect society’s beliefs.
Quantitative versus qualitative versus mixed methods
Quantitative and qualitative research approaches can complement each other and may even appear similar. For example, the use of a questionnaire may be viewed as qualitative research but may actually be quantitative, depending on how the survey is designed.
If the researcher asks respondents to answer on a scale (for instance, to give an answer from 1 ‘strongly disagree’ to 5 ‘strongly agree’), this is a form of quantitative research. If the response is open-ended and patients are not constrained to a scale or a choice of answers, the research is qualitative. However, as illustrated with the development of patient-reported outcomes (PROs), the scales and other outcome measures provided by researchers are best developed through qualitative methods that involve patients.
Research does not have to be entirely qualitative or entirely quantitative. A popular form of research is referred to as ‘mixed methods’ research, in which both qualitative and quantitative approaches are used. Researchers deliberately combine quantitative and qualitative data instead of analysing them separately. Although there are many definitions of exactly what constitutes ‘mixed methods’, one popular definition describes it as an approach or methodology that:
- Focuses on research questions that call for real-life contextual understandings, multi-level perspectives, and cultural influences;
- Employs rigorous quantitative research assessing magnitude and frequency of constructs and rigorous qualitative research exploring the meaning and understanding of constructs;
- Utilises multiple methods (such as intervention trials and in-depth interviews);
- Intentionally integrates or combines these methods to draw on the strengths of each; and
- Frames the investigation within philosophical and theoretical positions.2
Table 1 outlines the key differences between quantitative and qualitative research.
| Quantitative research | Qualitative research |
|---|---|
| Considered ‘hard science’ | Considered ‘soft science’ |
| Objective | Subjective |
| Deductive reasoning used to synthesise data | Inductive reasoning used to synthesise data |
| Focus: Concise and narrow | Focus: Complex and broad |
| Tests theory | Develops theory |
| Basis of knowing: Cause and effect relationships | Basis of knowing: Meaning, context |
| Basic element of analysis: Measurements and statistical analysis | Basic element of analysis: Words, narrative |
| Single reality that can be measured and generalised | Multiple realities that are continually changing with individual interpretation |
| Adapted from Keeler (2010)1 | |
Qualitative methods
Qualitative data collection methods vary between unstructured or semi-structured techniques. Some common methods include focus groups (group discussions), individual interviews, and participation/observations. The sample size is typically small, and respondents are selected to fulfil a given quota. Interactions between researchers and research participants are at the core of qualitative research methods. The understanding that comes from ‘meaning’ or ‘why’ or ‘how’ does not come from observed behaviour, but from what is being said and done by participants, or what is being felt by the researcher. Qualitative researchers may also gather data from documents or other written sources. In addition to gathering data that reflects the thoughts and expressions, qualitative research has been defined as having different characteristics than quantitative research. For example what is learned today in a specific group cannot be generalised or may change over time.
Differences in the relative perceived importance of characteristics and principles for qualitative research as well as the overarching purpose for the conduct of the research have led to different general types of qualitative research. This is similar to quantitative research, where experimental trials (such as randomised controlled trials) and non-experimental trials (such as observational trials) have been developed for unique purposes.
Beyond clinical effects: Qualitative research for decision-making and HTA
HTA processes attempt to provide decision-makers with the best possible (accurate and comprehensive) information. In order to provide accurate and comprehensive information, the use of synthesis (such as meta-analysis, network meta-analysis, modelling) and critical appraisal (such as quality checklists) predominate quantitative research in HTA processes. Similar approaches to synthesising qualitative research have been developed. These try to elaborate findings from qualitative research using an approach which is oriented in a similar way as quantitative research, thus allowing the qualitative data to be evaluated along with quantitative research in the HTA process.
How do qualitative methods fit into HTA processes?
Qualitative research can provide strong evidence of the needs and views of patients and help decision-makers and developers of medicines understand these needs and views. It can also be used to guide larger societal decisions about how to allocate limited available resources. Some questions that are important for research and reimbursement decision-making may illustrate this:
- Should we value care for the very sick or the very old more than others?
- How should we value technologies that reduce interactions with doctors and caregivers?
- Are there reasons that help explain sub-optimal use of medicines in practice?
Quantitative research can help us understand:
- How many people are affected by a disease
- What the economic burden of a disease is
- How many people might benefit from a particular medicine
- What the value of the benefit is
- How often the medicine might be used once access is provided
These are all important aspects to consider during decision-making.
The list below gives just a few examples where qualitative research can play a role.
- What diseases are important to address
- What is the need for new medicines
- Which outcomes are important
- Development of adequate Patient Reported Outcome Measures (PROMs) and Health-related Quality of Life (HRQoL) measures
Reimbursement and decision-making
- Provide robust patient input
- Determine need for current medicine
- Identify possible problems with alternatives
- Consider social values to guide decision-making
Implementation delivery and impact
- Evaluate reasons for sub-optimal adherence
- How patient experiences can be optimised
- Determine other factors to be taken into account
References
- Keeler, R. (2010). Nursing research and evidence-based practice: Ten steps to success. Sudbury, MA: Jones & Bartlett Learning, p. 276.
- Creswell, J.W., Klassen, A.C., Plano Clark, V.L., & Smith, K.C. for the Office of Behavioural and Social Sciences Research (2011). Best practices for mixed methods research in the health sciences. National Institutes of Health. Retrieved 4 July, 2021, from https://obssr.od.nih.gov/wp-content/uploads/2016/02/Best_Practices_for_Mixed_Methods_Research.pdf
A2-6.08-v1.1