Types of errors in research studies

What determines whether a study is valid? The validity of a study depends on the extent to which the results obtained are accurate in answering the research question posed. There are two validity criteria: external validity refers to the study’s ability to generalise the results obtained from the study sample to the reference population, while internal validity refers to errors made during the research process. With this in mind, there are two types of error that can affect the validity of a study:

• Random error: this is due to chance and cannot be avoided, though it can be reduced. It may arise from biological variability between individuals. In this case, the error can be reduced by increasing the sample size, choosing more precise measurement instruments, or taking measurements on multiple occasions.
• Systematic error or bias: this occurs in the design or data analysis of the study. It is a predictable error and must therefore be avoided. The most well-known biases are selection bias, information bias, and confounding bias, although many others exist.

Types of bias

1. Selection bias

This is a type of systematic error that occurs when study participants are selected incorrectly, resulting in a sample that is not representative of the population due to flawed inclusion and exclusion criteria or an inappropriate recruitment approach. The most important types are summarised below:

Consider an occupational therapy study investigating the effectiveness of a rehabilitation programme for patients with acquired brain injury. The aim is to compare functional outcomes between two groups: one following the rehabilitation programme and one continuing with their usual routine. An example of selection bias would occur if participants were not randomly assigned to the intervention (rehabilitation) or control (usual routine) group, and those allocated to the rehabilitation group happened to have more severe sequelae than those in the control group, with 60% of the rehabilitation group dropping out before completion while all control group participants completed the study. Another example of selection bias would be if, despite randomisation having been carried out, the control group — not receiving rehabilitation — chose not to attend the final assessment, thus constituting a dropout. Ways to prevent selection bias in this case:

• Randomising study participants.
• Ensuring an adequate sample size for randomisation to function correctly (the importance of calculating sample size).
• Defining clear inclusion and exclusion criteria.
• Minimising loss to follow-up by implementing participant retention strategies.
• Prior to allocation, providing participants with detailed information about the study, including the potential issues that may arise if they withdraw, so that they can make a more informed decision about their participation.
• Prior to allocation, informing participants that those assigned to the control group will receive the rehabilitation programme after the study has concluded.
• Attempting to collect final data from all participants, even those who did not attend all sessions.
• Conducting intention-to-treat analyses that is, analysing data according to the group to which participants were originally assigned, including all participants regardless of whether they dropped out.
• Performing sensitivity analyses stratified by number of sessions attended or other markers of treatment adherence.
• Considering potential confounding variables (see below).

2. Information bias

This is a type of systematic error that occurs during data collection and may lead to incorrect classification of participants with respect to exposure and/or outcome. It can be non-differential , occurring equally across all participants (exposed and unexposed / diseased and non-diseased) or differential, when it affects only a subgroup (e.g. only those with the disease, or only the unexposed). This type of bias can arise, for example, from using measurement instruments with poor sensitivity or specificity, applying incorrect diagnostic criteria, or imprecision in data collection. An example in occupational therapy would be the use of an assessment tool whose psychometric properties have not been evaluated, or whose properties have been evaluated but not in the population of interest. Ways to prevent it:

• Using assessment instruments whose psychometric properties have been evaluated.
• Using assessment instruments that are valid and reliable for the population in which they will be applied.
• Using instruments capable of discriminating between individuals with and without the condition.
• Training the staff who will administer the tools.
• Following the administration instructions for the tool faithfully.

Consider a study evaluating the effectiveness of an intervention programme on motor function improvement in patients who have undergone surgery for trapeziometacarpal osteoarthritis (rhizarthrosis). During the study, self-administered questionnaires are used to gather information on participants’ pain levels during cooking activities. As the questionnaire is self-administered, information bias could arise for various reasons: participants may underestimate or overestimate their pain, misunderstand the questions, be influenced by emotional factors in their responses, or find that cooking is not a meaningful activity for them. Ways to prevent it:

• Ensuring questions are clear for different types of respondents (varying educational levels, ages, cultural backgrounds, etc.).
• Ensuring the tools used have been tested for validity and reliability in this population (click to find out more)
• Training those who will explain the tool to participants and following the developers’ instructions faithfully.
• Guaranteeing the confidentiality and anonymity of respondents.
• Collecting other variables that may help to contextualise participants’ responses.

3. Confounding bias

This is a type of systematic error that occurs when an association between two variables is observed without accounting for the influence of a third variable — known as a confounding variable. This third variable must be independently associated with both variables (the exposure variable and the outcome variable) and must not lie on the causal pathway between them — that is, the occurrence of one must not require the prior occurrence of the other. An example in occupational therapy would be a study examining the impact of cocaine use on occupational performance in which questions about exposure to other substances (e.g. alcohol) are not collected. Since alcohol use is a risk factor for both occupational performance difficulties and cocaine use, failing to account for this variable in the analysis could distort the results. Ways to prevent or more precisely, to control it:

• Identifying and collecting all potential confounding variables in the study, which requires a thorough knowledge of the prior evidence on the topic.
• Using statistical analysis models that allow adjustment for the identified confounding variables.
• Conducting stratified analyses to examine the association between the exposure and outcome variables within different levels of the confounding variables.
• Conducting sensitivity analyses to assess the impact of confounding variables on the study results.

The following table provides a summary of the different types of errors and biases, along with their potential consequences and control measures:

  Since it is very difficult to eliminate all biases, it is important to declare them in the limitations section of the article, and to bear them in mind when reading other publications. The presence of these biases will affect the validity of the study, making it essential to plan an appropriate design and analysis with the aim of minimising them as far as possible.

Now that you know more… would you like to test yourself anonymously? You can do so using the form we have prepared for you.

References:

1. Rothman KJ, Greenland S. Modern epidemiology. 2nd. ed. Philadelphia: Lippincott-Raven Publishers; 1998.
2. Gordis L. Epidemiología. 3ª ed. Madrid. Elsevier; 2005.
3. Bases metodológicas de la investigación clínica y epide­miológica. En: Argimón Pallás JM, Jiménez Villa J (eds.). Mé­todos de investigación clínica y epidemiológica. Madrid: Elsevier; 2004. p. 8-15.
4. Errores en epidemiología. Errores sistemáticos. Factores de confusión y modificación del efecto. Evid Pediatr. 2016; 12:16
5. Hernández-Avila M, Garrido F, Salazar-Martínez E. Sesgos en estudios epidemiológicos. Salud pública Méx. 2000;42(5).

Casaña Escriche, E., & Muñoz Sánchez, R (2024, abril 11). Types of errors in research studies. PublicaTO – Scientific Skills in Occupational Therapy by InTeO.https://hacto.umh.es/2024/04/11/tipos-de-errores-en-los-estudios-de-investigacion/