Top 5 Tips for Reading a Veterinary Scientific Study

Sebastian Arlt, Dr.med.vet, DECAR, Freie Universität Berlin, Berlin, Germany

Marnie L. Brennan, BSc(VB), BVMS, PhD, PGCHE, DipECVPH (PM), MRCVS, FHEA, University of Nottingham, Loughborough, England

ArticleLast Updated January 20229 min readWeb-Exclusive

Results from scientific research can help provide optimal care. Daily decisions should lead to effective diagnostic procedures and therapeutic interventions with optimal risk:benefit ratios. It is therefore important to be able to select and evaluate scientific literature relevant to the field and/or patient. Critical evaluation aids in identification of strengths and weaknesses of a study and its relevance and validity in the clinic1; relevant information for critical evaluation is typically found in the methods and results sections.

The following are the authors’ top 5 tips for evaluating a scientific study.

1. Determining Relevance

For ease of application, the circumstances (eg, patient and condition being evaluated) and available resources (eg, therapies, equipment, required skills) in the study should be similar to those in the clinic. Other important considerations include compliance with local legal requirements, ethics, and wishes of the pet owner.2 For example, the study may recommend a surgical procedure, but surgery may not be the best course of action in a patient with high anesthetic risk; therefore, the recommendations from the study are less applicable to the case.

2. Identifying Study Design

Different study designs are best suited to address different questions (Table 1).3 Regardless of whether the study concerns diagnostics, therapeutic procedures or treatments, disease prevention, transmission, or another type of research, the evidence can be ranked based on the methodology used.4 Methodology most likely to minimize systematic errors is considered to have the least bias, and methodology with the most systematic errors results in the most bias. Systematic errors are not based on chance; they are the result of problems with the study design or methods used to obtain data.5 Increased levels of bias increase the likelihood of distorted results. Clinical decision-making should be based on evidence from the least-biased, but most applicable, study design available.

Common Study Types

Study Type*

Description

Meta-analysis

Quantitative statistical analysis combining data from several studies conducted as part of a systematic literature review

Systematic review

Method of collating and summarizing information from all published articles addressing a particular question

Follows defined and rigorous methods to search and select literature, assess quality, and make conclusions

Randomized controlled trial

Intervention study used to assess the effect of a treatment or intervention

Study subjects randomly allocated to either an intervention or a control group (ie, no treatment, placebo, current best treatment)

Ideally, everyone involved in the study should be blinded so no one knows which treatment each patient received

Cohort study

Prospective or retrospective study in which exposed and unexposed groups (cohorts) are observed over a period of time

Outcome (eg, disease) is measured at the end of the study period

Can identify risk factors associated with disease and estimate incidence

Diagnostic test validation study

Used to establish the usefulness of diagnostic tests for specific purposes

Patients are tested using both tests (often a new test compared with an accepted test), which are then compared to establish the sensitivity, specificity, and likelihood ratios of the new/repurposed test

Case-control study

Retrospective study comparing patients with (i.e., cases) and without (i.e., controls) the disease of interest and often carried out using clinical notes recorded by a veterinary clinic after interaction the pet owner

Patient histories are examined to identify risk factors for the disease

Cross-sectional study

Looks at a sample of the population at a single point in time, most commonly to determine the prevalence of a certain disease; can also include questionnaire-based studies

Case series

Description of the presentation, diagnosis, treatment, and outcome of several patients with the same disease or syndrome

Typically, there are no disease-free patients for comparison, and any differences in management are not considered

Case report

Description of a single case

Expert opinion

Can be an individual or a group of experts and provides some evidence

Useful when no other scientific, research-driven information is available

Consensus Statement

Although consensus statements based on peer-reviewed literature can be relatively robust, those that rely solely on expert opinion without attempts to gather opinions objectively (eg, using specific frameworks, such as the Delphi approach) may be less reliable

*Study types are listed in order from lowest to highest risk for bias.1,2

3. Considering Potential Bias

The level of evidence (eg, meta-analysis, randomized clinical trial, case report, expert opinion or experience) in a study indicates how prone it is to bias.6 The quality of the study, which is determined by an appraisal process, can also be an indicator of bias. Appraisal should include investigating factors, such as the type, age, sex, and number of study participants (ie, sample size); enrollment criteria; definitions of conditions; which and how examinations were performed; how participants were allocated to different groups; and whether the study was blinded (i.e., humans involved in the study did not know which treatment group participants were allocated to). Assessment for potential bias should also include whether clear inclusion and exclusion criteria are given and whether outcome measures are reasonable and relevant. Questions asked during the appraisal process depend on the study design and research question of interest (Tables 2-4). 

A common flaw in veterinary publications is not including a sufficient number of subjects or samples to draw robust conclusions.7 A sample size calculation or reasoning regarding the number of participants included in the study should be included in the methods, results, and discussion sections. 

In some studies, patients with a specific disease are enrolled without a clear case definition or a documented diagnostic procedure (including which test results indicated physiologic/not physiologic conditions) that can be used to confidently identify the presence of disease, making it difficult to draw a conclusion. How participants are allocated to different groups is important because unequal distribution (eg, including more severely ill patients in one group compared with another) can distort outcomes.8 In addition, a lack of blinding can influence recorded outcomes, as researchers may be (consciously or unconsciously) biased in the conduction or interpretation of a study when they know which patients received specific treatments.

Evaluation Guide For Intervention Trials

Step 1: Evidence level

Meta-analysis (statistical combination of the results of several studies) [5 points]

Clinical trial [3 points]

Case report [2 points]

Expert opinion or experience [1 point]

Step 2: Additional quality criteria (regarding corresponding evidence level)

Meta-analysis

Literature search is exhaustive and reproducible [2 points]

Included trials are clinically comparable [4 points]

Included trials are of high quality (ie, randomized, controlled, blinded) [2 points]

Results are discussed objectively and critically, including questions regarding comparability and bias [2 points]

Clinical trial

Trial comprises a sufficient number of participants or samples, including a sample size calculation to identify the appropriate number of participants or samples [2 points]

Essential information (eg, number included, breed, age, sex, inclusion criteria, housing) is given regarding participants [1 point]

Trial is composed of an adequate control group [3 points]

Trial is randomized [1 point]

Trial is blinded [1 point]

Examinations and interventions are described in detail, and results are presented completely [1 point]

Adequate statistical procedures are used, and any data that are incomplete or missing are documented 1 point]

Results are discussed critically [1 point]

References are extensive and current [1 point]

Case report

Essential information (eg, number included, breed, age, sex, inclusion criteria, housing) is given regarding participants [2 points]

Examinations and interventions are described in detail [2 points]

Results are discussed critically [2 points]

References are extensive and current [1 point]

Expert opinion or experience

Results are discussed critically [1 point]

References are extensive and current [1 point]

Step 3: Summation of points for an overall score

15-13 = very good; 12-10 = good; 9-7 = satisfactory; 6-4 = adequate; 3-2 = inadequate; 1 = fail

Evaluation Guide For Research On Diagnostic Tests

Study design

Disease/condition to be tested is clearly defined [1 point]

Clear, defined test results indicating physiologic/not physiologic conditions [2 points]

Clear inclusion and exclusion criteria for participants or samples are reported [1 point]

Appropriate number of participants or samples are included [1 point]

Procedures are described in detail [1 point]

Study is blinded [2 points]

Test characteristics

Test is compared with an acknowledged gold standard [1 point]

Sensitivity and specificity of the test are given [2 points]

Repeatability (same result obtained when test is repeated) is good [1 point]

Possible biases or other problems of the test (preanalytic/analytic) are discussed [1 point]

Practical relevance

Quality of the test results are discussed in context with other diagnostic tools for the given disease or condition [1 point]

Applicability and reliability of the test are discussed objectively [1 point]

Summation of points for an overall score

15-13 = very good; 12-10 = good; 9-7 = satisfactory; 6-4 = adequate; 3-2 = inadequate; 1 = fail

Evaluation Guide For Literature Reviews

Literature search and inclusion

Literature search was conducted systematically via databases and is well documented [4 points]

Search terms used are documented [2 points]

More literature was searched in reference lists of acquired articles (eg, hand searching*) [1 point]

Inclusion and exclusion criteria for papers are well-documented [2 points]

Assessment

Quality of each paper is assessed systematically [4 points]

Findings and conclusions are discussed objectively [2 points]

Summation of points for an overall score

15-13 = very good; 12-10 = good; 9-7 = satisfactory; 6-4 = adequate; 3-2 = inadequate; 1 = fail

*Hand searching is the examination of reference lists of included studies in order to identify other relevant citations.

4. Identifying Appropriate Results

Presentation of crucial information (eg, patient age and medical history, case or control definitions, diagnoses) should be examined.1 Lack of clarity on whether specific aspects of study design, methods, and results were not considered or just not reported by authors,3 and journal word count restrictions leading to fewer details provided, can cause difficulties for the reader. Special attention should be given to conclusions, as they may be based on weak or absent scientific data or go beyond the stated research question.3 

Transparent reporting is crucial; several reporting guidelines have been developed to ensure important details are not missed.9 Quality of a study decreases when critical appraisal is not possible due to poor reporting. 

Authors should include whether ethical approval was sought, as well as conflicts of interest and sources of research funding, as these may influence study design or interpretation.1

5. Evaluating Literature Guides

Several tools are available to guide critical assessment of a scientific study, including evaluation guides,10 which can help assess key features and quality. Summarizing rating points can provide confidence in recommendations based on study results, although additive scores may not always be necessary. Some tools are included here, but these do not cover all possible scientific research approaches, and some items may not be helpful or easily assessed in every case. 

The study design should be determined first, so the appropriate evaluation guide can be chosen. Additional criteria, including information content and objectivity, should be assessed using the appropriate guide. Determining whether the right statistics have been used may be challenging, but focusing on the design features that should be present can help determine study quality.

Conclusion

Critical reading of scientific studies can be time-consuming, but practice can make it easier to decide whether a study is relevant and valid and can be applied in the clinic.1 Identifying limitations can help results be interpreted and applied appropriately.

© 2023 Educational Concepts, L.L.C. All Rights Reserved.