We're sorry, but your current browser configuration is not compatible with this site. Please enable JavaScript or switch to a JavaScript-capable browser. Learn how to enable JavaScript

Content continues after advertisement

Top 5 Tips for Reading a Veterinary Scientific Study

Sebastian Arlt, Dr.med.vet, DECAR, Freie Universität Berlin, Berlin, Germany

Marnie L. Brennan, BSc(VB), BVMS, PhD, PGCHE, DipECVPH (PM), MRCVS, FHEA, University of Nottingham, Loughborough, England

Ethics & Human-Animal Bond

January/February 2022

Web-Exclusive

This is a filled error message

Account sign in.

To access full articles on www.cliniciansbrief.com, please sign in below.

Forgot password?

Create Account

Trusted content.

Tailored to you.

For free.

Create an account for free

Want free access to the #1 publication for diagnostic and treatment information? Create a free account to read full articles and access web-exclusive content on www.cliniciansbrief.com.

Results from scientific research can help provide optimal care. Daily decisions should lead to effective diagnostic procedures and therapeutic interventions with optimal risk:benefit ratios. It is therefore important to be able to select and evaluate scientific literature relevant to the field and/or patient. Critical evaluation aids in identification of strengths and weaknesses of a study and its relevance and validity in the clinic¹; relevant information for critical evaluation is typically found in the methods and results sections.

The following are the authors’ top 5 tips for evaluating a scientific study.

Determining Relevance

For ease of application, the circumstances (eg, patient and condition being evaluated) and available resources (eg, therapies, equipment, required skills) in the study should be similar to those in the clinic. Other important considerations include compliance with local legal requirements, ethics, and wishes of the pet owner.² For example, the study may recommend a surgical procedure, but surgery may not be the best course of action in a patient with high anesthetic risk; therefore, the recommendations from the study are less applicable to the case.

Identifying Study Design

Different study designs are best suited to address different questions (Table 1).³ Regardless of whether the study concerns diagnostics, therapeutic procedures or treatments, disease prevention, transmission, or another type of research, the evidence can be ranked based on the methodology used.⁴ Methodology most likely to minimize systematic errors is considered to have the least bias, and methodology with the most systematic errors results in the most bias. Systematic errors are not based on chance; they are the result of problems with the study design or methods used to obtain data.⁵ Increased levels of bias increase the likelihood of distorted results. Clinical decision-making should be based on evidence from the least-biased, but most applicable, study design available.

TABLE 1

COMMON STUDY TYPES^1,2

Study Type*	Description
Meta-analysis	Quantitative statistical analysis combining data from several studies conducted as part of a systematic literature review
Systematic review	Method of collating and summarizing information from all published articles addressing a particular question Follows defined and rigorous methods to search and select literature, assess quality, and make conclusions
Randomized controlled trial	Intervention study used to assess the effect of a treatment or intervention Study subjects randomly allocated to either an intervention or a control group (ie, no treatment, placebo, current best treatment) Ideally, everyone involved in the study should be blinded so no one knows which treatment each patient received
Cohort study	Prospective or retrospective study in which exposed and unexposed groups (cohorts) are observed over a period of time Outcome (eg, disease) is measured at the end of the study period Can identify risk factors associated with disease and estimate incidence
Diagnostic test validation study	Used to establish the usefulness of diagnostic tests for specific purposes Patients are tested using both tests (often a new test compared with an accepted test), which are then compared to establish the sensitivity, specificity, and likelihood ratios of the new/repurposed test
Case-control study	Retrospective study comparing patients with (ie, cases) and without (ie, controls) the disease of interest and often carried out using clinical notes recorded by a veterinary clinic after interaction the pet owner Patient histories are examined to identify risk factors for the disease
Cross-sectional study	Looks at a sample of the population at a single point in time, most commonly to determine the prevalence of a certain disease; can also include questionnaire-based studies
Case series	Description of the presentation, diagnosis, treatment, and outcome of several patients with the same disease or syndrome Typically, there are no disease-free patients for comparison, and any differences in management are not considered
Case report	Description of a single case
Expert opinion	Can be an individual or a group of experts and provides some evidence Useful when no other scientific, research-driven information is available
Consensus Statement	Although consensus statements based on peer-reviewed literature can be relatively robust, those that rely solely on expert opinion without attempts to gather opinions objectively (eg, using specific frameworks, such as the Delphi approach) may be less reliable

*Study types are listed in order from lowest to highest risk for bias.

Considering Potential Bias

The level of evidence (eg, meta-analysis, randomized clinical trial, case report, expert opinion or experience) in a study indicates how prone it is to bias.⁶ The quality of the study, which is determined by an appraisal process, can also be an indicator of bias. Appraisal should include investigating factors, such as the type, age, sex, and number of study participants (ie, sample size); enrollment criteria; definitions of conditions; which and how examinations were performed; how participants were allocated to different groups; and whether the study was blinded (ie, humans involved in the study did not know which treatment group participants were allocated to). Assessment for potential bias should also include whether clear inclusion and exclusion criteria are given and whether outcome measures are reasonable and relevant. Questions asked during the appraisal process depend on the study design and research question of interest (Tables 2-4).

A common flaw in veterinary publications is not including a sufficient number of subjects or samples to draw robust conclusions.⁷ A sample size calculation or reasoning regarding the number of participants included in the study should be included in the methods, results, and discussion sections.

In some studies, patients with a specific disease are enrolled without a clear case definition or a documented diagnostic procedure (including which test results indicated physiologic/not physiologic conditions) that can be used to confidently identify the presence of disease, making it difficult to draw a conclusion. How participants are allocated to different groups is important because unequal distribution (eg, including more severely ill patients in one group compared with another) can distort outcomes.⁸ In addition, a lack of blinding can influence recorded outcomes, as researchers may be (consciously or unconsciously) biased in the conduction or interpretation of a study when they know which patients received specific treatments.

TABLE 2

EVALUATION GUIDE FOR INTERVENTION TRIALS

Step 1: Evidence level
	Meta-analysis (statistical combination of the results of several studies) [5 points] Clinical trial [3 points] Case report [2 points] Expert opinion or experience [1 point]
Step 2: Additional quality criteria (regarding corresponding evidence level)
Meta-analysis	Literature search is exhaustive and reproducible [2 points] Included trials are clinically comparable [4 points] Included trials are of high quality (ie, randomized, controlled, blinded) [2 points] Results are discussed objectively and critically, including questions regarding comparability and bias [2 points]
Clinical trial	Trial comprises a sufficient number of participants or samples, including a sample size calculation to identify the appropriate number of participants or samples [2 points] Essential information (eg, number included, breed, age, sex, inclusion criteria, housing) is given regarding participants [1 point] Trial is composed of an adequate control group [3 points] Trial is randomized [1 point] Trial is blinded [1 point] Examinations and interventions are described in detail, and results are presented completely [1 point] Adequate statistical procedures are used, and any data that are incomplete or missing are documented 1 point] Results are discussed critically [1 point] References are extensive and current [1 point]
Case report	Essential information (eg, number included, breed, age, sex, inclusion criteria, housing) is given regarding participants [2 points] Examinations and interventions are described in detail [2 points] Results are discussed critically [2 points] References are extensive and current [1 point]
Expert opinion or experience	Results are discussed critically [1 point] References are extensive and current [1 point]
Step 3: Summation of points for an overall score 15-13 = very good; 12-10 = good; 9-7 = satisfactory; 6-4 = adequate; 3-2 = inadequate; 1 = fail

TABLE 3

EVALUATION GUIDE FOR RESEARCH ON DIAGNOSTIC TESTS

Study design	Disease/condition to be tested is clearly defined [1 point] Clear, defined test results indicating physiologic/not physiologic conditions [2 points] Clear inclusion and exclusion criteria for participants or samples are reported [1 point] Appropriate number of participants or samples are included [1 point] Procedures are described in detail [1 point] Study is blinded [2 points]
Test characteristics	Test is compared with an acknowledged gold standard [1 point] Sensitivity and specificity of the test are given [2 points] Repeatability (same result obtained when test is repeated) is good [1 point] Possible biases or other problems of the test (preanalytic/analytic) are discussed [1 point]
Practical relevance	Quality of the test results are discussed in context with other diagnostic tools for the given disease or condition [1 point] Applicability and reliability of the test are discussed objectively [1 point]
Summation of points for an overall score 15-13 = very good; 12-10 = good; 9-7 = satisfactory; 6-4 = adequate; 3-2 = inadequate; 1 = fail

TABLE 4

EVALUATION GUIDE FOR LITERATURE REVIEWS

Literature search and inclusion

Literature search was conducted systematically via databases and is well documented [4 points]
Search terms used are documented [2 points]
More literature was searched in reference lists of acquired articles (eg, hand searching*) [1 point]
Inclusion and exclusion criteria for papers are well-documented [2 points]

Assessment

Quality of each paper is assessed systematically [4 points]
Findings and conclusions are discussed objectively [2 points]

Summation of points for an overall score

15-13 = very good; 12-10 = good; 9-7 = satisfactory; 6-4 = adequate; 3-2 = inadequate; 1 = fail

*Hand searching is the examination of reference lists of included studies in order to identify other relevant citations.

Identifying Appropriate Results

Presentation of crucial information (eg, patient age and medical history, case or control definitions, diagnoses) should be examined.¹ Lack of clarity on whether specific aspects of study design, methods, and results were not considered or just not reported by authors,³ and journal word count restrictions leading to fewer details provided, can cause difficulties for the reader. Special attention should be given to conclusions, as they may be based on weak or absent scientific data or go beyond the stated research question.³

Transparent reporting is crucial; several reporting guidelines have been developed to ensure important details are not missed.⁹ Quality of a study decreases when critical appraisal is not possible due to poor reporting.

Authors should include whether ethical approval was sought, as well as conflicts of interest and sources of research funding, as these may influence study design or interpretation.¹

Evaluating Literature Guides

Several tools are available to guide critical assessment of a scientific study, including evaluation guides,¹⁰ which can help assess key features and quality. Summarizing rating points can provide confidence in recommendations based on study results, although additive scores may not always be necessary. Some tools are included here, but these do not cover all possible scientific research approaches, and some items may not be helpful or easily assessed in every case.

The study design should be determined first, so the appropriate evaluation guide can be chosen. Additional criteria, including information content and objectivity, should be assessed using the appropriate guide. Determining whether the right statistics have been used may be challenging, but focusing on the design features that should be present can help determine study quality.

Conclusion

Critical reading of scientific studies can be time-consuming, but practice can make it easier to decide whether a study is relevant and valid and can be applied in the clinic.¹ Identifying limitations can help results be interpreted and applied appropriately.

References

Dean R. How to read a paper and appraise the evidence. In Pract. 2013;35(5):282-285.
RCVS Knowledge. EBVM learning 2020. RCVS Knowledge website. Accessed March 22, 2021. http://learn.rcvsknowledge.org/ebvm-learning
Young JM, Solomon MJ. How to critically appraise an article. Nat Clin Pract Gastroenterol Hepatol. 2009;6(2):82-91.
Arlt SP, Heuwieser W. The staircase of evidence – a new metaphor displaying the core principles of evidence-based veterinary medicine. Vet Evidence. 2016;1(1).
Dohoo IR. Bias--is it a problem, and what should we do? Prev Vet Med. 2014;113(3):331-337.
Brennan ML, Arlt SP, Belshaw Z, et al. Critically appraised topics (CATs) in veterinary medicine: applying evidence in clinical practice. Front Vet Sci. 2020;7:314.
Simoneit C, Heuwieser W, Arlt S. Evidence-based medicine in bovine, equine and canine reproduction: quality of current literature. Theriogenology. 2011;76(6):1042-1050.
Hirst JA, Howick J, Aronson JK, et al. The need for randomization in animal trials: an overview of systematic reviews. PLoS One. 2014;9(6):e98856.
EQUATOR Network and UK EQUATOR Centre. Reporting guidelines for main study types. Equator Network website. Accessed March 30, 2021. https://www.equator-network.org
Pinchbeck GL, Archer DC. How to critically appraise a paper. Equine Vet Educ. 2020;32(2):104-109.a

For global readers, a calculator to convert laboratory values, dosages, and other measurements to SI units can be found here.

All Clinician's Brief content is reviewed for accuracy at the time of publication. Previously published content may not reflect recent developments in research and practice.

Material from Clinician's Brief may not be reproduced, distributed, or used in whole or in part without prior permission of Educational Concepts, LLC. For questions or inquiries please contact us.

Podcasts

Clinician's Brief:
The Podcast

Listen as host Alyssa Watson, DVM, talks with the authors of your favorite Clinician’s Brief articles. Dig deeper and explore the conversations behind the content here.

Clinician's Brief provides relevant diagnostic and treatment information for small animal practitioners. It has been ranked the #1 most essential publication by small animal veterinarians for 9 years.*

*2007-2017 PERQ and Essential Media Studies

It's Free & Simple

Delivered to Your Inbox

Newsletter

Join the Conversation

Facebook

Twitter

Home

Global Edition

About Us

Employment

Capsules

Advertise