What else should be appraised?

The level of evidence is a good indicator of how prone to bias the study is likely to be due to its design.

In most instances, however, there is overlap between the different levels, for example when looking at treatment, a well-designed cohort study may provide better evidence than a poorly designed controlled trial. Therefore, the appraisal should address aspects including the sample size, enrolment criteria, allocation, blinding, statistical methodologies and objectiveness in the discussion of results. Within the different study designs, you should assess possible biases that may have influenced the study outcomes.

FACT

You do not need to be a research scientist or a statistician to appraise the literature!

From a practical point of view, when you read papers, you should focus on the major issues that determine the quality of information. If you look through the literature, you will find that many articles are biased by poor reporting of crucial information like age and medical history of the enrolled animals, by inappropriate definitions or diagnoses of diseases, or by a lack of (or inappropriate) control groups (Dean 2013). This means that it is not your task to check or be familiar with every statistical procedure, but rather to be aware of key issues relevant to specific study designs. Below you can find, listed by types of EBVM questions, some resources provided by RCVS Knowledge and other organisations that highlight the key considerations to critically appraise different study designs.

Treatment

Evidence related to treatment often relates to intervention studies or randomised controlled trials. Occasionally, systematic reviews may address questions of treatment choice. There are a number of resources to guide you in appraising these study types:

RCVS Knowledge Appraisal Toolkit

Centre for Evidence-based Medicine Guidelines for Appraisal

Prognosis

Prognosis relates to the course of disease over time, and evidence related to prognosis can provide an invaluable source for the clinician when advising their client on the likely course of disease. Frequently, evidence related to prognosis is derived from cohort studies, and hence appraisal of this study type is particularly relevant in this setting.

RCVS Knowledge Appraisal Toolkit

Center for Evidence-based Medicine Guidelines for Appraisal

Risk factors

Evidence related to risk factors can be relevant to the practicing veterinarian when advising on the likelihood of disease given the specific animal’s characteristics, diet, environment, treatment, etc. When making judgements in relation to the likely casual association between a reported risk factor and disease, often the observational studies will need to be interpreted. Evidence from cohort, case-control and cross-sectional studies will need to be appraised.

RCVS Knowledge Appraisal Toolkit

Diagnosis

Veterinarians use diagnostic tests on a daily basis, yet often do not fully appreciate the evidence (or lack thereof) underpinning those tests when making major decisions for the animals they treat. Interpretation of evidence related to diagnostic tests is an important area of EBVM that can be overlooked.

Center for Evidence-based Medicine Guidelines for Appraisal

External validity

Another question that should be taken into account when appraising studies is whether the subjects (e.g. animals or specimens) used in the study are representative of the patient you are considering. Can the study results be transferred to your patient? Or are the subjects in the study you are considering so dissimilar that the results of that study do not apply to your patient?

The external validity of the paper is part of the whole assessment. It is important to understand the difference between the (average) effect on a population as compared to the effect for an individual patient. For example, on average, the effect could be beneficial, but be sure you consider that this average effect does not mean that it will produce that outcome every time in a given individual.