Making Sense of Medical Evidence: Participant'S Workbook

Making Sense of
Medical Evidence
An Interactive Workshop Advanc
Participant’s workbook
Advancing healthcare education

Table of Contents
1 Program overview
1 Evidence‑Based Medicine Journal Clubs
1 Program objectives
2 Part 1: Understanding medical evidence
4 Part 2: Understanding national guideline

development and evidence‑based
recommendations
6 Workshop evaluation form
6 Certificate of attendance
6 References
The information contained in this material is intended to be used for general guidance only and should not be relied on as professional
advice or instruction. Whilst every attempt has been made to ensure information is accurate, no guarantee can be provided as to the
accuracy of this material or its application in any particular situation. Practitioners are urged to seek appropriate professional guidance on
a case-by-case basis. Allori Pty Ltd does not accept responsibility or liability for reliance on any information contained herein.
© Allori Pty Ltd 2009 ALBOM.2.3

www.allori.edu.au
All rights reserved. No part of this publication is to be www.allori.edu.au Head office:
reproduced or transmitted in any form or by any means,
Head office: Level 1, 231 Burwood Rd, Hawthorn, VIC 3122
electronic, mechanical, photocopying, recording or otherwise,
Level 1, 231 Burwood Rd, Hawthorn, VIC offices
Allori has 3122 in Melbourne and Sydney, Australia.
without prior written permission of the Publisher.
Allori has offices in Melbourne and Sydney, Australia.
Sydney office: Melbourne office:
This educational initiative has been developed by Allori and is Sydney office: Melbourne
Phone: 1 3 0 0 office:
688 048 Phone: +61 3 8862 4700
supported by Boehringer Ingelheim. Phone: 1 3 0 0 6 8 8 0 4 8 Fax: Phone:+61 +61 3 8862
2 8572 82324700 Fax: +61 3 8610 0345
Fax: +61 2 8572 8232 Fax: +61 3 8610 0345
Program overview
Keeping abreast of the medical literature and understanding published clinical studies is an important
component of maintaining best practice and implementing practice change. As new studies emerge it’s
important to understand the significance of new results and their potential impact on clinical practice.
The Making Sense of Medical Evidence Interactive

Workshop is aimed at providing GPs with background Making Sense of Medical Evidence Interactive
information to the whole evidence‑based medicine Workshop will take 3 hours to complete.
process, by providing knowledge and encouragement
on how best to critically appraise the published
Earning each participant 6 RACGP Quality
literature to solve clinical questions, through the use of
Assurance and Continuing Professional
practical, validated and readily‑available tools.
Development (QA&CPD) category 2 points.
This workshop centres on an interactive and
participative meeting in which a recently published
clinical study or guideline is used as a case study.
At the end of the workshop participants will have This interactive workshop is to be held in Sydney,
the knowledge to effectively carry out a critical Melbourne, Perth, Adelaide and Brisbane during
appraisal of a clinical study or guideline, and have a 2009, as a 3‑hour meeting.
new found confidence in applying published data to
clinical practice.
Evidence‑Based Medicine Program objectives

Journal Clubs »» Understand the principles of evidence‑based
medicine
Recently, a new initiative – Evidence‑Based »» Apply the critical appraisal process to clinical
Medicine Journal Clubs (EBMJCs) – has been research findings, including clinical trials and
launched by the Royal Australian College of General guidelines
Pactitioner (RACGP). This new initiative provides »» Improve knowledge of, and interpretation of,
‘an opportunity for clinical teams to solve clinical statistical results
questions in a peer supported environment’. »» Recognise the best evidence
Allori has found that participation in the Making »» Apply the best evidence to clinical practice to
Sense of Medical Evidence Interactive Workshop assure clinical effectiveness
stimulates clinicians to subsequently undertake their »» Discuss the benefits of evidence‑based medicine
own EBMJCs. GPs are recommended to complete a »» Increase confidence to participate in, and/or
critical thinking course before participating in, or facilitate an evidence‑base medicine journal club
facilitating an EBMJC – an activity that the RACGP
is actively promoting, and can highly reward with
up to 40 Category 1 QA & CPD points.
For more information on RACGP EBMJCs,

please refer to the EBM Journal Club Application
Guide (2008–2010), and the QA&CPD Program
Handbook (2008–2010 triennium) for general
information and specific examples of EBMJCs.
Participant’s workbook page 1

Part 1: Understanding medical evidence
Evidence‑based medicine can be described as the
integration of best research evidence with clinical Helpful hint
expertise and patient values.1‑3
What is evidence‑based medicine?
Levels of evidence are a way of providing a
framework for handling or pigeon‑holing evidence. Evidence‑based medicine (EBM) is the
Different bodies recommend slightly different conscientious, explicit and judicious use of
designations of these ‘levels’ but the principles are current best evidence in making decisions about
similar. For example, The National Health and the care of individual patients. 2,3
Medical Research Council (NHMRC) has four
levels of evidence (I, II, III‑1, III‑2, III‑3 and IV).3,4
Critical appraisal fits into the evidence‑based medicine process by using published evidence to answer
a patient‑specific question, and help make a better clinical decision. This process involves searching the
literature, critically appraising it, and applying it.1,3
1. Asking patient‑specific questions in 2. Searching the literature

the PICO format
About 30,000 biomedical journals exist; these
Population, Patient, or Problem: What is the journals publish 3–6 million papers every year.
population or patient group? What are the most There are also over 20,000 reports of clinical trials
important characteristics of the patient? This may every year.3
include the primary problem, the disease, or a
The largest and probably best known general
co‑existing condition. Sex, age or ethnic background
biomedical database is MEDLINE, which
may also be relevant to the diagnosis or treatment of
is available free using the PubMED website.
a disease.
MEDLINE indexes over 4,800 journals, has over
Intervention: What do you want to do for the 12 million records dating back to 1996 and is
patient? What intervention or exposure are produced by the US National library of Medicine.
you considering (e.g. test, drug, surgery). What
Use the internet to find papers:
intervention strategy were study participants
exposed to? »» PubMED: www.ncbi.nlm.nih.gov/entrez
»» Cochrane Library: www.thecochranelibrary.com
Comparison to the intervention, or control: Are »» Bandolier: www.ebandolier.com
you trying to decide between two drugs, a drug and
»» US National Library of Medicine:
placebo, or two diagnostic tests?
www.nlm.nih.gov/entrez
Outcome(s) of interest: What are you trying to do »» UK National Library of Health:
for the patient (e.g. relieve or eliminate symptoms, www.library.nhs.uk
prolong life, reduce the number of adverse events, »» Embase: www.embase.com
improve functional performance), i.e. what do
you want to happen or prevent, AND/OR patient NB: The RACGP library offers GP members,
concern.1,3 services to assist with searching literature.
3. Using MeSH terms to search MEDLINE
Bibliographic databases contain records of published articles with content allocated to distinct fields e.g.
title, author, abstract. When the article is indexed it is also allocated a number of terms that describes the
article in terms of the population, intervention, comparator and outcomes. In MEDLINE, these are called
MeSH (Medical Subject Headings) descriptors. Key words can then be combined using Boolean operators,
such as AND, OR and NOT. For example, combining two key words with AND will retrieve records that
contain both words.3
page 2 Participant’s workbook
4. Understanding clinical trials
Interpreting p‑values: Understanding risk:

»» A p‑value is the probability (ranging from 0–1) »» Relative risk (RR) is the ratio of risk in the
that the results observed in a study could have treatment group (ART) to risk in the control
occurred by chance. By convention, a p‑value of group (ARC)
0.05 or below is accepted as being statistically
significant (i.e. statistical significance can occur [RR = ART / ARC ].5
by chance 1 time in 20), which is not very
unlikely. A more stringent rule is to use a p‑value »» Relative risk reduction (RRR) is 1 minus the
of 0.01 or below.3 relative risk.
Interpreting confidence intervals (CIs): [RRR = (ARC‑ART) / ARC].5
»» Confidence intervals provide a range around the
observed effect size, which can be a statistical
output, such as: an odds ratio, relative risk Helpful hint
(RR) or number needed to treat (NNT).
Conventionally, confidence intervals are created Use RR to interpret clinical trial results
at the 95% level (i.e. 95% of the time, properly
constructed CIs should contain the true value of If the RR or the odds ratio = 1, or the CI = 1,
the variable of interest).3 then there is no significant difference between
treatment groups.
Helpful hint If the RR > 1, and the CI does not include 1,
events are significantly MORE likely in the
Use CIs just like p‑values
treatment than the control group.
CIs can be used to look at statistical significance,
If the RR < 1, and the CI does not include
just like p‑values. If the 95% CI includes 1,
1, events are significantly LESS likely in the
infinity or zero, this is non‑significance at the
treatment than the control group.5
5% level (so ‘p’ is > 0.05).3
5. Using the Critical Appraisal Skills Programme (CASP) tool
Please carry out the following during the Part 1 workshop:

»» Nominate a spokesperson from your small group »» Consider the clinical significance of this paper,
(to later provide feedback to the larger group) and how these results may apply to your clinical
»» Using your CASP tool, discuss, appraise and
6 practice and/or patients.
complete your assigned CASP questions for the
clinical paper – The ONTARGET Investigators.
Telmisartan, ramipril, or both in patients at high Helpful hint
risk for vascular events.
PICO question:
NEJM 2008;358(15):1547‑1559.
In patients with vascular disease or high-risk
Refer to the ‘Understanding clinical trials’ diabetes (without heart failure) is an ARB, such
section to help refresh your memory of key as telmisartan as effective as an ACE inhibitor,
points covered in the Part 1 presentation such as Ramipril in reducing adverse cardiac
outcomes?
»» Consider whether the PICO question was
answered?

Part 2: Understanding national guideline development
and evidence‑based recommendations
Where possible, clinical practice guidelines should be
based on the systematic identification and synthesis Helpful hint
of best available scientific evidence. Guidelines are
becoming one of the critical links between the best What are clinical guidelines?
available evidence and good clinical practice.7
Systematically developed statements to assist
Guidelines should not be taken ‘on trust’ without practitioner and patient decisions about
an examination of how the recommendations have appropriate healthcare for specific clinical
been made, and whether good practice for guideline circumstances.8,9
development has been followed. “Guidelines still
need to be looked at with a degree of scepticism, and
bearing in mind that not all patients are the same.”3
Using the Appraisal of Guidelines Research & Evaluation (AGREE) Instrument
Please carry out the following during the Part 2

workshop: Helpful hint
»» Nominate a spokesperson from your small group How to answer the AGREE item questions: 10
(to later provide feedback to the larger group)
»» Using your AGREE Instrument,10 discuss, If you are confident that the criterion has
appraise and complete your assigned AGREE been fully met then you should answer
question(s) which refer to Part 7.3 (Secondary ‘strongly agree’.
Prevention and Anti-platelet Therapy) of
the Clinical Guidelines for Acute Stroke If you are confident that the criterion has
Management, National Stroke Foundation not been fulfilled at all or if there is no
2007.11 Note the reason for your responses information available then you should answer
‘strongly disagree’.
Due to time constraints, only five out of the
23 item questions (numbers 11, 15, 16, 17 If you are unsure that a criterion has been
and 21) will be covered during the small fulfilled, for example because the information
group discussions, but your moderator is unclear or because only some of the
has already appraised the full National recommendations fulfill the criterion, then you
Stroke Guidelines using all 23 AGREE item should answer ‘agree’ or ‘disagree’, depending on
questions, and will present these findings in the extent to which you think the issue has been
the Part 2 large discussion session addressed.
»» What is your overall assessment: Would you

recommend these guideline for use in clinical
To help answer your particular item question,
practice?
extracts from the AGREE user guide addresses the
issues and concepts for consideration.10
The full AGREE user guide for all 23 item questions

is available at www.agreecollaboration.org.

Domain 3: Rigour of development
Item 11. The health benefits, side effects and risks have been considered in formulating
the recommendations.
The guideline should consider health benefits, side effects, and risks of the recommendations. For example,
a guideline on the management of breast cancer may include a discussion on the overall effects on various
final outcomes. These may include: survival, quality of life, adverse effects, and symptom management or a
discussion comparing one treatment option to another. There should be evidence that these issues have been
addressed.
Domain 4: Clarity and presentation

Item 15. The recommendations are specific Item 16. The different options for
and unambiguous. management of the condition are
clearly presented.
A recommendation should provide a concrete
and precise description of which management is A guideline should consider the different possible
appropriate in which situation and in what patient options for screening, prevention, diagnosis or
group, as permitted by the body of evidence. treatment of the condition it covers. These possible
options should be clearly presented in the guideline.
An example of a specific recommendation is: For example, a recommendation on the management
Antibiotics have to be prescribed in children of of depression may contain the following alternatives:
2 years or older with acute otitis media if the a) Treatment with TCA, b) Treatment with SSRI, c)
complaint last longer than three days or if the Psychotherapy, d) Combination of pharmacological
complaint increase after the consultation despite and psychological therapy.
adequate treatment with painkillers; in these cases
amoxycillin should be given for 7 days (supplied Item 17. Key recommendations are
with a dosage scheme). easily identifiable.
An example of a vague recommendation is: Users should be able to find the most relevant
Antibiotics are indicated for cases with an abnormal recommendations easily. These recommendations
or complicated course. However, evidence is not answer the main clinical questions that have been
always clear cut and there may be uncertainty about covered by the guideline. They can be identified in
the best management. In this case the uncertainty different ways. For example, they can be summarised
should be stated in the guideline. in a box, typed in bold, underlined or presented as
flow charts or algorithms.
Domain 5: Applicability
Item 21. The guideline presents key review criteria for monitoring and/or audit purposes.
Measuring the adherence to a guideline can enhance its use. This requires clearly defined review criteria that
are derived from the key recommendations in the guideline. These should be presented. Examples of review
criteria are: a) The HbA1c should be <8.0%, b) The level of diastolic blood pressure should be <95 mmHg,
c) If complaints of acute otitis media lasts longer than three days amoxicillin should be prescribed.

Workshop evaluation form
The purpose of the evaluation form is to get feedback from participants on the
style, format and quality of the workshop. This information is important and
will help Allori to continually improve future educational programs. Completion
of the evaluation form is compulsory to the participant gaining their RACGP
QA&CPD points.
Certificate of attendance
Allori will email a personalised certificate of attendance to each participant
within a few weeks of the completion of the workshop, and upon receipt of the
meeting evaluation form.
References:
1. O’Brian K. Teaching Evidence Based Practice Workshop 2004. School of
Population Health, University of Melbourne. Available at: www.mh.org.au/
royal_melbourne_hospital/secure/downloadfile.asp?fileid=1001812.
2. Sackett DL et al. Evidence Based Medicine. How to practice and teach EBM.
Second Edition, Edinburgh. Churchill Livingstone 2000.
3. Moore A, McQuay H. Bandolier’s Little Book of Making Sense of the Medical

Evidence. Oxford University Press 2006.
4. H
ow to use the evidence: assessment and application of scientific evidence.
NHMRC. Commonwealth of Australia 2000.
5. Clinical Evidence: Helping you to practise evidence-based medicine. Available

at http://clinicalevidence.bmj.com/ceweb/resources/calculate_risk.jsp.
6. Public Health Resource Unit, UK, available at:

http://www.phru.nhs.uk/Pages/PHD/resources.htm.
7. A guide to the development, implementation and evaluation of clinical practice

guidelines. National Health and Medical Research Council (NHMRC) 1998.
Available at: www.nhmrc.gov.au/publications/synopses/cp30syn.htm.
8. SIGN 50. A guideline developer’s handbook. Scottish Intercollegiate

Guidelines Network. Revised Edition January 2008. Available at:
www.sign.ac.uk/pdf/sign50.pdf.
9. Field M, Lohr K. Institute of Medicine Committee to advise the Public Health

Service on Clinical Practice Guidelines. Clinical practice guidelines: directions
for a new program. Washington (DC): National Academy Press 1990.
10. Appraisal of Guidelines for Research & Evaluation (AGREE) Instrument.

The AGREE Collaboration 2001. Available at: www.agreecollaboration.org.
11. Clinical Guidelines for Acute Stroke Management. National Stroke

Foundation 2007 (Part 7, Secondary Prevention). Available at:
www.strokefoundation.com.au.

Notes:
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��

Notes:
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��

Notes:
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��

Notes:
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��

www.allori.edu.au
www.allori.edu.au Head office:
Head office: Level 1, 231 Burwood Rd, Hawthorn, VIC 3122
Level 1, 231 Burwood Rd, Hawthorn, VIC offices
Allori has 3122 in Melbourne and Sydney, Australia.
Allori has offices in Melbourne and Sydney, Australia.
Sydney office: Melbourne office:
Sydney office: Melbourne
Phone: 1 3 0 0 office:
688 048 Phone: +61 3 8862 4700
Phone: 1 3 0 0 6 8 8 0 4 8 Fax: Phone:+61 +61 3 8862
2 8572 82324700 Fax: +61 3 8610 0345
Fax: +61 2 8572 8232 Fax: +61 3 8610 0345

Making Sense of Medical Evidence: Participant'S Workbook

Uploaded by

Copyright:

Available Formats

Making Sense of Medical Evidence: Participant'S Workbook

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Making Sense of Medical Evidence: Participant'S Workbook

Uploaded by

Copyright:

Available Formats

Making Sense of

Advancing healthcare education

1 Evidence‑Based Medicine Journal Clubs

2 Part 1: Understanding medical evidence

4 Part 2: Understanding national guideline

6 Workshop evaluation form

© Allori Pty Ltd 2009 ALBOM.2.3

The Making Sense of Medical Evidence Interactive

Evidence‑Based Medicine Program objectives

For more information on RACGP EBMJCs,

Participant’s workbook page 1

1. Asking patient‑specific questions in 2. Searching the literature

3. Using MeSH terms to search MEDLINE

Interpreting p‑values: Understanding risk:

5. Using the Critical Appraisal Skills Programme (CASP) tool

Please carry out the following during the Part 1 workshop:

Participant’s workbook page 3

Using the Appraisal of Guidelines Research & Evaluation (AGREE) Instrument

Please carry out the following during the Part 2

»» What is your overall assessment: Would you

The full AGREE user guide for all 23 item questions

page 4 Participant’s workbook

Domain 4: Clarity and presentation

Participant’s workbook page 5

3. Moore A, McQuay H. Bandolier’s Little Book of Making Sense of the Medical

5. Clinical Evidence: Helping you to practise evidence-based medicine. Available

6. Public Health Resource Unit, UK, available at:

7. A guide to the development, implementation and evaluation of clinical practice

8. SIGN 50. A guideline developer’s handbook. Scottish Intercollegiate

9. Field M, Lohr K. Institute of Medicine Committee to advise the Public Health

10. Appraisal of Guidelines for Research & Evaluation (AGREE) Instrument.

11. Clinical Guidelines for Acute Stroke Management. National Stroke

page 6 Participant’s workbook

Participant’s workbook page 7

page 8 Participant’s workbook

Participant’s workbook page 9

page 10 Participant’s workbook

You might also like

4 Part 2: Understanding national guideline

1. Asking patient‑specific questions in 2. Searching the literature

3. Moore A, McQuay H. Bandolier’s Little Book of Making Sense of the Medical

5. Clinical Evidence: Helping you to practise evidence-based medicine. Available

6. Public Health Resource Unit, UK, available at:

7. A guide to the development, implementation and evaluation of clinical practice

8. SIGN 50. A guideline developer’s handbook. Scottish Intercollegiate

9. Field M, Lohr K. Institute of Medicine Committee to advise the Public Health

10. Appraisal of Guidelines for Research & Evaluation (AGREE) Instrument.

11. Clinical Guidelines for Acute Stroke Management. National Stroke