Learning Resources

Linguistic strategies for improving informed consent in clinical trials among low health literacy patients

Evidence-based guidance on how to improve informed consent processes for patients being invited to participate in clinical research.

| 0 Comments | Evaluated

Informed Health Choices Podcasts

Each episode includes a short story with an example of a treatment claim and a simple explanation of a Key Concept used to assess that claim

| 1 Comment | Evaluated

Informed Health Choices Primary School Resources

A textbook and a teachers’ guide for 10 to 12-year-olds. The textbook includes a comic, exercises and classroom activities.

| 0 Comments | Evaluated

Ebm@school – a curriculum of critical health literacy for secondary school students

A curriculum based on the concept of evidence-based medicine, which consists of six modules.

| 0 Comments | Evaluated

Confidence Intervals – CASP

The p-value gives no direct indication of how large or important the estimated effect size is. So, confidence intervals are often preferred.

| 0 Comments | Evaluated

Know Your Chances

This book has been shown in two randomized trials to improve peoples' understanding of risk in the context of health care choices.

| 0 Comments | Evaluated

Philosophy for Children (P4C)

P4C promotes high-quality classroom dialogue in response to children’s own questions about shared stories, films and other stimuli.

| 0 Comments | Evaluated

Thinking, talking, doing science

An experimental educational intervention in teaching science at primary schools.

| 0 Comments | Evaluated

Evidence for everyday health choices

A 17-min slide cast by Lynda Ware, on the history of EBM, what Cochrane is, and how to understand the real evidence behind the headlines.

| 0 Comments

Sunn Skepsis

Denne portalen er ment å gi deg som pasient råd om kvalitetskriterier for helseinformasjon og tilgang til forskningsbasert informasjon.

| 0 Comments

Dancing statistics: Explaining variance

A 5-minute film demonstrating the statistical concept of variance through dance.

| 0 Comments

Dancing statistics: sampling & standard error

A 5-minute film demonstrating the statistical concept of sampling and standard error through dance.

| 0 Comments

Dancing statistics: correlation

A 4-minute film demonstrating the statistical concept of correlation through dance.

| 0 Comments

Julia Belluz – Lessons from the trenches of evidence-based health journalism at Vox.com

20-minute talk by Julia Belluz on the need to bring the cultures of health journalism and EBM together.

| 0 Comments

Don’t jump to conclusions, #Ask for Evidence

An introduction to the ‘Ask for Evidence’ initiative launched by ‘Sense about Science’ in 2016.

| 0 Comments

Battling bad science

15-min talk by Ben Goldacre on ways evidence can be distorted.

| 0 Comments

How can you know if the spoon works?

Short, small group exercise on how to design a fair comparison using the "claim" that a spoon helps retain the bubbles in champagne.

| 0 Comments

English National Curriculum vs Key Concepts – Key Stage 3

A linked spreadsheet showing how the Key Concepts map to the Science National Curriculum in England at Key Stage 3 (ages 11-14).

| 0 Comments

DRUG TOO

James McCormick with another parody/spoof of the Cee Lo Green song ‘Forget You’ to prompt scepticism about many drug treatments.

| 0 Comments

Calling Bullshit Syllabus

Carl Bergstrom's and Jevin West's nice syllabus for 'Calling Bullshit'.

| 0 Comments

The surrogate battle – is lower always better?

James McCormick recruits a furious Fuhrer to point out that taking drugs to lower surrogate measures of ill health is a confidence trick.

| 0 Comments

Tom Hanks and Type 2 Diabetes

A 50-minute illustrated talk by James McCormack prompted by Tom Hanks’ announcement that he had been diagnosed with Type 2 diabetes.

| 0 Comments

Bohemian Polypharmacy

James McCormack recruits help from Queen to warn of the dangers of ‘Bohemian Polypharmacy’ in music.

| 0 Comments

Choosing Wisely

James McCormack using song and dance to warn about the negative effects of overtreatment.

| 0 Comments

Like a bridge overdiagnosis

James McCormack with another of his brilliant parodies, warning about the dangers of becoming inappropriately labelled as ill.

| 0 Comments

‘Tricks to help you get the result you want from your study (S4BE)

Inspired by a chapter in Ben Goldacre’s ‘Bad Science’, medical student Sam Marks shows you how to fiddle research results.

| 0 Comments

Reporting the findings: Absolute vs relative risk

Absolute Differences between the effects of two treatments matter more to most people than Relative Differences.

| 0 Comments

It’s just a phase

A resource explaining the differences between different trial phases.

| 0 Comments

Strictly Cochrane: a quickstep around research and systematic reviews

An interactive resource explaining how systematic and non-systematic reviews differ, and the importance of keeping reviews up to date.

| 0 Comments

The Princess and the p-value

An interactive resource introducing reporting and interpretation of statistics in controlled trials.

| 0 Comments

A Study Suggests…

An interactive resource introducing randomised controlled trials.

| 0 Comments

Teach Yourself Cochrane

Tells the story behind Cochrane and the challenges finding good quality evidence to produce reliable systematic reviews.

| 3 Comments

Explaining the mission of the AllTrials Campaign (TED talk)

Half the clinical trials of medicines we use haven’t been published. Síle Lane shows how the AllTrials Campaign is addressing this scandal.

| 0 Comments

Fish oil in the Observer: the return of a $2bn friend

Ben Goldacre draws attention to people’s wish to believe that a pill can be the solution to a complicated problem.

| 0 Comments

Building evidence into education

Ben Goldacre explains why appropriate infrastructure is need to do clinical trials of sufficient rigour and size to yield reliable results.

| 0 Comments

Anecdotes are great – if they convey data accurately

Ben Goldacre gives examples of how conclusions based on anecdotes and biased research can be damagingly misleading.

| 0 Comments

Studies of studies show that we get things wrong

Ben Goldacre gives examples of how conclusions based on anecdotes and biased research can be damagingly misleading.

| 0 Comments

Dodgy academic PR

Ben Goldacre: 58% of all press releases by academic institutions lacked relevant cautions and caveats about the methods and results reported

| 0 Comments

All bow before the mighty power of the nocebo effect

Ben Goldacre discusses nocebo effects, through which unpleasant symptoms are induced by negative expectations, despite no physical cause.

| 0 Comments

How do you regulate Wu?

Ben Goldacre finds that students of Chinese medicine are taught (on a science degree) that the spleen is “the root of post-heaven essence”.

| 0 Comments

Science is about embracing your knockers

Ben Goldacre: “I don’t trust claims without evidence, especially not unlikely ones about a magic cream that makes your breasts expand.”

| 0 Comments

NMT are suing Dr Wilmshurst. So how trustworthy are this company? Let’s look at their website…

Ben Goldacre celebrates Peter Wilmshurst, the doctor who blew the whistle on research misconduct in a study to which he was a contributor.

| 0 Comments

Over there! An 8 mile high distraction made of posh chocolate!

Ben Goldcare illustrates strategies used by vested interests to discredit research with ‘inconvenient’ results.

| 0 Comments

Brain imaging studies report more positive findings than their numbers can support. This is fishy.

Ben Goldacre explores how twice as many positive findings as could realistically have been expected from the data reported may have occurred

| 0 Comments

What if academics were as dumb as quacks with statistics?

Ben Goldacre introduces a statistical error that appears in about half of all the published papers in academic neuroscience research.

| 0 Comments

The strange case of the magnetic wine

Ben Goldacre shows how claims for the wine-maturing effects of magnets could be assessed with 50 people in an evening.

| 0 Comments

Screen test

Ben Goldacre notes that even if people realize that screening programmes have downsides, people don’t regret being screened.

| 0 Comments

Sampling error, the unspoken issue behind small number changes in the news

Ben Goldacre stresses the importance of taking account of “sampling variability” and confidence intervals.

| 0 Comments

The certainty of chance

Ben Goldacre reminds readers how associations may simply reflect the play of chance, and describes Deming’s illustration of this.

| 0 Comments

Publish or be damned

Ben Goldacre points out the indefensible practice of announcing conclusions from research studies which haven’t been published.

| 0 Comments

How myths are made

Ben Goldacre draws attention to Steven Greenberg’s forensically based illustration of citation biases.

| 0 Comments

Foreign substances in your precious bodily fluids

Ben Goldacre points out that there is no evidence giving strong support either to water fluoridationists or to anti-fluoridationists.

| 0 Comments

Is it okay to ignore results from people you don’t trust?

Ben Goldacre: why it’s important to consider vested interests when judging research, but not to dismiss research by people you don’t like.

| 0 Comments

Cherry picking is bad. At least warn us when you do it.

Ben Goldacre illustrates how biased ‘cherry picking’ and choosing from the relevant evidence can result in unreliable conclusions.

| 0 Comments

Why won’t Professor Susan Greenfield publish this theory in a scientific journal?

Ben Goldacre challenges senior Oxford professor to publish the evidence supporting her claim that computer games cause dementia in children.

| 0 Comments

The Man Who Swallowed The Pea, and Other Tall Tales

Tamara Ingamells’ lesson plan using the claim that MMR vaccination causes autism to help teenagers understand the importance of biases.

| 0 Comments

Weasels Are on the Loose

Weaseling is the use of certain words to weaken a claim, so that the author can say something without actually saying it and avoid criticism

| 0 Comments

Dragon Lesson Plan to investigate multivariate categorical data

Investigating multivariate data by sorting and organising a set of dragon cards to uncover information about the set.

| 0 Comments

Fast Stats to explain absolute risk, relative risk and Number Needed to Treat (NNT).

A 15-slide presentation on ‘Fast Stats’ to explain absolute risk, relative risk and Number Needed to Treat (NNT) prepared by PharmedOut.

| 0 Comments

Unsubstantiated and overstated claims of efficacy

A 32-slide presentation on misleading advertisements and FDA warnings prepared by PharmedOut.

| 0 Comments

Critical appraisal

University of New South Wales Medical Statistics Tutorial 4 addresses Critical Appraisal.

| 0 Comments

Probability and tests of statistical significance

University of New South Wales Medical Statistics Tutorial 6 addresses ‘Probability and tests of statistical significance’.

| 0 Comments

Bias – the biggest enemy

University of New South Wales Medical Stats Online Tutorial 5 addresses ‘Bias - the biggest enemy’.

| 0 Comments

Generation R – Pictionary research activity

GenerationR’s version of Pictionary using research concepts instead of usual game cards, allocated in different levels of difficulty.

| 0 Comments

Generation R – Clinical trials card-sorting exercise

Card-sorting exercise developed by GenerationR to familiarise children and young people with jargon terms used by clinical researchers.

| 0 Comments

Clinical Research Workshop

‘Clinical Research Workshop’ developed for young people by the Centre of the Cell.

| 0 Comments

Introduction to Evidence-Based Medicine

Bill Caley’s 26 slides with notes used as an ‘Introduction to Evidence-Based Medicine’.

| 0 Comments

Applying evidence to patients

A 27-minute talk on ‘Applying Evidence to Patients’, illustrated by 17 slides, with notes.

| 0 Comments

2×2 tables and relative risk

A 10-min talk on ‘2x2 tables and Relative Risk’, illustrated by 14 slides, with notes.

| 0 Comments

Appraisal of evidence and interpretation of results

A 14-min talk on ‘Appraisal of the Evidence and Interpretation of the Results’, illustrated by 19 slides, with notes.

| 0 Comments

Basic principles of randomised trials, and validity

A 8-min talk on ‘Basic principles of Randomised Trials, and Validity’, illustrated by 15 slides, with notes.

| 0 Comments

Defining clinical questions

An 8-min talk on ‘Defining Clinical Questions’ illustrated by 10 slides, with notes.

| 0 Comments

A way to teach about systematic reviews

81 slides used by David Nunan (Centre for Evidence-Based Medicine, Oxford) to present ‘A way to teach about systematic reviews’.

| 0 Comments

Applying the evidence

Six key slides produced by the University of Western Australia on applying evidence in practice.

| 0 Comments

Appraising the evidence

Six key slides produced by the University of Western Australia to introduce critical appraisal.

| 0 Comments

The power of the placebo effect

Emma Bryce’s video presents information about placebo effects: treatments not supposed to have an effect but which make people feel better.

| 0 Comments

Detectives in the classroom

Five modules of materials for promoting epidemiology among high school students.

| 0 Comments

Not all scientific studies are created equally

David Schwartz dissects two types of studies that scientists use, illuminating why you should always approach claims with a critical eye.

| 1 Comment

Taking account of the play of chance

Differences in outcome events in treatment comparisons may reflect only the play of chance. Increased numbers of events reduces this problem

| 0 Comments

Quantifying uncertainty in treatment comparisons

Small studies in which few outcome events occur are usually not informative and the results are sometimes seriously misleading.

| 0 Comments

Bringing it all together for the benefit of patients and the public

Improving reports of research and up-to-date systematic reviews of reliable studies are essential foundations of effective health care.

| 0 Comments

Applying the results of trials and systematic reviews to individual patients

Paul Glasziou uses 28 slides to address ‘Applying the results of trials and systematic reviews to individual patients’.

| 0 Comments

10 Components of effective clinical epidemiology: How to get started

PDF & Podcast of 1-hr talk by Carl Heneghan (Centre for Evidence-Based Medicine, Oxford) on effective clinical epidemiology.

| 0 Comments

Making the most of the evidence in education

A pamphlet to guide people using research evidence when deliberating about educational policies.

| 0 Comments

Critical appraisal of clinical trials

Slides developed by Amanda Burls for an interactive presentation covering the most important features of well controlled trials.

| 0 Comments

Tipsheet for reporting on drugs, devices and medical technologies

Questions that will be familiar to reporters covering health and medicine.

| 0 Comments

Caffeine Soft Drinks affect Human Heart Rate. Lesson Plan

A lesson to illustrate how medical researchers study the effects of drugs on people.

| 0 Comments

Life saving maths: How does vaccination work?

Vaccinating a large enough proportion of children means everyone is protected, including those who can't be vaccinated.

| 0 Comments

How to work out whether bacon sandwiches are harmful

The headline said there is a 20% greater risk of getting bowel cancer if you eat bacon sandwiches! Are they right?

| 0 Comments

Explaining the unbiased creation of treatment comparison groups and blinded outcome assessment

A class were given coloured sweets and asked to design an experiment to find out whether red sweets helped children to think more quickly.

| 0 Comments

Investigating how to remove bacteria from hands

Investigate the best way to remove bacteria from your hands.

| 0 Comments

Tips for understanding Intention-to-Treat analysis

Ignoring non-compliance with assigned treatments leads to biased estimates of treatment effects. ITT analysis reduces these biases.

| 0 Comments

Tips for understanding Absolute vs. Relative Risk

Absolute Differences between the effects of two treatments matter more to most people than Relative Differences.

| 0 Comments

Applying Systematic Reviews

How useful are the results of trials in a systematic review when it comes to weighing up treatment choices for particular patients?

| 0 Comments

Systematic Reviews and Meta-analysis: Information Overload

None of us can keep up with the sheer volume of material published in medical journals each week.

| 0 Comments

Combining the Results from Clinical Trials

Chris Cates notes that emphasizing the results of patients in particular sub-groups in a trial can be misleading.

| 0 Comments

Tips for understanding Non-inferiority Trials

A non-inferiority experiment endeavours to show that a new intervention is ‘not unacceptably worse’ than the comparison intervention.

| 0 Comments

How Science Works

Definitions of terms that students have to know for 'How Science Works' and associated coursework, ISAs, etc

| 0 Comments

Cyagen is paying for citations

Pharmaceutical company Cyagen offers researchers and other writers $100 or more for citing their products in publications.

| 0 Comments

GenerationR – The importance of involving children and young people in research

3/3, 22-min video at the launch of GenerationR, a network of young people who advise researchers.

| 0 Comments

Generation R – The importance of medical research in children and young people

2/3, 35-min video at the launch of GenerationR, a network of young people who advise researchers.

| 0 Comments

Generation R – The need to reduce waste in clinical research involving children

1/3, 14-min video at the launch of GenerationR, a network of young people who advise researchers.

| 0 Comments

No Power, No Evidence!

This blog explains that studies need sufficient statistical power to detect a difference between groups being compared.

| 0 Comments

Beginners guide to interpreting odds ratios, confidence intervals and p values

A tutorial on interpreting odds ratios, confidence intervals and p-values, with questions to test the reader’s knowledge of each concept.

| 0 Comments

Sample Size matters even more than you think

This blog explains why adequate sample sizes are important, and discusses research showing that sample size may affect effect size.

| 0 Comments

What is it with Odds and Risk?

This blog explains odds ratios and relative risks, and provides the formulae for calculating both measures.

| 0 Comments

Preclinical animal studies: bad experiments cost lives

This blog notes that few therapies that treat disease in animals successfully translate into effective treatments for humans.

| 0 Comments

Surrogate Endpoints in EBM: What are the benefits and dangers?

What are surrogate outcomes, their pros and cons, and why you should be cautious in extrapolating from them to clinical decisions.

| 0 Comments

The Systematic Review

This blog explains what a systematic review is, the steps involved in carrying one out, and how the review should be structured.

| 0 Comments

The Mean: Simply Average?

This blog explains ‘the mean’ as a measure of average; describes how to calculate it; and flags up some caveats.

| 0 Comments

Publication Bias: An Editorial Problem?

A blog challenging the idea that publication bias mainly occurs at editorial level, after research has been submitted for publication.

| 0 Comments

The Bias of Language

Publication of research findings in a particular language may be prompted by the nature and direction of the results.

| 0 Comments

Defining Bias

This blog explains what is meant by ‘bias’ in research, focusing particularly on attrition bias and detection bias.

| 0 Comments

Balancing Benefits and harms

A blog explaining what is meant by ‘benefits’ and ‘harms’ in the context of healthcare interventions, and the importance of balancing them.

| 0 Comments

Data Analysis Methods

A discussion of 2 approaches to data analysis in trials - ‘As Treated’, and ‘Intention-to-Treat’ - and some of the pros and cons of each.

| 0 Comments

Defining Risk

This blog defines ‘risk’ in relation to health, and discusses some the difficulties in applying estimates of risk to a given individual.

| 0 Comments

Traditional Reviews vs. Systematic Reviews

This blog outlines 11 differences between systematic and traditional reviews, and why systematic reviews are preferable.

| 0 Comments

P Value in Plain English

Using simple terms and examples, this blog explains what p-values mean in the context of testing hypotheses in research.

| 0 Comments

Cancer Screening Debate

This blog discusses problems that can be associated with cancer screening, including over-diagnosis and thus (unnecessary) over-treatment.

| 0 Comments

Surrogate endpoints: pitfalls of easier questions

A blog explaining what surrogate endpoints are and why they should be interpreted cautiously.

| 0 Comments

Misconceptions about screening

Screening should not be for everyone or all diseases. It should only be offered when it is likely to do good than harm.

| 0 Comments

Making sense of randomized trials

A description of how clinical trials are constructed and analysed to ensure they provide fair comparisons of treatments.

| 0 Comments

Randomized Control Trials

1/2, 40-min lecture on randomized trials by Dr R Ramakrishnan (Lecture 25) for the Central Coordinated Bioethics Programme in India.

| 0 Comments

Compliance with protocol and follow-up in clinical trials

Denis Black’s 10-min, downloadable, PowerPoint presentation on compliance, follow up, and intention-to-treat analysis in clinical trials.

| 0 Comments

Clinical Significance – CASP

To understand results of a trial it is important to understand the question it was asking.

| 0 Comments

Statistical Significance – CASP

In a well-conducted randomized trial, the groups being compared should differ from each other only by chance and by the treatment received.

| 0 Comments

P Values – CASP

Statistical significance is usually assessed by appeal to a p-value, a probability, which can take any value between 0 and 1 (certain).

| 0 Comments

Making sense of results – CASP

This module introduces the key concepts required to make sense of statistical information presented in research papers.

| 0 Comments

Screening – CASP

This module on screening has been designed to help people evaluate screening programmes.

| 0 Comments

Systematic Reviews – CASP

This unit looks at Critical Appraisal of systematic reviews.

| 0 Comments

Randomised Control Trials – CASP

This module looks at the critical appraisal of randomised trials.

| 0 Comments

Common Sources of Bias

Bias (the conscious or unconscious influencing of a study and its results) can occur in different ways and renders studies less dependable.

| 0 Comments

5 reasons why you might not get the best healthcare

Five reasons why patients may not always get the best care available.

| 0 Comments

Tamiflu: securing access to medical research data

A campaign by researchers has shown that Roche spun the research on Tamiflu to meet their commercial ends.

| 0 Comments

MMR: the facts in the case of Dr Andrew Wakefield

This 15-page cartoon explains the events surrounding the MMR controversy, and provides links to the relevant evidence.

| 5 Comments

Los intervalos de confianza en investigación

¿Para qué sirven los intervalos de confianza en los estudios de investigación?

| 0 Comments

Estadísticamente significativo

¿Qué significa estadísticamente significativo?

| 0 Comments

Toma de Decisiones Compartidas

¿Por qué nosotros, los pacientes, debemos participar en la toma de decisiones médicas importantes?

| 0 Comments

Watson en busca de la evidencia

Cómic acerca de conflictos de intereses y búsqueda de información.

| 0 Comments

Taller de Tratamientos Mágicos

Taller de Tratamientos Mágicos

| 0 Comments

The need to compare like-with-like in treatment comparisons

Allocation bias results when trials fail to ensure that, apart from the treatments being compared, ‘like will be compared with like'.

| 0 Comments

Why avoiding differences between treatments allocated and treatments received is important

Knowledge of which treatments have been received by which study participants can affect adherence to assigned treatments and result in bias.

| 0 Comments

The need to avoid differences in the way treatment outcomes are assessed

Biased treatment outcome assessment can result if people know which participants have received which treatments.

| 0 Comments

Avoiding biased selection from the available evidence

Systematic reviews are used to identify, evaluate and summarize all the evidence relevant to addressing a particular question.

| 0 Comments

Preparing and maintaining systematic reviews of all the relevant evidence

Unbiased, up-to-date systematic reviews of all the relevant, reliable evidence are needed to inform practice and policy.

| 0 Comments

Dealing with biased reporting of the available evidence

Biased reporting of research occurs when the direction or statistical significance of results influences how research is reported.

| 0 Comments

Using the results of up-to-date systematic reviews of research

Trustworthy evidence from research is necessary, but not sufficient, to improve the quality of health care.

| 0 Comments

Why treatment comparisons must be fair

Fair treatment comparisons avoid biases and reduce the effects of the play of chance.

| 0 Comments

Introduction to JLL Explanatory Essays

Professionals sometimes harm patients by using inadequately evaluated treatments. Research addressing uncertainties can reduce this harm.

| 0 Comments

Avoiding biased treatment comparisons

Biases in tests of treatments are those factors that can lead to conclusions that are systematically different from the truth.

| 0 Comments

Bias introduced after looking at study results

Biases can be introduced when knowledge of the results of studies influences analysis and reporting decisions.

| 0 Comments

Reducing biases in judging unanticipated effects of treatments

As with anticipated effects of treatments, biases and the play of chance must be reduced in assessing suspected unanticipated effects.

| 0 Comments

Recognizing researcher/sponsor biases and fraud

The vested interests of researchers and organizations tend to be reflected in reports of treatment research in which they are involved.

| 0 Comments

Video games and health improvement: a literature review of randomized controlled trials

This is a critical appraisal of a non-systematic review of randomized trials of video games for improving health.

| 0 Comments

Why comparisons must address genuine uncertainties

Too much research is done when there are no genuine uncertainties about treatment effects. This is unethical, unscientific, and wasteful.

| 0 Comments

Interpreting 95% Confidence Intervals

Gilbert Welch’s 9-min video on how 95% confidence intervals relate to p values.

| 0 Comments

Surrogate markers may not tell the whole story

A webpage explaining the limitations of using surrogate outcome markers in clinical research.

| 0 Comments

Why treatment comparisons are essential

Formal comparisons are required to assess treatment effects and to take account of the natural course of health problems.

| 0 Comments

Why treatment uncertainties should be addressed

Ignoring uncertainties about the effects of treatments has led to avoidable suffering and deaths.

| 0 Comments

Motivational Deficiency Disorder – a satirical look at disease mongering

Ray Moynihan’s 4-min video on ‘Motivational Deficiency Disorder’, illustrating ‘disease-mongering’.

| 0 Comments

Making Sense of Screening

Screening tests can cause harm. This guide helps you to make sense of claims about screening for health conditions.

| 0 Comments

‘Ask for Evidence’ lesson plan

A lesson plan and resources to give 13-16 year olds the opportunity to explore if what they see, read, and hear is true.

| 0 Comments

What are systematic reviews?

A 3-min video by Jack Nunn and The Cochrane Consumers and Communication group for people unfamiliar with the concept of systematic reviews.

| 0 Comments

Interactive PowerPoint Presentation about Clinical Trials

An interactive Powerpoint presentation for people thinking about participating in a clinical trial or interested in learning about them.

| 0 Comments

Testing Treatments

Testing Treatments is a book to help the public understand why fair tests of treatments are needed, what they are, and how to use them.

| 0 Comments

Annals Graphic Medicine: How screening is portrayed in the media

A cartoon series addressing the theme "Earlier is not necessarily better".

| 0 Comments

Eureka!

Cherry picking the results of people in sub-groups can be misleading.

| 0 Comments

Effectiveness Delusions

Cherry picking the results of people in sub-groups can be misleading.

| 0 Comments

Cecil and those pellets again…

If possible, participants in clinical trials should not know which treatment they are receiving.

| 0 Comments

Catch 22 – clinical trials edition

Fair comparisons of treatments in animals or highly selected groups of people may not be relevant.

| 0 Comments

House Calls Please

Beware reliance on 'experience' without reference to relevant evidence.

| 0 Comments

Composite Outcomes

Fair comparisons of treatments should measure important outcomes and avoid dependence on surrogate outcome measures.

| 0 Comments

Biomarkers unlimited

Fair comparisons of treatments should measure important outcomes and avoid dependence on surrogate outcome measures.

| 0 Comments

Data speaking for Itself

Cartoon about the many different ways of analyzing data

| 0 Comments

Department of Epidemiology

Presenting relative measures of treatment effects without presenting absolute measures can be misleading.

| 0 Comments

Goldilocks

Cartoon and blog about how poorly performed systematic reviews and meta-analyses may misrepresent the truth.

| 0 Comments

House of tottering cards

Poorly performed systematic reviews and meta-analyses may misrepresent the truth.

| 0 Comments

Cherry Picking

Cherry-picking results that only support your own conclusion may mean ignoring important evidence that refutes a treatment claim.

| 0 Comments

Straight ‘A’ student

Cherry picking results can be misleading.

| 0 Comments

Secret life of trials

The results of single comparisons of treatments can be misleading.

| 0 Comments

He said She said

Unsystematic presentation of research results can be misleading.

| 0 Comments

Desert Island Medical Journal

Small studies with few outcome events are usually not informative and can be misleading.

| 0 Comments

Forest Plot Trilogy

Synthesising the results of similar but separate fair comparisons (meta-analysis) may help by yielding statistically more reliable estimates

| 0 Comments

Significant Others

Statistical significance does not always mean meaningful or practical significance.

| 0 Comments

False Precision

The use of p-values to indicate the probability of something occurring by chance may be misleading.

| 0 Comments

Rumor has it

Anecdotes are an unreliable basis for treatment claims.

| 0 Comments

Personal “No Worse”

People with vested interests may use misleading statistics to support claims about the effects of new treatments.

| 0 Comments

Little Ogg

Lack of evidence of effects is not the same as evidence of no effects.

| 0 Comments

Lisa luxuriant hair

If possible, participants in clinical trials should not know which treatment they are receiving.

| 0 Comments

Suzie in despair

Apart from the treatments being compared, the comparison groups need to be similar.

| 0 Comments

World without bias

Overcoming biases is difficult but important. Treatment comparisons must be fair.

| 0 Comments

Evidence, the new Natural

Cherry picking results for reporting can be misleading.

| 0 Comments

Avoid despair about biases

People who choose to ignore biases may do themselves and others harm.

| 0 Comments

Does it work?

People with vested interests may use misleading statistics to support claims about the efects of new treatments.

| 0 Comments

Thousand dollar placebo

People with vested interests may take advantage of peoples' fears or hopes..

| 0 Comments

Promising treatments

'Promising' treatments greatly outnumber actual advances in treatment.

| 0 Comments

Right to remain anxious

Earlier testing is not always better, and can lead to overdiagnosis, overtreatment and anxiety.

| 0 Comments

De-awareness day

Earlier testing is not always better, and can lead to overdiagnosis and overtreatment.

| 0 Comments

Alicia

Earlier testing is not always better, and can lead to overdiagnosis and overtreatment.

| 0 Comments

Peer-Review

Even quality control steps, such as peer-review, can be affected by conflicts of interest.

| 0 Comments

Gertrud

Exaggeration and hopes or fears can lead to unrealistic expectations about treatment effects.

| 0 Comments

Conspiracy

Beware hidden agendas when assessing claims about treatment effects.

| 0 Comments

Fright Night

Even 'experts' can be uninformed.

| 0 Comments

Blackboard Professor

Beware reliance on 'experience' with no reference to relevant evidence.

| 0 Comments

Twilling Sisters

New treatments as well as old treatments can have unwanted effects.

| 0 Comments

Experienced

Beware reliance on 'experience' with no reference to relevant evidence.

| 0 Comments

Soy Lattes

Just because two things are associated, doesn't mean one thing caused the other.

| 0 Comments

Cause and Effect

Just because two things are associated, doesn't mean one thing caused the other.

| 0 Comments

Space-Diving Safety

Small studies with few outcome events are usually not informative and can be misleading.

| 0 Comments

Rhona Feeling Almost Cynical

Anecdotes are an unreliable basis for treatment claims.

| 0 Comments

Significus the Obscure

Cherry picking the results of people in sub-groups can be misleading.

| 0 Comments

Introduction to clinical trials: lay-friendly video

This lay-friendly video introduction to clinical trials was created by the European Communication on Research Awareness Needs Project.

| 3 Comments

Overview of study designs

11 slides and a 4-min commentary overviewing study designs for clinical research. (from Univ Mass Med School).

| 1 Comment

Systematic Reviews

3 slides and a 4-min commentary about systematic reviews and meta-analyses (from Univ Mass Med School).

| 0 Comments

Relative risk, absolute risk and number-needed-to-treat

8 slides and a 4-min commentary on relative risk, absolute risk, and number needed to treat (from Univ Mass Med School).

| 0 Comments

Attrition bias, publication bias, comparator bias and commercial bias

6 slides and a 3-min commentary on attrition bias, publication bias, comparator bias and commercial bias (from Univ Mass Med School).

| 0 Comments

P-values and the role of chance

Gilbert Welch’s 10-min video on p-values and assessing the likelihood that a difference between treatments is due to chance.

| 0 Comments

Understanding Confidence Intervals

A 4-min video explaining the concept of confidence intervals and how they are calculated, with helpful diagrams and examples.

| 0 Comments

Understanding Overdiagnosis bias

Gilbert Welch’s 14-min video discussing the risks of overdiagnosis bias and screening.

| 0 Comments

Calculating and interpreting absolute and relative change in an unwanted outcome after treatment

Gilbert Welch’s 6-min video explaining how to calculate and interpret absolute and relative change in an unwanted outcome.

| 0 Comments

Understanding lead-time bias

Gilbert Welch’s 10-min video explaining why survival ALWAYS rises following early detection -- even if no one is helped.

| 0 Comments

Mixed Messages about Statistical Significance

A webpage explaining the difference between statistical and practical significance.

| 0 Comments

Testing Treatments Audio Book

The Testing Treatments Audiobook enables visitors to the TTi site to select whichever chapters in the book they would like to listen to.

| 0 Comments

Science fact or fiction? Making sense of cancer stories

A Cancer Research UK blog, explaining how to assess the quality of health claims about cancer.

| 0 Comments

A simple fair testing plan for science lessons

A short worksheet to help students devise a Fair Test.

| 0 Comments

Worksheet for planning fair tests

A TES worksheet to guide primary school children in planning a Fair Test.

| 0 Comments

Drug trials in healthy volunteers

A 6-minute video illustrating an early phase clinical trial in healthy volunteers.

| 0 Comments

How are medicinal drugs tested?

A group of text files for teaching students about how medicinal drugs are tested.

| 0 Comments

Designing a fair test

Using an everyday example to try to help students realise what fair testing involves.

| 0 Comments

Double blind studies

A webpage discussing the importance of blinding trial participants and researchers to intervention allocation.

| 0 Comments

CEBM – Study Designs

A short article explaining the relative strengths and weaknesses of different types of study design for assessing treatment effects.

| 0 Comments

Introduction to Critical Appraisal

30-slide introduction by Jason Curtis, to Critical Appraisal.

| 0 Comments

DISCERN online

A questionnaire providing a valid and reliable way of assessing the quality of written information on treatment choices.

| 0 Comments

7 words (and more) you shouldn’t use in medical news

A webpage explaining that dramatic effects of medical treatments are very rare.

| 0 Comments

Observational Studies – does the language fit the evidence?

A webpage explaining observational studies and their advantage and disadvantages.

| 0 Comments

Means vs. Medians

Keith Bower’s 3-min video explaining how means (averages) and medians can be presented misleadingly.

| 0 Comments

Randomized Controlled Trial Protocols

A 1-hour videoed lecture explaining protocols for Randomized Control Trials (RCTs).

| 0 Comments

John Ioannidis, the scourge of sloppy science

A 8 min podcast interview with John Ioannidis explaining how research claims can be misleading.

| 0 Comments

Beware conflicts of interest

In this 5-min videoed TED talk, psychologist Dan Ariely explains how conflicts of interest can bias research.

| 0 Comments

Science Weekly Podcast – Ben Goldacre

A 1-hour audio interview with Ben Goldacre discussing misleading claims about research.

| 0 Comments

Statistical Significance and Practical Significance

Keith Bower’s 3-min video discussing the difference between Statistical Significance and Practical Significance.

| 0 Comments

Signals of overdiagnosis

Gilbert Welch’s 8-min video showing how population screening for disease leads to overdiagnosis.

| 0 Comments

Randomised Controlled Trials vs. Observational Studies

5-minute video explaining the difference between randomised trials and observational studies.

| 0 Comments

Type I and Type II errors, and how statistical tests can be misleading

Gilbert Welch’s 12-min video explaining Type I and Type II errors, and how statistical tests can be misleading.

| 0 Comments

How do you know which healthcare research you can trust?

A detailed guide to study design, with learning objectives, explaining some sources of bias in health studies.

| 0 Comments

CASP: making sense of evidence

The Critical Appraisal Skills Programme (CASP) website with resources for teaching critical appraisal.

| 0 Comments

Smart Health Choices: making sense of health advice

The Smart Health Choices e-book explains how to make informed health decisions.

| 0 Comments

Methodology of clinical trials

Eurordis training on the methodology of clinical trials for representatives of patients’ organisations.

| 0 Comments

How to read articles about healthcare

This article 'How to read health news behind the headlines', by Dr Alicia White, explains how to assess health claims in the media.

| 0 Comments

Can measurements show if a treatment works?

An article discussing errors to avoid when testing treatments.

| 0 Comments

The Fair Test (controlling variables)

TES PowerPoint presentation of 9 slides about the importance of controlling variables (i.e. conducting a fair test).

| 0 Comments

More than average confusion about what mean means mean

Cartoon and blog about how averages can hide important variations in effects.

| 0 Comments

Fair testing PowerPoint

A TES, 15-slide PowerPoint presentation by Keith Jarvis about Fair Testing, for 11-14 year olds.

| 0 Comments

Investigation: Designing a fair test

A teaching tool for teachers to illustrate how to carry out fair tests.

| 0 Comments

The basic principles of Evidence Based Medicine

A webpage explaining the foundations of systematic reviews.

| 0 Comments

Animal and Lab studies

A webpage explaining how results from animal studies may not be transferable to humans.

| 0 Comments

Commercialism

A webpage about commercialism and conflicts of interest in health research.

| 0 Comments

Be careful with composites

Kevin Lomangino's article discusses the limitations of composite outcomes and surrogate markers.

| 0 Comments

In defence of systematic reviews of small trials

An article discussing the strengths and weaknesses of systematic reviews of small trials.

| 0 Comments

Mega-trials

In this 5 min audio resource, Neeraj Bhala discusses systematic reviews and the impact of mega-trials.

| 0 Comments

Relative or absolute measures of effects

Dr Chris Cates' article explaining absolute and relative effects of treatment effects.

| 0 Comments

Evidence from Randomised Trials and Systematic Reviews

Dr Chris Cates' article discussing control of bias in randomised trials and explaining systematic reviews.

| 0 Comments

The perils and pitfalls of subgroup analysis

Dr Chris Cates' article demonstrating why subgroup analysis can be untrustworthy.

| 0 Comments

Reporting results of studies

Dr Chris Cates' article discussing how to report study results, with emphasis on P-values and confidence intervals.

| 0 Comments

Disease outcomes vs. patient-relevant outcomes

An e-Learning module consisting of 3 slides and 2-min commentary on why it is important to treat patients, not numbers from tests.

| 0 Comments

Evaluating Online Health Information

A 30 minute e-Learning presentation by Medline Plus explaining how to evaluate health claims found on the internet.

| 0 Comments

Reducing the play of chance using meta-analysis

Combining data from similar studies (meta-analysis) can help to provide statistically more reliable estimates of treatment effects.

| 0 Comments

Absolute versus relative risk – making sense of media stories

This blog by Sarah Williams should help people who want to understand risk in the context of health and medical stories in the press.

| 0 Comments

AllTrials: All Trials Registered | All Results Reported

AllTrials aims to correct the situation in which studies remain unpublished or are published but with selective reporting of outcomes.

| 0 Comments

Association is not the same as causation. Let’s say that again: association is not the same as causation!

This article explains how to tell when correlation or association has been confused with causation.

| 0 Comments

Routine use of unvalidated therapy is less defensible than careful research to assess the effects of those treatments

It is more difficult to obtain consent to give a treatment in a clinical trial than to give the same treatment for patients in practice.

| 2 Comments

Fair tests of treatments: a quick guide for journalists

When deadlines are pressing, how can journalists tell whether to believe claims about the effects of a new treatment or breakthrough?

| 1 Comment

The Gold Standard: What are randomised control trials and why are they important?

A four minute video by the MRC Clinical Trials Unit about the importance of randomised control trials.

| 0 Comments

Lesson plan for teaching secondary school students about double-blind trials

This lesson plan provides resources to run stimulating activities about fair tests of treatments in a classroom setting.

| 0 Comments

More is less: an investigation of unnecessary testing

This is a US radio production about unnecessary testing and the associated harm to patients and costs to the health system.

| 0 Comments

Viva la Evidence!

A brilliant song and video by James McCormack explaining the basics of evidence-based medicine.

| 0 Comments

Evidence Based Medicine Matters: Examples of where EBM has benefitted patients

Booklet containing 15 examples submitted by Royal Colleges where Evidence-Based Medicine has benefited clinical practice.

| 0 Comments

Ben Goldacre talks about Bad Pharma on C-SPAN

In this 90 minute videoed lecture, Ben Goldacre talks about key issues raised in his book 'Bad Pharma'.

| 0 Comments

Big data and finding the evidence

“Big data” is large-scale data processing technologies intended to generate insights into performance, behaviour and trends.

| 0 Comments

Some Studies That I Like to Quote

This short music video encourages health professionals to use evidence to help reach treatment decisions in partnership with patients.

| 0 Comments

What does the Cochrane logo tell us?

This video and animated slide presentation prepared by Steven Woloshin shows how the Cochrane logo was developed, and what it tells us.

| 3 Comments

Shared Decision-Making

This resource from the Health Foundation shows how shared decision-making can be made to work in a typical consultation.

| 0 Comments

On taking a good look at ourselves

Iain Chalmers talks about failings in scientific research that lead to avoidable harm to patients and waste of resources.

| 1 Comment

A poem about regression to the mean

Regression to the mean can lead us to think that an intervention has been effective when it hasn't. This poem illustrates it nicely.

| 0 Comments

Ben Goldacre on TED, on drug companies and hidden data.

Ben Goldacre's lecture at TEDglobal, in which he describes how pharmaceutical companies harm patients by failing to report negative outcomes

| 0 Comments

Communicating with patients on evidence

This discussion paper from the US Institute of Medicine provides guidance on communicating evidence to patients.

| 0 Comments

Surgery for the treatment of psychiatric illness: the need to test untested theories

Simon Wessely describes the untested theory of autointoxication, which arose in the 1890s and caused substantial harm to patients.

| 0 Comments

The placebo effect

A video by NHS Choices explaining what the placebo effect is, and describing its role in medical research and the pharmaceutical industry.

| 0 Comments

What did James Lind do in 1747?

A 2 minute Video clip of a BBC documentary recreating James Lind's celebrated experiment to test treatments for scurvy.

| 2 Comments

Manipulating doctors: testimony from an ex-drug rep

In this 10-min video, Gwen Olsen, a former pharmaceutical sales representative, talks about manipulating doctors to sell more drugs.

| 0 Comments

Therapeutic Touch: a schoolgirl shows how to test it

This 5-minute video provides an example of applying scientific method to dodgy treatment claims.

| 3 Comments

Help at last for the Annoyingly Cheerful

This humorous video by the Onion illustrates some of the tactics used to push unnecessary treatments or "sell" sickness.

| 1 Comment

What does a positive genetic test mean? The example of coeliac disease

Video tutorial explores the ways in which evidence about the effectiveness of genetic testing can be misrepresented in advertising.

| 0 Comments

Who has diabetes?

So how do we decide who has diabetes? When I was in medical school, our numerical rule was this: if […]

| 0 Comments

Doctors talk about guesswork in prescribing

In a fictional conversation between two doctors, a general practitioner makes the following point: ‘Tons of what we do is […]

| 0 Comments

From person to patient

Screening will inevitably turn some people who test ‘positive’ into patients – a transformation not to be undertaken lightly. ‘If […]

| 0 Comments

On being sucked into a maelstrom

In 2006, a patient in the UK, who happened to be medically trained, found herself swept along by the Herceptin […]

| 0 Comments

A tragic epidemic of blindness in babies

‘In the period immediately after World War II, many new treatments were introduced to improve the outlook for prematurely-born babies. […]

| 0 Comments

Anecdotes are anecdotes

‘Our brains seem to be hard-wired for anecdotes, and we learn most easily through compelling stories; but I am aghast […]

| 0 Comments

A new treatment for strawberry birthmarks

Treatments with dramatic effects are occasionally discovered by accident. Take the example of a condition that occurs in infants called […]

| 0 Comments

Mother’s kiss

Low-tech approaches can have dramatic effects too. Young children sometimes place small objects – plastic toys or beads, for example […]

| 0 Comments

Imatinib for chronic myeloid leukaemia

Impressive results have also been seen in patients given imatinib for chronic myeloid leukaemia [4], [5]. Before imatinib was introduced […]

| 0 Comments

Laser treatment of portwine stains

The birthmarks known as portwine stains are caused by permanent and malformed dilated blood vessels in the skin. Commonly occurring […]

| 0 Comments

Breast cancer

The treatment of breast cancer provides another example of professional uncertainty. There is considerable variability in the use of surgery, […]

| 0 Comments

Antibiotics in pre-term labour

Fair tests of treatments with hoped-for beneficial effects, and which are assumed to be harmless, can show that neither is […]

| 0 Comments

Caffeine for breathing problems in premature babies

Large variations in the treatments used for a particular condition provide clear evidence of professional uncertainty about the relative merits […]

| 0 Comments

Genetic tests: sometimes useful, often dodgy

Not so long ago ‘genetic testing’ was more or less confined to generally rare, single-gene disorders – for example, the […]

| 2 Comments

Lung cancer screening: early but not early enough?

Screening may detect disease earlier, but not always early enoughto make a difference (see Figure). Some cancers, for example lung […]

| 0 Comments

Prostate cancer screening: clear harms with uncertain benefits

Prostate cancer is the second most common cancer in men worldwide,14 and broadly falls into two types. Some men have […]

| 0 Comments

Breast cancer screening: well established but remains contentious

Since routine breast screening with mammography is well established in many countries one could well assume that mammographic screening must […]

| 0 Comments

Abdominal aortic aneurysm screening: proceed with care

At the other end of the age spectrum, abdominal aortic aneurysm screening can also be beneficial. The aorta is the […]

| 0 Comments

Phenylketonuria screening: clearly beneficial

Newborn babies are routinely screened for an inherited disease called phenylketonuria (PKU). Babies with PKU are unable to process phenylalanine, […]

| 0 Comments

New – but is it better?

Key points Testing new is necessary because new treatments are as likely to be worse as they are to be […]

| 0 Comments

Don’t be fooled by eye-catching statistics

Let’s say the risk of having a heart attack in your fifties is 50 per cent higher if you have […]

| 0 Comments

3: Statistics are confusing – should patients really have to look at the numbers?

The way that numbers are presented can be very daunting – or even downright misleading. But if you really do […]

| 1 Comment

Mutilating surgery

Until the middle of the 20th century, surgery was the main treatment for breast cancer. This was based on the […]

| 0 Comments

Involving citizens to improve healthcare

‘The confluence of interest between advocacy groups, those who sell treatments, and those who prescribe them makes for a potent […]

| 0 Comments

Pester power and new drugs

‘New drugs by their very nature are incomplete products, as full information about their safety, effectiveness and impact on costs […]

| 0 Comments

Lay people help to rethink AIDS

‘Credibility struggles in the AIDS arena have been multilateral: they have involved an unusually wide range of players. And the […]

| 0 Comments

A key partnership

‘People-focused research in the NHS simply cannot be delivered without the involvement of patients and the public. No matter how […]

| 0 Comments

Patients’ choice: David and Goliath

‘Who has the power to see that research questions actually address the greatest needs of patients in all their misery […]

| 0 Comments

Psoriasis patients poorly served by research

‘Few trials involved comparison of different options or looked at long-term management. The duration of studies is unconvincingly brief in […]

| 0 Comments

All it takes is to find the gene

‘It’s . . . hoped that the genetic revolution will cure every problem known to man. We will be able […]

| 0 Comments

Dodgy, devious and duped?

Writing a light-hearted article for a Christmas edition of the British Medical Journal, two researchers created a spoof company called […]

| 0 Comments

Doctors and drug companies

‘No one knows the total amount provided by drug companies to physicians, but I estimate from the annual reports of […]

| 0 Comments

Impact of “me-too” drugs in Canada

‘In British Columbia most (80%) of the increase in drug expenditure between 1996 and 2003 was explained by the use […]

| 0 Comments

Who decides what gets studied?

Clearly this situation is unsatisfactory, so how has it come about? One reason is that what gets studied by researchers […]

| 2 Comments

Questions that are important for patients

Researchers in Bristol decided to pose a fundamental question:  ‘To what extent are questions of importance to patients with osteoarthritis […]

| 0 Comments

Aprotinin: effect on bleeding during and after surgery

Research funders, academic institutions, researchers, research ethics committees, and scientific journals are all complicit in unnecessary research. As we explained […]

| 0 Comments

Stroke

Another example of unnecessary research, yet again because the results of preceding studies had not been gathered together and analyzed, […]

| 2 Comments

Respiratory distress in premature babies

Some research falls in between good and bad – it is plainly unnecessary. An example of such research concerns premature […]

| 0 Comments

Epidural analgesia for women in labour

The importance of assessing outcomes that matter to patients is clearly illustrated – in a very negative fashion – by […]

| 0 Comments

Psychiatric disorders

Regrettably, research is not always well done or relevant. Take the example of a distressing condition known as tardive dyskinesia. […]

| 0 Comments

HIV infection in children

The results of good research are also making a real difference to children infected with HIV (human immunodeficiency virus), the […]

| 0 Comments

Pre-eclampsia in pregnant women

Another outstanding example of good research concerns pregnant women. Worldwide, about 600,000 women die each year of pregnancy-related complications. Most […]

| 2 Comments

Stroke

Stroke is a leading cause of death and long-term disability. The death rate is between one in six and two […]

| 0 Comments

What research regulation should do

‘If ethicists and others want something to criticise in clinical trials, they should look at scientifically inadequate work, reinvention of […]

| 0 Comments

A commonsense approach to informed consent in good medical practice

‘What is missing in the debate surrounding informed consent is the true nature of patient understanding, what information patients want […]

| 0 Comments

Rethinking informed consent

‘[Some] have come to suspect that informed consent is not fundamental to good biomedical practice, and . . . attempts […]

| 0 Comments

Biased ethics

‘If a clinician tries a new therapy with the idea of studying it carefully, evaluating outcomes, and publishing the results, […]

| 0 Comments

In an ideal world

‘In an ideal world, wherever possible, we could be gathering anonymised outcome data and comparing this against medication history, making […]

| 0 Comments

Who says medical research is bad for your health?

‘Most discussion about the ethics of medical research addresses the question of how research should be regulated. Indeed, medical research […]

| 0 Comments

What regulatory systems do not do

Although regulatory systems for research impose onerous requirements on researchers before studies start, there are many things they conspicuously fail […]

| 2 Comments

Information and consent

Requirements relating to provision of information and consent for studies are one of the ways in which the regulatory system […]

| 0 Comments

Do regulatory systems for testing treatments get it right?

Although the level of regulation can be reassuring, current regulatory systems impose very onerous burdens on anyone wishing to study […]

| 0 Comments

Regulating tests of treatments: help or hindrance?

Key points Regulation of is unnecessarily complex Current systems of research regulation discourage that would make for better healthcare Despite […]

| 3 Comments

Instructions to authors to put research in context by the editors of the medical journal The Lancet

Systematic Review This section should include a description of how authors searched for all the evidence. Authors should also say […]

| 0 Comments

Could checking the evidence first have prevented a death?

‘In a tragic situation that could have been averted, Ellen Roche, a healthy, 24-year-old volunteer in an asthma study at […]

| 0 Comments

Science is cumulative, but scientists don’t accumulate evidence scientifically

‘Academic researchers have been talking about something called “cumulative meta-analysis” for 25 years: essentially, you run a rolling meta-analysis on […]

| 0 Comments

Marketing-based medicine

‘Internal documents from the pharmaceutical industry suggest that the publicly available evidence base may not accurately represent the underlying data […]

| 1 Comment

The importance of systematic reviews

‘Systematic reviews and meta-analyses have become increasingly important in health care. Clinicians read them to keep up to date with […]

| 0 Comments

Synthesizing information from research

More than a century ago, the president of the British Association for the Advancement of Science, Lord Rayleigh, commented on […]

| 0 Comments

Why did you start?

‘Few principles are more fundamental to the scientific and ethical validity of clinical research than that studies should address questions […]

| 0 Comments

Reports of new research should begin and end with systematic reviews

The report of a study [20] to assess the effects of giving steroids to people with acute traumatic brain injury […]

| 0 Comments

Wasted resources in healthcare and research

Failure to do systematic reviews of relevant, reliable research evidence does harm even when it is not harming patients and […]

| 2 Comments

Avoidable harm to people participating in research

Failure to assess all relevant, reliable evidence can also result in avoidable harm to people who participate in research. Researchers […]

| 0 Comments

Avoidable harm to patients

Recommended treatments for heart attacks that had appeared in textbooks published over a period of 30 years were compared with […]

| 0 Comments

What can happen if all the relevant, reliable evidence is not assessed?

In this section Avoidable harm to patients Avoidable harm to people participating in research Wasted resources in healthcare and research […]

| 2 Comments

Recognizing vested interests and spin in systematic reviews

What if the reviewers have other interests that might affect the conduct or interpretation of their review? Perhaps the reviewers […]

| 3 Comments

Reducing the play of chance in systematic reviews

Previously, we explained how the play of chance can be reduced by combining data from similar but separate studies – […]

| 0 Comments

Identifying all the relevant evidence for systematic reviews

Identifying all the relevant evidence for systematic reviews – irrespective of the language or format of the relevant reports – […]

| 0 Comments

Reducing biases in systematic reviews

Just as biases can distort individual tests of treatments and lead to false conclusions, so they can also distort reviews […]

| 0 Comments

Systematic reviews of all the relevant, reliable evidence

Whilst it is easy to state that we should review the results of a particular alongside other relevant, reliable evidence, […]

| 0 Comments

Is one study ever enough?

The simple answer is ‘hardly ever’. Very seldom will one fair treatment comparison yield sufficiently reliable evidence on which to […]

| 1 Comment

Assessing all the relevant, reliable evidence

Key points A single rarely provides enough evidence to guide choices in healthcare Assessments of the relative merits of alternative […]

| 0 Comments

What does “statistically significant” mean?

‘To be honest, it’s a tricky idea. It can tell us if the difference between a drug and a placebo […]

| 0 Comments

Obtaining large enough numbers in fair tests of treatments

Sometimes in tests of treatments it is possible to obtain large enough numbers from research done in one or two […]

| 1 Comment

What does a “significant difference” between treatments mean?

Well, this is a trick question, because ‘significant difference’ can have several meanings. First, it can mean a difference that […]

| 2 Comments

Assessing the role that chance may have played in fair tests

The role of chance can lead us to make two types of mistakes when interpreting the results of fair treatment […]

| 4 Comments

Taking account of the play of chance

Key point Account must be taken of ‘the ’ by assessing the confidence that can be placed in the quality […]

| 1 Comment

The Yellow Card Scheme

The Yellow Card Scheme was launched in Britain in 1964 after the thalidomide tragedy highlighted the importance of following up […]

| 0 Comments

Believing is seeing

The British doctor Richard Asher noted in one of his essays for doctors: ‘If you can believe fervently in your […]

| 0 Comments

Mistaking the cure

. . .‘it is alleged to be found true by proof, that by the taking of Tobacco, divers and very […]

| 0 Comments

Generating and investigating hunches about unanticipated adverse effects of treatments

Generating hunches about unanticipated effects of treatments Unanticipated effects of treatments, whether bad or good, are often first suspected by […]

| 2 Comments

Fair measurement of treatment outcomes

Although one of the reasons for using sham treatments in treatment comparisons is to help patients and doctors to stick […]

| 2 Comments

Helping people to stick to allocated treatments

Differences between intended and actual treatments during treatment comparisons can happen in other ways that may complicate the interpretation of […]

| 2 Comments

Dealing with departures from allocated treatments

For all the reasons given so far in this section, you will have realized that fair tests of treatments have […]

| 2 Comments

Following up everyone in treatment comparisons

After taking the trouble to assemble comparison groups to ensure that like will be compared with like, it is important […]

| 0 Comments

Ways of using unbiased (random) allocation in treatment comparisons

Random allocation for treatment comparisons can be used in various ways. For example, it can be used to compare different […]

| 0 Comments

Unbiased, prospective allocation to different treatments

In 1854, Thomas Graham Balfour, an army doctor in charge of a military orphanage, showed how treatment groups could be […]

| 0 Comments

Comparing apparently similar groups of patients who happen to have received different treatments in the same time period

Comparing the experiences and outcomes of apparently similar groups of patients who happen to have received different treatments in the […]

| 0 Comments

Comparing patients given treatments today with apparently similar patients given other treatments in the past for the same disease

Researchers sometimes compare patients given treatments today with apparently similar patients given other treatments in the past for the same […]

| 0 Comments

Treatments with moderate but important effects

This section addresses the common situation in which treatments may differ only moderately from each other, but these differences may […]

| 0 Comments

Treatments with dramatic effects

Sometimes patients experience responses to treatments which differ so dramatically from their own past experiences, and from the natural history […]

| 2 Comments

Comparing like with like

In this sub-section Comparisons are key (this page) Treatments with dramatic effects Treatments with moderate but important effects Comparisons are […]

| 2 Comments

So what are fair tests?

Most of us know that it can be a mistake to take a media report of some new medical advance […]

| 0 Comments

The need to go beyond impressions

If patients believe that something helps them, isn’t that enough?  Why is it important to go to the trouble and […]

| 0 Comments

The beneficial effects of optimism and wishful thinking

The psychological reasons for people attributing any improvement in their condition to the treatment they received are now better understood. […]

| 2 Comments

Why are fair tests of treatments needed?

In this sub-section Nature, the healer (this page) The beneficial effects of optimism and wishful thinking The need to go […]

| 2 Comments

Fair tests of treatments

Key points Fair tests of are needed because we will otherwise sometimes conclude that treatments are useful when they are […]

| 2 Comments

Addressing uncertainty is professional

‘One of the key attributes of professionalism . . . should be the ability to identify and address uncertainty in […]

| 0 Comments

Facing up to uncertainties: a matter of life and death

‘Failure to face up to uncertainties about the effects of treatments can result in avoidable suffering and death on a […]

| 0 Comments

Stepwise progress doesn’t hit the headlines

Science itself works very badly as a news story: it is by its very nature a subject for the “features” […]

| 0 Comments

Providing treatment as part of a fair test

So what should happen when there is important about the effects of new or old treatments that have not been […]

| 0 Comments

Addressing uncertainties about the effects of treatments

Where do we go from here? Clinicians need to be able to draw on resources that provide the best current […]

| 0 Comments

When practitioners disagree

In this sub-section Introduction (this page) Caffeine for breathing problems in premature babies Antibiotics in pre-term labour Breast cancer Introduction […]

| 2 Comments

Moderate treatments effects: usual and not so obvious

Most treatments do not have dramatic effects and fair tests are needed to assess them. And sometimes a treatment may […]

| 3 Comments

Dramatic treatment effects: rare and readily recognizable

In this sub-section: Laser treatment of portwine stains Imatinib for chronic myeloid leukaemia Mother’s kiss A new treatment for strawberry […]

| 2 Comments

Dealing with uncertainty about the effects of treatments

Key points of are rare about the are very common Small differences in the effects of different treatments are usual, […]

| 1 Comment

The Screening Circus

In 2009, a recently retired professor of neurology with a long-standing interest in stroke prevention learnt that neighbours had received […]

| 0 Comments

Don’t play poker with your genes

‘Acting on the knowledge of a single (or even a few) gene variants is similar to betting all your money […]

| 0 Comments

Selling screening

‘Selling screening can be easy. Induce fear by exaggerating risk. Offer hope by exaggerating the benefit of screening. And don’t […]

| 0 Comments

Discoverer of PSA speaks out

‘The test’s popularity has led to a hugely expensive public health disaster. It’s an issue I am painfully familiar with […]

| 0 Comments

Overdiagnosing prostate cancer

Prostate cancer has been described as the par excellence example of overdiagnosis. This does not mean that there are not […]

| 0 Comments

Don’t assume early detection is worthwhile

‘Screening for neuroblastoma illustrates how easily one can fall into the trap of assuming that because a disease can be […]

| 0 Comments

Is anyone normal?

Whole-body CT scans Among the tests on offer at private clinics are whole-body computed tomography (CT) scans to look at […]

| 0 Comments

What screening aims to achieve and why evidence matters

The examples we have already given show that, before rushing headlong into widespread screening, it is worth pausing a moment […]

| 3 Comments

Lessons from neuroblastoma screening

Experience with screening for neuroblastoma – a rare cancer that mainly occurs in young children – is instructive in several […]

| 2 Comments

Earlier is not necessarily better

Key points Earlier does not necessarily lead to better ; sometimes it makes matters worse programmes should only be introduced […]

| 2 Comments

The struggle for unbiased evidence

Researchers expected it would take about three years to enrol about 1,000 women in the two studies. Instead it took […]

| 0 Comments

Random allocation – a simple explanation

‘Randomisation is to minimise bias and ensure that the patients in each treatment group are as similar as possible in […]

| 0 Comments

The classical (Halstead) radical mastectomy

The radical mastectomy, devised in the late 19th century by William Halsted, was the most commonly performed operation for breast […]

| 0 Comments

Drastic treatment is not always the best

‘It is very easy for those of us treating cancer to imagine that better results are due to a more […]

| 0 Comments

We do things because…

‘We [doctors] do things, because other doctors do so and we don’t want to be different, so we do so; […]

| 0 Comments

Dare to think about doing less

So, more is not always better – and this message remains important. Today, in women with metastatic (widespread) breast cancer, […]

| 1 Comment

Bone marrow transplantation

However, the demise of mutilating surgery did not spell the end of the ‘more is better’ mindset – far from […]

| 0 Comments

Intensive treatments for breast cancer

The therapies advocated for breast cancer – so often in the news – provide some especially valuable lessons about the […]

| 2 Comments

More is not necessarily better

Key point More intensive is not necessarily beneficial, and can sometimes do more harm than good. Introduction A popular misconception […]

| 2 Comments

No wonder she was confused

In January 2004, a hysterectomy patient wrote this letter to The Lancet: ‘In 1986 I had a hysterectomy because of […]

| 0 Comments

Evening primrose oil for eczema

Even if inadequately assessed treatments do not kill or harm, they can waste money. Eczema is a distressing skin complaint […]

| 0 Comments

Hormone Replacement Therapy (HRT)

In women going through the menopause, hormone replacement therapy (HRT) is very effective in reducing the distressing hot flushes that […]

| 0 Comments

Diethylstilboestrol

At one time, doctors were uncertain whether pregnant women who had previously had miscarriages and stillbirths could be helped by […]

| 0 Comments

Drugs to correct heart rhythm abnormalities in patients having a heart attack

Dr Spock’s advice may have seemed logical, but it was based on untested theory. Other examples of the dangers of […]

| 2 Comments

Advice on babies’ sleeping position

Do not imagine that only can harm – advice can be lethal too. Many people have heard of the American […]

| 0 Comments

Hoped-for effects that don’t materialize

Key points Neither theory nor professional opinion is a reliable guide to safe, effective Just because a treatment is ‘established’ […]

| 2 Comments

Don’t be too certain

“Through seeking we may learn and know things better. But as for certain truth, no man hath known it, for […]

| 0 Comments

Herceptin

Commercial companies are not alone in trumpeting the advantages of new treatments while down-playing drawbacks. Professional hype and enthusiastic media […]

| 1 Comment

Mechanical heart valves

Drugs are not the only treatments that can have unexpected bad effects: non-drug treatments can pose serious risks too. Mechanical […]

| 0 Comments

Avandia

2010 saw another drug – rosiglitazone, better known by the trade name Avandia – hitting the headlines because of unwanted […]

| 0 Comments

Vioxx

Although drug-testing regulations have been tightened up considerably, even with the very best drug-testing practices there can be no absolute […]

| 0 Comments

Thalidomide

Thalidomide is an especially chilling example of a new medical treatment that did more harm than good. [1] This sleeping […]

| 0 Comments