New – but is it better?

Download a PDF of New - but is it better? Why fair tests of treatments are necessary

Key points

  • Testing new treatments is necessary because new treatments are as likely to be worse as they are to be better than existing treatments
  • Biased (unfair) tests of treatments can lead to patients suffering and dying
  • The fact that a treatment has been licensed doesn’t ensure that it is safe
  • Side-effects of treatments often take time to appear
  • Beneficial effects of treatments are often overplayed, and harmful effects downplayed

Introduction: Why fair tests of treatments are necessary

Anecdotes are anecdotes

‘Our brains seem to be hard-wired for anecdotes, and we learn most easily through compelling stories; but I am aghast that so many people, including quite a number of my friends, cannot see the pitfalls in this approach.’

Nick Ross, broadcaster.  Read the full quote.

Without fair – unbiased – evaluations, useless or even harmful treatments may be prescribed because they are thought to be helpful or, conversely, helpful treatments may be dismissed as useless. And fair tests should be applied to all treatments, no matter what their origin or whether they are viewed as conventional or complementary/alternative.

Untested theories about treatment effects, however convincing they may sound, are just not enough. Some theories have predicted that treatments would work, but fair tests have revealed otherwise; other theories have confidently predicted that treatments would not work when, in fact, tests showed that they did.

Although there is a natural tendency to think ‘new’ means ‘improved’ – just like those advertisements for washing machine detergents – when new treatments are assessed in fair tests, they are just as likely to be found worse as they are to be found better than existing treatments.

There is an equally natural tendency to think that because something has been around for a long time, it must be safe and it must be effective. But healthcare is littered with the use of treatments that are based on habit or firmly held beliefs rather than evidence: treatments that often do not do any good and sometimes do substantial harm.

James Lind

There is nothing new about the need for fair tests: in the 18th century James Lind used a fair test to compare six of the remedies then being used to treat scurvy, a disease that was killing vast numbers of sailors during long voyages.

He showed that oranges and lemons, which we now know contain vitamin C, were a very effective cure.

In 1747, while serving as a ship’s surgeon aboard HMS Salisbury, James Lind assembled 12 of his patients at similar stages of the illness, accommodated them in the same part of the ship, and ensured that they had the same basic diet. This was crucial – it created a ‘level playing field’.

Lind then allocated two sailors to receive one of the six treatments that were then in use for scurvy – cider, sulphuric acid, vinegar, seawater, nutmeg, or two oranges and a lemon. The fruit won hands down.

The Admiralty later ordered that lemon juice be supplied to all ships, with the result that the deadly disease had disappeared from the Royal Navy by the end of the 18th century.

A page from the Treatise on ScurvyOf the treatments Lind compared, the Royal College of Physicians favoured sulphuric acid while the Admiralty favoured vinegar – Lind’s fair test showed that both authorities were wrong.

Surprisingly, influential authorities are quite frequently wrong. Relying too much on opinion, habit, or precedent rather than on the results of fair tests continues to cause serious problems in healthcare.

Quack or cure?

This video provides an example of the application of the scientific method to the claims of quackery! Featuring Emily Rosa, the youngest person ever to publish in the prestigious Journal of the American Medical Association.

Today, uncertainties about the effects of treatments are often highlighted when doctors and other clinicians differ about the best approach for a particular condition.

In addressing these uncertainties, patients and the public, as well as doctors, have an important part to play. It is in the overwhelming interest of patients, as well as professionals, that research on treatments should be rigorous.

Just as health professionals must be assured that their treatment recommendations are based on sound evidence, so patients need to demand that this happens. Only by creating this critical partnership can the public have confidence in all that modern medicine has to offer.