Fair measurement of treatment outcomes

Download a pdf of Fair measurement of treatment outcomes

Although one of the reasons for using sham treatments in treatment comparisons is to help patients and doctors to stick to the treatments allocated to them, a more widely recognized reason for such ‘blinding’ is to reduce biases when the outcomes of treatments are being assessed.

Blinding for this reason has an interesting history. In the 18th century, Louis XVI of France called for an investigation into Anton Mesmer’s claims that ‘animal magnetism’ (sometimes called ‘mesmerism’) had beneficial effects. The king wanted to know whether the effects were due to any ‘real force’, or rather to ‘illusions of the mind’. In a treatment test, blindfolded people were told either that they were or were not receiving animal magnetism when in fact, at times, the reverse was happening. People only reported feeling the effects of the ‘treatment’ when they had been told that they were receiving it.

For some outcomes of treatment – survival, for example – biased outcome assessment is very unlikely since there is little room for doubt about whether or not someone has died. However, assessing most outcomes will entail some subjectivity, because outcomes should and often do involve patients’ experiences of symptoms such as pain and anxiety. People may have individual reasons for preferring one of the treatments being compared. For example, they may be more alert to signs of possible benefit when they believe a treatment is good for them, and more ready to ascribe harmful effects to a treatment about which they are worried.

In these common circumstances, blinding is a desirable feature of fair tests. This means that the treatments being compared must appear to be the same.

In a test of treatments for multiple sclerosis, for example, all the patients were examined both by a doctor who did not know whether the patients had received the new drugs or a treatment with no active ingredient (that is, the doctor was ‘blinded’), and also by a doctor who knew the comparison group to which the patients had been allocated (that is, the doctor was ‘unblinded’). Assessments done by the ‘blinded’ doctors suggested that the new treatment was not useful whereas assessments done by the ‘unblinded’ doctors suggested that the new treatment was beneficial. [8]  This difference implies the new treatment was not effective and that knowing the treatment assignment led the ‘unblinded’ doctors to have ‘seen what they believed’ or hoped for.

Overall, the greater the element of subjectivity in assessing treatment outcomes, the greater the desirability of blinding to make tests of treatments fair.

Sometimes it is even possible to blind patients as to whether or not they have received a real surgical operation. One such study was done in patients with osteoarthritis of the knee. There was no apparent advantage of a surgical approach that involved washing out the arthritic joints when this was compared with simply making an incision through the skin over the knee under anaesthesia, and ‘pretending’ that this had been followed by flushing out the joint space. [9]

Often it is simply impossible to blind patients and doctors to the identity of treatments being compared – for example, when comparing surgery and a drug treatment or when a drug has a characteristic side-effect. However, even for some outcomes for which bias might creep in – say, in assigning a cause of death, or judging an X-ray – this can be avoided by arranging for these outcomes to be assessed independently by people who do not know which treatments individual patients have received.