Testing Treatments interactive
Promoting critical thinking about treatment claims
Please tell us what you think of Testing Treatments.
All comments are moderated to prevent spam, but we welcome all of your views.
A clearly written and well argued account of an appropriate application of the scientific method of enquiry to an emotionally charged area of our lives (and possible avoidable death!).
I found the examples of (type 1 and type 2) errors – in understanding the results of flawed research – chilling.
As a result I hope, in the future, to be even more aware of the possible pitfalls and the potential for bias in both the design and reporting of health related research. Thank You.
We welcome all of your comments and feedback!
Please tell us about useful resources that help explain the importance of testing treatments.
Try this: http://www.ncbi.nlm.nih.gov/books/NBK66204/
I like the pictures of content that change on the home page. Is there any way to make these clickable so someone could click through to something that catches their interest as it flashes up?
Thanks Amanda, if you click on the image below the text it links you to the relevant TTextra
I can’t find the link to download the 2nd edition. can you please post it? All I can find is links to the hypertext edition.
Thanks for point this out JD, I have now rectified matters.
The PDF of Testing Treatments is now available to download here: http://www.testingtreatments.org/the-book/download-the-book/
Thanks for the link to the download!
Nice to hear the Testing Treatments book recommended by Ben Goldacre on Radio 4 today:
Thanks for this great link, Andre. Interesting to see how the same book can get such a difference reaction! (For anyone else who wants to listen to the review but not the whole programme the discussion starts at 20 minutes.) This website addresses many of the comments of Michelle Paver.
This is a great initiative. Like Andre, I heard about it on Radio 4 yesterday. Ben’s recommendation, although audacious, made a great change from the obscure & tedious books brought to our attention by some of previous guests over the years.
If you are getting the right treatment you feel effective you will probably not be here. It is those who need research & more assistance who are found on Forums such as Healthunlocked & one linked to care 2 USA [ Inspire?]
Helplines of Charities run by the non medical on corporate remits or a Nurse are a complete waste of time -assistance needs to be given often urgently .
Just a few comments in response to a recent query regarding the apparent discrepancy in length between the Chinese and English versions of Testing Treatments (1st edition):
(1) The Chinese version does not include the cover, Contents and Acknowledgement pages and the foreword by Ben Goldacre (which constitute the first 11 pages of the English version), nor does it include the References, Additional resources and Index (which constitute the last 21 pages of the English version).
(2) In addition to the difference in font size, the written Chinese text may be more concise (space-wise, although I’m certainly not a linguistic expert…) than the written English text. For these two versions, one page of text in Chinese can cover approximately two and a quarter pages of text in English without losing information. Therefore, taking into account the omission of contents at the front and end of the English version from the Chinese version (as stated above), the number of pages for the Chinese version seems reasonable.
(3) The only things that appear to be missing from the main text of the Chinese version (that I could identify after a page-by-page comparison) are the text boxes ‘The Which? verdict on health screens’ in Chapter 2 and ‘Two cheers for uncertainty’ in Chapter 4 of the English version. Other headings, text boxes and texts seem to agree well between the two versions. Like any translated work, the Chinese version may have minor loss of fidelity in some sentences, but overall I would have no hesitation to recommend the Chinese version to a Chinese reader, who can always consult the original English version if/when necessary.
I hope this helps.
Thanks for your advice. The second edition(Chinese) is going to come up, which will cover all contents that appear in English version.
That’s great news for Chinese readers. Look forward to seeing the new edition!
The Chinese version is now available. Please select “Chinese” from the drop-down menu under “Translations of this website”, above right.
Just a couple of suggestions:
1. Please change the name from TT extras to “blog”. I don’t think anybody knows what TT extras means. Everyone knows what a blog means.
2.I would love to see an RSS feed fr the articles. So it is easy to find updates in my google reader. More like a pull technique.
3.Also a facebook page where people can “like”.So any artcil or updates can be seen in their facebook feed.
Thanks for all the hard work!
2.I would love to see an RSS feed fr the articles. So it is easy to find updates in my google reader. More like a pull technique
Many thanks for these suggestions, which we will take into account. Another think we are looking at is making the linkage between TTextras and the book content a little more direct.
However, I think it’s a great idea to explicitly provide a blog. The problem is getting people to write it!
I can do one per month on the topic of your choice in a public friendly format using TT as a base. Please let me know if this would be useful
Comment for James McCormack. A good video “Some studies…”. One of the studies cited, the one about salt restriction, could be improved (or removed). In fact in that review, patients were given such an amount of furosemide that everyone would bet that with a salt restriction regime patients will suffer -for sure- a hiponatremia. The authors of the trials included in this review gave in fact 3 or more times the recommended amount of furosemide!!. Aplicability of the conclusions are then flawed. The methodology of the SR is ok, they follow PRISMA, they do anything for a sound paper, except the usual dosage of diuretics…
Hi Jose – thanks so much for your comment – I think about it this way.
1) There really is no recommended dose of furosemide – the “recommend” dose is the dose that, via titration by the clinician or patient, controls excess fluid/weight – Regardless, they state “approximately two-thirds of patients received furosemide 50 mg twice daily (comparable with a commonly seen 40 mg twice daily dosing in the community)”
2) Let’s assume that the doses in these studies are higher than those typically seen – how does that “flaw” a study – in my opinion it doesn’t. All that issue does is limit it’s generalizability to people who are on those doses.
3) Unless there is a relatively well described reason why these specific doses of furosemide somehow were the reason why the people in the salt restriction group had the magnitude seen for increased all cause mortality (RR 1.95, 95% CI 1.66 to 2.29), sudden death (RR 1.72, 95% CI 1.21 to 2.44), death due to HF (RR 2.23, 95% CI 1.77 to 2.81) and HF readmissions (RR 2.10, 95% CI 1.67 to 2.64) I’m not sure that I’m too concerned about the dose of furosemide.
3) What is presented in this review is the “best” available evidence that we have – “flawed” as it is. Unless there is RCT evidence that salt restriction actually improves outcome I would lean towards using this evidence rather than ignoring it because it is “flawed”. The bottom line is that anyone who recommends salt restriction in HF is doing it based on opinion – the lowest form of evidence and ignoring much higher levels of evidence.
This is a very interesting and useful resource – any plans to introduce it as part of the teaching materials at universities? Medical students may find this very enlightening – get ’em while they’re young!
Many thanks for your kind words.
I completely agree with you, students are absolutely a key group and we’d love to hear your (or anybody else’s) ideas on what we can to do to get TTi onto their radar.
At the moment we are talking to a broad range of potential users to find out more about how to make TTi useful to them, and this includes educators as well as health professionals, information workers, journalists, bloggers and other groups.
So now’s the time to tell us what you think!
Science AAAS is using Google video and Discus for interactive panel workshops. ThinkWell could perhaps work with you to set up an interactive Public Led Online Trial to road test learning. The public often gets complex papers thrown at the activists to shut them up and they have no way like CASP or TTI to vet the paper. People learn by doing. Recently I had a trust contact me who had no idea what an SR really was!
Hi, I love this Testing Treatments website, it has great content and I would blog post excerpts and links on our website. I can’t because older really valuable posts are lost. Same when I want to email someone so they can check it out. For instance I looked for one Post on the benefits of brain training but could not find it even through search. There is very little information via the menus so it discourages repeat visits. You could fix this with a location page where you list everything and not just latest blogs or even with a visible sitemap. Hope this can be fixed but great book to download in multiple languages, thanks for this!
We don’t have any posts about brain training; that’s probably why you didn’t find it!
Thanks for the suggestions though. We are listening to what people say and doing hands-on user research at the moment, so we will take this into account.
It was actually a post that included a reference to using steroids post brain truama. I wanted to find this paper and use it as I know physicians that have done this research since the study which is a waste of resources . The patient opinion data that no one took any account of reported this intervention as unhelpful. I was tying this in with a large brain training study that was done with inadequate not appropriately targeted training which received great press for its not significant findings but if you use inaccurate tools the results are not going to be stellar. If they had done an unbiased review they would have seen this. Some research does more harm than good….a link to the paper would be great!
Ah OK. Is this the one you mean?
It’s in the part of the site that deals with sample size. Both references are on that page.
Search for “steroid” found it.
LOL Helps to have the right word! Thanks very much. I think I came across it while looking for sample size info and lost both again. I got to comments again by clicking feedback as clicking on your comment took me your profile. The entire article was excellent and easy reading.
Excellent book. Currently integrating it into our new pharmacy curriculum.
Thanks Anthony. We’d be really interested to hear how you get on with that, and any suggestions you have to make the content more useful for your students.
I wonder if the trials do not automatically help “the majority” who react positively to a treatment. Every person and his/her metabolism is different so maybe big trials disadvantage the “rare metabolisms” as for them no treatments are developed? Thanks for comments.
I believe this is a great resource for patient and public to understand the importance of the clinical process
This website is really helpful, but it is not sufficiently on the radar. I have only stumbled upon it. I have been struggling with doctors fobbing me off with “It will be fine. The risk is low.” and refusing to go into detail. I then have to go off and spend quite some time googling relevant research (although I’m not a scientist, let alone a medical person) to make up my own mind and satisfy myself that the supporting evidence is there. Unfortunately I am now faced with a proposed treatment where it seems to me that the uncertainties and controversies have not been resolved. Indeed I have found out that a research review is underway hoping to clarify things. The doctor is playing it as if there is no uncertainty at all. I have printed off “Addressing uncertainties about the effects of treatment” and “Providing treatment as part of a fair test” as a structure for discussion next time, which will hopefully be a more intelligent conversation. I do hope your website will help doctors to have more “grown up” conversations with people who are intelligent enough to handle it and express an interest (by asking too many difficult questions!). More needs to be done to identify us as a subset who need a different sort of conversation. I know lots of people need the “touchy-feely” support that is widely available, but medical practitioners are actually making the stress worse for those of us who don’t need that, but do need a grown up conversation that isn’t afraid of discussing risk and uncertainty and guiding us to an independent rational decision, mindful of the evidence and its limitations, and not just going along with “doctor knows best”. Keep up the good work!
Many thanks FTF! Let us know how you get on; we’re delighted to know that our materials are helpful to you and will be very interested to hear how they went in your consultation.
What do you suggest if you find misconduct within a Cochrane group ? I’ve read ‘Bad Pharma’ and am ploughing thru’ Deadly Medicines but there’s been no suggestion as to who watches the watchers.
I would document it and contact the Cochrane team http://www.cochrane.org/contact/. Have you done this?
In terms of “who watches the watchers”, we all do. Labels and authority in and of themselves don’t guarantee quality. We should always think critically about any health claims.
In the case of Cochrane Collaboration, their methods are transparent as are their findings (or they are supposed to be anyway). This makes it easier to think critically about their output than pretty much any other source of information I can think of.
Douglas, I’ve received no reply regarding concerns raised offline to Minervation so they’re now publicly posted at FnMyalgia.com
Marzipan, did you contact me at Minervation? I didn’t receive anything from you.
Hi. I love this site. Nice work. I found a link that doesn’t work. In the “Correct box” for the question about placebos in the quiz, the link to Emily’s video is broken
Home > Test Your Knowledge
Does belief in the effects of a treatment influence the likelihood that those effects will be experienced?
Correct: In addition to this video by Ben Goldacre, you may like to watch this video on the *research done by American schoolgirl Emily Rosa* and you might like to read more about fair measurement of treatment outcomes.
Thanks Rebecca, we’ll sort that out asap.
I love this a book and I will be using it as a resource when I prepare materials for a talk I am giving on patient involvement in research. I have one criticism, which is that you refer to pregnancy sickness as morning sickness. We at Pregnancy Sickness Support (www.pregnancysicknesssupport.org.uk), the main charity helping women with severe pregnancy sickness and hyperemesis gravidarum, are campaigning to have this terminology consigned to history. Only 10% of women with pregnancy sickness have symptoms confined to the morning hours, therefore it is a ridiculously innaccurate description.
That issue is minor, however, compared to the fact that the term ‘morning sickness’ trivialises an illness which causes moderate to severe morbidity to most pregnant women. Women struggle to access treatment, they are dismissed by their doctors and are expected to struggle on while suffering an illness, which is, in the public consciousness considered to be a minor inconvenience. This erroneous public perception is perpetuated, in part, by a name which invokes the image of a minor problem which is over and done with by 12 noon.
Severe pregnancy sickness is anything but a minor inconvenience; it causes severe and long lasting poor health of the mother and is the cause of around 1000 terminations of wanted pregnancies every year in the UK. Most mothers who have terminated a pregnancy due to hyperemesis gravidarum were not offered the full range of treatments and report uncaring attitudes from healthcare providers. (Reference https://www.pregnancysicknesssupport.org.uk/documents/HGbpasPSSreport_docx.pdf)
We are glad that you have highlighted the lack of sense and attention to scientific evidence which led to the withdrawal of Bendectin, however, we would like you to refer to pregnancy sickness as pregnancy sickness. You may even want to put in brackets ‘previously known as morning sickness’.
Margaret O’Hara, trustee of Pregnancy Sickness Support
Thanks for your comments Margaret, I hope your talk goes well. Please let us know if you have anything you’d like share about how Testing Treatments helped you with patient involvement.
Re: the terminology issue, I take your point and can see the argument for a change. I guess our authors were working with established terminology. I’ll pass your comment on for consideration.
After listening to the Breakfast Session at Evidence Live I had a look at the website and I really appreciate all the information available here. But – I think a link is missing from the Fair Comparisons network page. I tried to access information from the link “Learning and teaching resources” and was unable. Just wanted you to know!
Greetings from Anneli
Many thanks for your kind words and eagle eyes. I’ve fixed the link now.
GET-IT provides plain language definitions of health research terms