Eminence vs. evidence

posted on: 2012-11-30 06:57

Alan Cassels is the author of the recently published Seeking Sickness: Medical Screening and the Misguided Hunt for Disease, and is currently writing a social history of The Cochrane Collaboration. Here he presents the essential differences – with illustrative examples – between ‘eminence-based medicine’ and ‘evidence-based medicine’.

Cross-posted with grateful acknowledgment from common ground, Western Canada's “biggest and best-loved monthly magazine dedicated to health, wellness, ecology and personal growth”.

What is ‘eminence-based medicine’? It means relying on the opinion of a medical specialist or other prominent health official when it comes to health matters, rather than relying on a careful assessment of relevant research evidence. You might be asking, “Who am I to question an ‘expert’, especially a physician, a specialist or a prominent medical researcher who knows so much more than me?” Here are a few recent examples of eminence-based medicine in the real world, which hopefully will leave you questioning.

Exhibit A
Recently, an Australian journalist shared with me the thoughts of a very prominent professor of medicine concerning cholesterol-lowering drugs. This national expert is one of the authors of Australia’s lipid guidelines – guidance for doctors on cholesterol-lowering or statin drugs – and he weighed in with his opinion around the benefits and harms of statins.

His answers sounded very authoritative, citing studies and reports that buttressed an unmistakable love of statins. He also cherry-picked his data, selecting and presenting research evidence that supported his love affair, concluding that most people, even people at moderate risk of heart disease, should be taking a statin. Saying that double-blind trials provide incontrovertible proof that statins reduce the risk of heart attacks and are among the “safest class of drug ever developed”, he maintained that more people needed to take these drugs. Risks? He said that the “risks of side effects are less than 0.1%”.

If you prefer evidence to expert testimony, you’d look for the most reliable research studies out there and compile them into an analysis that made sense, using methods that are clear and replicable. Above all, referring to single studies is bad, bad, bad because a single study only represents a small slice of research. If you present a study and I counter with a study that found the opposite, the resulting ping-pong game would leave us both exhausted, but no smarter. Instead of single studies, you need to demand, and depend upon, overviews of all available research in an area, such as a systematic review of the literature. This is the kind of stuff produced by The Cochrane Collaboration, and on statins it tells a very different story.

The Cochrane Review says that statins for primary prevention – drugs to prevent a person’s first heart attack – might have some benefit, but they can’t say for sure. The story is incomplete. Their overview of the best quality research is hampered by the fact that some companies refuse to release the full set of adverse event data. Other research suggests that 0.1% risk of side effects is a gross underestimate in the real world, and that as many as 20% of patients experience muscle weakening when taking statins. So whom do you trust – the expert claiming these are the safest drugs in the world, or the evidence that says we actually don’t have a full picture of the safety of these drugs? As one doctor once told me, “In God We Trust; all others must show data”. To which I say, “Amen”.

Exhibit B: ‘flu drugs
British Columbia (BC) launched a very aggressive anti-‘flu policy this fall, with healthcare workers being forced to get the ‘flu shot or wear a mask. The rationale is that anyone caring for patients shouldn’t also be passing on viruses to them and making them sick. Fair enough. But does the evidence support mandatory ‘flu shots for healthcare workers? Does the vaccine even work to prevent the spread of the virus?

BC Provincial Health Officer Dr. Perry Kendall thinks so and so do some of his colleagues, including Dr. Paul Van Buynder, Chief Medical Office of Fraser Health. A researcher friend of mine asked Dr. Van Buynder for the evidence used to support this new policy. He received a reference list that fits into the ‘shotgun’ category of literature reviews, containing dozens and dozens of references, some relevant studies as well as editorials, commentaries, eminent opinions, and other detritus. Hmmm. And this is somehow supposed to placate us as a reasonable ‘evidence base’?

When you use a shotgun, you’re likely to hit something – maybe. Included in the province’s ‘evidence’ to support the new policy was a whole range of studies and outcomes, including some deemed ‘biologically implausible’ - such as that vaccinating health workers reduces death by all causes, which is to say the ‘flu shot also prevents death by strangulation, gun shots, and zombies.

To counter this, I looked for the sniper rifle and spoke to Dr. Tom Jefferson, a Rome-based researcher who produces ‘flu vaccine reviews for The Cochrane Collaboration. He’s been doing systematic reviews of ‘flu vaccines and ‘flu drugs for over a decade, so he has more than a passing interest in the subject. Jefferson’s team examined four large cluster randomised trials and one cohort trial of nearly 20,000 healthcare workers. According to their findings, the ‘flu vaccine showed “no effect on specific outcomes: laboratory-proven influenza, pneumonia, or deaths from pneumonia”.

In other words, the ‘flu policy, while eminently agreeable, is unsupported by evidence that has been systematically collected, critically evaluated, and properly synthesised. BC is not the only jurisdiction to adopt a ‘trust us, we’re experts’ pose, but in my opinion, that response is only fit for underlings, not intelligent, responsible healthcare workers facing the pointy end of a syringe this season.

Exhibit C
When my book came out last year, the Vancouver Sun asked me if I’d like to publicly debate the prostate-specific antigen (PSA) test, a blood test used to detect signs of prostate cancer. “Whoopee”, I said. I jumped at the chance to step in the ring with a prominent urologist at University of British Columbia over the value of a very controversial test for a disease that, although occasionally fatal, mostly isn’t, while the PSA test can make many men incontinent or impotent due to unnecessary treatment. The motto for the PSA test, synthesised by the United States Preventive Services Task Force (USPSTF), which uses strong systematic evidence gathering and synthesis is this: ‘Just Don’t Do It’. That drove the urologists nuts.

I really wanted to hear the urologist stand up and tell me the USPSTF evidence is wrong and that we need to keep subjecting men to the PSA test. Two days before the debate, the urologist backed out and I understand why: eminence-based medicine can look very silly in a public debate. He probably felt I was going to wipe the floor with him, but if you know any urologists who would like to debate the PSA with me, please contact me via www.alancassels.com

A final note
A recent study from US Public Citizen found that, since 1991, there have been 239 legal settlements, totaling $30.2 billion in federal and state penalties, levied against US pharmaceutical companies. There’s a real laundry list of crimes, but defrauding the government, hiding drug safety information, and hawking drugs for purposes beyond which they are approved are the main ones. Drug companies have pledged to change, signed ‘corporate integrity agreements’ and indicated that they want to move on, promising a better future. We can be hopeful, but we also have to be realistic. Paying huge fines for illegal activity is one thing, but will they be still playing the eminence game? Will they continue to fund their own experts and do research that goes through a selective reporting of ‘the evidence’? Sadly, that’s probably going to be the case so you must immunise yourself: keep asking questions and questioning answers.

Comments

Re: Eminence vs. evidence

Reminds me too much of the cigarette safety studies  in recent years....

Re: Eminence vs. evidence

hmm nicely written, I would also love to re post the link on my page... keep the detailed and good work coming

  • These online gambling australia have a place for all types of people today, additionally a multilingual seasoned aid team that's surely obtainable previously mentioned the cellphone all times in the week.

Re: Eminence vs. evidence

Could I post the article and and link on my site?

Re: Eminence vs. evidence

You are welcome to post a link to the article, but please do not reprint the text in full without submitting a full request, detailing how and where it would be used, for the author's attention at web@cochrane.org. Thank you.

Re: Eminence vs. evidence

I would like to be able to tweet the link and/or post on my Facebook page. Is this feasible?

Zbys

Re: Eminence vs. evidence

Zbys - yes, please feel free to publicise the link to this blog post.

Nancy, Cochrane Web Team

Re: Eminence vs. evidence

I'm not entirely sure I can agree with the implicit message of this post, which seems dangerously close to "anyone who disagrees with the conclusions reached by the Cochrane Collaboration is wrong, and likely a paid shill of industry".

Does systematic review eliminate subjectivity and the influence of personal bias?  A Pubmed search on almost any subject shows that the "ping pong" of dueling expert opinions has largely been replaced by the ping pong of dueling meta analyses.  Statins are useful in primary prevention or they aren't. Tiotropium increases mortality or it doesn't. Varenicline poses a severe CV risk to patients with no CV disease history or poses no risk at all in this group.  Reboxetine has no demonstrable anti-depressant activity or is similar to other antidepressants. 

While Cochrane Group and others have extensively discussed the potential for selective trial publication to bias comprehensive reviews, the subjective process of selecting studies for inclusion and choosing the most appropriate statistical analysis seems to have played a major role in the conflicting conclusions obtained in many of these recent studies.

 Ping pong is alive and well.

Re: Eminence vs. evidence

John, I think you missed the message. The author isn't saying that Cochrane Reviews and Meta-Analyses lack subjectivity. Neither did the author say that Cochrane authors balk at criticism, nor that anyone who disagrees with the Collaboration is a "paid shill of industry," as you claim. The author is simply alluding to the fact that "experts" funded by industry sometimes are unwilling to share adverse event  data because they are funded by industry. Or, when they do share some of their data, they cherry pick. In almost every research study, including systematic reviews and meta-analyses, we cherry pick somewhat. That is partly why we pick primary outcomes and secondary outcomes, or pursue research that is not overambitious. But the difference between a well-done systematic review versus an "expert" who simply speaks for the sake of speaking is that the systematic review or meta-analysis is subject to greater scrutiny. Further, Cochrane reviews are often updated and when evidence is weak, results are still shared.

NB: These are my personal views and do not necessarily represent the views of the Collaboration

Post new comment

The content of this field is kept private and will not be shown publicly.
If you do NOT enter a subject, the first few words of your response will be used.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <p> <br> <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>

More information about formatting options

Updated on: November 30, 2012, 6:57

The Cochrane Official Blog is curated and maintained by the Cochrane Web Team. To submit items for publication to the blog, please email web@cochrane.org.

The Cochrane Blog presents commentary and personal opinion on topics of interest from a range of contributors to the work of The Cochrane Collaboration. Opinions posted on the Cochrane Blog are those of the individual contributors and do not necessarily reflect the views or policies of The Cochrane Collaboration.