data&donuts
  • Data & Donuts (thinky thoughts)
  • Time to make the donuts...
  • COLLABORATor
  • donuts (quick nibbles)
  • Data talks, people mumble
  • Cancer: The Brand
  • For writers only...
  • the grapheme
  • Diabetes: the brand
    • Educational strategies in cholesterol management
    • Tug on the thread of high-value-care...
    • Smarter is Better--let's work smarter, not harder
  • datamonger.health

data journalism, bath water, and babies

4/20/2016

 
Picture
All randomized controlled trials aren't good and all observational trials aren't bad. Whew. It feels good to get that information out there.

Too many heuristics floating around  to do any body any good. Lets take a sensible approach to evaluating the quality of evidence.

As journalists, writers, and healthcare stakeholders it is important to not grab that heavy peach. Low-hanging fruit is the manifestation of simplistic rules of thumb--RCTs are the standard for which we measure all clinical evidence for example.

First, a little context to the discussion of evidence-based medicine. One of the main themes of the Lown Annual Conference, is a must read.


As Evidence-based medicine (EBM) became more influential, it was also hijacked to serve agendas different from what it originally aimed for. Influential randomized trials are largely done by and for the benefit of the industry.

Meta-analyses and guidelines have become a factory, mostly also serving vested interests. National and federal research funds are funneled almost exclusively to research with little relevance to health outcomes. We have supported the growth of principal investigators who excel primarily as managers absorbing more money.

Diagnosis and prognosis research and efforts to individualize treatment have fueled recurrent spurious promises. Risk factor epidemiology has excelled in salami-sliced data-dredged articles with gift authorship and has become adept to dictating policy from spurious evidence. Under market pressure, clinical medicine has been transformed to finance-based medicine.

​In many places, medicine and health care are wasting societal resources and becoming a threat to human well-being. Science denialism and quacks are also flourishing and leading more people astray in their life choices, including health. EBM still remains an unmet goal, worthy to be attained.-- Evidence-based medicine has been hijacked: a report to David Sackett


​High Quality of the Evidence for Medical and Other Health-Related Interventions was Uncommon in Cochrane Systematic Reviews
 reports that of 1394 systematic reviews published on the Cochrane Database from January 2013 to June 2014, only 608 (43.6%) incorporated Grades of Recommendation, Assessment, Development, and Evaluation (GRADE). You may also be surprised by the quality of evidence reported for the first listed primary outcome:
  • High quality 13.5%
  • Moderate quality 30.8%
  • Low quality 31.7%
  • Very low level quality 24%

•Consideration of all outcomes--only 19.1% had at least one outcome with high quality of evidence.
•Reviews with high quality of evidence, only 25 had both significant results and a favorable interpretation of the intervention.
Dr. John Ioannidis also discussed findings of The Life Cycle of Translational Research for Medical Interventions providing an insightful graphic to demonstrate the problematic task of drug discovery. What you can observe shows the "median translation lag was 24 years (interquartile range, 14 to 44 years) between first description and earliest highly cited article. This was longer on average (median 44 versus 17 years) for those interventions that were fully or partially "refuted" (contradicted or having initially stronger effects) than for nonrefuted ones (replicated or remaining unchallenged) (P = 0.004)."
Picture

John Ioannidis, MD shared with us the following from Head-to-head randomized trials are mostly industry sponsored and almost always favor the industry-sponsor:

•Among trials published in 2011, 55/57 of non-inferiority trials with head to head comparisons sponsored by the industry demonstrated non-inferiority
•Success rate 96.5%

Flacco et al, J Clin Epidemiol 2015

Legend from article. The figure below describes the network of cosponsorship for the 12 most prolific companies.

Each company is shown by a node whose diameter is proportional to the number of trials sponsored.

Lines represent cosponsorship between companies, with thickness proportional to the number of trials cosponsored.

Dashed lines refer to comparison with only one cosponsored trial. The thickness of the autoloops is proportional to the number of trials where the respective company is the unique sponsor.

Red circles into each node represent the proportion of trials sponsored (or cosponsored) by each company that showed favorable results to the specific company (corresponding percentages are listed next to each node). BMS, Bristol-Myers Squibb.
Picture
Picture
I will continue to post insights and steps we can employ to carefully analyze the quality of evidence behind the clinical research and outcomes reported in the literature. My point is to be aware of heuristics that suggest "always" or "never". You can see that quality is deeper than just the design of the study.

If upstream is dirty, downstream will be muddy – Tibetan Proverb

Comments are closed.
    Browse the archive...
    Picture
    Follow @datamongerbonny
    Thank you for making a donution!
    donations=more content

    Snackable content for your business needs...

    I am a medical/health economics writer/ data analyst, ultra-runner, and mom. 

    In a world of "evidence-based" medicine I am a bigger fan of practice-based evidence.

    Question the quality of the evidence. The motivation for disseminating the evidence.

    Who stands to benefit the most from its uptake?

    ​Remember the quote by Upton Sinclair...


    “It is difficult to get a man to understand something, when his salary depends upon his not understanding it!”

    Data and Donuts has a free weekly newsletter!

    * indicates required
    Email Format
    Tweets by @graphemeconsult

    Categories

    All
    Aging
    Blogging
    Content Marketing
    Creativity
    Food Policy
    Healthcare
    Healthcare Blog
    Health Policy
    Medical Writer
    Presentation
    Value In Healthcare

    Follow the evolution of Alzheimer's Disease into a billion dollar brand
    Picture
Proudly powered by Weebly
✕