Support vs. illumination--how are we using statistics?

He uses statistics as a drunken man uses lamp posts - for support rather than for illumination--Andrew Lang, Scottish poet.

I am not a statistician. Having said that, I have on the other hand, taken multiple higher level courses in statistics over the years. I attend statistical conferences, workshops, and do a lot of independent reading in biostatistics, medical statistics, and multivariate analysis. I would argue the fact that this is perhaps more than the targeted readers of our most esteemed medical journals--or at least on par.

But I am often left scratching my head. In workshops, after a proper visualization history and priming I begin with a relatively simple graphic. An enthusiastic volunteer will often comply with my request for a headline for the graphic. The sort of headline we would like to see in a clinical journal not necessarily in digital or print journalism at large.

I would argue--based on dozens of data literacy in healthcare workshops--this is where we are losing our audiences. Complex curves and obtuse statistical models are not communicating effectively in the absence of context. Often, authors report intent-to-treat analysis but a granular examination of data describes censuring of data (the measured event doesn’t happen while the subject is still being monitored)--did they die or drop out of the trial?

​Why do I mention this? More than once I have raised my hand at international statistical conferences asking who the target audience is for the messages they are hoping to deliver. Esoteric phrases that include terms like heteroscedasticity, eigenvalues, or the relevance of long-rank stratification without explanation or clarification continue to isolate the importance of numeracy.

I am like you. I read the clinical literature especially when something highly anticipated is reported. You might be surprised that although there is no shortage of buzz around immuno-onocology drugs we have none that report benefit over standard of care or placebo based on overall survival. Benefit has been demonstrated in select patients with respect to progression-free survival but I would say there is much we don't know about patient selection, durable effects, and how to translate the findings.

BACKGROUND
An earlier analysis of this phase 3 trial showed that the addition of a cyclin-dependent kinase 4 and 6 (CDK4/6) inhibitor to endocrine therapy provided a greater benefit with regard to progression-free survival than endocrine therapy alone in premenopausal or perimenopausal patients with advanced hormone-receptor–positive, human epidermal growth factor receptor 2 (HER2)–negative breast cancer. Here we report the results of a protocol-specified interim analysis of the key secondary end point of overall survival.

As a long-time reader/listener of Vinay Prasad--most recently his podcast Plenary Session has been quite illuminating when interrogating the methodology of complex speciality drugs in oncology. Although I urge you to listen to the entire podcast--it is over 2 hours--I have pulled a few salient questions for consideration.

The point of randomized clinical trials is to examine if investigational drugs are better than currently available drug regimens before, during, and after the clinical trial. An interesting point observed during the podcast questioned the number of patients in both active and control arm that appear to not have been assigned subsequent therapy.

The methodology mentions intent to treat populations but it isn't clear (at least to me) if all patients were included in the analyses.

I have included the Kaplan-Meir curves and forest plots from both trials discussed on podcast. It is helpful to listen with the visual so you can follow the discussion, criticism, and limits of the insights to be gleaned from the what at first glance appeared to be pivotal findings.

Here is the Palbociclib over survival curve and forest plot...

Now compare to Ribociclib clinical trial...

We can also observe regional issues as mentioned by Dr Prasad during the podcast.

Why does this all matter? Because there is so much to appreciate and evaluate when looking at clinical research. It isn't just about running data or understanding the statistics, it is often about digging deeper.

Think of what is missing not just what is being displayed in the graphics. The pertinent negative...