I learned and promptly incorporated this creative method of looking at graphs and deciphering their meaning for a section of the data literacy courses I teach.
The graphs are released on Tuesdays, facilitated on Wednesdays and discussed with statistical follow-up on Fridays--all with support of the American Statistical Association.
Here is the beauty of the method:
An important habit to integrate into any presentation or workshop--or even anything you write, is the following.
Bring experts up on the stage with you. Clearly it isn't feasible to grab the actual scientist/physician, or expert but when you familiarize yourself with their body of work you can integrate their insights into your discussions. Plenary Sessions by Vinay Prasad MD, MPH is an essential part of my podcast habit especially since I work with a lot of oncology data.
You can easily apply the principles of the NY Times, What's Going on in This Graph lessons to your understanding of clinical publications. The recent JAMA Oncology journal article, Profiling Preexisting Antibodies in Patients Treated With Anti–PD-1 Therapy for Advanced Non–Small Cell Lung Cancer introduced a familiar bias as identified by Vinay Prasad and others, the guaranteed-time bias or how I recall learning about it--immortal-time bias.
In a nutshell, this means "immortal time is a span of cohort follow-up during which, because of exposure definition, the outcome under study could not occur". This can be by trial design, death, or the fact that the study outcome can not occur--because perhaps the study participant did not live long enough or dropped out of the study.
So what do you notice?
The objective of the JAMA Oncology article states the following, "To assess the safety and efficacy of anti–PD-1 treatment in patients with subclinical disease with advanced NSCLC and with or without preexisting autoimmune markers, including rheumatoid factor, antinuclear antibody, antithyroglobulin, and antithyroid peroxidase; and to assess potential clinical biomarkers that may be meaningfully and conveniently associated with clinical benefit or with irAEs following anti–PD-1 treatment."
I notice that you have to search until figure 3 to notice that there is no statistical difference in overall survival between patients with or without pre-existing autoimmune markers (rheumatoid factor, antinuclear antibody, antithyroglobulin, and antithyroid peroxidase).
I also noticed that Main Outcomes and Measures section ignore overall survival and only state the measures such as progression free survival (PFS) and immune-related adverse events (irAEs), which by the way, did report p-values < 0.05.
The antibody markers are either present at baseline or not. All good here. A hazard ratio of 0.72 with a p value of 0.19 are arguably nothing to write home about.
What do you wonder?
Now you might have missed this without being guided by the discussion in the Plenary Sessions podcast but the paper does away with the overall survival argument until we view graphs comparing irAEs. Why does this matter? Unlike the presence of antibodies at or before time "0", adverse events occur after time "0". You have to receive the treatment before you can experience an adverse reaction.
This is where the immortal-time bias becomes relevant.
What do you need in order to experience adverse events? You need to be alive. These irAEs have been associated with response to nivolumab. Perhaps the group with irAEs have longer overall survival because they had to survive to have the immune related adverse event.
See the problem here?
The potential for guarantee-time bias (GTB), also known as immortal time bias, exists whenever an analysis that is timed from enrollment or random assignment, such as disease-free or overall survival, is compared across groups defined by a classifying event occurring sometime during follow-up.
Figure 1 cleverly moved to the front of the article regardless of its dubious immortal time-bias and surrogate end-point.
Figure 2 below is looking at progression-free survival only with or without presence of any antibody
There are ways (likely known to the authors) to avoid this type of bias. For starters, use landmark survival analysis to plot all patients alive at say week 10. Did the patients with immune-related adverse events have better outcomes? Read this article if you are interested in details of avoiding time bias in clinical trials, Challenges of guarantee-time bias. Here is a visually helpful description from the research paper.
Patients who undergo transplantation seem to have longer survival times. However, patients in the transplantation group must survive at least until donor is available (blue), and their survival reflects total time before and after transplantation (blue and gold).
Have data questions? Send them along...
If a picture is worth a thousand words, a graphic is at least worth a million numbers.
Sign up for our newsletter!
Browse the archive...
Thank you for making a donution!
In a world of "evidence-based" medicine I am a bigger fan of practice-based evidence.
Remember the quote by Upton Sinclair...
“It is difficult to get a man to understand something, when his salary depends upon his not understanding it!”