Many of you have asked, why data & donuts? The name always generates a smile and an appreciation but few are aware of the organic origin. Have you worked in Academia?
Do you shudder when I mention Journal Club? Often the fellows were presenting their research findings to thin and dwindling crowds of their peers. All it took to get a little bump in audience participation was a well-placed box of donuts. The tag-line quickly became, come for the donuts, stay for the data.
Case in point. Industry migrated toward real world evidence without an infrastructure to support the knowledge shift. Experts in the field commented at DIA/FDA Statistics Forum, "We weren't trained how to model the emerging disparate data sources".
Only the data isn't waiting--pragmatic trials are evolving into a valuable tool to examine real world evidence.
In the pragmatic-explanatory continuum, a randomized controlled trial (RCT) can at one extreme investigate whether a treatment could work in ideal circumstances (explanatory), or at the other extreme, whether it would work in everyday practice (pragmatic). How explanatory or pragmatic a study is can have implications for clinicians, policy makers, patients, researchers, funding bodies, and the public.--Tosh and colleagues 2011
Accuracy of Medical Claims for Identifying Cardiovascular and Bleeding Events After Myocardial Infarction A Secondary Analysis of the TRANSLATE-ACS Study
I use medical claims data fairly regularly. Full disclosure though, I rely only on data with low cost or freely available. It isn't the magic bullet some mathy types claim but once you understand what it can do--It can be a valuable part of the healthcare and medical data story.
Guimaraes et al., asked the timely question
"Can medical claims be used to accurately assess cardiovascular and bleeding events after myocardial infarction in an all-ages population?
In short, the authors performed a post-hoc analysis of the TRANSLATE-ACS observational study to compare incidences of bill-identified events by either medical claims data or by physician adjudication to identify the accuracy in identifying potential outcomes.
The k statistic is a measure of inter-rater agreement (between the physician adjudicator and medical claims data). They are classifying the patients or events into "mutually exclusive" events.
Data were analyzed from January 30, 2015, to March 2, 2017. We calculated the total number of each event type when identified by medical claims vs when physician adjudicated. We also calculated cumulative incidence rates at the patient level of each event type and the combined outcome of death, MI, and stroke when defined by the 2 respective methods.
Event rates at 1 year were lower for MI, stroke, and bleeding when medical claims were used rather than physician adjudication. Moderate agreement between medical claims and physician adjudication was observed in ascertaining MI and stroke events, but agreement was worse for bleeding events. While medical claims may be a reasonable resource to assess MI and stroke outcomes, caution is still needed. Medical claims have limited accuracy in identifying bleeding events, which suggests the need for an alternative approach to ensure good safety surveillance in cardiovascular studies.
- Pragmatic Trials are important to move research quickly into practice
- Pragmatic Trials add Complications
- Can this study be answered using a pragmatic trial approach?
- Study design needs to be flexible
- EHR data is valuable--critical to understand the performance of ALL measures
- Appropriate analysis integrating design, randomization, and outcomes
- Many statistical questions still being addressed...transparency is always appreciated.