Think about the human genome, with its 3 billion base pairs: even in a very large randomised trial, say of 20 000 people, thousands of genetic differences are sure to arise between the groups. Some of these differences—which we do not yet know— might be important for prognosis. Randomisation guarantees that such differences are indeed due to chance. It means that statistical theory based on random sampling can be used to calculate confidence intervals that express the potential magnitude of such chance events...Vandenbroucke 2004 When are observational studies as credible as randomized trials?
I am not exactly sure when it is appropriate to affix the badge of data scientist or analyst to your lapel and boldly saunter out into the world. The battles are softly waged with words as statisticians claim certain inalienable rights as loose definitions of data literacy are assigned to the spoils. I am only responsible for my little corner of the world.
Through several post-graduate degrees I learned statistics and most recently completed an executive education program from the Fu School of Engineering at Columbia in Applied Analytics. The curriculum was quite a slog as mastery in Python came at the expense of real-time skills applicable to daily questions. I completed the course with a score of 97% but felt a little overwhelmed. I had questions. Questions that needed answers and answers that screamed for clarity. Slowly I began tackling Python library by library. I started with Pandas and began to gain proficiency outside of the need to pass an exam or submit a capstone.
So here we are. A data curious human navigates the world of health economics, health policy, and clinical medicine. Think about this. If we aren't intended to be able to clearly understand the research or media discussions in this space--who is the audience? My goal isn't to provide unyielding answers to these perturbations but to point and say, "What is that?" and "Why does this matter?"
When guiding or facilitating discussions around data literacy of fluency, I pull clinical research into the discussion and collaboratively step through the numbers to help inform colleagues about best practices or even self governance in relying on data to curate insights. For example, we may understand that randomized controlled trials are the "gold" standard for answering questions about tolerability, efficacy, and safety but is this always the case?
I am not debating the role of a well-designed study powered to answer a question but we need to understand the role of observational studies and the potential limits of RCTs. The majority of RCTs are limited by size and follow-up periods. If we are attempting to say anything meaningful about duration of response, safety or adverse events we need to look at case-control of large-scale observational studies.
To evaluate causal mechanisms of disease for example, we need to be able to navigate observational studies. How can we identify or adjust for confounding? Where do we begin while acknowledging that many confounders are insufficiently known or are unquantifiable?
What if we can create a visual representation of causal assumptions to identify potential pathways of confounding?
Directed acyclic graphs or DAGs have the variables connected by arrows that are all directed or pointing in the same direction. These represent direct causal effects. If we remove the arrow, the effect is no longer observed. And because causes (such as exposures) precede the effect (disease) we use a sort of chronology when creating a DAG.
Our a priori knowledge informs how we design the question.
directed acyclic graphs (Nephrology Dialysis Transplantation, Volume 30, Issue 9, September 2015, Pages 1418–1423)
A graphical presentation of confounding in DAGs. (a) The structure of confounding in DAGs. Since age is a common cause of CKD and mortality, confounding is present when we want to assess the causal relationship between the exposure CKD and the outcome mortality (b). The backdoor path from CKD via age to mortality can be blocked by conditioning on age, as depicted by a box around age in (c). Similarly, ethnicity is a common cause of obesity and decline in kidney function (d). The backdoor path from obesity via ethnicity to decline in kidney function can be blocked by conditioning on ethnicity. If ethnicity is not measured or not properly measured, residual confounding remains present.
As described by Suttorp and colleagues, the three criteria of confounding are:
1. Confounder must have an association with the outcome
2. Confounder must be associated with the exposure
3. Confounder must not be in the causal path from exposure to outcome (not a consequence of the exposure)
Confounding distorts the actual effects so now we need to remove as much of the impact as possible. The methods vary but it is critical to know how the confounding was identified (often if and methods used). To address confounding by age when evaluating the relationship between chronic kidney disease (CKD) and mortality, we might, for example, stratify by age.
Conditioning is the term used for adjusting for confounding and includes restriction, stratification, or multivariable analysis. The box around age in the figure above demonstrates that this confounder has been blocked.
What about the causal relationship between obesity and decline in kidney function as described in (d) above? If we make assumptions regarding race (research article conflates race and ethnicity--it is a social construct being misused as a biologic proxy but for purposes of the discussion I am not poking that tiger)-Differences in progression to ESRD between black and white patients receiving predialysis care in a universal health care system.-- and integrate prior research describing a faster decline in kidney function and progression to end-stage renal disease (ESRD) in blacks and higher obesity rates in African Americans (again, are we talking ethnicity or race?), we might define ethnicity as a confounder.
...It is, however, possible to identify confounding in a DAG that is impossible to adjust for. For instance, it could be that physicians did not record ethnicity, and ethnicity is thus unavailable in the data analyses. The investigator cannot adjust for a factor that is not measured. Similarly, it is possible that adjustments are only partly successful in controlling for confounding. For example, even if ethnicity was recorded and adjusted for in the analyses, some residual confounding can remain present.
More to come (stay tuned...)