data talks, people mumble

(Data) love in the time of Covid-19...

4/21/2020

I wish I could recall which came first, my love of data visualization or being mesmerized by Edward Tufte. Either way, I continue to be surprised by the narrative that accompanies the graphics he displays. After all, it is his "voice" I hear when I am observing and truly engaged with a map or any other graphic.

Following the onslaught of COVID-19 maps I was immediately reminded of the novel approach regarding the cholera epidemic in 1854.

John Snow resisted the tally of "each days bad news" in favor of looking for a cause-effect relationship. Take a look at the cluster of deaths over on Broad Street in the D region.

I imagine you have heard the story of the water pump being correlated if not causative in the outbreak--and once it was removed, the epic tale was history.

Not sure if this escaped your interest but we have had our own modern day water pump in theories about the widespread COVID-19 in New York. The part of the water pump is being played by subway turnstiles and is a fascinating read if nothing else. The following is a working paper that reads like a conversation--one of the reasons I enjoyed reading and interpreting the data. Full disclosure please be aware that this National Bureau of Economic Research paper is not peer-reviewed and is circulated for discussion and comments only.

THE SUBWAYS SEEDED THE MASSIVE CORONAVIRUS EPIDEMIC IN NEW YORK CITY

Abstract:
New York City’s multitentacled subway system was a major disseminator – if not the principal transmission vehicle – of coronavirus infection during the initial takeoff of the massive epidemic that became evident throughout the city during March 2020. The near shutoff of subway ridership in Manhattan – down by over 90 percent at the end of March – correlates strongly with the substantial increase in the doubling time of new cases in this borough. Maps of subway station turnstile entries, superimposed upon zip code-level maps of reported coronavirus incidence, are strongly consistent with subway-facilitated disease propagation. Local train lines appear to have a higher propensity to transmit infection than express lines. Reciprocal seeding of infection appears to be the best explanation for the emergence of a single hotspot in Midtown West in Manhattan. Bus hubs may have served as secondary transmission routes out to the periphery of the city.

For each station, the idea is first to compute the time trends in turnstile entries and coronavirus incidence, and then assesses whether there is a relation between the two trends across different subway stations (Fredriksson and Oliviera 2019). Unfortunately, there is a serious problem with this extraordinarily popular method of doing policy analysis (Bertrand, Duflo, and Mullainathan 2004). In particular, there is likely to be significant serial correlation in the outcomes among adjacent subway stations situated along the same line.

The problem, put differently, is that the individual subway stations are not epidemiologically independent entities.

Following the realization that looking at the individual subway stations may not be the appropriate unit of analysis, the discussion reveals the utility of considering subway lines. I will summarize the static model of epidemic propagation discussed in more detail in the paper but basically susceptible individuals are classified as S and their contact with infectious individuals is classified as I.

Incidence of new infection depends on the frequency of contact between S and I and the probability that there is transmission of infection.

The Goscé model offers a number of insights that are immediately applicable to the data from the New York City Flushing subway line. The first is that the rate of disease transmission is related to the number of trips and average number of stations per trip along the entire subway line, and not just to the number of entries at any one subway station. Second, passengers entering the subway line even at a remote, less populous station are slowing down the system, thus increasing the transit time that the S’s stay in contact with the I’s. Third, those uninfected S- passengers who cram shoulder-to-shoulder into a particular subway are increasing train-car density and thus raising the average number of other S-passengers infected by an I-passenger who happens to be standing in the middle of the train. Fourth, local trains – like the Flushing local – are more likely to seed epidemic infections than express lines. Finally, an entire subway line, rather than the individual stations or subway cars, is the appropriate unit of analysis.

An important consideration is the impact of reducing train service likely accelerated the spread of virus as commuters found themselves crammed into fewer cars for longer periods of time.

One distinguishing factor between the present study and prior work is that seasonal influenza has generally had a reproductive number R in the range of 1.2–1.4, while pandemic influenza has had an R in the range of 1.4–1.8, with the high end representing the 1918 pandemic (Biggerstaff et al. 2014). By contrast, we have estimated the R in New York City during the initial surge of infections in early March to be on the order of 3.4 (Harris 2020). An overall assessment of these research efforts may lead some scientific reviewers to conclude that cause-and-effect remains difficult to prove. Still, we doubt whether any public health practitioner would be reluctant to take action on the basis of the facts we now know.

Harris, J. E. 2020. The Coronavirus Epidemic Curve Is Already Flattening in New York City.
https://www.nber.org/papers/w26917,
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3563985: National Bureau of Economic Research Working Paper No. 26917, April 3, 2020

"Do what you can, where you are, with what you have" -- Teddy Roosevelt

4/14/2020

If you want data--we have data. The Johns Hopkins Interactive Resource provides the data that fuels many of the graphics exploding across the internet. I have shared thoughts in panel discussions and here, Data after coronavirus...what survives?

In a reality that may never return, I present topics in data literacy across a wide-variety of industries but mostly community or population level data. When you are live with full access to attendees it is synergistic to be able to offer clarifications, deep dives into topics that arise, or even question how data is sourced, prepared, analyzed, and communicated. More importantly--we can challenge how the data question was formulated. Often, this requires modifying questions to better serve the data available.

In publications with limited space for back-story and education regarding terms, models, or algorithms used we can unintentionally mislead. In fact, the format of my classes often begins with the viewing of art standing in for graphics, selected to reveal biases we may not even be aware of...

In the current environment where we are isolated, bombarded with information, and perhaps fearful this is evolving into a perfect storm for misinterpretation. I have noticed battles on twitter between statisticians, epidemiologists, data scientists, and even economists. Lots of grumbling about statistical models, weak assumptions, and who bears the right to pontificate or offer expertise.

Let me tell you my perspective for what it may or may not be worth. I believe that statistics, epidemiological principles, data science, and economics are all tools and information we need to understand as data professionals. Learning to read a visualization and to create them requires a deep understanding of a lot of edges from different industries.

But here is also the thing. I use many tools, like python for example, without the understanding of a developer. Perhaps this is easier than falling short in other skills because it is hard to reach a wrong conclusion if you can't even get the code to run! I don't feel inferior, and neither should you, when using statistical tools or complex analytic algorithms. If they are to be applied to complex problems we should be able to gain a workable amount of fluency to know what we know and hope for collaboration and conversation when we are wandering off in the wrong direction.

I would like to roll back this discussion to a few foundational elements of the maths and assumptions that underlie much of the confusion. And where I think we need the experts to clarify and engage in a narrative that elucidates not isolates.

The notion that we can manage without models and that sufficient quantities of data—big data—can take the place of models is a seductive one.--What is the Purpose of Statistical Modeling

We can gather data of the scale visualized in the COVID-19 Dashboard, but do the numbers indeed speak for themselves? It is vital to recognize that there are indeed different types of models. Only one of them, "data-driven, empirical, or interpolatory" can't be wrong. It is simply summarizing underlying data--empirical models can however; serve no purpose and have low value.

On the one hand we have theory-driven, theoretical, mechanistic, or iconic models, and on the other hand we have data-driven, empirical, or interpolatory models. Theory-driven models encapsulate some kind of understanding (theory, hypothesis, conjecture) about the mechanism underlying the data, such as Newton’s Laws of motion in mechanics, or prospect theory in psychology. In contrast, data-driven models merely seek to summarize or describe the data.--What is the purpose of statistical modeling?

David Hand, Professor of Mathematics and Senior Research Investigator at Imperial college in London and author of What is the Purpose of Statistical Modeling published in the Harvard Data Science Review, cautions that Theory-driven models can indeed be wrong or misleading. Think about the scope of COVID-19 visualized by confirmed cases, death tallies, and hospitalization rates perhaps not representing the actual reality they are intended to represent.

For example, if you are not familiar with data visualization or statistics and simply view the COVID-19 projections as published by the Institute for Health Metrics and Evaluation you may not realize that the light purple shaded graphic represents the uncertainty around the measures. I rely on these graphics to describe resource allocation projections but not knowing this vital piece of information if all you are doing is a quick glance can change the game.

The Financial Times has been my favorite resource. They are offering free access to COVID-19 stories (thankfully). I read my allotment of complimentary stories but the $60/month fee for full access is a little steep. I like the readability and well-annotated graphics.

The more you read and look at data visualizations the more there is to learn. Pre-attentive attributes guide our attention but aren't reliable for determining what information might indeed be missing.

We should consider the Breiman definition of information, "to extract information about how nature is associating the response variables to the input variables."

You might detect the illusion of prediction and information when we are missing many of the input variables to describe relative frequencies of disease (COVID-19 testing across the population regardless of symptoms), modes of transmission, estimates of actual number of cases. These limitations will indeed impact our ability to plan interventions and allocate health resources.

For prediction, data-driven models are ideal–indeed in some sense optimal. Given the model form (e.g. a linear relationship between variables) or a criterion to be optimized (e.g. a sum of squared errors), they can give the best fitting model of this form, and if the criterion is related to predictive accuracy, the result is necessarily good within the model family. In contrast, theory-driven models are required for understanding, although of course they can also be used for prediction.--David Hand, What is the purpose of statistical modeling

I was recently discussing limitations and potential harms of using readily available statistics reported with the rapidly accumulating data. The Tableau Dashboard although well-intentioned might tempt many to generate graphics of limited value. In times like these, I would definitely continue to explore and learn about the data through these free platforms but I would caution the data family to yield the outcomes and insights to professionals. Here is a summary of insights gleaned from a recent article in The Guardian, Coronavirus statistics: what can we trust and what should we ignore?

I would be cautious of data reporting a daily count of confirmed cases or new deaths.
We are not testing the entire population. Determinations of eligibility for testing is widely heterogenous. Consider counties where you have to be admitted to a hospital or exhibiting profound symptoms vs. exposure to a positive case to be confirmed vs. testing the whole population.

If the sickest of all of us are being tested--would you be surprised to see increasing death rates? What about deaths not attributed to confirmed cases but likely due to COVID-19? Are we testing the dead? What if the death occurs before the test results have been returned? How are co-morbidities being attributed on death certificates?

What about false negatives? We have an expanded pool of professionals applying swabs to nasal passages and throats--how are individuals previously tested as negative but now positive counted? How will home-testing impact the sensitivity and specificity of testing?

What methods are used to smooth the data so we can capture trends?

Logarithmic scales allow comparisons between populations--many have opinions on this but I think it reflects the exponential viral growth. Yes, you might miss the overall magnitude of the problem without the s-curve but when we have R-naught driving the spread of disease--I think log scale--as long as it is clearly defined, is helpful and relevant.

Data models can be useful but the media rarely provides the limitations of the chosen models or highlights the uncertainty.

The science behind antibody testing is beyond this discussion but I suggest you listen to this quick tutorial by Peter Attia MD. His podcast is one of only a few resources I read or listen to regularly about COVID-19.

Most of the big, attention-grabbing illustrations of data science in action are data-driven. But if theory-driven models can be wrong, data-driven models can be fragile. By definition they are based on relationships observed within the data which are currently available, and if those data have been chosen by some unrepresentative process, or if they were collected from a non-stationary world, then their predictions or actions based on the models may go awry.--David Hand, What is the purpose of statistical modeling?

Tempering the strong...

3/30/2020

Apparently I am an extroverted introvert. When in social situations I am calm, friendly, and have been accused of being mildly entertaining. I thrive both literally and figuratively on public speaking and being at the podium.The problem is--I prefer to stay home or with small groups of friends.

I tell you this to give you an idea of how I have been tempering the changes of the last month or so. Delays in projects, rescheduling of talks, and serious doubts about the status of conference appearances for the remainder of the year not withstanding--I remain pretty good. I have long been a creature of habit and more importantly, a remote worker. My last W-2 gigs (many years ago) were also jobs where I worked from a remote office and traveled to client locations or to the office on a quasi-quarterly basis.

If isolation tempers the strong, it is the stumbling-block of the uncertain.
Paul Cezanne

I am however learning a new skill. Like many of you I am often in the room with the visualizations I am sharing. It is pretty straightforward to add the necessary context and cautionary tales about bias and the power of pre-attentive attributes when you are pointing to elements within a graphic.

The free data hubs and sharing of raw data are creating exciting opportunities to tell an important narrative but often we haven't done our due diligence. The sources of the data--at least the high quality sources--share the methodology and a variety of resources to help with interpretation.

In a live situation it is second nature to show limits of data or extrapolations to be made, discuss aggregation and granularity, and define terms used. The challenge in our quick digital mindset is how to make sure the details are included in the comparisons.

Many of us have been watching the pandemic and either relying on data visualizations or recreating our own from raw data. The problem is--there are many missteps and fumbles around what the data is actually capable of contributing to the narrative.

If you are an epidemiologist or have studied epidemiology for public health many of the miscommunications are quite obvious. I think we could all use a better foundation in data literacy and fluency and what better place to start then with a map from Johns Hopkins Coronavirus Resource Center. The data is available for download in GitHub and you will find instructions and guidance. I recommend you read the resources providing information on the terms used to describe the pandemic and important guidelines regarding epidemiology.

Click to set custom HTML

When I view the map, the red sort of creates an ominous and deadly vibe. Yes, people are dying but perhaps we need to see context to understand--fear mongering will only get us so far. I barely noticed the green font depicting the number of people recovered. If the red dots are indicating confirmed cases it is much worse. Confirmed only means they were validated with a test--a test with its own biases and limitations. And we know in the US at least we are limited in testing or even providing the tests to populations of people in our communities.

Context is king when working with large complex datasets.

There are important considerations that need to accompany any visualization but COVID-19 data has a time horizon that is critical to clarify. For example, when were national measures enacted like shelter in place, or self-isolation (shown here by star symbols). What happens if we are only measuring confirmed cases in areas where tests are known to be largely unavailable or limited?

I personally prefer the selection of a logarithmic scale on the y-axis to better convey exponential viral growth. There is a lot to discuss in this graphic. Did you notice that the US does not have a marker indicating a national message to shelter in place? If we observe countries that have issued national orders--how long before the bend in the curve is evident?

Here are a few resources to help you make better visualizations...

The data issue...

2/21/2020

There is a weird facet of my personality that applauds irony in all its iterations. I was asked to speak at a local community event for "innovators and entrepreneurs" highlighting the United Nations Sustainable Goals--specifically equality. My suggestion to introduce the utility of census data and how to access, clean, and analyze for free was welcomed.

Unfortunately, in the absence of effective marketing -- the draw of census data is not exactly standing room only. Because as it turned out--they must have been all standing somewhere else. The attendees mill about drinking free beer and nibbling on heavy hors d'oeuvres and once the second round of talks begin--they are typically engaged in other conversations. Not to worry, I persist.

My motivation in agreeing to give talks for free from time to time is not unlike the comedian ducking into little venues to test out new material.

I prepare for the bigger events by learning from the smaller gatherings. It is easy to gain insights from the questions asked, requests for additional details, or invitations that follow where the good "bits" are and where there may be a little meat missing from the bone.

For example, I teach audiences how to read a job's report. Over a dozen previous attendees sent me a link to the New Yorker article--The Bullshit-Job Boom. They get me.

One of my favorite observations in the article I can relate to from my previous life--"I do digital consultancy for global pharmaceutical companies’ marketing departments. I often work with global PR agencies on this, and write reports with titles like How to Improve Engagement Among Key Digital Health Care Stakeholders. It is pure, unadulterated bullshit, and serves no purpose beyond ticking boxes for marketing departments. . . .

I was recently able to charge around twelve thousand pounds to write a two-page report for a pharmaceutical client to present during a global strategy meeting. The report wasn’t used in the end because they didn’t manage to get to that agenda point."

A book I carry with me everywhere--a textbook and guide through all of the discussions about economics, Measuring What Counts: The Global Movement for Well-Being begins with the following few sentences.

"The world is facing three existential crises: a climate crisis, and inequality crisis, and a crisis in democracy. Will we be able to prosper within our planetary boundaries? Can a modern economy deliver shared prosperity? And can democracies thrive if our economies fail to deliver shared prosperity?"

Back to the census data. To explore broad questions beyond GDP -- the overworked metric of gross domestic product-- it is vital and it is important to dig deeper. And better yet the insights are free once you can tackle the steep learning curve. What an opportunity to meet your potential clients, patients, or customers in the communities where they live. Identify the barriers to improved outcomes by identifying structural determinants and working for policies to ameliorate wide disparities in not only income but opportunity.

If we believe that we, as Americans, are bound together by a common concern for each other, then an urgent national priority is upon us. We must begin to end the disgrace of this other America.

And this is one of the great tasks of leadership for us, as individuals and citizens this year. But even if we act to erase material poverty, there is another greater task, it is to confront the poverty of satisfaction - purpose and dignity - that afflicts us all.

...Gross National Product - if we judge the United States of America by that - that Gross National Product counts air pollution and cigarette advertising, and ambulances to clear our highways of carnage. It counts special locks for our doors and the jails for the people who break them. It counts the destruction of the redwood and the loss of our natural wonder in chaotic sprawl. It counts napalm and counts nuclear warheads and armored cars for the police to fight the riots in our cities. It counts Whitman's rifle* and Speck's knife*, and the television programs which glorify violence in order to sell toys to our children.

Yet the gross national product does not allow for the health of our children, the quality of their education or the joy of their play. It does not include the beauty of our poetry or the strength of our marriages, the intelligence of our public debate or the integrity of our public officials. It measures neither our wit nor our courage, neither our wisdom nor our learning, neither our compassion nor our devotion to our country, it measures everything in short, except that which makes life worthwhile. And it can tell us everything about America except why we are proud that we are Americans.-- Robert F. Kennedy March 18, 1968 *mass murderers

We are all homeless...

1/20/2020

Whether measured by inequality, the nature of the jobs market, or even lifespan, America’s poor have little to fall back on. To Lind, this is no accident. Governments have been cutting back on social safety nets at a time when employers have restricted benefits to the most highly credentialed jobs.

As Warren Buffett, the billionaire investor, said in 2006: “There’s class warfare, all right, but it’s my class, the rich class, that’s making war and we’re winning.”--Populism and the smouldering rage of American poverty, Financial Times

If you said I was stubborn you wouldn't be a liar. I refused to acknowledge the signs of an oncoming cold. Figuring I could run it out I did an easy 10 mile run hoping my oxygenated lungs would expunge the irritants and I would be back to being shiny and new. Let's just say all didn't go as planned and I spent the afternoon sipping tea spiked with a bit of whiskey and watching a few documentaries.

Signs of Humanity was a brilliant surprise. Willie Baronet is an artist and professor in Texas. Well, after watching his documentary I can say with confidence--he is also a filmmaker. His story illustrates the humanity and compassion evident in his interviews of over 100 homeless people. Offering to purchase their signs, he collects them and creates art installations to bring awareness and conversation to the front line of our debates on community and policy change.

I have always know that poverty isn't simply one thing. It is a cascade of small and large tragedies that can leave us hopeless, bereft, and completely alone.

Watch Signs of Humanity. If you have a Prime account it is free.

As an analyst, I can only measure what I bring to the discussion. If you write about poverty, social determinants of health, or other variables with an easy numeric tally or comparator you are leaving data on the table. The tensions we hold can help inform and elevate discussions.

There is no "other" in discussions of poverty. In an economy where we must keep our fingers crossed that we don't lose our jobs (and the benefits they provide), become ill, or need to reduce work load to care for ailing parents--there is no floor. You can drop right down to the bottom at the blink of an eye.

Over the years we have been lucky. My husband and I had our parents during the fragile years of building our own little family. There were so many random challenges that didn't seem to care if we were highly educated and well compensated. His boss shot himself in the heart and we were left without a steady income that had seemed teflon over the prior 17 year period. I once worked in Pharma and as companies were sold, merged, and scrapped--I began to appreciate the fragility of long term security.

I work in healthcare for the human side of medicine--not the profit motives winding through our fragile US health system. I want us to ask better questions and to do a better job at questioning answers. We need to pay attention, become data literate, and share our stories.

Follow along with me as we explore census data, government data, and other resources to help add a human dimension to a much needed narrative...

The potential magnitude of chance events

1/8/2020

Think about the human genome, with its 3 billion base pairs: even in a very large randomised trial, say of 20 000 people, thousands of genetic differences are sure to arise between the groups. Some of these differences—which we do not yet know— might be important for prognosis. Randomisation guarantees that such differences are indeed due to chance. It means that statistical theory based on random sampling can be used to calculate confidence intervals that express the potential magnitude of such chance events...Vandenbroucke 2004 When are observational studies as credible as randomized trials?

I am not exactly sure when it is appropriate to affix the badge of data scientist or analyst to your lapel and boldly saunter out into the world. The battles are softly waged with words as statisticians claim certain inalienable rights as loose definitions of data literacy are assigned to the spoils. I am only responsible for my little corner of the world.

Through several post-graduate degrees I learned statistics and most recently completed an executive education program from the Fu School of Engineering at Columbia in Applied Analytics. The curriculum was quite a slog as mastery in Python came at the expense of real-time skills applicable to daily questions. I completed the course with a score of 97% but felt a little overwhelmed. I had questions. Questions that needed answers and answers that screamed for clarity. Slowly I began tackling Python library by library. I started with Pandas and began to gain proficiency outside of the need to pass an exam or submit a capstone.

So here we are. A data curious human navigates the world of health economics, health policy, and clinical medicine. Think about this. If we aren't intended to be able to clearly understand the research or media discussions in this space--who is the audience? My goal isn't to provide unyielding answers to these perturbations but to point and say, "What is that?" and "Why does this matter?"

When guiding or facilitating discussions around data literacy of fluency, I pull clinical research into the discussion and collaboratively step through the numbers to help inform colleagues about best practices or even self governance in relying on data to curate insights. For example, we may understand that randomized controlled trials are the "gold" standard for answering questions about tolerability, efficacy, and safety but is this always the case?

I am not debating the role of a well-designed study powered to answer a question but we need to understand the role of observational studies and the potential limits of RCTs. The majority of RCTs are limited by size and follow-up periods. If we are attempting to say anything meaningful about duration of response, safety or adverse events we need to look at case-control of large-scale observational studies.

To evaluate causal mechanisms of disease for example, we need to be able to navigate observational studies. How can we identify or adjust for confounding? Where do we begin while acknowledging that many confounders are insufficiently known or are unquantifiable?

What if we can create a visual representation of causal assumptions to identify potential pathways of confounding?

Directed acyclic graphs or DAGs have the variables connected by arrows that are all directed or pointing in the same direction. These represent direct causal effects. If we remove the arrow, the effect is no longer observed. And because causes (such as exposures) precede the effect (disease) we use a sort of chronology when creating a DAG.

Our a priori knowledge informs how we design the question.

directed acyclic graphs (Nephrology Dialysis Transplantation, Volume 30, Issue 9, September 2015, Pages 1418–1423)

A graphical presentation of confounding in DAGs. (a) The structure of confounding in DAGs. Since age is a common cause of CKD and mortality, confounding is present when we want to assess the causal relationship between the exposure CKD and the outcome mortality (b). The backdoor path from CKD via age to mortality can be blocked by conditioning on age, as depicted by a box around age in (c). Similarly, ethnicity is a common cause of obesity and decline in kidney function (d). The backdoor path from obesity via ethnicity to decline in kidney function can be blocked by conditioning on ethnicity. If ethnicity is not measured or not properly measured, residual confounding remains present.

As described by Suttorp and colleagues, the three criteria of confounding are:

1. Confounder must have an association with the outcome
2. Confounder must be associated with the exposure
3. Confounder must not be in the causal path from exposure to outcome (not a consequence of the exposure)

Confounding distorts the actual effects so now we need to remove as much of the impact as possible. The methods vary but it is critical to know how the confounding was identified (often if and methods used). To address confounding by age when evaluating the relationship between chronic kidney disease (CKD) and mortality, we might, for example, stratify by age.

Conditioning is the term used for adjusting for confounding and includes restriction, stratification, or multivariable analysis. The box around age in the figure above demonstrates that this confounder has been blocked.

What about the causal relationship between obesity and decline in kidney function as described in (d) above? If we make assumptions regarding race (research article conflates race and ethnicity--it is a social construct being misused as a biologic proxy but for purposes of the discussion I am not poking that tiger)-Differences in progression to ESRD between black and white patients receiving predialysis care in a universal health care system.-- and integrate prior research describing a faster decline in kidney function and progression to end-stage renal disease (ESRD) in blacks and higher obesity rates in African Americans (again, are we talking ethnicity or race?), we might define ethnicity as a confounder.

...It is, however, possible to identify confounding in a DAG that is impossible to adjust for. For instance, it could be that physicians did not record ethnicity, and ethnicity is thus unavailable in the data analyses. The investigator cannot adjust for a factor that is not measured. Similarly, it is possible that adjustments are only partly successful in controlling for confounding. For example, even if ethnicity was recorded and adjusted for in the analyses, some residual confounding can remain present.

The reason for this is that self-reported or physician-reported race does not always completely represent the racial background of an individual. When confounding is unknown, unmeasured or even partially measured and adjusted for, residual confounding will remain present. This is also the problem with confounding by indication. A physician's treatment decision is based on many factors, including the physician's preference and estimation of the patient's outcome, and it is almost impossible to completely measure all these factors. Randomized controlled trials are therefore the best way to avoid confounding by indication.--Graphical presentation of confounding in directed acyclic graphs

More to come (stay tuned...)

Resources

When are observational studies as credible as randomized trials?

Graphical presentation of confounding in directed acyclic graphs

What does race have to do with it?

12/11/2019

I don't have a bunch of answers. What I have are questions. In fact, when I talk about data fluency I prefer to "group think" solutions. We try and unpack the information available to us and trace a path to critical thinking.

Reading through the article below, once again probably launched more questions than it attempted to answer.

Low-Dose Aspirin for Primary Prevention of Cardiovascular Disease: Use Patterns and Impact Across Race and Ethnicity in the Southern Community Cohort Study--click for article

American Heart Association news summarized the findings, A racial gap in the heart attack benefits of aspirin?

"We think the reason aspirin use did not have a beneficial effect for African Americans could involve a different genetic response to aspirin therapy and poor control of other risk factors," said Dr. Rodrigo Fernandez-Jimenez, lead author of the study. He is a cardiologist and researcher at Centro Nacional de Investigaciones Cardiovasculares in Madrid.

But when I look at the classification of "black" race contrasted with birth country of origin you can see why lumping a diverse population under a poor proxy for biology or a substitute for a political construct makes no sense in medicine (or anywhere else for that matter).

Data visualization below is from my workshop on racial bias in data...think about how heterogenous genetic profiles would be in an aggregated classification of "race".

Reading through the graphics in the article I was unable to look at the variables classified by race--they were aggregated at too high of a level to invite inquiry or additional analyses.

For example, why isn't the data stratified by race AND risk? Is it me or am I unable to determine how many blacks are low, intermediate, or high risk stratified by education or health insurance status? Wouldn't access to healthcare be a potential driver of being "prescribed" low dose aspirin?

I recently examined how access to healthcare influences algorithms attempting to manage the health of populations within a large health system. The algorithm was adopted to help identify patients with complex needs that would benefit from an intervention to help mitigate co-morbid disease and higher risk profiles.

It is worth digging into the article to see how the referral to the intervention becomes more equitable once actual levels of disease are implemented instead of healthcare costs. When the algorithm is applied as developed, black patients have to be far more ill before being referred to the program as compared to white patients.

Back to the Low Dose aspirin study. To add more confusion or obfuscation--the measure regarding health insurance status is not included in the final graphic...

Keep an eye on the YouTube space if you would like to see the full presentation from Tableau Fringe Festival. They should have videos posted by the end of the month.

Measuring what we should, not what we can...

10/26/2019

Do you write survey questions? Use them as a measure of knowledge, competency, or behavioral change? Perhaps you are thankful that you don't have to write them, your job is simply to do the analytics.

Your enterprise analytics are set it and forget it. But what if I asked you about your process? I just finished grading assignments in a university class I am facilitating about Understanding Data.

They are lost when asked to describe discrete vs. continuous data types and distinguish between qualitative ordinal, binomial, and nominal data.

It made me want to do a re-wind on data gathering that I think you will find helpful. If you need hands-on help, sign up for the first tutorial in a series I am launching Thursday, Big Data on A Less Big Budget.

But let's not get ahead of ourselves. We start with data identification, sourcing, and gathering for a specific reason. The types of data we collect will determine the types of analyses we can do--and thus, how meaningful our insights might be.

If it was up to me, I would like to see more discrete choice modeling analytics using a Python library like ChoiceModels.

"discrete choice models, or qualitative choice models, describe, explain, and predict choices between two or more discrete alternatives"

But many teams want to keep relying on Likert scales or Likert-type questions, let me explain how this impacts your analytics plan.
Why don't we make sure we are at least doing the best we can.

Daniel Mingook Kim

In my experience, once you get past the murky distinction of nominal, ordinal, and numeric, there is little thought to how you need to adjust your analytics. The time to make these important decisions is immediately after you create a data question or draft a set of objectives for a need assessment.

To collect measures of attitude, character, and personality that better assess post-activity behavior of a healthcare provider after an educational intervention, you likely rely on Likert attitudinal scales and apply them to professional learning and/or behavior change.

The problem is you need to distinguish between Likert-type items and Likert scales. Likert-type questions are when you design single response questions that are not specifically related. You can list them separately and your intention is not to combine the responses or aggregate a score.

When you analyze Likert-type responses, you can sense an ordinal measurement--there is a directionality or a "greater than" or "lesser than" relationship. We think of this as ordinal because you can't say much about the distance between the measures or distinguish the intensity of the difference. Because of this you would use descriptive statistics like mode or median or frequencies to assess variability. This might also include chi-square.

Likert scales are a collection of similar questions where you create a composite score based on a quantitative measure of say how a healthcare provider selects a first-line therapeutic option for a patient with "mild to moderate" rheumatoid arthritis. Because the individual scores are generating a composite or overall measure, now we would use the mean and standard deviations as a measure of variability. These interval scale items would include Pearson's r, t-test, ANOVA, and regression.

This is why decisions around your data analytics start at the question development stage. What type of information are you hoping to gather? What data types do you need to answer your data question?
â€‹You will know this right away if you don't out source data gathering to PubMed published articles or pre-existing survey data.

Beware of low hanging fruit...

Summary of DCA from Columbia University Mailman School of Public Health...

Discrete choice models are appropriate for evaluating behaviors that operate within a framework of rational choice. Health care professional, when confronted with a discrete set of options will opt for maximal benefit or utility. The options most likely to improve patient outcomes for example and/or abide within a framework of evidence-based medical choice.

These decisions are made from both the choices presented at the point of care AND the characteristics of the professional making the final selection. It follows from this assumption that utility of a choice is a function of the characteristics of the possible choices and the characteristics of the person making the choice.

Here is a resource I found extremely helpful--reach out if you are interested in exploring DCA collaboratively.

E. Lancsar, J. Louviere. Conducting discrete choice experiments to inform healthcare decision making. A userâ€™s guide.PharmacoEconomics, 26 (2008), pp. 661â€“677

"Every reform movement has a lunatic fringe"

10/25/2019

It's conference time. Not just Tableau Conference 2019 but viewing my calendar of speaking obligations--quite a few others as well.

But just because we aren't supposed to pick favorites among our children doesn't mean I can't give extra smooches and love to Tableau. I have been to 5 or 6 over the years and they are one of the best production, content, entertainment, and simply experiential conferences I have ever had the pleasure of attending (I have to extend a slow clap to Qualtrics as well).

All of these years later, I had choices to make. When you attend about a half dozen as a speaker, there isn't much time to be in the audience. And let's face it, you should up your game and set your focus on being the one at the podium.

This year due to a new format and fewer speaker opportunities I was not invited to the main stage but was extended an opportunity to speak at the Tableau Fringe Festival. A spot for those of us with ideas worthy of sharing with the community-at-large but maybe not scalable to the main stage.

The videos will be uploaded to the Fringe Festival website in a few weeks so follow the link above. Until then, here are a few of the dashboards I included in my talk, a specific version of Data Talks, Meople Pumble titled Decision Making in The Face of Uncertainty. Which variables and data sources are needed for a dynamic description of poverty?

What's next?

I am teaching an online workshop on Halloween, Big data on a less big budget where we will step through a real world example and demonstrate how to use open source tools to create and answer a data question.

I will see you at Women in Tech Summit in Raleigh Durham November 8th after a busy week of National Press Club workshops and Academic Stand-up in NYC with a featured conversation with Angela Saini author of Superior: The Return of Race Science with a hands-on Python refresher nearby.

"Every reform movement has a lunatic fringe"--Theodore Roosevelt

Cautionary tale to "see and define" not "define and see"

10/4/2019

“For the most part we do not first see, and then define, we define first and then see. In the great blooming, buzzing confusion of the outer world we pick out what our culture has already defined for us, and we tend to perceive that which we have picked out in the form stereotyped for us by our culture.”--Walter Lippmann (American Journalist--Sept. 23, 1889—Dec. 14, 1974)

The basis for teaching data fluency stems from our very own heuristics. Mine too. None of us are above debate regarding our intentions or assumptions.

Allow me to share a specifically non-instagramy moment. Due to a confluence of events--apparently rodent friendly weather, living on a park with a creek, and a lively chicken coop--we became infested. Mainly the yard and coop but who knows? If truth is indeed "poisoned at its source" according to Walter Lippmann, we attacked with gusto.

After all, we defined the problem therefore the solution appeared to be quite straightforward.

Well let's think this through carefully. If the goal was a systemic ridding of the rodent population from the yard--we were advised by professionals to follow the protocol.

But often, in pursuit of an outcome we forget about the stench. In this case I am referring to fetid carcasses decaying just about everywhere you can imagine. Out of sight but definitely not out of olfactory range.

The unpleasant imagery may cloud your ability to see the data adjacent similarities but stay with me here. A good data question often requires the holding of tensions. The pursuit of a complete data set to indeed be empowered to curate empathy or insights. Culturally defined norms often make it unavoidably tempting to believe low hanging fruit and reinforce pre-existing assumptions.

We might ignore relevant information out of fear, repulsion, or conscious bias.

But eventually the stench of bad data will win. And we will need to clean up the mess and start over.

Where to connect--click on images for links to more info!

<<Previous