Navigating High-Value Care

Your data team: "ontogeny recapitulates phylogeny"

11/12/2018

The dubious history of Ernst Haeckel and debunking of his biogenetic law aside, I still love the catchphrase he generated--"ontogeny recapitulates phylogeny." The embryo does in certain instances, reflect evolutionary history--pharyngeal gill slits anyone? But it isn't as reminiscent of Lamarckian ideology as our modern day understanding of the complexities of genetics reveal.

I recently heard the phrase attributed to nascent data teams. Think of the pluripotent stem cell for example. Many a data project begins with a small nimble team with an undifferentiated skill set. Angela Bassa, Director of Data Science at iRobot, shares that one of the earliest paths to "value" is by answering easy data queries with an architecture that takes care of itself.

Over time, a data team evolves and grows the ability to mature into anything. This leads to the specialization and differentiation of mature data teams. This sounds tremendously like how data projects evolve in the wild. When the journey is new, you may find yourself dabbling in a bit of the data engineering and data architecture bits regardless of focused expertise in formulating data questions.

The truth of the matter is, everyone has data. Data scientists worth their salt have the skill of formulating a data question and selecting the right statistical and visualization tools to formulate a solution. We strive for replicability, interpretability, and fit for purpose.

Full disclosure. When I am launching new data strategies with clients I focus on team building. One of the best pieces advice, I "borrowed" from Amazon CEO Jeff Bezos.

Perhaps you have heard of the 2 pizza rule.

"It's simple. The more people you pack into the meeting, the less productive the meeting will likely be. The solution?

Never have a meeting where two pizzas couldn't feed the entire group."--Business Insider

If you scour adds for data scientists, what you may notice is a focus on tools rather than asking if you have the imagination to envision what an answer to a data question might look like. In my opinion, the method should be second to the actual problem a team seeks to answer. It reminds me of a data professional that works within a large hospital system in my neighborhood. There are so many legacy systems in his department, they can barely communicate. The seasoned IT team has preferred ways of computing while the newer team members are more likely to be comfortable with the latest and greatest.

How many pizzas do you think it would take to feed those folks?

Listen to a recent post about managing data science teams--this post is a bit of a summary--the episode is worth your ears...

Secret ingredients of a robust medical education strategy

11/3/2018

Headlines within Continuing Medical Education broadcast record attendance at events, "Innovative" learning strategies, and tactics that lead us to believe we are making significant inroads in closing the gaps between evidence-based medicine and actual physician behavior at the point of care.

All those successes include big data integration and improved data literacy, or do they?

First, a little context. The CME industry includes close to $3 billion dollars in educational investments from a wide variety of financial categories including registration fees, commercial support, advertising and exhibits income, private donations, and government grants.

The ACCME's Accreditation Criteria require providers to produce educational activities that are designed to change competence, performance, or patient outcomes. Providers are then required to analyze the changes that were achieved as a result of the activities.
Figure 4 illustrates the percentage of CME provided in 2017 that was designed and/or analyzed for changes in competence, performance, and/or patient outcomes.--2017 ACCME Data Report

Here is where the rigor starts to unravel. The measurement of competence and performance is inherently subjective or observational and simultaneously not evaluated utilizing sophisticated analyses. And the teeny tiny attention to actual patient outcomes is alarming.

Likert metrics, poorly constructed surveys utilizing outdated research methodology, and narrow focus to measure the activity instead of the actual physician level engagement or impact on patient outcomes appears to go unnoticed.

The Accreditation Council for Continuing Medical Education (ACCME) President and CEO writes in the annual report:

CME has responded to the changing health environment. Healthcare is practiced in teams--and clinicians need to learn how to work effectively as teams for the benefit of their patients. While CME traditionally has been focused on physicians, it has never excluded other professions. This data report shows that the number of interactions with learners other than physicians, such as nurses, pharmacists, and physician assistants, has increased nearly 90% over the past 10 years. Overall, learner interactions (physicians and other learners) grew 57% during that time.

Why does this growth matter? Degree and training programs are the foundation of clinician education; for the rest of their careers, clinicians rely on accredited continuing education to help them provide high-quality, safe, compassionate, and effective care to the patients they serve. Those activities help clinicians absorb and process rapidly changing medical information and practice an ever-diversifying set of complex skills.--Graham McMahon, MD, MMSc Accreditation Council for Continuing Medical Education (ACCME)

Not to be all sturm and drang but I disagree regarding the claim that CME has responded to the changing health environment. I spend a lot of time in Washington DC and rarely see CME colleagues participating in discussions around value-based reform at the FDA, Brookings Institute, PCORI, Kaiser Family Foundation, or National Press Club.

Following a presentation at Alliance I was asked to write an article for their open access publication, Almanac. I decided to pull the draft when I was challenged to provide citations from publications to support the findings and assertions regarding the importance and relevance of System's Thinking in medical education. How can innovation occur if we continue to simply rinse and repeat what we have been doing for decades? Why are so few willing to be first--seed a hypothesis and proceed from there?

A recent small survey of ~100 writers of CME needs assessments should stimulate a larger query of hypothesis test pathways to improve how we engage and evaluate learning in continuing medical education. What it revealed--at least to me--is a potential skill desert unable to provide relevance and insight in our healthcare big data ecosystem.

I have offered over the last few years to help improve the research methodology but my offer never received a response. I also reach out to colleagues with insights revealed from global travel and experience with comparative health systems--also crickets.

Respondents of the annual survey were primarily staff employees 56% (n=60) followed by freelancers 33% (n=35). I hope next year they will ask about roles within the company and link it to educational levels or areas of study. My concern is low data literacy among members of CME teams responsible for writing NAs or proposals seeking funds.

It would be a more rigorous study if unique identifiers of respondents allowed a stratification of data yielding a "tidy" data analyses--"Tidy data sets are arranged such that each variable is a column and each observation (or case) is a row"--wikipedia

It appears writer's of needs assessments defer to the client or employer to designate what to include in a "first-rate" assessment. Lacking a definition of "first-rate" is also problematic. We might be comparing apples to tennis balls here.

The top 3 highest ranked choices for what should be routinely included in needs assessments as measured in survey:

medical literature review 65% (n=70)--all publications are not created equal, how is data quality measured?
learning outcomes data 45% (n=48)--need more specifics here to be able to aggregate with confidence
clinical practice guidelines 42% (n=45)--funding source of guideline? Date of guideline?

This data should have been ranked--as it tells nothing about actual behavior when aggregated. If it was ranked, we could then assign probabilities.

Writers are also asked about tables, charts, or graphics--what to include in a "typical" needs assessment. Top 3 responses: (these also should have been ranked not "select all that apply" for valuable insights).

Alignment of practice gaps with learning objectives and desired outcomes 65% (n=68)
Results of clinician survey related to practice gaps 43% (n=45)
pre test/post test data from previous activities (with to without p values) 41% (n=43)

Coming in last (behind "I don't include tables, charts, or graphics", 17% (n=18) we find "Statistics showing effect size(s) from previous learning activities", 12% (n=12).

The problem here is that charts (my collective term for visualizations) are actually little arguments. They need to be read and interpreted and if necessary, challenged--not just slapped into a document.

I think I noticed a Likert scale in the bunch but without ranking we know very little. For example, the response of 5 for one question is not comparable to a response of 5 in a different question.

I do applaud the authors of the survey (both work in CME) for efforts to measure the practice environment but I believe we left a lot of data on the table. You can read more here for thoughts on improving our approaches to CME.

When you know better, you do better (in theory)

I have worked in CME for at least 15 years or so. I consider myself a recovering CME professional. I had to step out of the bubble to notice the limits of our specialized and industry specific practices. Work in health economics, health policy, and biostatistics highlights the limited scope of "analyses" integrated in design, development, implementation, and evaluation in CME.

If anything, the role of CME is even more critical as medical education frameworks evolve to address the shifting healthcare landscape. Providers are expected to negotiate health policy, health economics, and the rigor of clinical research in decisions made at the point of care. The patient is now a stakeholder and must be informed regarding risk/benefits of treatment options and impact of care on quality of life. Community resources are also engaged in many consultations to aid in improving patient outcomes challenged by social determinants of health. That's a lot of heavy lifting for a poorly constructed multiple choice question.

As fee-for-service is replaced by value-based healthcare the traditional educational structures must evolve along with the architecture of outcomes and assessment. Most importantly, we need to define “value” based on each stakeholder.

There are trade-offs especially in the evolving age of patient-centered care. Insights gleaned from population health are informative but critical thinking at the point of care requires the ability to synthesize multiple scenarios and outcomes both clinical and economic, risk benefits and cognitive biases and heuristics. Our challenge is to embrace the integrative thinking that allows us to consider
assumptions, current reality, and critically assessed practices all together.

Too often we focus on one element or we defend assumptions as truth in stark either–or terms. We often set up false dichotomies when the most meaningful solutions lay between polar positions. I am talking to you writer of multiple choice questions to evaluate provider competencies and practice behavior.

The increasing complexity of practice necessitates the development, refinement, and application of additional and better assessment methods. These opportunities must include an emphasis on competencies beyond medical knowledge and basic clinical skills, such as systems thinking, quality improvement, interprofessional teamwork, and patient safety, while concomitantly attending to identity formation, wellness, and resilience.

In the modern age of big data we have access to control groups generated from American Community Survey Census Data, real gaps in care reported within the National Practitioner Data Bank, and limitless sources of data pointing toward patient outcomes such as MEDdra and FDA databases. Why are we still recycling outdated insights or industry funded guidelines as primary sources of information to guide care and improve outcomes at the point of care?

One way to measure treatment benefits and patient preferences for treatment is through the use of a stated preference discrete choice experiment (DCE) survey.These surveys are designed to understand the strength of preference for the different constituents or attributes of a treatment intervention or care service.

The choice experiment methodology is a powerful means of capturing patient, caregiver or physician strength of preference for the attributes of a treatment and makes it possible to understand the relative importance of attributes with respect to the degree to which people are willing to trade between them.

If you have questions I can be reached here...or on twitter @datamongerbonny

Inspiration is cheap, rigor is expensive...

10/24/2018

You might not need statistics. Hold on. Before you start doing cartwheels down the hall--this only applies if there is no uncertainty in your data.

Perhaps you have data collected from a continuing medical education activity. You are only interested in how the attendees navigated through your course and what your data demonstrates about this group and this group only.

But what if you want to be able to extrapolate these findings to the broader population? Now we have introduced uncertainty. It is impossible to reach every single member of your target audience roaming around the planet. Now we are dealing with uncertainty and the mathematics needed to talk in probabilities--statistics--"inferring proportions in a whole from those in a representative sample"

How would primary care physicians, for example, that did not attend your educational activity answer the questions you asked throughout the engagement? Better yet, what attributes of your activity are members of the educational audience accessing at the point of care? Was it your activity that nudges a participant toward evidence-based behavior? Are you certain it wasn't another program or resource, colleague conversation, alternative type of media, or other fractional or incremental activity?

I primarily focus on ideas. First, looking at wide and shallow datasets and mining for hypotheses. These are only for inspiration. The rigor is downstream of the initial queries. Collaboratively a question is identified and the team finds the best tools for the job.

Survey design is often needed to gather data.

Does your team include someone with the skillset to carefully perform mathematical approximations for continuous data, categorical data, sliding scale data? How do you determine number of questions, correct for the number of questions, construct survey to minimize response bias, and satisficing?

Who evaluates the original purpose of survey? Is a survey the best tool? What is worth measuring?

In my case, the toolbox includes R language, Python, SAS, Qualtrics, and Tableau.

The biggest problem in collaboration usually rests around a Type III or IV error.

Type I error (incorrectly rejecting the null hypothesis)
Type II error (not rejecting the null when you should)
Type III error (you correctly reject the null hypothesis, but for the wrong reason)--Statistics How To

Type IV error is a specific type of Type III error (correctly reject the null hypothesis, but make a mistake interpreting the results)

A few examples:

Aggregation bias (the wrong assumption “what is true for the group is true for the individual”--think Simpson's paradox or ecological fallacies
Running the wrong test for your data

Data clients typically invest or inherit large datasets. Perhaps they purchased an extract of claims data, CMS data, or hospital system data (intentionally vague to not reveal client identities).

Here is the caution.

Many business teams are guilty of mining their data for inspiration. If it stopped there--we would be okay. The problem exists when the attempt is made to introduce statistical "rigor" into the same data set.

Cassie Kozyrkov, Chief Decision Scientist at Google Cloud does a great job clarifying why this is problematic. Think of sitting down in the morning with your cup of tea looking forward to a nice piece of sourdough toast. The toaster pops up and holy cow, your toast appears to have the image of Elvis carefully outlined on the lightly browned surface.

If you want to investigate the miracle--or better yet--the exploits of your toaster, would you put in a fresh slice of bread or put the Elvis toast back in the toaster?

Inspiration is cheap, rigor is expensive--Cassie Kozyrkov

(data)modeling--how much crap can you take?

10/15/2018

A lot of modeling is how much crap you can take--Lauren Hutton

My first gig working in pharma was a lesson in acronyms. I had a 12 month contract at GSK and I learned an exponential amount. Everyone used verbal shorthand and I jotted the scattered letters down to look up later.

I think it softened my skin. I have no problem asking the questions. The biggest surprise is how few of us challenge the answers.

Although I have experience as a bench scientist, academic writer at a large academic medical center, as well as a lot of agency work--the added layer of pharmaceutical experience was the real hat trick.

Starting my data journey, I often heard statisticians discussing data models as if they were a thing I could pull from a shelf. They sound vetted and official don't they?

A data model is quite simply an "accumulated set of discovered relationships" within a dataset. Make no mistake the models are tested, but if sufficient "target variables" or "class labels" are not identified and considered, you can imagine the limit of insights to be gleaned from analyses.

Approached without care, data mining can reproduce existing patterns of discrimination, inherit the prejudice of prior decision makers, or simply reflect the widespread biases that persist in society.

It can even have the perverse result of exacerbating existing inequalities by suggesting that historically disadvantaged groups actually deserve less favorable treatment.--Big Data's Disparate Impact Barocas & Selbst

Working in health policy and accessing databases filtered by race, economics, and a wide variety of demographic target variables--it is critical to consider all of the "possible values of the target variable into mutually exclusive categories".

A 360 perspective is imperative while data mining. The ability to target variables specified and parsed to avoid systematically disadvantaging protected classes leads to actionable insights. It is a necessarily subjective process but can be improved with data literacy and applied to non-binary classifications.

Here is an important graphic from The Senate: Affirmative Action for White People published in the NY Times--written by David Leonhardt. Civics lesson rewind: residents of Washington, D.C., and Puerto Rico do not have voting representation in the Senate.

Right now, about four million American citizens have almost no congressional voting power, not even the diluted power of Californians or Texans. Of these four million people — these citizens denied representative democracy — more than 90 percent are black or Hispanic.

They are, of course, the residents of Washington, D.C., and Puerto Rico. Almost half of Washington’s residents are black, and nearly all of Puerto Rico’s are Hispanic.

“Big Data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it.” – Dan Ariely

Walking down New Hampshire Avenue on my way back to my hotel, I stumbled upon the Democracy Tree. It is nestled between the roots of this majestic tree. Here is what it says:

This Tree is dedicated to the more than half million veterans, taxpayers, and citizens of the District of Columbia who, despite fighting in foreign wars, paying their full measure of taxes and continue to have no voting representation in the Congress of the United States of America.

"Taxation without Representation is Tyranny."
Foundry Democracy Project, Foundry United Methodist Church

The results are pretty outrageous. The Senate gives the average black American only 75 percent as much representation as the average white American. The average Asian-American has 72 percent as much representation as a white person. And the average Hispanic American? Only 55 percent as much. That’s right — the structure of the United States Senate treats a Hispanic citizen as only about half as important as a white citizen.

My point in all of this is surprisingly not political. The illustration here is to show that technology and advancements in our Big Data culture are forgetting the nuance of machine learning, AI, and algorithms--the insights are only as good as the data we feed into them.

"Data is frequently imperfect in ways that allow these algorithms to inherit the prejudices of prior decision makers."

The full quote below is an important example of how stewards of data (and aren't we all) have a unique responsibility to be thorough and informed as we explore and mine data. Garbage in, garbage out but worse yet--discrimination in, discrimination out...

Advocates of algorithmic techniques like data mining argue that these techniques eliminate human biases from the decision-making process. But an algorithm is only as good as the data it works with.
Data is frequently imperfect in ways that allow these algorithms to inherit the prejudices of prior decision makers. In other cases, data may simply reflect the widespread biases that persist in society at large. In still others, data mining can discover surprisingly useful regularities that are really just preexisting patterns of exclusion and inequality. Unthinking reliance on data mining can deny historically disadvantaged and vulnerable groups full participation in society.
Worse still, because the resulting discrimination is almost always an unintentional emergent property of the algorithm’s use rather than a conscious choice by its programmers, it can be unusually hard to identify the source of the problem or to explain it to a court.--Big Data's Disparate Impact Barocas & Selbst

Being the one that knocks...

10/12/2018

I know, I know, you didn't ask but I have some advice for you. Before you attend the conference or join the organization--think about who is being served by your attendance or membership. Are you "picking yourself?" as Seth Godin would say or are you part of someone else's narrative? I have joined organizations like the American Medical Writers Association but got very little in return. You become the product as they sell your data for a profit--I know--who doesn't anymore.

My pivot toward value and independence came when I found the right organizations. Most of them are focused in journalism so the art of storytelling and data literacy began to emerge.

I learned this lesson a single pain point at a time. When I didn't pick me, I ended up writing client biased content orchestrated by agency x, pharma company y, or medical education company z.

Earning press credentials provides access to medical and industry conferences--for free. When I was forced to tell sponsors what I was going to write about--I informed them I had no idea. Until I enter the space and have conversations, listen to the lectures or keynotes, or participate in the Q&A how can I possibly know what to write about? But that is how most journalism platforms work. You pitch or get pitched and then get to work. I reject this bias. I pick me. Or better yet, I pick you.

I reflected on this recently at a "meet and greet" for new members of the National Press Club in DC. After recent years of finally picking myself, I was invited to become a member. I think of the familiar creed among journalists,

Afflict the comfortable and comfort the afflicted

I now write about healthcare and all the different moving parts. Think of the piles of information and sort them by health policy, health economics, health law, clinical research, or whatever web that wipes across your mind.

We are taught to follow algorithms. Write the manuscript, book, article, or report as instructed. For a while, we comply, until we don't. We finally realize that you can't win by being more average than your competition. Better at following the blueprint. Think about your education or perhaps you have children navigating the educational system.

Watch them dutifully follow rubrics, guidelines, outlines, and doing what they are told. Our educational system is obsolete. We don't need these skills in modern society. We need creators, inventors, innovators--we need insights. If you have an idea of something you want to see in this world--listen to the short podcast by Seth Godin. Tag. You ARE it...

Well-meaning colleagues at the Press Club were excited to introduce me to members, experts in their fields, communicators, all to network--because I mentioned that I work for me. But it was weird. They were guiding me toward mentors. I reminded them that not all who wander are lost. I don't want a job. I am the job.

Who are you talking to right now? Who is it you think you see? Do you know how much I make a year? I mean, even if I told you, you wouldn't believe it. Do you know what would happen if I suddenly decided to stop going into work?

A business big enough that it could be listed on the NASDAQ goes belly up. Disappears! It ceases to exist without me. No, you clearly don't know who you're talking to, so let me clue you in. I am not in danger. I am the danger. A guy opens his door and gets shot and you think that of me? No. I am the one who knocks!--Walter White, Breaking Bad

Where I am headed next...and hope to see you!

Or reach out @datamongerbonny for scalable talks on data literacy, data sourcing, governance, or building your own data pipeline...

Knowledge is power but information is liberating

10/6/2018

The title of today's post is attributed to a statement made by Kofi Annan about education. On the surface, what could be more anodyne than knowledge right? But what if a large percentage of the collective, narrowly defines information based on a single perspective?

The remaining part of the quote, "Education is the premise of progress, in every society, in every family" might be the key element. I want to encourage you to pursue education daily. Many peers, mid-career or beyond, tuck into comfortable cozy careers either in a subject niche, algorithmic informed writing habit, or simply decide to focus on regulatory and compliance template driven writing. I am here to tell you--those days are waning. Big data is here and has invited CRISP-DM along for the ride. Briefly, the 6-phases of cross-industry standard process for data mining include understanding your data, preparation, modeling, evaluation, deployment, and business understanding.

You might be thinking--hey wait a minute--I am a writer not a data person. Okay. But here comes the hard no. If you aren't data informed or aware, what are you actually writing? Opinions?

The Government are very keen on amassing statistics - they collect them, add them, raise them to the nth power, take the cube root and prepare wonderful diagrams.--Josiah Stamp

Last week was a return to travel, workshops, and a smattering of data literacy work from the setting of a local conference on interdisciplinary statistics and combinatorics. I had grounded myself for a few months. Opting to step out of the frenzied and chaotic world of conference attendance and ask myself an important question--who is being served by me being in the audience?

Am I able to bring information immediately applicable at the point of care or am I listening to "experts" proclaim mind-numbing self-promotion and shameless marketing? Media credentials can get you front row access to most conferences but I would also argue--beware the chattering masses--they have heterogeneous motivations and goals. It is quite all right to return to square one--learn what you didn't know, you didn't know...

Thursday afternoon I traveled to North Carolina State University to attend a 2-hour workshop R for Document Creation (R Markdown). Because I am handling large datasets and would like the data to travel with the visualizations I am learning how to make this happen. I am learning R (with Python refreshers to follow) to expedite working with non-proprietary healthcare data. The datasets are large and often in need of cleaning and preparation for analysis. Check out your local universities. Free workshops are a great way to add a few new "shiny" tools to your skillset.

Here is an example of how Edward Tufte uses R markdown to prepare margin figures in his books and writings...The beauty of R--or at least for me--is the simplicity of the scripts. You can name them and save them for quick updates or application to additional data sets.

Using or learning R has been impactful and instrumental in allowing me to approach data visualization with a question informed by multiple datasets but not limited by interoperability or access. When you "knit" the code--voila the image will be published with the code either displayed or neatly hidden away--your choice.

This weekend, I wandered over to a local conference being held at an alma mater, UNC-Greensboro. Invited as media I scanned the conference brochure looking for sessions of interest (and was not disappointed).

I will be writing more in depth about the sessions but for now the familiarity of Simpson's paradox and refinement of my ability to interpret gene signature data were highlights.

You can amass data. That is the easy part. Big data lakes host a wide variety of potential information but you will need buckets--S3 or Hadoop clusters to organize. You will need pipelines to your data in order to begin the process. Upstream insights to confirm the right data is being generated (garbage in, garbage out), all 6 steps of CRISP-DM, and a governance strategy to protect your data and control access.

Follow along for information and guidance--or reach out for a hand. You never know what you might find swimming beneath the surface...

Clarity in the eye of the beholder...

9/20/2018

A recent article, STATISTICAL PARADISES AND PARADOXES IN BIG DATA (I): LAW OF LARGE POPULATIONS, BIG DATA PARADOX, AND THE 2016 US PRESIDENTIAL ELECTION reminds me of a great definition of statistics, "Principled thinking and methodology development for dealing with uncertainty." My intent is certainly not to be partisan but to highlight data that could just as easily been rendered in a behemoth healthcare data set. IPUMS describes the process well, "Think of census data needing to be cleaned, merged with new data, editing routines developed, and millions of strings coded into useful classifications. The data are far too large for manual inspection, requiring efficient data analysis and scalable approaches including machine learning."

The article is technical but can be parsed into tangible little morsels. Here are a few that interest me. Large-sample asymptotic is basically the approximation or limit of a graph--you have probably heard of the Central Limit Theorem and it has a role on the limits of Big Data. So I don't go down the rabbit hole of technical jargon this is a great analogy from the article (you should read it). Think of statistics and dealing with uncertainty--the author uses food metaphors--they help.

Fast food will always exist because of the demand—how many of us have repeatedly had those quick bites that our doctors have repeatedly told us stay away from?

But this is the very reason that we need more people to work on understanding and warning about the ingredients that make fast food (methods) harmful; to study how to reduce the harm without unduly affecting their appeal; and to supply healthier and tastier meals (more principled and efficient methods) that are affordable (applicable) by the general public (users).

I will bullet the most important findings in the article taken mostly verbatim:

difference between the sample average and the population average is the product of three terms: (1) a data quality measure, (2) a data quantity measure, and (3) a problem difficulty measure
the most critical—yet most challenging to assess—among the three is data quality.
When combining data sources for population inferences, those relatively tiny but higher quality ones should be given far more weights than suggested by their sizes.
data quality must be a relative one, and more precisely it should be termed as data quality for a particular study. This is because any meaningful quantification of data quality must depend on (1) the purposes of the analysis—a dataset can be of very high quality for one purpose but useless for another; (2) the method of analysis (e.g., the choice of sample average instead of sample median); and (3) the actual data the analyst includes in the analysis
Big Data Paradox-- The bigger the data, the surer we fool ourselves.

Fig 4. (please go to article for details) comparison of actual vote shares with 2016 Cooperative Congressional Election Study (CCES) estimates across 50 states and DC. "Color indicates a state’s partisan leanings in 2016 election: solidly Democratic (blue), solidly Republican (red), or swing state(green). The left plot uses sample averages of the raw data (n = 64,600) as estimates; the middle plot uses estimates weighted to likely voters according to turnout intent (estimated turnout nˆ = 48,106);and the right plot uses sample averages among the subsample of validated voters (subsample size, n = 35,829)."

Observe the only voting regions where the confidence intervals (barely) cover the actual results (Trump Data). And it should provide a clear warning of the Big Data Paradox: it is the larger turnouts that lead to more estimation errors because of systemic (hidden) bias, contrary to our common wisdom of worrying about increased random variations by smaller turnouts.

This article is one you may need to read and re-read several times. I find it important as we enter the age of personalized medicine. You can see here--even if the mathematics underlying the problem are complex--individualized predictions are approximations.

Because each of us is unique, any attempt to “personalize” must be approx- imative in nature. But this is a very different kind of notion of approximation, compared to the traditional large-sample asymptotics, where the setup is to use a sample of individuals to learn about the population they belong to.

In contrast, individualized prediction is about finding a sequence of proxy populations with in- creased resolutions to learn about an individual. This leads to an ultimate challenge for Statistics (and statisticians): how to build a meaningful theoretical foundation for inference and prediction without any direct data?--Xiao-Li Meng, Whipple V.N. Jones Professor of Statistics, Harvard Faculty of Arts & Sciences Editor in Chief of the Harvard Data Science Review

Why do so many of us view statistics and data literacy as something we just aren't built to comprehend?Sal Khan from Khan academy hits the nail on the head. If you are like me and annoyed by everything of value being tucked behind a paywall--his motto is, "You can learn anything. For Free. For Everyone. Forever."

Finally I now know why I am able to grasp advanced statistical principles (at least the big ideas) readily in my modern day work flow. He summarizes below in his brief and worthwhile talk below--only about 10 minutes but BOOM it is worth it. In a nutshell, when we are self-directed and learning or re-learning a skill or subject we tend to focus on what we don't know before moving on.

In a traditional academic model, we group students together, usually by age, and around middle school, by age and perceived ability, and we shepherd them all together at the same pace. And what typically happens, let's say we're in a middle school pre-algebra class, and the current unit is on exponents, the teacher will give a lecture on exponents, then we'll go home, do some homework.

The next morning, we'll review the homework, then another lecture, homework, lecture, homework. That will continue for about two or three weeks, and then we get a test. On that test, maybe I get a 75 percent, maybe you get a 90 percent, maybe you get a 95 percent. And even though the test identified gaps in our knowledge, I didn't know 25 percent of the material.Even the A student, what was the five percent they didn't know?

I never thought about the challenge of recalling academic courses and specializations in advanced Calculus, statistics, or programming languages. It was always about the test and let's face it, if you got an 85% on a test you were chuffed. Or at least I was. But think about the 15% of the material you didn't know? What if your car brakes were repaired only 85% of the way--would you consider it safe?

I thought I would share a few simple graphics I created to demonstrate clarity and what granularity can yield if we seek to understand what the small differences in a bar chart might mean if we are looking at the country as a whole.

The last graphic is of the electoral college votes--the only vote that matters in the US but you can see how additional clarity can once again be more insightful than reliance on a simple statistic about winning an election...thanks to Alberto Cairo for the image below.

Reach out for any help with data literacy or creating your own data stories--either in your professional or personal life--brainstorming is always free! twitter.com/datamongerbonny

data struts and frets his hour upon the stage...

9/2/2018

Out, out, brief candle! Life's but a walking shadow, a poor player That struts and frets his hour upon the stage And then is heard no more; it is a tale Told by an idiot, full of sound and fury, Signifying nothing.--Macbeth by Shakespeare

As similar drugs hit the market, the pundits race to see where the profits are likely to be highest. Case in point look no further then immuno-oncology drugs or other categories within personalized medicine.

"For now, sales are a useful metric fo starting and staying on a checkpoint (Inhibitor)"

I would argue that patient outcomes like "overall survival" will be the metric that delivers. We struggle to reconcile patient centered care with the aggregated data that leads to FDA approval. But meanwhile, premature media hype of early phase prognostic factors and clever pre-launch hype have become the harbors of safety investors have in the crosshairs.

Commercialization of disease has become a huge profit motive for shareholders. I have no problem if you are owning your decision to hedge the bets in favor of expanding market shares but I also see where patient as a buzzword has become like an awkward "comma" in the overall story. "What is that doing there and is it really necessary--It disturbs the flow of the narrative".

My shareholders expect me to make the most profit. That's the ugly, dirty truth.--Martin Shkreli

I have been painfully unsuccessful in talking about pharmaceuticals in a vacuum that does not include health policy, health law, health economics, and clinical research data. It bugs some people. I work as a consultant that views my role as being truthy and delivering perspectives often over-looked in the Google economy. The majority of calls I am on include a C-suite type masquerading as an expert without ever exhibiting doubt in the science or wanting engagement on other perspectives.

I recognize discussion topics from the first pass on what loads on page 1 of an online search. What I don't hear is the granular discussions from well-designed trials that add a depth to pre-clinical decisions or moiety designs.

Attempts to vilify the pharmaceutical industry without transparency of health law, policy, and economics underlying the framework continue to be short sighted. In our big data lives, there are now infinite data libraries and resources for query and analyses. Was it Martin Luther King that warned us all, "Nothing in the world is more dangerous than a sincere ignorance and conscientious stupidity"?

Upstream from diseases we can discover how laws influence health. No matter what side of the aisle you are on--changes to federal law, reversal of executive orders, changes to federal agency policy (ask yourself how Alex Azar, Health and Human Services Secretary, ended up with jurisdiction over immigration) and appointment of judges impact public health.

The Southeastern Association of Law Schools 2018 conference featured a session on health law:

WORKSHOP ON HEALTH LAW
Healthcare in the Era of the Trump Administration
The 2016 election has already ushered forth dramatic changes to our healthcare system, in efforts to repeal and replace the Affordable Care Act, in deregulation, in executive orders that alter the effects of the ACA, and in numerous other ways. This panel will focus on the state of healthcare in America in the era of the Trump presidency. What are the enduring effects of the Trump presidency on the state of healthcare delivery and healthcare finance (both in private and public insurance programs) now and into the future?
Moderator: Professor Deborah Farringer, Belmont University College of Law
Speakers: Professor Jennifer Bard, University of Cincinnati College of Law; Professor Zack Buck, The University of Tennessee College of Law; Professor Nicole Huberfeld, Boston University School of Law; Professor Nicolas P. Terry, Indiana University, Robert H. McKinney School of Law

I recommend listening to This Week in Health Law podcast if you want to access a wide variety of legal scholar discussions outside of the typical journalistic soundbites and tantalizing ledes. These are a few of the graphics from the event. The broad discussions include the role of trade tariffs on the cost of medical equipment, the fight against global guidelines on livestock antibiotics, suggested bans on CDC employees using words like "evidence-based, science-based, vulnerable, entitlement, diversity, transgender, and fetus", the phrase "climate change", sexual discrimination and Title VII protections, use "religious liberty" to deny medical care, and on and on...

Harvard scientists say current and proposed environmental policies could lead to an extra 80,000 deaths per decade. Think of losing the entire audience from the MetLife Dome --where the New York Giants and Jets play--every decade because of changes in policies that impact our clean air and water. Requiring raw data from landmark cases to be released to the public or risk overturning of established science.

Letter from Harvard leaders to EPA's Andrew Wheeler on proposed science policy by Emily Atkin on Scribd

Context is king. We will not be able to out "Innovate" our way out of global and national priorities that minimize the role of social determinants, health policy, social policy, economics, and the environment.

time to get our hands dirty...

I for one do not accept Shakespeare's existential nihilism from Macbeth. We just need to look at more data. Dirty hands get things done.

Find more data--not just the data that supports our objectives, bottom line, or shareholder interests.

Minding my own business

8/27/2018

A fellow trail runner suggested the title for a new post. When you are out on the trails you tend to share things and admittedly you likely think things are funny that may or may not be. Regardless, I cleaned up the actual suggested title and decided to go with the sentiment. How are you running your business? It can be your own small digital media company or the business of you within the confines of an employee/employer establishment.

There is never a lack of listicles published to help you decide to stay an employee or branch out on your own into the wild and woolly world of entrepreneurship. I was sort of pushed. Small companies are at risk of insolvency from external pressures and internal--sometimes both.

You can be brave out on your own but sometimes its easier with an established cast of characters. We know the gossip, the BS artist, the person that will always deliver during crunch time, the data person--what do you do if they are all you? Finger pointing and hushed laughter lose the luster when you have no one but yourself to blame. I was recently asked to write something described as "traditional". What does that even mean? I think it was supposed to be a pitch--Our Product is Great--Everyone Else Stinks--Our Product is the Best. I have many tools I rely on and couldn't imagine working without but lets face it, there is always room for improvement.

What drives your curiosity when you read about R&D driving up healthcare costs? How is R&D being defined? How much research are we talking about or is it primarily mergers & acquisitions and patent protection? How much of our tax dollars have funded the "Innovation" being heralded as the cost of doing business in our new global economy?

I detest organizations in the news business that park everything of interest behind a paywall. Even more when the discussions are around important considerations in healthcare. News we all need to know to make informed critical decisions. I know, I know, its all about making a buck but what about when it isn't?

It reminds me of the news headline on the evening news (remember the evening news?)

You Won't Believe What Household Item Will Kill You in About 10 Minutes.--News at 11.

I remember when the news was for all of us. Not just those of us paying for premium access. Don't worry. It makes me want to work harder. To ask more questions and question more answers.

I go where my curiosity leads me. Beth Macy wrote a fabulous book about the opioid crisis, Dopesick. Dealers, Doctors, and The Drug Company That Addicted America.

Not because it was in the news media but because she noticed something not being discussed when she was writing about the furniture industry--what was happening to communities in the wake of offshoring.

I always feel a little preachy when finishing a workshop presentation. The audience always seems surprised regarding the cost of limited data literacy. Once upon a time it became okay to roll our eyes around mathy topics. That time is gone.

When we can't go right to the source and decipher the statistical modeling or data sources we are at the whim and fancy of people that own the knowledge. I don't remember who said it but it went something like this: Discovery isn't followed by someone yelling EUREKA! It is usually the quieter, hmmmm...Isn't that interesting?

Those with the knowledge control how much of the truth valve to open.

What if it is only a small dribble?

"Liberty, prosperity" and streptomycin?

8/11/2018

I am a New Jersey gal. Even when I think I am a California gal or North Carolina gal--there are some things embedded in identity that you can't shake. For example, Bruce Springsteen sang quite convincingly from the stage that I should "get out while you're young"--and I did.

On a long recent drive to NYC my husband (Steve) mentioned New Jersey is seeking to adopt a state microbe. I had no idea. And then he shares NJ is only the second state to put it up for a vote. The first state? Oregon and the winning microbe--was brewers yeast. Saccharomyces cerevisiae proudly represents the many small craft breweries across the state.

Our likely winner here in the Garden State will be Streptomyces griseus, the producer of streptomycin.

Streptomycin was discovered by Albert Schatz, Elizabeth Bugie and Selman Waksman in January of 1944. It was truly remarkable in the breadth of pathogens that it attacked, including Vibrio cholerae (cholera) and Mycobacterium tuberculosis (TB), against both of which penicillin, the only other antibiotic at that time, had no effect. The discovery of streptomycin radically changed public health.

This producer of streptomycin (Streptomyces griseus), which was discovered in New Jersey soil and also isolated from the gullet of a healthy New Jersey chicken, was developed at the Department of Biochemistry and Microbiology, Rutgers, the State University of New Jersey, where its remarkable antibiotic properties were recognized. It was used in the development of large scale fermentation, a novel concept at that time. There had been war-time pressures to increase the production of penicillin via deep tank submerged fermentations. This was successful, evidenced by the general availability of penicillin on D-Day, June 6, 1944. The pressure for enhanced penicillin production continued. George Merck, in assessing needs for penicillin and those for the newly found streptomycin, overrode Merck's scientific board, which was focused only on penicillin, and committed extra funding for a major new project: the production of streptomycin.

Thus streptomycin joined penicillin in the development of the pharmaceutical industry in New Jersey. Within four years of the discovery of streptomycin in 1944, the initial yields in the milligram range were ramped up to 3,000 kilograms per month, a million-fold increase. This enabled animal and human trials by Drs. William Feldman and Corwin Hinshaw, at the Mayo Clinic in Rochester, MN.--Rutgers School of Environmental and Biologic Studies

I love a quote by NPR host, Ailsa Chang "That microbe has saved lots of lives, including Senator Bob Dole and Beatles drummer Ringo Starr. Now, it has one more challenge - becoming official state microbe."

I have been thinking about interesting data and facts that send us off in pursuit of more context and knowledge. Motivated by the abundance of non-proprietary data free for us to access in our personal and professional lives--I began teaching and writing about data literacy. Watch this space for a new creative project to help all of us use data more effectively and give voice to stories yet to be heard...

<<Previous

Forward>>

Browse the archive...

Follow @datamongerbonny

Thank you for making a donution!

In a world of "evidence-based" medicine I am a bigger fan of practice-based evidence.

Question the quality of the evidence. The motivation for disseminating the evidence.

Who stands to benefit the most from its uptake?

Remember the quote by Upton Sinclair...

“It is difficult to get a man to understand something, when his salary depends upon his not understanding it!”

Tweet to @datamongerbonny

When you know better, you do better (in theory)

Where I am headed next...and hope to see you!

time to get our hands dirty...

Data and Donuts has a free weekly newsletter!

Categories