data talks, people mumble

The data issue...

2/21/2020

There is a weird facet of my personality that applauds irony in all its iterations. I was asked to speak at a local community event for "innovators and entrepreneurs" highlighting the United Nations Sustainable Goals--specifically equality. My suggestion to introduce the utility of census data and how to access, clean, and analyze for free was welcomed.

Unfortunately, in the absence of effective marketing -- the draw of census data is not exactly standing room only. Because as it turned out--they must have been all standing somewhere else. The attendees mill about drinking free beer and nibbling on heavy hors d'oeuvres and once the second round of talks begin--they are typically engaged in other conversations. Not to worry, I persist.

My motivation in agreeing to give talks for free from time to time is not unlike the comedian ducking into little venues to test out new material.

I prepare for the bigger events by learning from the smaller gatherings. It is easy to gain insights from the questions asked, requests for additional details, or invitations that follow where the good "bits" are and where there may be a little meat missing from the bone.

For example, I teach audiences how to read a job's report. Over a dozen previous attendees sent me a link to the New Yorker article--The Bullshit-Job Boom. They get me.

One of my favorite observations in the article I can relate to from my previous life--"I do digital consultancy for global pharmaceutical companies’ marketing departments. I often work with global PR agencies on this, and write reports with titles like How to Improve Engagement Among Key Digital Health Care Stakeholders. It is pure, unadulterated bullshit, and serves no purpose beyond ticking boxes for marketing departments. . . .

I was recently able to charge around twelve thousand pounds to write a two-page report for a pharmaceutical client to present during a global strategy meeting. The report wasn’t used in the end because they didn’t manage to get to that agenda point."

A book I carry with me everywhere--a textbook and guide through all of the discussions about economics, Measuring What Counts: The Global Movement for Well-Being begins with the following few sentences.

"The world is facing three existential crises: a climate crisis, and inequality crisis, and a crisis in democracy. Will we be able to prosper within our planetary boundaries? Can a modern economy deliver shared prosperity? And can democracies thrive if our economies fail to deliver shared prosperity?"

Back to the census data. To explore broad questions beyond GDP -- the overworked metric of gross domestic product-- it is vital and it is important to dig deeper. And better yet the insights are free once you can tackle the steep learning curve. What an opportunity to meet your potential clients, patients, or customers in the communities where they live. Identify the barriers to improved outcomes by identifying structural determinants and working for policies to ameliorate wide disparities in not only income but opportunity.

If we believe that we, as Americans, are bound together by a common concern for each other, then an urgent national priority is upon us. We must begin to end the disgrace of this other America.

And this is one of the great tasks of leadership for us, as individuals and citizens this year. But even if we act to erase material poverty, there is another greater task, it is to confront the poverty of satisfaction - purpose and dignity - that afflicts us all.

...Gross National Product - if we judge the United States of America by that - that Gross National Product counts air pollution and cigarette advertising, and ambulances to clear our highways of carnage. It counts special locks for our doors and the jails for the people who break them. It counts the destruction of the redwood and the loss of our natural wonder in chaotic sprawl. It counts napalm and counts nuclear warheads and armored cars for the police to fight the riots in our cities. It counts Whitman's rifle* and Speck's knife*, and the television programs which glorify violence in order to sell toys to our children.

Yet the gross national product does not allow for the health of our children, the quality of their education or the joy of their play. It does not include the beauty of our poetry or the strength of our marriages, the intelligence of our public debate or the integrity of our public officials. It measures neither our wit nor our courage, neither our wisdom nor our learning, neither our compassion nor our devotion to our country, it measures everything in short, except that which makes life worthwhile. And it can tell us everything about America except why we are proud that we are Americans.-- Robert F. Kennedy March 18, 1968 *mass murderers

We are all homeless...

1/20/2020

Whether measured by inequality, the nature of the jobs market, or even lifespan, America’s poor have little to fall back on. To Lind, this is no accident. Governments have been cutting back on social safety nets at a time when employers have restricted benefits to the most highly credentialed jobs.

As Warren Buffett, the billionaire investor, said in 2006: “There’s class warfare, all right, but it’s my class, the rich class, that’s making war and we’re winning.”--Populism and the smouldering rage of American poverty, Financial Times

If you said I was stubborn you wouldn't be a liar. I refused to acknowledge the signs of an oncoming cold. Figuring I could run it out I did an easy 10 mile run hoping my oxygenated lungs would expunge the irritants and I would be back to being shiny and new. Let's just say all didn't go as planned and I spent the afternoon sipping tea spiked with a bit of whiskey and watching a few documentaries.

Signs of Humanity was a brilliant surprise. Willie Baronet is an artist and professor in Texas. Well, after watching his documentary I can say with confidence--he is also a filmmaker. His story illustrates the humanity and compassion evident in his interviews of over 100 homeless people. Offering to purchase their signs, he collects them and creates art installations to bring awareness and conversation to the front line of our debates on community and policy change.

I have always know that poverty isn't simply one thing. It is a cascade of small and large tragedies that can leave us hopeless, bereft, and completely alone.

Watch Signs of Humanity. If you have a Prime account it is free.

As an analyst, I can only measure what I bring to the discussion. If you write about poverty, social determinants of health, or other variables with an easy numeric tally or comparator you are leaving data on the table. The tensions we hold can help inform and elevate discussions.

There is no "other" in discussions of poverty. In an economy where we must keep our fingers crossed that we don't lose our jobs (and the benefits they provide), become ill, or need to reduce work load to care for ailing parents--there is no floor. You can drop right down to the bottom at the blink of an eye.

Over the years we have been lucky. My husband and I had our parents during the fragile years of building our own little family. There were so many random challenges that didn't seem to care if we were highly educated and well compensated. His boss shot himself in the heart and we were left without a steady income that had seemed teflon over the prior 17 year period. I once worked in Pharma and as companies were sold, merged, and scrapped--I began to appreciate the fragility of long term security.

I work in healthcare for the human side of medicine--not the profit motives winding through our fragile US health system. I want us to ask better questions and to do a better job at questioning answers. We need to pay attention, become data literate, and share our stories.

Follow along with me as we explore census data, government data, and other resources to help add a human dimension to a much needed narrative...

The potential magnitude of chance events

1/8/2020

Think about the human genome, with its 3 billion base pairs: even in a very large randomised trial, say of 20 000 people, thousands of genetic differences are sure to arise between the groups. Some of these differences—which we do not yet know— might be important for prognosis. Randomisation guarantees that such differences are indeed due to chance. It means that statistical theory based on random sampling can be used to calculate confidence intervals that express the potential magnitude of such chance events...Vandenbroucke 2004 When are observational studies as credible as randomized trials?

I am not exactly sure when it is appropriate to affix the badge of data scientist or analyst to your lapel and boldly saunter out into the world. The battles are softly waged with words as statisticians claim certain inalienable rights as loose definitions of data literacy are assigned to the spoils. I am only responsible for my little corner of the world.

Through several post-graduate degrees I learned statistics and most recently completed an executive education program from the Fu School of Engineering at Columbia in Applied Analytics. The curriculum was quite a slog as mastery in Python came at the expense of real-time skills applicable to daily questions. I completed the course with a score of 97% but felt a little overwhelmed. I had questions. Questions that needed answers and answers that screamed for clarity. Slowly I began tackling Python library by library. I started with Pandas and began to gain proficiency outside of the need to pass an exam or submit a capstone.

So here we are. A data curious human navigates the world of health economics, health policy, and clinical medicine. Think about this. If we aren't intended to be able to clearly understand the research or media discussions in this space--who is the audience? My goal isn't to provide unyielding answers to these perturbations but to point and say, "What is that?" and "Why does this matter?"

When guiding or facilitating discussions around data literacy of fluency, I pull clinical research into the discussion and collaboratively step through the numbers to help inform colleagues about best practices or even self governance in relying on data to curate insights. For example, we may understand that randomized controlled trials are the "gold" standard for answering questions about tolerability, efficacy, and safety but is this always the case?

I am not debating the role of a well-designed study powered to answer a question but we need to understand the role of observational studies and the potential limits of RCTs. The majority of RCTs are limited by size and follow-up periods. If we are attempting to say anything meaningful about duration of response, safety or adverse events we need to look at case-control of large-scale observational studies.

To evaluate causal mechanisms of disease for example, we need to be able to navigate observational studies. How can we identify or adjust for confounding? Where do we begin while acknowledging that many confounders are insufficiently known or are unquantifiable?

What if we can create a visual representation of causal assumptions to identify potential pathways of confounding?

Directed acyclic graphs or DAGs have the variables connected by arrows that are all directed or pointing in the same direction. These represent direct causal effects. If we remove the arrow, the effect is no longer observed. And because causes (such as exposures) precede the effect (disease) we use a sort of chronology when creating a DAG.

Our a priori knowledge informs how we design the question.

directed acyclic graphs (Nephrology Dialysis Transplantation, Volume 30, Issue 9, September 2015, Pages 1418–1423)

A graphical presentation of confounding in DAGs. (a) The structure of confounding in DAGs. Since age is a common cause of CKD and mortality, confounding is present when we want to assess the causal relationship between the exposure CKD and the outcome mortality (b). The backdoor path from CKD via age to mortality can be blocked by conditioning on age, as depicted by a box around age in (c). Similarly, ethnicity is a common cause of obesity and decline in kidney function (d). The backdoor path from obesity via ethnicity to decline in kidney function can be blocked by conditioning on ethnicity. If ethnicity is not measured or not properly measured, residual confounding remains present.

As described by Suttorp and colleagues, the three criteria of confounding are:

1. Confounder must have an association with the outcome
2. Confounder must be associated with the exposure
3. Confounder must not be in the causal path from exposure to outcome (not a consequence of the exposure)

Confounding distorts the actual effects so now we need to remove as much of the impact as possible. The methods vary but it is critical to know how the confounding was identified (often if and methods used). To address confounding by age when evaluating the relationship between chronic kidney disease (CKD) and mortality, we might, for example, stratify by age.

Conditioning is the term used for adjusting for confounding and includes restriction, stratification, or multivariable analysis. The box around age in the figure above demonstrates that this confounder has been blocked.

What about the causal relationship between obesity and decline in kidney function as described in (d) above? If we make assumptions regarding race (research article conflates race and ethnicity--it is a social construct being misused as a biologic proxy but for purposes of the discussion I am not poking that tiger)-Differences in progression to ESRD between black and white patients receiving predialysis care in a universal health care system.-- and integrate prior research describing a faster decline in kidney function and progression to end-stage renal disease (ESRD) in blacks and higher obesity rates in African Americans (again, are we talking ethnicity or race?), we might define ethnicity as a confounder.

...It is, however, possible to identify confounding in a DAG that is impossible to adjust for. For instance, it could be that physicians did not record ethnicity, and ethnicity is thus unavailable in the data analyses. The investigator cannot adjust for a factor that is not measured. Similarly, it is possible that adjustments are only partly successful in controlling for confounding. For example, even if ethnicity was recorded and adjusted for in the analyses, some residual confounding can remain present.

The reason for this is that self-reported or physician-reported race does not always completely represent the racial background of an individual. When confounding is unknown, unmeasured or even partially measured and adjusted for, residual confounding will remain present. This is also the problem with confounding by indication. A physician's treatment decision is based on many factors, including the physician's preference and estimation of the patient's outcome, and it is almost impossible to completely measure all these factors. Randomized controlled trials are therefore the best way to avoid confounding by indication.--Graphical presentation of confounding in directed acyclic graphs

More to come (stay tuned...)

Resources

When are observational studies as credible as randomized trials?

Graphical presentation of confounding in directed acyclic graphs

What does race have to do with it?

12/11/2019

I don't have a bunch of answers. What I have are questions. In fact, when I talk about data fluency I prefer to "group think" solutions. We try and unpack the information available to us and trace a path to critical thinking.

Reading through the article below, once again probably launched more questions than it attempted to answer.

Low-Dose Aspirin for Primary Prevention of Cardiovascular Disease: Use Patterns and Impact Across Race and Ethnicity in the Southern Community Cohort Study--click for article

American Heart Association news summarized the findings, A racial gap in the heart attack benefits of aspirin?

"We think the reason aspirin use did not have a beneficial effect for African Americans could involve a different genetic response to aspirin therapy and poor control of other risk factors," said Dr. Rodrigo Fernandez-Jimenez, lead author of the study. He is a cardiologist and researcher at Centro Nacional de Investigaciones Cardiovasculares in Madrid.

But when I look at the classification of "black" race contrasted with birth country of origin you can see why lumping a diverse population under a poor proxy for biology or a substitute for a political construct makes no sense in medicine (or anywhere else for that matter).

Data visualization below is from my workshop on racial bias in data...think about how heterogenous genetic profiles would be in an aggregated classification of "race".

Reading through the graphics in the article I was unable to look at the variables classified by race--they were aggregated at too high of a level to invite inquiry or additional analyses.

For example, why isn't the data stratified by race AND risk? Is it me or am I unable to determine how many blacks are low, intermediate, or high risk stratified by education or health insurance status? Wouldn't access to healthcare be a potential driver of being "prescribed" low dose aspirin?

I recently examined how access to healthcare influences algorithms attempting to manage the health of populations within a large health system. The algorithm was adopted to help identify patients with complex needs that would benefit from an intervention to help mitigate co-morbid disease and higher risk profiles.

It is worth digging into the article to see how the referral to the intervention becomes more equitable once actual levels of disease are implemented instead of healthcare costs. When the algorithm is applied as developed, black patients have to be far more ill before being referred to the program as compared to white patients.

Back to the Low Dose aspirin study. To add more confusion or obfuscation--the measure regarding health insurance status is not included in the final graphic...

Keep an eye on the YouTube space if you would like to see the full presentation from Tableau Fringe Festival. They should have videos posted by the end of the month.

Measuring what we should, not what we can...

10/26/2019

Do you write survey questions? Use them as a measure of knowledge, competency, or behavioral change? Perhaps you are thankful that you don't have to write them, your job is simply to do the analytics.

Your enterprise analytics are set it and forget it. But what if I asked you about your process? I just finished grading assignments in a university class I am facilitating about Understanding Data.

They are lost when asked to describe discrete vs. continuous data types and distinguish between qualitative ordinal, binomial, and nominal data.

It made me want to do a re-wind on data gathering that I think you will find helpful. If you need hands-on help, sign up for the first tutorial in a series I am launching Thursday, Big Data on A Less Big Budget.

But let's not get ahead of ourselves. We start with data identification, sourcing, and gathering for a specific reason. The types of data we collect will determine the types of analyses we can do--and thus, how meaningful our insights might be.

If it was up to me, I would like to see more discrete choice modeling analytics using a Python library like ChoiceModels.

"discrete choice models, or qualitative choice models, describe, explain, and predict choices between two or more discrete alternatives"

But many teams want to keep relying on Likert scales or Likert-type questions, let me explain how this impacts your analytics plan.
Why don't we make sure we are at least doing the best we can.

Daniel Mingook Kim

In my experience, once you get past the murky distinction of nominal, ordinal, and numeric, there is little thought to how you need to adjust your analytics. The time to make these important decisions is immediately after you create a data question or draft a set of objectives for a need assessment.

To collect measures of attitude, character, and personality that better assess post-activity behavior of a healthcare provider after an educational intervention, you likely rely on Likert attitudinal scales and apply them to professional learning and/or behavior change.

The problem is you need to distinguish between Likert-type items and Likert scales. Likert-type questions are when you design single response questions that are not specifically related. You can list them separately and your intention is not to combine the responses or aggregate a score.

When you analyze Likert-type responses, you can sense an ordinal measurement--there is a directionality or a "greater than" or "lesser than" relationship. We think of this as ordinal because you can't say much about the distance between the measures or distinguish the intensity of the difference. Because of this you would use descriptive statistics like mode or median or frequencies to assess variability. This might also include chi-square.

Likert scales are a collection of similar questions where you create a composite score based on a quantitative measure of say how a healthcare provider selects a first-line therapeutic option for a patient with "mild to moderate" rheumatoid arthritis. Because the individual scores are generating a composite or overall measure, now we would use the mean and standard deviations as a measure of variability. These interval scale items would include Pearson's r, t-test, ANOVA, and regression.

This is why decisions around your data analytics start at the question development stage. What type of information are you hoping to gather? What data types do you need to answer your data question?
â€‹You will know this right away if you don't out source data gathering to PubMed published articles or pre-existing survey data.

Beware of low hanging fruit...

Summary of DCA from Columbia University Mailman School of Public Health...

Discrete choice models are appropriate for evaluating behaviors that operate within a framework of rational choice. Health care professional, when confronted with a discrete set of options will opt for maximal benefit or utility. The options most likely to improve patient outcomes for example and/or abide within a framework of evidence-based medical choice.

These decisions are made from both the choices presented at the point of care AND the characteristics of the professional making the final selection. It follows from this assumption that utility of a choice is a function of the characteristics of the possible choices and the characteristics of the person making the choice.

Here is a resource I found extremely helpful--reach out if you are interested in exploring DCA collaboratively.

E. Lancsar, J. Louviere. Conducting discrete choice experiments to inform healthcare decision making. A userâ€™s guide.PharmacoEconomics, 26 (2008), pp. 661â€“677

"Every reform movement has a lunatic fringe"

10/25/2019

It's conference time. Not just Tableau Conference 2019 but viewing my calendar of speaking obligations--quite a few others as well.

But just because we aren't supposed to pick favorites among our children doesn't mean I can't give extra smooches and love to Tableau. I have been to 5 or 6 over the years and they are one of the best production, content, entertainment, and simply experiential conferences I have ever had the pleasure of attending (I have to extend a slow clap to Qualtrics as well).

All of these years later, I had choices to make. When you attend about a half dozen as a speaker, there isn't much time to be in the audience. And let's face it, you should up your game and set your focus on being the one at the podium.

This year due to a new format and fewer speaker opportunities I was not invited to the main stage but was extended an opportunity to speak at the Tableau Fringe Festival. A spot for those of us with ideas worthy of sharing with the community-at-large but maybe not scalable to the main stage.

The videos will be uploaded to the Fringe Festival website in a few weeks so follow the link above. Until then, here are a few of the dashboards I included in my talk, a specific version of Data Talks, Meople Pumble titled Decision Making in The Face of Uncertainty. Which variables and data sources are needed for a dynamic description of poverty?

What's next?

I am teaching an online workshop on Halloween, Big data on a less big budget where we will step through a real world example and demonstrate how to use open source tools to create and answer a data question.

I will see you at Women in Tech Summit in Raleigh Durham November 8th after a busy week of National Press Club workshops and Academic Stand-up in NYC with a featured conversation with Angela Saini author of Superior: The Return of Race Science with a hands-on Python refresher nearby.

"Every reform movement has a lunatic fringe"--Theodore Roosevelt

Cautionary tale to "see and define" not "define and see"

10/4/2019

“For the most part we do not first see, and then define, we define first and then see. In the great blooming, buzzing confusion of the outer world we pick out what our culture has already defined for us, and we tend to perceive that which we have picked out in the form stereotyped for us by our culture.”--Walter Lippmann (American Journalist--Sept. 23, 1889—Dec. 14, 1974)

The basis for teaching data fluency stems from our very own heuristics. Mine too. None of us are above debate regarding our intentions or assumptions.

Allow me to share a specifically non-instagramy moment. Due to a confluence of events--apparently rodent friendly weather, living on a park with a creek, and a lively chicken coop--we became infested. Mainly the yard and coop but who knows? If truth is indeed "poisoned at its source" according to Walter Lippmann, we attacked with gusto.

After all, we defined the problem therefore the solution appeared to be quite straightforward.

Well let's think this through carefully. If the goal was a systemic ridding of the rodent population from the yard--we were advised by professionals to follow the protocol.

But often, in pursuit of an outcome we forget about the stench. In this case I am referring to fetid carcasses decaying just about everywhere you can imagine. Out of sight but definitely not out of olfactory range.

The unpleasant imagery may cloud your ability to see the data adjacent similarities but stay with me here. A good data question often requires the holding of tensions. The pursuit of a complete data set to indeed be empowered to curate empathy or insights. Culturally defined norms often make it unavoidably tempting to believe low hanging fruit and reinforce pre-existing assumptions.

We might ignore relevant information out of fear, repulsion, or conscious bias.

But eventually the stench of bad data will win. And we will need to clean up the mess and start over.

Where to connect--click on images for links to more info!

Value is in the eye of the beholder...or maybe hidden in the LI algorithm

10/1/2019

The emerging age of "value" based medicine has outpaced important discussions on metrics and tools to examine value outside of profit motives and limited variables. The buzz-worthiness of terms like big data and value might leave marketing professionals frothing at the bit--the rest of us are or should be moving forward with business as usual.

There is no such thing as big data. Large unstructured datasets didn't simply arrive at the beckon of marketers. And don't get me started on value. Value for whom? If we are indeed in the age of "personalized" anything--why don't we start with refining these general terms to engender meaning and the appropriate granularity.

I originally wrote the bulk of this article on LinkedIn. I have no idea what drives their algorithm but I will share that where other platforms (including this one) yield hundreds of eyeballs in a single day--their metrics show that from thousands of followers on LI only 7 bothered to read it. Maybe it is a dog I don't know. What I do know is that it is packed with resources you might find interesting and useful.

If I am wrong...woof, woof.

I probably receive anywhere from 20 to 40 messages every week or so from individuals with questions about data literacy, specific data packages, or data visualization with Tableau. There are a lot of questions that I hear that I never thought to ask. I never asked for permission. Many people ask should I learn Python or R?

What is the best book to read to learn Tableau? These are personal and specific to individual interests and deserving of exploration. Mainly because like anything, your mileage may vary.
And then, there is the tired question about what is the difference between a data analyst, data scientist, data engineer, statistician, blah, blah, blah...I will leave you to explore this well worn debate elsewhere. There also is a bit of a territorial debate where member of these distinct groups poke fun at the other less worthy participants.

Here is what I learned. The brightest and best want to bring you along. They encourage, share resources, and are welcoming.

A recent podcast featuring Xiao-Li Meng, the Whipple V. N. Jones Professor of Statistics, and the Founding Editor-in-Chief of Harvard Data Science Review stopped me in my tracks. A year ago I was reading through his work and wrote a few blogs about his ideas. Curious about a pending part two to the discussion, Statistical Paradises and Paradoxes in Big Data(I): Law of Large Populations, Big Data Paradox, and the 2016 US Presidential Election, I sent him an email.

He promptly responded. Apologized about the delay in the Part II, sent me a copy of the article above and several others he thought might be of interest (along with access instructions). Meet him in the podcast below and learn about his role as Editor-In-Chief of The Harvard Data Science Review.

The Mission and Scope of the HDSR is captured by this quote below:

As an open access platform of the Harvard Data Science Initiative, Harvard Data Science Review (HDSR) features foundational thinking, research milestones, educational innovations, and major applications.

It aims to publish contents that help to define and shape data science as a scientifically rigorous and globally impactful multidisciplinary field based on the principled and purposed production, processing, parsing and analysis of data.

Read the tag-line and I dare you not to be excited if you are even slightly data adjacent.

A microscopic, telescopic, and kaleidoscopic view of data science

Here is one of the articles referenced in the journal section brilliantly labeled as CORNUCOPIA: impact, innovation, and knowledge transfer.

(A)Data in the Life: Authorship Attribution in Lennon-McCartney Songs
Apologies for the interactive experience that will likely derail your tasks at hand as you explore the features below. If the link brings you to 3 vizzes, pick number 3 to activate the visualization below.

I have been taught that items should be bundled in threes so here is one more. I recently read a great article in Medium, Anaconda is bloated--Set up a lean, robust data science environment with Miniconda and Conda-forge by Ted Petrou. Here is how this was a lifesaver. I work with large datasets in healthcare and medicine. The last thing I needed was a clunky giant installation of valuable but likely not needed programs on my MacBook. Thanks to Ted and his willingness to walk me through a few hiccups I am up and running. If you are a Python person and value just-in-time instruction I suggest you check out his workshops (a few free ones are there for a test drive) over on DUNDER DATA.

Although I completed an executive online course from Columbia School of Engineering in Applied Analytics--once I returned to the real world I wanted expertise at a granular level in the packages and libraries that I actually use daily. Ted is so clear and patient and provides the detailed explanations of why things work a certain way and how to customize your experience specific for your data needs.

Want to connect? Follow along on twitter or catch me live at Women in Tech Summit next month or RStudio conference in San Francisco in January!

Later this month I will be presenting at the Tableau Fringe Festival on October 25th! Announcements about the line-up will be out soon...

Survey says: learn simple code to liberate insights

8/19/2019

The National Health Interview Survey (NHIS) has monitored the health of the nation since 1957. NHIS data on a broad range of health topics are collected through personal household interviews.

For over 50 years, the U.S. Census Bureau has been the data collection agent for the National Health Interview Survey. Survey results have been instrumental in providing data to track health status, health care access, and progress toward achieving national health objectives.

I posted a simple glimpse of National Health Interview Survey( NHIS) data on LinkedIn describing a few tools for working with large public datasets. The questions were coming fast and furious into my messages.

NHIS is an important data source for many reasons and you should know how to access the findings, especially if you are working with population data or interested in social correlates of health.

Not sure why folks aren't comfortable asking questions in comments but nevertheless I decided to use the ease of the platform here to finish the thread.

Here is the original graphic I posted demonstrating how easy it is to get a quick read of your data. The data dictionary will help you understand the variables but when you work with this data a lot--it is in your veins.

I demonstrate below how you would change column headings and give you a hint regarding the next step. I only changed a few as an example.

Because many of the NHIS responses are coded in numeric responses these all need to be changed to categorical.
It becomes obvious when you look at the profiler data below.

If you know the codes for say F10DVCT DURING THE PAST 12 MONTHS, did {you/any member of the family} receive care from doctors or other health care professionals 10 or more times? Do not include telephone calls.
1 = Yes
2 = No
7 = Refused
8 = Not ascertained
9 = Don't know

You can look at the data and see how many households answered "yes" for example. Scroll document below:

surveydata.html
File Size:	51227 kb
File Type:	html

Download File

Would you like tailored instruction on how to work with your data?

For example, unlike public data, when we are the ones designing survey instruments we have latitude in producing high value platforms that don't require analytic contortions to evaluate the responses.

My quick insights are the following:

Likert? No
Multiple choice? No
Ranking? YES

We are adding online workshops to the calendar as I will be traveling extensively throughout September and not available for onsite programs or workshops until mid-October (limited availability).

Select one listed or suggest something specific!

If the data is unclear--what exactly are you writing about?

Ready to integrate Python with Tableau?

8/2/2019

As soon as I completed the Columbia School of Engineering Executive Education program in Applied Data Science I became responsible for translating successful weekly assignments and assessments into an effective and scalable work flow. It is one thing to complete a high-level course at a prestigious university--it is quite another to know how to gain actionable insights outside of a classroom.

My interest in large non-proprietary healthcare databases exceeded anything Excel could hope for and I was amazed at the automation attainable in understanding a few open source languages--R and Python. I believe this is what I needed under the hood of the dashboards and insights clients were requesting. You may or may not know about TabPy--API Tableau Python Server--allowing remote execution of Python Code within a Tableau workbook. Also in Tableau Prep but let's not get ahead of ourselves. The limiting reagent for me was getting everything updated, dowloaded, and connected.

A lot of what is discoverable at the moment is somewhat out of date and might cause you to sip a scotch a little earlier in the day than what is socially acceptable--allegedly.

So my point here is to update Toward Data Science or GitHub (although the Github link I am providing is the updated version shared with me by Nathan Mannheimer a product manager at Tableau).

Download the Github repository and follow instructions in the README.md file. I use juptyer lab so had specific steps to get everything up and running.

I activated my Anaconda environment and downloaded the required files. It is pretty easy. Hit the create button and name your environment--I opted for TabPy. The files I installed into this new env included Numpy, Matplotlib, Pandas, Conda, scikit-learn, ipython, and Spyder (so I could have a testing environment for calculations in Tableau). I updated Python, TabPy, and Anaconda--I had a bit of a circling the drain day when I didn't realize that links were using older versions...live it and learn it!

Once you have your environment created you can click and open your preferred coding window...

I am a loyal MacOS person so your mileage may vary--navigate to Terminal enter "Conda create --name TabPy python=3.7"

Making sure you are in the right directory "cd TabPy-0.7" in my case and now run the "source startup.sh"

Once you have everything properly installed head over to Tableau. Under the Help menu scroll down to Settings and Performance >> Manage External Service Connection.

These are the settings you want to select. Click "OK", close the window and open again to Test Connection.

If you have everything installed correctly--you will receive the blessed notification below. Once I sorted out the old links, outdated software, and steps for MacOs--it all came together but I am hoping to prevent you from the aggravation.

Any questions find me on twitter @datamongerbonny or at the next Healthcare TUG on August 14th...

<<Previous