I am embedding a quick file here so I can refer to it live in a presentation for Women in Tech Summit tomorrow...more to come!

In my experience, once you get past the murky distinction of nominal, ordinal, and numeric, there is little thought to how you need to adjust your analytics. The time to make these important decisions is immediately after you create a data question or draft a set of objectives for a need assessment.

To collect measures of attitude, character, and personality that better assess post-activity behavior of a healthcare provider after an educational intervention, you likely rely on Likert attitudinal scales and apply them to professional learning and/or behavior change.

The problem is you need to distinguish between Likert-type items and Likert scales. Likert-type questions are when you design single response questions that are not specifically related. You can list them separately and your intention is not to combine the responses or aggregate a score. 

When you analyze Likert-type responses, you can sense an ordinal measurement--there is a directionality or a "greater than" or "lesser than" relationship. We think of this as ordinal because you can't say much about the distance between the measures or distinguish the intensity of the difference. Because of this you would use descriptive statistics like mode or median or frequencies to assess variability. This might also include chi-square.

Likert scales are a collection of similar questions where you create a composite score based on a quantitative measure of  say how a healthcare provider selects a first-line therapeutic option for a patient with "mild to moderate" rheumatoid arthritis. Because the individual scores are generating a composite or overall measure, now we would use the mean and standard deviations as a measure of variability. These interval scale items would include Pearson's r, t-test, ANOVA, and regression.

This is why decisions around your data analytics start at the question development stage. What type of information are you hoping to gather? What data types do you need to answer your data question?
​You will know this right away if you don't out source data gathering to PubMed published articles or pre-existing survey data.

Beware of low hanging fruit...

Discrete choice models are appropriate for evaluating behaviors that operate within a framework of rational choice. Health care professional, when confronted with a discrete set of options will opt for maximal benefit or utility. The options most likely to improve patient outcomes for example and/or abide within a framework of evidence-based medical choice.

These decisions are made from both the choices presented at the point of care AND the characteristics of the professional making the final selection. It follows from this assumption that utility of a choice is a function of the characteristics of the possible choices and the characteristics of the person making the choice. 

Here is a resource I found extremely helpful--reach out if you are interested in exploring DCA collaboratively.

E. Lancsar, J. Louviere. Conducting discrete choice experiments to inform healthcare decision making. A user’s guide.PharmacoEconomics, 26 (2008), pp. 661–677

The videos will be uploaded to the Fringe Festival website in a few weeks so follow the link above. Until then, here are a few of the dashboards I included in my talk, a specific version of Data Talks, Meople Pumble titled Decision Making in The Face of Uncertainty. Which variables and data sources are needed for a dynamic description of poverty?

“For the most part we do not first see, and then define, we define first and then see. In the great blooming, buzzing confusion of the outer world we pick out what our culture has already defined for us, and we tend to perceive that which we have picked out in the form stereotyped for us by our culture.”--Walter Lippmann (American Journalist--Sept. 23, 1889—Dec. 14, 1974)

The basis for teaching data fluency stems from our very own heuristics. Mine too. None of us are above debate regarding our intentions or assumptions.

Allow me to share a specifically non-instagramy moment. Due to a confluence of events--apparently rodent friendly weather, living on a park with a creek, and a lively chicken coop--we became infested. Mainly the yard and coop but who knows? If truth is indeed "poisoned at its source" according to Walter Lippmann, we attacked with gusto.

After all, we defined the problem therefore the solution appeared to be quite straightforward.

Well let's think this through carefully. If the goal was a systemic ridding of the rodent population from the yard--we were advised by professionals to follow the protocol.

​But often, in pursuit of an outcome we forget about the stench. In this case I am referring to fetid carcasses decaying just about everywhere you can imagine. Out of sight but definitely not out of olfactory range.

The unpleasant imagery may cloud your ability to see the data adjacent similarities but stay with me here. A good data question often requires the holding of tensions. The pursuit of a complete data set to indeed be empowered to curate empathy or insights. Culturally defined norms often make it unavoidably tempting to believe low hanging fruit and reinforce pre-existing assumptions.

We might ignore relevant information out of fear, repulsion, or conscious bias.

​But eventually the stench of bad data will win. And we will need to clean up the mess and start over.

The emerging age of "value" based medicine has outpaced important discussions on metrics and tools to examine value outside of profit motives and limited variables. The buzz-worthiness of terms like big data and value might leave marketing professionals frothing at the bit--the rest of us are or should be moving forward with business as usual.

There is no such thing as big data. Large unstructured datasets didn't simply arrive at the beckon of marketers. And don't get me started on value. Value for whom? If we are indeed in the age of "personalized" anything--why don't we start with refining these general terms to engender meaning and the appropriate granularity.

I originally wrote the bulk of this article on LinkedIn. I have no idea what drives their algorithm but I will share that where other platforms (including this one) yield hundreds of eyeballs in a single day--their metrics show that from thousands of followers on LI only 7 bothered to read it. Maybe it is a dog I don't know. What I do know is that it is packed with resources you might find interesting and useful.

​If I am wrong...woof, woof.

I probably receive anywhere from 20 to 40 messages every week or so from individuals with questions about data literacy, specific data packages, or data visualization with Tableau. There are a lot of questions that I hear that I never thought to ask. I never asked for permission. Many people ask should I learn Python or R?

​What is the best book to read to learn Tableau? These are personal and specific to individual interests and deserving of exploration. Mainly because like anything, your mileage may vary.
And then, there is the tired question about what is the difference between a data analyst, data scientist, data engineer, statistician, blah, blah, blah...I will leave you to explore this well worn debate elsewhere. There also is a bit of a territorial debate where member of these distinct groups poke fun at the other less worthy participants.

​Here is what I learned. The brightest and best want to bring you along. They encourage, share resources, and are welcoming.

A recent podcast featuring Xiao-Li Meng, the Whipple V. N. Jones Professor of Statistics, and the Founding Editor-in-Chief of Harvard Data Science Review stopped me in my tracks. A year ago I was reading through his work and wrote a few blogs about his ideas. Curious about a pending part two to the discussion, Statistical Paradises and Paradoxes in Big Data(I): Law of Large Populations, Big Data Paradox, and the 2016 US Presidential Election, I sent him an email.
​
He promptly responded. Apologized about the delay in the Part II, sent me a copy of the article above and several others he thought might be of interest (along with access instructions). Meet him in the podcast below and learn about his role as Editor-In-Chief of The Harvard Data Science Review.

The Mission and Scope of the HDSR is captured by this quote below:

Read the tag-line and I dare you not to be excited if you are even slightly data adjacent.

A microscopic, telescopic, and kaleidoscopic view of data science

Here is one of the articles referenced in the journal section brilliantly labeled as CORNUCOPIA: impact, innovation, and knowledge transfer.

(A)Data in the Life: Authorship Attribution in Lennon-McCartney Songs
Apologies for the interactive experience that will likely derail your tasks at hand as you explore the features below. If the link brings you to 3 vizzes, pick number 3 to activate the visualization below.
​

I have been taught that items should be bundled in threes so here is one more. I recently read a great article in Medium, Anaconda is bloated--Set up a lean, robust data science environment with Miniconda and Conda-forge by Ted Petrou. Here is how this was a lifesaver. I work with large datasets in healthcare and medicine. The last thing I needed was a clunky giant installation of valuable but likely not needed programs on my MacBook. Thanks to Ted and his willingness to walk me through a few hiccups I am up and running. If you are a Python person and value just-in-time instruction I suggest you check out his workshops (a few free ones are there for a test drive) over on DUNDER DATA.
​
Although I completed an executive online course from Columbia School of Engineering in Applied Analytics--once I returned to the real world I wanted expertise at a granular level in the packages and libraries that I actually use daily. Ted is so clear and patient and provides the detailed explanations of why things work a certain way and how to customize your experience specific for your data needs.
​

Want to connect? Follow along on twitter or catch me live at Women in Tech Summit next month or RStudio conference in San Francisco in January!

Later this month I will be presenting at the Tableau Fringe Festival on October 25th! Announcements about the line-up will be out soon...

I posted a simple glimpse of National Health Interview Survey( NHIS) data on LinkedIn describing a few tools for working with large public datasets. ​The questions were coming fast and furious into my messages.

NHIS is an important data source for many reasons and you should know how to access the findings, especially if you are working with population data or interested in social correlates of health.

Not sure why folks aren't comfortable asking questions in comments but nevertheless I decided to use the ease of the platform here to finish the thread.

Here is the original graphic I posted demonstrating how easy it is to get a quick read of your data. The data dictionary will help you understand the variables but when you work with this data a lot--it is in your veins. 

I demonstrate below how you would change column headings and give you a hint regarding the next step. I only changed a few as an example.

Because many of the NHIS responses are coded in numeric responses these all need to be changed to categorical.
It becomes obvious when you look at the profiler data below.

​If you know the codes for say F10DVCT DURING THE PAST 12 MONTHS, did {you/any member of the family} receive care from doctors or other health care professionals 10 or more times? Do not include telephone calls.
1 = Yes
2 = No
7 = Refused
8 = Not ascertained
9 = Don't know

You can look at the data and see how many households answered "yes" for example. Scroll document below:

Would you like tailored instruction on how to work with your data?

​For example, unlike public data, when we are the ones designing survey instruments we have latitude in producing high value platforms that don't require analytic contortions to evaluate the responses. 

My quick insights are the following:

Likert? No
Multiple choice? No
Ranking? YES

​We are adding online workshops to the calendar as I will be traveling extensively throughout September and not available for onsite programs or workshops until mid-October (limited availability).

​Select one listed or suggest something specific!

If the data is unclear--what exactly are you writing about?

As soon as I completed the Columbia School of Engineering Executive Education program in Applied Data Science I became responsible for translating successful weekly assignments and assessments into an effective and scalable work flow. It is one thing to complete a high-level course at a prestigious university--it is quite another to know how to gain actionable insights outside of a classroom.

My interest in large non-proprietary healthcare databases exceeded anything Excel could hope for and I was amazed at the automation attainable in understanding a few open source languages--R and Python. I believe this is what I needed under the hood of the dashboards and insights clients were requesting. You may or may not know about TabPy--API Tableau Python Server--allowing remote execution of Python Code within a Tableau workbook. Also in Tableau Prep but let's not get ahead of ourselves. The limiting reagent for me was getting everything updated, dowloaded, and connected.

A lot of what is discoverable at the moment is somewhat out of date and might cause you to sip a scotch a little earlier in the day than what is socially acceptable--allegedly.

So my point here is to update Toward Data Science or GitHub (although the Github link I am providing is the updated version shared with me by Nathan Mannheimer a product manager at Tableau).

Download the Github repository and follow instructions in the README.md file. I use juptyer lab so had specific steps to get everything up and running.

​I activated my Anaconda environment and downloaded the required files. It is pretty easy. Hit the create button and name your environment--I opted for TabPy. The files I installed into this new env included Numpy, Matplotlib, Pandas, Conda, scikit-learn, ipython, and Spyder (so I could have a testing environment for calculations in Tableau). I updated Python, TabPy, and Anaconda--I had a bit of a circling the drain day when I didn't realize that links were using older versions...live it and learn it!

Once you have your environment created you can click and open your preferred coding window...

I am a loyal MacOS person so your mileage may vary--navigate to Terminal enter "Conda create --name TabPy python=3.7"

Making sure you are in the right directory "cd TabPy-0.7" in my case and now run the "source startup.sh"

Once you have everything properly installed head over to Tableau. Under the Help menu scroll down to Settings and Performance >> Manage External Service Connection.

These are the settings you want to select. Click "OK", close the window and open again to Test Connection.

If you have everything installed correctly--you will receive the blessed notification below. Once I sorted out the old links, outdated software, and steps for MacOs--it all came together but I am hoping to prevent you from the aggravation.

​

Any questions find me on twitter @datamongerbonny or at the next Healthcare TUG on August 14th...

But here is where measuring objective reasoning and cognition are simultaneously manipulative and also an integral component of understanding why we do what we do.

As an analyst, I constantly review data collection strategies, collected data, and the output from survey instruments. Most industries, with the exception of continuing medical education (CME), are interested in influences of behavior. Shared decision making has guided patients to perhaps ask questions but we have done nothing to evaluate the quality of the responses they may or may not be provided.

Not just measuring how someone answers a multiple choice question and deciding you have influenced their practice of medicine but what drives their decision making?

How likely are they to change behavior based on results from a new randomized controlled trial? How do they evaluate objective findings? Are they concerned about industry funding? Are they influenced by their peers? How do they seek answers to a clinical questions? What publications do they follow? How data literate are they? Do they consider costs at the point of care? Do they use probabilities to evaluate risk/benefits?

The list goes on and on. If you aren't using some form of decision analysis or stepping outside of summary statistics--what do you know for certain?

Are we okay thinking that just because you present an audience with information that it will be distilled and operationalized?

Don't be a turnip...

Often, when working on data projects, I come across useful quick snippets of code that help with the Herculean task of sourcing data. I have quite an elaborate scheme of organizing a library of resources in my Dropbox folders. Here is one I recently discovered and have put to immediate use.

In the last two posts I relied on df.info() to explore a few datasets. A lot of times you just want to hop in and hop out but there are occasions where you need a bigger lens to look at big data. Here is where Pandas Profiling is superb. Read the article below for more details but I am pulling the most relevant steps into the blog for your review and direct application to healthcare datasets.

10 Simple hacks to speed up your Data Analysis in Python

I had to revise some of the code to make it work for me. I tend to use Jupyter notebooks and running commands from the shell ! work better for some reason. This is the code I ran instead of the installation code from the article.

Now we need to import the necessary packages--pandas allows you to convert your tabular into data frames. The link in red is a file I imported from my dropbox files of data sources. To grab the link from your dropbox file simply copy the link provided when you click "share"--replace the "0" with the "1" you see in the file URL below.

​I do a lot of work evaluating population health data. This sample is from Community Health Status Indicators (CHSI)--a bit dated but a good practice dataset to experiment with and learn a few new data skills. I have an extensive checklist of curated data sources shared with workshop attendees or clients but I am happy to share an edited sample with anyone -- reach out to me either on twitter or LinkedIn.

When you are evaluating datasets, the ability to have an overview of number of variables, missing data, and the types of variables is quite handy. You can also review a list of variables highly correlated with each other and therefore eliminated from comparison. All of the visuals are loaded with the simple code above. I broke it into segments for the purpose of our conversation but the html link will take you to the full output.

Certain data questions won't require all of the variables in a particular table. Each variable is summarized here and provides useful information on how the variables were coded (categorical for example), distinct counts and missing data are also provided as well as a graphic where warranted. My best practice is usually to review in Python (or R) and use Tableau Prep to remove unwanted columns. More on that in future post.

You can also view sample data here as well although I tend not to focus too much here since we are able to generate  a lot of summary data elsewhere.

One more option to view your data dynamically instead of simply in a static readout is using the code below. The # is commented out information and will not interfere with your code.

The x-axis below is the total number of data points while the Y axis is the value of every feature for that particular data point. Hit play to see interactivity.

​Obviously a curated dataset is much less crowded but you can select variables for comparison using tools in upper right menu.

You can reach out for specific and scalable conversations about formulating data questions, sourcing nonproprietary data, tidying and cleaning data, analysis, visualization, and communicating your findings.

We now do on-demand webinars in addition to onsite workshops tailored to your data conversations.

Here is a link to the html version of the notebook if you want to scroll around.

Many clients and collaborative partners are excited to visualize their data. The previous post, The voyage of data discovery, and this one are attempts to create awareness around not only the tools but the complexities of formulating a question. This is all happening upstream from our final visualizations.

I have done a lot of work data sourcing. Finding open source data containing databases for query and exploration. The challenge is the scale of big data. Navigating to clinicaltrials.gov is a great example. Here is how I tackle looking around the tables in Python.

I gave a teaser about simple code I use for R last time. Direct any questions to the comment box or reach out on twitter @datamongerbonny.
Again, I am using jupyter notebook as I do work in data literacy and teach workshops--all requiring detailed descriptions. And the biggest reason? This is how I was taught Python in an executive education program at Columbia School of Engineering.

Alternatively you can run code right into your text editor. I use this if for my eyeballs only but it's a little messy if you are trying to create a narrative. This image demonstrates a snapshot only of how many individual files are in the database. I like to explore them individually or combine them in meaningful ways depending on the question to be formulated or answered.

Explaining the details of accessing your files or creating them is a bit beyond this post but we do have workshops forming all the time. I aim to keep it general for those that might just need a nudge or can reach out with a question. If you have downloaded a csv file you can import into Anaconda where you have your Python saved. This way, when you call the file, it is pulled into the console.

For example I imported the pandas module (or library) and assigned the common alias pd. The reason for the alias is to simplify the writing of code. The basic format is alias (pd), and function (read_csv) with the parentheses containing the arguments.

These tables are quite large so often during exploration I will call head( ) for the top 5 rows in each column. You can get an idea of what data type is in each column. On tables unfamiliar to me I will often use the info( ) function. It's a more granular overview--how many rows, columns, datatypes, and issues like memory usage. This is helpful if you are limited in memory and will need to edit your data request for specificity.

Having a quick list like the one above is handy when you are writing code and need to recall the column names and tables where they are located. I will continue to work through data sourcing and tidying before we begin steps to visualization. Remember it all begins with a well articulated question before we can identify and curate the best available data for generating insights and answers.
​