data talks, people mumble

Keep making noodles...

6/18/2019

"Po: Maybe I should just quit and go back to making noodles.

Oogway: Quit, don't quit? Noodles, don't noodles? You are too concerned about what was and what will be."

Often, when working on data projects, I come across useful quick snippets of code that help with the Herculean task of sourcing data. I have quite an elaborate scheme of organizing a library of resources in my Dropbox folders. Here is one I recently discovered and have put to immediate use.

In the last two posts I relied on df.info() to explore a few datasets. A lot of times you just want to hop in and hop out but there are occasions where you need a bigger lens to look at big data. Here is where Pandas Profiling is superb. Read the article below for more details but I am pulling the most relevant steps into the blog for your review and direct application to healthcare datasets.

10 Simple hacks to speed up your Data Analysis in Python

I had to revise some of the code to make it work for me. I tend to use Jupyter notebooks and running commands from the shell ! work better for some reason. This is the code I ran instead of the installation code from the article.

Now we need to import the necessary packages--pandas allows you to convert your tabular into data frames. The link in red is a file I imported from my dropbox files of data sources. To grab the link from your dropbox file simply copy the link provided when you click "share"--replace the "0" with the "1" you see in the file URL below.

I do a lot of work evaluating population health data. This sample is from Community Health Status Indicators (CHSI)--a bit dated but a good practice dataset to experiment with and learn a few new data skills. I have an extensive checklist of curated data sources shared with workshop attendees or clients but I am happy to share an edited sample with anyone -- reach out to me either on twitter or LinkedIn.

When you are evaluating datasets, the ability to have an overview of number of variables, missing data, and the types of variables is quite handy. You can also review a list of variables highly correlated with each other and therefore eliminated from comparison. All of the visuals are loaded with the simple code above. I broke it into segments for the purpose of our conversation but the html link will take you to the full output.

Certain data questions won't require all of the variables in a particular table. Each variable is summarized here and provides useful information on how the variables were coded (categorical for example), distinct counts and missing data are also provided as well as a graphic where warranted. My best practice is usually to review in Python (or R) and use Tableau Prep to remove unwanted columns. More on that in future post.

You can also view sample data here as well although I tend not to focus too much here since we are able to generate a lot of summary data elsewhere.

One more option to view your data dynamically instead of simply in a static readout is using the code below. The # is commented out information and will not interfere with your code.

The x-axis below is the total number of data points while the Y axis is the value of every feature for that particular data point. Hit play to see interactivity.

Obviously a curated dataset is much less crowded but you can select variables for comparison using tools in upper right menu.

data collaborations...

You can reach out for specific and scalable conversations about formulating data questions, sourcing nonproprietary data, tidying and cleaning data, analysis, visualization, and communicating your findings.

We now do on-demand webinars in addition to onsite workshops tailored to your data conversations.

Here is a link to the html version of the notebook if you want to scroll around.

Exploring large datasets part deux...

6/17/2019

Many clients and collaborative partners are excited to visualize their data. The previous post, The voyage of data discovery, and this one are attempts to create awareness around not only the tools but the complexities of formulating a question. This is all happening upstream from our final visualizations.

I have done a lot of work data sourcing. Finding open source data containing databases for query and exploration. The challenge is the scale of big data. Navigating to clinicaltrials.gov is a great example. Here is how I tackle looking around the tables in Python.

I gave a teaser about simple code I use for R last time. Direct any questions to the comment box or reach out on twitter @datamongerbonny.
Again, I am using jupyter notebook as I do work in data literacy and teach workshops--all requiring detailed descriptions. And the biggest reason? This is how I was taught Python in an executive education program at Columbia School of Engineering.

Alternatively you can run code right into your text editor. I use this if for my eyeballs only but it's a little messy if you are trying to create a narrative. This image demonstrates a snapshot only of how many individual files are in the database. I like to explore them individually or combine them in meaningful ways depending on the question to be formulated or answered.

Explaining the details of accessing your files or creating them is a bit beyond this post but we do have workshops forming all the time. I aim to keep it general for those that might just need a nudge or can reach out with a question. If you have downloaded a csv file you can import into Anaconda where you have your Python saved. This way, when you call the file, it is pulled into the console.

For example I imported the pandas module (or library) and assigned the common alias pd. The reason for the alias is to simplify the writing of code. The basic format is alias (pd), and function (read_csv) with the parentheses containing the arguments.

These tables are quite large so often during exploration I will call head( ) for the top 5 rows in each column. You can get an idea of what data type is in each column. On tables unfamiliar to me I will often use the info( ) function. It's a more granular overview--how many rows, columns, datatypes, and issues like memory usage. This is helpful if you are limited in memory and will need to edit your data request for specificity.

Having a quick list like the one above is handy when you are writing code and need to recall the column names and tables where they are located. I will continue to work through data sourcing and tidying before we begin steps to visualization. Remember it all begins with a well articulated question before we can identify and curate the best available data for generating insights and answers.

The art and science of asking questions is the source of all knowledge--Thomas Berger, American novelist

The voyage of data discovery...

6/15/2019

The real voyage of [data] discovery consists not in seeking new landscapes, but in having new eyes--Marcel Proust

The ability to grab nonproprietary data for question generation and exploration might be the most neglected resource. For example, continuing medical education companies create programs at an immense profit often based on hunches, heuristics, or whatever the funder or sponsor deems worthy. What might the landscape or outcomes consider if there were better survey analytics or relevant metrics analyzed?

I have tried. Most often I am told that the preferred mechanism or platform is to keep the status quo. Useless pre- and post-evaluation measuring nothing of value or multiple choice formats that do not mirror how physicians take care of patients. The ever present Likert style questions confusing unimodal and multimodal scales. I will say two things here--again. Values on a Likert scale are not comparable.

A score of 5 on one question is just that--a five on one question. Second, if you start using ranking scales you will improve your data quality and analyses exponentially. Why?

How are we communicating about point of care interventions in the absence of decision choice experiments, determinations of value for each stakeholder, and collaborations with patients?

Think about probabilities. In medicine we measure probabilities. What is your risk for a disease, what are the chances you will respond to treatment--think about large scale risk/benefit questions. Today we aren't tackling surveys though--you can read Write Better Surveys. Period. for the price of a cup of fancy coffee or stay tuned to future posts.

I wrote a quick post over on LinkedIn that generated a lot of interest and not a small amount of private messages and queries. As much as I would prefer to engage colleagues publicly to foster broad discussion I get it--you are a shy bunch. I shared my method for jumping into a large dataset and looking around. Obviously, there is more work to be done but once you can see what questions might be able to be answered with data--you can ask better questions and begin questioning data.

A big dataset with a lot of options for data curiosity is the ClinicalTrials.gov database of the US National Library of Medicine. A friend was curious about the reason clinical trials are terminated and he had some static spreadsheets with aggregated answers. I thought it might be interesting to dig a little deeper so gave him the interactive visualization above, studies reporting "low accrual" or studies reporting reasons other than "low accrual" available at Tableau Public and this one as well, actively recruiting clinical trials for colorectal cancer.

I typically start over on the website for downloading content for analysis--Clinical Trials Transformative Initiative (CTTI)'s Database for Aggregate Analysis of ClinicalTrials.gov (AACT). I suggest reading the page to better understand your options for data but if you are a beginner, scroll down to the link for the Clinical Trials project. This simplifies the download of aggregated data from the publicly available relational database.

Once you click on download, it will take you to the following options. If you want to run a local copy of the AACT database (static) you can install PostgreSQL server and viola, there you go. I no longer use this option as from time to time the code will break and not accept my password. It creates this loop of resetting password and waiting and waiting.

I started using pipe-delimited files and working in R. I will show you a brief line of code or two so you can see how easily this option might work for you. The advantage of pipe-delimited files? They look like this column 1|column 2|column 3|column 4 so instead of being separated by a delimiter that might also be a value, such as a comma (in csv for example)--you have the pipe character.

Due to the size of a large database, it isn't recommended that you open it up in Excel for example. I like to work in RStudio. Because I teach a lot of courses in R, I prefer R Studio Cloud (the data and environment stay with each instance of a project) but for now let's just look at RStudio.

If you want to learn more we can set up a remote session where you can log into my cloud environment and interact live while we explore a database of interest (see below)...

I pulled a single table from the downloaded file and here is an example of how you can quickly examine data with a few lines of code. I quickly sampled 200,000 samples for quick information on column headings and can make a quick estimate regarding what I might find. For example, look at the column "reason".

We have over 50 other tables to explore for granularity by condition, clinical trial phase, year of start or completion, results availability, or clinical trial design to name a few.

Here is the list of other tables in the dataset. Make sure you review the data dictionary to help you select tables of interest.

Closer examination of the tables in the database are informative and I am happy to explore the data with you to help guide you through how you might use either R, Python, or Tableau Prep to clean up data for visualization. Stay tuned...

The data that didn't bark...

5/29/2019

Are you a Sherlock Holmes fan by any chance? One of the more famous tomes involves the disappearance of a valuable racehorse.

The "curious incident of the dog in the night-time" is easily explained: the dog made no noise, because no stranger was there. As Holmes explains: "I had grasped the significance of the silence of the dog, for one true inference invariably suggests others.... Obviously the midnight visitor was someone whom the dog knew well. It was Straker who removed Silver Blaze from his stall and led him out on to the moor"--The Adventure of Silver Blaze (Wikipedia)

Obviously there are other reasons the dog may not have barked but the most dramatic of course, was described by Holmes. This reminded me of silent data. Hear my out--I am going some where with this.

We are accustomed to looking at the same things the same way. I attended the Committee on Informing the Selection of Leading Health Indicators for Healthy People 2030. I remain unimpressed by the Healthy People 2020 metrics for many reasons. Sure it's a start but the problem remains--if data is old, aggregated to too high of a level, and chock full of low hanging fruit--what are we striving for?

Amy O'Hara from the Massive Data Institute at McCourt School of Public Policy, Georgetown University discussed data sources for objectives and the Leading Health Indicators. I could relate with her tasks the most. It is all about what you measure. The data that is now being gathered to expand definitions of poverty was not gathered to be shared. These were all internal databases not designed for harmonization--hence we have this massive problem with data quality and intersectionality.

Should we do what we have done before because it is easy and the data is ready?

The non-barking dog--or should we evolve our questions and take the practical steps to build a system that we can improve. Incentivize people to understand what is being measured. Head over to HealthyPeople.gov and you will see what I mean.

If reducing diabetes-related deaths (age adjusted, per 100,000 population) is the target, how are we going to focus on the disparities in counties, zip codes, and communities? We are not measuring the right things. I would argue that within National Target Met (green states) areas are many communities suffering from disease. We need granularity. We need new measures--we need a barking dog.

Akimbo is one of those podcasts that delivers time and time again. Seth's clarity and insight can help redirect your focus if you start monitoring the wrong things.

Seth Godin argues we need to place the right metrics on the table. What story do we tell ourselves, our co-workers, and customers? It matters...

Money has one advantage. It is easy to measure but that isn't the point. We want change we can be proud of. Lots of organizations measure money but they are leaving a trail of obesity or disease, discomfort or unhappiness. This creates systems that can't sustain for the long haul.--Seth Godin

What are you measuring?

Finding true merit in a deep river...

5/20/2019

I have been thinking about the complexity of the questions we ask and the ease with which we accept simple answers. Often, a question isn't even articulated. We have data. Lots of it and many of us want answers. But how do we navigate the noise? A solitary clear bell among a cacophony of distraction.

Professionally, I work a lot with publicly available data sources such as IPUMS and US Census Data. I know that race is not a useful variable. I question what you are measuring when I see poverty rates reported, or gender captured in a survey. Here is the problem. First, race is a social construct. If you want to measure social determinants--measure them. Don't use race as a proxy. Same in medical cohorts. If you stratify by race you need to identify what you are looking for--before the analyses.

How should we look at poverty? I look at the interests of the public good--what constitutes social welfare. Shortcuts inclusive of solely income stratification will never illuminate problems or solutions. How about aligning edges of variables that measure social inequality, power, relationally, social context, complexity, and social justice as described by Collins and Bilge below.

Positing that contemporary configurations of global capital that fuel and sustain growing social inequalities are about class exploitation, racism, sexism, and other systems of power fosters a rethinking of the categories use to understand economic inequality. Intersectional frameworks reveal how race, gender, sexuality, age, ability, and citizenship relate in complex and intersecting ways to produce economic inequality--Intersectionality, Patricia Hill Collins and Sirma Bilge

Unemployment, poverty, racial and gender discrimination, homelessness, illiteracy, poor health, and similar social problems within a nation-state constituted threats to the public good that were just as prominent as external enemies beyond its national borders.--Intersectionality, Patricia Hill Collins and Sirma Bilge

A recent article in The New Yorker, The Philosopher Redefining Equality reminded me of the intoxicating siren song of Baader-Meinhof phenomenon--a type of cognitive bias.

Elizabeth Anderson is the chair of the philosophy department at University of Michigan. Her belief that "equality and freedom are mutually dependent, enmeshed in changing conditions through time" captured my attention and improved my awarenesses of the fluctuations of what we mean when we use terms like "value".

"Her first book, “Value in Ethics and Economics,” appeared that year, announcing one of her major projects: reconciling value (an amorphous ascription of worth that is a keystone of ethics and economics) with pluralism (the fact that people seem to value things in different ways)."

"... Instead of positing value as a basic, abstract quality across society (the way “utility” functioned for economists), she saw value as something determined by the details of an individual’s history."

Andersonism holds that we don’t have to give up on market society if we can recognize and correct for its limitations—it may even be our best hope, because it’s friendlier to pluralism than most alternatives are. And we shouldn’t commit ourselves to an ideal system of any sort, whether socialist or libertarian, because a model set in motion like a Swiss watch will become a trap as soon as circumstances change. Instead, we must be flexible. We must remain alert. We must solve problems collaboratively, in the moment, using society’s ears and eyes and the best tools that we can find.

I spend a lot of time explaining the granularity of variables routinely pulled into data queries. You might not be measuring what you think you are measuring.

We certainly don't need any more noise drowning out important conversations--do we?

True merit, like a river, the deeper it is, the less noise it makes--Edward Wood

data lying idle on a shelf is wasted ammunition...

4/17/2019

"If you want to build a ship, don’t drum up people to collect wood and don’t assign them tasks and work, but rather teach them to long for the endless immensity of the sea."
- Antoine de Saint Exupery

A few friends invited me to Beaufort, North Carolina for the weekend. Not my typical coastal haunt but saying we had fun is a huge understatement.

Yes we did end up on a sail boat but the part that got me thinking--induced by staring out of the car window on the way down--we are all selling something.

All of the shops want you to purchase and they hoist the large banners to prove it. Everything is on sale and everything is a bargain and you need it all now. Clearly this isn't only in small coastal communities. Think about all the news, magazines, podcasts, newsletters, and data now being packaged for subscription service.

I doubt auto shops want you to know how simple it is (in most cases) to change your own oil. Restaurants are feeding the trend to make us feel good about not cooking--curbside pick-ups to consume at home around your own kitchen table abound. What if doing our own taxes, shopping at farmer's markets, or cleaning our own homes became de rigueur?

I would argue that many of us can manage our own data questions once we learn the ropes. Or at the very least learn to understand the limits of insights gleaned from information we happen to be collecting--either actively or passively.

The graphic below is a perfect example of how often we might be tempted to rely on others for insights. Roger Peng, demonstrates how important a good data question is to the goal of data science. The buzz is all around for machine learning but we all can contribute to creating high quality questions--answerable by good data (strength of evidence).

When you begin looking for data--in my case mostly open source--you need to be fully transparent. I began snooping around a few years ago to see what I could bring data clients to help them appreciate the immediate need for a data governance plan.

In many cases, there was not a single person charged with managing company data. In healthcare this can be problematic for sure. It was this process that led me to Nicola Askham. Nicola is the leading data governance coach in the UK and we quickly began having conversations. Over the course of a year or so we realized that many of our discussions might bring value to both of our networks. We decided to host a free Data Governance Webinar and invite one and all.

In healthcare, responsibility for the scale of data we are facing in modern analytics can be overwhelming. Now that insights have been liberated from dusty medical charts stored in file cabinets--now what? Do we have policies in place that allow us to be confident in the quality of our data?

Missing data, the wrong data, or incorrectly merged records can all compromise patient outcomes and most importantly patient safety.

If you are new to the conversation, begin here with Nicola's blog post, How to Write a Good Data Governance Policy. Send along any questions and we will address them in the webinar.

And then join us tomorrow at 11:30 EST--Data Governance for Healthcare.

Smooth sailing ahead or if things work out--the "beam reach"...

"A book lying idle on a shelf is wasted ammunition."--Henry Miller

Watch data governance for healthcare webinar here:

Best outcomes guaranteed?

3/27/2019

Meet your continuing medical education (CME) faculty...

Not sure how likely CME is to influence decisions at the point of care. Even large companies are evaluating participants based on multiple choice or Likert questions--not looking at probabilities or risk/benefit considerations facing health care providers on a per patient basis.

I explored the reported details from financial disclosure statements on Open Payments Data and noted the following compensation rates for a single CME panel activity.

Here is a typical CME activity in Rheumatology.

Expert Faculty A--BELOW the National Mean by $1,279.50/ABOVE the Specialty Mean by $1,632.60
Expert Faculty B--ABOVE the National Mean by $208,641.34/ABOVE the Specialty Mean by $206,573.10
Expert Faculty C--ABOVE the National Mean by $226,765.21/ABOVE the Specialty Mean by $220,628.17
Expert Faculty D--ABOVE the National Mean by $16,668.49/ABOVE the Specialty Mean by $10,531.45
Expert Faculty E--ABOVE the National Mean by $120,973.39/ABOVE the Specialty Mean by $114,836.35

Now before you consider me a curmudgeon, I did a little research first. I am not about targeting the small and scrappy companies--I am about the big companies with large pockets that honestly should know better.

I did a quick glance of the accompanying slide compendium and noted a lack of effect sizes, preponderance of p-values, a rogue hazard ratio here and there and a blatant lack of statistical rigor. The data and findings were based on abstracts only so we are even deeper in the weeds when trying to make sense of the insights being recommended by the panel.

Here are a few reasons we need effect sizes--we need to make assumptions about a normal distribution, p-values depend on the size of the effect AND the size of the sample. You can play this out by either having a huge effect size in a small population or a tiny effect size in a very large population.

If your objective is to educate physicians about the latest in clinical research--shouldn't the emphasis be on the significance of the effect, not the sample size?

What CME can learn from Courtyard Marriott

Because I work with a lot of teams that rely on survey data--I am focused on how to generate data with robust and rigorous research methodology. Unfortunately in CME we seem to be tethered to old ways of thinking and continually limit ourselves based on the limitations of existing frameworks.

One of the first sources of a useful data structure (conjoint analysis) came to me from a 1989 paper, Courtyard by Marriott: Designing a Hotel Facility with Consumer-Based Marketing Models.

This looks complicated but the figure below describes 50 factors that describe hotel features and services and the associated (167) levels are categorized under seven facets (External factors, Rooms, Food, Lounge, Services, Leisure, Security).

You can easily define facets (or attributes) in whatever therapeutic area you are targeting for intervention. Think how we can create metrics if we instead focus on rheumatoid arthritis for example. Combination therapy, route of administration, location of administration (home or infusion clinical), frequency of administration, time needed for infusion, experience with DMARD, chance of efficacy, onset of action, cost, etc...

I use examples from RA because of the complexity of measuring physician behavior with current standards of multiple choice or Likert. It is impossible. There is always a heuristic, trade-off, or utility consideration that you are not bothering to measure.

Look at what we can generate if we build data models. I created a data model using information from a paper by Scalone and colleagues, Patients', physicians', nurses', and pharmacists' preferences on the characteristics of biologic agents used in the treatment of rheumatic diseases.

What you are left with are deep insights regarding considerations at the point of care. Discrete choice modeling looks at several factors within a decision RELATIVE to each other, forcing respondents to make tradeoffs among different options.

What we learn from this type of data allows us to gain deeper insight into which factors respondents find most important.

Here is how we do it:

•Putting together an attribute list
•Define competitive space or preferences for a product or behavior
•Construct Experimental design
•Combinations of attributes are not correlated but balanced across products or behaviors
•Pre-test--instrument is tested for clarity, length and efficacy.
•followed by an editing and revision cycle that ultimately produces the field survey instrument.
•Send out to audience
•Set up data
•choosing variables to include in the modeling process
•Estimate model
•Build simulator

Agreed it would be easier to write simplistic multiple choice questions that isolate respondents from the actual influences at the point of care--but to what end?

The data has no value and is not moving the needle on improving the quality of healthcare.

If that troubles you--send me a tweet or email to discuss your data needs. Brainstorming is always free!

Now offering customized webinar learning live...

An adventure is no fun if its too easy!

3/13/2019

If you follow advances in immunotherapy the amazing pace of discovery is mind boggling. Depending on the pathway, the ability to target oncogenes and yield clinical improvement is beginning to become a reality. Although maybe not as quickly or enthusiastically as phase II study results would like...

If there is an enzyme to target all the better. The problem with RAS is the target is a protein--so we need to be able to insert activity at a single location requiring intricate folding and precision.

The proteins of the Hedgehog (Hh) family when deregulated may lead to defects and cancer in vertebrates and invertebrates.

They are called hedgehog because if you view the mutant drosophila melanogaster embryos you can envision little "hairs" poking out reminiscent of a hedgehog.

In fact, the pathway in humans feature 3 paralogous genes: Sonic hedgehog, Indian hedgehog, and desert hedgehog.

An adventure is no fun if its too easy!--Sonic the HedgeHog

If you have been stopping in at data & donuts over the years it isn't a big secret that I enjoy listening to podcasts. Really informative and eclectic conversations like Two Scientists Walk Into a Bar featuring Jane Grogan, Principal Scientist of Cancer Immunology at Genentech.

The format of the podcast features a disembodied voice (the producer) asking some of the lay person questions as the scientists focus on conversations around human biology and often cancer immunology and tumor micro-environments.

Oncogenes are an interesting area of study. They are basically genes that have potential role in specific cells but can also initiate cancer pathways when mutated.

Now that we have identified a wide array of oncogenes you may be curious--why is it so hard to directly drug tumor cells? In the same way that tumor cells respond and direct their own environments (stromal infiltration for example) they also respond to their drug environments and develop resistance.

You may also be surprised to learn that only a few of the known oncogenes are indeed "druggable". For example what if the mutation means that the gene is missing? You need to understand the signaling pathway downstream to know how to target.

The sonic hedgehog pathway is important for development of every organ in the body. What happens to developmental pathways if mutations are activated?

You can read a little more about the pathway below:

A dual mechanism of activation of the Sonic Hedgehog pathway in anaplastic thyroid cancer: crosstalk with RAS-BRAF-MEK pathway and ligand secretion by tumor stroma

Join me for a free webinar on March 29th at 1:00 PM EST.

The 5 second rule: how dirty is your data?

Wikipedia defines the 5 second rule: a defined window where it is permissible to pick up food after it has been dropped and thus exposed to contamination...allowing them to eat a dropped piece of food, despite the potential reservations of their peers.

Hopefully your data doesn't have weird fuzzy things sticking to it or unidentifiable lint or dirt--but what if you aren't certain? How can we make determinations regarding the reliability of our data sources?

The fifth lane of data science: storytelling

2/4/2019

If you are using data in any capacity I recommend listening to Data Framed. I always manage to learn a little something and also be sort of a data voyeur. The podcast reliably features working data scientists willing to share their process and data culture.

I think it's imperative that as many people know about how to handle their own personal data, as well as how to handle data within their own domain. I think every domain uses data in some aspect. Either you're collecting it, either you're manipulating it, or you're using it. It doesn't matter what discipline you're in. You need to know how to manage it.

You need to know when you have enough data to make sound decision. You need to know when you don't have enough data to make sound decisions. You have to know how to ask questions. It's about being curious. It's about trying to understand what that data is trying to tell you, not what you're trying to force the data to tell you.--Brandeis Marshall, Associate Professor of Computer Science at Spelman College

I especially like the consideration of data science as five lanes. Data collection and cleaning is the first lane followed by storage and management, analysis, visualization, and storytelling. Brandeis reminds us that even if you manage the first 4 lanes--if the story doesn't make sense--it all falls flat.

Depending on your data origin story you may have different experiences along the way but I began as a medical writer. I think of medical writing and data integration as having an explanatory role. A client would have a complex clinical study or complicated research question requiring an explanation. Here is the data--please tell the "so what" story that will make an audience care.

Now as I lean closer to applied data science in my educational and professional experience, I am more often than not, assuming an exploratory role. Ground zero is now formulating a question and discovering if we have the data to answer.

Simple statistics can tell you the shape of your data, variance, correlations, clusters--and provide a visual of where to go next.

Inspiration is cheap, but rigor is expensive. And if you are not willing to pay up, don't expect that there is some magical formula that's going to give it to you.--Cassie Kozyrkov

Where do you get the skills? Workshops are expensive and time consuming. I was fortunate and able to step away to take refresher courses in data analytics but that isn't for everyone.

The conversation from Data Framed actually estimated 10s of thousands of dollars and two-week bootcamps running from 9 to 5 every day. Not exactly what I would consider true "access" for those of us wanting to skill up.

We are developing a series of data literacy workshops. Simply a few hours of instruction on a variety of topics--beginning with needs assessments. It may sound specific but what is a needs assessment if not a process for identifying a question.

How do you formulate questions, identify proper data sets, access, collect, manipulate, use and operationalize?

What are the outcomes you hope to measure? How should they be measured? Can they be measured accurately? What are the best types of survey questions? How do I analyze the data?

Register at the link Continuing Medical Education: change isn't necessary, survival is not mandatory.

Folks that have already registered have received instruction on downloading open source software, a free coupon to background resources and a few introductory sample sets to manipulate before the April 11th workshop date.

Stay tuned for archived workshops in data governance, Python for healthcare, R programming for journalists, and a wide variety of topics requested by you...

Public Use Microdata Samples (PUMS) of US Slave Population

2/3/2019

There are many data sources freely available once you achieve a certain level of data literacy and graphicacy. IPUMS is a curated collection of census and survey data critical for comparative research of individuals, families, or communities.

One of the advantages of IPUMS is the weighted data provided. For example, in a given data sample, each singular person in a "flat" or unweighted file represents 100 people.

In the weighted files, individuals are not represented as 100 people but their actual representation in the general population is provided.

The Tableau story below is being created to help guide an audience through data exploration using visualization. It is a rough depiction of how visualizing data can be instrumental in generating hypothesis.

In honor of Black History month, I visualized data from the 1850s and 1860s regarding the US slave population. Once the presentation is live I will be happy to post--but for now you can view the story as a stand alone collection of visualizations meant to suggest opportunities for additional exploration.

Scroll across captions to advance images...

<<Previous