let's make your surveys suck less...

About a week ago a reluctant client gave me a call. He mentioned being a follower of this blog for a few years but felt like surveys they create in-house were perfectly functional--until during a live webinar I randomly used a survey template that his company had created--and was a wee bit critical.

Now let me be honest. I don't pull the smaller companies to task but focus on the bigger companies with deeper pockets and analytics departments. It is frustrating to see low quality multiple choice questions written in the absence of any concern toward research methodology or rigorous question design.

The good news is I have been invited onsite to a few locations to deliver scalable solutions. It is much easier to drive change in an organization when the process is implemented across a team or a company.

I am not particularly clever about how to market these opportunities but several forward thinking clients decided what they needed. We scheduled a few live customized webinars to support the in-person training and now we are off to the races.

Let me share a few of the insights that seemed the most helpful and easily integrated into processes that either already exist--or should.

I have nothing against Survey Monkey. If you are a skilled analytics professional or a subject matter expert with learning platform expertise and can distinguish satisficing question responses and how to avoid them--go get that banana.

---

I find Qualtrics to be just the right temperature--not too simplistic and accompanied by adequate support and direction to help the novice at least become aware of the possibilities beyond low value question design.

We won't delve into the statistical layer that makes it all possible--not yet--but conjoint analysis is the right tool to answer complex decisions like those made at the point of care. Health care providers and patients consider multiple attributes (characteristics) and features of an intervention or recommendation in complicated ways that it is impossible to measure or address if you are only asking in a Likert or multiple choice format.

​I can recognize when teams aren't up to the challenge. In these cases, just use ranking questions. At least you can safely talk about probabilities if you know how options are prioritized.

---

You might find a little context helpful. If you want to prioritize a behavior, examine what influences decisions, predict how cost impacts therapy selection, or uncover competitive advantages from another product--conjoint analysis is the right way to go.

If I had a nickel for every client that mentions question fatigue as a reason to not develop an impactful survey instrument I would be writing this from Bora Bora with a drink floating a little paper umbrella. My standard argument? Participants/respondents are willing if there is a high-value trade-off.

We are trying to uncover, "Are health care providers/patients/payers willing to sacrifice______ in order to have ______?

First step is to define the attributes of whatever intervention/behavior/commodity we are attempting to measure value decisions. Within these attributes are levels. The statistical modeling will draw combinations of attribute levels and present choice sets to the respondents.

​When presented with choices--respondents select the higher level of utility.
​

• Discrete choice experiments allow estimation of the relative importance of different aspects of care, the trade offs between these aspects, and the total satisfaction or benefit respondents derive from health care services. Monetary values of attributes may be indirectly estimated by including time-based attributes such as "waiting time"...Ryan et al. Use of discrete choice experiments to elicit preferences

Below is a framework that you can follow to design a DCE of your own. The first step is to consider the attributes and assign levels. This works surprisingly well in healthcare and there are many resources to help you get started.

There isn't likely to be anything ready to go out of the gate but if you step through some of the methods carefully you can build your foundation and then perhaps reach out for the analytic platform.

Using discrete choice experiments to inform the design of complex interventions

The critical component is to realize the limitations of low-value data gathering when it does not capture the considerations at the point of care. Health economics considers cost-benefit and trade-offs. Building a better survey strategy can serve as a first step in evaluating characteristics leading to improved behaviors and patient outcomes.

---

You can send an email to schedule a free 15 minute discussion or schedule directly below...

Schedule Appointment