“It was as true," said Mr. Barkis, " as turnips is. It was as true," said Mr. Barkis, nodding his nightcap, which was his only means of emphasis, " as taxes is.”
But here is where measuring objective reasoning and cognition are simultaneously manipulative and also an integral component of understanding why we do what we do.
As an analyst, I constantly review data collection strategies, collected data, and the output from survey instruments. Most industries, with the exception of continuing medical education (CME), are interested in influences of behavior. Shared decision making has guided patients to perhaps ask questions but we have done nothing to evaluate the quality of the responses they may or may not be provided.
Not just measuring how someone answers a multiple choice question and deciding you have influenced their practice of medicine but what drives their decision making?
How likely are they to change behavior based on results from a new randomized controlled trial? How do they evaluate objective findings? Are they concerned about industry funding? Are they influenced by their peers? How do they seek answers to a clinical questions? What publications do they follow? How data literate are they? Do they consider costs at the point of care? Do they use probabilities to evaluate risk/benefits?
The list goes on and on. If you aren't using some form of decision analysis or stepping outside of summary statistics--what do you know for certain?
Are we okay thinking that just because you present an audience with information that it will be distilled and operationalized?
Don't be a turnip...