I migrated my understanding of outcomes research outside the penumbra of CME and observed what was happening industry adjacent. When the stakes are higher in hospital systems, medical practices, and in organizations with an ethical health economics platform there was much to learn. The first thing to go <<Insert happy dance>> were poorly designed learning objectives and pre- and post-tests.
Read more thoughtful suggestions and a summary by Steven E. Nissen, Chairman of the Department of Cardiovascular Medicine at the Cleveland Clinic and Professor of Medicine at the Cleveland Clinic Lerner College of Medicine at Case Western Reserve University, Reforming the continuing medical education system published in 2015.
The delivery of medical education relies on the traditional methods for didactic learning such as lectures and seminars in customary settings such as auditoriums or classrooms. The content is largely driven by the educator without any assessment of the true educational needs of the learner. Using these traditional delivery methods, there are few opportunities to assess whether the educational program actually changed physician behavior or resulted in improvement in patient outcomes.--Steven E. Nissen, MD
"As a result of the education, you plan to: (Learner checks all that apply)."
Across two experiments, people showed evidence of poor metacognitive awareness of belief change about a social science topic. Following belief change, recollections of initial beliefs were generally inaccurate and biased toward current beliefs. In mediation analyses in both experiments, results showed that the relationship between the belief consistency of the text read by a person and their accuracy at recollecting their initial belief was mediated by their level of belief change.
These results suggest that recollections of previous beliefs are constructive, and influenced by the beliefs of the person at the time of the recollection. We propose that changes in beliefs do not tend to be moni- tored as they happen. Rather, we propose that people sim- ply use salient information at the moment of recollection to try to reconstruct their previous beliefs.
- After reading, subjects reported their current beliefs and attempted to recollect their initial beliefs.
- Subjects reading a belief inconsistent text were more likely to change their beliefs than those who read a belief consistent text.
- Recollections of initial beliefs tended to be biased in the direction of subjects’ current beliefs. In addition, the relationship between the belief consistency of the text read and accuracy of belief recollections was mediated by belief change.
- This belief memory bias was independent of on-line text processing and comprehension measures, and indicates poor metacognitive awareness of belief change.
Awareness of belief change is an under studied issue, but may be an important factor in people’s metacognitive understanding of their own comprehension. If people show little awareness of changes in their own beliefs, they may erroneously conclude that their beliefs are more fixed than they actually are, and consequently may be less willing to engage with information that is contrary to their beliefs.
1. Use a decision analysis format. Medicine is not practiced in a multiple choice format. There are trade offs. I have data to prove it. This graphic is from an educational program asking providers to rank considerations at the point of care. We ask multiple questions in this format to help contextualize decisions at the point of care (1 is highest priority, 8 is the lowest). Think about the granularity of insight when cross-tabulations of these findings are available for a wide variety of therapeutic decisions.
2. Provide context of original response when querying about current behavior.
3. Because learners grasp currently available tools and insights when making decisions, duplicate the post-activity milieu similarly. Asking learners about their current behavior in a vacuum is dangerous. The brain hates gaps and will fill them with whatever is quickly recalled.
4. This is a small study, I get it. But look at the bibliography of the Wolfe and Williams article. Follow the thread to the foundational research.
5. Listen to the podcast below.
In the show, psychologists Michael Wolfe and Todd Williams, take us though their new research which suggests that because brains so value consistency, and are so determined to avoid the threat of decoherence, we hide the evidence of our belief change. That way, the story we tell ourselves about who we are can remain more or less heroic, with a stable, steadfast protagonist whose convictions rarely waver ...
This can lead to a skewed perception of the world, one that leads to the assumption that mind change is rare and difficult-to-come-by. And that can lead to our avoiding information that might expand our understanding of the world, because we assume it will have no impact.