Are medical education professionals leaving data on the table?

What is assessment 2020?

Medical education professionals are struggling to develop relevant and rigorous assessment  to address the evolving US healthcare system.  The majority of assessment questions that I review are written outside of the point of care or actual doctor/physician model of healthcare. Multiple choice questions are listed with answers that seek to solicit a "correct" response to be compared pre- and post-education as proof of the effectiveness of an educational product or intervention. Unfortunately this is not how medicine is practiced. There are few "right" and "wrong" answers to address the heterogenous patient populations uniformly in the majority of clinical encounters. Clinicians would also never make such determinants of care without ever speaking directly to their patients. This should make you question the utility of our current physician engagement strategies within medical education.

Clinical trial evidence that informs evidence-based care is heavily influenced by industry and limited by dubious statistical methods. At the very least, recruitment seeks to enroll a homogenous population that bears little resemblance to the clinical patients outside of strict inclusion criteria. The data generated consists of averages, not specific information to inform outcomes at the patient level. Now that we have introduced fee for value, how are we educating professionals to evaluate downstream cost and consequences of a particular care algorithm? What does high-value care look like at the individual patient level?
 
Assessment 2020 task force  provides the following clinical scenario and discusses potential limitations of our current evaluation strategies:

A 70-year-old man is referred to you for advice on treatment following a pulmonary embolism. The pulmonary embolism was diagnosed when he presented with acute shortness of breath and chest discomfort three months ago.  He was treated initially with low molecular weight heparin, and then with warfarin, aiming for an INR of between 2.0 and 3.0. No underlying risk factors for the embolic event were identified. The patient has had no bleeding episodes during therapy. His only other active medical issue is hypertension, which is controlled with a diuretic. His referring physician asks for your recommendation concerning anticoagulation at this time.


The best recommendation now is:


A)    Stop the warfarin, and start low dose aspirin.

B)    Continue his current dose of warfarin for three more months, then switch to low dose aspirin.

C)    Continue warfarin indefinitely, but reduce the dose, aiming for an INR of 1.5 to 2.0.

D)   Continue his current dose of warfarin indefinitely. 

This question resembles the majority of questions used to evaluate participants knowledge or practice behavior. The problem is how do we integrate the patient-centric model described in the evolving care continuum?  Follow the scenario below...

Here, any of these could be acceptable strategies.Suppose it has been difficult to maintain an INR of 2.0 to 3.0, and the patient has been greatly troubled by the need for monitoring. Let’s say he also enjoys biking, but has curtailed this activity because of concern about the risk of bleeding. If I explain to him in a way that is easy to understand that trials have shown a risk of a recurrent DVT of 7-9% per year with no treatment, and that this could be reduced by starting low dose aspirin to about 5% per year. If he prefers this risk to continued warfarin therapy (which could further reduce his risk), then this would be an appropriate strategy. 

But suppose he—or a family member—is very concerned about a recurrent pulmonary embolus and has not been bothered by the warfarin therapy or monitoring; in this case, continuing his current dose indefinitely would give him the lowest risk of a recurrent thrombotic event. Option B could reduce his chances of post-thrombotic symptoms, and if he were willing to continue warfarin for a few more months, might be best for him. If he had concerns about bleeding as well as a recurrent blood clot, then option C might be best.

So as you can see, it is impossible to measure the true competence or behavior of a clinician when we are not considering the patient-clinician interaction, value assessment, and downstream outcomes that influence care across the entire cycle of care--not just an episodic interaction.

While it might be tempting to boil down treatment decisions to multiple-choice answers, those who practice medicine know that the knowledge we gain through interactions with patients is critical to making the right decisions and the best recommendations.

As we look at the future of physician assessment through Assessment 2020, one of our tasks is to consider how we can account for the “hidden curriculum” in treatment.

• If a patient’s values and preferences are elemental to decisions about care, how can they be incorporated into the testing environment?
• How should physicians best tested on core knowledge in an environment that does not allow for patient interaction (i.e., multiple-choice tests)?--Dr. Harlan Krumholz, Chair of the Assessment 2020 Task Force

I have published a few blogs about the complexities of EHR integration and continue to partner with practices to address the challenges ahead. Stay tuned for discussions of how to make EHR data actionable and better able to address "value" gaps at the point of care...

Thoughtful discussions about content development and outcomes analytics that apply the principles and frameworks of health policy and economics to persistent and perplexing health and health care problems

freelance writer--health economics, health policy, clinical medicine and insight analyst