A recent article in thebmj, Maximising the impact of patient reported outcome assessment for patients and society discusses the importance and limitations of patient reported outcomes in directing appropriate high-value healthcare to patients. Data governance is typically missing or tossed in as an afterthought so this figure and the contextual integration of the process is important--and captured my attention.
My expertise has defined my major collaborative efforts as designing survey instruments, providing an overview of a data governance strategy or where to integrate into existing systems, and as peer reviewer for Patient Centered Outcomes Research Institute (PCORI) data methods and implementation.
Many clients rely on Patient-Reported Outcomes Measurement Information System (PROMIS) measures developed with National Institutes of Health (NIH) funding. For the most part, I am not a fan. I can appreciate the need for standardization of instruments for interoperability but it is a bit of a Heisenberg Uncertainty dilemma--I would argue the generalization of measures influences the outcome.
"The uncertainty principle says that we cannot measure the position (x) and the momentum (p) of a particle with absolute precision. The more accurately we know one of these values, the less accurately we know the other."--Alok Jha
PRO instruments are anything but standard. A loose classification by the University of Oxford Patient Reported Outcomes Group identifies a variety of instruments, for example, disease-specific, population-specific, dimension specific, generic (SF-36), individualized, summary items, utility measures and so on.
Limitations include but are not limited to the lack of application or comparison to general population, few response categories, broad focus, lack of application to real world, many developed in the 90s or older, pre-dating the digital landscape and advanced knowledge of cognitive frameworks, and many other reasons to be wary of the generalized approach to gathering subjective data.
Many clients rely on Patient-Reported Outcomes Measurement Information System (PROMIS) measures developed with National Institutes of Health (NIH) funding. For the most part, I am not a fan. I can appreciate the need for standardization of instruments for interoperability but it is a bit of a Heisenberg Uncertainty dilemma--I would argue the generalization of measures influences the outcome.
"The uncertainty principle says that we cannot measure the position (x) and the momentum (p) of a particle with absolute precision. The more accurately we know one of these values, the less accurately we know the other."--Alok Jha
PRO instruments are anything but standard. A loose classification by the University of Oxford Patient Reported Outcomes Group identifies a variety of instruments, for example, disease-specific, population-specific, dimension specific, generic (SF-36), individualized, summary items, utility measures and so on.
Limitations include but are not limited to the lack of application or comparison to general population, few response categories, broad focus, lack of application to real world, many developed in the 90s or older, pre-dating the digital landscape and advanced knowledge of cognitive frameworks, and many other reasons to be wary of the generalized approach to gathering subjective data.
The way I prefer to develop patient perspective includes both validated PRO instruments--with aforementioned cautions--and actual patient data reflective of actual patient experiences--reported by patients. This can be complicated from an analyses point of view due to the large contribution of free-text and multi-modal questioning but when over-generalized standardized questions are augmented with ranking questions for example you can actually create a less linear perspective on how patients value inputs and identify missing measures or considerations. Think cinematic, not linear.
An additional limitation can be attributed to low rigor of survey questions, choice architecture, limited disease state knowledge in question design, satisficing, and improper analyses. The rise of "do-it yourself" survey instruments has unwittingly polluted the data landscape with worthless noisy and low-value data.
Look no farther than personalized medicine to see where we are losing the trees through the forest. We attribute differences in microbiology and biochemical profiles to meaningful clinical outcomes that for the most part, have not materialized.
An additional limitation can be attributed to low rigor of survey questions, choice architecture, limited disease state knowledge in question design, satisficing, and improper analyses. The rise of "do-it yourself" survey instruments has unwittingly polluted the data landscape with worthless noisy and low-value data.
Look no farther than personalized medicine to see where we are losing the trees through the forest. We attribute differences in microbiology and biochemical profiles to meaningful clinical outcomes that for the most part, have not materialized.
“Research is formalized curiosity. It is poking and prying with a purpose. ” -Zora Neale Hurston
Here is a pre-proof short little book I wrote several years ago about survey writing. It isn't meant to make you an expert but it is meant to instantly improve your survey development and the quality of the data you are collecting. I was an invited reviewer of an academic text, The Palgrave Handbook of Survey Research and distilled the most important but often overlooked habits of good survey design.
| write_better_surveys._period-shared.pdf |
We are launching our new data portal where data literacy is embedded with storytelling, analyses, and visualization--Datamonger.health--take a look.