I look at a lot of data. Likely not the first time that you heard or read this about me. I have also been writing a book about Improving Numeracy in Medicine--also something you probably knew.
This podcast presents an interesting perspective and potential solution for how we provide citations in qualitative (and I would argue applicable to quantitative research as well) research.
How should evidence be presented? What about studies we don't include in our analyses--are they part of our historical archive? There is so much useful insight in this lecture that I can't recommend it enough.
One of the biggest shifts in analyses--at least in my small professional world--is consideration of qualitative research. Medical narrative, notes, and textual EHR data contains relevant historical and patient-level insights in clinical decision making at the point of care.
Andrew Moravcsik is Professor of Politics and International Affairs, and Director of the European Union Program in the Department of Politics and Woodrow Wilson School at Princeton University is the speaker in the London School of Economics public lecture. This overview of qualitative data transparency is a timely topic applicable to developing optimized processes for both textual and numerical data.
What is research transparency? The requirement that emperical researchers disclose how they are reaching their conclusions. Not the same as replicability but unique to social sciences with a goal to debate the conclusions. You may not reach the same conclusion but the thread followed to reach conclusions is visible. The goal for qualitative research isn't to replicate the findings. Obviously scientists adhere to the scientific method but the focus in strictly quantitative methods is on reproducability.
Can transparency be improved in social sciences? Definition refined to create ethical norms--3 dimensions of research transparency:
1. data transparency--think of data as evidence. You have access to the data used to reach research claim.
2. analytic transparency--the link between data and causal or descriptive inferences made. Explicate the link by using annotated footnote or quantitative algorithm.
3. procedural transparency--processes used to process, generate, or choose an analytic method. Why was this data selected and not other datasets?
Quantitative research has a traditional history of data sharing but what does this mean in qualitative research? Quantitiative standards are not wholly applicable to qualitative scholars. Epistemologic structure is narrative and involves storytelling as arrays vs. datasets that are analyzed in their entirety in quantitative reseearch. We look at textual data vs. statistical. DIscursive footnotes explain a link to a specific part of data.
Research communities limit the practicality of different levels of transparency. Infinite documentation isn't readily available:
A workable compromise is needed. Conventional citations are often erroneous or point toward non-specific content. Quantitative research has morphed the standards of article length towards shorter lengths for publication. It is impossible to show qualitative sources in that amount of length.
Formats like (author, year) don't suit a qualitative world for data transparency. How do you find the actual source? Too many to locate or point to entire chapters of a book, and many are just erroneous. Many citations in political science and I would argue this is similar to clinical science as well it isn't obvious why a citation supports a specfic claim.
How do we archive? At first glance it seems to support a complete expansion of the research process. What is the background number of documents? What they cited? Looked at? Copied? Read? You obviously can't archive everything.
A potential solution includes one-click access to data and sources, interpretation of sources--a context for the research question--in expanded appendices. In active citation, the links remain internal to the document vs. hyperlinking outside of the article. I hope this brief introductory summary will encourage you to listen to the full podcast. What is evidence and are we beginning to notice when it is missing?
Thoughtful discussions about content development and outcomes analytics that apply the principles and frameworks of health policy and economics to persistent and perplexing health and health care problems.
Sign up for our newsletter!
Browse the archive...
Thank you for making a donution!
In a world of "evidence-based" medicine I am a bigger fan of practice-based evidence.
Remember the quote by Upton Sinclair...
“It is difficult to get a man to understand something, when his salary depends upon his not understanding it!”