I am mesmorized by research practices--both quantitative and qualitative. An artifact of the transition to EHR seems to be an expansion of interest in analyzing textual data.
What we lack is a complete understanding of where the disciplines of qualitative and quantitative diverge. Here is a conversation that I hope will encourage better questions and higher quality research for both.
This podcast presents an interesting perspective and potential solution for how we provide citations in qualitative (and I would argue applicable to quantitative research as well) research.
How should evidence be presented? What about studies we don't include in our analyses--are they part of our historical archive? There is so much useful insight in this lecture that I can't recommend it enough.
Andrew Moravcsik is Professor of Politics and International Affairs, and Director of the European Union Program in the Department of Politics and Woodrow Wilson School at Princeton University is the speaker in the London School of Economics public lecture. This overview of qualitative data transparency is a timely topic applicable to developing optimized processes for both textual and numerical data.
What is research transparency? The requirement that emperical researchers disclose how they are reaching their conclusions. Not the same as replicability but unique to social sciences with a goal to debate the conclusions. You may not reach the same conclusion but the thread followed to reach conclusions is visible. The goal for qualitative research isn't to replicate the findings. Obviously scientists adhere to the scientific method but the focus in strictly quantitative methods is on reproducability.
Can transparency be improved in social sciences? Definition refined to create ethical norms--3 dimensions of research transparency:
1. data transparency--think of data as evidence. You have access to the data used to reach research claim.
2. analytic transparency--the link between data and causal or descriptive inferences made. Explicate the link by using annotated footnote or quantitative algorithm.
3. procedural transparency--processes used to process, generate, or choose an analytic method. Why was this data selected and not other datasets?
Quantitative research has a traditional history of data sharing but what does this mean in qualitative research? Quantitiative standards are not wholly applicable to qualitative scholars. Epistemologic structure is narrative and involves storytelling as arrays vs. datasets that are analyzed in their entirety in quantitative reseearch. We look at textual data vs. statistical. DIscursive footnotes explain a link to a specific part of data.
Research communities limit the practicality of different levels of transparency. Infinite documentation isn't readily available:
- You are limited by ethics (confidentiality)
- Book citations are also limited--intellectual copyright law.
- Logistics--do you cite ALL documents you look at?
- RIghts of first use--if you list your entire dataset how do you guarantee you won't be scooped
- Individual journal laws
Formats like (author, year) don't suit a qualitative world for data transparency. How do you find the actual source? Too many to locate or point to entire chapters of a book, and many are just erroneous. Many citations in political science and I would argue this is similar to clinical science as well it isn't obvious why a citation supports a specfic claim.
How do we archive? At first glance it seems to support a complete expansion of the research process. What is the background number of documents? What they cited? Looked at? Copied? Read? You obviously can't archive everything.
A potential solution includes one-click access to data and sources, interpretation of sources--a context for the research question--in expanded appendices. In active citation, the links remain internal to the document vs. hyperlinking outside of the article. I hope this brief introductory summary will encourage you to listen to the full podcast. What is evidence and are we beginning to notice when it is missing?
Thoughtful discussions about content development and outcomes analytics that apply the principles and frameworks of health policy and economics to persistent and perplexing health and health care problems.