My choice of education or skill development begins with MOOCs like this one Unlock Value in Massive Datasets. Data is the blockbuster drug and if you don't know how to understand how the data was gathered or possess the confidence to challenge spurious claims you will be at a distinct professional disadvantage.
What else do we do in our sphere of influence? We look at numbers either reported in the media, in a clinical abstract or preferrably paper-- or perhaps in freely available public data.
I am going to leave you with a simple checklist for your data. And an invitation to join a few workshops. The workshops will last a few weeks not an hour, will cost practically nothing--especially if you enroll early, and I will summarize the high-level points here before we launch so you can decide if its your cup of tea.
Go check it out...Improving data literacy in medicine and clinical research. Join early for only $10. We will step through a bit of the book--Improving Numeracy in Medicine--address additional issues of scrutinizing evidence with screencasts, video, and audio.
- Do you know who funded the study?
- Have you accessed the clinical study reports?
- Are the results reported in actual clinical endpoints or outcomes? (morbidity, mortality, symptom relief, emotional/physical functioning and health-related quality of life)?
- If surrogate endpoints were used, are they adequately explained?
- Are research questions, outcomes and populations for analyses determined in advance?
- If identified as RCT be certain patients and physicians were not able to select cohorts
- Did authors present confidence intervals inclusive of clinical benefit?
- Sufficient number of participants?
- Random allocation of study subjects to their groups?
- Is there adequate concealment of allocation?
- Demographics the same between groups except for the subject of interest
- Are measurement methods valid and the same between groups? “Validated” may not really be valid.
- Could high discontinuation rates distort the outcomes resulting in under reporting of safety problems or otherwise create a distortion due to such issues as subjects using other interventions?
- Are missing data likely to distort results?
- If appropriate, was analysis done by Intention-to-Treat (all patients evaluated in assigned groups) with missing variables assigned by reasonable methods which will not favor the intervention?
- Were assumptions used for modeling reasonable?
- Was safety assessed and reported?
- Have results been confirmed in other valid studies?
- Are benefits AND harms considered
- Absolute and relative risks reported?
- Effect sizes reported
- NNT or NNH reported as well?