This draft contains potential questions for Dr Ben Goldacre. Have something you’d like to ask? Add your question below.
It was a not so sweet Valentine’s Day treat for five medical journals as a paper from University of Oxford’s Evidence-Based Medicine Data Lab was published on Thursday.
Discrepancies between pre-specified and reported outcomes are an important source of bias in trials. Despite legislation, guidelines and public commitments on correct reporting from journals, outcome misreporting continues to be prevalent.
The paper offers an exposé of the impotence of CONSORT, Consolidated Standards of Reporting Trials. The first CONSORT rules were published in 2001 and to date 585 journals endorse the 2010 version of the standards. The rules were intended to make trials more scientifically sound, but despite endorsing these rules, more than three-quarters of papers published in the 5 journals over a period of 6 weeks violated these rules.
Moreover, when the researchers sent letters to the journals, not all of them were published; and in fact the New England Journal of Medicine and Journal of the American Medical Association declined to publish a single correction letter.
Potential Interview Questions
- You started the EMBDataLab in 2015 at university of Oxford; what would you say is your lab’s greatest accomplishment to date?
- Would it be fair to characterise your career trajectory as starting out going after low hanging fruit in bad science like charlatans, and now moving up onto the scientists themselves?
- CONSORT stands for Consolidated Standards of Reporting Trials. Are there situations where these standards might be too restrictive?
- What’s the process like for getting papers retracted / amended by third parties, i.e. is there a formal process for that (letters) or is it more expected that amendments be done by the authors themselves? Did you notify the authors of the papers themselves at all? (Or would that have ruined the study?)
- Why did it take a year and a half for your new comPARE paper to get published?
- JAMA and NEJM claimed they didn’t have the space to publish all responses; what’s your response to that?
- In the past you’ve complained about academic publication model for medical trials. What do you think should replace it? Who should be responsible for publishing the trials?
- On your website you say a “pre-specified outcome is much less likely to give a false-positive result.” – why is this? Couldn’t it be the case that we should just let algorithms find the relevant correlations anyway?
- The main outcome you examined in this paper was outcome switching, are there any other things that you’d like to measure besides that?