The Problem With Science

This book tells the story of how a cadre of dedicated, iconoclastic scientists raised the awareness of a long recognized preference for publishing positive, eye catching, but irreproducible results to the status of a genuine scientific crisis. Most famously encapsulated in 2005 by John Ioannidis' iconic title: Why Most Published Research Findings are False," awareness of the seriousness of the crisis itself was in full bloom sometime around 2011-2012 when a veritable flood of supporting empirical and methodological work began appearing in the scientific literature detailing both the extent of the crisis and how it could be ameliorated. Perhaps most importantly of all, a number of mass replications of large sets of (a) published psychology experiments (100 in all) by the Open Science Collaboration, (b) preclinical cancer experiments (53) which a large pharmaceutical company considered sufficiently promising to pursue if the original results were reproducible, and (c) 67 similarly promising studies upon which an even larger pharmaceutical company decided to replicate prior to initiating the expense and time consuming developmental process. Shockingly, less than 50% of these 220 study results could be replicated, thereby providing unwelcomed evidence thatIoannidis' projections (and others performed later) were not simply pejorative flights of fantasy but possibly underestimates of the actual crisis at hand. Fortunately a plethora of practical, procedural behaviors accompanied these demonstrations which were quite capable of greatly reducing the prevalence of future irreproducible results. Therefore the primary purpose of this book is to provide guidance to practicing and aspiring scientists regarding how (a) to change the way in which science has historically been both conducted and reported in order to avoid producing false positive, irreproducible results in their own work and (b) ultimately to change those institutional practices (primarily but not exclusively involving the traditional journal publishing process and the academic reward system) that have unwittingly contributed to the present crisis. For what is actually needed is nothing less than a change in the scientific culture itself. A culture which will prioritize conducting research correctly in order to get things right rather than simply getting published. Hopefully this book can make a small contribution to that end"--