New working paper out!— this one has been in the making for a while (with Luigi Zingales, Tommaso Valletti, John Barrios, Shashank Singh and Joshua Levy).
In “The Conflict-of-Interest Discount in the Marketplace of Idea,” we study how different forms of conflicts of interest—such as money, career, data access, reputation, and ideology—impact trust in economic articles and science more broadly.
We use randomized vignettes to elicit how selected economists (from the NBER and CEPR), other academic economists and a general sample of the US population (through a YouGov poll) increase their trust in or discount the results of different academic articles once they are exposed to disclosures that reveal different forms of conflict of interest impacting that given paper.
Our results show that conflicts of interest—such as receiving money for work as an expert witness or being granted restricted access to databases by companies interested in the results of a study—have a significant negative impact on the trustworthiness of the academic work by the same scholars. By some measures, a conflicted paper is worth a third or even a quarter of a paper by non-conflicted scholars. However, there is a lot of variation: using public access data increases trust in papers, while ideology and even reputational concerns impact trustworthiness less significantly. We have many more results broken down by expertise, ideology, career affiliation, etc. The overall message, though, is consistent: conflicts of interest negatively impact trust in academia.
Our results are timely. Trust in science has been declining, and authorities have been increasingly criticizing the breakdown between academic independence and pure advocacy (as per recent speeches/comments by Jonathan Kanter and Andreas Mundt ).
As our paper highlights, one of the main problems is the overall negative externality generated by conflicts of interest: researchers internalize the benefits (money, data), but no one internalizes the costs—e.g. the articles never written because they antagonize companies that control the necessary databases. The overall distrust in the current disclosure system doesn’t help either: when we asked economists whether they adequately disclose their conflicts, 95% said “yes” or “mostly”. When we asked whether their peers had done so, most said they did not trust their disclosures.
This negative externality is mainly born by the vast majority of scholars performing serious, independent work (and, particularly young scholars).
We hope our results will help advance this vital conversation by providing a technical baseline for discussions on how external influences can impact academic research.
Please take a look, share it around, and send us comments! This is very much a work in progress.
https://lnkd.in/eGb29DyH
Australian Centre for Evaluation, Department of Treasury. PhD candidate at University of Queensland
2wThis is such an interesting paper but also a call to arms for more timely evaluation. Confirms suspicions that evaluation timing is often not well aligned with budget/investment decision making