Can We Test for Bias in Scientific Peer-Review?

Abstract

Science rests upon the reliability of peer review. This paper suggests a way to test for bias. It is able to avoid the fallacy -- one seen in the popular press and the research literature -- that to measure discrimination it is sufficient to study averages within two populations. The paper’s contribution is primarily methodological, but I apply it, as an illustration, to data from the field of economics. No scientific bias or favoritism is found (although the Journal of Political Economy discriminates against its own Chicago authors). The test’s methodology is applicable in most scholarly disciplines.