Implicit criteria weighting and evaluators' backgrounds in grant peer review

Grant peer review (GPR) is a cornerstone of modern research funding, yet the process by which reviewers integrate multiple criteria into a single overall evaluation—known as commensuration—remains poorly understood. We study the implicit weights that reviewers assign to PI qualifications, research quality, and societal relevance, using a unique dataset of 2105 review reports for 586 proposals submitted to the Swedish Foundation for Strategic Research (2011–2017), complemented by manual coding of comments and interviews with screeners and funding officers. Commensuration influences roughly 29% of final scores. The research quality criterion dominates evaluations, receiving about twice the weight of relevance. This commensuration pattern is surprisingly stable across pools of reviewers from different fields and geographic areas and irrespective of the reviewers' profile and their presumed ability to assess relevance. PI qualifications generally play a minor role, tempering concerns about pervasive reputational advantages, and tend to be used more when evaluators feel less competent, hinting that reputation is used as a fallback consideration when the reviewer lacks confidence. Overall, the results suggest that commensuration may systematically contribute to prioritizing technical merit over relevance even beyond the intentions of the funders.