Research domains
This could apply to funders, institutions, or even output assessment.
Context and considerations
Biases can take different forms in research assessment. The most commonly known form of bias is that of conflicting interests. Most research funders have a Declarations of Interest which are mandatory for reviewers to complete. These declarations ask reviewers to declare any potential interests they may have towards specific applications so that reviewers do not review applications for which they may have conflicting interests. The concept is less frequently applied in institutions (e.g., hiring and promotion), but it is just as relevant in these areas.
Conflict of interests declarations also apply to scientific publications, both for authors to declare any vested interest that could have biased their findings, and from peer-reviewers, whose collaborations could have an impact on their assessment of submitted manuscripts. The International Committee of Medical Journal Editors (ICMJE) Disclosure of Interest form and associated documentation is used by many medical and generalist journals as a standard to define what constitutes a conflicting interest.
Yet, conflicts of interests are generally linked to financial and observable interests such as collaborations, partnership, and personal relationships. Biases can go far beyond these observable elements. Having a process to make biases visible and transparent and to discuss frequent biases in assessment practices could help bring the review process to a deeper, more reflective level.
Challenges and mitigations
Challenge: While some biases may be easy to bring visibility towards (e.g., procedures or methodological preferences, disciplinary biases, theorethical assumption biases, etc.) hidden and unconscious biases are difficult to bring to the surface and they could be challenging to discuss in a review committee.
Mitigation: Individual work for review committees to identify their biases, for example by doing unconscious biases exercises or by following a short training on biases could help raise awareness without bringing the discomfort and fear of being exposed. Pairing individual exercises with an option for the review committee members to discuss their impressions of these exercises with other reviewers could enable a better understanding and a safe approach to making biases more visible, transparent, and surmountable.
Relevant resources and literature
The Harvard Implicit Bias test is increasingly used in research institutions and funders to highlight existing biases and raise awareness https://implicit.harvard.edu/implicit/takeatest.html
“Journals and Funders Confront Implicit Bias in Peer Review” https://www.aaas.org/news/journals-and-funders-confront-implicit-bias-peer-review
“Identifying Implicit Bias in Grant Reviews” https://iaphs.org/identifying-implicit-bias-grant-reviews/
PREreview, a peer review platform to support open preprint reviews, made a video and resources on bias which could be useful in creating more awareness and building tools to identify and declare biases in review processes.
Although not specific to a research assessment process, a paper by Hawwash and colleagues (2018) Perspective: Consideration of Values When Setting Priorities in Nutrition Research: Guidance for Transparency provides a template to make values explicit in priority setting for research funding priorities. The framework proposed in this article could also serve as inspiration for building a table to make bias explicit in assessment processes.