Research domains
This assessment can fit in any domain where individuals or research is being assessed.
Context and considerations
SCOPE is an acronym, where S stands for START with what you value, C for CONTEXT considerations, O for OPTIONS for evaluating, P for PROBE deeply, and E for EVALUATE your evaluation. SCOPE can be used as a guide when changing or introducing assessment processes, or even when assessing whether assessment processes in place are adequate.
START with what you value
Begin evaluation by identifying what evaluators, evaluatees, and other stakeholders, truly value about the entity being evaluated, avoiding being driven solely by external influences or relying exclusively on available data sources to prevent the “Streetlight Effect.”
CONTEXT considerations
Consider the context of your evaluation by focusing on the specifics: identify who you are evaluating (considering entity size and discipline) and why you are conducting the evaluation (i.e., for understanding, benchmarking, monitoring, incentivising behaviours, or rewarding).
OPTIONS for evaluating
When evaluating, explore both quantitative and qualitative options. Exercise caution when using quantities to represent qualities.
PROBE deeply
In your evaluation, consider potential discrimination, gaming of the approach, unintended consequences, and weigh the cost-benefit.
EVALUATE your evaluation
Evaluate your evaluation by assessing whether it achieved its goals, considering both formative and summative aspects. Use SCOPE as a framework for evaluating your evaluation.
The five stages of SCOPE operate under three main principles:
- Evaluate only where necessary. Evaluation is not always the right strategy. When it comes to incentivising behaviours, for example, it may be more fruitful to enable them than to evaluate them.
- Evaluate with the evaluated. Any evaluation should be co-designed and co-interpreted by the communities being evaluated.
- Draw on evaluation expertise. We should apply the same rigour to our evaluations that we apply to our academic research.
Challenges and mitigations
Challenge: Introducing assessments that recognise (and therefore incitivise) desired behaviours can help increase the occurrence of such behaviours amongst evaluated. However, assessments do not always make these behaviours easy, and as a result the behaviours are not upheld.
Mitigation: The best incentive is sometimes to make a behaviour easy and normative, rather than to measure and reward behaviours. Consequently, before undertaking an evaluation, it is important to ask ‘Do we need to evaluate at all?’
Challenge: Even if assessments are built with the best intentions, they can fail to represent the breadth and the values of the evaluated communities.
Mitigation: Including the evaluated and co-designing assessment with them – in particular being intentional about including a breadth of diverse profiles – can help support a more reflective and adequate assessment process.
Challenge: Assessment processes sometimes follow traditions without aiming to fulfil a specific purpose.
Mitigation: Producing a simple evaluation statement can be a helpful start, for example “We wish to evaluate X (an entity - e.g., researchers) for Y (a purpose - e.g., recruitment)” or “We wish to Y (a purpose - e.g., monitor) Z (a behaviour or quality - e.g., open research)”
Challenge: Sometimes assessments generate consequences that were not intended and that go against the purpose of the assessment.
Mitigation: Probing for unintended consequences while designing the assessment can help avoid or predict possible unintended consequences to monitor. The use/misuse/abuse/stress case matrix developed by Eric Meyer and Sarah Wachter-Boechtter in Meyer, E. & S. Wachter-Boettcher. (2016) Design for Real Life. A Book Apart
Evaluating success
The final stage of SCOPE is to run the evaluation, and then evaluate your evaluation. This is particularly important as often unintended consequences do not come to light until after the evaluation has been performed. You may wish to ask:
- What VALUE did you get from the evaluation?
- In what CONTEXTS might you evaluate your evaluation?
- What are your OPTIONS for evaluating your evaluation?
- Can you PROBE the evaluation outcomes to identify any unintended consequences or discriminatory effects?
Further details and guiding questions are available in the full guide to the SCOPE Framework
Relevant resources and literature
https://inorms.net/scope-framework-for-research-evaluation/
Templates from funders and institutions
Case examples and literature
Other resources
A short video introducing SCOPE can be seen at https://vimeo.com/397209608/a9bf413c32.
Hogan, S. (2025). Assessment reform advocate seeks shift from metrics ‘overuse’, Research Professional (20 February). [Discusses the use of SCOPE as a value-led alternative to metrics overuse].
Himanen, L., Conte, E., Gauffriau, M., Strøm, T., Wolf, B., & Gadd, E. (2024). The SCOPE framework – implementing ideals of responsible research assessment. F1000Research, 12(1241). https://doi.org/10.12688/f1000research.140810.2
Himanen, L. & Gadd. E. (2019). Introducing SCOPE – a process for evaluating responsibly. The Bibliomagician. https://thebibliomagician.wordpress.com/2019/12/11/introducing-scope-aprocess-for-evaluating-responsibly/
Case Studies or Implementation Examples
Many case studies of organisations and groups that used the SCOPE framework are available at https://inorms.net/scope-framework-for-research-evaluation/.