Blog from Sophia Ronketti Commissionership – Community of Practice 2

The UK Evaluation Society is currently holding a pilot, community of practice series on ‘Commissionership – towards better practice’, with sessions being held once a month until June 2023f. The sessions are an opportunity for commissioners and evaluators to discuss their experience and perceptions of issues in the commissioning process.

Each session has a particular focus.  The second community of practice session held on 16^th February focused on the purpose and audience for an evaluation.

The session raised two particular issues, experienced by both evaluators and commissioners, around the purpose and audience of an evaluation. This sparked an insightful discussion amongst participants reflecting on challenges they have experienced and ways in which they have sought to mitigate or overcome these issues.

Issue 1 – An unclear purpose and audience

The first issue raised was on the purpose and audience of an evaluation not being clearly defined or articulated by commissioners. The two most fundamental questions for a successful evaluation are perhaps “what are you going to use this information for” and “who is going to use it?”. These questions should be thought about by commissioners before a Terms of Reference (ToR) is developed, and not left to the evaluator or evaluation team to think through. A lack of clarity in ToRs was also one of the frustrations of commissioning raised in the first community of practice session held in January – you can read the blog from this session here.

Another interesting point raised was around the power imbalance in a commissioner-evaluator relationship. Perhaps, moving towards a more collaborative or partnership-style relationship between commissioner and evaluator, rather than a transactional/provider relationship, could support more dialogue at initial stages around issues/areas which need to be clarified, without the fear, by evaluators, of failing to secure the work.

One potential approach to this could be exploring a partnership type contract involving a more collaborative and shared relationship between commissioner and evaluator.

The importance of clarifying the definition of evaluation is also linked to purpose – a ToR should make it clear whether the work is a management/performance review, an impact assessment, or another type of evaluation. Working through this and making this clear in any ToR will help clarify the purpose of the evaluation project and may support wider organisational buy-in on the learnings from the evaluation, from key audiences.

Pre-tender conversations, either 1 to 1 or in a group, or allowing for questioning between the commissioner and evaluator during the commissioning process may also help clarify the purpose and audience of an evaluation. Open conversations between evaluator and commissioner at the very beginning of any evaluation to clarify and ensure that both parties are on the same page regarding the purpose and audience of an evaluation are key for the evaluation to run smoothly. This supports exploring a more collaborative and open partnership between commissioner and evaluator – and stresses the need for expectations by the commissioner and the evaluator on purpose and audience (and hence outputs) to be agreed, explicitly, up front.

Issue 2 – A changing purpose and audience

The second issue raised was around the purpose and audience of an evaluation changing over the course of the work. This can be a particular challenge for evaluators when the primary purpose of an evaluation changes from ‘learning’ to ‘accountability’ which may then not be feasible depending on the evaluation design and approach being taken.

Often, commissioners are not technical experts in evaluation approaches or even subject matter. This makes the evaluators role to communicate the evaluation approach and methodology initially, or the trade-off if change is needed, even more important – and commissioners listening to this.

It is recognised that change is likely for any evaluation running over multiple years. Particularly the priorities, interests or context may change, influencing the purpose or stakeholders of an evaluation. However, this links back and re-inforces the previous point about ensuring the relationship between commissioner and evaluator is more partnership focused and supports an open dialogue. Doing so, would help to combine the technical expertise of the evaluator with the policy or programme/intervention expertise from the commissioner to shape the evaluation.

Considerations

Two main points to consider by either evaluator or commissioner, from the session are:

1. The purpose and audience of any evaluation should be clarified and agreed at the very beginning of any intervention and is key in influencing the design of the evaluation.
2. Partnership style contracts or relationships between evaluators and commissioners might be a potential way to overcome the power imbalance between parties and support clarity of the purpose and audience of an evaluation.

Getting involved

The next community of practice session is scheduled for 23^rd March 2023 at 1300 to 1400 GMT with a focus on ‘Governing for Change during an evaluation’. To register, please sign up here.

If you are a commissioner, evaluator, or in procurement of evaluation, and would like to join the planning group for the series – please get in touch with hello@evaluation.org.uk Attn: Commissionership.