Commissionership: What is it? What does it involve?

We are coming from the view that the skill of commissioning evaluations is underplayed, under researched and under supported relative to the overall system of evaluation and broader evaluative practice.  This belies its importance, as how an evaluation / evaluative exercise is commissioned has consequences for its direction, quality of process, outcome, and ultimate usefulness.   Commissioning involves making key strategic, technical, and operational choices about an evaluation at the outset, which determine the key parameters – context, focus, shape, governance, and procurement – within which the evaluation will be undertaken.  Crucially the commissioning process should articulate assumptions, particularly, about institutional / organisational, external context impacting the commissioning agency, as well as the context of what is to be evaluated, and how the information sought will be used and by whom.

These choices also impact the type and quality of the professional relationship with the stakeholders involved, and most particularly with the evaluator / evaluation team.  Indeed, the role of commissioner of an evaluation is arguably the most powerful position in the whole evaluation process, as the role signs off on the terms of reference, and is a key player in determining what happens to the information generated by the evaluation, thereafter.

The commissioning authority may be a separate entity from the evaluation manager, or it may be combined in one and the same role and person.  This difference in itself needs to be explicit and its implications for the evaluation made clear and understood by all parties.

To highlight the importance of what we are exploring here, we are giving it a name: ‘commissionership’.    In much current evaluation guidance, ‘commissionership,’ as we understand it, is not separately identified, but is commonly fused within sections on evaluation management.  However, evaluation management is most often presented as a set of tasks which an evaluation manager must undertake to establish and implement an evaluation.  It is our contention that ‘commissionership’ involves more than the delivery of the sum of the parts of step by step implementation; it requires a strategic and relatively sophisticated understanding and co-ordination of context, vision, technical knowledge, governance, costs, selection, and delivery, all with regard to optimizing the use of information generated by the evaluation.  The real value and critical bearing of robust, ‘Commissionership,’ gets lost in the mesh of detailed evaluation implementation.  Yet it is a pre-implementation part of the overall evaluation exercise process which merits much closer attention than it currently receives, which needs greater articulation and greater recognition as a discrete and monitorable stage in the overall evaluation process, and which should be recognised as a key perspective and role with knowledge and skills, in its own right.

Aspects in the scope of ‘Commissionership’ include:

1.  Identification of who decides what to evaluate, and its implications

2.  The Commissioner’s context for seeking an evaluation – making explicit institutional/organisational imperatives and pressures l eg level of urgency, pressure to show progress and impact, show profile, inform decision-making; external pressures on the matter to be evaluated

3.  Selection of type of evaluation eg is it commissioned to provide an external assessment to stakeholders, or to provide an assessment in conjunction with stakeholders, or to facilitate stakeholders themselves to assess? eg is an impact, process, value for money or other type of evaluation required?

4.  Decision on whom should undertake the evaluation exercise eg Internal led, external evaluator/s, or a mix?

5.  Selection of analytical framework and methods, to ensure the exercise can provide the information sought.

6.  Theory of change – understanding how this will be used in the evaluation. Or if one should be reconstructed, if none already exists?

7.  Leading on the drafting of the Terms of Reference including:

• Content/subject
• Context and purpose of the evaluation eg. internal organisational drivers for the evaluation, external pressures on the policy/issue/work context, why the evaluation is needed, what information is sought, for whom, and by when
• Scope and focus of the evaluation g., what is to be evaluated, timings, key context parameters, flexibility in the scope, data required and availability, evaluation questions
• Deliverables including what is required, when, in what formats, and length and presentation style
  • Scheduling (e.g., overall duration, intervals for review) and consequences of key delays
  • Governance
    • Expectations of roles, inputs, and communications between all parties, and external communications
    • Scrutiny process both during and for the final deliverable, and sign off for the contract
    • Ethical issues related to the evaluation and how they will be handled, including data management and storage
    • Substantive change in demand/direction/scope during the evaluation
    • Procurement of Evaluation Team/candidate
      • Skills & competencies sought from the evaluator/s,
      • team/candidate selection criteria & interview process
      • Budget and costs , what is covered
      • Safeguarding issues and insurance issues in conflict areas
      • Risks incl., how to change team members if not performing

8.  Use of information generated by evaluation

• Dissemination process, to whom, in what form and timings, who undertakes which aspects
• Integration in the appropriate decision/policy-making process
• Other ways of using evaluation to make a difference

Useful articles

Jayne Cox et al, Cecan publications

https://www.cecan.ac.uk/wp-content/uploads/2020/09/Jayne-Cox-Report-final.pdf

To explore ‘Commissionership’ further, and drawing on our own experience, The Society is piloting ‘Community of Practice’ sessions for UK Evaluation Society members, from January 2023 onwards.

Do join!  Come, share your own experience and build on the experience and insights of others.

The first 3 sessions are already listed!

Commissionership Community of Practice 1

Frustrations of a Commissioner; Frustrations of an Evaluator – Towards better practice

12 January 2023 1300- 1400 GMT Zoom online

Commissionership Community of Practice 2

The purpose and audience for an evaluation – Towards better practice

16 February 1300 – 1400 GMT Zoom online

Commissionership Community of Practice 3

Governing for change during an evaluation – Towards better practice

23 March 1300 -1400 GMT Zoom online

Members only

If you would like to join the ad hoc group of members who are preparing the sessions contact hello@evaluation.org.uk attn Commissionership Group.

UK Evaluation Society ad hoc ‘Commissionership’ Group

November 2022