About the 2021 Conference theme: Evaluation and Evaluative Thinking
Evaluation has increasingly become a specialist field, with notable methodological developments in both quantitative and qualitative approaches. However, the area of evaluative thinking has also emerged as important.
Definitions vary, for example, in the work of Thomas Archibald it is embedded in the idea of critical thinking. However, our keynote Thomas Schwandt suggests that while Evaluative Thinking might be understood as an individual undertaking that requires cognitive capacities for analytical and critical thinking along with a set of intellectual dispositions, evaluative thinking can also be depicted as a collaborative social practice, which this Conference would like to emphasise. Evaluative thinking can be applied to both formal evaluations and informal evaluative efforts, includes an ongoing process of reflection, and tends to blend monitoring and evaluation. It also lends itself to a critical appraisal of evidence of social practice, policy and the big global issues of our time, including the way evidence is now seen as something that can be alternative or ‘fake’ if it does not correspond to particular political positions or outlooks.
We might access resources from a wide range of disciplines in order to engage in evaluative thinking. So, to understand evaluative thinking as embedded practice means that it should operate less as a separate and independent function but more as an integrated part of a wider set of concerns. This idea does have implications for everyday practice. For example, the Education Department in New South Wales, Australia, has expressed the idea as a series of practices and it argues that to think evaluatively is to engage in the following:
- suspending judgement, considering alternative explanations and allowing new evidence to change our mind
- questioning assumptions, particularly about the pathway of cause and effect
- selecting and developing solutions that are informed by a strong evidence base and are responsive to our context and priorities
- valuing the lessons we can learn from all our experiences, disappointments as well as triumphs
- wrestle with questions of impact and effectiveness, not just activity and implementation
- maximise the value of existing data sources already available to us, mindful of their limitations
- work to improve the strength of our evidence base as we go.
To think evaluatively then can be helped by a culture which encourages reflection, collaborative ways of working and where things might be tried out without the automatic risk of blame.
This means that evaluative and reflective practice should be part of ‘the way we do things around here’ where all colleagues seek, learn and think critically about the evidence that underpins their actions. Embedding this way of thinking rarely happens by accident. It implies a shift away from seeing evaluation as a necessary evil or a ritualistic necessity, to one in which attributing value to an intervention or programme of some kind is built in as a core dimension of good practice.
We might understand the attempt to cultivate evaluative thinking as ‘nurturing’ practices which support and encourage it. The idea of nurturing is important in this respect because it suggests how some organisational environments can be much more conducive to evaluative thinking than others. The idea of a ‘learning organisation’ is a useful way of capturing these characteristics. So, in promoting what ‘evaluative thinking’ offers, we can point to some positives:
- it helps to derive value from deep understanding and diverse perspectives, potentially able to maximise what people can offer an organisation
- it helps to reduce risk of managerial over-intervention because it encourages deeper understanding and transparency from diverse perspectives
- it encourages the use of collaborative capability and critical questioning to inform decision-making.
The kinds of collective questions we might engage in while adopting this perspective include:
- Do you really understand why it is the way it is?
- Do you understand how to improve it?
- Why do you think that particular group responded in that way?
- How can we involve particular stakeholders in examining why things are done in that way?
- On what basis can we communicate and devolve responsibility better?
- What kind of knowledge resources might we need to help us improve?
How might we set up ways of collecting evidence, analysing and using it which can address these kinds of questions and help us attribute value to what we do?
This way of thinking strongly implies ‘using’ evaluations to guide future actions and ‘next steps’ and is particularly relevant to working in complex environments. In day-to-day terms, it means that people leading specific initiatives need to emphasise the expectation that an evaluative dimension is required at an early design stage.
By focusing on evaluative thinking in the Conference theme, we wish to signal that the Conference includes a broad scope of contributions, including from those working in a range of fields where systematic inquiry and evaluative judgement inform learning and decisions, even if these are not typically seen as evaluations. Learning cultures, capabilities for evaluative thinking and embedding evaluation and evaluative thinking in implementation processes are all suitable areas to explore in the Conference.
Whether you are new to evaluation or very experienced, from academia, government, consultancy or civil society, you will have a lot to share, learn and debate at this Conference of the leading professional organisation for evaluation in the UK. Register now!