“Means, Motive, Opportunity” – ER2

In order to frame this enquiry, let’s begin with a small the exploration of the motivations behind commissioning and performing the evaluations in the first place. Though examples here are from social research, these are easily compared with parallels in any intervention, including investment in the development of the science, technology (and business support facilities and services (for example STFC, 2014:5-7).

Firstly, an important part of the evaluation research is process evaluation (Rossi, 1972:34), used in order to improve on the delivery of the intervention, or – as beautifully listed in an interview with Waverley Care (a Edinburgh charity) CEO – “what we need to stop doing, what we want to keep doing and what we are not doing that we should be doing”. When working along this strand of evaluation, it is crucial that the researcher provides recommendations that can be acted upon. The best way to carry out such evaluation is often to focus on a specific small area of the intervention, for example how does an organisation collect feedback and implement changes reflecting the concerns raised by internal and external customers. Having said that, conclusions and recommendations can often be very general.

In the process evaluation, there is further check on the identifying emerging needs and (geographical, social, economic) individualisation of the delivery of outputs. This is particularly important for social projects (such as the Waverley Care), where there is significant variation across the different locales in which they work. However, this is also important in terms of social and geographical inclusiveness of science and technology investment. Hence, evaluation research in this context can provide important checks on the “fairness” of the intervention whilst it is underway.

Then there is the often missed – but in my opinion very important objective in evaluation – the inward facing component, i.e. the improvement of morale of the people engaged in the programme/intervention/organisation by celebrating their success. It is very important for the staff to appreciate the whole picture, “take a step back” to frame their work within a wider context. This is both a good motivation for future work as well as a huge morale boost as one can see how they personally and as a team are making a significant difference to people’s lives.

Finally, the primary motivation for impact evaluation is (always?), to understand the impact/difference an intervention/organisation is making. Evaluation is often considered important for funding applications, i.e. both assessing the need for the intervention as well as monitoring the delivery of outcomes (to evaluate the VALUE generated).

My research is similarly linked to the need for accountability when spending public money (Nutley, Walter and Davies, 2007:254) and in particular the effectiveness of the investment in natural sciences research (mainly cost benefit analysis), which is currently epitomised in cost benefit analysis, but that is already the topic of the next post…