Cost benefit analysis is an attractive evaluation method, as it can provide concrete, often quantified, data about interventions, usually in a form which is easily communicated to the clients, policy makers, funders and the general (lay) public. In its core and at its best, cost benefit analysis is a very direct and straightforward evaluation process, whereby inputs and outcomes are weighted against each other and logical conclusions about the efficacy of a programme can be reached.
However, all three of these elements – inputs or costs, outcomes or benefits, and efficacy or the relationship between the two – are highly contestable. To begin with, defining your parameter space and acknowledging constrains and assumptions is the key element of this approach to evaluation. These decisions, even if very well argued for, are ultimately just decisions; a global cost benefit analysis, if such a thing was ever possible, would need to encompass much of the factors and effects left on the other side of the dividing line for the evaluation to be a true representation of the net impact of the programme.
Secondly, even though the aim is to have a quantified data as possible – best if every input and impact are turned in some sort of monetary measure – both costs as well as benefits are often indirect or intangible. In Cellini and Klee’s most stark example (2010, p. 500): what is “the value of wilderness or an increased sense of community”? Furthermore, even if a measure can be put to notions such as wellbeing, another – perhaps most challenging of all – decision has to be made, namely what ratio between costs and benefits defines effectiveness of even efficiency?
However, in my limited experience, cost to benefit analysis is effective if the intervention being evaluated is narrow and well defined in terms of the available resources, the scope and the intended outcomes, or better still, when all of the above have an intrinsic monetary value attached. The intended outcomes I look for in my research are related to innovation and consequently increased economic activity, contributions to GDP, business growth, job creation, etc., hence quantification of these parameters is not very difficult as they often come as monetary values to begin with.
The most challenging for me is to benchmark the efficacy of this cost to benefit ratio and, to be honest, even though it would be to a degree possible to put a judgment on how significant the benefits have to be to deem a programme a success, I prefer to correlate these ratios to background trends such as global economic activity, comparisons to global GDP growth, global business and job creation, and add qualitative data where possible, as I believe the later provides a broader judgment on how the intervention is impacting those in and close to it.
This advanced cost benefit analysis can then feature prominently in a new paradigm of impact evaluation – the Correlated Quantified Impacts (QCI) – the topic of the next post.
Space Innovation: Discovering the Future 