The OECD’s stand of “What is Impact Assessment” [Mar/2022]
Impact assessment is a central tool for policymakers to provide information on the outcomes of policy and regulation. At the end of 2021, the European Commission reconfirmed the importance of the tool by providing an updated version of its framework standardizing impact assessment approaches (see here). But what actually is impact assessment? The OECD tries to answer this question by breaking down the nature of impact assessment and its real-world policy application in its publication “What is impact assessment?”.
The core of impact assessment is found to link causation from an intervention to an outcome while measuring the induced changes along the way. For this, the focus must be on a narrow and well-defined set of impacts to analyse, to allow for detailed effects in a clearly delineated framework. Different than in an evaluation, impact assessment does not aim to give an all-inclusive view or answer policy questions of the bigger picture but instead highlight one set of potential effects in rather isolation. In this exercise, it is especially important to avoid misinterpretation. While one set of effects might be highlighted extensively by an impact assessment, it can come at the cost of unknowingly neglecting other factors. Therefore, the OECD suggests experimenting with the assumptions included in the exercise. Moreover, individual context between observers is likely to vary enough to make “best practice” conclusions almost impossible to establish. The unique role of people and institutions need to be considered on a case-by-case basis. However, this also presents the opportunity to solve issues in different ways without the limitation of only one perfect solution.
Two common applications of impact assessment in public policy are defined:
- Ex-Ante requires policy cycle analysis beforehand to inform policy makers of future, expected effects of current interventions
- Ex-Post evaluates the previous policy cycle and tries to understand to what extend an intervention managed to address the issue it was intended to address.
However, the OECD raises the question whether our general interpretation of impact might not be in line with its actual application anymore. Given the assumed linear relationship between research and society (in which changes in research are the cause for changes in society), it makes sense to anticipate that impact assessment causes the desired outcome. However, the paper suggests reverse causality between the two factors and a range of additional unaccounted influences that are neglected in the current models. One should acknowledge that “the mechanisms of impact are likely to be diverse; certainly one generic story about how impact occurs will not fit all circumstances”.
The full document with detailed information can be found here.
For the OECD follow-up publication “What is Impact Assessment? Mechanisms”, please click here.