Between Learning and Legitimation
The purposes of evaluation are to monitor and control policies, to identify possibilities for rationalization, to constantly improve internal processes, and to provide a detailed understanding of the potential effects of future projects.
This sort of list may appear to be the result of a somewhat disorganized, semi-scientific brainstorming session that, by enumerating the various functions of evaluation, suggests they are perfectly compatible with one another and can be put into practice simultaneously.
The assumption is that effective evaluation can serve all these purposes at once: the monitoring and control of policies by the management, the production of legitimacy in the eyes of clients, the production of learning effects, and the assessment of future projects. But a brief look at just two of the central functions of evaluation – learning and legitimacy – shows that this assumption cannot be correct.
Its role in allowing organizations and their members to ‘learn’ consistently comes out on top in lists of the functions of evaluation. Learning implies structures being altered: whether the structures of human consciousness, the structures facilitating the processing of experience, or the structures of organizations. Learning is often triggered by an impulse that makes it more difficult to maintain existing structures in their present forms. When mistakes in a language test are corrected, this creates an incentive to avoid making these mistakes in future. When a boss receives a negative assessment in a 360-degree feedback process, this makes it harder for him or her to maintain the leadership regime in its current form – at least this is the theory.
The second function of evaluations is the production of legitimacy. It is a characteristic trait of modern organizations that they seek to gain acceptance. Special programmes, positions, or even whole departments are created with the sole purpose of securing legitimacy, yet the efforts that go into the production of legitimacy have to be repeated anew with almost every decision. Reference is made to secure employment and gender equality. Comprehensive investment calculations are produced in order to justify each project. Staff are allowed to participate in new initiatives in order to ensure that decisions made have a ‘broad basis’.
These two functions differ significantly with regard to the extent to which they can be communicated within an organization. Learning through evaluation is – to use Robert Merton’s phrase – a ‘manifest’, a visible, function. Manifest functions can freely be used as arguments. It is possible to appeal to the learning function in seeking more resources to carry out evaluations, and to invoke the necessity of this function at meetings without being frowned at. By contrast, the function of providing legitimacy is, in Merton’s terms, latent. In this context, ‘latency’ does not imply that this is an unconscious process, as in Sigmund Freud’s psychology, for example; rather, it refers to the fact that it is difficult to name this function openly. You cannot easily address the aim of securing legitimacy in meetings with colleagues from different departments. And a human resources department is not likely to be granted funding to undertake an evaluation if the only justification for it is the aim of legitimizing the department’s own work.
From a sociological perspective, there is no reason to consider the manifest function, learning, more important than the latent function, providing legitimacy. We know from empirical work in organization studies that organizations, or units within organizations, cannot survive solely by achieving a high degree of technical rationality through ongoing learning. Their survival also depends on them achieving a high level of legitimacy within their environment. Producing information that demonstrates that one’s own actions are meaningful is as important for an organization as rationalizing its internal processes.
The difficulty with evaluations results from the fact that the two functions – learning and legitimacy – cannot be optimized at the same time. To put it bluntly, attempting to use evaluations in order to legitimize certain policies obstructs learning processes within the organization. And, by the same token, attempting to use evaluations to initiate learning processes often makes it harder to secure legitimacy for proposed policies.
Let us first look at the first claim. The degree of legitimacy provided by a cost-benefit analysis demonstrating that the implementation of a particular measure will lead to cost savings of 200,000 euros, or a return on investment of 541%, is no doubt very high, but at the same time such forms of evaluation do not create many possibilities for learning. What can a team of advisors learn from the fact that a measure has ‘only’ led to savings of 150,000 euros, rather than 200,000 euros? Setting aside the degree of uncertainty involved in such calculations, what can we learn from the fact that a measure will yield a return on investment of 487%, rather than 541%?
But there is something else to this that is even more problematic. The pressure to secure legitimacy leads to weaknesses in learning processes. As a result of this pressure, mistakes are not flagged as mistakes, fake successes are presented as real successes, and every suggestion for ways in which a measure might be improved carry a disclaimer to the effect that, of course, the measure has already been a great success overall. In seeking to produce legitimacy, it is not as though the members of an organization consciously decide to renounce learning for the sake of legitimacy. Rather, the pressure of legitimation is such that the data and figures originally marshalled in support of a claim of legitimacy take on a life of their own. By letting off some rhetorical fireworks at the end of a business seminar, the coach leading it may engineer a brilliant satisfaction rating of 1,3. But the focus on this final grade may mean the many minor weaknesses in the seminar go unnoticed. When that 751% return on investment figure is actually being calculated, there may well be some reflection on exactly how it is being arrived at, but then, as it later gets repeated, the figure takes on a life of its own.
However, we can also look at the problem of evaluation from the opposite perspective: any attempt at learning always entails the risk of losing legitimacy. Openly addressing a mistake may gain you the respect of others, but by the time knowledge of this mistake spreads uncontrollably across the organization this creates a justification problem. You may talk openly about a flaw in a measure you have implemented with the hope of learning from your mistakes, but this also risks your professionalism becoming subject to doubt.
As I have presented it, the evaluation dilemma – the need to produce legitimacy versus the need to produce learning opportunities – relates to a more general tension between rational decision-making and ‘action rationality’, as identified by Nils Brunsson. On the one hand, organizations depend on rational decision-making, and thus on ongoing learning processes. As many alternatives as possible must be considered, and proper thought must be given to their possible consequences and unwanted side effects. On the other hand, organizations must also act rationally; that is, individuals must be willing to take particular actions and to work in a particular direction. We know that people are more likely to be motivated to act when only a few actors are involved in a decision, when only the positive consequences of a decision are considered, and when the favoured solution is compared only to obviously unsuitable alternatives.
But – and this is Brunsson’s most crucial claim – decision-making can only be made more rational at the expense of action rationality, and vice versa. If rational decision-making means considering as many alternatives as possible, scrutinizing the consequences of all alternatives, and involving as many people as possible in decision-making, then, as Brunsson convincingly argues, the rationality of decision-making comes at the expense of the motivation for action. If there are so many alternatives, why should we opt for one of them in particular? But the opposite is also true: a decision that is action rational – involving few actors, considering a limited number of alternatives, and, if possible, avoiding the analysis of potential problems – is less rational as a decision.
