Given the multiplicity of actors involved in an emergency response, it is important that coordination is explicitly considered - the intervention of a single agency cannot be evaluated in isolation from what others are doing, particularly as what may be appropriate from the point of view of a single actor, may not be appropriate from the point of view of the system as a whole.
(Hallam, A. (1998) Evaluating Humanitarian Assistance Programmes in Complex Emergencies, ODI, London)
Coordination is consistently identified as being significant to the performance of the humanitarian community. However, coordination activities are rarely evaluated at the field level. Where coordination is discussed, it is usually in the context of system-wide evaluations that result in extremely broad recommendations, failing to provide specific lessons to draw on for future coordination.
It is generally agreed that the success of coordination depends on how much it improves the effectiveness and efficiency of the response. Unfortunately, it is difficult to identify cause-and-effect for coordination activities, which makes it difficult to evaluate coordination only in those terms.
It is therefore important to develop clear guidance on how to evaluate coordination.
- OECD DAC laid out an expanded set of criteria (www.oecd.org/dataoecd/9/50/2667294.pdf)
- ALNAP (2006) Evaluating humanitarian action using the OECD-DAC criteria provides a good starting point.
In most evaluations, coordination is treated as a subset of broader humanitarian action. Another approach is to evaluate joint activities carried out by NGOs, placing coordination in the wider context.
The evaluation of coordination should reflect the objectives of the coordination mechanism itself. I.e. what did it initially set out to do, did it do it, and were the objectives relevant to improving humanitarian response in the first place? Ideally, an evaluation approach should be decided in consultation with the members of the coordination mechanism, and also with external stakeholders such as national government, UN agencies, and donors.
Appendix 2 includes some possible evaluation approaches. These four approaches range from a logical framework approach with a focus on evaluating the perceived usefulness of outputs and their impact on practice, to modifying the OECD-DAC criteria to accommodate evaluating coordination as one means of achieving a more effective humanitarian response.
The evaluation approach that coordination bodies develop will depend on specific circumstances, including the:
- Type of coordination being evaluated;
- Operational or security constraints;
- Available budget and other support resources;
- In addition to other relevant criteria.