Conventional M&E methods are not very suited for dealing with complex interventions such as advocacy. Causal pathways often cannot be fully known prior to an intervention, that what needs to be measured may not be known prior to an intervention, and processes shaping a programme and its results are not stable. Recently, M&E methods have been developed that seek to engage with complexity, including Complexity-responsive Evaluation, Complexity-Aware Monitoring, Developmental Evaluation, Contribution Analysis, Outcome Mapping and Outcome Harvesting. Nevertheless, they have certain limitations when it comes to monitoring and evaluation of advocacy as they do not address four key issues:

Measurement and evidence

Current M&E methods focus on the measurement of achievements, often using indicators to classify these. Outcome indicators used for advocacy are often quantitative in nature, such as 'number of elected officials who publicly support the campaign'. Also, qualitative indicators are employed, including e.g., strengthened organizational capacity, or uptake of certain arguments in political debate. To a degree, such indicators can be used to ‘measure’ achievements of advocacy programmes. However, rather intangible outcomes such as increased credibility of the organization’s positions in the eyes of policymakers, or increased support for a certain policy position among policymakers are hard to measure. At the same time, these are crucial achievements.

In addition, these methods do not critically reflect on the issues of evidence. They easily gloss over the fact that when it comes to advocacy, evidence of outcomes is often not available or accessible, because they are intangible, invisible, or politically sensitive. Some methods that focus on evidence-gathering, (contribution analysis and process tracing being the most prominent ones), are appealing in their sensitivity to complex contexts and processes. At the same time, implementation of such methods, especially when it comes to complex programmes involving many actors (as often found in advocacy for development) is highly resource-intensive, while not able to resolve key mentioned challenges of evidence availability and accessibility that would often be confronted.

The meaning of achievements and failures

While advocacy achievements may be measured in some cases or to some degree, they need to be interpreted to make sense. For example, a number of elected officials publicly supporting a campaign does not tell us whether the campaign has actually changed their viewpoints. Qualitative tools measuring change do exist, including e.g. Most Significant Change, Outcome Harvesting, and Outcome Mapping. However, while such approaches can bring out results in qualitative terms, they still pay little attention to the question of what an outcome means in light of the Theory of Change of a programme, and in the context in which the outcome is achieved. As mentioned earlier: advocacy results are often steps in the direction of impact, rather than results that have a direct impact on constituencies. The nature and significance of an advocacy outcome can therefore often only be interpreted in light of the hypothesized pathways of change and desired outcomes that are described in a Theory of Change. Furthermore, current tools are highly intervention-centred, paying little attention to the context in which a particular intervention is to make a difference and how contextual factors influence and interact with intervention activities. In advocacy, an organization's intervention will be but one factor in a complex process involving many actors and factors. Current tools thereby tend to provide a rather decontextualized presentation of a programme and its results when it comes to advocacy. What a programme's achievements mean in the broader context in which it operates remains hard to say.

By focusing on outcomes first and foremost, existing methods provide little insight into the actual work of advocates. The judgments and analyses behind strategies contributing to achievements and failures remain black-boxed. As a consequence, they cannot support decision-making and strategizing about the next advocacy steps: what went into the mix of efforts, knowledge, relations and skills to achieve outcomes, what dilemmas and challenges came up, and how these were confronted and eventually solved (or not). Existing methods, therefore, offer little opportunity for the identification of lessons to strengthen future advocacy theories, approaches, and strategies.

Communication

Existing M&E methods for advocacy are mostly focused on producing reports for internal and external accountability purposes. There are two basic limitations with this, constricting the usefulness of such reports for communication.

Internal communication in organizations and networks
Existing methods do little for internal communication about advocacy work within organizations and networks. Reports that focus on what comes out of the ‘black box’ of advocacy do not convey the nature and significance of advocacy work and the knowledge and skills involved. In addition, current M&E methods ignore the fact that advocacy is not done by one organization on its own, but engages multiple actors. They do little to bring together stakeholders (one’s organization, allies, and partners), to interpret the significance of the advocacy work and its link with its achievements. They do not address questions of interpretation of achievements, and failures and challenges are not collectively addressed. Collectively obtained experiences and lessons are therefore not used for planning next actions. Unpacking the inside of the ‘black box’ will help organizations to understand and learn about advocacy work. Explaining advocacy and the relation with its results to the internal audiences supports advocates in their own organization. Collective interpretation and sense-making of advocacy work in relation to results has the potential to strengthen coalitions and networks and the impact of their collaboration.

External communication
Commonly used M&E methods do little for external communication. They tend to lead to reports that are technical in nature and therefore inaccessible for audiences beyond a small circle of experts. These reports are not well-suited to convey advocacy achievements, which often are steps towards impact rather than forms of impact. Advocacy results easily remain out of view or meaningless for supporters and wider publics. In addition, lack of impact on constituencies can be mistaken for a lack of significance if the significance of the results is not articulated. For example, achieving an adjustment to a policy document may result from great collective efforts made over many years. However, it should be clarified how that change is or may become helpful to constituencies or society at large. This is not simply because policy processes are technical. Again, the outcomes of advocacy for development are often interim in nature, banking on future policy processes to attain the ultimate legitimacy of positive impacts such as increased access to clean water, food security, or a living wage. In existing M&E methods, this is not considered. We therefore expect that explaining advocacy and the relation with its results through Narrative Assessment stories to supporting donors and the public at large can help build support for past and future action.