Squaring the project circle and making evaluations matter

The complex environments in which we sail demand that we invest more in evidence-based strategy reflection. What do we learn from our own monitoring data? What does feedback from our stakeholders teach us? But especially: how and when does it feed into our strategic planning?

Recognising the dichotomy between the information needed to propose relevant strategy adjustments and our (donor-inspired) evaluation practice, we have decided to embed our evaluation practice in a more meaningful project management cycle. Time to update the dynamics that underpin our work!

Most project officers, programme managers and executives will have come across the project cycle at some point in their careers. Courses on project cycle management build around the following image and adapt it to the line of business to which they cater:

One cannot refrain from nodding in approval of the seemingly infallible logic this image presents. It goes without saying that one needs to evaluate the project at hand, for the new project cycle to reflect the valuable lessons learnt during the previous one. Also, with so much visual emphasis on the preparation of a project, what could possibly go wrong during its implementation?

Reality, however, for development projects that last 3 years or less (that is, the majority of projects) looks more like this:

Strategic planning takes place at certain intervals, often at turning points within an organisation and sometimes coinciding with the run-up to a new funding cycle of a big donor. These cycles tend to follow one another neatly to avoid funding gaps, with deadlines for submitting funding requests a few months before the previous cycle ends.

Assuming that change does not occur in one funding cycle – as is often the case in development projects that target structural change – soliciting organisations need to identify and formulate new proposals well ahead of the end of their ongoing programmes. Except if something went badly wrong or ground-breaking new dynamics have come to dominate your line of work, this proposal writing does not necessarily involve heavy, strategic planning. Staff on the ground, fired up by their programme managers and fundraising officers, concoct proposals that promise to consolidate the gains of the last project while forecasting even more life-impacting changes in the next one (which, in turn, will need consolidation in the project thereafter).

Meanwhile, evaluations typically take place towards the end of an ongoing project cycle. This timing is especially representative for evaluations that were budgeted as part of the project, for they need to be accounted for within the project’s fiscal boundaries, while at the same time reflecting as much of the anticipated change as possible.

The above-mentioned dynamics drastically interrupt the logic of the project cycle. Not only do organisations identify and formulate new proposals before the evaluation of the ongoing project has even started, funders engage in the appraisal of proposals while evaluations take place. They decide on new funding without having received the evaluation reports on the performance of development organisations. Put differently, without considering evidence on the capacity of organisations to deliver promised change, donors decide on the funding (dis)continuity of these organisations in function of budgetary availability and the political priorities of the day. A case in point are the Dutch and Belgian governments’ financing decisions on the most recent multi-year development cooperation programmes.

When reality cannot be moulded into the model, why not consider reshaping the model based on reality? At Rikolto, we propose to revisit the project cycle putting the different phases into perspective and reshaping the final part of the cycle:

Strategic planning, and the identification and formulation of new proposals should not make up over half of the cycle (even if that was only visually the case). Sure, we need to conduct a proper context analysis that addresses the change dynamics in our context of intervention, as this allows us to identify the entry points in the food system that we can influence. From there, we can design a coherent plan towards the desired change, while building on the good practices and lessons learnt from previous and ongoing projects.

However, organisations with qualified staff on the ground, such as Rikolto, can easily reduce the time spent on this initial phase. The gut feelings of our colleagues in the field, nurtured by years of experience and informal sensors, provide a valuable balance to overly schematic or complex visualisations that illustrate desired pathways of change.

At the core of Rikolto’s ambition to change the recipe of the food system are implementation practices that focus on continuous learning and adaptive planning, essential skills when reality comes waltzing in the change diagrams. It should not surprise that these practices make up the brunt of our project cycle. They drive the iterations (lean build-measure-learn cycles) in pursuit of sustainable solutions that make the agri-food businesses of our partners thrive.

Visually shifting the focus from planning to implementation goes hand in hand with our search for methodologies and practices that foster evidence-based reflection and adaptation. We used to have dozens of check-lists, screening tools and templates for the initial stages of project identification and design but assumed that learning and strategy adjustment would automatically take place when updating intervention frameworks and compiling annual reports.

Data-driven learning dynamics and problem-driven iterative adaptations top nowadays the work agenda of the international PLA team (which gathers all regional and international M&E experts of Rikolto).

Considering that most of what we do takes place in volatile, uncertain, complex and ambiguous contexts, evaluations need to play a double role. They should help us uncover the change dynamics beyond our sphere of influence and understanding the impact of our work (past the outcomes we monitor regularly), in view of improving our change strategies.

In so doing, evaluations are also elementary in being accountable towards the people whose livelihoods we seek to improve, the partners with whom we work, the donors (large and small) that support us financially and, last but not least, ourselves.

All the regional offices and international clusters of Rikolto are currently implementing projects that fall under a five-year strategic progamme (2017-2021). In this cycle, we have decided to put the greatest emphasis on a mid-term evaluation, with data collection taking place at the end of 2019 and data analysis in the first months of 2020.

This timing will allow first of all to have covered sufficient time to be able to detect impact beyond our direct partners – an essential condition for meaningful impact evaluations. Furthermore, we anticipate that a thorough mid-term evaluation at two-thirds of our programme implementation shall yield results that can actually be used to define an appropriate exit strategy.

The last year and a half of the current programme can then focus on key areas that will ensure the sustainability of the accomplished results after the end of the collaboration with our partners. In other cases, it will allow us to timely inscribe improved strategies and future collaborations that address critical outstanding challenges into new programme proposals.

Finally, putting this much emphasis on this mid-term evaluation should leave a solid impact analysis in place, easing the burden of the end-line assessment and making its result more quickly available. Beyond serving final accountability purposes, the end-line evaluation can then suggest tweaks and improvements to the new programme strategy whose implementation will already be ongoing by the time these evaluation results will have been published.