This decentralized evaluation was jointly commissioned by a consortium of partners comprising the Southern Africa Development Community (SADC) Secretariat, Foreign Commonwealth and Development Office (FCDO), the Swiss Agency for Development and Cooperation (SDC), Landell Mills and the World Food Programme (WFP) and covered the period March 2017 to March 2022. The evaluation serves a summative accountability function by presenting the progress made towards attaining planned programme results, and a formative learning function to inform strategic and operational decision-making about future programming. Data collection took place from 12th July to 13th August 2021.
The evaluation focused on assessing the relevance, coherence, effectiveness, efficiency impact, and sustainability of the programme as well as the value for money.
The evaluation objectives included: 1) to provide an objective/impartial assessment of whether planned outputs were delivered and whether they led to anticipated outcomes, 2) to understand how the programme led to or contributed to outcomes, 3) to examine programme achievements, identify why they have or have not been made, and identify any broader consequences, positive or negative, intended or unintended, that have occurred as a result of the programme, and 4) to provide recommendations on future programming, implementation methods, strategic orientations, and partnerships.
Some key evaluation findings included:
- The programme remained relevant to the needs, priorities, and policies of Member States and of SADC throughout the region.
- The programme was sensitive and responsive to the SADC regional context, policies, and specific Member States challenges.
- The RVAA programme was well aligned with relevant SADC programmes and policies.
- RVAA was implemented within a framework of policies addressing food and nutrition security and well-being in Member States and the region.
- Using the institutionalisation index, the programme was on track to establish training, communication, and advocacy tools, record best practises for learning sharing, and provide technical assistance for phase-out and sustainability.
- Capacity was strengthened through training to enhance knowledge and to improve tools in many areas of the Vulnerability Assessments and Analysis (VAA) system and the conducting of VAAs.
- There was evidence that legitimacy, credibility, and influence of the RVAA programme has improved at national and regional levels.
- The programme developed VAA guidelines and standards during COVID-19, which assisted most member states continue conducting VAAs despite restrictive measures.
- Despite this flexibility, COVID- 19 weakened the programme resulting in some unmet targets.
- Gender integration was identified as a priority for RVAA Phase 3 along with climate change, HIV/AIDS, urban and markets, poverty, and chronic analysis.
- Internal factors that constrained programme achievements included revising the Theory of Change (ToC) and Logframe ToC refinement averted this.
- In some Member States, human resources and technical capacity also threatened the VAA system.
Key recommendations from the evaluation included:
- Expedite finalization of the sustainability plan and intensify capacity-building support focusing on issues needed for sustainability including prioritization of mobilising resources, quality assurance assessments, and partnerships and network management.
- Strengthen the SADC Secretariat's regional coordination of technical support for the RVAA programme to engage participation and ownership consistent with an emphasis on harmonization and regional integration rather than a multi-country or individual Member State focused perspective.
- Integrate gender, children, and people with disabilities in VAAs so National Vulnerability Assessment Committees (NVACs) can include it in Member State assessments.
- Improve RVAA conceptual clarity and monitoring, evaluation, and reporting (MER) among all stakeholders
- Cascade technical capacity-building to provinces and districts to help Member States mobilize resources and skills.
- Consolidate Communication and Advocacy Initiative to strengthen VAA in influencing policy and resource mobilization.
- Through cross-learning, promote nimble approaches to VAA by using new technology.
- Ensure RVAA programme handover to SADC includes ongoing processes and responsibilities.
- Promote the VAA system to play a preventive and forward planning advisory function on vulnerability beyond the supply of data.
