|
Summative report of the Masibambane II programme
David Everatt, Phillip Ravenscroft, Dave Still, Matthew J Smith, Nobayethi Dube, Nicky McLeod, Jim Gibson, Cindy Illing, Derek Hazelton, Junior Khanyi, Patrick Mbanjwa, Ross Jennings and fieldwork teams from Q&A and Field Focus
Department of Water Affairs and Forestry (DWAF); Strategy and Tactics
August 2007
|
|
|
[Download complete version  - 3.3Mb ~ 19 min (244 pages)]
[ Share with a friend 
]
|
Overview
-
Methodology: A multi-method summative evaluation of Masibambane Phase II (MSB II) has been completed. The evaluation involved a survey of 1025 people living in communities where the sector has delivered services (with direct beneficiaries) since 1994. Two telephonic surveys were also undertaken, one with WSA Managers, another with PMU Managers. Mini-surveys complemented these, such as a minisurvey of national sector departments around sector-wide (SWAP) issues, and a snap survey at the KwaZulu-Natal provincial co-ordinating meeting. In-depth interviews were conducted with over 80 people in the sector, and 5 provinces and 17 projects were visited, for qualitative insights. It should be noted from the outset that these are very rich datasets that deserve greater analysis by the sector over time, since their value extends far beyond the evaluation alone.
-
The results: This evaluation focused on outcomes as well as outputs. In other words, we analysed what was done in the sector with the support of Masibambane II, as well as why it was done and how sustainable the results will be over time. And the findings in these two areas are quite markedly different. MSB II scored extremely well in many key output areas, including 'hard' issues such as progress towards targets, and 'softer' issues such as institution and capacity support. But there are critical weaknesses in outcome-related areas, including environmental sustainability, gender mainstreaming and civil society involvement - but above all, operations and maintenance (O&M).
-
The evaluation uses a standard scoring methodology. The axes of the evaluation were efficiency, effectiveness, relevance, impact and sustainability. These were drawn together to rate MSB II overall, using the common categories of highly unsatisfactory, less than satisfactory, satisfactory, or highly satisfactory. Achieving a 'highly satisfactory' score is possible, but requires exceptional performance in all 5 key areas across all (or at least the vast majority) of the areas under study. In the case of MSB II, the evaluation team found a mixture of good and bad; and the threats facing the sector in the area of sustainability made it impossible to give an overall score higher than 'satisfactory'. This is nonetheless a very good result, built on very solid performance, and areas where improvement are required are set out clearly in the report and accompanying recommendations.
-
Above all, the relationship between provision and aftercare needs to be better balanced: at the moment, WSAs are under intense pressure to meet water and sanitation targets, and they prioritise this (in what they do, what they spend on and what they monitor) above the financial or operational sustainability of the services they provide. This is not sustainable in the long term. As the sector moves into phase III of MSB, the focus should be on water for sustainable growth and sustainable development, with a concomitant shift from delivery to asset management.
|
|
