A well-designed evaluation framework is essential to provide evidence for project efficacy
The Monitoring, Evaluation and Learning (MEL) experts from Novametrics led the development of digital and paper tools specific for the project (Novametrics, 2020).
MEL tools
- Training Register
- LDHF Practice Session Register
- Supportive Supervision Tool
- Respectful Maternity Care (RMC) Assessment
- Capacity Building Log
- Midwives Association Capacity Assessment Tool (MACAT)
Novametrics trained and provided ongoing support to the country teams enabling the collection of comparable standardized project data.
The baseline, midline and endline project evaluation included 137 sample health facilities (17% of 805 total implementation sites).
For more information about the Novametrics report contact ICM.
Impact, Outcome and Output Indicators
Impact 1: Improved Maternal Outcomes
- Sampled health facility sites case fatality rate: Postpartum Haemorrhage
- Sampled health facility sites case fatality rate: Eclampsia
- Sampled health facility sites maternal deaths as % of all births
Impact 2: Improved Neonatal Outcomes
- Sampled health facility sites case fatality rate: Neonatal Asphyxia
- Number of neonatal deaths <24 hours of birth in sampled health facility sites as % of all births
- Number of stillbirths in sampled health facility sites as % of all births
- % of sampled health facility sites births in which newborn was referred or admitted to newborn nursery
Outcome 1: Improved confidence and competence of project trainees
- % of training participants observed in simulation or practice as having required skills
- % of training participants rated as having the required knowledge
- Average % increase in training participants’ confidence score from pre-test to post-test
Outcome 2: Improved Quality of Midwifery Care
- % of cases of eclampsia in sampled health facility sites that are treated with magnesium sulphate
- % of women giving birth in sampled health facility sites who received a uterotonic immediately after birth
- % of newborns in sampled health facility sites who are not crying immediately after birth and receive bag and mask ventilation immediately
- % of women in sampled health facility sites who receive pharmacological relief and/or sedation prior to manual removal of placenta
- Average Respectful Maternity Care (RMC) score at sampled health facility sites
Outcome 3: Strengthened Midwives’ Association (MA) Capacity
- Number of MA members
- Number of MA members who joined in the last 12 months
- MACAT overall score (mean of the 7 domain scores)
Output 1: A cohort of active and high-quality HMS and HBS Masters Trainers and Practice Coordinators
- Number of certified HMS and HBS Master Trainer Facilitators (MTFs)
- Number of certified HMS and HBS Master Trainers (MTs)
- % of sampled health facility sites with at least one HMS and/or HBS Practice Coordinator
Output 2: Intervention sites are equipped with HMS and HBS training resources that are accessible as required for training and practice sessions
- % of sampled health facility sites receiving supportive supervision visits that have all the HMS and HBS training resources in good working condition
- % of sampled health facility sites receiving supportive supervision visits which have all the required clinical service equipment in good working condition
- Number of sampled health facility sites receiving supportive supervision visits in which, in the last 3 months, one or more scheduled supervised practice session did not take place due to the practice resources being inaccessible
Output 3: High quality HMS and HBS training delivered to midwives, other healthcare providers, and students
- Number of HMS and HBS champions trained in at least one training session
Output 4: Low-Dose, High-Frequency (LDHF) HMS and HBS practice sessions delivered to midwives, other healthcare providers, and students
- Number of supervised HMS and HBS practice sessions in the sampled health facility sites in the preceding 3 months
- Average number of HMS and HBS champions attending supervised practice sessions in the sampled health facility sites
- Number of unsupervised HMS and HBS practice sessions in the sampled health facility sites in the preceding 3 months
Output 5: Regular and comprehensive supportive supervision provided to MA’s and implementation sites
- % of sampled health facility sites that had at least one supportive supervision visit in the preceding 3 months
- % of supportive supervision visits that resulted in a supportive supervision tool being fully completed and returned to the MA Project Manager within 1 week of the visit
- Number of calls or visits between ICM and MA in the preceding quarter, during which at least on capacity building action was agreed
- % of MA capacity building actions that have been completed
Continuous feedback from monitoring data improves the quality of project activities
The consistent support from the MEL team increased the ability of the MA to be responsive to changes in the project.
Frequent communication between the MEL team and ICM enabled timely support to be provided to the MA who subsequently mitigated risk and unexpected challenges to project implementation.
Continuous quality improvement activities were observed during the project as improved function of the MA, emergency preparedness in health facilities, review of curricula in education institutions and better organisation of training and LDHF practice sessions.