A multi-country impact evaluation
CHNIS was generously funded by Support Offices in Australia, Canada, Germany and the United States.
The Child Health and Nutrition Impact Study (CHNIS) was a five‐year research collaboration, launched in 2012, between World Vision and the Johns Hopkins Bloomberg School of Public Health, designed to measure the impact and cost-effectiveness of a three-part package of programmes on the health of mothers, babies and children. The three bundled, community-strengthening models - Timed and Targeted Counseling (TTC), Citizen Voice and Action (CVA), and Community Committees (COMM) were central to WV's 7-11 Strategy. Implementation of the bundle was carried out in four countries - Cambodia, Guatemala, Kenya and Zambia - and compared to matched comparison sites in the same countries that implemented only COMM. In size and scope, CHNIS was groundbreaking for World Vision, yielding rich knowledge and understanding around our core project models as well as how to improve implementation quality and how to undertake high-quality research and impact evaluations.
The Child Health and Nutrition Impact Study was a two-arm, quasi-experimental evaluation, with two intervention sites per country receiving a package of three approaches (see below) versus two matched comparison sites in each country receiving COMM plus any pre-existing CHW approach. In both intervention and comparison areas, World Vision's field operations were funded exclusively through private funds. The baseline was completed over a three-month period and the end line was completed five years later over the same timeframe to minimise seasonal effects. A mixed-method approach was taken combining results from population-based surveys with qualitative interviews with implementing teams, community members, CHWs and health facility staff. The study used the difference-in-differences (DiD) statistic to quantify as a percentage point difference whether intervention sites outperformed comparison sites. The package of three approaches evaluated:
Timed and Targeted Counselling: CHWs counsel families using storybooks followed by dialogue and negotiation for better health practices. Home visits are timed during pregnancy and early childhood to target health messages when they are most relevant. Of the three approaches, TTC is the most directly linked to health and nutrition behavioural outcomes.
Citizen Voice and Action: CVA empowers community members to monitor health services and build relationships with local governments to hold duty bearers accountable for the delivery and quality of health services. CVA groups facilitate a constructive dialogue between community members and decision-makers. CVA illuminates client-facing service quality elements, but only indirectly addresses clinical performance.
Community Health Committees: COMM empowers community groups to provide an enabling environment for health practice improvements. Committees diagnose and address barriers that families commonly face to reduce their daily risks or obtain needed health services. Committees link health facility staff to community representatives and can provide a support system for CHWs.
Results: Key findings and recommendations from the study
For community health systems strengthening:
- Community members and health facility staff highly valued the TTC, Community Health Committees (COMM), and Citizen Voice and Action (CVA) approaches.
- Across all countries, qualitative evidence found CVA improved facility-based care and both COMM & CVA contributed to making health services more patient-centred.
- Across all countries, home visits by CHWs doing TTC appeared to have a positive impact on health and nutrition behaviours compared to pre-existing CHW services.
- Greater attention to the timing, resourcing, streamlining, sequencing and staffing configurations of interlinked approaches is needed to boost their synergistic impact.
- Qualitative evidence that CHWs are well respected and appreciated in their communities, that caregivers value CHW services in all study sites and feel better supported by them to overcome barriers to adopting desired behaviour in intervention sites.
- Programmes had challenges reaching target levels of coverage and suffer from ongoing CHW turnover.
For maternal and child health
- Findings from across the 4 countries suggest that World Vision programmes support an improved continuum of care from the antenatal to the postpartum period.
- Results on exclusive breastfeeding and timely care-seeking for childhood illness showed that intervention sites outperformed comparison sites in three countries (Cambodia, Guatemala and Kenya).
- Children receiving their first dose of Vitamin A showed greater improvements in the intervention versus comparison sites for Cambodia, Guatemala and Kenya.
- Zinc treatment for diarrhoea persists as a gap in the delivery of a proven and practical intervention across all countries. Initial signs of progress towards coupling zinc with oral rehydration therapy reported in Zambia only.
- Our results show community health workers (CHWs) in all study sites can effectively promote behaviour adoption by the families they serve, but with no conclusive evidence that the World Vision package performed better overall.
- World Vision programmes are contributing to improved nutrition status with positive trends in three countries (Cambodia, Guatemala and Kenya).
- Improvement in wasting was significantly greater in the intervention versus comparison sites in Cambodia and Kenya.
- Results for exclusive breastfeeding are strongly positive. Improvements were seen across most study sites and intervention sites outperformed comparison sites in three countries (Cambodia, Guatemala and Kenya).
- Alarming and rising rates of maternal and childhood anaemia where found in Kenya and Cambodia. This population-based anaemia data is not commonly available and represents a strong call to action.
- Nutrition-sensitive approaches including WASH, food security and agricultural support also appear to have contributed to improved outcomes.
- A. Edward, K Dam, J Chege, A. Ghee, H Zare, & C Chhorvann. (2016). Measuring pediatric quality of care in rural clinics ꟷ a multi-country assessment ꟷ Cambodia, Guatemala, Zambia and Kenya. International Journal for Quality in Health Care. 28. 10.1093/intqhc/mzw080.
- Edward AE (2018) Impact of Impact of Community Oriented Interventions on Pediatric Care-seeking Practices- A Multi-country Study in Cambodia Prev Med Commun Health: DOI: 10.15761/PMCH.1000105
- A Edward, Younghee Jung, Grace Ettyang, Jane Chege, Annette E Ghee. (2018). Applying an Equity Lense to Maternal Health Care Continuum in Rural Communities of Cambodia, Guatemala, Kenya and Zambia. Internal Medicine Review, February 2018.
- A. Edward, Y. Jung, C. Chhorvann, A.E. Ghee, J. Chege. (2019) Association of mother's handwashing practices and pediatric diarrhoea: evidence from a multi-country study on community-oriented interventions J PREV MED HYG 2019; 60: E93-E102
- A Paden, AE Ghee, A Martirosyan, J Chege, B Aidam (2016). Reviewing implementation fidelity to leverage impact in a multi-country maternal and child health and nutrition study Annals of Global Health 82(3):519-520 · May 2016
- Submitted BMJ: Can people-centered community-oriented interventions improve Skilled Birth Attendance? Evidence from a quasi-experimental study in rural communities of Cambodia, Kenya, and Zambia
- Accepted: J Qual Health Care. Can social accountability mechanisms using community scorecards improve quality of pediatric care in rural Cambodia?
Guatemalan Madres Guías Gather to Share Experiences
Working together to build a new bathing center in Muniile, Zambia: Community groups work in synergy to address Maternal and Child Health
“The community did not know they have the voice to influence things to happen” - Sharon, World Vision CHNIS coordinator in Magoye ADP.
In Magoye ADP over the last two years, all 7 Rural Health Centers have seen the capacities of their COMM groups built (COMMs are called Neighborhood Health Committees or NHC in Zambia). Capacity building supported by World Vision has enabled the 208 NHC members to function effectively, supervising community-based cadres of Community Health Workers within a particular catchment area. During this same period, CVA groups were mobilized and trained. Previously, communities waited for things to be provided by local government. They believed that requesting something would be seen as “[acting] against the state”. But following CVA training, a change in mindset was visible. Community members are now knowledgeable about policies and standards within local public facilities particularly education and health, and know they have a legal right to demand that these standards are upheld. The community mindset change spurred by CVA training, combined with strengthened NHCs started to reinforce each other to improve Maternal and Newborn health in Magoye.
The story of a dilapidated and unsanitary bathing facility at Muniile health center illustrates the transformative power of this approach. As ttC trained Safe Motherhood Awareness Groups (a term used to describe Community Health Workers in Zambia) ramped up home visits to ensure that all women receive skilled childbirth care at the Muniile health center, the NHC group members quickly realized that the condition of its bathing shelter was a barrier for women considering where to give birth. Using their COMM training the NHC made an action plan to help address the problem, and engaged community members including the CVA group. The CVA group then advocated for local government to make improvements, and helped identify resources for the bathing shelter. As a result, the community began the construction of the bathing shelter at Muniile to offer new mothers a safe facility to use following delivery. It was completed this year.