National Monitoring and Evaluation Strategy - Jan 2010 Final Draft
National Monitoring and Evaluation Strategy - Jan 2010 Final Draft
National Monitoring and Evaluation Strategy - Jan 2010 Final Draft
REPUBLIC OF LIBERIA
MINISTRY OF HEALTH AND SOCIAL WELFARE
Over two decades of provision of health care services, the Ministry means of monitoring and
assessing achievements in the sector was through Demographic and Health Surveys (DHS),
rapid assessments and specific studies. The ushering in of a democratically elected government,
coupled with the restoration of peace and stability, has set the stage for transition from relief to
development. This paradigm shift provides an opportunity to reform the sector, improve the
health and well-being of Liberians by increasing access to quality and affordable health care,
and addressing critical health problems through effective and professional monitoring and
evaluation of the system.
Implementing the National Monitoring and Evaluation Framework and Strategy will be a major
step in the pursuit of the National Health Policy vision statement: a nation with not only
improved health, but also equal access to health care. This vision is attainable not only by
commitments from all stakeholders, provision of resources but also by initiating and
implementing a robust monitoring strategy, having clearly defined bench mark indicators that
will be used to evaluate the health care delivery system periodically.
The M&E strategy provides the roadmap for measuring achievements of the National Health
Policy Plan and the Basic Package of Health Services. The Strategy defines data collection,
management and dissemination processes. It also document means by which the health sector
will be monitored, reviewed and evaluated. The strategy includes milestone, progress, outcome
and impact indicators.
With the full implementation of this strategy, gaps in the health care delivery system will be
identified, improvement in data collection and management will be addressed and prompt
interventions will involve to bring relief to our people are experiencing health and health related
problems.
We are grateful to all those who committed their efforts, time and resources to the preparation
of national M&E Framework and Strategy. We are confident that the implementation of the
national M&E Framework and Strategy is both critical and doable. We encourage all actors and
programs to join us in this drive towards the transformation and development of the health
sector. This document forms the basis for monitoring and taken prompt interventions that will
eventually skew our appalling rates of morbidity and mortality in the sector.
2
Acknowledgement
The 2009 National Monitoring and Evaluation (M&E) Framework and Strategic Plan was initiated
by the Ministry of Health and Social Welfare M&E Unit, the United Nations Development
Program, the National AIDS Control Program , the National Malaria Control Program , the
National Leprosy and Tuberculosis Program, and the Health Management Information System
Unit. USAID through BASICS provided technical support to the development of this document.
A participatory method was used to develop this plan. Specifically the approaches used included
the following: 1) Review of national and international documents, 2) Consultative meetings with
the MOH/SW senior management and National Program’s M&E focal persons. 3) Establishment
of National M&E Technical Working Group from MOH/SW and its stakeholders.4) five days
Technical workshop involved the MOH/SW departments, the M&E Technical Working Group and
other partners. 5) The final draft review meeting was held during the Health Sector Review
Meeting.
The Ministry of Health and Social Welfare expresses its appreciation to all programs,
organizations and individuals that provided assistance and support in the planning and
elaboration of this Framework and Strategic Plan. For the formulation of this document we owe
the following individuals special thanks and appreciation:
Dr. Eisa Hamouda, International M & E Specialist; Dr. Chet, BASICS HMIS Consultant; Mr. Luke
Bawo, BASICS HMIS Technical Assistant; Dr. Louis Kpoto, Director of Epidemiology; Mr. Nmah
Bropleh, Assistant Minister for Planning and Policy Development; Mr. C. Sanford Wesseh,
Assistant Minister for Statistics; Mr. Benedict Harris, Health financing Officer; Mrs. Jessie E.
Duncan, Assistant Minister for Preventive Services; Mr. S. Tornorlah Varpilah, Deputy Minister
for Planning; Dr. Tarbeh Freeman, Dean, A.M. dogliotti Collage of Medicine-University of
Liberia; Mrs. Yah Zolia, Assistant Program Manager for M&E, NMCP; Mr. Joe Bondo, Director
National M&E Unit; Mr. George P. Jacobs, M&E Assistant; Ms. Edith Horace, M&E Assistant; Mr.
Joe S. Kerkula, Sr., M&E Assistant; Ms. Beatrice Lah, NACP; Mr. Sonpon Blambo Sieh, M&E
Officer, NACP; Mr. Jallah Gouyon, M&E Officer, NDS; Mr. Moses Badio,Data Manager, NACP;
Mr. Ernest Cholopray, Deputy Program Manager for M&E, NLTBCP; and Mr. Roland Nyanama,
M&E Officer UNDP.
Once more, I am pleased to recognize and appreciate the dedicated sacrifices and
commitments of partners and individuals who have contributed immensely to the elaboration of
the National M&E Framework and Strategic Plan. It is my fervent hope that this document is
implemented to its fullest and that the M&E Unit continues to work with these individuals,
programs and organizations as we struggle towards reforming and improving the health
delivery system in Liberia.
3
List of Acronyms and Definitions
4
PDA Personal Digital Assistant
PR Principal Recipient
PRS Poverty Reduction Strategy
TB Tuberculosis
TM Traditional Mid-wives
TTM Trained Traditional Mid-wives
TWG Technical Working Group
UNDP United Nations Development Fund
UNFPA United Nations Population Fund
UNICEF United Nations Children Fund
USAID United States Agency for International Development
WHO World Health Organization
5
Table of tables
Table of Figures
Figure 1. Result Hierarchy and Corresponding Monitoring & Evaluation Event .............. 13
Figure 2. Framework for measuring results ................................................................... 15
Figure 3. Monitoring, Evaluation, and Research Institutional framework. ..................... 25
Figure 4. M&E Structure at the county .......................................................................... 26
Figure 5. MOH&SW Routine Data Flow Chart ................................................................ 36
6
Table of Content
1. Introduction................................................................................................................... 8
1.1 Context ...................................................................................................................... 8
1.2 Overview of Health Sector ............................................................................................ 8
1.3 National Health Policy and Plan (2007 – 2010) ............................................................... 9
7
1. Introduction
1.1 Context
After decades of turmoil, Liberia is enjoying peace and stability, under the watch of a legitimate
government, recognized and supported by the international community.
According to the provisional results of the 2008 Population and Housing Census of Liberia, the
population is 3,489,072 (Goverment of the Republic of Liberia, 2008). The population annual
growth rate is 2.1%. The overall sex ratio is 102.3(number of males per 100 females) virtually
the same as that for 1984. Population density is 84 per square mile. The mean household size
declined from 6.2 in 1984 to 5.1 in 2008. The total national population is seen to be unevenly
distributed among the counties. Ever since 1984, the population distribution favors the ‘big six’
– Montserrado, Nimba, Bong, Lofa, Grand Bassa and Margibi counties; in descending order of
magnitude. They account for 75.2% of the total population. The South-East is very sparsely
settled. The current fertility rate is estimated to be 5.2 (LDHS, 2007)) a substantial decrease
compared to 6.2 in 1999-2000. Almost one in three young women age 15- 19 has already
begun childbearing. The use of modern family planning methods among women is 11 % (LDHS,
2007).
8
tuberculosis, sexually-transmitted diseases (STDs), worms, skin diseases, malnutrition, and
anemia are the most common causes of ill health. Malaria accounts for over 40% of OPD
attendance and up to 18% per cent of inpatient deaths. HIV prevalence rate estimates vary
widely, but the 2008 antenatal sero-survey estimated and average of 5.4% with slight reduction
from previous years of 5.7 (NACP, 2008). Existing data are inadequate to draw firm conclusions
about internal variations in HIV prevalence. It appears that Monrovia and the south-eastern
region have higher HIV prevalence rates than the rest of the country.
Nutrition
According to 2007 LDHS, 19% of under five children are underweight and 8% are wasted (thin
for height). In addition, 39% of children under five are stunted. One fifth of children are
severely stunted. Stunting ranges from 30% in Monrovia to 45% in the South Eastern. These
figure show slight change in children’s nutritional status compared to previous findings from the
2006 CFSNS. In 2006 approximately 27% of children were underweight, 7% are wasted, while
39% are stunted (CFSNS, 2006). No updated data available on micronutrients’ deficiency.
However, the 2006 CFSNS estimated iron deficiency anemia at 87% in children 6-35 months,
58% in non-pregnant women 14-49 years, and 62% in pregnant women aged 14-49 years.
Vitamin A deficiency affects 52.9% of children 6-35 months and 12% of pregnant women. Zinc
supplementation for children has not yet been introduced. One in three children under six
months of age in Liberia is exclusively breastfed (LDHS, 2007). This figure is similar to the 2006
UNICEF’s report where 35% of children below 6 months of age are exclusively breast-fed
(UNICEF, 2006).
The formulations of the National health policy and National Health Plan in 2007 were milestones
in the Government of Liberia’s commitment to rebuild and develop the health sector to
“effectively deliver quality health and social welfare services to the people of Liberia”.
9
The National Health Plan sets forward a framework for shifting from humanitarian to
development and from vertical to integrated health systems development. This framework is
based on four components as summarized in the table below.
The major challenges in the existing M&E and HMIS relate to the harmonization, integration,
coverage gap and delays in reporting; this contributes to the inadequate use of information as
the basis for decision-making in planning and management. In addition, parallel reporting
systems with programmatic and donor-supported initiatives result in multiple reporting formats
and an increased administrative workload. Below is highlight of details of main systems’ gaps
and constraints:
Although people use selected indicators for monitoring interventions (like EPI coverage),
there is no systematic measuring of the BPHs implementation at the peripheral level,
using standardized set of indicators. As a result there is a gap in collecting the necessary
information.
The quarterly review reports are still based on raw figures and absolute levels of
accomplishment (rather than on performance indicators) and on a long list of data items
10
concerning activities implemented (ranging from key to marginal services), without any
prioritization. This approach is not suitable for performance comparison and trend
analysis, and may undermine the overall monitoring exercise.
In the context of decentralization and health sector reform, demands for monitoring the
performance of the health sector necessitate clear statements on planned targets and
measurement of actual achievements. These processes require explicit standards for
measuring performance, clear specifications of the relationship between inputs and
outputs, and use of valid indicators to compare the actual achievements with the
planned targets. In this perspective, performance monitoring should rely on a minimum
set of key indicators and focus on the implementation of the activities and the
intermediate steps that determine how inputs are transformed into outputs, linked to
the ultimate desired outcome.
11
REPUBLIC OF LIBERIA
MINISTRY OF HEALTH AND SOCIAL WELFARE
12
PART I: NATIONAL MONITORING AND EVALUATION POLICY
Monitoring and evaluation take place at two distinct but closely connected levels, whereby
monitoring focuses mainly on products and service outputs that emerge from processing inputs
through the Annual Plans; and whereby evaluation focuses mainly on outcomes and impact.
Performance Monitoring and Evaluation Chain is the series of ongoing routine monitoring;
annual reviews and five-yearly evaluation make up the performance M&E chain. The former
takes place more frequently at lower levels and focuses mainly on outputs, whereas the latter
takes place less frequently at higher levels and focuses more on outcomes and impact.
( Daily, Monthly,
Quarterly)
Review
(Quarterly, Annually)
Evaluation
( Mid-term, Five
Yearly)
All Performance Monitoring and Evaluation events are aimed at the systematic collection and
analysis of information to track changes from baseline conditions to the desired outcome and to
understand why change is or is not taking place. These functions are closely linked to decision-
making processes at service delivery, programme and policy levels. They provide consistent
information to service providers, programme managers and other stakeholders for the
improvement of interventions and strategies. And they allow for holding policy makers and
managers accountable. They differ, however, in their specific objectives, focus and
methodology, and how they are conducted and used.
13
Table 2. Main features of routine Monitoring, Annual Conference and Evaluation
Routine Monitoring
Annual and Quarter Evaluation (Mid-Term, 5
(Daily,
Review/Conference yearly)
monthly/quarterly)
To track changes from To track and validate To validate what results were
baseline conditions to mainly outputs and achieved, and how and why
Objective
desired output. outcome to some extent they were or were not
achieved.
Focuses on the inputs Focus on the annual plan Compares planned with
and outputs of annual targets mainly on output intended outcome
plans. and outcome achievement. Focuses on how
and why outputs and
Focus strategies contributed to
achievement of outcomes.
Focuses on questions of
relevance, effectiveness,
sustainability and change.
Tracks and assesses Evaluates annual Evaluates achievement of
performance (progress performance by outcomes by comparing
towards outcomes) comparing indicators indicators before and after 5
Methodology through analysis and before and after. Relies year NHP. Relies on
comparison of on monitoring data from monitoring data on
indicators over time. routine HMIS. information from external
sources.
HMIS HMIS (monitoring report) Surveys (harmonized to meet
Supervision report Annual Rapid Assessment 5 yearly evaluation)
Information
Activity report for Annual Conference Research report
Sources
Annual Conference reports
Observations
Continuous (3 monthly) Annually by key 5 yearly
and systematic by partners with or External evaluators and
Conduct
Programme Managers without help of partners.
and key partners. external facilitator
Alerts managers to Provides input to the Provides managers with
problems in planning of next annual strategy and policy options.
Use performance and plan.
provides options for
corrective actions.
Service providers Programme managers Policy and strategic planners
Main users Programme managers (public, private) Development partners
(public, private) Development partners
14
Logical Framework in Monitoring and Evaluation
An effective M&E system has a clear logical pathway of results which encompass the major
levels that include inputs, outputs, outcomes and impacts. Figure 1 demonstrates these
interconnections where:
Inputs: are resources that are put into a program in order to achieve the delivery of services;
Processes: are activities carried out for the achievement of one’s goals
Outputs: are tangible products that are necessary to achieve the objectives.
Impact: is the overall and long-term effect of an intervention, for example, measurable health
changes that are associated with outcomes, particularly reduced mortality and morbidity.
At the bottom of figure 1, are the actions for improved monitoring and evaluation.
1
Figure 2. Framework for measuring results
Inputs Process Outputs Outcomes Impact
Increased
Funding National
Efficiency
Quality,
products, information
Safety,
Governance, HR, medical
Efficiency
Health system monitoring
Quality,
Efficiency
Access, safety, quality, efficiency
Improved services:
service Improved
GOL Implementa utilization and survival
contribution tion intervention Child mortality
monitoring coverage Maternal
Pool mortality
Reduced
Funding Reconstructio Adult mortality
inequity (e.g.
n, BPHS gender,
Vertical Improved
urban/rural)
Programs Capacity nutrition
building Responsiveness
N.H.Plan Programmes No drop-off non- Reduced
health sector
Institutions interventions
morbidity
HR (e.g. water & HIV, TB,
Accountabili sanitation) malaria
ty
Aid Performance
Implementati Health system Coverage Improved
Impact
monitoring equity
M&E Action
evaluation
15
2. Purpose of Health Sector Performance Monitoring and Evaluation.
The aim of Health Sector Performance Monitoring and Evaluation Policy is to provide
information that will enable tracking of progress to enhance the health sector’s efficiency, and
improve the quality and coverage of health services. The overall goal of the policy is to set the
guiding principle of performance measurement, explain the concepts and use of monitoring and
evaluation within the health sector as well as define the roles and responsibilities of various
M&E actors.
The M&E policy will be operationalized through a national monitoring and evaluation strategy
with guidance on the use of the M&E logical framework and other specific measurement issues
circulated by MOH/SW in consultation with partners.
All inputs in the National Health Plan and Annual Work-Plans are designed to acting in concert
to achieve results. The monitoring of these results represents a distinct shift away from the past
interest in monitoring inputs to outputs.
The MOH/SW and its partners at both national and decentralized levels are committed to the
development and operation of a comprehensive Monitoring and Evaluation (M&E) systems
(NHP, 2007). The following principles guided the development of the national M&E Framework
and Plan:
1. Building strategic partnership for M&E: The NHP builds on partnership among various
stakeholders. This principle also applies to the development of the national M&E system.
2. Mainstreaming the M&E system will be incorporated into the NHP and will be used to
monitor the indicators and strengthen policy monitoring and evaluation
3. Enough financial resources will be mobilized and allocated for the strengthening of the
M&E system.
4. Standardized core set of indicators. The national M&E system will have a core set of
national indicators.
5. Simplicity: Data collection, analysis and the dissemination for information to the
stakeholders will be simplified and made user friendly.
6. Data Quality Assessment (DQA): The MOH/SW and HMIS will put in place DQA protocols
to verify the completeness and accuracy of the data collected. These will ensure both
internal self-assessment and external verification of data/information.
7. Data collected at the county or national levels will be used for decision-making.
8. Timeliness and Reliability of Data: data collected, disseminated and used through a good
M&E system will be timely and reliable. All programs, CHTs, and partners will be
required to be transparent and accountable to the M&E system they have and the data
they collect and provide to MOH/SW.
16
4. The roles, responsibilities of performance monitoring and evaluation
Result monitoring will be a continual and systematic process of taking decisions based on
systematically collected and analyzed data to improve the performance of the health sector
towards achievement of Millennium Development Goals, Poverty Reduction Strategy, and health
sector targets.
MOH/SW and its partners carry out result monitoring by tracking outputs and measuring their
contributions to outcomes by assessing the change from baseline conditions. They oversee the
key outputs – the specific products and services that emerge from processing inputs through
their respective programmes – because they can indicate whether a strategy is relevant and
efficient or not. Relevance in a results-based context refers to whether or not an input in the
Annual Plans contributes to the achievement of a key outcome.
To conduct effective outcome monitoring, MOH/SW and its partners will establish baseline data,
select outcome indicators of performance and design mechanisms that include planned actions
such as field visits, stakeholder meetings and systematic analysis or reports. The M&E activities
will be conducted in the following sequence:
All health facilities starting from level 1 to level 6 will be monitoring their outputs on a monthly
basis. Such monthly monitoring will help service providers detect the problems at an early stage
and take corrective measures so that the annual targets of the program are met.
County and the Central MOHSW will carry out quarterly monitoring at their respective levels to
ensure that programme results are produced as planned. Such monitoring will trigger
supportive supervision and follow-up visits to the next lower level of the health system.
Annual Reviews will primarily focus on the performance of the annual plan and therefore they
can be termed as annual interim evaluations. At the same time, an Annual Review will analyze
the performance towards the achievement of long-term goals and targets as formulated in the
PRS, National Health Policy and National Health Plan.
Result evaluation will be an evaluation of the five-year National Health Plan, BPHS, and
Decentralization approach, all intended to bring about a health outcome and impact. The
evaluation will assess whether or not outcomes are being achieved. It will help to clarify the
reasons for any underperformance, highlight the unintended consequences, recommend actions
to improve performance for future programming, and generate lessons learned. Thus, the
evaluation will provide input to the next cycle of programme planning.
17
5. Frequency and participation in Monitoring and Evaluation
The frequency of M&E activities and the involvement of parties in the above PM&E functions will
be as follows:
MOHSW Programme /
Quarterly Self Nov, Feb, May, Aug
Programmes/Divisions Division Managers
DHMTs
Annual Review Facility Facility In-charge July
Chair-FMC
CHB
Annual Review County CHO August
Programme
M&E Sub
Annual Review National Managers; DM November
Committee
Planning
M&E technical
Evaluation National DM Planning TBD
working group
1. Performance Indicators
A core set of the national level performance monitoring and evaluation is included in the
National M&E Strategy. These core national indicators were selected and agreed upon in a
participatory manner through various consultation meetings and national workshops.
Measuring Equity
Narrowing inequity and demonstrating improvements in health in line with national targets is
generally not possible without having information about health status, health determinants,
service utilization and the effect of services on the health of populations at sub-national and
local levels. Therefore, the information on all of the service coverage, outcome and impact
indicators will be disaggregated by geographic area.
Information on the following indicators will be disaggregated, in order to examine whether
there are any discrepancies in utilization of BPHS between genders or levels of poverty.
Furthermore, CHT and MOHSW headquarters will monitor geographical inequities by comparing
health facilities and counties respectively.
18
Information Gender Poverty
OPD utilization X X
Clients satisfied with services X X
The HMIS will work closely with various stakeholders at both national and county levels to
coordinate collection of data that will be used to generate information products. The data
collection strategy for the routine national essential indicators and dataset (NEIDS) at facility
and county level has already been developed and rolled out through the DHIS. This strategy
entails data collection from the community, health facility (public and private), district, county to
the national levels. The HMIS shall establish data quality assessment protocols in a participatory
and consultative manner with all stakeholders.
Different M&E events in the series will utilize information from various reports depending on the
type of M&E event in the series.
In order to ensure harmonization of information at the Ministry, the Department of Planning will
be responsible for information products at the Ministry of Health. The M&E and Research Unit of
the Department of Planning, in collaboration with the HMIS Unit, will develop information
products that will be disseminated to stakeholders at both national and county levels. The
information products that will be developed include the following:
4. Information Dissemination
Monitoring and evaluation should not end with the production of reports. The reports need to
be adequately shared and disseminated to the health sector stakeholders, so that M&E can
serve as an instrument for ensuring the achievement of national health goals.
Besides the dissemination as stated above, any documents that help improve understanding,
planning and management of health services will be disseminated as widely as possible, using
modern technology, where appropriate.
M&E and Research Unit of MOH/SW is tasked with the responsibility to ensure that data
collected and reported on national indicators are of high quality and can be assessed and
verified. The Central M&E and Research Unit will also work with vertical program M&E Units in
coordinating the assessment of their programs’ data quality. The assessment and verification of
19
data can be done through a developed DATA QUALITY ASSESSMENT (DQA) TOOL—a single
integrated tool that ensures that standards are harmonized and allows for joint implementation.
Monitoring and evaluation are carried out at different intervals and with a common purpose.
The table below presents some of the specific purposes of using different types of M&E results.
The existing governance structures will oversee sector performance at the various levels of the
health system. The different governance structures will be overseeing the performance as
shown in tables below.
20
Table 6. Summary of key role of M&E Stakeholders.
The MOH/SW shall contract individual(s)/ firm(s) to develop the national evaluation plan for the
health sector. This plan will serve as the framework for the Health Sector Evaluation (HSE). The
plan will specify the evaluation scope, the implementation and data collection methodology, the
TOR for the evaluators, and the advisory board. Integrating the principles of the evaluation
framework into all the Health System operations we hoped to stimulate innovation toward
21
outcome improvement and to detect the Health Plan effects. More efficient and timely detection
of these effects will enhance our ability to translate findings into practice.
The health sector evaluations will be guided by the national health plan framework (PBHS, HR,
Infrastructure, Health Financing and Partnership and Support Systems).
22
REPUBLIC OF LIBERIA
MINISTRY OF HEALTH AND SOCIAL WELFARE
23
PART II: MONITORING AND EVALUATION STRATEGY
1. Develop clear M&E strategies using standardized M&E and supervisory guidelines;
2. Regularly monitor progress and achievements of NHP components as a whole and
improvements in service delivery, quality of care and financial performance;
3. Evaluate the impact, effectiveness and cost-effectiveness of the BPHS components;
4. Define the roles of stakeholders in the systematic collection, collation, analysis and use
of data in order to avoid duplication of efforts;
5. Improve information sharing and dissemination of information and the use of data for
planning.
1. Develop indicators that link health service outputs and outcomes to inputs
2. Provide evidence for policy formulation, assessing quality of service, preparing budget
and program plans
3. Build capacity to monitor the implementation of contractual arrangements
4. Strengthen capacity for operational research
5. Improve the application of appropriate technology for data collection, storage, analysis,
and dissemination of health information
6. Establish national working groups (MER-TWG, HMIS-TWG, etc) to develop guidelines
7. Develop performance measures for benchmarking inter-district and inter-county
comparisons
8. Liaise with the LISGIS to enhance the design of the national households survey
9. Forge linkages between research and routine health information systems
10. Create national database of research findings and a forum for dissemination
11. Streamline the deployment and use of information and communication technology
through the implementation of IT policy
12. Develop in-service training for staff to improve capacity for using information
technology.
24
2.1 National Level
The MOH/SW has the mandate of coordinating, monitoring, and evaluating health services in
the country. The MOH/SW has created a Program Coordination Team (PCT) to coordinate
partners and resources for implementing the NHP&P. The PCT consists of Deputy Ministers and
supported by technical experts and headed by the CMO/DMHS. Within the proposed M&E
systems, the PCT shall be mandated with policy-making on M&E, oversight of M&E functions,
and creating an enabling environment for M&E. A Central M&E and Research Division is
proposed by this framework. The division will be under the Bureau of Vital and Health Statistics
within the Department of Planning (Figure 2). The M&E and Research division will be tasked
with operationalizing M&E and Research Policy and the development of standards in
consultation with partners. The MOH/SW has also established a national M&E Technical
Working Group (TWG) charged with the responsibility of providing technical assistance in the
course of implementation of the national monitoring and evaluation roadmap, and providing
training to the MOH/SW and to stakeholders in the M&E of programs.
Universal Birth
M&E and Research Information Technology
Registration
Health Management
M&E
Information System
Research
2.2 County
The CHTs are required by the NHP and its M&E policy to monitor and evaluate health programs
implemented by different stakeholders including NGOs, CBOs and the private sector. They are
also in-charge of submitting program activity data to the MOH/SW Central M&E and Research
Unit, organizing county M&E coordination involving communities, and participating in national
M&E working groups and reviews.
The M&E Policy proposes a monitoring and evaluation structure at the county level to be
coordinated by a County M&E and Research Officer (Figure 3). The officer will supervise the
Data Manager/ Registrar. Both of them will report to the M&E Officer. The Data Manager will
be assisted by a Data Clerk(s).
25
Figure 4. M&E Structure at the county
The NGOs and private sector at the county/ district level are to collaborate with the CHT in the
implementation of the county M&E Plan, submit their programme activity data and information
products to the CHTs and DHOs, and participate in M&E and Research related activities and
coordination.
Core national indicators that were decided upon in a participatory manner were selected for
inclusion in this framework. This framework contains the national priority indicators. A second
set of 125 indicators (including the national priority ones) is under review for inclusion into the
HMIS strategy. The second list of indicators, the data sources, and the operational definitions
for the indicators are presented in the Annex.
The National M&E system has primarily three types of indicators. They include the following:
Impact indicators (to measure the long-term results in the results framework)
Outcome-level indicators (to measure actual or intended changes);
Output indicators (to measure tangible products that are necessary to achieve the
objectives).
A set of community-related indicators that measure all other efforts that takes place
within communities need to be developed.
26
Final Draft
component
Denominator
Reference
Hierarchy
Program
National
Baseline
Target
Numerator
Result
data name
data name
Type
Indicator
1. ID
National
source
source
Data
Data
2. 1
Infant mortality rate N Health Status NHP 71 Impact 5Y Number of deaths to Census Total live births Census
children under 1 DHS DHS
3. 2
Under-five mortality N Health Status LPR 111 94 Impact 5Y Number of deaths to Census Total number of Census
rate S under 5 children DHS live births DHS
4. 3
Maternal mortality N Health Status LPR 994 895 Impact 5Y Number of maternal Census Total number of Census
ratio S deaths in a given DHS live births DHS
year due to
pregnancy related
causes during
pregnancy or within
42 days of childbirth
5. 4
Total fertility rate N Family NHP Impact 5Y Total number of Census Number of Census
Planning children that would DHS women in group DHS
be born to a group
of women if all lived
to the end of their
childbearing years
and bore children
according to a given
set of age-specific
fertility rates
6. 5
HIV Prevalence in N HIV/AIDS LPR 1.5 1.5 Impact 5Y # of HIV cases DHS 15-49 aged Census
general population S (new+ old) HIS population estimate
7. 6
Percentage of health G HIV/AIDS BPH Input A Number of health AHFC Total number of AHFC
facilities providing S facilities providing health facilities
HIV laboratory HIV laboratory
services services
8. 7
Health expenditure N Finance NHP Input A Sum of the public NHA Estimated mid- Census
per capita and private year population
expenditure in
health
9. 8
Percentage of health G Child Health BPH Input A Number of health AHFC Total number of AHFC
facilities having birth S facilities having birth health facilities
registration program registration program
10. 9
Percentage of health G Maternal BPH Input A Number of midwives HRIS Number of HRIS
professionals trained health S trained in life saving certified midwives
in and providing i life skills currently in the
saving skills (LSS) health system
11. 1
Percentage traditional G Maternal BPH Input A Number of HRIS Number of HRIS
0
midwives trained in health S traditional midwives traditionally
home based life trained in home trained midwives
saving skills (LSS) based life saving
skills
12. 1
Percentage of health G TB BPH Input A Number of health AHFC Total number of AHFC
3
facilities providing TB S facilities providing health facilities
laboratory services TB laboratory
services
13. 1
Percentage of health G CP BPH Input A Number of health AHFC Total number of AHFC
4
facilities running CHV S facilities having CHV health facilities
program program
14. 1
Contraceptive M Family LPR 11 15 Outcome A Total number of DHS Total woman of Census
5
prevalence rate Planning S users using modern HIS child bearing age estimate
(Percentage of family planning in the catchment
women 15-49 using methods at a point population (mid
modern contraceptive of time year)
methods)
15. 1
Percentage of N Malaria NHP Outcome 5Y # of children under DHS Total children Census
6
children under age 5 age 5 sleeping under HIS under age 5 estimate
sleeping under insecticide treated
insecticide treated bed nets
bed nets
16. 1
Community Health G Human BPH Outcome A Number of Census Total population HRIS
7
Volunteer rural resource S community health
population ratio workers
28
17. 1
Health professionals N Human NHP Outcome A Total population HRIS Number of health HRIS
8
to the population resource professionals by
ratio by category of category of
pharmacist, doctors, dentist,
environmental nurses, midwives,
technician, pharmacist at
administrators, and work in any
others per 1000 sector in the
population country
18. 1
Percentage of vital G Civil BPH Outcome Q # of births CR # of estimated CR
9
registration for birth registration S registered births
19. 2
Cure rate among N TB MD Outcome A Number of new DHS Total number DHS
0
smear positive TB G sputum positive HIS new sputum HIS
cases (Under Directly cases who are positive TB
Observed Treatment proved smear patients in the
Short Course) negative at the end same cohort
of treatment
20. 2
percentage of annual N Finance NHP Output A Amount spent in a FMIS Total allocation FMIS
1
budget utilized year as planned for the year
21. 2
Percentage of M Child Health MD Output Q Number of children DHS Estimated Census
2
children under one G under one who HIS number of under estimate
fully immunized received pentavalent 1 population
(Measles as proxy) 3
22. 2
Percentage of N Maternal MD Output Q # of deliveries DHS Expected # of Census
3
deliveries conducted health G conducted at health HIS deliveries estimate
at health facilities by facilities by skilled
skilled health health personnel
personnel
23. 2
Percentage of N Malaria NHP Output Q # of pregnant DHS # of pregnant DHS
4
pregnant women women (attending HIS women attended HIS
(attending ANC) ANC) received 2 or ANC
receiving two or more more doses
Intermittent Intermittent
preventive treatment preventive treatment
(IPT) (IPT)
29
24. 2
Percentage of G Human BPH Output A Number of AHFC Number of AHFC
5
position filled by resource S professionals of establishment of
category of doctor, each category at each category of
dentist, nurses, work professionals
midwives, position
pharmacist,
environmental
technician,
administrators, and
others
25. 2
Percentage of health M QA NHP Output A Number of health AHFC Number of health AHFC
7
facilities providing full facility providing full facilities expected
BPHS BPHS to provide full
BPHS
26. 2
Percentage of health N supplies NHP Output A Number of public SLMIS Number of health AHFC
8
facilities reporting no health facilities facilities
stock outs of without stock outs of
essential drugs essential drugs for
more than a week at
a time
27. 2
Percentage of health G QA BPH Output Q # of health facilities HIS # of health HIS
9
facilities supervised S supervised by DHMT facilities
by CHT members in members using
the last 3 months integrated
supervision checklist
28. 3
County Health Board G Management BPH Output A Number of health AHFC Number of health AHFC
0
established S facilities having facilities
functional health
committee
29. 3
Percentage of G Management BPH output A Number AHFC Expected number AHFC
1
coordination/manage S coordination/manag of coordination /
ment meeting held in ement meeting held management
the last three months meetings
30. 3
Percentage of health G Infrastructure BPH Output A Number of health AHFC Number of health AHFC
2
facility rehabilitated S facilities completed facilities planned
rehabilitation for rehabilitation
31. 3 N BPHS NHP Output A Number of health AHFC Total number of
3
% of health centers centers and functional Health
and hospitals with hospitals with centers and
30
emergency transport emergency transport Hospital
system system
32. 3
% of County with N BPH Output A Number of county AHFC Total number of
4
emergency prepared S health team with functional health
plan emergency facilities
IDSR preparedness plan
33. 3
% of Health facility N Infrastructure BPH Output A Number of Health Total number of
5
with operating hand S facility with functional Health
pump or an operating hand facilities
equivalent safe water pump or an
source equivalent safe
water source
34. 3
% of county health N Finance NHP Output Q Number of county Total number of
6
office that maintains health office that county health
an active financial maintains an active teams
ledger financial ledger
35. 3
% of county health N Governance NHP Output Q Number of county Total number of
7
board that have health board that CHT
regular meeting have regular
during the last three meeting during the
months last three months
36. Couple Years N Family NHP outcome
Protection (CYP) Planning
37. Percentage of health N Maternal NHP output
facilities providing Health
EmONC services
31
Final Draft
There are two primary Integrated Supportive Supervision areas: (1) Clinical Supervision for
the purpose of assessment of the quality and consistency of health care delivery and (2)
Program Integrated Supportive Supervision intended to supervise program implementation
and evaluate the accuracy of reporting and data collected. . Neither type of supervision is
currently carried out on a regular basis at all facilities in Liberia. This is a result of issues
with coordination between programs as well as staffing, financial and logistic constraints,
particularly at the county and district levels.
At the central level, responsibility for conducting Integrated Supportive Supervisions falls on
the Department of Health Services and the Department of Planning respectively. An
integrated supervision checklist has been developed. This checklist shall be revisited and
automated using the PDA software. Moreover, the clinical supervision has not yet been
systematically introduced at the county or district level. Therefore, even when the vertical
programs such as NACP, NLTCP and NMCP have tried to share responsibility for clinical
supervision with county level staff, their counterparts on the CHT have not been motivated to
assist in the work. .
While clinical and program supervisions require distinct personnel, expertise and tools, they
share five goals. First, to incorporate a system of analysis and review which leads to remedial
action to improve performance and in turn improve the health sector indicators; second, to
provide continuous on the job training in health facilities; third, to increase the involvement and
commitment of staff at both the county and the district level; fourth, to ensure that private and
NGO-supported health facilities are participating fully in the national health strategy; finally,
they seek to ensure the equitable provision of services to all sectors of the community,
including remote and hard-to-reach areas. The ongoing M&E system strengthening is an
opportunity to integrate and strengthen the supervision by defining roles of agencies and staff,
combining resources at a central level and distributing them appropriately to the counties and
districts, and creating a timeline for both the program and the clinical supervisions. These
objectives will be best accomplished by a clear supervisory plan of action endorsed and
supported by both the Departments of Health Services and Planning.
Both clinical and program supervision should be performed regularly at all levels of the health
sector by specifically trained supervisors whose roles and responsibilities have been specifically
defined by the MOHSW. The intervals of Integrated Supportive Supervision visit should be set
for the various levels of health systems. The district level supervisor should visit a local facility
approximately once per month, a county level supervisor approximately once/quarter and a
central level supervisor(s) approximately biannually. Each level supervisor should expect to
travel 10-15 days per quarter. Integrated Supportive Supervisions should be increased when
problems are noted.
A pilot WHO PDA project is in progress to improve and automate the conduct , documentation
and sharing of findings.
Data quality assessment (DQA)
The assessment and verification of data will be carried out through a developed DATA
QUALITY ASSESSMENT (DQA) TOOL—a single integrated tool that ensures that standards are
harmonized and allows for joint implementation.
The Central M&E unit in collaboration with national programs and partners shall also work to
enhance the data quality and address challenges and factors that influence data quality (table
below)
Table 8. Data are of quality when the following are contained in its dimension
Dimension Description
Completeness Data exhibits completeness if nothing needs to be
added, e.g. no blank space is left
Accuracy The degree to which data correctly reflect the real world
of an event being described
Reliability The degree to which the same result can be obtained by
repeating the same data capture process
Timeliness Data are current and information is on time. Reporting
as per schedule
Confidentiality Interviewees/clients are assured that whatever data
collected are kept private or secret according to national
and international standards
Integrity This dimension protects data from deliberate bias or
manipulation for political or personal reason(s)
Precision Data have sufficient details ,e.g. disaggregated by age,
sex etc.
Data quality checks shall be done at least twice per year by the central Monitoring &
Evaluation Unit and more frequently at the County and Facility level.
33
Cross verification---cross verification of programmatic results
with other data sources
Spot-checks of actual service delivery
Produce report Production of report will be done by M&E and
Research/HMIS Unit and submitted to PCT/MOHSW and the
vertical M&E Units for their programs.
The data collection strategy for the routine national essential indicators and dataset (NEIDS) at
facility and county level has already been developed and rolled out through the DHIS. This
strategy entails data collection from the community, health facility (public and private), district,
county, and national levels. It will involve monthly and quarterly progress reports coming from
health facilities run by public and private (profit and non-profit organizations) and then submitting
them to the CHTs, and copied to their national organizations. The modalities and the actual
process of doing so will be agreed upon and a county M&E operational plan developed in a
participatory and inclusive manner. The Central M&E Unit will supervise this exercise and make
sure that a county-level operational manual/guideline is developed and disseminated widely to all
stakeholders.
To ensure accurate, comprehensive and timely reporting, the HMIS has rolled out a
comprehensive DHIS training in all 15 counties. The M&E and Research division will coordinate
future capacity building and training programs in M&E and research at all levels, especially in the
areas of data collection, analysis, interpretation, production of information products and use of
the data for decision making and programming. Strengthening other M&E and Research and
research activities at the CHTs level will be a key priority.
Data Sources
In order for the national M&E and Research system to function, core data sources that feed into
the HMIS have been identified (for details refer to HMIS Strategy). There are two major
categories of data sources:
Data source for routine (output) program indicators and dataset: these are routine data and
reports from various levels. They include the routine data from health facilities and NGOs at
counties levels
Data sources for Impact/Outcome assessment such as the periodic population based
national surveys like LDHS, population census, special studies, e.g. operational research,
Health Facility Surveys.
For health delivery, the WHO proposed multiple data sources on a wide range of indicators
related to availability and access, quality, safety, efficiency and equity of services will be used.
The MOH/SW shall utilize such data sources and, in addition, the Ministry shall use GPS devices
and PDA based questionnaires to allow mapping of results, rapid data processing and report
production that shall be integrated with the DHIS.
34
Table 10. List of specific data sources for BPHS, HR, Infrastructure, and support systems
National Plan’s Data collection methods Descriptions
Pillars
BPHS Facility reports Regular facility data reported to regional and
national levels by service providers.
County key informant survey Periodic survey of all districts or equivalent
administrative unit within a country. Interviews
with district medical teams.
Facility census Periodic census of all public and private health
care facilities within the country.
Facility survey Periodic survey of a representative sample of
public and private health care facilities within
the country.
HUMAN RESOURCES Health Training Institution Periodic assessment done at health training
Assessment institutions to determine number of health
personnel
Health facility assessment Periodic assessment of public and private health
care facilities within the country
MOH Payroll Payroll listing provides list of personnel currently
on payroll and therefore in active employ
Labor Force Survey Survey that provides list of personnel currently
employed
Registry of professional Registry usually contains information on health
bodies and regulatory boards professionals and regulatory boards
INFRASTRUCTURE Registry of professional Registry usually contains information on health
regulatory entities facilities that are registered and functional
Assessment of facilities Periodic assessment of physical conditions of
public and private health care facilities within
the country
SUPPORT SYSTEMS Multiple data sources Periodic assessment all support systems of
depending on the support public and private health care facilities within
system. e.g. logistics the country
HEALTH FINANCING & Assessment or survey, Questionnaire usually as add on to other
PARTNERSHIP partner reports, and surveys;
coordination meetings Reports from partners presented at meetings
This section presents the reporting linkages and data flow in general – from the community to
health facilities through CHTs to Central MOH/SW (See Diagram below). Specific data flow for
each data source will be described in detail in the M&E AND RESEARCH Implementation Plan
Reporting to MOH/SW
An effective national M&E AND RESEARCH system requires that data flow structures and
reporting mechanisms are clearly defined to avoid double counting. The Diagram below is a
pictorial presentation of data flow and the information linkages from the community level (&
health facilities) to central MOH/SW.
35
Figure 5. MOH&SW Routine Data Flow Chart
GOL
All Community Health Volunteers (CHVs), Trained Traditional Midwives (TTMs), and Traditional
Midwives (TMs) within the catchment area of a health facility are to report community based
health data to the certified midwife and/or the Community Health Volunteer Supervisor based at
that health facility on a monthly basis and submit same to the OIC.
All health facilities OICs within a given district are to report to the District Health Officer (where
applicable) on a monthly basis. In an instance where there is no DHO, all health facilities are to
report directly to the CHT on a monthly basis.
Reporting from District Health Officer (DHO) to County Health Team (CHT)
All DHOs within a given County are to collect reports from the various health facilities OICs within
their respective districts and submit to the CHT on a monthly basis.
All NGOs, Private, Faith-based and Concession health facilities, operating in the counties are to
report copies of their health data to the CHT on a monthly basis.
36
Report from CHT to central M&E AND RESEARCH/HMIS
All data units of the CHTs are to submit health data/report to the central M&E/HMIS through their
respective CHOs on a monthly basis.
Reporting from M&E and Research to PCT/MOHSW, NDS and National Programs
M&E and Research will process, analyze and interpret data from all 15 counties and report directly
to PCT/MOHSW and will also share data received from the various counties with NDS and National
Programs (including Mental Health, EPI, ONCHO, Environment Health, Family Health, etc). The
NDS/Supply Chain and National Programs will then share reports with their respective Donors and
Partners.
Quarterly Service Coverage Report: This report provides information on coverage statistics
per BPHS area, and is essentially based on the main interventions as envisaged in the National
Plan. It therefore depends largely on the information that the CHTs submit and subsequent
analysis report produced by the M&E and Research Unit at the MOH/SW.
Ad hoc reports: In addition to the specific information products listed above, some
stakeholders/partners might have specific information needs at some stage. Although the MOH/SW
encourages the use of existing information products, ad hoc ones assist if there are any specific
and ad hoc information needs that are not covered in the above information products. Such a
request is made in writing to the MOH/SW.
37
8. Dissemination and use of information
The CHTs will undertake dissemination of M&E and Research findings in their respective counties
to NGOs, Private, Faith-based and Concession organizations and communities through organized
meetings with county development committee, etc
Health information Storage – the appropriate technology will be put in place for the secure
storage of health-related data. Storage devices that contain sensitive information will be kept away
from unauthorized access.
Health information availability – the appropriate technology will be in place to ensure the IT
system (computers, servers, and internet connection) is on-line and ready for access at all times.
This will involve the use of the following backup systems:
Power backups – the use of uninterruptible power supply (UPS) units to ensure that IT
systems that hold data are available at all times, especially in the case of power outages.
Data Backup – the use of external backup drives for storing daily system and files
backup, which will be stored at an off-site location as part of disaster recovery plans and
for systems’ restore.
Equipment backup – purchase of additional spare parts of IT equipment for the
replacement, in the instance where equipment becomes faulty.
Health Information transmission (Data transfer) - electronic connectivity (via the
Internet and other networks) for the transfer of electronic data, through the use of CD-
ROMs, memory sticks, encrypted email, and secured file transfer protocol (ftp), or secured
38
web services that permits electronic communication among health workers will be secured
in order to safeguard the information from unauthorized access, use, and disclosure.
Information Security: The appropriate technology that will preserve the confidentiality,
integrity and availability of data will be established through:
Electronic security - depending on the location (county or central level) of where the
data will be stored, access to personal computers, laptops, and servers will be secured
through the use of passwords, or other means of securing (encryption) access to the
stored information. Data stored on local or wide area networks with large numbers of
computers or internet access (county and central levels) will use technologies such as
firewalls and routers to limit access to those entitled to the data. Different levels of access
will be created depending on different purposes for the information, known as “role-
based” access.
Physical Security – data and information stored in electronic format, that is data stored
on personal computers, laptops, servers, backup and other storage devices both at the
county and central levels will be physically secured, such as by being stored in a locked
cabinet, within a locked room, and within a secured building to protect against
unauthorized person having physical access to them.
The public health sector shall work in close partnership with all stakeholders in health including
private medical practitioners and complimentary health care providers. The Ministry of Health and
Social Welfare shall continually seek the opinion of health service users in planning,
implementation, monitoring and evaluation of all health programs, projects and activities at central
and the peripheral levels. The table below summarizes key roles and responsibilities of
stakeholders in monitoring and evaluation of the National Health Plan.
The Role of the Central Monitoring & Evaluation and Research Division
The Central M&E and Research Division has the mandate of coordinating monitoring and
evaluation of programs and health services in Liberia.
Coordinate, supervise, and provide technical assistance to track progress of the NHP
activities at all levels.
Develop a national coherent plan for monitoring progress and evaluation of outcomes of
the implementation of the NHP and a standardized data collection plan, analysis, simplified
and comprehensive reporting format with collaboration amongst partners/stakeholders.
Supervise CHTs and facilities to audit and verify the credibility of data for analysis,
reporting and establish data quality assessment protocols to coordinate and guide data
collection and analysis for quality assurance.
Generate and disseminate simplified (user friendly) national information products. The
MOH/SW will ensure and facilitate the annual national disseminations and review activities;
Coordinate and support capacity building and training at central and county levels to
ensure that the M&E systems at the CHT level are functional.
Organize periodic Integrated Supportive Supervision visits and reviews of county-based
M&E systems and develop capacity building plans.
39
Central and county Monitoring and Evaluation Unit’s Tracking Tools
Four different tools will be used by M& E Unit for monthly, quarterly, annually and 5 yearly
performance monitoring and evaluation purposes, as shown below. For each indicator, a separate
textbox will provide space to record a succinct conclusion or interpretation of the observed value.
The role of the Monitoring, Evaluation and Research Technical Working Group
The role and function of the National Monitoring & Evaluation TWG is to provide technical
guidance to develop and implement Monitoring and Evaluation framework and Plan at central and
county levels. This TWG will meet monthly to ensure the successful implementation of the M&E
plan. The M&E unit serves as the secretariat.
Terms of Reference
The TWG will report to the Deputy Minister for Planning, Research and Development
Provide technical guidance to develop the integrated national monitoring and evaluation
plan.
Provide technical support for training needs assessment relating M&E and HMIS
Provide technical support for M&E and HMIS training needs assessment and capacity
building.
Develop SOP and trainers guide relating to M&E and HMIS
Identify and prioritize core list of national indicators (program specific, PRS and health
system performance indicators)
Develop data collection plan and schedule at central and county levels.
Advise on Integrated Supportive Supervision and review processes in relation to the
implementation of the data collection at all levels (this require further study though)
Advice on systematic Integrated Supportive Supervision and review processes as it relates
to the data collection implementation.
Ensure that data collected is disseminated to all stakeholders
Regularly evaluate the functions and quality of the M&E system
Advocate the use of information for decision-making
Advocate for the establishment of an independent clearing house for health related
research
Provide technical support to operational research at the Ministry
The CHTs will collect BPHS activities data at county level and monitor program activities and
indicators (progress, output, proxy and outcome).
Register and submit names of NGOs and CBOs involved in the health sector activities in the
counties to the MOH/SW; develop a database to track all partners in the health sector;
submit reports to the CHO.
Coordinate Integrated Supportive Supervision of M&E at the county level; coordinate and
supervise M&E at the county level.
40
Facilitate and demand that standardized forms be used for data collection from the
NGOS/CBOs; Facilitate and ensure the use of standardized forms for partners and health
facilities and submit monthly data forms to CHD.
Sensitize and advocate for the use of information products for decision making.
Disseminate information from the MOH/SW and sensitize partners at the county/community
levels.
Other stakeholders such as the research organizations, the private sector, and the development
partners are expected to:
Conducting periodic reviews of progress towards achieving the national health plan is essential
to the planning process. The reviews allow identification of operational challenges, best
practices; lessons learned and lead to preparation of a work plan for the subsequent period.
Since 2008, the Health Services Department has been organizing quarterly health sector review
meetings with the County Health Teams. The Planning Department participated in the meetings,
but was not closely involved in defining or implementing its recommendations. Similarly, one
Annual Health Sector review meeting was conducted in 2008, with better coordination between
the Health Services and Planning Departments.
For now and because the CHTs lack the capacity to conduct their own review meetings, the
MOHSW central office should conduct review meetings until the CHT capacities have been built
to conduct review meetings. However, it is proposed that the review meetings be held bi-
annually at the central level and quarterly at the county level once the capacities of the counties
have been built. This would be prudent to allow the counties time for the implementation of the
National health plan and will also give time at the central level to prepare for a more thorough
review of the implementation processes aiming at improving the efficiency and effectiveness of
the National Health Plan.
What to review
The National Health Plan sets forward a framework for shifting from humanitarian to
development and from vertical to integrated health systems development. This framework is
41
based on five components that will be reviewed during the quarterly and annual review
meetings:
The Process
A planning and review committee should be established to include Health Services, Planning and
Research, and administration. The establishment of such a committee will lead to a more
efficient, effective, and impactful implementation of the national health plan. The body will
create a more coordinated feedback loop among Central departments, programs, County Health
Teams and partners.
The planning and review committee shall be the body responsible for organizing and
implementing the quarterly and annual review meetings. The committee will be responsible for
implementing strategies designed for the review, coordinating all meetings, and ensuring the
system of feedback is established between the central level, county level, and other partners.
The committee will also establish the costs of the review and locate funding.
Members: members of the planning and review committee will be responsible for organizing
quarterly and annual review meetings. The membership shall be but not limited to the following
the department of health services, department of planning, administration and health sectors
partners with M&E Unit serving as secretariat.
The Steering Committee will be the appropriate mechanism to follow up on the implementation
of the recommendations of the review. Findings and recommendations of the review meetings
will be handled as follows:
By small technical groups and task forces: In this approach several working groups
or task groups will look at the results of the review and endorse what they see as main
findings and recommendations.
MOH/SW at central and county level: will develop an action plan with a timeline for
the implementation of recommendations. A final report will be presented in the next
review meeting.
The Ministry shall conduct mid-term evaluations of the progress made in accomplishing the
national health plan. In the remainder of the current five year plan, two evaluations shall be
42
conducted. The first, in 2009, shall be a mid-term evaluation that will be carried out by internal
evaluators. The second, in 2011, shall be a final evaluation that will be conducted as a mixed
review by an external evaluator in collaboration with an internal evaluation team. The results of
the second review will be used to plan for the next five year plan/policy review.
The goals for the evaluation of the national health plan are:
The National Health plan is currently in its implementation stage. It has been implemented from
2007 to early 2009 in number of counties. One annual review meeting of the health system was
held July 14, 2008. CHTs of various counties presented progress reports on the implementation
of the National Health plan strategies. There is still much to be done to complete the
implementation of the National Health Plan. The BPHS which is the cornerstone of the National
Health Plan has been implemented in 40% of functional health facilities (BPHS Review, Dr.
Bernice Dahn, 2008).
The MOH/SW and partners will agree on the organization and conduct of the NHP. Once agreed
upon, the MOH/SW shall contract individual(s)/ firm(s) to develop the national evaluation plan
for the health sector. This plan will serve as the framework for the Health Sector evaluation
(HSE). The plan will specify the evaluation scope, the implementation and data collection
methodology, and the TOR for the evaluators and the advisory board. Integrating the principles
of the evaluation framework into all the Health System operations will stimulate innovation
toward outcome improvement and will allow us to detect Health Plan effects. More efficient and
timely detection of these effects will enhance our ability to translate findings into practice.
The health sector evaluations will be guided by the national health plan framework (PBHS, HR,
Infrastructure, Health Financing and Partnership and Support Systems). The evaluation shall be
organized to cover the following (thematic) areas:
43
12. Strengthening of the M & E and Research Systems at all MOH/SW Levels
In line with the MOH/SW mandate to coordinate the national M&E and Research and having one
national M&E System, the execution of the national plan for the M&E system requires
commitment and partnership. The following will be necessary for the successful execution of the
M&E system:
Developing a national operation plan for data collection and reporting at the national and
county levels;
Strengthening the Department of Planning to receive and coordinate the distribution of
the M&E support coming from different development partners and international NGOS;
Strengthening the national M&E System including HMIS and other sub-systems;
Establishing and strengthening the M&E units at the CHTs;
Rolling out capacity building / training activities on M&E at the county levels. An M&E
Training manual will be developed and a training work plan agreed upon by the
stakeholders.
The goal is to enhance the knowledge and skills of M&E persons to effectively coordinate,
conduct data collection and analysis, and prepare information products. The specific activities
include the following:
development of the M&E training program/manual in collaboration with the training unit;
Training of the M&E focal points in relevant M&E /data technology (PDA);
Organizing national and county level seminars and workshops for the exchange of new
ideas in M&E.
Institutional Strengthening
The specific areas of interventions are included in the HMIS document. These areas include:
44
Developing the CHTs level M&E annual data collection plan – with clear activities and
timeframe;
Establishing a baseline for core National indicators, where there are not available;
Developing and building consensus on a standardized, user friendly data collection
instrument (s) and reporting formats;
Establish Data Quality Assessment Protocols and coordinating Integrated Supportive
Supervision.
Organize dissemination workshops in all the counties for sharing good practice and
lessons learned
Below is a national implementation workplan with an indicative budget for M&E systems
strengthening? Specific operational budget for M&E annual work-plans at the county level will
have to be developed.
45
Table 12. Implementation Plan for Strengthening of the M&E and Research Systems
Areas for interventions 2009 2010 2011 Indicative Source of
Budget funding
(3 years)
Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
Harmonization and Implementation of the National M&E Plan
Developing and building consensus on a national work plan X X $20,000.00
for data collection and reporting at the national and district
levels;
Strengthening the Department of Planning to receive and X X X X $80,000.00
coordinate the distribution of the M&E support coming from
health and development partners (office space, staffing,
computers, scanner, digital camera, etc)
Hire 30 M&E officers, data managers (2 per county) X X $351,000.00
Hold biannual national and county M&E coordination X X X X X X $480,000.00
meetings. For Sensitization, advocacy and sharing of
information.
Facilitation and development of the M&E training manual X X X $10,000.00
Training the M&E staff at the national level X X X X X $75,000.00
Training the M&E staff at the county level X X X X X $400,000.00
Training of the M&E focal points at national level in relevant X X
M&E / data technology (PDA)
Organizing seminars and workshops at national and county X X X $105,000.00
levels for the exchange of new ideas in M&E.
Sub-Total $1,521,000.00
Institutional capacity building
Maintain and upgrade DHIS database at national & in 15 X X
counties to include Community based programs
Procurement of computers, PDAs, telephone and internet X X
facilities.
Develop and disseminate integrated supervision checklist X X X
(applying the PDA software)
Establishing 15 county-based resource center X X X $450,000.00
Capacity Building for vertical programs M&E
46
Areas for interventions 2009 2010 2011 Indicative Source of
Budget funding
(3 years)
Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
Establishing a baseline for core National indicators, where X X X X $5,000.00
there are not available
Establish Data Quality Assessment Protocols Technical X X X $50,000.00
Assistance
Sub-Total $555,000.00
Strengthening coordination of health-related Research and Surveys
Develop a national research strategy X X $50,000.00
Carrying out of operational research (research grants) X X X X $250,000.00
47
Bibliography
2. LISGIS. (2007). Liberia Demographic and Health Survey. Monrovia: Liberia Institute of Statistics and
Geo-Information Service (LISGIS), Statistics House.
3. NACP. (2008). ANC, HIV/AIDS' sero-survey. Monrovia: National AIDS Control Program.
4. Economist Intelligence Unit Liberia Country Report. September 2006.
5. Government of Liberia, USAID, WHO. Liberia Sector Rapid Assessment Validation and Strategy
Design Workshop 2006.
6. Government of Liberia and UN Republic of Liberia: Comprehensive Food Security and Nutrition Survey.
2006.
7. Government of Liberia and UNDP Liberia: National Human Development Report. 2006.
8. Government of Liberia. Interim Poverty Reduction Strategy. Final Draft. 2006.
9. International Monetary Fund. Liberia. First Review of Performance under the Staff-Monitored Program.
September 2006.
10. Msuya C. and Sondorp E. Interagency Health Evaluation Liberia. Final Report. September 2005.
11. Republic of Liberia. Comprehensive Food Security and Nutrition Survey (CFSNS). 2006.
12. Republic of Liberia. Ministry of Health and Social Welfare. Rapid Assessment of the Health Situation in
Liberia. June 2006.
13. UNDP Liberia Common Country Assessment. 2006.
14. UNICEF The Official Summary of the State of the World's Children 2006.
15. United Nations and World Bank. Liberia Joint Needs Assessment. Sector Working Paper. Health and
Nutrition. February 2004.
16. World Health Organization. World Health Statistics 2006.
17. Ministry of Health and Social Welfare (2009), HMIS Policy
18. Ministry of Health and Social Welfare (2009), HMIS Strategy
48