Monitoring, Evaluation and Learning (Mel) Guide: Using MEL To Strengthen Your Organisational Effectiveness
Monitoring, Evaluation and Learning (Mel) Guide: Using MEL To Strengthen Your Organisational Effectiveness
Monitoring, Evaluation and Learning (Mel) Guide: Using MEL To Strengthen Your Organisational Effectiveness
1
Contents
Introduction ................................................................................................................. 3
What is MEL ................................................................................................................. 4
Organisational MEL .................................................................................................... 5
Project MEL ............................................................................................................... 5
The Monitoring, Evaluation and Learning cycle ............................................................. 7
Planning for MEL ....................................................................................................... 8
Key Elements of MEL ................................................................................................. 9
MEL Planning Tools ...................................................................................................10
Data ...........................................................................................................................13
Types of data ...........................................................................................................13
Sources of data ........................................................................................................13
Methods for data collection ........................................................................................14
Ensuring quality of data ............................................................................................14
MEL In Practice ............................................................................................................16
Who should be involved in your MEL work? .................................................................16
Who will manage or oversee MEL? .............................................................................16
How much will your MEL cost? ...................................................................................16
How much time might it take? ...................................................................................17
Working well with consultants ....................................................................................17
Learning ......................................................................................................................17
How can you develop your organisations skills further? ...................................................18
Resources ...................................................................................................................19
TEMPLATE – PLANNING MEL ........................................................................................21
2
Introduction
This MEL guide has been developed to support your international development organisation
to increase knowledge, confidence and success in Monitoring, Evaluation and Learning
(MEL). It aims to strengthen your organisational MEL practices; enhance your capacity to
become more effective; and maximise your impact; through review and learning with
partners and the communities you work with. Additionally, it will equip your organisation to
meet funders’ requirements on MEL.
You do not need to read this guide from cover to cover. The table of contents will
guide you to sections that may address your specific needs. It is however recommended
that you read section 2 if your organisation is new to MEL.
3
What is MEL
Monitoring and evaluation can help you to work out what difference you make through your
projects. Your organisation and partners can then go on and learn from this to improve your
performance in future.
MEL activity may already be taking place in your organisation; this guide will help you to
take stock of what you are already doing, put plans and systems in place and learn from it.
There are two levels of MEL, organisational MEL and project MEL.
Organisational MEL
At the organisational level, your MEL will focus on monitoring the performance of your
organisation as a whole. The areas considered in organisational MEL include:
Your organisation’s mission;
Your organisational structure;
Your planning, implementation and MEL processes;
The relevance of your projects to your mission; and
Your organisational capacity.
Organisation X’s mission is to create and promote opportunities for girls to become
empowered (personally, socially and economically) in order to be able to contribute to the
development of their communities.
Project MEL1
Project MEL focuses on the following for the specific project:
monitoring targets
results
changes
challenges
1
Due to the target audience of this guide we will refer to projects as a unit of intervention. Larger
organisations may look at MEL from a programme perspective and terminology may differ between
organisations.
5
This guide focuses mainly on Project MEL, but the tools can equally be applied to the
organisational level. There are however, many other tools that can be used for
organisational level MEL. Resources and guidance on organisational MEL can be found in the
resources section.
6
MEL: the basics
The Monitoring, Evaluation and Learning cycle
MEL is an integral part of project design, implementation and completion; MEL is done at all
stages within the project cycle The MEL cycle helps you to position MEL in the life of your
project, as shown on the diagram below.
The MEL cycle starts with the initial needs assessment, this is when you identify a
particular need or problem in a specific community which your organisation is able to
address. Ideas are generated by learning from existing projects, projects which have ended
or simply through conversations with partners, communities and funders. Then you develop
an in depth understanding of the need or problem, its underlying causes, and how it affects
the target communities. This is also the stage to think through potential solutions with your
partners and target communities. Having identified a need you then need to design a
project. The project will be the most credible intervention to address the problem. A critical
stage in the MEL cycle is MEL planning. This is where you, in collaboration with your
partners and target communities, decide the main MEL activities. You then go on to put in
place a realistic plan for undertaking them. MEL planning is a critical stage in the MEL
cycle where you should consider planning for the following MEL activities: conducting a
baseline, undertaking routine monitoring, undertaking mid-term and final evaluations;
deciding how to go about reflecting on information and learning from it to improve project
performance.
7
What is a baseline?
A baseline study will establish how a target group/area is prior to the implementation
of the project. It is so important to work from a good baseline as this is what you will
compare to your project results to show the change your project has made.
‘bad’ baselines affect the quality and validity of project MEL
During project implementation monitoring will be an ongoing process. You should allow
time for MEL activities in staff workplans and timetable regular checks with partners to
ensure monitoring is taking place. It is also recommended that you conduct a mid-term
evaluation. This allows you to take stock of where your project is and whether you are on
target to achieve project outcomes. At the completion of your project a final evaluation
should be conducted. This is the stage of the MEL cycle when you evaluate, in collaboration
with your partners and communities, whether your project brought about any real change in
the lives of the communities you work with. Once the final evaluation is complete you should
take time to review it and take note of any learning that can be gained. This stage also
provides opportunity to use this learning as input to new project ideas.
Additionally, it specifies:
how the information generated will be used;
the resources that will be needed to carry out MEL activities; and
how the project will be accountable to stakeholders in the process.
A MEL plan, resulting from the MEL planning process, is simply the working document
produced that guides you on how and when to implement MEL activities.
Before starting the MEL planning process, providing answers to the following questions
will enable you keep the planning exercise focused:
Some further questions you might want to consider at this stage are:
o What is the current status of project MEL? What has been done so far and by whom?
o Are there staff with specific MEL responsibilities who should be involved in the MEL
planning process?
o To what extent are project participants, stakeholders and partners involved in MEL?
8
o What are partners already doing and how does this relate to the project MEL?
o How do project staff perceive MEL?
o What are some of the issues/constraints involved in project MEL?
Good Practice
Consider how to actively involve partners
in the MEL planning process. Are your
partners local? Are they international?
How can you involve them in the process?
Indicators
Indicators are clues, signs or markers that measure one aspect of a project and show how
close a project is to its desired path and outcomes. They are observable and measurable
characteristics that can be used to show changes or progress a project is making toward
achieving a specific change. Indicators are relevant at all levels.
Figure 3 and table 5 provide examples of goals, outcomes, outputs, activities and indicators.
9
MEL Planning Tools
There are several ways/tools that can help you to visually represent your goals, outcomes,
outputs, activities, and related indicators. For clarity this guide gives two examples often
used in international development, there are however many others.
The tools identified in this guide are Weavers Triangle and Log Frame. These are explained
in the sections below with references for additional reading. Both tools allow you to present
your project intervention in a logical way. They help you assess whether the solutions you
have identified to tackle a particular issue will actually bring about change.
Weavers Triangle2The Weavers Triangle is a simple tool that can help you think about
your project and do planning, monitoring and evaluation of your work. It can both help you
develop your organisation’s strategy and plans for its projects, as well as work out whether
you are being successful in achieving what you set out to achieve. For small organisations a
Weaver’s Triangle is the best place to start.
2
The Weavers Triangle was originally developed by Jayne Weaver who worked for the Charities Evaluation
Services. For more information see www.ces-vol.org.uk and http://www.effectiveness. nidos.org.uk
10
Logical Framework
Similar to the Weavers Triangle, a Logical Framework (often referred to as a Log Frame) is
a tool which allows you to specify your project’s goals, outcomes, outputs and indicators to
show how they link and contribute to each next stage in a logical manner. It also allows you
to identify sources of data as well as the assumptions, i.e. the conditions necessary for
outputs and outcomes to be delivered. The Log Frame comes is in the form of a matrix, as
presented in table 5.
Theory of Change
The Theory of Change approach is an increasingly popular approach to development planning and
MEL, which aims at promoting critical thinking and learning, and to unpack and challenge
assumptions about how change happens. This Guide does not deal specifically with Theory of
Change – please see the Resource section for references.
11
Table 5: Example of a Log Frame
Project Description Indicators Data Sources Assumptions
Goal To reduce gender disparities in enrolment, Net enrolment rate School Records, National That all enrolments to
retention and completion at primary level (male/female) Statistics schools are recorded and
available
Outcome Girls claim their right to primary education Number of girls who School records, district There is not situation of
in districts X and Y in order to transition to complete primary six (in budget conflict, drought or famine
secondary school, gain qualifications, and comparison to boys) to disrupt the school year
improve their life chances and wellbeing. Girls, parents, community
leaders, teachers
Output Improved understanding of the importance Perception of the Girls, parents, community Parents and community
of girls’ education importance of girls leaders, teachers members are able to
education attend awareness raising
events
Improved quality of teaching at primary Number of teachers Training records, teachers Teachers are supported to
level supported to provide attend training
quality education
Activity Community outreach sessions on Uptake of outreach Girls, parents, community Staff can gain access to
importance of girls education sessions members target areas
Provide volunteer mentors to teachers Number of teachers Teachers Teachers are given
mentored permission by employer to
work with mentor
12
Data
What data you decide to collect, how and when are key to the success of your project.
Including partners in data collection enriches the process by providing insight into findings.
Those engaged locally are the only ones who have intimate knowledge of factors that can
play a big part in affecting, and indeed distorting, both the data collection process and
results. These could be practical factors such as local holidays or transport limitations
affecting logistics, or sensitive cultural issues such as ethnic group divisions or gender
relations affecting how and who you speak to. Involving communities in MEL activities
proves your accountability to them as project participants as well as building community
MEL capacity and strengthening community ownership of your project.
Types of data3
There are two main types of data: qualitative and quantitative. Quantitative data is
numerical data, i.e. data that can be counted for example numbers, amounts, proportion.
Qualitative data is descriptive, expressed in words or visual/auditory images, and gives a
more holistic picture of a situation than one can get with quantitative data. Qualitative
approaches are particularly useful for describing the range and nature of issues in a given
situation; for eliciting perceptions, beliefs, and explanations; and for spontaneous discovery
of issues emerging from the field. Qualitative data can best be described in words, opinions,
levels of understanding, etc. or diagrams and pictures or videos.
Sources of data
Data is collected from two main sources, primary and secondary. Primary sources refers
to data collected directly from target communities. Secondary sources refers to data from
written records.
3
Taken from: Mary Picard, Ph. D; 2000 – 2001, Materials for the Design, Monitoring and Evaluation (DME)
course, CARE Kosovo; P. 16 & 17.
13
Methods for data collection
The main methods for collecting data from primary sources are individual interviews,
focus group discussions, observation and surveys. Each of these methods can be
structured to provide qualitative or quantitative data. Secondary sources data is collected by
documentary review. This is simply reviewing relevant records and reports to find
information related to your data collection objective. Documentary reviews can help you
identify already existing information and keep data collection from primary sources focused
and at reduced cost. It is normally the first step in the data collection process where you
identify gaps in information.
Data collected from primary sources can be done in participatory and non-participatory
ways.
After deciding what data to collect and how, you will need to determine how frequently you
want to undertake MEL activities. This ranges from routine monthly tracking activities and
quarterly learning and reflection to annual reviews and evaluations.
Voice and Inclusion: ensure that the perceptions, beliefs and explanations by
target communities are included in the data to provide a clear picture of who is
affected by the project and how.
Appropriateness: ensure that the right methods are used to collect different
types of data.
Triangulation: ensure that data is collected using both quantitative and
qualitative methods, as well as primary and secondary sources of data This
allows you to cross check whether the information provided is a true reflection of
reality.
Contribution: develop questions that will elicit responses on how change
happens, the contribution of your project to bringing about change, and the
factors outside your project influencing or contributing to change.
4
Tom Barton; Guidelines for Monitoring and Evaluation: CARE Uganda; 1997
5
See www.bond.org.uk
14
Transparency: ensure transparency in your dealings with partners and
communities, for example transparency around project budgets and the use of
project resources.
You should ensure your field plan meets ethical considerations for data collection. These
include:
voluntary participation: this requires that respondents for data collection are
not coerced into participating in any study.
informed consent: this means that prospective respondents must be fully
informed about the procedures and risks involved in the study and must give
their consent to participate.
risk of harm : this requires that you do not put respondents in a situation where
they might be harmed as a result of their participation in the study.
confidentiality: this requires assuring respondents that information will not be
made available to anyone who is not directly involved in the study.
not raising expectations beyond what is in the control of your project.
Other ethical issues to consider throughout the MEL cycle relate to managing expectations
of project benefits; providing feedback to communities after data collection exercises; and
involving communities in key MEL processes.
15
MEL in Practice
Who should be involved in your MEL work?
Who should be involved in MEL will depend on your project strategy on participation and
partnership. Like many international development organisations, yours may already be
committed to participatory principles. Involving various stakeholders in MEL planning is a good
way to set a participatory tone right from the beginning of your projects. A MEL planning
process is also a good opportunity to engage your partners and communities and to familiarize
them with your projects. Additionally, it is an opportunity to build the capacity of partners and
target communities in MEL, and for you and your partners to understand your various roles and
responsibilities.
Generally, all those who are expected to benefit from MEL should be involved in the planning.
In practice, it may not be possible to include all project stakeholders in all steps of the MEL
planning process. In this case, consider setting up a MEL core group to work on the details.
Your project’s MEL plan and level of participation of partners and communities will help you
determine how much of the budget to allocate for each MEL activity.
16
How much time might it take?
The time spent on MEL planning varies from project to project and depends on the type of MEL
plan envisaged. It is common practice to devote one week, in a workshop, to plan for MEL, and
up to another week to finalise the outputs. This can take slightly longer if your partners and
communities participate remotely.
It is recommended that you develop a MEL plan before you start project implementation. This
will enable you figure out how long MEL activities might take, who has responsibility for which
MEL activities and how much it might cost. Table 3 below provides indications of how long
specific MEL activities might take.
Learning
The learning part of MEL is the one most at risk of being overlooked or forgotten.
Collection, analysing and making sense of data is a great learning opportunity for your
organisation. At the project level, you will learn what works well in a particular context or what
does not work well, which aspects of a project has more influence the achievement of results,
which strategies can be replicated etc.
At the organisational level, you can compare results across projects to determine which ones
contribute to achieving your organisation’s mission; you can also aggregate results from
different projects (depending on project similarities or in response to cross cutting
organisational indicators) to understand the wider reach of your organisation; or you can
aggregate learning from different projects to guide the strategic development of new projects
and funding opportunities.
You can facilitate both levels of learning through formal or informal learning and reflection
meetings of all stakeholders. It is good practice to share learning at both project and
organisational levels, and results achieved by projects (positive and negative) with your
partners, communities, and funders, in response to their needs and the benefits MEL provides
for them; as well as the general public (through your website). This strengthens accountability
and transparency.
The risk is often that the learning stays with the people who were directly involved and is not
shared with the organisation and is lost when the specific people leave. So You will need to
ensure there is appropriate documentation of processes and reports (paper based, photos,
17
videos etc.); and appropriate storage (filing - electronic, paper based) of MEL outputs in order
to keep learning within your organisation when key staffs leave.
NIDOS offers a number of services to help your organisation strengthen its effectiveness.
Mentoring NIDOS has a number of mentors, each with their own set of skills, and all with
extensive experience of working with international NGOs. The mentoring process can focus on
specific issues or identified needs within an organisation and help participants reflect and
question existing practice. Having a mentor means access to someone external who can bring
fresh eyes and help your organisation by asking useful questions, recording decisions and
making constructive suggestions. If you are interested in learning more about the mentoring
programme, please contact [email protected]
Training Each year NIDOS runs a number of training courses in response to member
feedback. Past courses include; producing good log frames, project management, social media,
effective budgeting, fundraising strategy and IATI. NIDOS can also organise tailored/in-house
courses; if you would like to find out more about this please contact [email protected]
Mel Working Group NIDOS members with a particular interest in MEL can join the MEL
Working Group. The group offers NIDOS members a space where they can share approaches to
MEL, discuss challenges, learn about innovative tools, invite external guests to present, as well
as organising ad-hoc training. All NIDOS members are welcome to join. If you are interested in
joining, or would like more information, please email [email protected].
Support for recruiting MEL consultants for specific assignments. NIDOS keeps a
database of external consultants that members can access (however, these consultants do not
work for NIDOS nor are endorsed by NIDOS). Consultants have specific expertise in a number
of areas, including MEL. Please contact [email protected] if you would like more
information.
18
Resources
Electronic platforms
Pelican Initiative: Platform for Evidence-based Learning & Communication for Social Change
The Pelican Initiative focuses on the central question: How can we learn more from what we do
while at the same time having the biggest possible impact on the social change processes in which
we engage?
Better Evaluation
An international collaboration to improve evaluation practice and theory by sharing and
generating information about options (methods or processes) and approaches
Participatory Methods
The Participatory Methods website, run by the Institute for Development Studies, provides
resources to generate ideas and action for inclusive development and social change.
A Step-by-Step Guide to Monitoring and Evaluation (School of Geography and the Environment,
University of Oxford, 2014).
This resource is designed to help groups working on community led approaches to climate
change and energy conduct their own Monitoring and Evaluation (M&E). It aims to provide an
19
accessible background to the principles of M&E, together with selected links to resources and
approaches that may be useful for your group.
Handbook on Planning, Monitoring and Evaluation for Development Results (UNDP, 2009).
The handbook was developed to help UNDP staff and partners to become more results-oriented
and improve their focus on development changes and real improvements in people’s lives.
Monitoring, Evaluation and Learning Guidance for International Applicants and Grantees (Comic
Relief, 2013)
This document provides guidance for applicants and grantees on Comic Relief’s expectations in
terms of MEL. The aim is to help applicants and grantees to understand Comic Relief’s MEL
requirements and the kind of support Comic Relief can offer, both during the application
process and over the life of the grant.
An evaluation of Big Lottery Fund’s Monitoring and Evaluation Policy for grant holders (Big
Lottery Fund, 2013)
This report summarises the findings of a research commissioned by the Big Lottery Fund to find
out how Fund projects were using the evaluation and monitoring element of their budget and
whether the allocation of up to 10% of grant budget towards monitoring and evaluation was
felt to be appropriate; the impact of wider learning for projects; and the effectiveness of BIG’s
approach.
Theory of Change
Understanding Theories of Change in International Development (Danielle Stein and Craig
Valters, 2012)
This is a review of the concepts and common debates within ‘Theory of Change’ material,
resulting from a search and detailed analysis of available donor, agency and expert guidance
documents.
A guide to monitoring and evaluating policy influence (Overseas Development Institute, 2011).
This paper provides an overview of approaches to monitoring and evaluating policy influence,
based on an exploratory review of the literature and selected interviews with expert informants.
Monitoring and Evaluation of Policy Influence and Advocacy (Overseas Development Institute,
2014)
This paper explores current trends in monitoring and evaluating policy influence and advocacy;
discusses different theories of how policy influence happens; and presents a number of options
to monitor and evaluate different aspects of advocacy interventions.
20
TEMPLATE – PLANNING MEL
Specific MEL activity Who’s involved Who’s responsible Expected dates Expected duration Cost
1 Needs assessments and
baselines, including
consultations with
communities.
2 Presentation of project to
communities.
3 Evaluations, including
consultations with
communities, analysis of
findings with communities,
and presentation of
preliminary/final findings to
communities for final
feedback and input.
4 Routine monitoring,
reflection and learning, with
partners and communities.
This may change during project implementation; that’s fine this is intended to be a starting point
21