SOLSTICE M&E Plan V2

Download as pdf or txt
Download as pdf or txt
You are on page 1of 55

SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

SOLSTICE

Monitoring & Evaluation Plan


Version 2.0

26 September 2018

i
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Contents

Acronyms ........................................................................................................................... iii

1 Introduction .................................................................................................................. 1

1.1 Purpose of this plan ................................................................................................ 1

1.2 Project summary ..................................................................................................... 2

2 SOLSTICE Theory of Change ...................................................................................... 5

2.1 General approach ................................................................................................... 5

2.2 Full Narrative .......................................................................................................... 6

3 Logical Framework ..................................................................................................... 10

4 Indicators .................................................................................................................... 16

5 SOLSTICE as an equitable partnership..................................................................... 28

6 Evidencing Interdisciplinarity .................................................................................... 29

7 Developing and monitoring SOLSTICE network ...................................................... 31

8 Roles & Responsibilities ............................................................................................ 36

9 Data Flow .................................................................................................................... 37

10 Data Management ................................................................................................... 37

10.1 Storage................................................................................................................... 37

10.2 Analysis ................................................................................................................. 38

10.3 Privacy................................................................................................................... 38

11 Appendices ............................................................................................................. 39

Appendix 1. Capacity development matrices ................................................................... 39

Appendix 2. Equitable partnership survey (PDF).............................................................. 52

Appendix 3. Network profiling questionnaire to be used in social network analysis (PDF) 52

ii
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Acronyms

SOLSTICE - WIO Sustainable Oceans, Livelihoods and food Security Through Increased Capacity in
Ecosystem research in the Western Indian Ocean, www.solstice-wio.org

RCUK Research Council United Kingdom, www.rcuk.ac.uk

GCRF Global Challenges Research Fund, www.rcuk.ac.uk/funding/gcrf/

M&E Monitoring and Evaluation

ToC Theory of Change

WIO Western Indian Ocean

iii
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

1 Introduction
SOLSTICE-WIO (www.solstice-wio.org) is a four-year collaborative project funded by the UK Global
Challenges Research Fund (GCRF) RCUK Collective Fund. Launched in October 2017, SOLSTICE-WIO
brings together local knowledge, international research expertise and state-of-the-art technologies
to address challenges facing the Western Indian Ocean (WIO) region in a cost-effective way. The
total budget for SOLSTICE is £8,000,000 covering the four year project duration which ends on 31
December 2021. The SOLSTICE team comprises researchers from the United Kingdom, South Africa,
Tanzania and Kenya with “in-kind” support from the wider Western Indian Ocean community.

1.1 Purpose of this plan


Purpose of this plan is to streamline and coordinate Monitoring and Evaluation (M&E) effort of the
SOLSTICE project. In particular, the M&E Plan aims to assist with the following:

• Identifying evidence needs to ensure proposed project outputs are leading to measurable
outcomes and impacts
• Identifying the stakeholders and user groups which need to be engaged to maximise project
impacts
• Strengthening program design: clear indicators of success allow Leadership Team to
optimise project activities and sharpen research questions
• Enabling clear communication: Leadership Team and project PIs are able to communicate
based on shared understanding of the different levels of outputs, outcomes and impacts.
• Maximising collaboration with existing networks and projects with shared vision and
objectives

The M&E Plan was created by the SOLSTICE M&E team (see Roles and Responsibilities) and includes
SOLSTICE Theory of Change, Logical Framework and the Key Performance Indicators. The M&E Plan
is a living and evolving document and is subject to change. All proposed changes to the M&E Plan
must be discussed with the M&E Team and approved by SOLSTICE Leadership Team. All versions of
the M&E Plan including the latest one can be found on the project website www.solstice-wio.org

1
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

1.2 Version updates

Version Milestone Key changes/additions

Theory of Change
M&E Plan v1 Created for 6 months
Log Frame
GCRF review (March
KPIs
2018)

The following new elements added:


M&E Plan v2 Revised for the Stage
Gate Review reporting
Assessing equitability
(September 2018) after
Assessing interdisciplinarity
receiving GCRF
Developing and monitoring SOLSTICE network
reporting requirements
Preliminary results of social network analysis
Capacity assessment matrices
Equitable partnership survey
Network profiling survey
Institutional Capacity KPIs (preliminary)

1.3 Project summary

Title SOLSTICE

Starting Date 01.10.2017

Duration 31.12.2021

2
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Partners 1. National Oceanography Centre (NOC), UK


2. Nelson Mandela University (NMU), South Africa
3. Plymouth Marine Laboratory (PML), UK
4. Institute of Marine Sciences (IMS), Tanzania
5. Tanzania Fisheries Research Institute (TAFIRI), Tanzania
6. Western Indian Ocean Marine Science Association (WIOMSA), Tanzania
7. Kenya Marine and Fisheries Institute (KMFRI), Kenya
8. Coastal Oceans Research and Development – Indian Ocean, East Africa (CORDIO-
EA), Kenya
9. Bayworld Centre for Research and Education (BCRE), South Africa
10. Rhodes University, Department of Ichthyology & Fisheries Science, South Africa
11. South African Environmental Observation Network (SAEON), South Africa
12. University of Cape Town (UCT), School of Economics, South Africa
13. Environment for Development (EfDT), Tanzania
14. Heriot-Watt University (HW), UK
15. The Scottish Association for Marine Science (SAMS), UK
16. South African Squid Management Industrial Association (SASMIA), South Africa

Target Area World leading research in Sustainable Living Marine Resources

Beneficiaries The primary beneficiaries are policy makers and resource managers tasked with
delivering sustainable management of marine living resources and climate adaptation
options.

Ultimate beneficiaries include:

- Commercial and artisanal fishers and their families will benefit from better yields and
greater stability of sustainably managed fisheries or from guidance on alternatives where
existing practices are unsustainable.

- Enterprises engaged in processing, marketing, distributing and exporting seafood will


benefit from higher yields and greater stability of optimally managed fisheries due to
improvements in fishing practices arising from research recommendations.

- The tourism sector, dependent on attractive, ecologically sound natural environments


and a culinary culture where seafood figures prominently, will benefit from their
continued availability.

- Fishers, their families and the general public will benefit from a better understanding of
the marine environment and the services provided by healthy marine ecosystems.

The primary academic beneficiaries of SOLSTICE-WIO will be international research


scientists working in the Western Indian Ocean on a wide range of marine topics,
including physical oceanography, marine ecology, fisheries science and economics, ocean
governance, socio-anthropology, climate change impacts, climate change adaptation and
mitigation.

3
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Cost £8,000,000

Funding Source Global Challenges Research Fund, UK

The overall aim of SOLSTICE-WIO is to strengthen capacity in the WIO to address


Goals
challenges of food security and the sustainability of livelihoods of the 60 million people
dependent on the region's marine ecosystems. This is reflected in the main objectives of
the project:

1. To grow marine environmental research capability to address challenges facing the


WIO region in a cost-effective way via state-of-the-art technology transfer, collaborative
environmental and socio-economic research and hands-on training;

2. To strengthen the capacity of UK marine scientists to apply leading-edge technologies


in developing countries, and work with regional and local experts to ensure that their
research addresses local and regional needs.

3. To strengthen the ability of WIO scientists to effectively deliver evidence-based


environmental and socio-economic information to support policy development and
implementation at national and regional levels.

4. To ensure future sustainability of marine research capability in the region by training


and mentoring early career scientists and post-graduate students from the WIO and by
developing on-line resources for use in distance learning and hands-on training of marine
scientists outside the partner organisations and beyond the duration of the project.

5. To ensure on-going support for an Ecosystem Approach to Fisheries in the WIO by


building lasting strategic research partnerships between UK marine science and regional
centres of excellence, between these centres and other WIO research organisations, and
between marine scientist and government agencies and NGOs mandated to deliver
sustainable development and exploitation of marine living resources in the WIO.

4
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

2 SOLSTICE Theory of Change


2.1 General approach
SOLSTICE is an international multi-disciplinary research project. Its goal to produce world-class
research, capacity development in the world-class research and evidence-based applications to
support the sustainable management of the living marine resources.

SOLSTICE Theory of Change present a conceptual view of the changes that SOLSTICE research and
capacity development must influence if it is to successfully contribute to development impacts for
people and ecosystems. It shows how and why SOLSTICE research is planned to help influencing
thinking, attitudes and behaviours amongst its user communities, so that long term improvements
can be brought about. SOLSTICE Theory of Change also draws attention to assumptions which are
made to allow this conceptual progression. All development initiatives are only as sound as its
assumptions which must be met for the goals to become achievable.

The Theory of Change recognises the SOLSTICE is located within a wider set of international, national
and local initiatives in the area of sustainable use of living marine resources, environmental issues
and impacts of climate change. These all offer opportunities for collaboration and co-production of
outputs.

It is important to remember that SOLSTICE Theory of Change is a conceptual model, not a literal
representation of a linear process. Its main purpose is to provide an analytical framework for
understanding the design of the planned research and the main type of changes the project must
achieve for long-term success.

The schematic of the SOLSTICE Theory of Change is presented in Figure 1. It describes a hierarchy of
changes starting at the lower level with Program Outputs achieving a number of Program Outcomes
leading to short-to-medium term changes. At this level ToC acknowledges that SOLSTICE is not the
only initiative working in the area of living marine resources and cutting edge technologies. Thus
attribution of the Outcomes and, further down the line, the Impacts to SOLSTICE outputs becomes
challenging.

It is important to remember that SOLSTICE is only a four-year project with a substantial component
dedicated to the world-leading primary research in marine environment and ecosystems. The process
of generating and publishing primary research results to act as evidence in the decision-making
process takes time comparable with the project lifespan. Thus the project Leadership Team and GCRF
as the project funding agency need to be prepared to evaluate the success of the project at the level
of the Outcomes rather than the Impacts. Furthermore, GCRF as a funding agency and NOC as a
leading organisation must find the way of carrying on Monitoring and Evaluation efforts beyond the
end of the project with the necessary budgetary implications.

5
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

2.2 Full Narrative


At the core of the SOLSTICE project design is the notion that insufficient research capacity,
inadequate monitoring programs and poor integration of science into fisheries and management
practices are the key roadblocks for the ecosystems approach to fisheries (EAF) in the Western
Indian Ocean, or put simply “You cannot manage what you don’t understand”.

Following requirements of the GCRF “Grow” call, SOLSTICE focuses its effort on producing four key
outputs:

Output1: Strong and self-sustaining scientific transdisciplinary WIO-UK networks producing high
quality intensive body of knowledge on ecosystems dynamics, human dependence on them, their
future trends and human responses

Output 2: Capacity developed in WIO to conduct interdisciplinary ecosystem research that meets
the needs of EAF, policy, industry and markets. Capacity developed in UK to meet the needs of ODA
research.

Output 3: Strong body of evidence produced by the network in each Case Study addressing societal
challenges and providing strategy options co-created with stakeholders and based on ecosystem
approach

Output 4: Transfer of cost saving technologies underpinning ecosystem research to overcome


limited investment into research infrastructure in WIO.

The first three outputs should be common to all projects funded by this call, while the fourth one
(technology transfer) is unique to SOLSTICE.

One important assumption will have to prove sound for the Output 1 and 2 to lead to desirable
outcomes, namely that upskilled researchers remain in employment either in their institutions or in
general area of management of the marine environment. If successful, we expect to achieve
Outcome 1: Networks grow, involve newly trained researches and stakeholder participation and
begin to attract new funding. It is important to recognise that attracting new funding is a critical
feature for a sustainable network and long lasting legacy of the project.

SOLSTICE recognises four key groups of players it must influence to realise its impacts: research
(academia and NGOs), private sector industry, policy makers and communities. Through its
engagement activities, SOLSTICE aims to overcome a wide spread resistance to employment of new
technologies and to achieve its Outcome 2: Key players become aware, receptive and show intention
to invest into new technologies and adopt ecosystem approach.

6
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

In providing a strong body of research evidence in each of the case studies and ensuring the optimal
pathways for communication between science and policy, SOLSTICE will achieve its Outcome 3: Case
studies provide strong body of evidence for action with a measurable outcome.

At this level ToC acknowledges that SOLSTICE is not the only initiative working in the area of living
marine resources and cutting edge technologies. Thus attribution of the Outcomes and, further
down the line, the Impacts to SOLSTICE outputs becomes challenging. However SOLSTICE recognises
that synergies with other activities in this area present excellent opportunities for achieving the long
term legacy and sustainability of the results.

It is important to remember that SOLSTICE is only a four-year project with a substantial component
dedicated to the world-leading primary research in marine environment and ecosystems. SOLSTICE
research underpins all four Outputs. The process of generating and publishing primary research
results to act as evidence in the decision-making process takes time comparable with the project
lifespan. Thus the project Leadership Team and GCRF as the project funding agency need to be
prepared to evaluate the success of the project at the level of the Outcomes rather than the
Impacts. Furthermore, GCRF as a funding agency and NOC as a leading organisation must find the
way of carrying on Monitoring and Evaluation efforts beyond the end of the project with the
necessary budgetary implications.

Nonetheless, the project Theory of Change and its Log Frame recognises achievable and measurable
impacts, albeit put them into the longer term category, potentially beyond the end of the project.

These are:

Impact 1: Sustainable networked centres of excellence established in South Africa and Tanzania

Impact 2: Institutional uptake of new technologies has a measurable impact on management of


living marine resources

Impact 3: Strengthened fisheries management

Impact 4: Uptake of research-based evidence in policy

Here again, we need to recognise critical assumptions which are a prerequisite to the project
achieving its impact and providing the long term legacy. SOLTICE recognises three such assumptions
at the level of Impacts:

1. Network is successful in leveraging additional funding.

2. Case studies move rapidly along impact pathways and become flagship examples

3. Key influential stakeholders endorse technology-based approach.

7
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Each of these assumptions is only partially based on the success of SOLSTICE effort and is under
influence of the factors outside of SOLSTICE control. For instance, success in securing funding
requires both: scientific excellence of the applicant (SOLSTICE objective) but also availability of
private or governmental funds to bid for (mostly outside of SOLSTICE area of influence).

ToC aspires to contribute to further Impacts which are next to impossible to attribute to SOLSTICE or
any other individual program:

1. Further underpinning of the blue economy and creation of alternative livelihoods

2. Sustainably managed marine ecosystems, economic grows, and food security.

It must be recognised, that SOLSTICE ToC is a generic framework designed to cover simultaneously
SOSLTICE general approaches and three very different Case Studies set in three countries with
diverse economic, scientific, social and cultural conditions. As the Case Studies progress, each of
them is expected to develop their own, much more specific Theory of Change.

8
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Figure 1. SOLSTICE-WIO Theory of Change diagram, to be read from the bottom up, following flow
arrows and touch-points.

9
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

3 Logical Framework

Summary of quantified SMART


targets for each level in the Indicators (quantitative and qualitative) Means of Verification Assumptions
log-frame

Pre-SOLSTICE: NOC/PML leading expertise in marine technologies: Robotics, Modelling, Remote Sensing and Environmental Economics; Existing model
outputs, data sets, software tools and socio-economic frameworks; NMU-NOC Innovation Bridge, Newton Chair and new institutional building at NMU to
Inputs work as a hub for technology transfer.

SOLSTICE: Man-hours and face-to-face interactions (meetings, training, collaboration activities) as detailed in the budget; Local interest and involvement from
9 WIO partners

Stakeholder engagement workshops; collaborative research (“learning by doing”); co-supervision of students; training courses; MOOC; filed work and
Activities
associated training; exchange visits

10
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Summary of quantified SMART


targets for each level in the Indicators (quantitative and qualitative) Means of Verification Assumptions
log-frame
• Number of collaborative cross-country • Web of Knowledge (WoK), internal project
Output 1: Strong and self- interdisciplinary papers manuscript tracker WIO partners have
sustaining scientific • Number of co-supervised (WIO-UK) MSc & PhD • Project descriptions incl. use of incentive to produce high
transdisciplinary WIO-UK students actively using technologies as part of technologies, supervisors and progress quality peer-reviewed
networks producing high quality projects available on project website; 6 month publications.
intensive body of knowledge on • Number of collaborative workshops project reviews
Outputs ecosystems dynamics, human • Number of exchange visits • Statistics submitted by project PIs; surveys Additional requirements
dependence on them, their • Number and bidding amount of further funding of visiting scientists and workshop for PhDs agreed by
future trends and human applications (successful and unsuccesful) participants partners aligning projects
responses • Statistics of participation in other program’s with SOLSTICE:
workshops, SOLSTICE invited speakers, publications, use of
presence on advisory boards technologies

11
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Summary of quantified SMART


targets for each level in the Indicators (quantitative and qualitative) Means of Verification Assumptions
log-frame
• Number of training courses and publications • Questionnaires before and after training
Output 2: Capacity developed in resulted from them sessions measuring change in knowledge Importance and potential
WIO to conduct interdisciplinary • Number of users of SOLSTICE-based MOOC • Statistics on MOOC attendance and use of of remote sensing data
ecosystem research that meets and MOOC-l off-line version for local training and modelling is
the needs of EAF, policy, industry • Number of SOLSTICE-aligned PhD and MSc • Reports and feedback on use of MOOC light appreciated and strong
and markets. Capacity co-supervised and degrees awarded in community management initiatives demand exist
developed in UK to meet the • Number of exchange visits and collaborative • Submission and successful defence of
needs of ODA research. publications and funding applications doctoral and master theses
resulting from them • Peer reviewed publications
• Number of success stories on use of • Policy reports Importance of economic
infrastructure provided to WIO • Presentations at conferences, departmental analyses in integrated
• Number of success stories for upgrading UK seminars, association meetings management
technologies to meet ODA requirements acknowledged and
• Number of responses in director-level adopted
consultation on research funding
• Number of briefs with recommendations on
increased capacity of WIO institutions and early
career scientists to generate research funding in
EAF
• Number of funding organisations receiving
SOLSTICE recommendations

12
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Summary of quantified SMART


targets for each level in the Indicators (quantitative and qualitative) Means of Verification Assumptions
log-frame
• Number of refs to case study reports and data • References to project webpage and
Output 3: Strong body of bases YouTube videos Meaningful indicators for
evidence produced by the • Number of stakeholder workshops • Number of workshops and participants the case studies will be co-
network in each Case Study • Number of participants representing • WoK, internal project manuscript tracker created with stakeholders
addressing societal challenges stakeholders • Meetings with relevant stakeholders to during the kick-off
and providing strategy options • Number of peer-review publications review final reports meetings.
co-created with stakeholders and • Number of case study synthesis papers • Citations of reports and peer reviewed
based on ecosystem approach • Number of translational products: (e.g. Policy papers in policy and management
Briefing Notes, infographics) co-developed with documents
stakeholders
• Number of reports on economic/management
strategies by case study (minimum in one case
study)

• Number of MSc & PhD students actively using • 6 months student project reviews
Output 4: Transfer of cost technologies as part of projects Project and publication
saving technologies • N of cross-discipline papers using support of • Review of publication by M&E group reviews are carried out by
underpinning ecosystem technologies • Project report on cost-benefit analysis M&E group
research to overcome limited • Cost-benefit analysis employing technologies employing technologies
investment into research • N of technical demonstrations in Tanzania and
infrastructure in WIO. SA

13
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Summary of quantified SMART


targets for each level in the Indicators (quantitative and qualitative) Means of Verification Assumptions
log-frame

Outcome 1: Networks grow, SOLSTICE delivers


involve newly trained researches 1. Regular network analysis survey carried out convincing demonstration
and stakeholder participation 1. Strength, extent and other key parameters of by M&E team for all network members of the technologies in case
network as measured by social network analysis (baseline, mid-project, end-project)
and begin to attract new funding studies

Outcome 2: Key players become 2. number of requests for briefs, demonstrations, Number of locally relevant
aware, receptive and show cost estimates of employing new technologies; 2. Interviews with key players; surveys applications of
intention to invest into new number of requests for training and capturing changing attitudes to technologies developed in
participation in case studies; number of technologies carried out by M&E team
Outcomes technologies and adopt collaboration with WIO
additional researchers and managers becoming
ecosystem approach involved into case studies and demonstrations experts are growing

Outcome 3: Case studies provide 3. Number of papers, conference presentations, There is sufficient capacity
strong body of evidence for reports presenting analysis of underpinning 3. Publication records, conference papers, in the region to
ecosystem dynamics and recommended desktop reviews
action with a measurable successfully use modelling,
management or policy actions.
outcome remote sensing and
economic decision
frameworks

14
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Summary of quantified SMART


targets for each level in the Indicators (quantitative and qualitative) Means of Verification Assumptions
log-frame

Impact 1: Sustainable networked 1. Number of new funding applications with the use 1. Regular survey of project participants and Network is successful in
centres of excellence established in of technologies and addressing aspects of food partner institutions carried out by M&E panel leveraging additional
South Africa and Tanzania security (baseline, mid-project, end-project) funding

Number of new PhD/MSc students using


technologies
Case studies moved rapidly
Number of developing country researchers that along impact pathways and
contribute to production of high quality became flagship examples
multidisciplinary research publications on the link
between ecosystem dynamics and food security

Impact 2: Institutional uptake of 2. Number of organisations directly and indirectly 2. Interviews with partner institutions and key Key influential stakeholders
new technologies has a using data products generated by technologies (i.e. stakeholders; statistics of data use, expression of endorsed technology-based
measurable impact on measured through data portal metrics, publications, interest and requests for evidence. References approach
Impacts management of living marine from non-academic publications (reports and
funding applications, MOOC participation) and use
resources the results as evidence in decision-making process briefs)

3. Number and type of cases where case study Upskilled researchers have
outcomes were evident in management decisions support of their institutions
Impact 3: Strengthened fisheries (mention in reports and strategy documents). 3&4. Continuous collection of data on the direct and remain in employment
management Number of managers trained in MOOC and MOOC- and indirect use of project outcomes in
light. Number of UK REF impact stories built of case management and policy. Surveys of MOOC
studies. participants.

4. Number and type of interventions where case Altmetrics to capture exposure and reach of
study outcomes were considered in policy relevant project outputs.
Impact 4: Uptake of research-
activities (mention in policy documents and climate 15
based evidence in policy
adaptation plans). Number of REF impact stories
built of case studies.
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

4 Indicators

Log-frame reference Output 1: Strong and self-sustaining interdisciplinary network

1. Number of collaborative cross-country interdisciplinary papers (including UK


Indicators
and at least one WIO partner)
2. Number of co-supervised (WIO-UK) MSc&Phd students actively using
technologies as part of projects
3. Number of collaborative workshops (including UK and at least one WIO
partner)
4. Number of exchange visits
5. Number and bidding amount of further funding applications

Baseline refers to the 5 year period prior to 1st October 2017


Baseline
1.
2.
3.
4.
5.

Target Target refers to the end of four year project period

1. 30
2. 15
3. 10
4. 10
5. 10

Data Collection Questionnaire sent to institutional leads

Tool Excel

Frequency 6 monthly

Responsible Sofia Alexiou

Reporting Details will be submitted by institutional leads to S. Alexiou every six months for
the reporting period.

Quality Control This group of indicators is straightforward and easily verifiable as workshops,
supervision and exchange visits are organised centrally. The questionnaire needs to
recognise that peer-reviewed publications have the following stages: submitted;
under revision; resubmitted; accepted; in press; published.

16
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Log-frame reference Output 2: Capacity developed in WIO to conduct interdisciplinary ecosystem


research that meets the needs of EAF, policy, industry and markets

Indicators 1. Number of training courses delivered


2. Number of publications resulted from training courses
3. Number of users of SOLSTICE-based MOOC (including off-line version)
4. Number of users of SOLSTICE-based MOOC-light for fishers and public
5. Number of PhD and MSc degrees awarded with SOLTICE co-supervision
6. Number of exchange visits resulting in collaborative publications and
funding applications
7. Number of success stories on use of infrastructure provided to WIO
8. Number of users of the data bases
9. Number of responses in director-level consultation on research funding
10. Number of briefs with recommendations on increased capacity of WIO institutions
and early career scientists to generate research funding in EAF
11. Number of funding organisations receiving SOLSTICE recommendations
Baseline refers to the 5 year period prior to 1st October 2017
Baseline
1.
2.
3.
4.
5.
6.
7.
8.
Target Target refers to the end of four year project period

1. 4
2. 10
3. 300
4. 50
5. 10
6. 10
7. 10; user stories (qualitative, interview-based)
8. 100
Data Collection 1. Reports by PIs delivering training
2. Reports by PIs delivering training
3. Statistics on MOOC attendance (web-based)
4. Report by institutional PIs on use of off-line version for local training
5. Report by institutional PI on state of PhDs and MScs
6. Questionnaires before/after training; reports by PIs
7. Interviews with users (as identified by institutional leads)
8. Web site statistics
Frequency 6 monthly

Responsible S. Alexiou

Reporting Details will be submitted by institutional leads to S. Alexiou every six months

Quality Control Quantitative indicators in this group are straightforward and easily verifiable.
Project members collecting qualitative indicators will attend M&E training including
approaches to collection of qualitative indicators.

17
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Log-frame reference Output 2: Capacity developed in UK to meet the needs of ODA research
1. Number of success stories for upgrading UK technologies to meet ODA
Indicators
requirements (conference abstracts, interviews)
2. Number of training courses attended by UK team members leading to step
change in ODA research capacity
3. Number of young career scientists applications for ODA funding as Co-I or PI
4. Number and amount of ODA-type funding applications
Baseline refers to the 5 year period prior to 1st October 2017
Baseline
1.
2.
3.
4.

Target Target refers to the end of four year project period

1. 5; user stories (qualitative, interview-based)


2. 5; user stories (qualitative, interview-based)
3. 3
4.10

1. Interviews with technology WP PIs, conference abstracts


Data Collection
2. Interviews with attendees (questionnaires)
3. Reports by PIs
4. Reports by PIs
Tool Excel

Frequency 6 monthly

Responsible S. Alexiou

Reporting Details will be submitted by UK WP leads to S. Alexiou every six months for the
reporting period.

Quality Control Quantitative indicators in this group are straightforward and easily verifiable. All
data are expected to be held either at NOC or at PML and thus easily obtainable.
Project members collecting qualitative indicators will attend M&E training including
approaches to collection of qualitative indicators.

18
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Output 3: Strong body of evidence produced by the network in each Case Study
Log-frame reference
addressing societal challenges

Indicators 1. Number of references to case study reports and data bases


2. Number of stakeholder workshops
3. Number of participants representing stakeholders
4. Number of peer-review publications addressing the case study issues
5. Number of case study synthesis papers
6. Number of translational products: (e.g. Policy Briefing Notes, infographics) co-
developed with stakeholders
7. Number of reports on economic/management strategies by case study
(minimum in one case study)
Baseline refers to the 5 year period prior to 1st October 2017
Baseline
1.
2.
3.
4.
5.
6.
7.
Target refers to the end of four year project period
Target
1. 20
2. 10
3. 100
4. 30
5. 3
6. 9
7. 2
Data Collection 1. Altmetrics, data submitted by institutional leads, website statistics
2. Project workshop reports
3. Project workshop reports
4. Project manuscript tracker, WoK (likely after the end of the project)
5. Project manuscript tracker, WoK (likely after the end of the project)
6. Report by project PIs
7. Report by project PIs

Tool Excel

Frequency 6 monthly

Responsible S. Alexiou

Reporting Details will be submitted by workshop organisers and institutional leads to S.


Alexiou on ongoing basis

Quality Control Quantitative indicators in this group are straightforward and easily verifiable.
Project members collecting qualitative indicators will attend M&E training including
approaches to collection of qualitative indicators.

19
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Output 4: Transfer of cost saving technologies underpinning ecosystem research


Log-frame reference
to overcome limited investment into research infrastructure in WIO.

Indicators 1. N of MSc & PhD students actively using technologies as part of projects
2. N of cross-discipline papers using support of technologies
3. Cost-benefit analysis employing technologies (indicator: costs saved)
4. N of technical demonstrations in Tanzania and SA
5. N of stakeholders attending technical demonstrations
6. Number of presentations on use of technologies in key partner and
stakeholder institutions delivered by SOLSTICE researchers
Baseline refers to the 5 year period prior to 1st October 2017
Baseline
1.
2.
3.
4.
5.
6.

Target Target refers to the end of four year project period

1. 10
2. 10
3. £1,000,000
4. 2
5. 50
6.30

1. Internal project tracker


Data Collection
2. Internal project tracker and WoK
3. Cost-benefit analysis
4. Technical demonstration report
5. Technical demonstration report

Tool Excel

Frequency 6 monthly

Responsible S. Alexiou

SOLSTICE socio-economist (cost-benefit analysis)

Reporting Details will be submitted by workshop organisers and institutional leads to S.


Alexiou on ongoing basis

Quality Control Quantitative indicators in this group are straightforward and easily verifiable.

20
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Outcome 1: Network grow, involve newly trained researches and stakeholder


Log-frame reference
participation and begin to attract new funding

Indicators 1. Strength, extent, centricity and other key parameters of network as measured
by social network analysis
Selection of network parameters for monitoring and their targets to be identified
after baselining exercise in April 2018
Note that social network analysis is an area of science rather than a monitoring
exercise. Ongoing development of the methodology is expected.

Baseline refers to the 5 year period prior to 1st October 2017


Baseline
1.
2.
3.
4.
5.

Target Target refers to the end of four year project period

1.
2.
3.
4.
5.

Data Collection A survey with the network profiling indicators will be completed by all project
partners, in-kind contributors and associated PhD students.
The questionnaire will cover: co-authored publications, co-supervised students, co-
organised workshops and co-developed funding applications.

Tool Social Network Analysis package, Matlab

Frequency Baseline (April 2018, backdated to 1st Oct 2017)


Mid-term assessment (March 2020)
End of project assessment (December 2021)
Long-term impact assessment (TBD)

Responsible K. Popova (analysis), S. Alexiou (data collection)

Reporting A project report on the results of the network analysis will be produced by K.
Popova and S. Alexiou

Quality Control The methodology was tested in the framework of international GULLS project. If
required, a professional advice on network analysis will be requested from Dr.
Ingrid van Putten, CSIRO who performed the analysis for GULLS.

21
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Outcome 2: Key players become aware, receptive and show intention to invest
Log-frame reference
into new technologies and adopt ecosystem approach

1. Number of requests for briefs and demonstrations from stakeholders


2. Number of requests for cost estimates of employing new technologies;
Indicators 3. Number of requests for training and participation in case studies;
4. Number of additional researchers and managers becoming involved into case
studies and demonstrations
5. Number of instances of WIO media coverage
6. Interviews with key players; surveys capturing changing attitudes to
technologies carried out by M&E team
Baseline refers to the 5 year period prior to 1st October 2017
Baseline
1.
2.
3.
4.
5.

Target Target refers to the end of four year project period

1. 20
2. 20
3. 50
4. 100
5. 20
6. qualitative

Data Collection 1. requests via website or project PIs


2. requests via website or project PIs
3. requests via website or PMO
4. 6-monthly reports by institutional leads

Tool Excel

Frequency 6-monthsly

Responsible S. Alexiou

6 monthly reports by institutional leads


Reporting
Ongoing recording of all requests via website

Quality Control Quantitative indicators in this group are straightforward and easily verifiable.

M&E panel may consider qualitative indicators in form of “Success stories” if a


particularly far-reaching event occurs. Project members collecting qualitative
indicators will attend M&E training including approaches to collection of qualitative
indicators.

22
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Outcome 3: Case studies provide strong body of evidence for action with a
Log-frame reference measurable outcome (e.g. projections of the squid dynamics leads to a proposal
of pension fund)

1. Number of papers, conference presentations, reports presenting analysis of


underpinning ecosystem dynamics and recommended management or policy
Indicators actions.
2. Success stories dedicated to each of the proposed actions (qualitative
measure)

Baseline Baseline refers to the 5 year period prior to 1st October 2017

1.

Target Target refers to the end of four year project period

1. 10
2. 5

Data Collection 6-monthly reports by institutional leads


Interviews; desktop reviews

Tool Excel

Frequency 6-monthsly

Responsible S. Alexiou

Reporting 6 monthly reports by institutional leads

Quality Control Quantitative indicators in this group are straightforward and easily verifiable.

Coverage of the success stories will be decided on the case-by-case basis

23
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Impact 1: Sustainable networked clusters established in South Africa and


Log-frame reference
Tanzania

1. Number and bidding amount of new funding applications with the use of
Indicators
technologies and addressing aspects of food security
2. Number of new PhD/MSc students using technologies
3. Number of developing country researchers that contribute to production of
high quality (journal IF>1) multidisciplinary research publications on the link
between ecosystem dynamics and food security (SOLSTICE network researchers
only, post-SOLSTICE Special Issue only)

Baseline refers to the 5 year period prior to 1st October 2017


Baseline
1.
2.
3.

Target Target refers to the end of four year project period

1. 10
2. 10
3. 30

1-3. Data submitted by institutional leads


Data Collection
Regular survey of project participants and partner institutions carried out by M&E
panel (baseline, mid-project, end-project)

Tool Excel

Frequency 6 monthly

Responsible S. Alexiou

SOLSTICE socio-economist (cost-benefit analysis)

Reporting Details will be submitted by UK WP leads to S. Alexiou every six months for the
reporting period.

Quality Control Quantitative indicators in this group are straightforward and easily verifiable. The
questionnaire needs to recognise that peer-reviewed publications have the
following stages: submitted; under revision; resubmitted; accepted; in press;
published.

24
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Impact 2: Institutional uptake of new technologies has a measurable impact on


Log-frame reference
management of living marine resources

Indicators
1. Number of organisations directly and indirectly using data products generated
by technologies
2. Number of cases where data products generated by technologies are used as
evidence in decision-making process (each accompanied by a “success story”)

Baseline refers to the 5 year period prior to 1st October 2017


Baseline
1.
2.

Target Target refers to the end of four year project period

1. 20
2. 10

Data submitted by institutional leads


Data Collection
Desktop reviews
Altmetrics (references from non-academic publications e.g. briefs)
RCUK REF impact stories

Tool Excel

Frequency 6 monthly

Responsible S. Alexiou

SOLSTICE socio-economist (cost-benefit analysis)

Reporting Details will be submitted by UK WP leads to S. Alexiou every six months for the
reporting period.

Quality Control Quantitative indicators in this group are straightforward and easily verifiable.

25
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Log-frame reference Impact 3: Strengthened fisheries management

Indicators
1. Number and type of cases where case study outcomes were evident in
management decisions (mention in reports and strategy documents),
accompanied by success stories.
2. Number of managers trained in MOOC and MOOC-light.
3. Number of UK REF impact stories built of case studies.

Baseline refers to the 5 year period prior to 1st October 2017


Baseline
1.
2.
3.

Target Target refers to the end of four year project period

1. 20 (Most likely to happen after the end of the project).


2. 100
3. 2 (Most likely to happen after the end of the project).

Data submitted by institutional leads


Data Collection
Desktop reviews
Altmetrics (references from non-academic publications e.g. briefs)
RCUK REF impact stories
MOOC statistics; Surveys of MOOC participants.

Tool Excel

Frequency 6 monthly

Responsible S. Alexiou

SOLSTICE socio-economist (analysis of management plans)

Reporting Details will be submitted by WP leads to S. Alexiou every six months for the
reporting period.

Quality Control A professional M&E advice/bought-in service might be though at this stage for the
Indicator 1.

26
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Log-frame reference Impact 4: Uptake of research-based evidence in policy

Indicators
1. Number and type of interventions where case study outcomes were
considered in policy relevant activities (mention in policy documents and
climate adaptation plans), accompanied by success stories.

2. Number of REF impact stories built of case studies.


Baseline refers to the 5 year period prior to 1st October 2017
Baseline
1.
2.

Target Target refers to the end of four year project period

1. 20 (Most likely to happen after the end of the project).


2. 2 (Most likely to happen after the end of the project).

Data submitted by institutional leads


Data Collection
Desktop reviews
Altmetrics (references from non-academic publications e.g. briefs)
RCUK REF impact stories
Survey of MOOC participants

Tool Excel

Frequency 6 monthly

Responsible S. Alexiou

SOLSTICE socio-economist (analysis of policies)

Reporting Details will be submitted by WP leads to S. Alexiou every six months for the
reporting period.

Quality Control A professional M&E advice/bought-in service might be though at this stage for the
Indicator 1.

27
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

5 SOLSTICE as an equitable partnership


General approach
Creating an equitable partnership is one of the key objectives of the project. We follow general
recommendations from (1) ESPA Brief “Research for development impact: The role of equitable
partnerships”, and the (2) UKCDS report “Building Partnerships of Equals”, and (3) personal
experience of Directors and PIs from ESPA and BELMONT projects in the ODA setting.

Measures used in SOLSTICE to facilitate and continue Equitable Partnerships

• Project partnership development and co-design meetings were run in WIO before official
start of the project (four workshops, NOC investment)
• Annual Advisory & Leadership (A&LP) panel meeting (April 2018) dedicated ½ day session (2-
day meeting) to discuss equitable partnerships giving voices to both WIO and UK institutions
• An Equitable Partnership Survey (EPS) (anonymous) was developed in May and then
executed in July 2018 (Full survey can be found in Appendix 2). This will be repeated at 24
and 36 months. It is the main mechanism that SOLSTICE uses to demonstrate equity
between its partners.
• Survey indicators and interim responses from the teams on challenges and benefits of
SOLSTICE partnerships are reviewed by the A&LP on annual basis.
• The survey aims to evaluate and provide evidence for the following characteristics of
equitable partnerships (as defined by GCRF):

1. Transparency (joint decision-making and clarity of communication)


2. Joint Ownership (capacity to co-design, co-produce and co-benefit from the research results
and project activities)
3. Mutual Responsibility (capacity of partnership to express and facilitate sharing of skills,
creativity, results, knowledge and promote the advancement of Case Studies to benefit of all
partners)

Pros and Cons of working in ODA partnership (overall assessment of equitable partnerships)

Although the above characteristics provide a good foundation to build equitable partnerships,
further steps are needed to ensure that equitable partnership actually works.

In ODA-style research, balancing the interests of Northern and Southern researchers, institutions,
disciplines, network members in different career stages, political agendas, and cultural differences is
not easy. And yet an equitable partnership can be defined as a partnership where each member’s
benefits outweighs the challenges such a partnership brings.

Hence key benefits and challenges were identified for WIO (by WIO partners) and for UK (by UK
partners) independent of each other during the first Annual Advisory/Leadership Panel meeting.
These benefits and challenges were then rated by each member of the partnership via the online
EPS (Appendix 2). Survey aims to assess if benefits outweigh challenges both for UK and WIO team
members.

SOLSTICE will continue to use the results from the EPS (questionnaire) as equitable partnership
indicators.

28
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

6 Evidencing Interdisciplinarity
Conducting Interdisciplinary Research (IDR) is a hotly debated topic articulated in a growing
volume of dedicated literature (including Nature). While there is varying consensus on what IDR
is, there is less clarity on (1) what constitutes quality in IDR, (2) how to evaluate and report on
IDR. Of course, once a project produces peer-reviewed publications, interdisciplinarity can be
evaluated more readily via cited bibliography. Until then, project interdisciplinarity can only be
reported using an IDR framework. Although many theoretical frameworks of IDR exist, we have
not come across any which seem appropriate in an ODA research setting.

We therefore developed our own pragmatic framework of interdisciplinarity based on the PIs
experience with similar ODA projects. This utilises recommendations in the ESPA policy brief ―
Interdisciplinary research for development impact: How can funders walk the talk?, and in Tang et
al. (2014) (The relationship between interdisciplinarity and impact, UKESRC report).

OPTIMUM (Objective-Practice-Team-Integration-Management-Uniqueness-Motivation) – A
pragmatic framework for designing, managing and evidencing interdisciplinarity in large projects.

Objectives

Most IDR aims to develop responses to social and economic problems. Narrow objectives are likely to have
solutions. Broader objectives less so with the generated knowledge only partly addressing them.

SOLSTICE mostly belongs to the first category of solution-oriented objectives driven by the case
studies, but they are set within the wider GCRF challenges.

Practice

Method of deploying human resources, i.e. deployment of a large number of part-time senior researchers vs. a
small number of full-time junior researchers. The former maximises number of disciplines in the project,
outputs and ability to expand by attracting additional funding but introduces challenges of coordinating and
integrating the network. The latter simplifies coordination, integration but minimises spread of disciplines and
potential for expansion and maximises the risk of relying on a few key individuals.

SOLSTICE is predominantly the former category, employing 33 senior scientists working part-time
and 7 full-time early career scientists. The larger number of senior scientists has been advantageous
attracting more than 20 MSc/PhD students into the SOLSTICE team.

Team

Refers to composition of team and cognitive distance between disciplines. IDR requires a combination of single
discipline experts as well as T-people (generalists). Warning: assembling single discipline experts
does not guarantee the interdisciplinarity (hence “I” – Integration).

SOLSTICE team comprises (from network profiling survey, Appendix 3):


• Geosciences 33%
• Technologies 13%
• Fisheries 23%
• Socio-economics, governance 16%
• T-shaped scientists 15%

29
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Integration

Integration of single disciplines into a multidisciplinary effort is expensive, time consuming, requires face-to-
face interactions, and occurs at expense of advancement of single disciplines (which are most likely to produce
highest ranking peer-reviewed publications; their advancement is also critical to the project).

SOLSTICE from an interdisciplinary perspective, integrates 3 key outputs: (1) Selection of case studies
(achieved), (2) “Foundation study” to identify gaps and set hypothesis (achieved first draft), and (3)
“Synthesis study” at end of the project to synthesize disciplinary efforts and interdisciplinary
connections and recommendations into a policy output.

We are also monitoring the growing number and strength of emerging interdisciplinary connections
using network profiling method (see Category 6) and graphical Social Network Analysis (M&E v2).

Management

This refers to how strongly, and by what methods, integration is driven. Management of integration is
expensive both in time and budget.

SOLSTICE drives integration strongly for a small set of clearly identified outputs from an early stage of
the project. We use a combination of a small number of actual interdisciplinary workshops with regular
virtual Webex meetings with coordinators identified both in WIO and UK. Connections between
specialised studies (projects) are constantly promoted.

Uniqueness

What are you doing to make it interdisciplinary? Uniqueness of your project is most likely to be its bestselling
feature for securing future funding. It is worth considering it as a special case worth extra effort.

SOLSTICE’s unique feature is marine robotics. An interdisciplinary study linking progress of marine
robotics in the region (ocean technology) to how marine robotics is perceived by local fishermen
communities (social science) has already emerged and is underway.

Motivation

Interdisciplinarity is often perceived to be bad for early career scientists (ECS) and takes more time to produce
output. Also, evidence suggests interdisciplinary papers are cited less than specialised. Single discipline
researchers are strongly motivated by advancement of their own disciplines.

SOLSTICE uses well defined Case Studies with strong societal significance and leveraging of funding in
similar interdisciplinary areas as key motivations

30
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

7 Developing and monitoring SOLSTICE


network
General approach

Scientific networks are at the center of all international programs, especially those concentrated
around “wicked problems” such as food security. Well-organised networks harness capacity to solve
shared but complex problems by unifying scarce resources, create Capacity Development
opportunities, and hasten the development of a common knowledge and understanding. A growing
volume of literature is now dedicated to scientific network analysis and theoretical frameworks. And
yet, what constitutes a research network and how a network becomes a network remains obscure in
many cases.

SOLSTICE aspires to developing a research network which is (1) international, (2) interdisciplinary
and (3) focused around the challenge of food security and sustainable use of living marine resources.
Desirable characteristics include research excellence, sustainability (ability to secure future funding)
and potential to address complex problems (research capability). The ultimate ambition is for this to
evolving into a global network.

SOLSTICE is not starting from scratch. Rather it combines several partnerships already in existence to
form an effective, well-focused research network. These include (1) a BELMONT Forum GULLS and
ESPA program GLORIA projects between the UK, South Africa, Madagascar, India, Brazil and
Australia, (2) the well-established and connected Western Indian Ocean Marine Science Association
(WIOMSA), and (3) links between national and inter-governmental institutions. Consequently, core
institutions in the SOLSTICE Food Security UK-WIO wide network are NOC, PML, NMU, WIOMSA,
TAFIRI, KMFRI, IMS and CORDIO.

Defining SOLSTICE network

The term “Network” is highly subjective, and so is a notion of what constitutes “network strength”.
Given the opacities of the issue, we have developed a new, simple but quantitative method that
allows us to objectively assess the building of the SOLSTICE output-based network and its
effectiveness. It also allows comparisons with other networks. In our method, connection strengths
between two researchers (in a network) is identified by the following:
1. Number of co-authored peer-reviewed papers

2. Number of co-authored reports commissioned by a regional or global NGO, World Bank or


regional/national government (Must be publicly available online and list main and contributing
authors)

3. Number of co-supervised PhD and M students (including connections between students and
supervisors)

4. Number of co-organised international workshops (defined: minimum of two countries


participating with formal record publicly available online listing organisers and participants).

5. Number of co-applicants for a funding grant (PIs, Co-Is) (includes successful and unsuccessful
applications)

31
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Evaluating the SOLSTICE network

Baselining of the SOLSTICE network has been completed via a SOLSTICE Network Survey (Appendix
2). After baselining, development of the network will be monitored using 6-monthly project KPIs
which contain the five criteria identified above for all the network participants. If the network
expands substantially beyond its original size, we may only be able to track two out of five
parameters (peer-reviewed publications and commissioned reports).

We employ Social Network Analysis (preliminary graphical analysis of baselining is shown below)
which demonstrates the following characteristics of the network: (1) expansion of the original
(baselined) network, (2) strength (efficiency), (3) interdisciplinary and (4) international spread
(including strength of the links between UK and developing countries). Social Network Analysis is
best reported via graphics but ResearchFish doesn’t allow this. Therefore, we simply using a set
network development indicators:
• Number of individuals involved into the network (67)
• Number of countries (4)
• Number of UK-WIO connections (39)
• Degree of interdisciplinarity (see Category 5)
• Averaged strength of WIO-UK (1.9) and interdisciplinary connections (effectiveness) (4.2,
preliminary estimate)

We may adjust the above indicators and/or method of their calculations as our network theory
develops.

Facilitating and promoting the SOLSTICE network

SOLSTICE is firmly focused on deliverables including peer-reviewed publications, production of


policy-relevant material, facilitation of collaborative workshops involving stakeholders and further
funding applications. These facilitate and develop the network strength and expansion.

As it develops, the SOLSTICE network will be promoted at WIO regional and international
conferences. We intend to publish the network development method, results and practical
recommendations in a peer-reviewed journal (Popova et al in prep).

Preliminary results of the SOLSTICE network profiling using Social Network Analysis

32
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Figure 1. Preliminary results of SOLSTICE network baselining via Social Network Analysis. Dots represent
network members (individual scientists). Lines represent connections between members (measured via co-
authored publications, reports, etc. See full criteria in the text). Members are coloured by disciplines showing
that interdisciplinary connections already exist in SOLSTICE.

33
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Figure 2. Preliminary results of SOLSTICE network baselining via Social Network Analysis. Dots represent
network members (individual scientists). Lines represent connections between members (measured via co-
authored publications, reports, etc. See full criteria in the text). Members are coloured by countries showing
strong clustering within countries and weak international connections.

Preliminary network statistics


Number of connected nodes: 67
Number of unconnected nodes: 24
Total connections: 574
UK-WIO connections
Number of connections: 19 (solely from nodes 46 (Katya) and 49 (Val))
- Tanzania: 3
- Kenya: 3
- South Africa: 13
Average weight of connections: 1.99
- Tanzania: 1.67
- Kenya: 1
- South Africa: 3.31
Minimum weight: 1
- Tanzania: 1
- Kenya: 1
- South Africa: 1
Maximum weight: 23
- Tanzania: 4
- Kenya: 2
- South Africa: 23

34
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Interdisciplinary connections

Number of connections:
Discipline ES OT F SE GS
ES - 5 3 2 12
OT 5 - 3 2 4
F 3 3 - 7 10
SE 2 2 7 - 8
GS 12 4 10 8 -

Average weight of connections:


Discipline ES OT F SE GS
ES - 3.4 2.67 6 3.58
OT 3.4 - 3.67 1.5 5.75
F 2.67 3.67 - 5.14 7.2
SE 6 1.5 5.14 - 2.5
GS 3.58 5.75 7.2 2.5 -

ES – Earth Sciences
OT – Ocean Technologies
F - Fisheries
SE - Socio-economics, governance
GS – Generalists or T-shaped scientists

Figure 3. An example of a network profiled for


GULLS project (a project similar in its size,
objectives and disciplines to SOLSTICE) and
connections between the project partners
(courtesy of Dr.I.VanPutten, CSIRO).

35
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

8 Roles & Responsibilities


Role Responsibilities

K. Popova SOLSTICE Co-Director

NOC Chair of the M&E panel


• M&E data analysis
• Applications of the social network analysis methods to impact evidence
• Regular reviews of ToC and log-frame
• Overall responsibility for M&E assessment and reports

S. Alexiou SOLSTICE Project Coordinator

NOC M&E officer


• Data collection, flow and archiving
• Coordination of reporting from WIO
• Preliminary analysis of data
• Collation of reporting material for baselining, mid term and final M&E reports;
• Annual reporting to GCRF

M. Roberts SOLSTICE Co-Director

NOC/NMU • 6-monthly review of indicators and their suitability for evidencing project outcomes
and impacts in the area of Science into Policy and Governance
• 6-monthly review of indicators and their suitability for evidencing of the impact in
the SA and TAN network clusters
• Production of the “Success Stories”

V. Byfield SOLSTICE WP4,5 Lead (Capacity Development and Communication)


NOC • 6-monthly review of indicators and their suitability for evidencing project outcomes
and impacts in the area of Capacity Development.
• Ensuring that the project web resources have adequate automated user reporting
systems.
• Ensuring that project data bases have adequate automated user reporting systems.
• Ensuring that MOOC user statistics is fit for purpose. MOOC reporting.
• Capacity development questionnaires.
• Training course evaluations.
• Institutional capacity baselining
• Production of the “Success Stories”

E. Papathanasopoulou SOLSTICE WP2 Lead (Socio-Economics)

PML • 6-monthly review of indicators and their suitability for evidencing project socio-
economic outcomes and impacts in the Case Studies
• Contribution to the baselining, mid term and final M&E reports

SOLSTICE M&E Panel will meet every 6 months (October and April) for reviews of data over the
reporting period and review of ToC, Log-Frame and formulation of the Key Performance Indicators.

M&E Panel will report annually to the Leadership Team and the Advisory Panel.

36
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

9 Data Flow

10 Data Management
10.1 Storage
Data collected during M&E process will be mainly stored in Excel spreadsheet where it can be
analysed. Some data will be collected through automated analytics, ie: number of downloads from
our website etc, which will then be exported to Excel and PDFs for analysis, interpretation and
reporting. All data will be stored on the NOC secure server, behind encryption firewalls. Files are
stored digitally on the NOC shared storage system, which is only accessible by NOC users and each
drive having its own permissions locking them down to specific users. These drives are backed up
nightly and held off site for up to a year.

37
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

10.2 Analysis
Majority of the data will be analysed in Excel. For the social network analysis Matlab tool box will be
used.

The interface between Excel network data base and Matlab will be streamlined to ensure smooth
operation of the analytical tools and clarified by October 2018.

The interface between Excel and data base usage statistics will be clarified by October 2018.

The interface between Excel and model output usage statistics will be clarified by April 2019.

The interface between Excel and MOOC usage statistics will be clarified by October 2020.

10.3 Privacy
SOLSTICE and its participant partners will abide by and comply with the EU’s General Data Protection
Regulation (GDPR), ec.europa.eu/info/law/law-topic/data-protection_en. Any personal data that
may be collected during M&E will processed lawfully, transparently, and for a specific purpose. Once
that purpose is fulfilled and the data is no longer required, it will be deleted.

38
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

11 Appendices
Appendix 1. Capacity development matrices
GCRF anticipate for the Capacity Development to happen at three levels:
Individual: involving the development of researchers and teams via training and scholarships, to design and undertake research, write up and publish
research findings, influence policy makers, etc.
Organisational: developing the capacity of research departments in universities, thinks tanks and so on, to fund, manage and sustain themselves.
Institutional: changing, over time, the 'rules of the game' and addressing the incentive structures, the political and the regulatory context and the resource
base in which research is undertaken and used by policy makers.

We use Research Capacity Assessment Matrices (RCAMs) as the main framework and operational tool to assess CD at the individual, organisational and
(partially) institutional level. The matrices follow the general structure:

• Levels of skills of individuals


• Access to infrastructure
• Access/Involvement to international expertise and networks
• Support at organisational Level
• Support at Institutional Level

These focus on:


• Remote sensing for marine ecosystems and climate change impact
• Ocean Model Analysis
• Marine robotics
• Biogeochemical monitoring
• Conducting interdisciplinary projects in EAF and sustainable use of living marine resources
• Conducting ODA research (UK)

39
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

No Category Please indicate the present status of your institution for each of the categories listed. Your responses should be specifically related to:
Use of autonomous marine robotic systems to monitor marine ecosystems and climate change impact
1 2 3 4 5 Simple test question for
Level 3
1 Skills to undertake No skills available to work Some basic skills available, Some data processing skills are Good understanding of Broad skillset available, competent skills If provided with raw binary file,
processing and with data obtained by but core training in data available from one or two oceanographic data processing are available in processing, analysis and multi-variable data from ocean
analysis of data robotic systems. processing and analysis is individuals but further training techniques, analysis and interpretation of new observations with gliders could you produce a
collected from required. in processing and analysis interpretation. Further training or the ability to lead on peer reviewed processed and quality
marine robotic Basic oceanographic data specific to marine robotics is assistance is required to develop publication in international journals. controlled dataset and produce
platforms interpretation skills are not required. Basic oceanographic interpretation into peer reviewed Expertise is being continuously broadened plots demonstrating changing
available. data interpretation skills are publications. and brought up to date via regular training. mixed layer depth and relative
available. Some mentorship is available location of the chlorophyll
from senior colleagues. maximum throughout the
deployment?
2 Infrastructure of No access to marine No access to robotic Able to participate in Access to marine robotic systems Direct access to Marine robotic systems If requested, could you plan the
marine robotic robotic systems or platforms and technical experiments with marine and relevant technical support either by ownership or via a national deployment of an ocean glider
systems and ability technical support. No support but limited access to robotic platforms undertaken from partners. Able to direct the facility, including access to technical to provide 1 month of repeat
to manage datasets access to data collected data collected by partner by partner institutes. Sufficient use of robotic systems to meet support. Data quality protocols follow transects between two fixed
to international from marine robotic institutes. Limited skills or skills available to manage (e.g. own science and technical internationally recognised standards. stations in the West Indian
standards. systems. infrastructure to manage provide quality assurance and objectives. Every effort is made Proven track record of reports/peer- Ocean, and deliver the resulting
observational data collected control) and securely store data to ensure high data quality reviewed literature presenting data multivariable data at a suitable
using robotic platforms. collected. following “best practice” gathered using marine robotic platforms. quality for use in a peer
guidelines. Research into future development of reviewed publication?
robotic systems is planned/ or undertaken.
3 Access to No contacts/collaborations Limited contact or Working in collaboration with Working in collaboration with Taking a leading role in international Can you host a workshop to
international with marine robotics collaboration with marine international groups to address international partners addressing groups that coordinate or develop provide training for local
expertise / networks international community or robotics international scientific or technical questions scientific and technical questions oceanographic research proposals scientists and technologists on
identified partnerships community (scientists and in the marine environment in the marine environment using associated with marine robotics. Provide the deployment and recovery of
with other institutions technologists). Attendance of using robotic systems. Have robotic systems. Able to request lead authorship of peer reviewed papers in ocean gliders using
using marine robotics. relevant personnel at developed suitable contacts assistance and training as international journals regarding operation internationally recognised
international meetings or within the marine robotics required. Representation on and application of marine robotic systems. expertise?
workshops that include international community to international groups associated Providing assistance and training to
emerging users of marine provide training and assistance with marine robotics. Co- international partners.
robotics. in operation and data analysis. authoring papers with
international partners.
4 Recognition and Use of marine robotic There is some appreciation of Use of robotics to conduct Some investment has been made The use of marine robotics to conduct Can you produce a cost-benefit
support at systems for ocean research the benefits of using robotic oceanic research is perceived into facilities and training of ocean research is highly valued by analysis of the use of marine
organisational level is not considered a systems for oceanic research as a valuable activity and personnel to help deliver local management and receives sufficient robotic systems for use in a
valuable undertaking for but it is insufficient for opportunities for local objectives with marine robotics funding to enable outputs to feed into future marine monitoring
policy and business significant investment. development have been to deliver policy needs and policy needs and to support business programme?
development. identified. business development with the developments without the need for partner

40
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

assistance of external partners. contribution.

5 Recognition and The use of robotic systems The use of robotic systems to Use of robotic systems appears Use of robotic systems and the A national facility or coordination Can you identify two current
funding by National to investigate marine investigate marine ecosystem in funding calls and national impact of climate change on the programme exists that enables open access nationally funded projects in the
/African/ Global ecosystem function or function or climate change marine programmes. Use of ocean are a regular subject of to marine robotics and technical support last year, where marine robotic
Institutions, policy climate change impacts impacts does appear in robotic systems is identified as funding calls or a required that is actively encouraged to be accessed systems form a critical
making and funding does not appear in funding funding calls and national one of the priority areas in component of funding calls. through targeted funding calls. component of the research
bodies calls and national marine marine programmes, but is capacity development by Strong national and/or programme?
programmes. identified as one of the areas national and international international capacity
where capacity development African bodies. development efforts exist in this
is needed. area.

41
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

No Category Please indicate the present status of your own skills (1-3) and your institution/country (4,5) for each of the categories listed. Your responses should be specifically related to:

Ocean Model Analysis* for marine ecosystems and climate change impact
*
Please note that this area refers only to analysis of existing models rather than new model development
1 2 3 4 5 Simple test question for Level 3

1 Computing Limited or no skills or Some basic skills are available Some data processing skills are Adequate skills in data analysis and Adequate skills in data analysis and If provided with a web link to a
facilities and data available to work but core training in data available but up-to-date and visualisation are available visualisation are available and expertise is 100Mb netcdf file containing 3D
processing skills with model outputs processing and visualisation advanced training in data analysis being continuously brought up to date monthly mean fields of Chl-a, would
is required and visualisation is required you be able to plot surface field, a
crossection and extract annual cycle
in contrasting locations?
2 Access to model No guidance material Some relevant material Relevant material and access skills Relevant material and access skills Models are used effectively to guide Given a 1 month notice, can you get
data relevant to or training on access and/or training is available are available but scientific expertise are available. Models are used policy and blue economy. Quality of the an access to suitable model output
the region and to models available but not enough expertise to is insufficient to interpret and use effectively to guide policy and blue models for the given region is assessed and produce a visualisation of a
analytical skills to implement it the models effectively industry and user feedback is regularly given to surface field of primary production
interpret the data international model developers. in your area and its future
projections under a variety of
climate scenarios?
3 Access to Don’t have any Have some contacts with Work in collaboration with groups Work in close collaboration with Work in close collaboration with groups Do you have an ecosystem
international contacts/collaboratio modellers (physical and/or in other countries doing ecosystem groups in other countries doing in other countries doing model modelling component in your
model development and analysis.
expertise / ns with ocean biological) in other countries model development and analysis. Co-author papers with these development and analysis. Co-author current project even if done by
networks (physical and/or and/or institutions and aware groups and can request papers with these groups and can request another institution?
biological) modellers of their effort and progress. assistance/training in access and assistance/training in access and analysis
analysis of their model output if
in other countries of their model output if required. Write
required.
and/or institutions funding proposals in collaboration with
these groups.
4 Recognition and Modelling is not There is some appreciation of Modelling is perceiveda Modelling
as is valued by Modelling is valued by management, its Can you name three institutional
support at perceived as a potential value of modelling valuable asset and it is seen as management, its use is encouraged use is encouraged and institutional projects or project proposals in the
but funding is insufficient
organisational valuable tool for but it is insufficient for being worth the investment funding is allocated last year where ecosystem
level policy and business investment modelling was a component?
development

42
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

5 Recognition and Ecosystem modelling Ecosystem modelling and Ecosystem modelling and analysis Ecosystem modelling and climate Ecosystem modelling and climate change Can you name three current
funding by and analysis of analysis of climate change of climate change projections for change projections for marine projections are a regular subject of national programs or national
National /African/ climate change projections for marine marine environment occasionally environment appear in funding funding calls or a required component of funding calls issued in the last year
Global Institutions, projections for environment occasionally appear in funding calls. Modelling is calls. Modelling is identified as one funding calls. Strong national and/or where ecosystem modelling was a
policy making and marine environment appear in funding calls and identified as one of the areas of the priority areas in capacity international capacity development component?
funding bodies do not appear in national marine programs. where capacity development is development by national and efforts exist in this area
funding calls and needed international African bodies.
national marine
programs.

43
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

No Category Please indicate the present status of your own skills (1-3) and your institution/country (4,5) for each of the categories listed. Your responses should be specifically related to:

Satellite remote sensing* for marine ecosystems and climate change impact
*
Please consider all the main satellite technologies and data types used for marine applications (ocean colour, SST, altimetry, SAR) when assessing status
1 2 3 4 5 Simple test question for
Level 3
1 Computing facilities and Limited or no skills or available Some basic skills are available Ability to use visualise and Adequate ability to process and Adequate skills in satellite data If provided with NetCDF data of
data processing skills to work with satellite data but core training in the use of process some types of satellite visualise marine data from most analysis and visualisation are SST, chlorophyll-a, SAR, gridded
marine data form different data is available, but up-to-date satellite technologies is available for all the main SSH and SSHA, and along-track
satellite technologies is needed and advanced training in the full available, and the techniques satellite technologies, these are altimetry, would you be able to
range of relevant marine data can be applied in synergy when frequently used in synergy to process and visualise at least 3
types is needed required improve available information, of these data types to reveal
and expertise is being features of interest?
continuously brought up to date
2 Access to satellite data No access to satellite data or Some relevant data is available Data access and analysis skills Relevant material and access Earth observation is used Given a 1 month notice, can you
relevant to the region information on how to analyse not enough expertise to use are available for some satellite skills are available for all the effectively in synergy with other get access to suitable satellite
and analytical skills to and interpret different data analyse and interpret it in technologies, but scientific main satellite technologies for information to guide policy and data to undertake a study of
interpret the different types. relevant contexts expertise is insufficient to marine monitoring. Data from a blue economy. Data from some of the environmental
data types identify, access and interpret all range of sensors are used to different sensors is used in factors influencing ecosystem
the different types of data guide policy and blue industry. synergy, to increase accuracy of variability in your region
relevant to marine ecosystem Algorithms are adapted to information, and feedback from
dynamics improve accuracy of some data users guides development of
products. new or improved algorithms.
3 Access to international Don’t have any Have some contacts with marine Work in collaboration with Work in close collaboration with Work in close collaboration with Do you include satellite data in
expertise / networks contacts/collaborations with remote sensing scientists in groups in other countries doing groups in other countries doing groups in other countries your work, even if the expertise
marine remote sensing. Co-
marine remote sensing other countries and/or who are involved in marine author papers with them and working with satellite data. Co- to analyse some of the required
scientists in other countries institutions and aware of their remote sensing can request assistance/training author papers with these groups data sets used come from
and/or institutions effort and progress. in accessing, analysing and can request assistance/training another group?
interpreting data products
if required. Write funding
where local expertise is lacking
proposals in collaboration with
these groups.

44
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

4 Recognition and support Satellite remote sensing is not There is some appreciation of Remote sensing is perceived as Satellite remote sensing is Remote sensing is valued by Can you name three
at organisational level perceived as a valuable tool for potential value of using remote a valuable tool that is worth the valued by management, its use management, its use is institutional projects or project
is encouraged but funding is
research, or for policy and sensing data, but it is investment needed to access insufficient to fully take encouraged and institutional proposals in the last year where
business development insufficient for investment in and analyse relevant data in the advantage of available data funding is allocated to ensure remote sensing was a
adequate infrastructure and future that relevant facilities and component?
support for skills development expertise are available
5 Recognition and funding Satellite remote sensing for Satellite remote sensing for Satellite remote sensing for Satellite remote sensing for Analysis and interpretation of Can you name three current
by National /African/ marine applications does not marine applications occasionally marine applications occasionally marine applications appear in marine satellite data are regular national programs or national
Global Institutions, appear in funding calls and appear in funding calls and appear in funding calls, and is funding calls. Developing subject of funding calls or a funding calls issued in the last
policy making and national marine programs. national marine programs. identified as one of the areas capacity to take advantage of required component of funding year where the use of marine
funding bodies where capacity development is global data archives is identified calls. Strong national and/or satellite data was a component?
needed as a priority by national and international capacity
international African bodies. development efforts exist

45
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

No Category Please indicate the present status of your own skills (1-3) and your institution/country (4,5) for each of the categories listed. Your responses should be specifically related to:

Biogeochemical monitoring of marine ecosystems and climate change impact

1 2 3 4 5 Simple test question for


Level 3
1 Capability and skillsets Limited or no skills available to Some basic skills are available Narrow multidisciplinary skillset Broad skillset available, good Broad skillset available, If asked to conduct regular
to undertake organise and implement but restricted to single available, some appreciation of understanding of competent analysis and monthly sampling would you be
multidisciplinary science fieldwork or collect and process disciplinary science (i.e. multidisciplinary science but up- multidisciplinary science, can interpretational skills, expertise able to plan, implement, analyse
samples chemistry or biology not both) to-date and advanced training in analyse and interpret diverse is being continuously broadened and interpret multiple
methodologies is required datasets but recognises training and brought up to date via parameters to produce an
is required regular training annual cycle of the upper
ocean?
2 Analytical infrastructure No analytical infrastructure Some analytical equipment Relevant equipment available Relevant equipment available. Analytical instruments are used If a visitor asked to run water or
and ability to produce available. No ability to analyse exists but range of measurable but scientific expertise is Instruments are used regularly regularly and effectively across water derived samples through
datasets samples or produce final parameters is limited. Limited limited. Instruments not used and effectively to address multidisciplinary projects. Data all of your existing instruments
datasets. No ability to conduct expertise available to use effectively or data quality is scientific problems following methodologies follow “best would you be able to do so
fieldwork (sample collection) instruments efficiently. unknown. “best practice” guidelines (if practice” guidelines (if quickly and efficiently providing
available). available), data quality is the visitor with datasets?
assessed to an internationally
recognised standard with
appropriate reference materials
3 Access to international No contacts/collaborations with Have some contact with Work in with Work in close collaboration with
collaboration Work in close collaboration with Are you able to draw in outside
expertise / networks observational/analytical relevant scientists in other groups in other countries doing groups in other countries doing groups in other countries doing expertise to help inform
marine ecosystem research and
oceanographers in other countries and/or institutions marine ecosystem research, pelagic marine biogeochemistry. marine ecosystem research and fieldwork planning, sample
countries and/or institutions and are aware of their effort pelagic marine biogeochemistry Co-author papers with these pelagic marine biogeochemistry. analysis or data interpretation?
and progress. and climate change impact on groups and can request Co-author papers with these For measurements your home
assistance/training if required.
the ocean. groups and can request institute cannot yet perform
assistance/training if required. could you call upon your
Write funding proposals in contacts to facilitate this?
collaboration with these groups.

46
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

4 Recognition and support Regular or routine sampling of There is some appreciation of Regular monitoring is perceived Routine sampling is valued by Routine sampling is valued by Have high level discussions
at organisational level coastal waters is not perceived the benefits of routine as a valuable activity and it is management and government management, it feeds into /reports produced a plan for a
but funding is insufficient
as a valuable undertaking for monitoring but it is insufficient seen as being worth the policy and institutional funding routine sampling strategy for
policy and business for investment investment is allocated national coastal waters?
development
5 Recognition and funding Biogeochemical sampling to Biogeochemical sampling to Biogeochemical sampling to Biogeochemical sampling Biogeochemical sampling and Can you name three current
by National /African/ investigate the impacts of investigate the impacts of investigate the impacts of appears in funding calls. the changing ocean are a regular national programs or national
Global Institutions, climate change or climate change or climate change or Routine sampling is identified as subject of funding calls or a funding calls issued in the last
policy making and anthropogenic activities does anthropogenic activities anthropogenic activities one of the priority areas in required component of funding year where biogeochemical
funding bodies not appear in funding calls and occasionally appear in funding occasionally appears in funding capacity development by calls. Strong national and/or sampling was involved?
national marine programs. calls and national marine calls. Routine sampling is national and international international capacity Did they involve more than
programs. identified as one of the areas African bodies. development efforts exist in this nutrients and chlorophyll
where capacity development is area measurements?
needed

47
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

No Category Please indicate the present status of your institution for each of the categories listed. Your responses should be specifically related to:

Conducting interdisciplinary projects in Ecosystem Approaches to Fisheries and sustainable use of living marine
resources
For a score of 2 and above, evidence (e.g. proposals, publications) and budget estimates must exist where appropriate
1 2 3 4 5 Simple test question for
Level 3
1 Scientific Project Does not have capacity Has sufficient expertise to Has sufficient expertise to Has sufficient expertise to design Has sufficient expertise to design an Over the last three years, have
Design (defined here for scientific project design a single discipline design an interdisciplinary an interdisciplinary project covering interdisciplinary project covering areas you submitted an
as a proposal design project addressing issues project covering three or more areas from physical oceanography from physical oceanography through interdisciplinary funding
developed for an relevant to the marine of the following areas: physical through marine ecosystems to marine ecosystems to fisheries, taking proposal covering three or
internal or external ecosystems and fisheries oceanography, marine fisheries, taking account of climate account of climate change impacts, socio- more of the following areas:
funding body) ecosystems, fisheries, climate change impacts, socio-economics economics and governance to address physical oceanography, marine
change impacts, socio- and governance to address pressing pressing societal issues ecosystems, fisheries, climate
economics, governance societal issues involving latest technologies change impacts, socio-
economics, governance
2 Funding generation Does not have evidence Has evidence of successful Has evidence of successful Has evidence of successful funding Has a capacity and expertise to generate Over the last three years, have
(defined here as a of generating funds for funding generation for a funding generation for a generation for a multi-institutional, funding for a multi-institutional, multi- you won any competitive
successfully funded a project project as described above project as described above multi-national project which national project which includes all of the funding for a proposal as
proposal by an internal includes all of the above above described above?
or external body)
3 Executing a research Has no evidence of Has evidence of successful Has evidence of successful Has evidence of successful Has evidence of successful completion of Over the last three years, have
project project completion completion of a project as completion of a project as completion of a multi-institutional, a multi-institutional, multi-national you successfully completed a
described above described above multi-national project which project which includes all of the above project as described above?
includes all of the above
4 Publishing results of Has no track record of Has a track record of peer- Has a track record of peer- Has a track record of peer- Has a track record of peer-reviewed Do you have peer-reviewed
research projects publications reviewed publications reviewed publications reviewed publications describing publications describing research findings publications describing research
describing research findings describing research findings of research findings of the above of the above findings of a project as
of the above the above described above?

5 Work with Has no experience of Can identified key non- Has an experience of effective Has an experience of involving Has an experience of involving Have you run a stakeholder
stakeholders communicating with academic stakeholders of the communication of the research stakeholders in the design of stakeholders in the design of projects and information event or similar?

48
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

non-academic project outputs results to the non-academic projects obtaining their support as cash or in-kind
stakeholders stakeholders investment

6 Informing Does not conducts Conducts research of Has evidence of producing Has a track record of the use of Has a track record of the use of scientific Have you produced a policy
policy/governance research of potential potential relevance to policy policy-relevant material some scientific results in results of interdisciplinary projects in brief or a stakeholder report?
relevance to policy policy/governance policy/governance

49
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

No Category Please indicate the present status of your institution/country (4,5) for each of the categories listed. Your responses should be specifically related to:

Capacity to carry out international development research addressing the SDGs (UK only)
1 2 3 4 5 Simple test question for
Level 3
1 Skills at individual No Some PIs have experience of Some PIs have experience of A range of PIs has previous experience A range of PIs has previous experience In the last three years, have the
level experience/skill conducting research in ODA conducting research in ODA context. of conducting research and delivering of conducting research, delivering organisation submitted a
s of working in context. No skills of M&E in the Some limited ability of M&E in the projects in ODA context. Early career projects and securing funding in ODA funding proposal for ODA-
ODA projects. organisation. organisation. researchers have opportunities to and SDG context and publishing compliant research with strong
contribute to ODA research and synthesis results in peer reviewed M&E plan, log frame, theory of
understand additional challenges of literature. Early career researchers change including partners from
ODA. have opportunities to contribute to ODA countries?
Organisation has some M&E expertise ODA research and understand
but can get access to M&E skills via additional challenges of ODA.
external contractors. Organisation has M&E experts and
stakeholder engagement experts for
ODA context.
2 Tools, data sets, No capacity or The organisation has a capacity (if Recognition that the effort is needed Organisation is working towards Large data sets are adapted to low Does the organisation have
instruments are recognition of the need arises) for adapting the towards adapting the large data sets adapting the large data sets to low broadband environment. Data examples of quick adaptation of
adapted to ODA the need to large data sets to low broadband to low broadband environment; broadband environment; developing processing and visualisation tools are research tools and methods for
conditions adapt to ODA environment and data processing developing data processing and data processing and visualisation tools developed for unskilled users. work in ODA context?
challenges tools for unskilled users and visualisation tools for unskilled users. for unskilled users. Some work Instruments and analytical capabilities
adapting instruments and analytical Recognition for the need of adapting underway for adapting instruments are adapted for small boats/shore
capabilities for small boats/shore instruments and analytical capabilities and analytical capabilities for small deployment and lack of research
deployment and lack of research or small boats/shore deployment and boats/shore deployment and lack of infrastructure.
infrastructure. lack of research infrastructure. research infrastructure.
3 Access to No North-South PIs have some contacts in ODA PIs have existing and PIs have existing trans-disciplinary
national PIs have strong existing trans- In the last three years, has the
international links exist countries international collaborations and track national and international North- disciplinary national and international organisation been able to
South networks which have some
expertise / record of publications with ODA track record of publishing ODA North-South networks which are respond to an ODA funding call
networks country researchers research in SDG context and some active in publishing ODA research and at a short notice using existing
joint North-South funding applications pursuing new funding opportunities networks with ODA countries
with projects in excess of £1M and ODA experts in UK?

50
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

4 ODA research ODA work is Organisation accepts that ODA ODA context is valued by the ODA context is highly valued by the Organisation has a strong track record Can leading PIs get involved into
capacity at discouraged. work should be a part of the organisation. PIs are recognised for organisation. PIs are recognised for of delivering large ODA projects. ODA ODA research without
their work in ODA/STG context and
organisational research portfolio their work in ODA/STG context. output expectations are adjusted context is highly valued by the damaging/slowing their career
level accordingly. Organisation has ODA organisation. PIs are recognised for progression?
experienced PMOs, coms and other their work in ODA/STG context and
science support functions.
output expectations are adjusted
accordingly. Organisation has ODA
experienced PMOs, comms and other
science support functions.
5 ODA research There are no There are some funding There are funding opportunities for Complexities of research in ODA/SDG Complexities of research in ODA/SDG Does the organisation have any
capacity at funding opportunities for leading western leading western research context are recognised by national and context are clearly recognised by ODA-compliant research funded
institutional opportunities research organisations to become organisations to become involved into international funding bodies. There national and international funding by external bodies?
(national and involved into research with ODA research with ODA countries. Policy- are funding opportunities for leading bodies. Clear evaluation frameworks
international countries. making bodies are aware of ODA western research organisations to of North-South collaborations exist.
funding and projects. become involved into research with There are funding opportunities and
regulatory bodies) ODA countries. Policy-making bodies evaluation incentives for leading
level are aware and receptive of science western research organisations to
into policy process with ODA projects. become involved into research with
ODA countries. There are appropriate
journals, conferences and reports for
dissemination of such research. Policy-
making bodies are aware and
receptive of science into policy
process and actively collaborate with
ODA projects.

51
SOLSTICE, Monitoring & Evaluation Framework 26 September 2018

Appendix 2. Equitable partnership survey (PDF)

Appendix 3. Network profiling questionnaire to be used in social network


analysis (PDF)

52

You might also like