Loading…
MERL Tech DC 2018 has ended
Simple
Expanded
Grid
By Venue
Wednesday, September 5
 

9:00am EDT

Workshop: Big Data and Evaluation (Separate registration required)
Join us for a one-day, in-depth workshop on big data and evaluation where you’ll get an introduction to Big Data for Evaluators. We’ll provide an overview of applications of big data in international development evaluation, discuss ways that evaluators are (or could be) using big data and big data analytics in their work. You’ll also learn about the various tools of data science and potential applications, as well as run through specific cases where evaluators have employed big data as one of their methods. We will also address the important question as to why many evaluators have been slower and more reluctant to incorporate big data into their work than have their colleagues in research, program planning, management and other areas such as emergency relief programs. Lastly, we’ll discuss the ethics of using big data in our work. Co-hosted by the World Bank's Independent Evaluation Group (IEG).

This workshop has an additional cost of $150 - please be sure you have registered here.

Here's the agenda for the day.
Speakers
avatar for Michael Bamberger

Michael Bamberger

Independent Consultant, Independent Consultant
Michael Bamberger, jmichaelbamberger@gmail.com, Independent Evaluation Consultant. 40 years of experience conducting and teaching development evaluation in Latin America, Africa and the Middle East and Asia. Working for the past few years on the opportunities and challenges for integrating... Read More →
avatar for Kerry Bruce

Kerry Bruce

Executive Vice President, Social Impact
Kerry Bruce is Executive Vice President at Social Impact and a global health and evaluation executive with more than 20 years experience living in Asia and Africa and strong world wide experience in all sectors. She’s an innovator in the big data and ICT4D space and enthusiast for... Read More →
avatar for Peter York

Peter York

Chief Data Scientist, BCT Partners
Peter York is BCT Partner’s Chief Data Scientist. He has over 20 years of experience in research, evaluation and data analytics, as well as serving as a national spokesperson for social impact and impact measurement issues for the government, nonprofit and philanthropic sectors... Read More →

Final MERL Tech Slides Sessions 1 and 2 pdf
Final MERL Tech Slides Sessions 6 and 7 pdf
Final MERL Tech Slides Session 3 to 5 pdf
Final MERL Tech Slides Session 8 pdf
Wednesday September 5, 2018 9:00am - 5:00pm EDT
World Bank, "I" Building, Room I-1-200 Independent Evaluation Group at the World Bank – “I” Building, Room: I-1-200, 1850 I St NW, Washington, DC 20006
  Pre-Workshop, Exploring the Future of MERL

9:00am EDT

Workshop: What can Blockchain offer to MERL? (Separate registration required)
Tired of the blockchain hype, but still curious on how it will impact MERL? Join us for a full day workshop with development practitioners who have implemented blockchain solutions with social impact goals in various countries. Gain knowledge of the technical promises and drawbacks of blockchain technology as it stands today and brainstorm how it may be able to solve for some of the challenges in MERL in the future. Learn about ethical design principles for blockchain and how to engage with blockchain service providers to ensure that your ideas and programs are realistic and avoid harm.

This workshop has an additional cost of $150 - please be sure you have registered here.

Here is the agenda for the day.
Speakers
avatar for Shailee Adinolfi

Shailee Adinolfi

Director, Blockchain Solutions, ConsenSys
Shailee Adinolfi is currently Director, Blockchain Solutions at ConsenSys, a venture production studio harnessing the power of ethereum. She was previously Vice President of Account Management and Marketing at BanQu Inc., which creates Economic Identities (patent pending) for people, assets and organizations on the distributed ledger, linking them to the global economy. At FHI 360, Shailee was a Technical Advisor, Digital Financial Services and Project Director for the USAID Mobile Solutions Technical... Read More →
avatar for Michael Cooper

Michael Cooper

Founder, Emergence
Michael Cooper is a former Associate Director for Policy and Evaluation who has 13 years experience with the U.S. Department of State, USAID and Millennium Challenge Corporation. In this capacity, he had technical oversight on dozens of independent mixed-method evaluations, creation... Read More →
avatar for Valentine J Gandhi

Valentine J Gandhi

Founder, The Development Cafe
Dr Valentine (Val) Joseph Gandhi is an Implementation Scientist, Development Economist, Policy Advisor, Poverty and Livelihood Impact Evaluation Specialist, ICT4D and Knowledge Manager, currently based between Jakarta and Nairobi, with over 15 years of interdisciplinary, quantitative... Read More →

Wednesday September 5, 2018 9:00am - 5:00pm EDT
Chemonics 1717 H St. NW, Washington, DC 20016
  Pre-Workshop, Exploring the Future of MERL
 
Thursday, September 6
 

8:30am EDT

Registration and Meet and Greet
Join us bright and early to grab a coffee or tea, get your name badge, and network with new colleagues and old friends!
Thursday September 6, 2018 8:30am - 9:00am EDT
Reception Area 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Community Building, Building bridges, connections, community and capacity

9:00am EDT

Welcome to MERL Tech 2018!
Welcome to MERL Tech! What is MERL Tech and why do we gather? What can you expect at this year's conference? What are plans for the space?

Welcome to MERL Tech slides
Speakers
avatar for Linda Raftree

Linda Raftree

Organizer & Independent Consultant, MERL Tech
Linda Raftree supports strategy, program design, research, and technology in international development initiatives. She co-founded MERLTech in 2014 and also works as an independent consultant. Linda advises Girl Effect on digital safety, security and privacy and supports the organization... Read More →

Thursday September 6, 2018 9:00am - 9:20am EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Community Building, Building bridges

9:20am EDT

Opening Talk: The Long Road Home... A Brief History of ICT for Development (ICT4D)
How did we get here? From Blockchain technology to social media, the technology landscape around us is changing now faster than ever before. Every day we are facing new challenges and trying to do the right thing, while at the same time trying to keep up with the latest new tech. But how did we get here? Is there anything that we can learn from what we have done in the past? Is it not a good time to re-think our approach to ICT4D? In this talk we will look at the development of ICT4D in the past 70 years, and try to use our history to look at where we are going, but also at important factors that will define our ability to cope with the new world technology is presenting us with.

Anahi's Slide Deck
Speakers
avatar for Anahi Ayala Iacucci

Anahi Ayala Iacucci

Senior Director, Humanitarian Programmes, Internews
Anahi Ayala Iacucci leads the Internews Humanitarian team as the Senior Director for Humanitarian Programs. Anahi plays a key role in developing Internews’ global Humanitarian program strategies; while also closely supporting teams in the field and managing Internews Emergency Response... Read More →

Thursday September 6, 2018 9:20am - 9:35am EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Community Building, Strengthening the evidence base on MERL Tech and/or ICT4D approaches

9:45am EDT

What's the State of MERL Tech?
What's happened at the intersection of MERL and Tech since 2014? What has changed? What's stayed the same? Where have there been unexpected turns? What tools and approaches have died out and what new ones have entered the mix? Let's map out the past 5 years and visualize the future of the space.

Session slides

Summary of some of the key research questions
Speakers
avatar for Michael Bamberger

Michael Bamberger

Independent Consultant, Independent Consultant
Michael Bamberger, jmichaelbamberger@gmail.com, Independent Evaluation Consultant. 40 years of experience conducting and teaching development evaluation in Latin America, Africa and the Middle East and Asia. Working for the past few years on the opportunities and challenges for integrating... Read More →
avatar for Maliha Khan

Maliha Khan

Founder, Daira
Maliha Khan is an international development practitioner who focuses on dealing with issues of complexity in measurement and accountability systems; designing initiatives for social change, and leading organizational change processes. An expert in ethical issues and data, she led... Read More →
avatar for Linda Raftree

Linda Raftree

Organizer & Independent Consultant, MERL Tech
Linda Raftree supports strategy, program design, research, and technology in international development initiatives. She co-founded MERLTech in 2014 and also works as an independent consultant. Linda advises Girl Effect on digital safety, security and privacy and supports the organization... Read More →
ZT

Zach Tilton

Consultant, Digital Impact Alliance

Thursday September 6, 2018 9:45am - 10:30am EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Community Building, Building bridges

10:30am EDT

Long Break
Take a nice break and re-connect with old friends and meet new ones!
Thursday September 6, 2018 10:30am - 11:00am EDT
Reception Area 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Community Building, Building bridges

11:00am EDT

Building Digital Information Retrieval at Scale for Education and the Social Sector
This session brings a combination of theory and practice, with hands-on components demonstrating scaling approaches that suit M&E project, system, and national scale levels. The session will demonstrate several approaches to mobile, tablet or phone based collection and retrieval that are sustainable at scale. Members of the panel have designed, built, and assisted with large scale information systems--many of which started as smaller scale M&E efforts. This session will be particularly appropriate for those interested in monitoring quality—be it education or health.
Speakers
avatar for Joy Cheng

Joy Cheng

Technical Officer II, Digital Solutions, FHI 360
KM

Kurt Moses

Director Policy & Information Systems, Digital Solutions, FHI 360
Kurt D. Moses has helped 600 organizations, at every level of education, from primary through higher education, in the developing world, OECD countries, and the United States. Using strategic planning, organizational development, and improved management information systems (MIS)--including... Read More →

Thursday September 6, 2018 11:00am - 12:00pm EDT
Balcony D 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Sharing experiences

11:00am EDT

Designing for equity: tools and approaches for assessing local equity to maximize impact and improve service delivery
Development programs prioritize serving the poor, yet face challenges when trying to determine whether they are reaching their intended population. This session will explore the importance of and challenges to equity measurement, and introduce commonly-used wealth assessment tools, including the EquityTool—a free, scientifically-validated tool helping managers use data to adjust and improve service delivery, and drive inclusivity in their programs. We will analyze the approach and implications of a pilot analysis conducted by Living Goods branches in Kenya in 2017 to illustrate how measurement can be used in local health systems to make global health care more equitable. By incorporating the EquityTool into the Medic Mobile app, Community Health Workers (CHWs) collected equity data for over 90% of families within their catchment areas. Data collected during the pilot was analyzed to assess characteristics of households in the pilot sample, describe how often targets for desired health outcomes are being reached, and observe patterns of health inequity across and within pilot branches. We will invite participants to share their own experiences in how accurate beneficiary targeting (or a lack thereof) has affected program implementation, and encourage the discussion of other commonly-used wealth assessment tools. Participants will leave the session with an appreciation for the importance of real-time equity metrics in inclusive development, and an understanding of how to easily evaluate the relative wealth of beneficiaries in their own interventions.

Session slides

Speakers
avatar for Rebecca Chapman

Rebecca Chapman

Communications Manager, Metrics for Management
Rebecca specializes in the collection and translation of data into powerful narratives and actionable results. She previously served as a Project Manager at Equal Access International, where she supported the use of media to empower communities in West and Central Africa to affect... Read More →
avatar for Karla Feeser

Karla Feeser

Research Analyst, Metrics for Management
Drawing on her time with non-profit organizations, academic institutions, and state and federal public health agencies, Karla has a wide range of expertise in research methodology, data collection and analysis. Prior to providing technical assistance at Metrics for Management, Karla... Read More →
avatar for Sara Hollis

Sara Hollis

Impact Analyst, Medic Mobile
Sara works to ensure that Medic Mobile implementations and tools have a desired and measurable impact in communities where they are deployed. She is excited about the potential that mHealth, Human Centered Design, and engaging data visualizations have in improving surveillance and... Read More →

Thursday September 6, 2018 11:00am - 12:00pm EDT
Academy Hall Breakout B 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Strengthening the evidence base on MERL Tech and/or ICT4D approaches

11:00am EDT

How to use Kumu to develop a robust programme Theory of Change
Because of its wide availability and ease of use, Microsoft PowerPoint is a popular software for representing Theories of Change (ToC). However, there are serious limitations to PowerPoint: the limited space can force the ToC to be overly simplified, version control makes it hard to develop ToCs in a participatory manner and it is difficult to edit the ToC once finished.

We will first discuss a range of alternative tools to develop ToCs and where they are best utilised. We will then present how we used Kumu to develop a robust ToC on a programme in Pakistan. Kumu is an online data visualisation platform that helps with the organisation of complex information into interactive relationship maps and lends itself well to the development of ToCs. In a hands-on presentation, we will present the steps involved in, and the benefits of using Kumu.

We will first demonstrate how we constructed stakeholder maps to represent the current state of the health system in Pakistan. We will then show how we used these maps, in combination with system mapping, to develop a robust programme ToC. Because all components are clickable, a deep dive into every aspect of the ToC is possible, which allows us to present a narrative, include photos and links to external sources. We will present how elements and connections can be decorated and labelled automatically, based on programme and external data.

It is hoped that session participants will gain an appreciation of the benefits of using Kumu in ToC development, and how it can be applied to their own work.
Speakers
avatar for Mark Oldenbeuving

Mark Oldenbeuving

Manager - Design and Innovation, Palladium
Mark Oldenbeuving is part of Palladium’s Design and Innovation team, based out of London. Mark has experience in the design, monitoring and evaluation of international development projects in South America, Africa and Asia. Mark is an experience facilitator of Theory of Change and... Read More →

Thursday September 6, 2018 11:00am - 12:00pm EDT
Academy Hall Breakout C 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Sharing experiences

11:00am EDT

Integrating MERL with program design: Presenting an approach to balance your MERL strategy with four principles
This session is for program managers and monitoring and evaluation officers who work in complex contexts, and due to limited resources, often have to make tradeoffs between program design (D) and MERL activities. When programs prioritize one aspect of D-MERL over others or continue to work in silos, issues can arise: a compromise in evidence quality and integrity; uninformed decision-making; poor selection of methodological processes; and ineffective organizational learning.

In this interactive session, we will introduce the BalanceD-MERL approach developed by the BalanceD-MERL consortium (funded by the USAID Global Development Lab’s MERLIN program). This approach is an intentional bridge between program design and utilization-focused MERL. Audiences will leave this session understanding how to operationalize the BalanceD-MERL approach, as well as its four core principles, into their programs. We will also present newly developed tools, beginning with the BalanceD-MERL maturity matrix. This matrix provides practical guidelines (through the use of guiding questions) on developing effective and efficient MERL strategies that are designed to the needs of the program. Next, we will share a practice-oriented case study that captures lessons learned from our experience piloting the approach and designing the MERL strategy (with the Performance Management Information System) in a water, sanitation and hygiene program launched in India. Discussion with the audience will encourage feedback on how technology can be better leveraged to bridge the gaps between program design and MERL, and how the BalanceD-MERL approach can be made more user-friendly. Attendees are highly encouraged to share lessons learned from their own experiences in balancing monitoring and evaluation with other activities.
Speakers
avatar for Heather Esper

Heather Esper

Senior Program Manager, William Davidson Institute at the University of Michigan
Heather Esper leads the Performance Measurement Initiative at the William Davidson Institute at the University of Michigan. She has 11 years of experience in leading partnerships, research and assessments of organizations’ economic, social and environmental performance and outcomes... Read More →
avatar for Yaquta Fatehi

Yaquta Fatehi

Program Manager, The William Davidson Institute at the University of Michigan
Yaquta Fatehi is a Program Manager at The William Davidson Institute at the University of Michigan. She works with both the private sector and the development sector to identify and improve socio-economic impacts of their low-income stakeholders. Combining social and business/operations... Read More →

BalanceD MERL session v4 pdf
Thursday September 6, 2018 11:00am - 12:00pm EDT
Angle 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Sharing experiences

11:00am EDT

MERL+ML+AID: USAID works to get the recipe right!
USAID's Bureau for Food Security and the Global Development Lab are actively engaged in learning about how remote sensing, machine learning methods, and predictive analytics could improve how we measure, monitor, and learn from the activities we invest in.  As part of this process, we will present on some of the projects we're managing that employ innovative approaches to monitoring, evaluation, and learning, and how viable these tools are compared to traditional MEL tools.  We will also discuss the ethics and risks of some of these emerging technologies and methods.
Speakers
avatar for Aubra Anthony

Aubra Anthony

Strategy & Research Team Lead, USAID
Aubra Anthony leads the Strategy & Research team in USAID’s Center for Digital Development within the Global Development Lab. Her team investigates the social, political, and ethical dimensions accompanying the introduction of digital technologies in developing economies. Her portfolio... Read More →
avatar for Farzana Ramzan

Farzana Ramzan

Monitoring and Evaluation Advisor, USAID
Farzana Ramzan, Ph.D., is a Monitoring, Evaluation, and Learning Advisor with USAID’s Bureau for Food Security.  Farzana advises her team and the Bureau on emerging technologies and innovations in MEL, and will be managing a new award that applies remote sensing and machine learning... Read More →
avatar for Peter Richards

Peter Richards

Economist, USAID
Peter Richards is an economist with the Bureau for Food Security, where his work explores how new and innovative technologies are helping us to better understand the changing social and environmental landscapes in which USAID operates, and the Agency's impact on them.  Prior to joining... Read More →

Thursday September 6, 2018 11:00am - 12:00pm EDT
Balcony E 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Exploring the Future of MERL

11:00am EDT

Mobile Data Collection combined with Business Intelligence tools to support survey management and data driven decision making
A critical part of collecting data is learning from it to better inform decisions and drive critical outcomes.  Therefore, the ability to have a tool which integrates raw data with off-the-shelf and customized platforms for analysis and further visualization is critical.

This session will be around technologies and platforms used for not only improving the data collection process but also for what you can do with your data once it’s been collected. A panel of three will share their practical experience in using different tools for mobile data collection and the associated tools and services used to the most out of their raw data – creating useable analysis and visualizations. Each presenter them will go over the following questions:

•    What tools were used?
•    Why these tools where chosen?
•    How were these tools used?
•    What were the challenges?

Through Q&A, the final purpose of this session is to spark discussion around the opportunities offered by these technologies and gather insight from the attendees. The panelists will approach the session from distinct perspectives, as a technology developer (SurveyCTO), an implementing organization (Save the Children) and, lastly, an independent evaluation firm (IMC Worldwide).
Speakers
avatar for Audra Blanchfield

Audra Blanchfield

Director of Marketing and Strategic Engagement, Dobility, Inc. | SurveyCTO
Audra is Dobility's relationship manager. Leveraging her passion for ICT4D and years of experience leading impact evaluation processes in Africa and Asia, she focuses on ensuring organizations – small and large – gain access to affordable, high-quality data through SurveyCTO.Audra... Read More →
avatar for Nael Jean-Baptiste

Nael Jean-Baptiste

Senior MEAL EFSL Specialist, Save the Children
I’m a professional with more than ten years of experience in Monitoring and Evaluation (M&E) which include seven years in the field. I’m skilled in developing and managing M&E and Learning systems as well as in designing M&E components in support to proposal development. I have... Read More →
avatar for Mike Klein

Mike Klein

Director, IMC Worldwide
Michael Klein is a director of International Solutions Group, a company that works with governments, U.N. agencies, international organizations, NGOs and other companies to improve the implementation of humanitarian aid and development programming projects. He is based in Washington... Read More →

Thursday September 6, 2018 11:00am - 12:00pm EDT
Vista 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Sharing experiences

11:00am EDT

The Challenge and (desperate) Need for Organizational Data Literacy
The present and future of MERL is data driven, as is the whole of International Development. Without data literate organizations, the bleeding-edge approaches that unlock our data and insights will fail, or at best be utilized by a fraction of the development community. This round-table discussion of technologists from different parts of the development community will discuss what it takes to see data skills and knowledge become pervasive across organizations big and small. Not everyone has to be a data scientist, but we need to raise the knowledge level of all development staff to really maximize the power of data.ding
Speakers
CG

Charles Guedenet

MEL Advisor, IREX
Charles Guedenet serves as a Monitoring, Evaluation, and Learning Advisor within IREX’s Center for Applied Learning and Impact (CALI) where he is responsible for measuring IREX’s global impact and supporting organizational learning, accountability, and data informed decision-making... Read More →
avatar for Anahi Ayala Iacucci

Anahi Ayala Iacucci

Senior Director, Humanitarian Programmes, Internews
Anahi Ayala Iacucci leads the Internews Humanitarian team as the Senior Director for Humanitarian Programs. Anahi plays a key role in developing Internews’ global Humanitarian program strategies; while also closely supporting teams in the field and managing Internews Emergency Response... Read More →
avatar for Maliha Khan

Maliha Khan

Founder, Daira
Maliha Khan is an international development practitioner who focuses on dealing with issues of complexity in measurement and accountability systems; designing initiatives for social change, and leading organizational change processes. An expert in ethical issues and data, she led... Read More →
avatar for Ben Mann

Ben Mann

Director, Development Results, DAI
Ben Mann is a data wonk and technologist who spends way too much time thinking about ways to measure impact of development projects. An "enlightened engineer", he thrives on challenges that bring policy and technical implementation together. Ben leads DAI's Managing for Development... Read More →
LS

Leslie Sage

Director of Data Science, DevResults
Leslie Sage is the Director of Data Science at DevResults, a company that provides data management software for the international development community. Leslie implements data management platforms around the world, both building new sites for clients and training users in software... Read More →

Thursday September 6, 2018 11:00am - 12:00pm EDT
Academy Hall Breakout A 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Facing our challenges and shortcomings

12:00pm EDT

Lunch and Demo Tables
Grab a lunch and visit demo tables on the first or second floor!
Thursday September 6, 2018 12:00pm - 1:00pm EDT
Reception Area 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Community Building, Building bridges, connections, community and capacity

1:00pm EDT

Hard Talk: Adaptive Management - is MERL helping or hindering its potential for success?
Adaptive Management: Is MERL helping or hindering its potential for success?

Adaptive Management is this year’s must-have accessory for any development programme – but is it living up to its potential, why aren’t more programs adopting it, what happens when it goes wrong?

DIAL (the Digital Impact Alliance) will be hosting a fun, interactive, panel show to look at the above questions through a MERL Tech lens.

The panelists, with help from the audience will challenge some common myths and debate the key questions:
  • Where is Adaptive Management being adopted and what successes and failures are we seeing?
  • To what degree does our integration of MERL technologies help or hinder taking a more adaptive approach?
  • How can the MERL Tech community do more to help adoption and success of adaptive approaches – and should we be doing this?

Adaptive Management and MERL Resources
Adapting Aid case studies from Mercy Corps and IRC (2016) - https://www.mercycorps.org/research/adaptive-management-case-studies

Making M&E work for adaptive management requires strategic skills (INTRAC, May 2018) - https://www.intrac.org/making-work-adaptive-management-requires-strategic-skills/

Monitoring and evaluation: five reality checks for adaptive management (ODI, Dec 2017) - https://www.odi.org/comment/10588-monitoring-and-evaluation-five-reality-checks-adaptive-management

USAID Learning Lab: https://usaidlearninglab.org/node/14636 -- has both monitoring toolkit and evaluation toolkit

Harvard PDIA in practice series - notes and a podcast series https://bsc.cid.harvard.edu/pdia-practice-series

 Oxfam AM rules of thumb (June 2018) https://oxfamblogs.org/fp2p/seven-rules-of-thumb-for-adaptive-management-what-do-you-think/

 The fashion for agile management is spreading (The Economist, July 2018) https://www.economist.com/business/2018/07/05/the-fashion-for-agile-management-is-spreading?cid1=cust/ddnew/email/n/n/2018079n/owned/n/n/ddnew/n/n/n/nNA/Daily_Dispatch/email&etear=dailydispatch&utm_source=newsletter&utm_medium=email&utm_campaign=Daily_Dispatch&utm_term=2018079

 How can Monitoring, Evaluation and Learning support adaptive management? (July 2018) https://oxfamblogs.org/fp2p/simplicity-accountability-and-relationships-three-ways-to-ensure-mel-supports-adaptive-management/

 How USAID can bridge the disconnect between its procurement process and evidence (July 2018) https://www.cgdev.org/blog/usaid-thinks-about-procurement-and-program-design-it-should-keep-evidence-mind

 How can we support the Adaptive Development Community?https://drive.google.com/drive/folders/1NQm-21WlTGmojVAXUJak_Bd21ogneuJ6

Innovations in evaluation: How to choose, develop and support them (May 2018) https://www.betterevaluation.org/sites/default/files/Innovations%20in%20evaluation_BetterEvaluation-UNICEF-EvalSDGs.pdf

Email mhaikin@digitalimpactalliance.org to add your resource
Speakers
avatar for Dave Algoso

Dave Algoso

Open CoLab
Dave Algoso is a social change strategist, organizer, facilitator, and consultant at large. He's advised NGOs like Mercy Corps and Oxfam America, and donor agencies like UNICEF, the World Bank, and USAID. His new project is called Open CoLab.
avatar for Amanda Berman

Amanda Berman

Monitoring Officer, Johns Hopkins CCP
Amanda works for the Johns Hopkins Center for Communication Programs (CCP) as a Monitoring Officer in the Research & Evaluation division. She has worked extensively in digital health and GIS – from mobile data collection to app creation, SMS surveys and spatial analysis – in an... Read More →
avatar for Julia Coffman

Julia Coffman

Director, Center for Evaluation Innovation
Julia Coffman founded the Center for Evaluation Innovation. She has more than 20 years of experience as an evaluator, and now specializes in the evaluation of advocacy, public policy, and systems change efforts. For 15 years Julia worked with the Harvard Family Research Project... Read More →
avatar for Matt Haikin

Matt Haikin

Manager, Insights and Impact, Digital Impact Alliance
I have been active in social-tech for 20+ years and an ICT4D researcher, practitioner and evaluator since 2011.I now work at DIAL where I am helping frame and undertake our research with a particular interest on engaging with and supporting local digital-development ecosystems in... Read More →
avatar for Laura O'Brien

Laura O'Brien

Senior Manager, MEL, Digital Impact Alliance
I am particularly interested in the MERL of technology in the international development context, and how adaptive management can be integrated to responsibly support these programs.
avatar for Veronica Olazabal

Veronica Olazabal

None, None
Veronica Olazabal, Director of Measurement, Evaluation and Organizational Performance at The Rockefeller Foundation is an award-winning impact measurement and management expert with a professional portfolio ranging 15 years and four continents (Africa, Asia, North and South America... Read More →

Thursday September 6, 2018 1:00pm - 1:50pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Hard Talk, Facing our challenges and shortcomings

2:00pm EDT

Adapting agile for MERL
Developing and implementing monitoring and evaluation strategies, particularly on massive multi-year projects, is a long journey. Teams develop various tech tools for data collection, analysis, and visualization, often managed with traditional project management tools like Gantt charts and PMPs. Agile techniques offer an alternate framework for managing development projects. In this session, we’ll unpack how the principles and practices of agile software development can be adapted and used by monitoring and evaluation professionals working within project implementation teams. These principles, from the Agile Manifesto (agilemanifesto.org) include valuing: - Individuals and interactions over processes and tools - Working software over comprehensive documentation - Customer (user) collaboration over contract negotiations - Responding to changeover following a plan Some of these principles may seem, on the surface, to be at odds with traditional M&E, but they are flexible and adaptive. They align to emerging approaches like using theories of change, iterative design and implementation, and complex systems theory. We’ll use a case example of developing a monitoring and evaluation dashboard with an agile framework to provide a tangible example of how these principles can be successfully applied. Facilitators will include a trio of diverse experiences: Trent Hone (Agile Coach), Amanda Makulec (data visualization designer & former M&E specialist), and Gina Assaf (Digital Design Team Lead). Participants will leave with a basic understanding of agile principles and select practices, equipping them to seek opportunities to use agile in their own workflows or collaborate effectively with software development partners who use this approach to managing development projects. often use traditional project management tools and approaches to develop workplans and manage progress, which can be linear and limiting.
Speakers
avatar for Gina Assaf

Gina Assaf

Digital Design Team Lead, Souktel
Combining technology, design and international development, Gina Assaf leads digital design for Souktel Digital Solutions, a company that designs and develops solutions for the humanitarian aid sector. She has worked with the UN and USAID, and their implementers, designing solutions... Read More →
avatar for Trent Hone

Trent Hone

Managing Consultant, Excella
Trent Hone is a Managing Consultant with Excella Consulting and an award-winning naval historian. He works with software and IT organizations to improve their art of practice, increase effectiveness, and accelerate learning. He has helped dozens of government and commercial teams... Read More →
avatar for Amanda Makulec

Amanda Makulec

Data Visualization Lead, Excella
Amanda Makulec is the Data Visualization Lead at Excella and has spent nearly a decade working in analytics, information systems, and data visualization. She finds creative ways to help people use data to answer questions and make decisions. At Excella, she leads cross-functional... Read More →

Adapting Agile for MERL Sept 2018 pdf
Adapting Agile for MERL Activity Sept 2018 pdf
Thursday September 6, 2018 2:00pm - 3:00pm EDT
Angle 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Building bridges

2:00pm EDT

Application maintenance isn’t sexy, but critical to success: how to ensure sustainability and maintainability in the non-profit/development agency space
Software tools are like cars. They start out all shiny and new and over time they need maintenance. We would never expect not to wash our car, change the oil, check and replace breaks, fix the engine if it caused the car to be completely stranded at the side of the road. So it is with software. Software needs care and maintenance once it’s deployed and at each maintenance or version update, thorough regression testing. Furthermore, software deployed in an international development setting, needs to have ongoing funding and human resources to manage software updates necessitated by environmental changes, software defects or the need for enhancements. In this session we will explore donors’ desire to build sustainable software products yet their reluctance to embrace a realistic model for ongoing maintenance and sustainability. I will cover the kinds of issues that typically arise after a software system is implemented and why it’s critical to include maintenance and testing activities in your initial planning and budget. Using real, live software projects spanning DHIS2, occasionally-connected facility-central server applications, to ecommerce and WordPress applications as examples, I will illustrate maintenance failures and how they may impact your project as well how to remedy them. Specific focus will be on the importance of hardware and system configuration, data entry form issues, software version updates, software process and business rule documentation, testing documentation and defect tracking. I’ll also review how to estimate maintenance costs and make suggestions for how to structure maintenance when transferring software ownership to local ministries or organizations. Finally, I will provide a checklist for development of your maintenance plan that can be a reference in your next software development project.
Speakers
avatar for Stacey Berlow

Stacey Berlow

Managing Partner, Project Balance
Stacey Berlow is a Managing Partner at Project Balance, a small women-owned business specializing in data driven software solutions. With over 20 years of product and project management experience and she has successfully directed multi-million, large-scale, multinational, and offshore... Read More →

Thursday September 6, 2018 2:00pm - 3:00pm EDT
Balcony E 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Facing our challenges and shortcomings

2:00pm EDT

From Traditional Evaluation to Data Analytics – PCI’s Experience Assessing the Impact of a Digital Social Enterprise
Over 225 million pastoralists across Africa rely on livestock as their primary source of economic, social, and physical well-being. With dramatic changes in climate and land use, finding pasture within traditional grazing areas has become high-risk and resource intensive. To address this issue, in 2013 PCI created paper-based community grazing maps overlaid with current vegetation conditions using satellite imagery. In partnership with Fordham University, PCI is conducting a randomized control trial (RCT) to rigorously evaluate the impact that the maps have on pastoralist’s ability to find pasture, improve their herd condition and value, and reduce the need for emergency aid. In pursuit of sustainability and scalability, PCI started transitioning away from paper map distribution in 2016, to a mobile application called AfriScout. While the traditional evaluation approach has yielded valuable data to help drive the digital development of AfriScout, the mobile app opens up a whole new world of using data science and data analytics to mine user generated data and continue to monitor and evaluate the impact of the app.

This session is for people who want to learn how an iNGO is transitioning from a traditional M&E approach to a new approach that combines analyzing user-generated data with survey data to monitor and evaluation a digital social enterprise. We will present on the methodology used to evaluate the maps, the pivot to AfriScout, and how PCI will use platform generated data to continuously evaluate and improve the application and demonstrate impact. We hope the presentation will lead to a lively debate about evaluating social innovations and the future of M&E.
Speakers
avatar for Helene Purcell

Helene Purcell

Teaching Fellow, Fordham University
Helene Purcell is in the 4th year of her PhD in economics at Fordham University in New York City. Her research focuses on development economics, specifically climate change adaptation, and she is currently working on research surrounding risk preferences and natural disasters in Indonesia... Read More →
JR

Jason Rubin

MERL Technical Advisor, PCI
Jason has been working in M&E for international development since 2007 across a variety of technical areas including democracy and governance, capacity building, economic development, and climate change adaptation. He got his first taste of MERL Tech as a consultant working to convert... Read More →

Thursday September 6, 2018 2:00pm - 3:00pm EDT
Balcony D 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Strengthening the evidence base on MERL Tech and/or ICT4D approaches

2:00pm EDT

I need that data yesterday! Three experiences of building M&E technology solutions
Do you ever dream about clean reliable data that can be manipulated and reported on across your various teams, stored effectively and always accessible to all who need it, when they need it? These three organizations do! This session is for those who are exploring or implementing a technology system or data warehouse solution to respond to their organizational needs for information. Join us to learn about the nuances, complications and victories of three mid-size nonprofit organizations who are at various stages of developing and executing organizational wide technology solutions for M&E. Panelists will describe the relationship between their M&E framework and the approach to their technology solution. Whether you are just starting out or knee deep into your own system, participants will leave the session with a very comprehensive and practical list of do’s and don’ts.

Session slides
Speakers
MB

Madeleine Boyd

Development and Communications Coordinator, Vital Voices Global Partnership
As a Development and Communications Coordinator at Vital Voices, Madeleine manages the Vital Voices monthly giving program, revenue reporting, and donor acknowledgement process. A data nerd at heart, Madeleine oversees development analytics, online fundraising efforts and donor information... Read More →
avatar for Nick Hamlin

Nick Hamlin

Data Scientist, GlobalGiving
As GlobalGiving’s Data Scientist, Nick wears many hats. Whether leading the organization’s data strategy, building and maintaining the GG Rewards algorithm, or designing experiments to measure program impact, Nick lives to make social good data accessible, understandable, and... Read More →
KH

Kelly Heller

M&E Specialist, Vital Voices
Kelly is the Monitoring and Evaluation Specialist for the Leadership & Global Activation department at Vital Voices. In this role she coordinates the design and implementation of a monitoring and evaluation framework to strengthen the individualized support that the organization offers... Read More →
avatar for Lauren Rice

Lauren Rice

Monitoring and Evaluation Manager, The Urban Alliance
Lauren is the Monitoring and Evaluation Manager at Urban Alliance. As an internal evaluator, she is responsible for national data collection and management, reporting, development of evaluation tools and liaising with external evaluators for The Urban Alliance’s second RCT. She... Read More →
AG

Alejandra Garcia Villamil

Director of MEL, Vital Voices
Alejandra Garcia-Diaz Villamil has over 13 years of research, evaluation, and monitoring experience within Latin America, Africa, and the United States. Her prior work experience focused on maintaining monitoring and evaluations systems for ProMujer, Community Research Institute at... Read More →

Thursday September 6, 2018 2:00pm - 3:00pm EDT
Academy Hall Breakout A 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Building bridges

2:00pm EDT

Innovations in data capture and management in DHIS2-based systems: WhatsApp and USSD
This session will feature two expert presentations that discuss innovations in data capture and system management in DHIS2-based systems. The first presentation, from Pact and the USAID Kizazi Kipya project in Tanzania will highlight how the project identified and met data collection needs through design and build of an easy-to-use USSD app integrated with DHIS2. Kizazi Kipya has collected millions of data points across multiple intervention areas, requiring complex systems management to track and aggregate data for reporting, learning, and analytics. To collect data quickly and at low-cost from over 20,000 community case workers (CCWs) serving one million OVC, the project developed a USSD app for direct, real-time data entry. USSD technology uses the GSM network rather than internet and allows for rapid, cost-effective data entry by decentralizing the process. USSD data entry improves data quality through data validation checks with the DHIS2 server.

The second presentation, from Akros and the EU DHIS2 Strengthening Project in Zambia, looks at how project teams developed WhatsApp support groups to aid in the management of data entry and data use in the DHIS2-based HMIS. WhatsApp support groups were not the initial solution to what was a critical need for strong remote support networks for users of the HMIS, who are all over the country; it was only identified after other support strategies failed and ultimately remains successful because it is already widely used and therefore is a very accessible way for users to connect to one another and request help.

Through the presentation of these two innovations, we hope to incite discussion around how to creatively address system needs in ways that fit the context of your implementation – that is, where possible, using tools that are intuitive to users and easily integrated into their daily work.

Julie's session slides
Tumainiel's session slides
Speakers
avatar for Julie Doherty

Julie Doherty

Zambia Country Director, Akros
TM

Tumainiel Mbwambo

Health Informatics Officer, Pact Tanzania
Tumainiel holds a Bachelor’s degree in information technology from the Institute of Finance Management in Dar es Salaam. He has 7 years of experience in data systems development and administration, statistical analysis and data management. He is currently working as a Health Informatics... Read More →

Thursday September 6, 2018 2:00pm - 3:00pm EDT
Vista 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Sharing experiences

2:00pm EDT

Measuring impact with remotely sensed imagery and machine learning
This session will explore techniques for analyzing free or inexpensive satellite and aerial imagery to monitor economic, agricultural, and environmental programs through the use of machine learning algorithms. The session is for people who are interested in learning about new data sources and analytical methods to complement or augment existing MERL techniques. We will explore the topic through a selection of case studies. Attendees will leave the session with an understanding of the potential and limitations for using different types of remotely sensed imagery (satellite, aerial, drone) as well as a general knowledge of different categories of analytical tools and how they can be applied to different sectors.
Speakers
avatar for Chuck Conley

Chuck Conley

Account Manager, Esri
Chuck Conley is an Account Manager with Esri’s Nonprofits & Global Organizations Team. He is passionate about advancing the use of Geography and GIS technology in the humanitarian and development fields. For the first 17 years of his GIS career he worked to directly support a wide... Read More →
avatar for Brendan O'Neill

Brendan O'Neill

Solution Engineer, Esri
Brendan O'Neill is a Solutions Engineer on Esri’s Nonprofit and Global Organizations team and supports the World Bank, World Food Programme, International Atomic Energy Agency, United Nations Refugee Agency (UNHCR), and USAID among others. Brendan focuses on formulating enterprise... Read More →

Thursday September 6, 2018 2:00pm - 3:00pm EDT
Academy Hall Breakout B 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Sharing experiences

2:00pm EDT

Unleashing the power of data and technology for gender just programming and development interventions
Join Oxfam and the International Food Policy Research Institute (IFPRI) in this highly participatory session exploring how does technology add value in answering gender questions and, in ultimately implementing gender just programming. This session will start with an intro on the role of data and technology and what insights it could give us for improving inclusion, and for having gender just programs and interventions. Then we will use a world café format to have a deep dive discussion of IFPRI’s work on investigating gender and time use through the use of new technologies, and Oxfam’s gender analytics program of repurposing data to improve programming. This will be followed by a full group discussion and debrief.

Repurposing data for gender just programming
Find out how we systematically reanalyze qualitative monitoring data and significant change diaries to detect gendered patterns using qualitative analysis software, and how we are using these findings for organizational development.
Discover what happens when we re-analyze quantitative data of completed impact evaluations (using RCTs, Difference in differences (DD)) to learn about the gendered impacts of various programming in resilience and agricultural interventions. Are our agricultural interventions equally effective for men and women in Haiti, Senegal, or Ethiopia? What are the patterns of education and asset ownership among men and women and how does this affect food security?
In the presentation, we will answer these questions as well as discuss how we use qualitative and quantitative data analysis software to consistently mine our evaluation data for gendered patterns and how to bring it back to program design and implementation.

Investigating gender and time use through the use of new technologies
How women and men choose to allocate their time and activities is an important component of many development projects, yet studies rarely collect data on time use, partly due to a lack of reliable, cost-effective tools. Existing standard methods for measuring time use include time diaries and stylized survey questions, but these tools can be cognitively and logistically challenging, leading to measurement error and reporting bias. Moreover, these standard methods measure quantity of time and not, importantly, quality of time (i.e., how a person experiences their time, such as the energy intensity of different tasks), which may play an important role in mediating project effects.

In this presentation, we will clarify the distinction between quantity and quality of time, highlight the importance of measuring both when investigating gender-related research questions, and discuss potential technology-based solutions to problems associated with measuring quantity and quality of time. We describe our efforts on a proposal currently in development that investigates the potential of smartphones for reducing recall bias in the collection of time use data through a randomized controlled trial. We also outline some recent examples of gender and nutrition research using new technologies such as activity tracking devices, heart monitors, and wearable cameras.
Speakers
EF

Emma Fawcett

Evaluation, Learning, and Effectiveness Advisor, Oxfam America
Emma is the Evaluation, Learning and Effectiveness Advisor at Oxfam America
HM

Hazel Malapit

Researcher, International Food Policy Research Institute (IFPRI)
FS

Florence Santos

Global Monitoring, Evaluation, and Learning (MEL) Manager, Oxfam America
GS

Greg Seymour

Researcher, International Food Policy Research Institute (IFPRI)

Thursday September 6, 2018 2:00pm - 3:00pm EDT
Academy Hall Breakout C 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Sharing experiences

3:00pm EDT

Short Break
Take a quick break before heading off to your 3.15pm session!
Thursday September 6, 2018 3:00pm - 3:15pm EDT
Reception Area 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Community Building, Building bridges

3:15pm EDT

Blockchain solutions for MEL data management and end use
This session will give an overview of what the blockchain is and how it offers solutions to MEL's unique data challenges to include management and increased access to data in a manner that facilitates end use. While in its infancy, the blockchain offers solutions that protect PII and create more streamlined processes for data quality and dissemination that can create opportunities for end use beyond what current processes and protocols allow.

This session will present an outline of these technical blockchain solutions in addition to some of the barriers to implementing these solutions in current practices and culture. The session is intended to be participatory in order to gain feedback from practitioners on their experience with the issues at hand and to invite them to an online platform that aims to be a gathering place for MEL practitioners to learn about and discuss opportunities in this space.
Speakers
avatar for Shailee Adinolfi

Shailee Adinolfi

Director, Blockchain Solutions, ConsenSys
Shailee Adinolfi is currently Director, Blockchain Solutions at ConsenSys, a venture production studio harnessing the power of ethereum. She was previously Vice President of Account Management and Marketing at BanQu Inc., which creates Economic Identities (patent pending) for people, assets and organizations on the distributed ledger, linking them to the global economy. At FHI 360, Shailee was a Technical Advisor, Digital Financial Services and Project Director for the USAID Mobile Solutions Technical... Read More →
avatar for Michael Cooper

Michael Cooper

Founder, Emergence
Michael Cooper is a former Associate Director for Policy and Evaluation who has 13 years experience with the U.S. Department of State, USAID and Millennium Challenge Corporation. In this capacity, he had technical oversight on dozens of independent mixed-method evaluations, creation... Read More →
avatar for Valentine J Gandhi

Valentine J Gandhi

Founder, The Development Cafe
Dr Valentine (Val) Joseph Gandhi is an Implementation Scientist, Development Economist, Policy Advisor, Poverty and Livelihood Impact Evaluation Specialist, ICT4D and Knowledge Manager, currently based between Jakarta and Nairobi, with over 15 years of interdisciplinary, quantitative... Read More →
avatar for Nick Martin

Nick Martin

Founder & CEO, Tech Change
Nick is an educator, technologist, and social entrepreneur with 15 years of experience in international development and social innovation. He was also part of LinkedIn's first Creator Accelerator program (#licreatoraccelerator)

Thursday September 6, 2018 3:15pm - 4:15pm EDT
Academy Hall Breakout B 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session

3:15pm EDT

From words to wisdom: Exploring free tools for qualitative data analysis
IMPORTANT: TO GET THE MOST FROM THIS SESSION, BRING YOUR LAPTOP AND SOME QUALITATIVE DATA TO PLAY WITH! (survey responses, reports, interview notes, or anything else!)Let's say you've just conducted thirty key informant interviews. Or you've just collected 200 open-ended responses in a survey for trainees. How can we begin to make sense of all this qualitative data? In this hands-on workshop, we'll explore 3-4 specific free tools that MERL practitioners can use to code and categorize qualitative information (key informant interviews, beneficiary survey results, etc.) to prepare it for visualization. This session emerges from a growing demand among the MERL community to "visualize qualitative information", but it's not a data viz workshop. Instead, we'll explore a few new tools you probably haven't heard of (Aylien, OverviewDocs, DataBasic) that helps you make sense of qualitative data (through quick online document coding, semantic analysis, etc.). This will help you prepare qualitative data so it can be more easily visualized later. We'll also introduce some interactive, creative, "offline data viz" methods of collecting qualitative data, as a fun way to prime the workshop. You'll leave primed to use these tools in your work, as well as armed with a better understanding of the work flow required to turn qualitative data into visualizations. We'll get hands-on with each tool in the session, so bring your own device!
Speakers
avatar for Samhir Vasdev

Samhir Vasdev

Digital Development Adviser, IREX
Samhir Vasdev is the Technical Adviser for Digital Development at IREX, a global development and education organization. There, Samhir drives the organization’s efforts to foster better use of data to inform decisions, learning, and action within IREX and its programs. For example... Read More →

Thursday September 6, 2018 3:15pm - 4:15pm EDT
Vista 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Building bridges

3:15pm EDT

Good, cheap, fast: Pick two
International development organizations and software providers are interdependent. What are the hallmarks of a fruitful relationship with your software provider? What are the best practices for managing that partnership, setting mutual expectations, and ensuring long-term success? Chemonics and DevResults will team up to discuss 1) Chemonics’ successes and lessons learned with rolling out enterprise-wide M&E system, 2) DevResults’ observations across scores of organizations who are working to do the same, and 3) the collaboration, compromise, and tangible steps taken on both sides to ensure success with data management for M&E. Chris Gegenheimer and Chris Huey of Chemonics, along with Leslie Sage of DevResults, will examine choices about staffing, timing, ongoing support, feature requests, resource allocation, and more. The old saying about software is, “good, cheap, fast: pick two”, but Chemonics and DevResults have spent years maximizing the balance between all three. Join our presentation and Q&A to find out how.
Speakers
avatar for Christopher Gegenheimer

Christopher Gegenheimer

Director MEL Technology, Chemonics International
CH

Christopher Huey

MEL Manager, Chemonics International
Christopher Huey is a Monitoring, Evaluation, and Learning manager with nine years of development experience, specifically in youth development and human rights advocacy. He has a strong background in technology and information management systems and is particularly interested in... Read More →
LS

Leslie Sage

Director of Data Science, DevResults
Leslie Sage is the Director of Data Science at DevResults, a company that provides data management software for the international development community. Leslie implements data management platforms around the world, both building new sites for clients and training users in software... Read More →

Thursday September 6, 2018 3:15pm - 4:15pm EDT
Angle 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Sharing experiences

3:15pm EDT

The TEGA Selfie App – an innovative mobile research app that empowers girls to show us their world
Improving outcomes for girls is a priority for the sector, but hearing from girls authentically is an ongoing challenge for evaluators. Girls are usually interviewed by someone outside their community and social customs mean family members are often present. Do girls really speak openly in these situations? We have noticed that girls speak most candidly when they self-report, on their own in private. So we designed TEGA (Technology Enable Girl Ambassador) Selfie - a mobile-based, girl operated research app that empowers girls to self-report via a smartphone over several months. Without anyone listening, girls can share their views more truthfully. Girls aged 15-17 are trained to use the TEGA Selfie research app on smartphones through in-person and mobile learning. Over months, they complete surveys in the app and interview a panel of close family and friends. Via the app, they send the findings to Girl Effect’s highly secure, multi-media content hub that houses the data and instantly analyses it programmatically.

TEGA Selfie gives us an unprecedented longitudinal and holistic view into girls’ worlds. We can capture and analyse data faster than ever. With better, faster research, we have the opportunity to transform programming. Our session is for anyone who wants to include more girls’ voices, more authentically, in development. In our session, we will present the TEGA Selfie method, including a live demo of the app. We’ll share a case study and learnings from TEGA Selfie monitoring in Nigeria. In Kano and Edo, we are using TEGA Selfie to monitor and evaluate a Girl Effect digital service for teenage girls called Girls Connect. Girls call an IVR line to hear informative stories and get information and advice from mentor. Monitoring is being done with adolescent girls (our TEGA Selfies) and their families. We will share the challenges, learnings and findings from the project. We’ll close with a Q and A and a idea sharing session focused on how to improve, innovate and best use TEGA Selfie. We want to use this opportunity to bring together experienced development professionals to share challenges honestly and come up with creative and innovate solutions to improve how we monitor projects as a sector.

Georgie's slides
Speakers
GW

Georgie Whiteley

Research Specialist, Girl Effect
Georgie Whiteley is a Research Specialist at Girl Effect, working on Girl Effect's mobile based peer to peer research method, TEGA. She had worked on research projects in Bangladesh, Nigeria and the USA. Her background is in qualitative research.

Thursday September 6, 2018 3:15pm - 4:15pm EDT
Balcony E 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Sharing experiences

3:15pm EDT

USAID Open Data Policy and Intellectual Property Rights
This session will present an overview of USAID's development data directives (e.g., USAID's Development Data Policy, USAID's Public Access Plan) for effectively managing USAID-funded data, especially federally funded scientific research data. It will focus on data management issues related to intellectual property rights, licensing, copyright, and data sharing and use. The discussion attempts to highlight data management best practices that USAID and the development data community can advance together to promote responsible use of data.

Session slides
Speakers
avatar for Anna Brenes

Anna Brenes

Data Steward, USAID Bureau for Food Security
Anna began working for USAID in 2012. She works closely with BFS staff and partners to provide guidance and support on USAID's development data directives and the development of analytical geospatial tools. Prior to working with USAID, Ms. Brenes worked 3 years with the State of Minnesota... Read More →
GG

Gayle Girod

Chief Innovation Counsel, USAID
Gayle joined the USAID Office of General Counsel in May 2011 from World Vision.  She is the Chief Innovation Counsel, supporting the Agency to use its authorities creatively to achieve our development objectives.  Gayle previously practiced government contracts law at three DC law... Read More →
MS

M. Scott Thompson

Data Scientist, USAID
Scott provides project leadership through USAID's Data Services team to advance the availability, accessibility, and re-use of USAID-funded data. Dr. Thompson's work at USAID focuses on enterprise data efforts, such as long-term curation, interoperability, and data sharing and publication... Read More →

Thursday September 6, 2018 3:15pm - 4:15pm EDT
Academy Hall Breakout A 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session

3:15pm EDT

Using mHealth for training and quality assurance of frontline RMNCH workers in Sindh, Pakistan
This session will share findings from an evaluation of the Roshan Mustaqbil (“Bright Future”) application (App), created by the Johns Hopkins Center for Communication Programs Health Communication Component (HCC) Pakistan program. Roshan Mustaqbil was developed to support the Pakistan Lady Health Worker (LHW) program in Sindh Province, specifically to enhance the LHWs' communication skills and interpersonal effectiveness. The App has two components - mLearning and mCounseling, and supports the LHWs and their clients as both an educational tool and a technical resource.

The evaluation of the App found high usability rates and increased perceived effectiveness on users' performance among LHW. Positive findings included sixty-five percent of LHWs who strongly agreed that training through the App had increased their knowledge of FP topics, and 70% who strongly agreed that the training increased their overall ability to provide information and counsel clients on maternal health issues. Seventy-six percent of LHWs strongly agreed that their use of the App had increased their effectiveness as a LHW and  80% strongly agreed the App provided an advantage over traditional learning methods. Positive findings were sustained in results from a 3-month post assessment.

Session participants will learn about key technical and training considerations for mHealth approaches to improve the interpersonal and professional effectiveness of front line workers delivering RMNCH services to clients in rural and communities with limited resources.
Speakers
KK

Kuor K

STA R&E, CCP
avatar for Saifra Sohail

Saifra Sohail

Senior Research Data Analyst, CCP

Thursday September 6, 2018 3:15pm - 4:15pm EDT
Balcony D 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Strengthening the evidence base on MERL Tech and/or ICT4D approaches

3:15pm EDT

When MERL takes over: Balancing data and information needs with digital development
This session will explore the tension between collecting everything possible versus the ability to develop and implement a viable and usable technology product. Too often, there is a desire amongst international development organizations to collect more and more MEAL data, especially given the increasing ease of data collection with technology like tablets and phones. This desire can sometimes be so strong that it comes at the expense of everything else. If this happens during a tech development, it can cause major issues, and add time and resources to the process.

Save the Children and Praekelt Foundation will lead a discussion on what happens when the desire for more and more data hampers the functionality of a final tech product, and ways to mitigate this. We will use the example of the Dooit Android application, which was developed jointly with Girl Effect Mobile, Praekelt, and Save the Children as a case study for the group to examine. Dooit was launched in Indonesia in 2017 that helps youth save money and reach savings and budget goals on their own. We will use examples of what went well and not so well during this process to frame the discussion, encouraging participants to share similar experiences related to MEAL and technology. Finally, the group will come up with a series of approaches that can be used for tech development that balance the need for data and design functionality.
Speakers
avatar for Jonathan McKay

Jonathan McKay

Director, Girl Effect
Jonathan McKay is the Director of 'Create' at Girl Effect — a new team that brings together writers, designers, software developers and strategists to create brands, products and experiences that girls love. With a background in digital marketing including building mobile campaigns... Read More →
CY

Christopher Ying

Senior Specialist, MEAL and Research, Save the Children US
Christopher Ying is a specialist in the field of monitoring and evaluation with global experience. He currently serves as the Senior Specialist for MEAL and Research for the Skills to Succeed Program at Save the Children US, which focuses on youth employment. His previous position... Read More →

Thursday September 6, 2018 3:15pm - 4:15pm EDT
Academy Hall Breakout C 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Sharing experiences

4:15pm EDT

Short Break
Take quick break before the 4.15 Lightning Talks start!
Thursday September 6, 2018 4:15pm - 4:30pm EDT
Reception Area 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Community Building, Building bridges, connections, community and capacity

4:30pm EDT

WhatsApp is What’s Up: How IP Messaging Can Increase Engagement and Sustainability
With over 1.5 billion users worldwide, WhatsApp proves to be the next SMS: a way to reach millions of people who may not have access to information otherwise. Beyond sustainability, the richer and more interactive content on WhatsApp allows for increased engagement.  We will share examples of how we've seen this play out in mobile health programmes and what the potential is for WhatsApp in MERL.

Ambika's slides
Speakers
avatar for Ambika Samarthya-Howard

Ambika Samarthya-Howard

Director of Communications, Praekelt.org

Thursday September 6, 2018 4:30pm - 4:35pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Lightning Talk, Exploring the Future of MERL

4:35pm EDT

MERL on the money: Are we getting funding for data right?
What if I told you that we all have access to an additional $100+ million for MERL data – all we’ve got to do is invest in smarter M&E.

This lightning talk aims to explore the costs and benefits of taking a data ecosystems approach to MERL. Sharing lessons learned from work with UNICEF and the Gates Foundation, World Bank, and others, we’ll argue the key to unlocking additional funds is investing smarter in recipient country data systems: the oft-forgotten public good that can keep on giving.

Paige's slides
Speakers
avatar for Paige Kirby

Paige Kirby

Senior Policy Advisor, Development Gateway
Paige drives innovation work around the SDGs, leads strategic communications, and supports partnership development at Development Gateway. She works with development organizations and country governments to support the use of data and technology in decision-making. Paige has supported... Read More →

Thursday September 6, 2018 4:35pm - 4:40pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Lightning Talk, Facing our challenges and shortcomings

4:40pm EDT

Dancing with data
This lightning talk will summarize challenges and lessons learned from IREX's experience creating VIBE: the Vibrant Information Barometer. After many years of running the Media Sustainability Index, IREX designed a new tool to better measure complex information systems (not just media sectors). This talk will share insights from our struggle to 1) make different types of data play nicely and 2) to build a coherent narrative from the synthesis that provides people with a fresh perspective and practical, useful information.
Speakers
avatar for Tara Susman-Peña

Tara Susman-Peña

Senior Technical Advisor, IREX
Tara Susman-Peña is senior technical advisor at IREX, where she is working to adapt and expand the global development nonprofit’s media and information literacy program, Learn to Discern, for the United States and other countries. An expert on media and communications technology... Read More →

Thursday September 6, 2018 4:40pm - 4:45pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Lightning Talk, Sharing experiences, examples, challenges, and good practice

4:45pm EDT

Progressive web apps are the future of digital development (really!)
Progressive Web Apps (PWA) are an approach to web apps that use modern capabilities to deliver app-like experiences. In other words, they bring the best of both mobile sites *and* native applications to users. They're reliable, they work even when offline, they're secure, they're engaging, they're cheaper and simpler to build, and they're easier to use. Literally everyone wins.

PWAs will be the easiest way to deliver high-quality digital services for all types of audiences -- to mobile phone users at the last mile of connectivity filling out surveys to project staff updating progress in the field to program staff monitoring targets on a spotty WiFi connection in downtown Washington, D.C.

You can get started now -- PWAs can help you in your everyday tasks, whether you need to take notes in the browser, or record a voice memo (remember that they work offline!). We pushing our tech teams and partners to build PWAs, we will improve the value and ease of digital services for everyone.

Gabriel's slides
Speakers
avatar for Gabriel Krieshok

Gabriel Krieshok

ICT4D Specialist, Abt Associates
Gabriel Krieshok is the ICT4D Specialist at Abt Associates, where he works with his Data Science and Enabling Technologies colleagues to solve all types of global challenges. Gabriel has had roles in product design, UX design, innovation labs, data science teams, web development... Read More →

Thursday September 6, 2018 4:45pm - 4:50pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Lightning Talk, Exploring the Future of MERL

4:50pm EDT

Why No One is Using your Dashboard
Today, every program seems to have a dashboard (or at least the promise of creating one) with lofty goals of “evidence based decision-making” and “meaningful data use.” Have we become so focused on building something that we fail to build tools that actually get used? In this lightning talk, Amanda will talk about three ways dashboards often fail to deliver on their promises and what we can do as developers, evaluators, and end users to address these design foibles from the start.
Speakers
avatar for Amanda Makulec

Amanda Makulec

Data Visualization Lead, Excella
Amanda Makulec is the Data Visualization Lead at Excella and has spent nearly a decade working in analytics, information systems, and data visualization. She finds creative ways to help people use data to answer questions and make decisions. At Excella, she leads cross-functional... Read More →

Why No One is Using your Dashboard MERL Tech 2018 AMakulec pdf
Thursday September 6, 2018 4:50pm - 4:55pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Lightning Talk, Facing our challenges and shortcomings

5:00pm EDT

Hard Talk on Dashboards – Everyone wants them, no one uses them, Why?
Dashboards: everyone says they want them, but many sit unused on web portals and shared folders. During this hard talk session, four data enthusiasts who work on dashboards every day will share their experiences on pitfalls that go beyond color palettes and tech stacks, and explore how we can be more mindful of the human side of dashboard design and use.

The panelists will debunk common assumptions and practices, share stories of their own dashboard design failures, and surface disagreements in our development community about why dashboards fail, succeed, or sit barely used on shared drives.

==========

Favorite Resources from the Panelists

Big Book of Dashboards - great primer on dashboard development; examples are in Tableau workbooks but the concepts are widely applicable
Flowing Data - cross cutting blog that hits on design, tools, development processes and more
New series from Ken Flerlage - SQL for Tableau - highlighting how data viz developers can benefit from learning some basic coding skills

Blogs on dashboard design tricks (primarily Tableau-focused, but ideas can be more widely applied):
Ryan Sleeper
Viz Wiz (Andy Kriebel)
Data + Science

Chart selection heirachies, for ideas on what chart to use when:
Visual Vocabulary
Data Viz Catalogue
Data Viz Project

Blogs on data viz best practices:
Policy Viz
Storytelling with Data
Ann K Emery
Evergreen Data

Inspiration for what to do (and not to do) in data viz design:
WTF Visualizations
Junk Charts
Makeover Monday (#makeovermonday on Twitter)
SWD Challenge (and #SWDchallenge on Twitter)
Energy in America (Tableau dashboard example Amanda called out)
Speakers
RB

Ritika Bhasker

Senior Data Scientist, DevResults
Ritika Bhasker: ritika@devresults.com Data Scientist, DevResults. Ritika Bhasker is a data scientist with a background in global governance and political science. She's passionate about using data to help identify and solve problems, and build out accessible ways to help others do the same. Prior to joining DevResults... Read More →
avatar for Amanda Makulec

Amanda Makulec

Data Visualization Lead, Excella
Amanda Makulec is the Data Visualization Lead at Excella and has spent nearly a decade working in analytics, information systems, and data visualization. She finds creative ways to help people use data to answer questions and make decisions. At Excella, she leads cross-functional... Read More →
KS

Kristine Schlenker

Health Informatics Advisor, USAID / Public Health Institute
Kristy is a Health Informatics Advisor with over a decade of experience in analytics, data management, performance management, and research. She currently leads an interagency group focused on advancing data access and data quality across PEPFAR – serving as a key liaison between... Read More →
RS

Robert Segan

Deputy Director Global Health Solutions, QED
A global health specialist with more than 12 years of experience in international development and work in emerging markets, Rob previously led a team that contributed to more than a $1B in winning proposals for a range of public and private sector clients including USAID, DFID, CDC... Read More →
avatar for Tim Shifflett

Tim Shifflett

Deputy Director of Client Innovations, MSI
Tim provides field facing technology support for donor funded projects, focusing on remote data collection systems and and information portals. He has supported data visualization projects for humanitarian and development organizations in Afghanistan, Yemen, Sudan, and other nations... Read More →

Hard Talk Dashboards MERL Tech 2018 pdf
Thursday September 6, 2018 5:00pm - 6:00pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Hard Talk, Facing our challenges and shortcomings

6:00pm EDT

Happy Hour Reception and Fail Fest!
We all love to talk about our successes, but it's through failure that we may learn the best lessons. Join us for a Happy Hour reception and support some brave souls who will get up on stage to tell us about moments where they have failed, how they picked themselves back up again, and what they learned in the process!
Speakers
avatar for Wayan Vota

Wayan Vota

Co-Founder, MERL Tech
Wayan Vota is a co-founder of MERL Tech and Kurante. He is on the forefront of utilizing new and emerging technologies to magnify the intent of communities to accelerate their social and economic development. Wayan has worked at FHI360’s Tech Lab, Development Gateway, Inveneo, The... Read More →

Thursday September 6, 2018 6:00pm - 6:30pm EDT
Reception Area 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Fail Fest, Facing our challenges and shortcomings

6:30pm EDT

Check Yourself Before You Tech Yourself: Common Pitfalls In Deploying Mobile Tools
With over 500 projects in 50+ countries, Dimagi has witnessed success. We’ve worked with awe-inspiring organizations to design, develop and deploy mobile apps that improve lives in the Majority World. We’ve also witnessed humbling failure. Phones break and aren’t replaced, trainings can’t keep pace with attrition, tech becomes outdated as program requirements evolve, data collection is duplicated and an undercompensated workforce becomes overburdened. You know the stories.

The allure of technology is strong, and its potential is powerful. However, it’ potential to fail, if not well planned and executed, is also very real. This presentation will focus on some of Dimagi’s hardest-learned lessons about common pitfalls in deploying mobile applications for data collection and service delivery. The session will draw on our vast experience implementing mobile tools in low resource settings, from small, 10-user projects, to a national scale project with over 100,000 users, and will showcase several different case studies and examples.

This lightning talk is designed for folks or organizations looking to adopt technology, like CommCare, for MERL. Tech isn’t always the best idea, and it’s certainly not a silver bullet to fix programmatic issues. My hope is that participants will leave the session with questions and criteria to help them evaluate their systems for tech readiness so that they are in a better position to time tech interventions, anticipate risks and avoid facing costly consequences.

Erin's slides
Speakers
EQ

Erin Quinn

Customer Success Manager, Dimagi
Erin Quinn is a Customer Success Manager at Dimagi where she works with diverse clients ranging from grassroots organizations to government ministries to enable the effective use of mobile technologies in education, health, and agriculture. Erin has worked on 15+ mobile technology... Read More →

Thursday September 6, 2018 6:30pm - 6:35pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Fail Fest, Facing our challenges and shortcomings

6:35pm EDT

How to Fail at Mobile Surveying Without Really Trying
This Fail Fest Lightning Talk will explore lessons learned from survey research projects utilizing mobile technology. The use of mobile technology in survey research has become increasingly more prevalent across the world, but with this growth comes both obvious and subtle lessons learned. Like all modes of research, those that utilize technology have strengths and weaknesses, and these often differ depending on the mobile mode being used. In this lightning talk we will discuss successes and failures related to several modes of data collection, including computer-assisted telephone interviews (CATI), interactive voice response (IVR), mobile web and short message service (SMS) surveys. For example, respondents must have a mobile phone and be literate to participate in SMS surveys, while CATI surveys require that enumerators speak and understand local languages and dialects and that respondents have time to participate in a survey.

Representatives from GeoPoll and RTI International will provide real examples of how mobile surveys can fail by overlooking both simple and complex issues related to questionnaire design, sampling, and programming surveys into technology platforms. At the end of this short talk, session participants will understand that although technology can improve speed and quality of research, the details and nuances of any survey research project still apply. They will also learn that a single issue with a technology platform can derail an entire survey research project during the design, implementation, or analysis stages of data collection.
Speakers
CL

Charles Lau

Research Survey Methodologist, RTI International
avatar for Tavian MacKinnon

Tavian MacKinnon

Program Manager, GeoPoll

Thursday September 6, 2018 6:35pm - 6:45pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Fail Fest, Facing our challenges and shortcomings

6:45pm EDT

Open Data: Are your donors screwing you over? (TBC)
Pending
Speakers
avatar for Anahi Ayala Iacucci

Anahi Ayala Iacucci

Senior Director, Humanitarian Programmes, Internews
Anahi Ayala Iacucci leads the Internews Humanitarian team as the Senior Director for Humanitarian Programs. Anahi plays a key role in developing Internews’ global Humanitarian program strategies; while also closely supporting teams in the field and managing Internews Emergency Response... Read More →

Thursday September 6, 2018 6:45pm - 6:50pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Fail Fest, Facing our challenges and shortcomings

6:50pm EDT

We Wrote the Book on Evaluation Failures. Literally.
The newly released book “Evaluation Failures” collected stories from 22 contributors about their failures in research, evaluation, and life in general. Join one of the contributors as he shares his own stories of failure and relates some of the more memorable episodes of calamity from the book.  
 
Isaac's slides
Speakers
avatar for Isaac Castillo

Isaac Castillo

Director of Learning and Evaluation, Venture Philanthropy Partners
Isaac D. Castillo is the Director of Outcomes, Assessment, and Learning at Venture Philanthropy Partners (VPP). Isaac has over 20 years of performance management, evaluation, and outcome measurement experience. He currently manages, refines, and evolves VPP’s approaches and processes... Read More →

Thursday September 6, 2018 6:50pm - 6:55pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Fail Fest, Facing our challenges and shortcomings

7:00pm EDT

Fail Fest: Drink Up and Close Out!
Enjoy a few more drinks before closing out the evening. See you bright and early tomorrow!
Speakers
avatar for Wayan Vota

Wayan Vota

Co-Founder, MERL Tech
Wayan Vota is a co-founder of MERL Tech and Kurante. He is on the forefront of utilizing new and emerging technologies to magnify the intent of communities to accelerate their social and economic development. Wayan has worked at FHI360’s Tech Lab, Development Gateway, Inveneo, The... Read More →

Thursday September 6, 2018 7:00pm - 7:30pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Fail Fest, Facing our challenges and shortcomings
 
Friday, September 7
 

8:30am EDT

Registration and Morning Networking
Grab a coffee and some breakfast, pop by a demo table, then grab a good seat to get ready for the day!
Friday September 7, 2018 8:30am - 9:00am EDT
Reception Area 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Community Building, Building bridges

9:00am EDT

Welcome to MERL Tech Day 2
Welcome to Day 2 of MERL Tech!
Speakers
avatar for Linda Raftree

Linda Raftree

Organizer & Independent Consultant, MERL Tech
Linda Raftree supports strategy, program design, research, and technology in international development initiatives. She co-founded MERLTech in 2014 and also works as an independent consultant. Linda advises Girl Effect on digital safety, security and privacy and supports the organization... Read More →

Friday September 7, 2018 9:00am - 9:10am EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Community Building, Building bridges, connections, community and capacity

9:10am EDT

Opening Talk: Data Science for MERL in 20 Minutes or Less
Ever wonder "What the heck is Data Science and can it really make my life easier?" This live experiment will show how a few lines of code (written on-stage) can scrape, analyze, and visualize data that MERL teams rely on every day to measure outputs, outcomes, and impacts. A tag-team of narration & live-coding, audience members will learn a few approaches for utilizing the power of Data Science and get access to an open-source solution (posted on GitHub!) for getting web data to measure performance indicators. Did we mention that the two "wild & krazy guys!" who will present are "more fun than a barrel of monkeys!" and are quite entertaining?
Speakers
GM

Greg Maly

Principle Data Scientist, DAI
Greg Maly, Principle Data Scientist at DAI’s Center for Digital Acceleration, has spent his career working at the intersection of global development and diplomacy, technology, and the arts. In previous roles, he conducted data driven conflict analysis research at the U.S. Department... Read More →
avatar for Ben Mann

Ben Mann

Director, Development Results, DAI
Ben Mann is a data wonk and technologist who spends way too much time thinking about ways to measure impact of development projects. An "enlightened engineer", he thrives on challenges that bring policy and technical implementation together. Ben leads DAI's Managing for Development... Read More →

Friday September 7, 2018 9:10am - 9:30am EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Plenary, Exploring the Future of MERL

9:30am EDT

Hard Talk: Will Technology and Big Data Replace Monitoring Evaluation, Research and Learning?
The most challenging aspects of purposeful Monitoring, Evaluation, Research and Learning (MERL) are most often: 1.) selecting informative metrics and 2.) performing MERL efficiently. Many individuals in MERL assume that technology and big data will dramatically improve both the selection of informative metrics and MERL efficiency. However, as a community, we also understand that existing structural constraints in the MERL ecosystem may either hinder or render ineffective technology-enabled tools. In addition, the biases present in large data sets may mean that metrics selected using machine learning are no more informative than those already in use and that MERL will ultimately be ineffective at improving institutional performance. In this interactive panel, we will discuss the existing structural barriers to the successful implementation of novel technology in the MERL space and what realistic expectations should be regarding the ability of big data to improve MERL outcomes.

In this interactive discussion, our participants will both try and answer the question:

Will technology and big data improve MERL or will existing structural issues in our communities, paired with known biases in existing data sets, make MERL all-the-more challenging?

Session slides
Speakers
AC

Alexa Courtney

CEO, Frontier Design Group
Alexa Courtney has over 17 years of experience working in the US and globally in South Asia, Africa, and Europe; co-designing strategies with organizations to better adapt and create impact and leading research, assessment, and evaluation teams. She pioneered the application of design... Read More →
avatar for Madeleine Gleave

Madeleine Gleave

Chief Data Scientist, Nithio
Madeleine Gleave is Chief Data Scientist at Nithio, a new analytics and finance platform dedicated to scaling off-grid electricity access across Africa. Madeleine was previously the Advanced Implementation Specialist for Dharma.ai, where she led the implementation of Dharma’s data... Read More →
SV

Samuel V. Scarpino

Chief Data Scientist, Dharma.ai
Samuel V. Scarpino is the Chief Data Scientist and Dharma.ai and is an Assistant Professor in the Network Science Institute at Northeastern University--with appointments in Marine & Environmental Sciences, Physics, and Health Sciences. Scarpino's research spans a broad range of topics... Read More →

Friday September 7, 2018 9:30am - 10:30am EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Hard Talk, Facing our challenges and shortcomings

10:30am EDT

Measuring impact with remotely sensed imagery and machine learning
In this lightning talk we will introduce 3 case studies for how organizations are using satellite and drone imagery to monitor the impact of their programs. The talk is meant to alert attendees to the potential for leveraging free or inexpensive imagery and machine learning algorithms to provide new insights and inspire them to think of ways to apply these data sources and methods to their own work
Speakers
avatar for Brendan O'Neill

Brendan O'Neill

Solution Engineer, Esri
Brendan O'Neill is a Solutions Engineer on Esri’s Nonprofit and Global Organizations team and supports the World Bank, World Food Programme, International Atomic Energy Agency, United Nations Refugee Agency (UNHCR), and USAID among others. Brendan focuses on formulating enterprise... Read More →

Friday September 7, 2018 10:30am - 10:35am EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Lightning Talk, Sharing experiences, examples, challenges, and good practice

10:35am EDT

Ready, set, fly! How to close a data gap from the sky using drones
Have you ever thought about having data from the sky to give a full picture of your project? The speaker and his team went to the Dominican Republic and flew 2 drones to gather data – beyond what would normally be collected on the ground - about a World Bank supported irrigation project. The team also worked with local partners over a week on joint capacity building, bringing them along to collect data with the drone expert and evaluator. The drones captured images of 1,300 ha of rice fields and irrigation infrastructure. This lightning talk gives you an idea about how drones can be a powerful tool in MERL of infrastructure projects compared with other alternatives such as satellite images, aerial photos and physical inspection – and transferring knowledge and tools to development partners.

Katsu's slides
Speakers
KH

Katsumasa Hamaguchi

Evaluation Officer, The World Bank
Katsumasa joined Independent Evaluation Group (IEG) of the World Bank Group in 2015 and has been leading innovation in monitoring, evaluation and learning using untraditional methodologies including drones and Social Network Analysis (SNA). He recently proved value of using drones... Read More →

Friday September 7, 2018 10:35am - 10:40am EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Lightning Talk

10:40am EDT

We live in a 3D world, so let's map it!
Amanda's slides
Speakers
avatar for Amanda Berman

Amanda Berman

Monitoring Officer, Johns Hopkins CCP
Amanda works for the Johns Hopkins Center for Communication Programs (CCP) as a Monitoring Officer in the Research & Evaluation division. She has worked extensively in digital health and GIS – from mobile data collection to app creation, SMS surveys and spatial analysis – in an... Read More →

Friday September 7, 2018 10:40am - 10:45am EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Lightning Talk

10:45am EDT

Satellite data for decision making and situational awareness?
I will discuss the use of satellite data for decision making and situational awareness in some Creative projects. Lessons can be learned from what went well and what went less well...

Daniel's slides
Speakers
DB

Daniel Berger

Data Lead, Creative Associates International

Friday September 7, 2018 10:45am - 10:50am EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Lightning Talk, Sharing experiences

10:50am EDT

Mobile Tracking of Land Parcel Ownership in Zambia
Land ownership can be key to economic stability especially in Zambia where farming is a means to self sufficiency. However, often there is no formal record of ownership. In 2015, Akros worked with the USAID Tenure for Global Climate Change Program and the Chipata District Land Alliance (CDLA) to develop a DHIS2-based land tenure reporting platform. The tool enables village-level surveillance of land tenure changes and land disputes, as well as electronic submission of the forms needed to initiate certificate changes and comply with village land committee procedures. In 2017, Akros and TGCC developed an application specifically for use by chiefs to review land parcel data—including a mobile registry of land parcels and owners built from the current DHIS2 reporting system, allowing them to view land ownership history for each individual parcel in their community. This lightning talk will discuss the gains seen through the reporting platform since 2015, as well as early responses to the chief application.

Julie's slides
Speakers
avatar for Julie Doherty

Julie Doherty

Zambia Country Director, Akros

Friday September 7, 2018 10:50am - 10:55am EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Lightning Talk, Sharing experiences

10:55am EDT

Maslow’s Hierarchy of M&E software UX
As the basic needs of data reporting become commoditized, organizations realize they can get more out of data. They can work “smarter, not harder,” achieve better results, and reach new levels of evidence-based decision making. What is the role of technologists in helping move an organization through this process? Diego Mijelshon shares some of the secret sauce that guides the evolution of DevResults, a company that helps people meet their objectives and raise the bar for organizational growth and use of data.

Diego's slides
Speakers
avatar for Diego Mijelshon

Diego Mijelshon

Senior Engineer, DevResults
Diego is one of the senior engineers in DevResults.With over twenty years of experience in software development with an Information Systems background, he lives by the Alan Key motto “Simple things should be simple, complex things should be possible”. He’s lived all of his life... Read More →

Friday September 7, 2018 10:55am - 11:00am EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Lightning Talk

11:00am EDT

Long Break
Find some friends -- new or old -- to catch up with. Visit one of the demo tables on the first or second floor.
Friday September 7, 2018 11:00am - 11:30am EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Community Building, Building bridges, connections, community and capacity

11:30am EDT

Building bridges between data science and development evaluation
The purpose of this session is to stimulate dialog between data scientists and evaluators on three questions: (1) potential applications of data science in strengthening evaluation methodology, (2) why have evaluators been slower to adopt data science methodologies than their colleagues in other parts of the development community, and (3) how to build bridges between data scientists and evaluators.

The session will begin by reviewing a number of promising ways in which big data tools and analytical techniques can strengthen how development programs are evaluated, and why evaluators have been slow to adopt these potentially powerful tools.

This will be followed by two presentations illustrating how different big data tools are being applied in evaluation. The first presentation will describe how Girl Effect is leveraging user comments analytics to evaluate the effectiveness of its media campaigns in increasing the agency of teenage girls in four different countries. The second presentation will describe how the Global Environment Facility (GEF) is using geospatial science and satellite data to strengthen the evaluation of environmental programs protecting forests, fisheries and other natural resources. Both presenters will address the following questions:

• The main steps in the design and utilization of the different data science techniques in evaluations.
• The financial and technical resources required for the design, implementation and analysis of the respective studies
• How much time was required for the implementation of these studies and how did this compare requirements for traditional evaluation approaches?
• How were the different big data sources accessed and would these sources be easily accessible to other agencies (or is it necessary to have high level contacts with government agencies or the organizations that produce the data?)
• What was the value-added of using the big data? Were the additional costs and effort justified by the improved quality of the evaluation?
• What were the main challenges [methodological, resource considerations, organizational, political and cultural] in using these tools and techniques?
• What advice would the presenters give to other agencies considering the use of these tools and techniques?

Participants will then be invited to: (1) share their experiences in the use (or decision not to use) big data in their evaluations, and (2) to discuss the opportunities and challenges for building bridges between data scientists and evaluators. MERL Tech is considering organizing future training sessions to develop a common framework between data scientists and evaluators, so ideas discussed in this session will provide a useful input.

Claudia's slides
Speakers
avatar for Anupam Anand

Anupam Anand

Evaluation Officer, GEF-IEO, World Bank Group
Anupam Anand has more than 12 years of combined experience in international development, academia, and teaching. He implements innovative multidisciplinary approaches to development evaluation. At the IEO, he has led evaluations on biodiversity, Illegal Wildlife trade, and land degradation... Read More →
avatar for Michael Bamberger

Michael Bamberger

Independent Consultant, Independent Consultant
Michael Bamberger, jmichaelbamberger@gmail.com, Independent Evaluation Consultant. 40 years of experience conducting and teaching development evaluation in Latin America, Africa and the Middle East and Asia. Working for the past few years on the opportunities and challenges for integrating... Read More →
avatar for Claudia Abreu Lopes

Claudia Abreu Lopes

Data Science and Social Change Consultant, Independent
Claudia Lopes is a Data Science and Social Change Consultant. She has 10 years of experience working in academia and partnering with international development organisations, mainly in East Africa and Latin America. She is interested in methods to analyse large volumes of citizen-generated... Read More →

Friday September 7, 2018 11:30am - 1:00pm EDT
Academy Hall Breakout A 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Building bridges, connections, community and capacity

11:30am EDT

I survived a machine learning pilot
We don't doubt that machine learning can help us achieve MEL outcomes more efficiently, accurately, consistently, etc. But taking the plunge—defining a relevant problem, finding a machine learning tech partner, etc.—can be a huge challenge, especially at a resource-constrained NGO. This session will try to make it easier for others in the MERL community to take that plunge.

For the past 6 months, IREX has been collaborating with Lore, an AI startup, to test how machine learning can help us monitor the impact of our media trainings at a scale, efficiency, and consistency that far outpaces those of traditional human "coders". We've trained a computer to examine thousands of news articles in Mozambique, to automatically measure the quality of journalists’ content. We've turning this experience into an explainer for NGOs and global development organizations that provides practical tips about how to (and how not to) use machine learning technology to tackle their challenges.

Join this casual "Q&A" session, where we'll share those lessons and tips and discuss concrete, actionable steps to commons questions, ranging from "What does a TOR for a machine learning MEL project look like?" to "How much time do you need to spend 'teaching' the machine?"

We've heard enough talk about how frontier technology like machine learning can transform our work as MEL practitioners. At this session, let's unpack that black box and walk the talk!
Speakers
HE

Hedeer El-Showk

Co-founder, Lore AI
Hedeer El Showk is the co-founer of Lore Ai, an artifical intelligence technology startup helping people effectively work with "big text" through natural language processing and smart automation. Hedeer holds an M.Sc in Mathematical Finance from the Courant Institute at New York University... Read More →
AG

Alexandre Gavaza

M&E Manager, IREX
Alexandre Gavaza is a professional with over 15 years of experience working for NGOs (Health and Media), firms in the construction, beverages, extractive and metallurgical industry. He is a Data Specialist accredited by Microsoft, which confers his mastery in the structuring and implementation... Read More →
NS

Neeran Saraf

Senior ICT Advisor, Zeva
avatar for Samhir Vasdev

Samhir Vasdev

Digital Development Adviser, IREX
Samhir Vasdev is the Technical Adviser for Digital Development at IREX, a global development and education organization. There, Samhir drives the organization’s efforts to foster better use of data to inform decisions, learning, and action within IREX and its programs. For example... Read More →

Friday September 7, 2018 11:30am - 1:00pm EDT
Balcony E 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session

11:30am EDT

M&E + GIS + Mobile: The power of location, anywhere, anytime, any device
Harness the power of location to improve coordination and operational efficiency in your M&E activities. Reduce or replace reliance on paper. Ensure that all workers, from the field office to the home office, use the same authoritative data, reducing errors, boosting productivity, and saving money.
  • Plan: Leverage the power of location to understand where M&E work needs to be done and how to effectively schedule and dispatch resources.
  • Coordinate: Quickly respond to the unexpected by understanding, in real-time, what resources are available and where they are needed. Easily assign tasks and monitor their status.
  • Navigate: Consistently meet deadlines by using the most efficient routes so M&E teams get there on time and get the job done.
  • Locate and Capture: Boost productivity with location-aware mobile apps for data capture and situational awareness.
  • Monitor: Stay on top of M&E operations by monitoring, tracking, and reporting real-time data feeds to focus on what matters most.
  • Empower Stakeholders: Leverage easy-to-understand dashboards to support informed decision-making.
Speakers
avatar for Chuck Conley

Chuck Conley

Account Manager, Esri
Chuck Conley is an Account Manager with Esri’s Nonprofits & Global Organizations Team. He is passionate about advancing the use of Geography and GIS technology in the humanitarian and development fields. For the first 17 years of his GIS career he worked to directly support a wide... Read More →
avatar for Brendan O'Neill

Brendan O'Neill

Solution Engineer, Esri
Brendan O'Neill is a Solutions Engineer on Esri’s Nonprofit and Global Organizations team and supports the World Bank, World Food Programme, International Atomic Energy Agency, United Nations Refugee Agency (UNHCR), and USAID among others. Brendan focuses on formulating enterprise... Read More →

Friday September 7, 2018 11:30am - 1:00pm EDT
Balcony D 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Sharing experiences

11:30am EDT

Measuring complexity: A real-time system analysis tool
Over the last 10 years, Root Change has learned that trust and relationships are at the heart of social change. While we know achieving and sustaining development outcomes depends on the contributions of multiple actors embedded in thick webs of social relationships and interactions, traditional MEL approaches have failed to help us understand those complex dynamics. A new theory of measurement built around ways to measure trust, relationships, and accountability between development actors is essential.

We have designed Pando, a technology platform, to help development actors visualize, learn from and engage with the social systems where they work. Grounded in social network analysis (SNA), Pando users collect relationship data to visualize their social systems in real-time. Pando is also integrated with Keystone Accountability’s Feedback Commons, an online tool that allows users to collect and analyze qualitative feedback from system actors about levels of trust and relationship quality. Key features of Pando include network mapping, feedback surveys, visualizations, and dashboards on changing levels of trust, social capital and influence among development actors, their partners and constituents.

Our participatory session will explore this new approach. We will invite conference attendees to map their relationships on Pando by email invitation prior to the session. Mapping will also be possible at our Pando demo booth. During our 1.5 hour session we will dive deeper into the MERL Tech ecosystem and prominent use cases from past Root Change work to explore how ONA metrics and feedback are revolutionizing our thinking around the measurement of social capital and complex systems.

Attendees should plan to bring their laptops to the session.
Speakers
EB

Evan Bloom

Co-Founder and Managing Partner for Innovation, Root Change
Evan is the co-founder and managing partner of Root Change, where he sets strategic direction for Root Change’s organizational strengthening initiatives and action research agenda. He loves to tinker in the Root Change “lab” on new capacity building technologies and to exchange... Read More →
AS

Alexis Smart

Senior Technical Officer and Pando Product Manager, Root Change
Alexis works closely with partners and local change makers to co-design and facilitate system strengthening initiatives to address complex social problems. She leads Capacity 2.0 (C20) Local Works in Malawi and serves as the product manager for Pando, an online network visualization... Read More →

Friday September 7, 2018 11:30am - 1:00pm EDT
Academy Hall Breakout C 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Building bridges

11:30am EDT

Moving from data to action: How can real-time data enable adaptive management?
Using digital tools (such as mobile phone based text messaging, web-based applications, social media platforms, etc.) or large digital datasets (such as satellite or cell phone tower data) for collecting real-time data helps programs and projects respond quickly to community needs or potentially changing circumstances on the ground. However, these digital tools and datasets are often not well understood or mapped into decision-making processes. Examples of how real-time data has been used to manage development and relief efforts adaptively:
  • Tracking migration flows following natural disasters to understand population locations and best locate relief efforts
  • Gathering demographic and assessment data following trainings, in order to improve outputs and outreach for future trainings
  • Analyzing real-time disease outbreak data to understand where medical resources will be most effectively deployed
  • Analyzing of radio and social media to understand and adapt communication outreach

In this session, we will engage participants in a rigorous and inclusive conversation on these and other uses of real-time data to enable adaptive management in development programs. The presenters will provide overview on USAID’s newly released portfolio of research on Real-time Data for Adaptive Management (RTD4AM), followed by an activity demonstrating the importance of real-time data in enabling adaptive management and its potential challenges. Then, the participants will have a chance to provide feedback and/or input into USAID’s upcoming guides intended to overcome barriers to real-time data use in development. We look forward to continuing to build the community of practice of adaptive management within the MERL community.

Session slides
Speakers
avatar for Erica Gendell

Erica Gendell

Program Analyst, USAID/Lab
Erica Gendell is a Program Analyst at the U.S. Global Development Lab at the U.S. Agency for International Development (USAID). She works on inclusive digital development and advancing the use of data and digital technologies in USAID programs. Before USAID, Erica worked for CSRA... Read More →
avatar for Rebecca Saxton-Fox

Rebecca Saxton-Fox

ICT Policy Advisor, USAID
Rebecca Saxton-Fox is an ICT Policy Advisor on the Development Informatics team at USAID. She works on advancing the appropriate use of data and digital technologies in USAID programs, with a particular focus in the past few years on development and promotion of responsible data practices... Read More →

Friday September 7, 2018 11:30am - 1:00pm EDT
Academy Hall Breakout B 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Sharing experiences

11:30am EDT

Responsible Data: Ready, Set? Stop and Prep.
MERLTechies face pressure to collect more and more digital data. But In the grind for bigger, better, faster data, organizations often skip moments to pause and examine readiness to ethically and securely collect, store, transfer, process, and analyze data. This session offers a safe space to assess our organizations’ responsible data readiness. Most importantly, the session will prepare participants to push back against pressure and lead tough, but urgently needed conversations to address data risks and ethical hazards. Participants will candidly self-assess responsible data readiness and explore frameworks (legal, compliance, and principles) to foster organizational buy-in to address gaps. Using role play and organizational personas, participants will practice having difficult conversations with key stakeholders about the gaps and issues identified , with facilitators serving as coaches and thought partners. Finally, participants will reconvene to discuss what they learned, what gaps they will address going forward, and how they will tactically bridge difficult data conversations.
Speakers
avatar for Laura Walker McDonald

Laura Walker McDonald

Director of Innovation, Global Alliance on Humanitarian Innovation
Laura Walker McDonald is Director of Innovation at GAHI, the Global Alliance on Humanitarian Innovation. She has spent eight years working on inclusive, appropriate technology for social change, at SIMLab and FrontlineSMS, and prior to that worked for the Red Cross in London. She... Read More →
avatar for Alexandra Robinson

Alexandra Robinson

Strategy and Impact Manager, Humanity United
Alexandra Robinson manages strategy, impact measurement, ICT, and data/ tech ethics across Humanity United's counter-trafficking and conflict prevention portfolios. She began her career building real time data counter-trafficking systems in Nepal, and more recently led M&E of the... Read More →

Responsible Data Ready, Set Stop and Prep pptx
Friday September 7, 2018 11:30am - 1:00pm EDT
Angle 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Facing our challenges and shortcomings

11:30am EDT

Up your data game: How to use R to wrangle, analyze, and visualize data faster and better
This workshop will introduce participants to the possibilities of using the free and open-source R programming language and software environment for data processing, analysis, and visualization. For many, Microsoft Excel is the tool of choice when working with data. By simply clicking, pointing, and dragging things with your mouse, you can accomplish many data-related tasks. What R offers is the ability to do the same tasks and much more in a fraction of the time it would in Excel, with larger data sets, fewer errors, and more transparency. Participants will see satisfying examples of using R to do common data tasks and, working back from those examples, learn key R programming skills in the process. The workshop facilitator will use a fun and freely-accessible dataset and examples from popular R “packages” to perform common data tasks, including cleaning, exploring, and plotting data. Participants will see right away what data science can do for them and the many advantages of using R.

To actively learn new R skills, participants will need R and Rstudio, a free and open-source tool for using R, on their computers; however, those without laptops or R will still benefit from a demonstration of R and a discussion of its features. A handout with links to the presentation and the R examples used along with additional details on where they can learn more will be provided.

Session slides
Speakers
CG

Charles Guedenet

MEL Advisor, IREX
Charles Guedenet serves as a Monitoring, Evaluation, and Learning Advisor within IREX’s Center for Applied Learning and Impact (CALI) where he is responsible for measuring IREX’s global impact and supporting organizational learning, accountability, and data informed decision-making... Read More →
avatar for Jonathan Seiden

Jonathan Seiden

Senior Specialist, Education Research, Save the Children

Friday September 7, 2018 11:30am - 1:00pm EDT
Vista 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Building bridges

1:00pm EDT

Lunch and Demo Tables
Grab a lunch and visit demo tables on the first or second floor!
Friday September 7, 2018 1:00pm - 2:00pm EDT
Reception Area 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Community Building, Building bridges, connections, community and capacity

2:00pm EDT

Blockchain Learning Agenda: Practical MERL Workshop
What do we as MERL practitioners need to know to understand the opportunities and challenges that blockchain technology presents for development?  Our team set out to develop a data-based case study of blockchain pilot projects, to begin understanding what works, what doesn’t, and why, from a MERL perspective. We discovered that MERLing blockchain is not possible based on how little substantive information is available regarding what works, what doesn’t, and why.

None of the many “case studies” available online provided the requisite granularity to achieve what we set out to understand. So, we’re taking this opportunity to share our experiences and thinking with the larger MERL community to crowd-source input on what kind of Learning Agenda would be needed under what circumstances to get the most out a future chance to review data on what works, what doesn’t, and why, to help vet the appropriateness of blockchain for MERL functions, and monitoring during implementation, if it is appropriate, in the form of a mini-workshop during this session.

Participants in this mini-workshop will collaboratively identify key features of a MERL blockchain Learning Agenda, and leave with a better understanding of how important it is to have a learning agenda to help guide them through the process of vetting the appropriateness of blockchain to support their MERL functions, through a 40-minute mini-workshop, and will leave with a early version that can be used in the initial stages of blockchain vetting (a polished version will be distributed electronically to participants after the conference).
Speakers
avatar for John Burg

John Burg

Policy, Planning, and Learning Fellow, CAMRIS International
CM

Christine Murphy

Monitoring, Evaluation, and Learning Specialist, USAID (contractor)
JP

Jean-Paul Petraud

Senior Evaluation Specialist, USAID

Friday September 7, 2018 2:00pm - 3:00pm EDT
Academy Hall Breakout B 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session

2:00pm EDT

Creating a MERL culture at your Nonprofit
When creating a data-driven organization, sometimes the investments with the highest return aren't investments in technology or even people, but rather in culture. To really take advantage of data you need to have a culture that values evidence and rigor. The right technology will not help your organization to use data effectively if a MERL culture is not in place. In this session, you will learn how to go about creating a MERL culture in your nonprofit. In addition to some practical tools you can use, two organizations will share the lessons they have learned trying to create a MERL culture. The presenters will speak from their perspective about how they have navigated myriad obstacles and created real value for their organizations.
Speakers
avatar for Eric Barela

Eric Barela

Director, Measurement & Evaluation, Salesforce.org
Eric Barela is currently the Measurement & Evaluation Senior Manager with Salesforce.org, where he leads efforts to evaluate the impact of its global philanthropic efforts.  Prior to joining Salesforce.org, Eric was the Chief Knowledge & Impact Officer at Partners in School Innovation... Read More →
avatar for Morgan Buras-Finlay

Morgan Buras-Finlay

Measurement & Evaluation Manager, Salesforce.Org
Morgan Buras-Finlay is currently the Measurement & Evaluation Manager with Salesforce.org, where she evaluates the impact Salesforce of technologies. Prior to joining Salesforce.org, Morgan was the Director of Evaluation, Learning & Impact, at the Mission Economic Development Agency, where her work focused on the development of organizational MERL capacity and culture in the nonprofit sector. Morgan received a Masters in Social Work from Columbia University... Read More →
avatar for Kosar Jahani

Kosar Jahani

Director, Impact, Samaschool
Kosar is currently the Director of Impact at Samaschool, an organization that equips low-income job seekers to benefit from independent work while advocating for an economy in which all workers thrive. She leads research projects that measure the organization's social impact, as well... Read More →
avatar for Brian Komar

Brian Komar

VP-Global Impact Engagement, Salesforce.org
Nonprofit innovation; CRM; evaluation; digital media; social enterprises, CSR, and cultural advocacy
avatar for Andrew Means

Andrew Means

Founder, Big Elephant Studios
Andrew Means is a serial social entrepreneur leading organizations that drive the social sector’s use of data and technology. He is co-founder of BrightHive and founder of Big Elephant Studios and Data Analysts for Social Good. He has previously held leadership positions at Uptake.org... Read More →

Friday September 7, 2018 2:00pm - 3:00pm EDT
Academy Hall Breakout A 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Sharing experiences

2:00pm EDT

Evidence-based decision-making & impact: The future of MERL through technical and qualitative tools
Participants will engage in a hands-on session that highlights a few different tools and technologies that can ensure that actionable, quality data is available and accessible to decision-makers as quickly and easily as possible. The Future of MERL is not to collect & report on data in a traditional way; instead it is to iteratively gather quantitative and qualitative data that is relevant, accessible and used to help inform decision-making in real-time.

Session slides
Speakers
GM

Greg Maly

Principle Data Scientist, DAI
Greg Maly, Principle Data Scientist at DAI’s Center for Digital Acceleration, has spent his career working at the intersection of global development and diplomacy, technology, and the arts. In previous roles, he conducted data driven conflict analysis research at the U.S. Department... Read More →
KB

Komal Bazaz Smith

Director of Learning & Adaptive Management, DAI
Komal Bazaz Smith works at the nexus between social impact, strategy and community engagement. She is passionate about using evidence, stories and real-time feedback to continuously improve our ability to support social innovators and initiatives. Komal has almost 20 years of multi-sectoral... Read More →

Friday September 7, 2018 2:00pm - 3:00pm EDT
Vista 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Sharing experiences

2:00pm EDT

From call times to comment analysis: combining digital data sources to understand change
This session is designed to help M&E teams design monitoring and evaluation platforms for tech interventions, with a particular focus on how to use multiple digital data sources to understand the performance and impact of a platform. Websites and Interactive Voice Response (IVR) lines are now commonly used in development programmes to create awareness, motivate community members, and influence knowledge, attitude and behaviour change. Each channel offers the opportunity for users to actively contribute to engage with the platform and participate in surveys and discussion. This panel will discuss the measurement of two such platforms from Girl Effect, both aimed at mid-adolescent girls. Girls Connect is a phone service that relies on IVR technology to deliver content to girls on demand. Callers can access story episodes, and have the option of being connected to a real-life Role Model to further build on the lessons and advice shared in the story content. Springster is a website with short articles, comments from users and interactive polls and surveys. Both platforms have extensive back end capabilities. The types of data generated and collected by each platform varies widely, from online surveys to Google Analytics to call center data to comment analysis. The interplay between these multiple data sources can be complex - and to fully understand impact, we must carefully utilise multiple data sources. This presentation will show how the measurement framework for each platform ‘makes sense’ of multiple data sources by using a common framework to describe reach, engagement, participation and impact. The presenters will discuss the importance of differentiating the ‘levels’ of the framework in order to better understand pathways to impact. The session will include a participatory activity in which session attendees will have the opportunity to apply the measurement framework to their own projects.
Speakers
AJ

Amy Jones

Evidence Manager, Girl Effect
Amy Jones is responsible for the monitoring and evaluation of Girl Effect’s brands and products. Amy has experience in both qualitative and quantitative research methods, with a focus on incorporating principles from behaviour science. Before joining Girl Effect, Amy worked with... Read More →

Friday September 7, 2018 2:00pm - 3:00pm EDT
Academy Hall Breakout C 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Sharing experiences

2:00pm EDT

The ethics of data: Who benefits and who risks? A framework for assessment and decision-making
When collecting, storing and sharing data, we know we need to be mindful of risks of that data being misused. But how to we identify those risks? And how do we balance them against the project needs for the data?

This interactive workshop will walk you through the process of analyzing the benefits of your data, the risks associated with it with some simple metrics to identify which data needs to be collected and which needs to be protected.
Speakers
avatar for Siobhan Green

Siobhan Green

CEO, Sonjara
Siobhan Green is co-founder of Sonjara, an ICT4D firm specializing in technology evaluations, assessments, and building data rich applications for the NGO and international development community. She was the lead investigator for the USAID Global Development Lab Responsible Data Guidelines... Read More →
MK

Mala Kumar

Senior UX Specialist, Sonjara, Inc.

Friday September 7, 2018 2:00pm - 3:00pm EDT
Angle 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Facing our challenges and shortcomings

2:00pm EDT

The other CoP: Building Communities of Practice
Communities of Practice are designed to engage learners in a process of learning constructed around common interests, ideas, passions, and goals. Oxfam, RTI International, and Tetra Tech have each used various methods and technical solutions to engage Monitoring, Evaluation, and Learning CoPs within their respective organizations. Have you been considering starting a CoP at your organization? Do you know what would make a CoP successful? What does it takes to start a CoP? In this session, practitioners from Oxfam, RTI International, and Tetra Tech will share their experiences in starting and maintaining a CoP.

The session will highlight the characteristics of CoPs within these organizations; demonstrate the value of developing a Community of Practice; provide practical tools and examples that others can use to start a CoP; and share learnings from working with Communities of Practice. The session will help you identify the needs that your team can address by creating a CoP as well as talk through the obstacles you may encounter. Whether you’ve been managing a CoP for a long time or looking to just start one, the session will provide a space for discussion, feedback, and sharing of experiences.

Session slides
Speakers
avatar for Molly Chen

Molly Chen

MERLA Specialist, RTI International
Molly Chen is an international development professional specialized in monitoring, evaluation, research, learning and adapting (MERLA) at RTI International. She leads the overall strategy, design, and implementation of MERLA for RTI International’s Global Health Division’s technical... Read More →
HM

Haneen Malallah

Knowledge, Learning, & Accountability Advisor, Oxfam
Haneen Malallah has worked extensively in the social sector in managerial, consulting, and advisory capacities. Her engagements include work with multiple international non-governmental organizations based in the US and assignments with entities operating nationally in Jordan and... Read More →

Friday September 7, 2018 2:00pm - 3:00pm EDT
Balcony D 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Building bridges

2:00pm EDT

The Taming of the Metrics with the Act Against AIDS Metrics Database (AAAMD): MERL with Ingenuity
Launched in 2009 by CDC and the White House, Act Against AIDS (AAA) is an umbrella initiative that combats complacency about HIV and AIDS in the United States by raising awareness among all Americans. AAA launched with three distinct efforts requiring monitoring and evaluation and rose to as many as 18 distinct efforts in 2016. During this time, data across the AAA suite were systematically collected and processed through customized Microsoft Excel (MS-EXCEL) spreadsheets. The expansion of the AAA initiative necessitated a multi-level approach to process evaluation data collection. Monitoring and evaluating AAA requires a collaborative process with multiple stakeholders, including contract staff and community partners, as well as CDC evaluators, campaign implementation staff, and project officers. To ensure prompt, reliable, and uniform quantitative process data is collected across AAA efforts, the AAA evaluation team developed and released the AAA Metrics Database (AAAMD) in 2016.

The AAA evaluation team took a user-centered design approach to the AAAMD, ensuring the interface accounted for users with varying levels of digital and data literacy. It allows CDC to access process data for all AAA efforts in real-time from a single web portal. The AAAMD has been recognized for substantially improving efficiency of quantitative data collection, saving hundreds of hours of labor, and greatly improving productivity.

A panel session with Q&A at the end will provide a detailed walk-through the development, launch, and management of the AAAMD and how it could be used as a model for MERL teams needing to streamline data collection on a similar scale.
Speakers
IP

Ichhya Pant

Senior Associate, ICF International
Ichhya Pant works at the intersection of health, evaluation, data and information and communication technologies (ICTs) with a focus on vulnerable population such as immigrants, refugees, women and children. She is committed to promoting a culture of health worldwide by harnessing... Read More →

Friday September 7, 2018 2:00pm - 3:00pm EDT
Balcony E 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Break Out Session, Strengthening the evidence base on MERL Tech and ICT4D approaches

3:00pm EDT

Ice Cream Social!
When your brain is full, what better remedy than brain freeze? Join us for an ice cream social.
Friday September 7, 2018 3:00pm - 3:30pm EDT
Reception Area 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Community Building, Building bridges, connections, community and capacity

3:30pm EDT

How to win in 20 years
The social sector is changing. Today’s leaders might be tomorrow’s relics. In this lighting talk Andrew will paint a picture for what the future might look like. A future driven by outcomes and validated by data where money will flow to organizations making change on the ground. Gain a glimpse into the future and how you might prepare for it.
Speakers
avatar for Andrew Means

Andrew Means

Founder, Big Elephant Studios
Andrew Means is a serial social entrepreneur leading organizations that drive the social sector’s use of data and technology. He is co-founder of BrightHive and founder of Big Elephant Studios and Data Analysts for Social Good. He has previously held leadership positions at Uptake.org... Read More →

Friday September 7, 2018 3:30pm - 3:35pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Lightning Talk, Exploring the Future of MERL

3:40pm EDT

Scaling storytelling with data science at GlobalGiving
At GlobalGiving, we connect thousands of organizations working in 170 countries with the money, information, and ideas they need to thrive. To carry out this broad a mission with less than 50 full-time staff, we augment our human resources with technology in service of increased effectiveness and scale. In this lightning talk, we’ll describe a new implementation we’ve been working on over the last year to leverage natural language processing techniques and cloud-based microservices to provide better and faster feedback to our nonprofit partners about the reports they publish on their work. We’ll share what we’ve learned about using data science to help our partners tell better stories about the incredible work they do, as well as how we’ve used technology to make sure that our staff is able to spend their limited time focusing on the situations that require particular individual attention. We’ll also touch on how we can use these new tools to help ensure that members of the communities in which our partner organizations work have their personal privacy protected while still having their stories heard. We won’t get too deep into the nuts and bolts in this short session, so this presentation will be geared towards everyone, not just audience members with technical backgrounds.

Nick's slides
Speakers
avatar for Nick Hamlin

Nick Hamlin

Data Scientist, GlobalGiving
As GlobalGiving’s Data Scientist, Nick wears many hats. Whether leading the organization’s data strategy, building and maintaining the GG Rewards algorithm, or designing experiments to measure program impact, Nick lives to make social good data accessible, understandable, and... Read More →

Friday September 7, 2018 3:40pm - 3:45pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Lightning Talk, Sharing experiences, examples, challenges, and good practice

3:45pm EDT

Evaluating the Technology Incubator Fund: Do small grants make a big difference in appetite for technology in programs?
This lightning talk will discuss the findings of an evaluation of Oxfam America’s Technology Incubator Fund (TIF). Since 2013, the TIF has been granting $25,000 to Oxfam program teams interested in implementing a technological solution to their programmatic problems. Projects funded have ranged from mobile data collection tools to global program monitoring systems to websites dedicated to engaging youth in political processes.

Five years and 17 grants later, we sought to understand the effectiveness of TIF as a tool in solving programmatic challenges and increasing appetite for technologies within Oxfam. The lightning talk will share what worked and what didn’t in a small fund with such ambitious goals.

Amy's slides
Speakers
avatar for Amy Hood

Amy Hood

Program Coordinator, LEAD & ICT4D, Oxfam America

Friday September 7, 2018 3:45pm - 3:50pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Lightning Talk

3:50pm EDT

MERL Tech and the American Evaluation Association: Let's build a bridge
The purpose of my talk is to make a case for MERL Tech attendees to get more involved with the American Evaluation Association (AEA) and to provide some avenues for increased involvement. I learned at last year’s MERL Tech DC conference that many in the MERL Tech crowd do not see a place for themselves at the AEA table because there is a perception that AEA is too academic and traditional in its understanding of evaluation. Funny enough, many members of AEA think the same thing. There is a growing call for practicing evaluators to share their learnings as a way of building up a practical knowledge base. MERL Tech attendees can really contribute to this knowledge base because they’re so focused on learning from what actually happens during an evaluation. Also, many AEA members can benefit from the wisdom of MERL Tech attendees with respect to using technology effectively in their practice. In addition to making the case for increased MERL Tech attendee involvement, I will share some ways attendees can get involved, from presenting at the annual conference to getting involved with local affiliates and Topical Interest Groups to blogging about their experiences. Full disclosure: I’m a current AEA Board Member and a practicing evaluator. I want to make sure those of us working in the trenches feel like our contributions are valued within AEA.

Eric's slides
Speakers
avatar for Eric Barela

Eric Barela

Director, Measurement & Evaluation, Salesforce.org
Eric Barela is currently the Measurement & Evaluation Senior Manager with Salesforce.org, where he leads efforts to evaluate the impact of its global philanthropic efforts.  Prior to joining Salesforce.org, Eric was the Chief Knowledge & Impact Officer at Partners in School Innovation... Read More →

Friday September 7, 2018 3:50pm - 3:55pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Lightning Talk, Building bridges, connections, community and capacity

3:55pm EDT

Mapping ICT4D Evidence Base: How much evidence is there really?
There are mixed opinions on whether there is adequate high-quality evidence supporting the effectiveness of ICT4D interventions in low- and middle-income countries. My colleague Hannah Skelly and I conducted a systematic search and screening process to catalog all publicly available impact evaluations across 11 intervention categories and nine development sectors. In this talk I will show the resulting evidence map and discuss the key conclusions about evaluation methods and evidence clusters and gaps for ICT4D.

Annette's slides
Speakers
avatar for Annette Brown

Annette Brown

Director, Research and Evaluation, FHI 360
Annette N. Brown, PhD leads the Research and Evaluation Strategic Initiative at FHI 360 and serves as editor-in-chief for the R&E Search for Evidence blog. Prior to joining FHI 360, Dr. Brown headed the Washington, DC office of the International Initiative for Impact Evaluation (3ie... Read More →

Friday September 7, 2018 3:55pm - 4:00pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Lightning Talk, Strengthening the evidence base on MERL Tech and ICT4D approaches

4:00pm EDT

Irresponsible innovation? Time for ICT4D to grow up and get its M&E house in order
In ICT4D, we literally do not know what we are doing.

Innovative solutions are piloted and trialled - but ​few scale successfully, and we aren't sure why​. We duplicate past experiments with little intentionality about testing different methodologies or iterating on an idea. We don't understand our impacts, intended and unintended, so we may be causing harm to the most vulnerable. Risk identification and mitigation ahead of deployment is generally poor. And innovations can be solutions looking for problems: disconnected from the context, real needs and capacities, and the wider field’s existing efforts.

Understanding the impact of our work through evidence, holding ourselves to account against sound ethical and best practice frameworks, and acting on this knowledge to improve practice over time, would improve outcomes; allow organizations to take better risks; and most importantly reduce harm to populations. These goals are weakened by underinvestment, a lack of openness around learning and evaluation, technical challenges and uncritical discourse. Monitoring and evaluation of innovative projects carried out by staff unaccustomed to new technologies may miss relevant assessment criteria. Pilots, innovation challenges, and other common funding mechanisms may have very light reporting requirements. And we don't have a culture of transparency around learning and evaluation.

But it’s time we professionalized. It's time to stop ‘innovating’ and start taking incremental steps towards excellence, fuelled by evidence.

In this provocative panel, we will discuss the evidence gap is - how many evaluations SHOULD we have? What SHOULD we know? How can we challenge policymakers to commit to starting to change the culture, and establishing a norm of evidence-based action in ICT4D?
Speakers
avatar for Annette Brown

Annette Brown

Director, Research and Evaluation, FHI 360
Annette N. Brown, PhD leads the Research and Evaluation Strategic Initiative at FHI 360 and serves as editor-in-chief for the R&E Search for Evidence blog. Prior to joining FHI 360, Dr. Brown headed the Washington, DC office of the International Initiative for Impact Evaluation (3ie... Read More →
avatar for Michael Eddy

Michael Eddy

Vice President of Analytics and US Country Lead, Global Innovation Fund
Michael Eddy is the Vice President of Analytics and US Country Lead at GIF, bringing a passion for innovative finance and the use of data and evidence for better decision-making. Prior to joining GIF, he co-founded and built Instiglio, where he designed the first impact bond in... Read More →
avatar for Anahi Ayala Iacucci

Anahi Ayala Iacucci

Senior Director, Humanitarian Programmes, Internews
Anahi Ayala Iacucci leads the Internews Humanitarian team as the Senior Director for Humanitarian Programs. Anahi plays a key role in developing Internews’ global Humanitarian program strategies; while also closely supporting teams in the field and managing Internews Emergency Response... Read More →
avatar for Laura Walker McDonald

Laura Walker McDonald

Director of Innovation, Global Alliance on Humanitarian Innovation
Laura Walker McDonald is Director of Innovation at GAHI, the Global Alliance on Humanitarian Innovation. She has spent eight years working on inclusive, appropriate technology for social change, at SIMLab and FrontlineSMS, and prior to that worked for the Red Cross in London. She... Read More →

Friday September 7, 2018 4:00pm - 5:00pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Hard Talk, Facing our challenges and shortcomings

5:00pm EDT

Day 2 Summary and close out
Sum up the day and be on your way!
Speakers
avatar for Maliha Khan

Maliha Khan

Founder, Daira
Maliha Khan is an international development practitioner who focuses on dealing with issues of complexity in measurement and accountability systems; designing initiatives for social change, and leading organizational change processes. An expert in ethical issues and data, she led... Read More →
avatar for Linda Raftree

Linda Raftree

Organizer & Independent Consultant, MERL Tech
Linda Raftree supports strategy, program design, research, and technology in international development initiatives. She co-founded MERLTech in 2014 and also works as an independent consultant. Linda advises Girl Effect on digital safety, security and privacy and supports the organization... Read More →

Friday September 7, 2018 5:00pm - 5:30pm EDT
Academy Hall Main Room 1825 Connecticut Avenue NW, 8th Floor, Washington, DC 20009
  Community Building, Building bridges, connections, community and capacity
 
Filter sessions
Apply filters to sessions.
Wednesday, September 5
Thursday, September 6
Friday, September 7
Academy Hall Breakout A
Academy Hall Breakout B
Academy Hall Breakout C
Academy Hall Main Room
Angle
Balcony D
Balcony E
Chemonics
Reception Area
Vista
World Bank, "I" Building, Room I-1-200
  • Break Out Session
  • Community Building
  • Fail Fest
  • Hard Talk
  • Lightning Talk
  • Plenary
  • Pre-Workshop
  • LEVEL
  • Advanced MERL
  • Advanced Tech
  • Any level MERL
  • Any level of MERL or Tech
  • Any level Tech
  • Approaches
  • Beginner MERL
  • Beginner Tech
  • critical perspectives
  • debates
  • frameworks
  • Hard talks
  • Intermediate MERL
  • Intermediate Tech
  • methods
  • Research
  • Popular