MERL Tech DC 2018

In ICT4D, we literally do not know what we are doing.

Innovative solutions are piloted and trialled - but ​few scale successfully, and we aren't sure why​. We duplicate past experiments with little intentionality about testing different methodologies or iterating on an idea. We don't understand our impacts, intended and unintended, so we may be causing harm to the most vulnerable. Risk identification and mitigation ahead of deployment is generally poor. And innovations can be solutions looking for problems: disconnected from the context, real needs and capacities, and the wider field’s existing efforts.

Understanding the impact of our work through evidence, holding ourselves to account against sound ethical and best practice frameworks, and acting on this knowledge to improve practice over time, would improve outcomes; allow organizations to take better risks; and most importantly reduce harm to populations. These goals are weakened by underinvestment, a lack of openness around learning and evaluation, technical challenges and uncritical discourse. Monitoring and evaluation of innovative projects carried out by staff unaccustomed to new technologies may miss relevant assessment criteria. Pilots, innovation challenges, and other common funding mechanisms may have very light reporting requirements. And we don't have a culture of transparency around learning and evaluation.

But it’s time we professionalized. It's time to stop ‘innovating’ and start taking incremental steps towards excellence, fuelled by evidence.

In this provocative panel, we will discuss the evidence gap is - how many evaluations SHOULD we have? What SHOULD we know? How can we challenge policymakers to commit to starting to change the culture, and establishing a norm of evidence-based action in ICT4D?