MEL support to Westminster Foundation for Democracy (WFD)

What we did

Innovation through remote video interviews

We developed an asynchronous outcome map framework and tools to collect feedback via velfies (video selfies), where participants of training programmes answer a set of questions that are sent in advance. The end product is a 10-minute video combining all the 12 participants’ feedback from five countries.

When the COVID-19 pandemic started, WFD was in the middle of their African Liberal Network Women’s Leadership Programme.  As with all the training at the time, the face-to-face programme had to move into the online world. They had been collecting feedback from participants via standard surveys, but they wanted to probe participants’ experience of the transition more deeply and to have their input on the direction of the programme going forward.

Overall, our Monitoring, Evaluation and Learning (MEL) advice to WFD’s Political Party Officers (PPOs) ranged from the development and tracking of outcome maps, the design of new projects to tips on how to capture learning and adapt programmes accordingly.

As shown in the video below, our solution to ask participants to film themselves in asynchronous velfies provided evidence of outcome maps’ progress markers, feedback to adjust the programme, identification of case studies and, at the same time, stories to follow up.

 

What participants thought of the African Liberal Network Women’s Leadership Programme

Client Feedback

“The feedback we gathered through reporting was often dry and difficult to put into context. Shifting to a visual format gave these women a voice to share their experiences in their own words, resulting in much more compelling and useful stories.”
Graeme Ramshaw Director of Research and Evaluation, WFD
“The idea of capturing the participant’s feedback via velfies was both an obvious thing to do, but also a big step into the unknown for us. I found Isabela’s willingness to ‘learn-by-trying’ approach just what we needed to realise our ideas. In addition to a video that really captures the different personalities and reflections of the programme, Isabela provided an excellent Sense Learning analysis. We now have a feedback product that can have multiple uses and audiences, as well as some insightful learning from our Women’s Leadership Programme.”
Adele Poskitt Head of Multi-Party Office, WFD

Lessons Learned

Not all is a garden of roses

Even though videos have been used in evaluations and there is a vast literature base about participatory video evaluations (PVE), using video for more mundane and day-to-day tasks of MEL – such as collecting feedback – is not widely done. I hope that this feedback video inspires other practitioners and donors to explore new ways of carrying out MEL, making it more creative and useful to all involved.

The advance of technology has given us the many tools we have today. Yet, despite having all this technology at our fingertips, it is easy to fall into the habit of using the same tools we have always used.  The list below presents some of the advantages and disadvantages we found in capturing participants’ feedback via asynchronous videos (instead of via online surveys).

ADVANTAGES

  • Answers in the video were richer and deeper compared with answers in previously written surveys.
    • The video format brings stories to life and transmits emotions difficult to get across in written form.
  • Artificial intelligence helps to speed up and keep translation and subtitles costs down
    • Online tools such as Sonix (automated transcript and subtitle) and Akbar (translation) do 70% of the work, with a human translator/subtitle expert required only to double-check transcript/translation and subtitles timing.
  • Cloud environment (such as Dropbox) makes it possible to collaborate with video-editors in countries with lower exchange rates and helps keep the costs down.
  • MEL feedback becomes a non-extractive activity, where participants also get something out of it.
    • Participants received a velfies guide with tips about self-recording.
    • The process of providing feedback was also an opportunity for participants to practice speaking to the camera.
    • Participants received feedback on their videos and how they could improve them in the future.
  • Videos have multi-use and the material can be used beyond MEL, such as communications.
    • Shorter videos (30 sec to 1 min) can be extracted and used in social media.
  • People remember 22 times more with video than when data is given via a text.
    • We believe that the messages in this feedback video are more memorable than in a report.

DISADVANTAGES

  • Variable image and audio quality of different participants, because they did not always follow instructions.
  • The costs and time involved in engaging with participants, video-editing, translation and subtitles are higher than in written surveys.
    • MEL specialists often do not have video editing skills, and outsourcing costs might prove prohibitive.
  • Resistance from those averse to change, as with any new practice.

To Learn More

The support was part of our partner’s INTRAC semi-independent Monitoring, Evaluation and Learning Support Unit (MEL Support Unit). The experience of using video tools as part of data collection for MEL was unique. If you want to read more about the experience, we summarize it in the blog post Video selfies” are a winner amongst all the new virtual working tools.

Contact us

Let’s explore how we can help you and your project

And don’t forget to follow us on social media!