BBC World Trust - Usability Testing of IVRS for maternal health education

IVRS Usability Testing

Context:

BBC WST initiated an IVRS for frontline healthworkers focusing on shaping demand and practices to improve family health in Bihar, India. At this time, they wanted to improve the IVRS based on the user needs generated through usability testing.

The IVRS training tool is divided into courses accessible to frontline health workers through their mobile phones by dialing a number, similar in nature to a customer care service. Specific instructions about navigating through the training course are provided at the beginning and end of each lesson by a male voice. 

Research Location:

The districts (in Bihar) selected for research were based on parameters such as geographical distribution, urban and rural distribution and frontline health worker distribution. We conducted research in three of the eight priority districts, namely Samastipur, East Champaran and Patna. These particular districts were selected as representatives of the three dominant regional languages of Bihar, namely Maghai, Maithli and Bhojpuri. Each selected district was representative of one distinct language in order to validate the receptivity of script of the IVRS content across different regions in Bihar.
 

My role: 

I led a team of researchers that travelled to Bihar, India along with the necessary lab equipment that we set up in an empty room, along with an observation room. In order to understand the user-friendliness of the prototype, task based usability testing was conducted followed by an interview with frontline health workers (ASHAs, AWWs, and ANMs) in a controlled lab environment. 
Each user was asked to perform specific tasks related to navigating through various sections of the certificate training course. For instance, they were asked to call up the IVR number to listen to the welcome message, and access the lessons within the chapter, navigate from one lesson to the other, and play the quiz at the end of the chapter.

Findings:

Upon completion of all tasks, we sought specific inputs around relevance of content, presentation of content (e.g., language, speed, voice and tone), and course structure and length. We also probed into cost perception and effective promotion channels to allow successful implementation of the service

The observations and insights were extracted for different aspects (of the frontline health workers responses to IVRS) like navigation, content type and presentation, value and cost perception and promotion. These were used to formulate recommendations to address the shortcomings in the IVRS and inform improvements and changes.