Evaluating user experience of learning AdWords through Youtube videos

Google AdWords is an online advertising service that enables advertisers to compete to display brief advertising copy to web users.

Context:

To improve the experience of help videos and increase the number of resolutions in a year, Google decided to create a video curriculum that is based on user needs and expectations. Google had seen an increase in the number of video uptake rates but was not able to proportionally relate it to the case escalations. The video engagement team launched a program to study user engagement with help videos to understand how AdWords users are consuming videos, what are their needs and expectations.

My Role:

I led the qualitative research consisting of interviews and usability testing and conducted the same with AdWords users. To ensure we are covering the right audience studies were conducted with AdWords users who prefer to watch a video than reading text to solve a problem. Over 15 participants were recruited for the study to uncover their needs and frustrations around video learning experiences and to get feedback on existing AdWords help videos which enabled me to define specific use cases of video learning the pain points in existing video formats. A combination of critical incident method and think aloud was used to understand user behavior towards videos.

After the study, findings were collated in the form of a user journey consisting of pain points and challenges that users face. I created recommendations for video types and formats and shared with the vide engagement team. As the user research lead, I participated in the creation of the new video curriculum and the insights from the study heavily influenced the new curriculum and design of the video program.

Result:

The new curriculum led to a higher uptake rate in multiple video types that were introduced in the following year.

Few Findings

• Novice users rely heavily on visual learning aids like videos and/or articles with images/videos. We should focus on ease of learning for the novice user and study usability over time as users develop expertise in using the help features.
• Users consume videos to learn about AdWords in both, a reactive and a proactive manner. Reactive users have mission critical task/s on the help platform where it does matter how fast people find a solution. These users need short, quick and interface heavy, hence easy (and fast!) to learn.
• It's also important to note that mission critical tasks might lead a user to not solely rely on videos to solve a problem, if it's not the fastest way to solve their problem.
• Proactive users are willing to spend time on watching videos and learning about AdWords in general or about something specific, which is mission driven, but not critical.

Reasons why users prefer NOT to watch videos:

• People feel they are not in control of the content they are consuming while watching videos, because of the linear nature of videos. When people read text, they are able to scan through and skip sections of text that they think are not applicable to them.
• People find reading faster than listening to a person talk, esp. in videos where there is a host leading the video. People could read the subtitles fasters than hear the person talk and that led them to find an article and read that instead.