Background and Project Focus

For a class project, three classmates and I decided to perform usability testing on Duolingo’s ability to streamline its language-learning process. The project was inspired by our team’s past experiences using the app and familiarity with its cultural media presence. Preliminary research we performed and background knowledge suggested that there were many complaints about the app’s cognitive overload on its users. As such, we focused our usability testing around identifying points of confusion and users’ ability to transform the app into a comfortable learning environment. 

Usability Testing

For our study, we recruited five participants who had never used Duolingo before and five participants who had some experience with the app before, regardless of the duration or recency. We then performed hour-long usability testing sessions in which we walked participants through four tasks in the Duolingo app. During this process, we tracked qualitative observational data regarding the participants’ ability to navigate through the task, taking note where they encountered difficulties and frustrations. Throughout usability testing, we also kept track of users’ expectations to compare against the actual implementation, allowing us to consider what more intuitive options may look like. Our analysis involved performing a visual thematic analysis, where we identified commonalities in the participants’ experiences. During the analysis process, we recognized that there was very little difference between the two user groups we had initially divided the participants into. Taking this into consideration, we pivoted to analyze the entire sample as a whole without much regard for comparative analysis. 

Findings and Suggestions

After conducting our analysis, we looked through our various findings and considered the severity of each trend that we observed. We did so by identifying the most debilitating pain points we observed and the frequency at which they occurred. The most prominent problem that the majority of our participants had difficulty simply navigating to the notification settings in the Duolingo app. Much of the confusion was caused by a fairly prominent bell icon, which is often associated with notification settings, leading to a completely unrelated page. Learning that the notifications settings were under the profile in the app was very unexpected for most of the participants. Furthermore, the process of adjusting notifications also proved tedious, as participants had to toggle off sixteen individual types of notifications, spread apart four separate pages. Several participants remarked on the inefficiency of the format, expressing that they would appreciate having a single button that silences all the notifications at once. 

Even after turning off notifications within the app, most participants were not completely certain that they had successfully turned off the notifications, claiming that they would prefer stronger feedback than the icons turning from blue to gray. Some participants mentioned that they would expect turning Duolingo’s notifications off to be more difficult, especially given the app’s humorous social media reputation of being difficult to turn off. In our report, we discussed the idea of a save button with a confirmation screen or a pop-up that helps clarify the user’s new settings. Additionally, some participants mentioned the buttons themselves were fairly small, pointing out that the already tedious task required a level of precision to avoid mistakenly turning notifications back on.

Much of Duolingo’s terminology, much of which helps establish the app’s game-like education method and paid content, also became a point of cognitive overload. Participants expressed confusion over terms like “streak freezes,’ “hearts,” “smart scheduling,” “personalized practice,” and “unlimited legendary.” Given that users are confronted with these terms very early on, it would make sense to have some form of directory to define or sort these terms to reduce initial cognitive overload that could interfere with a first-time user’s experience.

Delivering Insights

To finish up our project, we presented our insights to the rest of our class. Our discussion of our findings included both the high-severity issues listed above and a nuanced discussion of lesser issues, such as branding and profile setup. Our presentation included quotes as qualitative evidence (not included here due to confidentiality agreements) as a demonstration of how that informed our thematic analysis. Treating our audience like a group of stakeholders, we provided multiple actionable suggestions, enabling hypothetical designers to have more freedom to consider an elegant solution to the observed issues. While doing so, we discussed the feasibility of addressing certain issues, with respect to Duolingo's infrastructure as a business and their intent, to further identify which problems to prioritize.