A 2017 survey among nearly 48,000 college students indicated that 64% of college students sometimes feel very lonely in the past year. This perceived loneliness had not only affecting their academic performances but also threatening their mental health, leading to dramatic increase in demand mental health services. As a result, POP was developed to make it easier and more comfortable for college students to create meaningful social connections and fight against potential mental issues. With the release of their prototype, they are seeking to understand the usability and collecting users' feedback to improve their app.
POP is a student only social network application that helps students to connect and build deeper friendships that can make them truly happy and fulfilling under a safe environment.
‍
To gain a whole picture of the current prototype and identify potential usability problems, I conducted a heuristic evaluation. By applying the evaluation criteria from Neilsen and categorize the discovered issues into four levels, I was allowed to provide early feedback back to the design team and make a more efficient plan for further usability testing.
Neilsen's 10 Usability Heuristic Principles:
For all the usability issues discovered, they would be classified to different level based on its seriousness, urgency and influence. There are four different levels:
Level 0 - Good design
Level 1 - Serious issues that must be fixed
Level 2 - Minor issues that could be fixed
Level 3 - Further test needed
After evaluated the app prototype, I summarized my findings into a report in which I identified the severity of each issue IÂ discovered with suggestions to fix them. The walk through of the prototype could be separated into three main phases: account registration, setting up personal profile, and matching with friends. I listed some issues in each phase as examples to explain some of the common issues that were discovered in the prototype.
The prototype had some good designs in error preventions in the registration phase.
Take account registration for example, when typing into the email for register an account, the app would provide instructions of the requirement of registration email and password that not only allow users to learn what would be required in setting up their accounts but also provide clues whether their current set up is matching those requirements.
However, there were also consistency problems such as the backward arrow signs had different direction in different pages, which also created the problem of not matching between system and real world. An arrow sign directing to the top usually means folding up items rather than moving back to the previous page. This was an issue that must be fixed before launching.
Another obvious consistency problem in the registration phase occurred at the forgot password page. As users might forget which email they used for registration, the reminder under the forget password page should also aligned to the registration page that informed users an EDU email must be supplied here. It was also a problem related to error prevention and help users recognize, diagnoses, and recover from errors as this reminder instruction was not sufficient to allow users correctly use the forgot password function.
Suggestion:
1. Align the backward arrow signs, make them direct to left.
2. Align the reminder in forgot password page with the registration page.
Proceeding to the personal profile phase, many of the issues were related to flexibility and efficiency of use and helping users recognize, diagnoses, and recover from errors.
‍
Take selecting major in profile set up as an example, users were asked to tap the input box and search for their majors. However, the app didn't asked users to identify themselves as graduates or undergraduates, users might not find their major from the first few attempts. Moreover, the switch of undergraduates vs graduates was just a reminder below the input box, users might not even recognize that this switch option exists, leading them to keep try and fail on searching for their majors.
Suggestion:
Make the selection of “graduate” or “undergraduate” an earlier process before searching and inputting their majors. As a result, users can recognize there are different searching content under these two categories.
Another example for issues of consistency and matching between real world is the setting of personality. Users were allowed to identify their personality while setting up the profile to get users better matches in the future. Nonetheless, these self-identified scale were only slide bars without any level or description. People might have difficulties understanding of the personality tendency and not aligned to what extends of tendency these slide bars indicated.
Suggestion:
Apply level structures into each personality with brief example or explanation so that users can better understand what it means.
As for matching with friends, there were some good designs for flexibility and efficiency of use.
An example from viewing matches, users were allowed to see what interests they have in common that brought them to match for each other. It not only acknowledged users of the rationale of the results from algorithm but also gave more information for users to start their conversation and ice-break with their matches.
An example for issues of recognition rather than recall was the "unmatch" function. Unmatch could only be found in the pop up list after tapping the report flag of the match. In many scenarios, users would unmatch with certain matches without reporting anything about them. Accordingly, these users would not be able to find the unmatch button. Moreover, the unmatch was displayed with an icon of close window, with a separate line at the bottom of the window. This layout might lead users to assume that it was a close window button rather than an unmatch button if they were not paying attention.
Suggestion:
Unmatch should serve as an option in the matches' profile. When people wanted to disconnect with someone on social media applications, the process is usually entering to that person’s profile page and find the option there.
Evaluation ReportSince POPÂ only serves for college students, it was essential to screen out the participants we wanted to participate in the usability testing. As a result, I created a participant screener as a tool to identify whether the participants we recruited met our criteria and manage whether I have recruited the ideal amount and distribution of participants I wanted.
Started with introducing the objective of the study, followed with logistic, participants requirement, recruitment target, participant quotas and screening questions, with this tool, I end up recruited five qualified participants within a week.
To successfully conduct the usability testings, I planned the testing script to not only prepare myself for the testing but also organize the testing plan and make sure the important issues I would like to test were all covered. Moreover, by listing the ice-break questions, transitions between different sections and tasks, the testing went more smoothly with the script.
‍
When proceeded to the testing phase, the POP prototype got some new updates, as a result, the testings were conducted using the new version of prototype.
‍
Most of the testings were conducted onsite, while only the session with participant 2 was conducted in a remote setting. In the onsite setting, participants are asked to enter the prototype through their own mobile devices and I would sit beside them to facilitate the testing and observe how they use the prototype. While in the remote setting, the participant accessed the prototype with her laptop and shared her screen using online meeting tool. I facilitated the testing session through the online meeting tool and provided an additional text instruction to the participant. With the text instruction, participant had a better overview of the whole session and would not be lost following the instruction to complete all tasks.
‍
Both settings followed the same process with four main tasks for participants to complete:
For each task, there were related follow up questions to identify their thoughts on certain usability issues and a five-point scale for a general usability rating for completing each task.
I classified the issues found from the testing into two categories, usability issues and feature/system issues, to better addressed the influence level of these issues for future improvements.
For all the usability issues I found, there were four issues that were much more serious than the others. I highlighted these four issues followed by my recommondations as below:
After the completion of account registration, it directly moved to the main page and show up with a potential match profile to the user.
From the usability testings:
Add a transition page after the completion of registration and profile setting that allows users to choose whether to view the profile the just set up or start matching with friends. With the transition page, users were allowed not only to choose where they would like to visit first but also acknowledge they have completed their profile setting and could be proceeding to the next page.
The interest category icons are not meaningful in this page. It just caused confusion and occupied space.
From the usability testings:
Remove the interest category in the matched profile since it already showed the common interest they shared. The space could stay empty to make less pressure for users to view or add some other useful information to help users learn more about this person. (such as a brief introduction of themselves)
When editing their profiles, after users selected new items in the interest list, there was no confirmation that inform them whether they have successfully added and made the changes they wanted. (same as removing items)
From the usability testings:
Simply add an CONFIRM button in the page for users to confirm saving the changes they made. Moreover, adding a pop-out confirmation window if users attempted to leave the page before saving all their changes can also provide a better error prevention for users.
While the app provided a filter feature for users to set up some criteria for matching with new friends, there were issues about where to enter this feature and how it would work.
From the usability testings:
Move the entrance to the filter setting to the main page (e.g. adding a hidden slide button). Since users were viewing the matches in this page, it would be more rational for them to find the filter here.
A further research for what users expected to have the filter feature on matching would be necessary. Since POPÂ committed to provide equality and equity to all groups of students with the ability to connect with others, the usage of filter and what conditions were allowed to set up in the filter would need more research.
For feature/ system issues, I concluded them into three further research questions:
POP highlighted its unique differentiators as:
1. Providing steps of progress to help users connect with others.
2. Staying anonymous before making to the progress of the connection.
3. Students from all groups are allowed to make connections without obstacles.
However, from the feedback of the participants, those features were not attractive enough for them to use this new app in addition to the apps they have already been using. Therefore, a further research on these features as well as what users are expecting for a new social network app would be beneficial for the development of POP.
Following the previous question, the interaction progress was the feature that the participants had the most diverse preference. The participants loved this feature and thought it was very useful otherwise they thought that it was a totally waste of time for making connections. Hence, this feature could be a breaking point but still needed more research and adjustments.
As the vision of POP was to help students suffering isolation and loneliness to have an approach of making connections, the current features such as filters and interaction progress were actually stopping students with certain traits losing the opportunities to make new connections. Accordingly, a further research on the needs of these groups of students would really allow POP to become the new approach for them to conquer their loneliness and build healthy relationships with others.
Copy right © 2021 Chun-Wei Wang. All rights reserved.