Role & Responsibility

Methods & Tools

Project Background

A 2017 survey among nearly 48,000 college students indicated that 64% of college students sometimes feel very lonely in the past year. This perceived loneliness had not only affecting their academic performances but also threatening their mental health, leading to dramatic increase in demand mental health services. As a result, POP was developed to make it easier and more comfortable for college students to create meaningful social connections and fight against potential mental issues. With the release of their prototype, they are seeking to understand the usability and collecting users' feedback to improve their app.

Client Kick-Off Meeting

What is POP?

POP is a student only social network application that helps students to connect and build deeper friendships that can make them truly happy and fulfilling under a safe environment.

Vision of POP

Social Network Inside Campus

Heuristic Evaluation

To gain a whole picture of the current prototype and identify potential usability problems, I conducted a heuristic evaluation. By applying the evaluation criteria from Neilsen and categorize the discovered issues into four levels, I was allowed to provide early feedback back to the design team and make a more efficient plan for further usability testing.

Evaluation Criteria

Neilsen's 10 Usability Heuristic Principles:

Issue Levels

For all the usability issues discovered, they would be classified to different level based on its seriousness, urgency and influence. There are four different levels:

Level 0 - Good design

Level 1 - Serious issues that must be fixed

Level 2 - Minor issues that could be fixed

Level 3 - Further test needed

Evaluation Findings

After evaluated the app prototype, I summarized my findings into a report in which I identified the severity of each issue I discovered with suggestions to fix them. The walk through of the prototype could be separated into three main phases: account registration, setting up personal profile, and matching with friends. I listed some issues in each phase as examples to explain some of the common issues that were discovered in the prototype.

1. Account Registration

The prototype had some good designs in error preventions in the registration phase.

Take account registration for example, when typing into the email for register an account, the app would provide instructions of the requirement of registration email and password that not only allow users to learn what would be required in setting up their accounts but also provide clues whether their current set up is matching those requirements.

However, there were also consistency problems such as the backward arrow signs had different direction in different pages, which also created the problem of not matching between system and real world. An arrow sign directing to the top usually means folding up items rather than moving back to the previous page. This was an issue that must be fixed before launching.

Another obvious consistency problem in the registration phase occurred at the forgot password page. As users might forget which email they used for registration, the reminder under the forget password page should also aligned to the registration page that informed users an EDU email must be supplied here. It was also a problem related to error prevention and help users recognize, diagnoses, and recover from errors as this reminder instruction was not sufficient to allow users correctly use the forgot password function.

1. Align the backward arrow signs, make them direct to left.
2. Align the reminder in forgot password page with the registration page.

2. Setting Up Personal Profile

Proceeding to the personal profile phase, many of the issues were related to flexibility and efficiency of use and helping users recognize, diagnoses, and recover from errors.
Take selecting major in profile set up as an example, users were asked to tap the input box and search for their majors. However, the app didn't asked users to identify themselves as graduates or undergraduates, users might not find their major from the first few attempts. Moreover, the switch of undergraduates vs graduates was just a reminder below the input box, users might not even recognize that this switch option exists, leading them to keep try and fail on searching for their majors.

Make the selection of “graduate” or “undergraduate” an earlier process before searching and inputting their majors. As a result, users can recognize there are different searching content under these two categories.

Another example for issues of consistency and matching between real world is the setting of personality. Users were allowed to identify their personality while setting up the profile to get users better matches in the future. Nonetheless, these self-identified scale were only slide bars without any level or description. People might have difficulties understanding of the personality tendency and not aligned to what extends of tendency these slide bars indicated.

Apply level structures into each personality with brief example or explanation so that users can better understand what it means.

3. Matching with Friends

As for matching with friends, there were some good designs for flexibility and efficiency of use.

An example from viewing matches, users were allowed to see what interests they have in common that brought them to match for each other. It not only acknowledged users of the rationale of the results from algorithm but also gave more information for users to start their conversation and ice-break with their matches.

An example for issues of recognition rather than recall was the "unmatch" function. Unmatch could only be found in the pop up list after tapping the report flag of the match. In many scenarios, users would unmatch with certain matches without reporting anything about them. Accordingly, these users would not be able to find the unmatch button. Moreover, the unmatch was displayed with an icon of close window, with a separate line at the bottom of the window. This layout might lead users to assume that it was a close window button rather than an unmatch button if they were not paying attention.

Unmatch should serve as an option in the matches' profile. When people wanted to disconnect with someone on social media applications, the process is usually entering to that person’s profile page and find the option there.

Evaluation Report

Testing Plan

Participant Screening

Since POP only serves for college students, it was essential to screen out the participants we wanted to participate in the usability testing. As a result, I created a participant screener as a tool to identify whether the participants we recruited met our criteria and manage whether I have recruited the ideal amount and distribution of participants I wanted.

Started with introducing the objective of the study, followed with logistic, participants requirement, recruitment target, participant quotas and screening questions, with this tool, I end up recruited five qualified participants within a week.

Testing Script

To successfully conduct the usability testings, I planned the testing script to not only prepare myself for the testing but also organize the testing plan and make sure the important issues I would like to test were all covered. Moreover, by listing the ice-break questions, transitions between different sections and tasks, the testing went more smoothly with the script.

Usability Testing

Testing Process

When proceeded to the testing phase, the POP prototype got some new updates, as a result, the testings were conducted using the new version of prototype.
Most of the testings were conducted onsite, while only the session with participant 2 was conducted in a remote setting. In the onsite setting, participants are asked to enter the prototype through their own mobile devices and I would sit beside them to facilitate the testing and observe how they use the prototype. While in the remote setting, the participant accessed the prototype with her laptop and shared her screen using online meeting tool. I facilitated the testing session through the online meeting tool and provided an additional text instruction to the participant. With the text instruction, participant had a better overview of the whole session and would not be lost following the instruction to complete all tasks.
Both settings followed the same process with four main tasks for participants to complete:

  • Task 1 - Onboarding: Complete registration and set up a personal profile.
  • Task 2 - Overall Click Around: After making to the main page, freely explore the app.
  • Task 3 - Match and Message: Match with a friend a start conversation.
  • Task 4 - Profile Editing and Filter: Edit personal profile and set up matching filter.

For each task, there were related follow up questions to identify their thoughts on certain usability issues and a five-point scale for a general usability rating for completing each task.

Findings and Recommendations

I classified the issues found from the testing into two categories, usability issues and feature/system issues, to better addressed the influence level of these issues for future improvements.

Usability Issues

For all the usability issues I found, there were four issues that were much more serious than the others. I highlighted these four issues followed by my recommondations as below:

1. Users mistook the first profile they saw as theirs after completing account setting

After the completion of account registration, it directly moved to the main page and show up with a potential match profile to the user.

From the usability testings:

  • 4/5 participants did not notice that they have completed the registration and personal profile setting process when they made to the main page.
  • 4/5 participants mistook the matched profile in the main page as their own profile.
  • 3/5 participants expected to see their complete profile right away after setting up.
  • There was no navigation for users to identify where they currently were or were about to heading to.


Add a transition page after the completion of registration and profile setting that allows users to choose whether to view the profile the just set up or start matching with friends. With the transition page, users were allowed not only to choose where they would like to visit first but also acknowledge they have completed their profile setting and could be proceeding to the next page.

2. Interest category icons

The interest category icons are not meaningful in this page. It just caused confusion and occupied space.

From the usability testings:

  • 5/5 participants did not recognize these icons and asked to clarify what they were after seeing the page.
  • 4/5 participants thought that those icons were not connected to the interest categories.
  • 3/5 participants thought that those icons were causing confusion for them to understand the content on the page.
  • The various icons were not providing information to the users but only making the screen crowded and confusing.


Remove the interest category in the matched profile since it already showed the common interest they shared. The space could stay empty to make less pressure for users to view or add some other useful information to help users learn more about this person. (such as a brief introduction of themselves)

3. Need confirmation for making changes

When editing their profiles, after users selected new items in the interest list, there was no confirmation that inform them whether they have successfully added and made the changes they wanted. (same as removing items)

From the usability testings:

  • 5/5 participants were confused whether they have successfully made the changes while editing their profile.
  • 3/5 participants accidentally left the profile with the content that they were not desired to.
  • Without a confirmation information, users could only check whether they have successfully made the changes by reviewing their whole profile.


Simply add an CONFIRM button in the page for users to confirm saving the changes they made. Moreover, adding a pop-out confirmation window if users attempted to leave the page before saving all their changes can also provide a better error prevention for users.

4. Filter issues

While the app provided a filter feature for users to set up some criteria for matching with new friends, there were issues about where to enter this feature and how it would work.

From the usability testings:

  • 3/5 participants were not able to complete the task of setting up filter because they could not find the feature.
  • 4/5 participants rated setting up filter as a difficult task to complete.
  • 3/5 participants were questioning some of the settings in the filter would significantly reduce the matches of certain groups, which was against the original intention of this app.


Move the entrance to the filter setting to the main page (e.g. adding a hidden slide button). Since users were viewing the matches in this page, it would be more rational for them to find the filter here.

A further research for what users expected to have the filter feature on matching would be necessary. Since POP committed to provide equality and equity to all groups of students with the ability to connect with others, the usage of filter and what conditions were allowed to set up in the filter would need more research.

Feature/ System Issues

For feature/ system issues, I concluded them into three further research questions:

  • Did the unique feature of POP really attracting to users?

POP highlighted its unique differentiators as:
1. Providing steps of progress to help users connect with others.
2. Staying anonymous before making to the progress of the connection.
3. Students from all groups are allowed to make connections without obstacles.

However, from the feedback of the participants, those features were not attractive enough for them to use this new app in addition to the apps they have already been using. Therefore, a further research on these features as well as what users are expecting for a new social network app would be beneficial for the development of POP.

  • What interactions do users need for social networking apps?

Following the previous question, the interaction progress was the feature that the participants had the most diverse preference. The participants loved this feature and thought it was very useful otherwise they thought that it was a totally waste of time for making connections. Hence, this feature could be a breaking point but still needed more research and adjustments.

  • Did the current version of app really serve the equality and equity as it planned ?

As the vision of POP was to help students suffering isolation and loneliness to have an approach of making connections, the current features such as filters and interaction progress were actually stopping students with certain traits losing the opportunities to make new connections. Accordingly, a further research on the needs of these groups of students would really allow POP to become the new approach for them to conquer their loneliness and build healthy relationships with others.

Let's chat!

Copy right © 2021 Chun-Wei Wang. All rights reserved.