Design Brief
Project Quaker is a service design project for the provost office at the University of Pennsylvania in Fall 2020.
As the university gradually reopens in the spring semester and the weekly testing capacity increases up to 40,000, it was very important to leverage an effective testing strategy on top of other safety protocols to control the spread of COVID within the Penn community and limit the spillover effect on the Philadelphia community. The university's COVID task force was eager to implement any solutions that can help with student compliance issues.
We need to ensure that people understand that testing is on top of other things. You can’t give up on safety measures. Testing is an extra component.
Penn's Chief Wellness Officer
01 Research
Tools
Online survey
Benchmark research
1:1 stakeholder interviews
Field Observations
Randomized interviews at testing site
information audit
social media audit
Benchmark research
1:1 stakeholder interviews
Field Observations
Randomized interviews at testing site
information audit
social media audit
We conducted intensive research using many tools to help us understand all aspects of the problem and COVID-19 in general.
We used benchmark research of different university's COVID policies to help us have a productive conversation about what Penn's new policy should be like with our clients and other stakeholders. Our survey gave us a broad understanding of the student demographic and their general frustration and confusion. Some of the responses led to in-depth interviews with the individual students.
The 1:1 stakeholder interviews with students, admins, behavioral marketing experts, health experts, the randomized interviews at Penn's testing site and Penn students' social media audit gave us quotes about how people really perceive the pandemic and the testing experience.
Additionally, because of the developing nature of the COVID-19 pandemic, we had to constantly educate ourselves and re-align ourselves with our client throughout the project and our research continued throughout the project.
02 Insights Synthesis
After research, we synthesized the insights using quantitative and qualitative analysis and used graphs to communicate our findings clearly with our clients.
Tools
Website audit
Journey Map
Persona
Interviews
Surveys
Journey Map
Persona
Interviews
Surveys
Quantitative Result
We sent out a google form survey to Penn students, and learned that:
47.1%
got tested weekly, as recommended
91.4%
got covid information mainly from university wide emails
68.1%
did not understand the contact tracing or Red Pass process
48.5%
had participated in indoor gatherings of 10+ people in the past week
Interviews
From 18 in-depth interviews with Penn students (undergrads, grads, PhD, etc), we heard:
"Public health is not enough as an incentive”
"I don’t really know what’s happening and I can’t find clear info online or through emails"
“It's awkward to be spitting in the middle of the room with others who are unmasked and it took me forever.”
From 12 interviews with university admins, public health experts and behavioral scientists we heard:
"Students need to feel like they're part of the Penn community."
“Scents can help people salivate but it could contaminate the sample depending on how you use it.”
”A regret lottery system might work well when you want to incentivize people to participate in medical procedures”
Journey MapS
To fully understand how a student go from incentives to receiving a test, we mapped out all the steps they might take. This convoluted journey map shows us that there are multiple drop off points and students can easily become one time testers or not get tested at all.
When Penn ran saliva testing pilots, we each went through the testing process ourselves and observed others at the testing site to measure the differences between nasal swab tests and saliva tests. This new process involves about 20 steps, 9 changed steps comparing to the nasal swab test and overall more time consuming.
Information Audit
We compiled the table below to illustrate what COVID-related information can be found on each channel. It is worth noting that none of the channels had all the information and that it was extremely difficult to find the program or school-specific information on any of these channels.
User Personas
From the interviews, survey and our observation, we found 5 main typologies that Penn students fall into: the "responsible undergrads", the "required testing grads", the "normal lifers", the "I'm safe, so why test?", and the "positivity scared".
03 Ideation
Based on insights, we generated our problem statements and HMW statements for each of the three areas identified: incentives, information and student experience. Then, we started ideating in the three areas and used a few tools to help narrow down to the solutions that are impactful and implementable.
Tools
clustering
impact vs effort, 2x2 matrix analysis
assumption mapping
impact vs effort, 2x2 matrix analysis
assumption mapping
Problem statements
Incentives:
Students lack the motivation to get continuously tested.
HMW incentivize Penn students to comply with the university’s testing frequency requirements?
Information:
Students have to visit multiple sites to get their COVID-19 information. When they reach the information, they aren’t able to fully comprehend the information given.
HMW help students streamline their information gathering and improve comprehension?
Experience:
Students are not educated about what to expect and how to safely produce a sample when they arrive at the testing center.
HMW educate test takes prior to and during the test to set expectations and optimize the process?
Students lack the motivation to get continuously tested.
HMW incentivize Penn students to comply with the university’s testing frequency requirements?
Information:
Students have to visit multiple sites to get their COVID-19 information. When they reach the information, they aren’t able to fully comprehend the information given.
HMW help students streamline their information gathering and improve comprehension?
Experience:
Students are not educated about what to expect and how to safely produce a sample when they arrive at the testing center.
HMW educate test takes prior to and during the test to set expectations and optimize the process?
Brainstorming
We brainstormed solutions individually and put each idea on a post-it note. Then, we clustered them into each of the HMW area
Evaluation
With all of our proposed solutions, we used two 2x2 matrixes to evaluate priority and risks involved with each idea.
04 Solutions
With our clients, we narrowed down to 6 solutions in the 3 focus areas.
Because of time constraint, we were only able to fully prototype and test out two of the six solutions, please see the Other Solutions to see prototypes of the other four.
Six Proposed Solutions
Tested Solutions
COVID-19 Website:
Redesigning Penn’s COVID website so that it can become the one-stop-shop where students can find all COVID information needed.
Redesigning Penn’s COVID website so that it can become the one-stop-shop where students can find all COVID information needed.
Aromatherapy:
Create a mechanism to help students produce saliva at a faster rate.
Create a mechanism to help students produce saliva at a faster rate.
unTested Solutions
Incentives:
Leverage school traditions as a motivation(e.g. in-person graduation, Hay Day, sponsored BBQ events)
Leverage school traditions as a motivation(e.g. in-person graduation, Hay Day, sponsored BBQ events)
Texting:
Create text reminders for the saliva test so students remember to follow specific rules before their tests.
Create text reminders for the saliva test so students remember to follow specific rules before their tests.
Infographics:
Convey positivity and contact tracing information to the infrequent testers in a different format.
Convey positivity and contact tracing information to the infrequent testers in a different format.
Signage:
At the testing site, display signs that can guide the student prior to and during their tests.
At the testing site, display signs that can guide the student prior to and during their tests.
05 Prototype & Testing
We fully prototype and test out two of the six solutions, a website redesign and using aromatherapy to speed up saliva production. However, we also provided preliminary designs for the other solutions.
Solution 1: COVID-19 website redesign
Tools
interview
a/b testing
usability testing (before and after)
figma
excel
a/b testing
usability testing (before and after)
figma
excel
Background
When we were researching how students were gathering their COVID related information, we found that 47.9% of the students listed the university’s COVID website as their top 3 ways to get information. However, this percentage drops by half when students were asked to rate their top 3 most informative sources.
This showed us that students were using the university’s COVID website, but they just had a hard time finding the exact pieces of information they needed. As the fall semester progressed, Penn had been positioning this website to be the central repository of all COVID-19 information on campus.
This showed us that students were using the university’s COVID website, but they just had a hard time finding the exact pieces of information they needed. As the fall semester progressed, Penn had been positioning this website to be the central repository of all COVID-19 information on campus.
We found out the top four reasons why students were visiting the website and learned that students would also like to know about the testing locations and hours.
Current Status Quo
To evaluate how user friendly Penn's original COVID website was, we observed each other and Penn students use it to complete various tasks (to schedule a test, find testing site info, Spring semester information, student campus contract, etc). Many students were unable to locate the information or complete the task we gave them.
From the observations, we identified 6 design issues that caused the inefficiency and confusion:
1. Multiple navigation menus
2. Buried links
3. Key info presented as paragraphs
4. Large blocks of texts
5. Scheduling link buried in text
6. Long scroll with no access to navigation menu
2. Buried links
3. Key info presented as paragraphs
4. Large blocks of texts
5. Scheduling link buried in text
6. Long scroll with no access to navigation menu
Prototype
We then created a redesign of this website using Figma to address the needs we found. Some of the key changes we made were:
1. Buttons for easy access to the most useful links
2. Recent announcements and updates as notification banner
3. Consolidate navigation bars into one that sticks on top
4. Add “New” tags for the newest updates
5. Show the 3 most important stats above the fold
6. Create a testing site directory and campus map
2. Recent announcements and updates as notification banner
3. Consolidate navigation bars into one that sticks on top
4. Add “New” tags for the newest updates
5. Show the 3 most important stats above the fold
6. Create a testing site directory and campus map
Testing and results
To test our Figma prototype, we first ran usability tests with students. We used the university’s COVID website as control and our Figma prototype as the experiment and asked students to perform tasks such as “schedule a covid test” using the website. We recorded the amount of time it took them to perform each task. We ran 17 usability tests in total and noticed that the original site was more frustrating and time consuming for students to navigate. A lot of students simply gave up when they couldn’t find the link to perform the tasks. The two bar graphs below show the incomplete task rate on original site and the amount of time for students to complete each task using either site.
In general, 70% of the participants agreed that the redesigned site is easier to use. There was a 99.2% success rate for participants to complete tasks using the prototype. There was a 61% decrease in the amount of time it took participants to complete tasks.
Since we worked very closely with our clients, some of the changes we proposed were already implemented into Penn COVID-19 website by the time we officially handed off the project.
Since we worked very closely with our clients, some of the changes we proposed were already implemented into Penn COVID-19 website by the time we officially handed off the project.
Solution 2: Aromatherapy at the testing sites
Tools
pretotype
benchmarking
pilot studies (controlled experiment)
field observations
on site randomized interview
benchmarking
pilot studies (controlled experiment)
field observations
on site randomized interview
Background
After we decided that salivation time reduction would be the main challenge we want to tackle, we started looking into how people salivate and what are being used to help with this process. A lot of our interviewees suggested us leverage human senses and a few of them mentioned the cookie scent in DisneyWorld.
We wanted to try and see if aromatherapy would work. We were put in touch with Dr. Dani Reed from the Monell Chemical Senses Center because she has 20 years of experience in the field of taste and smell and had collected 10,000 saliva samples. Dani suggested testing lemon as a sour odorant and peach as a sweet odorant for our prototypes and sent us a peach odorant sample for testing.
We wanted to try and see if aromatherapy would work. We were put in touch with Dr. Dani Reed from the Monell Chemical Senses Center because she has 20 years of experience in the field of taste and smell and had collected 10,000 saliva samples. Dani suggested testing lemon as a sour odorant and peach as a sweet odorant for our prototypes and sent us a peach odorant sample for testing.
Pretotype & Prototype
Along with our clients, we came up with the following design criteria to help us achieve the final solution.
Thus, we tested various scent carrying materials and scents to determine the combination that fits our criteria the best.
Thus, we tested various scent carrying materials and scents to determine the combination that fits our criteria the best.
The materials tested were mostly craft supplies that were easily accessible: cotton balls, cotton swabs, watercolor paper, sketchbook paper, posterboards, cardboard, and printing paper.
The scents tested were lemon extract oil and peach extract oil as they were suggested by Dr. Dani.
The scents tested were lemon extract oil and peach extract oil as they were suggested by Dr. Dani.
Posterboard was determined to be the material that fits all of our requirements. We also chose lemon instead of peach scent since the lemon scent promoted more saliva production in our team members. This was in line with a study that the Monell Center sent us.
Testing and results
We ran controlled experiment at the Houston Hall saliva test pilots for two days. The only variable between the two days was the use of aromatherapy strips.
On both days, we recorded time each test takers spent to supply saliva sample and intercepted test takers as they existed the space for an interview.
On both days, we recorded time each test takers spent to supply saliva sample and intercepted test takers as they existed the space for an interview.
On the control day, we timed 32 test takers and interviewed 20 students as they left the site. The average time students took to administer the saliva sample was 4:10 and the median time was 4:08.
On the experiment day, we conducted 38 interviews and observed 55 people.
There was an increase in first time test takers and a few outliers who took longer than 11 minutes, which may have resulted in longer than expected average times of 4:05.
There was a significant reduction in the time students took to produce a saliva sample while using the lemon scent strips as the median time was reduced by almost 21% to 3:16.
Overall, 70% of students who used the lemon scented strips said that it helped them generate saliva and the perceived difficulty and duration of producing a sample decreased after scented strips were used.
On the experiment day, we conducted 38 interviews and observed 55 people.
There was an increase in first time test takers and a few outliers who took longer than 11 minutes, which may have resulted in longer than expected average times of 4:05.
There was a significant reduction in the time students took to produce a saliva sample while using the lemon scent strips as the median time was reduced by almost 21% to 3:16.
Overall, 70% of students who used the lemon scented strips said that it helped them generate saliva and the perceived difficulty and duration of producing a sample decreased after scented strips were used.
The exit interviews provided us more insights than the quantitative data:
“I liked the lemon scent because it made me feel that the room is clean”
“I used to think about sour things but these strips are better and easier”
“It seemed to go MUCH faster. I loved it!"
"I'd love to have it as an option moving forward!”
“I really liked the scent strip from my last test, do you have any more of them?”
We also learned from staffs that incorporating the scented strips had no disruption to their workflow and some staffs were already advocates for this added aid.
Other solutions
Besides redesigning the covid website and implementing aromatherapy at testing sites, we worked on 3 other solutions addressing the information-gathering problems. These would work alongside the new website to provide students needed information. We encountered testing constraints but hope that prototypes are still helpful to our client.
Texting
Infographics
On-site signage
06 Impact & Learning
Our extensive research helped lay the foundation for two successful prototypes, both of which are on their path to implementation for Spring 2021 to help keep Penn's campus and the city safer during the COVID-19 pandemic.
It was a very rewarding and eyeopening opportunity to have worked on this project. I learned how to work in a rapidly changing environment and context (the pandemic). I experienced and learned the importance of being agile, flexible and empathic while being firm with the bottomline and deadline. This project involved many stakeholders who had different perspectives and conflicting needs and wants and I learned how to juggle all the factors and ultimately voice for the people who would be impacted by our solution the most. It also showed me that the design need to be grounded in solid user research and constant communication. I was also very fortunate to have supportive and driven teammates who also believed in the human-centered design process and collaborated with me on making all the timeline for the project.
