Navigating Pizza Perfection: A Papa Johns Usability Journey

About this project

In this student project, I worked with a group to understand how users interact with the Papa Johns site and discover pain points when performing common tasks like customizing orders and contacting customer service.

Project Duration: May-June 2023

UX Discipline: Research, Usability

Methods & Deliverables: Group collaboration, screener development, task creation, moderated usability testing, affinity diagramming

Context

Ordering takeout online has become ubiquitous. 60% of Americans order takeout or delivery at least once a week, and with Papa Johns as one of the nation’s top grossing pizza chains, it’s crucial that their site be intuitive and enjoyable in order to maintain a competitive edge.

Screener & Tasks

Initial Business Concerns

Initially, our group of four researchers were tasked with recruiting Advanced users. We determined that these users would be both technologically adept and very familiar with ordering pizza online. We then selected the following business concerns to address:

1. If a user does a Google search for Papa John’s and clicks to go their home page, it might be unclear what they need to do to order a pizza.
2. When they finally do get to the order page, the choices there may seem overwhelming to users.
3. Should we challenge to users to sign in or create an account before going to checkout is going to deter users from continuing? Do users actually see the option to order as a guest? How can we strike the right balance of users creating accounts?
4. How easy it is for customers to order combo pizzas (e.g., half pepperoni, half sausage)?

Screener

Working together, we brainstormed what characteristics our users would need to screen for to meet the Advanced user criteria: someone who is over 18, has above average computer skills, and regularly orders food online via desktop, at least once a week.

These questions were created with the flexibility of recruiting over the phone or online using multiple choice, multiselect, and one short answer. The goal of each task is listed below to explain what goals it addresses.

1. How old are you?
   1. Under 18
   2. 18-22
   3. 23-30
   4. 31-40
   5. 41-50
   6. 51-60
   7. 61-70
   8. 71-80
   9. 80+

Ensures a user is the appropriate age. More open-ended phrasing aims to avoid people saying an inaccurate age to enter the study.

2. What kinds of tasks do you perform online in a typical week? (Check all that apply)
   1. Playing games online
   2. Online shopping
   3. Checking social media
   4. Sending emails
   5. Reading articles
   6. Watching media or streaming (e.g., YouTube, Netflix)
   7. Other

Aims to understand a user’s computer skills. If asked over the phone, user can start listing tasks off instead of having the options read to them. If more than three options are checked, they can be considered advanced.

3. About how often do you or your family cook meals at home versus buying food from a restaurant each week (could be in person or delivery)?
   1. Rarely order out (<= 1x week)
   2. Sometimes order out (2-3x week)
   3. Often order out (4-6x week)
   4. Constantly order out (>7x week)

Serves as a warm-up. We don’t want people to assume that this screener is about eating out from the very beginning, so trying not to bias the answers by adding in “cooking meals at home.” This wouldn’t disqualify the user until we understand their ordering habits.

4. When you do order out, how do you typically order food?
   1. In person
   2. Over the phone
   3. Online
   4. Using a mobile app

Looking for users who frequently order online, although any additional information can’t hurt. Those who don’t order online at all would be disqualified.

5. When was the last time you ordered food online, either for yourself or your family, or for someone else like an event?
   1. Within the past week
   2. Within the past month
   3. Over a month ago
   4. Never

Looking for users that say, “within the past week.” Coupled with #3, this will tell us who regularly orders food online to meet our Advanced user criteria.

6. Have you ever ordered pizza online?
   1. Yes
   2. No

We’re looking for online pizza ordering regulars, so those who say “No” are disqualified.

7. How often do you order pizza online?
   1. More than once a week
   2. Once every 2-3 weeks
   3. Once a month
   4. Once every few months
   5. Rarely

This will narrow our search down to the die-hard pizza fans

8. Where do you typically order pizza online from? (Check all that apply)
   1. Papa Johns
   2. Dominos
   3. Donatos
   4. Pizza Hut
   5. Little Caesar’s
   6. Other national franchise
   7. Local stores/franchises
   8. Other

It isn’t required that they order from Papa Johns specifically but will help to know how familiar they are with our products.

9. When placing online food orders, do you prefer to create an account and sign in or check out as a guest?
   1. Create an account & sign in
   2. Check out as guest
   3. Other
10. To the above question, why or why not create an account?

This short answer will help us understand what benefit users see with an account and can help define user types for testing, those who prefer account creation and those that don’t.

Tasks

With our target users in mind, we developed the following tasks to test site usability against the business concerns.

1. You have several friends over and you’re getting hungry for Papa Johns pizza. Open your browser and perform a search to get you there and start an online order.

question to answer: Do people know what to do once they’re on the homepage?

2. You and your friends want to order a pizza but are reaching sharp discord about whether pineapple goes on pizza. To please everyone, you need to order a pizza with ham, half with pineapple and half with another topping.


question: Can people easily create a combo pizza?

3. Now, to please yourself, create a large pizza you personally would like.

question: Are the menu choices too overwhelming for the user?

4. Finally, order a meat lovers pizza. Since you don’t want the onions that come with it, you need to remove that topping off your pizza and instead, add some jalapeños.

question: Are the customization choices too overwhelming for the user?

5. With the pizzas in your cart, go check out. You don’t actually have to finish checking out but enter information as if you are. How did you find that experience?

question: Can users successfully place an order as a guest with minimal frustration?

6. You like to order from Papa Johns on a regular basis, but you don’t want to have to recreate your usual every time. Can you find a way to save an order?

question: Can users understand that they would need to create an account for this job?

7. Go ahead and create a new account. For this exercise, feel free to use this test email and phone number (supply test data). How did you find this process?

question: Is the account creation process streamlined enough that users would be able to do this quickly?

8. Still with the pizzas in your cart, try the checkout process again. How would you compare this to checking out as a guest? Which do you prefer?

question: What do individual users prefer? Do they see an advantage to checking out with an account or not?

9. You’re planning a big event and need 10 pizzas. How would you go about doing this?

question: Would users expect to place a large order in the same ordering process or would they think to look elsewhere?

10. Look at the ordering page. If you were to reorder this in a way that made sense to you, how would you do it?

question: does the user’s conceptual model differ from the way we’ve organized our page, and is there a way to make ordering less overwhelming?

Usability Testing

Project Restructure

After submitting the Screener and Tasks for Advanced users, the business (course) decided to take this usability test in a new direction.

Rather than focusing on Advanced users, we were tasked with recruiting Average users over the age of 50 who had never ordered online before. Additionally, the third business concern that would challenge users to sign in was abandoned for the time being and replaced with two others: can users find a way to sign up for email coupons without creating an account, and can users find a way to contact support in the case of an issue with an order.

With these changes in mind, the final task list was as follows:

Ice breaker: Search Google for pizza in your area.

• Are you able to find what you need?
• Is there a pizza place close by?
• Is it the type of pizza you normally like?

Task 1: There are a bunch of people at this party and you need to order 3 pizzas. They are:

1. Mushroom and pepperoni.
2. Half onion and half sausage with light sauce.
3. A specialty pizza, because you want some variety

This task would help assess the ordering process, specifically, how easy is it to navigate the menu and customize pizzas?

Task 2: You want to sign up for deals and coupons but you don’t want to register. You just want to give them your email. Can you do this and what do you think you will receive by email?

This task would help assess signing up for offers, seeing if people can find what they need through the information available on the site.

Task 3: Your pizza arrives and it is terrible! Your driver was rude and you are really upset about what just happened. You call the local store but get nowhere. You need to contact the corporate office. How would you do that?

This task would help assess seeking customer service and if participants can easily and readily request help

Method

Four individual moderators each conducted usability testing on one participant. Three moderators were interviewed in person and one remotely, each session lasting, on average, 20 minutes. All sessions were recorded with participant consent and reviewed by the team for analysis.

The four participants selected had never ordered pizza online before and were over the age of 50. P1, P2, and P4 were women, and P3 was a man. Finally, all participants had average computer skills and seemed confident navigating the search engine in their ice breaker task.

Synthesis

After all members of the team shared their recordings, I reviewed all four sessions and took notes using an Excel spreadsheet as my affinity diagram. From these notes, I determined task success rates and found patterns in user expectations and behavior. 

Results

High Level Results

Participant success and satisfaction varied widely across each task.

Follow-up Research Questions

This study highlighted several pain points worth exploring in subsequent research. Below are some questions that emerged:

Ordering process:

• Set up a task to test error recovery and see how participants approach it to determine more intuitive recovery measure.
• Try to understand the various pizza deals and if participants feel as confident ordering with them as through the regular customization process.
• Survey customers on popup content and feelings around them, and whether they find popups helpful or simply intrusive.

Coupon signup:

• Survey on email-only coupons: is signing up for coupons without creating an account desirable?
• What are participants expecting in the footer when it comes to email signup? Text fields or links with specific content?
• What coupon information is helpful to include alongside loyalty program information?

Customer service:

• Learn what kind of contact methods people prefer when they’ve experienced an incident.
• Redesign the contact form to be more streamlined and approachable.

Final Report

The final report with detailed synthesis can be found below, in the link and the document viewer.