4 Lessons Learned From An Impromptu Usability Test

Working at a rapid pace revealed some best practices that I’ll abide by on future tests.

 

Photograph by Vlada Karpovich via Pexels

 

For a recent design project (link coming soon!), I wanted to run usability testing before moving the prototype from mid-fi to high-fi. Initially, I thought I’d have a week to write the script, recruit five participants, run the tests, and organize the data. This timeline felt tight but doable.

However, due to a shift in the client’s timeline, I only had four days. Cramming seven days’ of work into four days meant that working quickly wouldn’t be enough — I had to work rapidly. (The alternative was that we move forward without any testing, which I was unwilling to do!)

I buckled down and swiftly implemented a usability test. The sessions were ultimately fruitful, and I’m glad that I pushed myself to get it done. Unsurprisingly, in my rush to get things done, I made mistakes along the way. While these mistakes didn’t derail the sessions, they gave me some insights to carry into future usability tests.

 
  1. Do a tech rehearsal beforehand.

This was a remote test, so the participant would open the Figma link, share their screen, and interact with the prototype to complete certain tasks. My first session included some avoidable tech hiccups:

  • The prototype link didn’t work. When I sent the link, the participant landed on a page that said, “This prototype does not exist.” I was thrown off by the error message. The prototype did exist — it was right in front of me! It took me almost five minutes of troubleshooting to realize that I hadn’t updated the link sharing permissions. When I set the link to “Anyone can view,“ she was able to open the prototype right away.

  • I forgot to give the participant screen-sharing permission in Zoom. This took me a few seconds to fix, but it broke the flow of the session. Next time, I’ll update the screen-sharing permissions before the participant enters the Zoom room.

  • The prototype view setting created a snag. The prototype automatically opened to the 100% view setting. The participant could see the entire prototype screen, but the window size made it impossible to interact with the lower part of the screen. We had to pause in the middle of a task so I could explain how to adjust the Figma window to “Fit to screen.” This issue was resolved in about 30 seconds, but it was yet another interruption that took us out of the flow of the test.

 

2. Recruit more participants than you need.

It’s a best practice to run a usability test on five participants. On day one of my testing timeline, I recruited five people and scheduled sessions over the next few days. Everything went to plan until the final test. Unfortunately, the fifth participant got sick and had to cancel at the last minute.

At this point, I was at the end of day three. By the end of day four, I needed to transcribe the sessions, organize the data, and pinpoint key insights. It wasn’t feasible to find another participant, run a test, and process their data in time. So, for this test, I made due with four participants instead of five.

Life is unpredictable and everyone has busy schedules, so it’s understandable if someone needs to drop out! Next time, I’ll recruit more participants than I need. If everyone shows up, I’ll have even more data than needed. If someone has to drop out, I’ll still have the desired number of sessions.

 

3. Write a lean script.

I wanted to run each session in 30 minutes. This timeframe felt fairly manageable from the participant’s perspective and I’d be able to get plenty of data in that time. Thus, I had thirty minutes to do the following:

  • Greet the participant

  • Introduce the project

  • Facilitate four tasks

  • Pose follow up questions during/after each task

This is a lot to accomplish in thirty minutes! Three sessions fit into this timeframe, but just barely. The fourth session went over time. Fortunately, the participant was able to stay longer to finish up the final task.

After reflecting on the sessions, I realized the one of the tasks could have been omitted. It was the least urgent section of the prototype. While I received some good feedback, the task wasn’t essential. In the future, I’ll use more strict criteria for writing the script. With a leaner script, I can keep the session focused on the most urgent questions. With fewer tasks to facilitate, I can leave more time for follow up questions. Plus, fewer tasks is a lighter lift for the participant.

 

4. Stagger the sessions.

Due to other conflicts in my schedule and the participants’ schedules, I had to run three sessions fairly close together on the same day. When I was recruiting participants, I was so focused on scheduling sessions that fell within my timeline that I didn’t think to incorporate breaks for myself.

I had to listen back to each recording so I could transcribe the sessions. When I listened back to the third session, I noticed a dip in my facilitation. While I guided them through the tasks without incident, I missed some opportunities to ask follow up questions. I noticed moments when the participant made an interesting statement that I just accepted without further questioning.

Facilitation fatigue crept in because I hadn’t staggered the sessions properly. As a former teacher, I’m well aware that facilitating is demanding work. When possible, I’ll make sure there are breaks between sessions, so I can approach each one with the same level of awareness and attention.

Previous
Previous

Canceling A Membership Should Not Be A Long And Winding Road

Next
Next

“Get Organized With The Home Edit” Cleverly Customizes Design Patterns To Meet Client Needs