Saxony Partners Design Method of the Week: Usability Testing

In today’s first ever Saxony Partners Design Method of the Week, we cover Usability Testing. This is critical to projects and companies that have mature user experience practices. These mature practices also fall higher on the Design Strategy Altitude Model, which we will be covering in a later week.

What is it?

Usability testing is a methodology that facilitates not only validation of the quality of an interface or physical product, but also the discovery of usability issues. A usability test is when users perform a set of tasks and the performance of the system is recorded. Conducting usability testing and incorporating feedback into the product continuously throughout discovery, design, and development is the ideal way to build high quality products. These products, along with good marketing strategy, will have fewer usability issues and have a much higher chance of beating competitors.

How Should a Test Occur?

Usability Testing is conducted remotely via the web, in users’ environments (e.g., cubicles, vehicles, job sites), or in a usability testing lab that utilizes recording technology and two way mirrors. Additionally, there are important roles that are ideally filled by different people to better facilitate good testing:

  • Recruiter – This person is tasked with finding users that use or may likely use the system assuming good marketing and few usability issues.
  • User – This is a person that the recruiter, if they have done their job correctly, is an actual user or potential user of the product. If they are discovered to be neither, they should still be compensated for their time and dismissed from testing.
  • Facilitator – The facilitator should be a senior usability tester who is experienced in using a script and aware of when it is acceptable to deviate from the script to ask follow-up questions as issues arise when the user uses the interface.
  • Note Taker – This person records any note worthy occurrences such as questions the users ask or challenges they have during tasks.
  • Observers – These should ideally be product owners, developers, and designers who might otherwise discount the usability testing results as inconclusive had they not been present. If these stakeholders are not available they should be given access to videos, quotes, or good summaries of what happened while they were not present.
  • Presenter – This person should be good at public speaking and is often the facilitator of the test. They should push to present the results to decision makers for the product. They’ll be able to summarizes the results, go into great detail of what the results mean, suggest solutions, and be prepared to answer questions.

The outcome of usability testing is:

  • System Usability Scale Scores are given by users and will show how the product rates when compared against thousands of other products that were evaluated in the same way.
  • Usability Issues are discovered by observing the users as they interact with the system. They will help inform what should be focused on when designing solutions. The following are ways of identifying which tasks had issues:
    • Time on Task
    • Difficulty

There are some potential issues that can occur with usability testing

– Is the right person presenting? Addressing reluctance of decision makers to accept conclusions from testing is a key aspect of Usability Testing. The person presenting should be ready to answer questions and provide valid support for their conclusions.

– Sometimes technology breaks during a test. Perhaps a recording device fails or the application fails. This is an opportunity to have back up recording devices and things such as paper prototypes or wireframes so that the test can still be useful.

– Often you won’t understand why a user did a certain action. Having users think out loud and reminding them to do so during a test will help identify where the human computer interaction failed. This will help produce better solutions by focusing on these failed interactions.

Additional Information

For more details, see our page devoted to this method: Usability Testing