James SpillaneUsability testing, as a part of a web or mobile development process, involves bringing in actual or representative users to try realistic tasks. The goal of usability testing is to make sure that crucial tasks can be completed with effectiveness and efficiency and that users feel good about their experiences.
In many cases, user research is not a project unto itself. Rather, it is part of a larger web-centric project effort. These larger projects often have a project manager. This person has the task of coordinating the usability effort into the broader project schedule. The project manager generally has some kind of technology management background, but is often not someone who has extensive experience with usability.
This lack of knowledge about usability isn’t inherently a problem, for this is the reason why my company and I are pulled into the project in the first place. However, based on my experiences and lessons learned, here is a “cheat sheet” of things the project manager needs to be thinking about.
“I’ve got an incredible design team. I don’t need to have anyone evaluate it.”
I never send out any kind of formal document without having one of my staff review it. No matter how good a writer I think I am, I am 100% sure that I’ll have typos in that document that I won’t find no matter how much I try to look for them. I’ll miss them because it’s my baby. Similarly, a design team, no matter how good they are, will not be able to see issues with something they developed. A design team may also wish to do their own testing, where the designers also operate as usability test moderators.
Recommendation: Don’t assume that a design team, no matter how amazing they are, will be able to see flaws in their creation. Although someone who is an interface designer can also be a usability evaluator, I’d highly recommend that they don’t evaluate a creation that they had a large part in designing. The risk of asking biased questions is much higher, and the risk of not seeing flaws all that much greater. Ideally, someone who has had no role in the creation should be evaluating the site.
“Usability testing is too expensive.”
I sometimes hear the contention that the budget doesn’t allow for usability testing. Once figuring in not only labor hours but also other direct expenses, such as participant recruiting costs, participant incentives, and facility rental, costs may seem like they are really starting to add up.
Recommendation: There are a number of arguments to be made for the cost-justification of usability testing. Even as early as the mid-1990s there were books and articles published on this very topic (Classic revised in 2005: Cost-Justifying Usability). In addition, usability testing doesn’t have to be terribly expensive. First, calculate out what a typical usability study might cost, with an optimal number of representative users in an optimal location with optimal incentives. Look at the final number: is it really too high? If so, consider reducing the number of participants; even if not optimal, at least a fair bit of valuable information is likely to emerge from testing. Similarly, I like to produce full written reports, along with a PowerPoint summary report, but if budget (my allotted hours) doesn’t permit, I might just produce the PowerPoint summary report and make certain to at least convey the general ideas of the findings. In the more uncommon instance that the budget is larger than the base optimal projections, a usability researcher can always add what I’d consider “deluxe” options, like a highlights video from the sessions.
“You’ll tell my team to scrap the site and start from scratch!”
In addition, the stakeholder may believe that inherent in any usability test comes a risk that they will be told that they should be starting from scratch.
Recommendation: In my experience, with a good development team that has a reasonably good sense of design principles, the testing often uncovers some larger changes but usually uncovers a number of problems that can be fixed with minimal impact to the development schedule. Even if the stakeholder sees some larger-scale recommendations, assuming that there are no show-stoppers, these should be cataloged and saved for a future release. All the minor tweaks and low-hanging fruit, however, could be done immediately, and the stakeholder could see a large value-add for minimal cost.
“But we really can’t afford testing.”
Sometimes the budget is really tight, and the client feels that they absolutely can’t do usability testing.
Recommendation: Don’t worry. Maybe it won’t be optimal, but at a minimum, consider a quick expert review. I’ve done reviews for clients with results given as quick bullet points in an email, which helps save time and thus cut cost. Also, consider a usability test with only a small number of users that is observed by the client. Instead of spending time writing a report or even a summary, consider a quick debrief meeting a day or two later to talk about trends and observations. Again, not optimal, but at least there is some evaluation time squeezed in.
“Our schedule is too tight for usability testing.”
Often, time tables are tight and stake holders may feel rushed. When stakeholders hear that a usability study may run over a period of three days, and that the report will take a few more days, they may get the impression that the usability evaluator expects all work to stop during this period. Stakeholders will express the concern to the project manager that there simply isn’t enough time for usability testing.
Recommendation: Let the stakeholder know that not only do you know that time is tight, but that the usability practitioner does as well. Work does not have to stop for the usability test to take place. Further, if the development environment is flexible enough, and ideally if there is perhaps an extra day or two between sets of participants, there may be enough time not only to make changes to a prototype, but also to test out some of the changes with the next round of participants or to adjust the test plan and drill down and zero in on key problems very rapidly.
“We’re already doing software testing – we’ve got it covered!”
Sometimes project managers are aware that there is this testing approach called “usability testing,” but they figure it’s not much different than the software/functional testing that they are already doing. They may have spreadsheets of actions that a user is supposed to be able to do, and they need to make sure that all of the design specifications are met. While this software testing is certainly important, project managers may not realize that usability testing is focused on the success of actual users using the website, instead of just verifying that the functionality exists and is working as initially intended. What if those initial intentions and assumptions were incorrect for the expected audience group?
Recommendation: Make sure to consider the needs of the actual users of the site, not just the specifications that have been handed down to you by the stakeholders.
“Usability testing will increase my project scope dramatically.”
You have a looming deadline and even if you do have time to squeeze in a usability test, you’re afraid that the results are going to be less than perfect. You’re envisioning at least one whopper of an issue and don’t want to have to delay the deadline in order to get that issue resolved.
Recommendation: You don’t necessarily need to resolve that whopper issue before launch. The deadline might be hard and fast, and that’s okay. It’s better to at least know going in what the major issues are. But more importantly, you’re likely to find a number of small tweaks that can be made quickly and won’t increase project scope. It’s okay to start your list of fixes for the next release, even before this one is out!
“I’ve done testing for my desktop web projects, but do I need to do testing for similar mobile content/apps as well?”
You have done a good job integrating testing into websites in the past. But now, you are doing a mobile project. Maybe it’s your first. Maybe you’re “converting” your desktop site to a mobile site or maybe you’re porting functionality to a new mobile app. Since things worked smoothly on the desktop version, do you need to concern yourself with mobile version testing too?
Recommendation: Absolutely. Usability is very much framed by context and audience. The context of desktop use is likely to be different than the mobile context. Users will be apt to have different situations when they use the mobile version than the desktop version. Also, the users are apt to be different, and only a subset of your desktop users are likely to be using mobile. It’s important to categorically understand these differences, which will inform both who you will be recruiting for your test and also what scenarios you might be testing with.
“I really do want to do usability testing, but the stakeholder is only permitting me one shot at it.”
A stakeholder may concede that usability testing is important but say that only one round of testing will be supported. At what point should the testing be done? At the beginning of development to get things off on the right track? Late in the development cycle? Right before launch to confirm things work?
Recommendation: If you only have one shot at testing, the best time to do testing is at the latest point you can where changes to the website would not be very problematic. At a minimum, I’d suggest that there be click-through HTML prototypes, but these do not necessarily need to have the actual polished look of the finished product. That said, if you have the okay to test with enough (perhaps 10 or 12) representative users, consider spreading these users out, maybe half at the HTML prototype stage and half once everything is more developed. Perhaps you could even divide your users over three points.
“Does Search Engine Optimization (SEO) matter to usability?
I’ve heard rumors that SEO is related to usability, but I think of SEO as a way to sell more products.
Recommendation: At its core, SEO is about getting found, and no product can be usable without being able to be found. Optimizing a page for search so the right users find the page at the right time can be critical. Also, keep in mind that users who already know a product or information is on your site will often type in your site (or brand) name into an engine like Google or Bing along with the associated keywords. So make sure that what they want to see actually comes up! When doing usability testing, consider including a search engine component. If there is already something that can be searched for (an old site), ask them to search for it. See what search terms they use. And don’t forget that internal SEO matters too. While some people will navigate through your site, others will consistently use internal site search. If appropriate, incorporate some tasks related to using the internal search engine into a usability study.
Usability vs. the three constraints of project management
The project management triangle has three constraints: time, cost and scope. It is important for the project manager to understand how usability relates to these three constraints.
Time: On the one hand, introducing usability evaluation would seem to introduce more time into the effort. On the other hand, just like spending time defining good requirements can improve the ability to stay on track later on, so too can usability testing make sure that things stay on schedule by reducing back tracking.
Cost: Any cost for usability evaluation would seem to make the overall project cost go up. On the other hand, though, like time, the cost is likely justified by the prevention of the need to spend labor hours later on fixing what turns out to otherwise not be usable.
Scope: Usability should be part of an initial project scope, and it would be wise for project managers to assume that usability testing will find some things that need to be changed. If the changes required are too significant, and the project deadlines can’t be met if certain recommendations are taken, then it would be better to place these (theoretically significant) recommendations into a future release than to avoid doing the usability testing at all.
Usability and Project Sponsor Buyoff
Project sponsor buyoff is critical for any project. How does usability fit into this important consideration?
Recommendation: Considering usability from the outset and including this into any project will hopefully show a sponsor that you are considering your end users and value project quality. So including it may be an asset from the start. If a project sponsor questions the usability evaluation efforts, be prepared to explain the decision and defend this approach, perhaps by explaining advantages to time, cost and scope. Try to help the project sponsor to understand that including a usability component doesn’t indicate a lack of trust in the team, but rather is an appropriate step for any project.
Adapted from Cory’s DC Usability Blog.
Read more:
This article was written for Business 2 Community by James Spillane.
Learn how to publish your content on B2C
Join our Telegram channel to stay up to date on breaking news coverage
Add B2C to your Google News Feed
Great tip about doing search engine usability tests to see what your users are actually searching for.
I think the key to get stakeholder approval is doing a few usability tests up front. Once they see real people interacting with their site and struggling, they’ll want to invest more resources in testing and fixing it. (And hopefully testing again..)