Clean up the UI before the usability study

Sometimes you need to make changes before running a usability study, instead of waiting for the results. If you make the changes, you won’t have to watch all the participants struggle with the same obvious problems. This lets the study focus on deeper issues.

These problems may in the user interface or the underlying business rules. If it’s possible, work with engineering and product management to fix the problems beforehand.

For example, a few years ago, we tested an online document database. The initial search UI opened too many windows, and we expected to hear about that from all of the study participants. The development team created a parallel version of the product that fixed this problem.

This change, along with some others, allowed participants to focus on searching for information instead of on how the system worked. It was a successful strategy: A participant who had used the system before the study explicitly said that he liked the single-window model better. We were able to find a lot of other problems instead of hearing everyone talk about all the windows.

Question What have you done to make usability studies run more smoothly?


3 Responses to Clean up the UI before the usability study

  1. Mary says:

    The interesting question is how do you decide what needs to be cleaned up before the study? On what basis do you decide how many windows are too many?

    I did a study last winter in which I performed a heuristic evaluation, the site was cleaned up per the results, and then I ran the tests. It worked well.

    • Hal Shubin says:

      I guess it’s based on experience, although that’s not a guarantee.

      The folks who produced that software thought they had a good model. When they explained it, it did make sense. But it was just too complicated because it required manipulating too many windows, more than anyone would have used for searching in any other application. As we know, people don’t want to learn a special way of working for each application they have to use.

      We made a number of recommendations after the study, and they implemented quite a few of them. One way they knew they’d improved the app was that they got a lot fewer complaints. (People complain more than they praise, apparently.)

      How did you know what things should change before the study you mentioned?

  2. Karl Puder says:

    Another way that this comes up happened when I was at DEC. A hardware engineer appeared at my cube and asked for a few minutes of my time. He showed me a prototype box of a new device and asked me to interact with it (I have forgotten his exact words). I commented that one of the switches was rather small and difficult to manipulate. His response when we were done was that a lot of people had said that.

    The impression I got was that since this was an ad hoc HF study, once a trend appeared in the results they could just stop and go address that problem rather than waste any time (or their pool of virgin test assistants) on any particular count of evaluations.

Leave a reply. We'd love to hear from you.

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s