It is important to standardize user testing so that you get consistent, reliable results.
This chapter will take you deep into the world of moderated testing, which is arguably one of the most difficult testing methods. We have four expert moderators to help us understand the five-step process and all the variables. You will be in good hands!
A moderated usability testing
You will need to choose the right usability testing method for you at the beginning of your usability test process.
> Research goals (what you are trying to achieve)
> Your resources are the amount of time and money that you have available to invest.
> The audience that you wish to test
Moderated usability testing is one of the best methods to gain useful insight. A trained moderator observes and interacts directly with participants and provides valuable feedback. This method is also one of the most difficult because of all the variables involved, from the location chosen to the moderator’s ability to obtain valuable answers from participants.
Our ability testing is a 5-step process
The following points apply if you wish to test a prototype or non-transactional website.
Step 1: Plan the session
The most important part of the whole process is planning the details for the usability testing session. How you proceed with the testing process and what you get will be determined by the decisions you make.
Find out the nature of your study
Definition of Problems/areas you want to concentrate on: What is the purpose? Which areas of your website could benefit most from usability testing
You want to test the type of users: These are typically representative of your users. However, you might want to drill down on a segment (e.g. users who have made a purchase within the last 30 days).
Questions that you want to Ask: What are the questions you want to be asked by users about your website’s content? What are you trying to find? (Note: We go into detail in the usability test questions chapter.
Logistical details about your usability testing sessions
> Location — Will you test in your office? In a research laboratory? Online?
> Timetable When will the testing sessions be held? This is especially important if you invite participants to a lab for research.
> Moderators: Who will conduct the testing sessions? Moderating user testing without influencing results takes skill and practice. You should either hire trained moderators or arrange training for yourself.
> Recording setup — recording testing sessions allows you to look back on them later and capture all data that the moderator may have missed or did not have the time to record. You will need to familiarize yourself with the equipment and how it is installed before you can take advantage of audio or video recording. You want to capture the screen of each participant, their speech, and their body language in an ideal setting. This can all be done easily in a test lab.
All this information can be gathered in one place. Bonus points for creating a template that you can reuse multiple times. This will serve as your guide to the next steps, such as recruiting participants and designing the session.
Step 2: recruiting participants
How and who you hire will depend on what your testing goals are (e.g. how much information and how long you need your sessions to achieve them) and your budgetary limitations.
These are the most popular methods to find people for your study.
> Hiring an agency If you are looking for a specific group of people (such as web-savvy oncologists or single mothers under 35), hiring a specialized agency is the best way to locate them. These agencies have a lot of resources and are able to find the best candidates quickly.
> Make your website. If you have a user base, then recruit them.
Note: You must first take step 1 and plan how you’ll run the test. An on-page poll may help you connect with volunteers around the globe, but only a small fraction of them will be available if you’re testing at a lab.
l Use social networking: If you have a following on social media, make use of your channels to reach potential participants.
l You can recruit your clients by reaching out to clients/customers and asking if they are willing to help (provided that they have given consent to being contacted for these initiatives). It’s not a good idea to spam your clients unnecessarily.
You must compensate your participants for their efforts, regardless of how you recruit them.
Step 3: Design the task(s)
The task design and recruiting participants are both done simultaneously. After you have figured out the why and the how of your research and waited for confirmation from participants, it is time to design the test.
This means that you will carefully plan the scenarios and tasks you’ll be guiding your participants through. You want clear and tangible results.
One of the tips to remember is the most important functions when designing scenarios. An e-commerce website’s top task is often to sell something. Therefore, you will want to create scenarios that guide the user through the purchase process.
Step 4: Run the session
You or your moderators must follow a protocol when conducting the usability testing session. While this protocol allows for some customization, it ensures a consistent experience for all participants.
Let’s say you run the session: Here is what you do.
Warm-up and introductions
If conducting an in-person test make sure that your participant is comfortable with the testing environment (e.g., the height of the desk, placement of the mouse, etc.). It is essential that your participant understands what will happen during the session. If you are doing it remotely, ensure they can hear you clearly.
You can record the session if you need to (e.g., to be able to look back on it later). If you’re conducting in-person testing you might even have a form printed that you can ask them to sign.
Ask your subject a few friendly questions to help them relax. For example, what distance they have traveled to get there, how many times they’ve used user testing, and so on. It is important for new interviewees to feel comfortable and relaxed in the testing environment. The first few minutes should be spent getting to know one another and building rapport. This will help you transition smoothly into the testing phase. The tester may not even realize you are changing gears as you begin to gather more information.
Collect pre-testing data
Use predetermined questions to gather demographic and psychographic information during your conversation. You might ask the following questions for your e-commerce test:
> When was the last time they ordered online
> How many times have they purchased something online in the last 6 months
> How they usually go about finding products they desire
> What influences their purchasing decisions
Transfer to the first task
To transition your participant to the first task, use the rapport you have built. There are usually three to four scenarios that you would like to try, but it depends on your participant's mood and skill level. It is a good idea to become a certified moderator. You must be able to recognize when your participants get frustrated. This could mean you need to change the task or switch them to something easier. Or you might find a highly skilled participant who completes the task in record time. This is where you will need to be adept at probing the reasons they did what they did.
Note-taking
Ideal situations have another person taking notes so that you can focus on your relationship with your participant.
Questions to follow up and wrap up
You can ask follow-up questions at the end of your session and gather the participant’s final feedback. Make sure to thank them.
Moderation: Dos and don’ts
It takes a lot of skill to manage a moderated section. This involves building a relationship with the subject and then naturally leading them through the tasks. The four veteran testers shared their top tips on being a moderator.
> Do use clear, neutral instructions. It is important to ensure that your question does not allow for the interpretation.
> Do not write down the tasks. Or at least don’t copy the task list verbatim. It can make the proceedings too formal. Personalizing the task and the wording you use will make the participant feel more comfortable.
> Watch for verbal cues, body language, and body language. Even though users may not explicitly state that they are confused, a skilled moderator will be able to tell from their actions. A user who used to be silent before saying “Hmm! “Or sighing in frustration.
> Do not speak too often. It is important not to interrupt the user’s thinking processes. Give them a task and ask them to talk about their thoughts and actions. Then, observe quietly how they do it.
> Keep an even tone. Do not disagree too much with the user; this could influence their final opinions. Be neutral in your body and speech.
> Do not take control of the task. The user must be fully in control from the moment the test begins. Do not let them navigate or use their mouse.
> What are the best ways to interrupt? You can deflect any questions or interrupts by using the boomerang, echo, and Columbo techniques, which are detailed in the test questions chapter.
> Do not look at their screens too often. Sometimes, too much attention to a user’s screen can lead to a change in their behavior.
Step 5: Analyzing the insights
Once you have collected all the data, it is time to analyze them and draw conclusions. This should be done as soon as possible following testing to ensure that your observations remain fresh in your mind.
You can identify the most common or serious problems users have encountered and take them to the doctor for further investigation.
Do not address all the problems that occurred; instead, focus on the most important issues and prioritize them for resolution.
