Yet another usability testing participant in their car! Not driving thankfully but parked in a car park whilst Andy (one of our Senior UX Consultants) moderated the session remotely.
Even though covid forced us to move all of our Usability Testing from our London labs to purely remote, we’ve actually been loving the format. And we’ve been really busy this last year with user research to drive our programmes, despite my initial concerns.
Moderated usability testing has always been one of the most valuable and powerful methods we use for helping our clients to design and optimise their digital products and services. Helping to ensure that they are human-centric and adding as much value as possible to customers and the business. That’s why it plays such an important role in our Opportunity Audits and alongside all of our online experimentation (A/B testing) programmes.
Why moderated usability testing is more effective than unmoderated usability testing?
The moderation side of task-based usability testing is extremely important in our experience and the key reason why moderated usability testing is so much better than unmoderated. Moderated sessions with professionally recruited participants that match the target audience deliver so many more useful insights than unmoderated usability testing, and this is largely due to the moderator.
A great moderator is able to put the participant at ease; introduce the scenario; set the tasks in the right way; check that the participant really understands before starting; adapt the flow on the fly, and crucially dig deeper with follow-on questions and mini-tasks to really uncover the ‘why’ behind the customer problems.
This greater level of insight and understanding empowers the team to make far better solutions. That’s why A/B test ideas from user research like this result in higher overall win rates.
What are the pros and cons of going remote for moderated usability testing?
So the big question for us when we realised we’d have to move to 100% remote, was whether we’d still be able to deliver such high-quality research? Would the technical set-up and remoteness get in the way? Would the ‘social distance’ between moderator and participant adversely affect the way we build a repour with the participant? Would clients still be able to and willing to watch the sessions and participate in note-taking?
Below I’ve summarised some of the pros and cons based on our experiences over the last year.
The Cons
- We miss out on the body language because the camera only catches the participants’ face.
- There are certainly technical challenges such as internet connection issues and confusion over the software. This is almost inevitable in these situations where we are so reliant on the participants’ own set-up of their phone or computer and internet connections. So it’s important that we allow a little bit more time for set-up and also that we have backup plans in place. For example, we’ve had to rearrange a number of the sessions to the following day.
- A few technical challenges for clients logging in to watch and participate. We resolved this by providing better instructions and also a quick software orientation session beforehand.
The Pros
- Whilst we miss out on the body language, we still get to see the expressions on the users’ face – which is in itself very telling! But this is only possible if using specialist software that allows the user to share the screen at the same time as sharing their camera.
- We get to test in the participants home (or car!), which is great as it’s a more natural experience for them as they are in their own environment.
- We have access to a far wider variety of users from all over the UK as well as in different countries, because it no longer matters where they live, as long as they have an internet connection.
- It’s also just a lot more convenient for users to fit a session into their schedule as there’s no travelling involved. So we find we have a lower drop-out rate.
- Clients can watch the sessions and participate live from wherever they are in the world too.
What does the future look like for our usability testing?
The great news is that the Pros of conducting sessions remotely have far outweighed the Cons, and we’ve been delighted with the results.
The team have had to adapt and we’ve changed our processes, but this effort has very much paid off.
In conclusion, even when things open up and we are able to run in-person usability testing in the labs again, I’m pretty sure that we’ll be sticking to remotely conducted moderated usability testing for the vast majority of our user research.
