“Hope is not a strategy.” – Vince Lombardi
Whilst we’ve shared insights on measurable, quantitative data in previous blogs, it should also be noted that we believe qualitative research is equally as important. But why you might ask? Essentially, qualitative research helps you understand the motives behind specific user behaviours, which in turn allows you to paint a broader picture about your customers and their potential pain points.
When approaching experimentation, many businesses base a lot of emphasis on gut-feeling when deciding what to test on their website, which often leads to more tactical approaches. The idea that they “hope” their website changes will be for the better indicating they may not have a concrete reasoning behind why their visitors are not converting. Ensuring you use a strategic approach for experimentation is essential and something we ensure all our clients have at Creative CX. Simply put, our strategic approach ensures every experiment is well aligned with the strategic objectives of your business while also having the highest value for your customers. You can see why this is more forward-looking and superior to your gut-feelings.
Ensuring you use a strategic approach for experimentation is something you will have heard us talk about often at Creative CX – we believe it is far superior to using a tactical approach! Essentially, this is where you make an effort to ensure that each experiment is well aligned with the strategic objectives of the business and your team.
There’s no doubt: qualitative research is an important part of the experimentation process, though it is somewhat overlooked by businesses. You might be wondering how you should go about gathering qualitative data, which tools you should use, and what you could find out from them. If so, find out all this and more in our latest blog.
What is Qualitative Research?
The use of qualitative research has been around long before websites even existed.
By nature these types of research are there to provide us with more rich and contextual data, often seen in science to provide data into beliefs, values, feelings and motivations that underlie behaviours. In our case, we use qualitative research to find out the ‘why’ behind specific user behaviour. In parallel with quantitative research, it allows us to gain a fuller understanding of our customers and the pain points they might be experiencing.
Why is this important?
At Creative CX, most of our experiments focus on a specific customer problem we’re attempting to solve. But in order to do this, we need to have a deep understanding of our client’s customers and why they behave the way they do on our client’s websites. And the only way we can achieve this is through qualitative research.
But as data is a key component in identifying your customer problems, it’s this information that can help shape your strategy, choose the suitable next steps, or change course if necessary. You might have heard the expression “no data, no party” before and it’s quite true; until you have data to support your idea, that’s all it is: an idea.
When to use qualitative research
Qualitative research is best used when you’re looking for the motivation behind a particular action. This could be something like wanting to know why bounce rates have increased on your website’s cart page.
As you probably know, an increase in bounce rate on a page could be for a myriad of reasons. Could it be because the page is confusing or untrustworthy to users? Or is it simply because they’re exiting the page in order to find a discount code before proceeding with their purchase? Each of these answers would lead us to completely different customer problems, hypotheses, and therefore experiments.
At Creative CX, we take a question-led approach to data. Emphasis is placed on friction points but it is also why users were motivated to convert on a particular website too. Taking this approach allows us to be efficient, gaining impactful and actionable insights that can drive our experimentation programme. We like to ask the kind of ‘whys’ that can’t be uncovered using analytics or quantitative data alone.
Mastering which tools
Below are just some of the tools and techniques we use at Creative CX to gather qualitative data that drives our customer problem statements. Look out for our articles coming soon on how you can master these!
- Click/Heat Map Analysis
- Usability testing
- Heuristic Reviews
- Analytics Reviews
- On-Site Polls
- Customer Service Interviews
At Creative CX we also place a great deal of value on our Opportunity Audits, it is what we suggest to clients first to really drive their programmes – find out more here! This is a powerful combination of research, analysis, ideation & journey mapping methods that produce impressive findings. Again, highlighting how important qualitative research is in driving innovation. Learn more about becoming a champion of qualitative and quantitative research triangulation in our article!
Improving your Strategic Advantage
So you know qualitative research will help you to identify specific customer problem statements, which will in turn will help to provide foundation for future iterative tests. However, when it comes to running qualitative research alongside live experiments, a lot of businesses miss a trick, especially when it comes to gathering learnings among different site variations.
Let’s say you are facing low customer interaction with a new product that has launched on your homepage. Using analytics, you found that users were not interacting with the new product featured on the homepage or the messaging surrounding it. As a result, you changed various elements of the feature to further investigate the issue and identify the main pain points. You performed several tests, including design, placement and prominence.
The experiment is giving you flat results, despite all data pointing towards positive results.
If the experiment has been live for the expected test duration and there has been no significant changes, this may make it difficult to deduce any insights, making it harder to drive iterations. This is where qualitative insights could really help you to understand user behaviour further. One way of gathering this research could be through setting up a quick, one-question, exploratory survey for users on the homepage.
In this example, you might ask: “What is preventing you from learning more about our new product?”
A number of responders might explain that the value proposition used confusing or unclear jargon, creating a disconnect between what they expect the product to be and what it actually is.
So, even after you tried improving different aspects of the copy surrounding the new product feature, it turns out that it was the confusing wording that caused the low customer interaction with your new product launch. By knowing this sooner you could have reached a successful test in less time and fewer iterations!
This highlights the importance of running qualitative research alongside your live tests. Not only does qualitative research help you solve problems you probably didn’t even know you had, but it also makes your experimentation programme more efficient.
Learn how users interact with your test variants
When you create experiments, your variants should be designed in order to isolate changes and gather insights about your customers. However, it’s important to note that you can also learn a great deal from qualitative research – and better yet, you can usually see results for this in real-time.
Here’s an example: you are about to implement a new feature that you believe will have a significantly positive impact on the page’s conversion rate. You have a few variants you want to test, but you have just realised that the feature will sit below the fold. What if users don’t even see the change because they don’t scroll that far down? You might have engagement metrics that are tracking the clicks, but how do you find out how users are interacting with the page?
Quantitative data such as scroll maps can help point towards which variant performs best but won’t explain the why. This is where qualitative research can, once again, save the day. With the right tools, such as on-site polls, you can essentially find out why the better performing variant is best received by the user. These techniques essentially track how users interact with the page and its elements, including your variations.
Running an on-site poll while the test is live will provide you with insights into why one variant might be performing better than another. Something as simple as:
Q1: How simple was it to find that content? (Score of 1-5)
Q2: Why?
This is an incredibly valuable way of uncovering the qualitative findings. The responses from the poll underpin exactly how they navigate through the site so you learn where and why the user stops scrolling. This helps you understand why a user might be bouncing off the page or dropping off, which could give you ideas about how you might need to tweak the page’s design and content.
For instance, you might realise that you need to tweak the page’s design to ensure users are focused on the content you want them to see. You could do this by adding visual cues, which quite literally point users in the direction of the page’s CTA, form or important information, or adding better eye paths, which designs the page in such a way that it draws the user’s eye to a certain area of the page.
Qualitative research, therefore, gives you further opportunities for testing and iteration.
Impact on iterative tests: some you win, some you learn
Sadly, solving all your customer’s problems in the first experiment is highly unlikely. There is probably an infinite number of possible solutions to your customer’s problem, so testing your hypothesis multiple times is the way forward. Even if your test’s a winner, it’s likely you can use your learnings from this test and previous experiments to find an even better solution, improving the customer experience that step further.
And better yet, qualitative research offers an additional layer of learning that allows you to learn more about your customers and, in turn, make your experiments more efficient.
What do we mean by this? Well, let’s imagine that one of your iterations places the ‘Add to Basket’ button above the fold on a product detail page (PDP), which results in an increase of button clicks and a jump in revenue of approximately 8% – that’s great! But where do you go next? Well, you can learn a lot from qualitative data about why this has happened.
For instance, you might find that, looking at the screen recordings and heatmaps, users were more likely to engage with the ‘Add to Basket’ button because it was more visible on the page. You can therefore turn this learning into an iterative test hypothesis. With more specific test hypotheses, we can more confidently pinpoint how to improve customer experiences without wasting loads of time on iterative tests.
Qualitative research helps provide insights that answer the ‘why’ questions in experimentation, which will, in turn, feed your experimentation programme and offer you a deeper understanding of your customers. This helps ensure you get the most out of your experiments.
To find out more about how best to use qualitative research in your experiments, feel free to reach out to us on LinkedIn or get in touch through our website.
