If you’re reading this, you’ve probably heard all about the benefits that experimentation can bring to an organisation and the relationship between a strong testing ethos and business success. Maybe you’ve just started a new role or joined a new team and want to know how to get the most out of experimentation. Perhaps you’ve just got a new testing tool and are using that as a trigger to reset your approach. It could even be that you’ve realised that your results have stagnated recently and it’s time for a revamp.
In any case, it’s important to understand how you’re going to approach experimentation and generate maximum value. With experimentation programmes typically requiring investments in technology, team members and possibly agency support, kickstarting your testing in the right way and gaining momentum early on is vital in demonstrating value.
For the purpose of this article, we’re going to focus on the strategic steps required to boost your programme. So we’re assuming that you’ve successfully gone through (or are about to complete) the technical onboarding process. In other words, you have your platform of choice implemented on the site and plenty of development resources readily available to build your tests ideas.
There are three main things to consider to ensure you’re kicking off with experiments that are going to have an impact on your business objectives and stand the greatest chance of solving the issues identified.
1. Align your experimentation strategy and tests with business goals and KPIs.
This is a golden rule for all your testing and one that you’ll hear us at Creative CX go on and on about! Before you rush straight into building your first test, it’s important to take a step back and make sure that the effort you’re about to put in is well aligned with the strategic objectives of the business and your team. Doing so is going to ensure that you’re maximising the potential returns of any test against your strategic measures of success and that you’re not simply running experiments for the sake of it.
If you don’t already know what your business goals are, and how they link to your teams performance, then it’s essential you spend time with stakeholders to determine what your North Star metric is. Once you have that in place, you can work down to identify the KPIs you are going to be targeting with your tests. On a test by test basis you should be checking the relationship between the primary metric and your KPIs to ensure you are aligned on how success will be defined. If there’s a mismatch then you should be challenging why the test is being run ahead of others which are going to directly relate to your team’s goals.
2. Use data and research to identify pain points, then fuel your ideation sessions.
Once you know the KPIs that your team are focusing on, you can start to get into the detail of the specific user problems you are looking to address. Quantitative sources (such as the numbers you get from your web analytics platform) will help you to identify where in the journey your users are struggling. Layering in qualitative research will allow you to get closer to the problem and uncover exactly why users are struggling and the specific obstacles or frictions they are facing. There are strengths and weaknesses for individual qualitative research techniques, so combining multiple sources of research and data is a key aspect of gaining useful insight into how your existing (and potential!) customers behave.
When you have identified a number of user pain points, organise these into problem statements. Problem statements should clearly define what the issue is and be backed up with a number of data sources from your research. Try to be customer-centric with your definitions, explaining the issue from the users viewpoint will help you to craft watertight problem statements, for example:
‘Users don’t see…’
‘Users don’t know that…’
‘Users are distracted by…’
‘Users aren’t able to…’
Problem statements done? You’re ready to start exploring possible solutions, this is where the ideation phase begins! The objective here is generate as many creative solutions as possible for the snag points and blockers that you’ve just listed as problem statements. The more solutions and executions you can come up with, the better. Use different brainstorming techniques and ideation methods to keep things interesting; where possible try to involve other programme stakeholders – they’re unlikely to share your viewpoint and your combined input should be more valuable by virtue of bouncing ideas off each other. Don’t settle for easy executions and force yourselves to be imaginative!
3. Use a prioritisation framework to establish which tests you run first.
Now that you are armed with a list of brilliant new test ideas, you need to work out which is going to be built first. All tests have an associated cost to build, whether it be budget spent on agencies, internal staff cost or just time not spent on another project. As it’s unlikely your team will have unlimited resources (not to mention unlimited traffic to test on!) you will need to pick and choose which tests get built first.
With different stakeholders contributing to the programme this is a potential minefield of opinion, so we highly recommend using a prioritisation framework to remove any bias and subjectivity. A simple model which can be used to prioritise experiments is known as PIE (Potential, Importance, Ease) can be used at the start of your testing programme. Score test ideas under each category to establish how much improvement a test is likely to offer, how important the page is (in terms of volume of traffic and the value it drives) and how difficult it will be to build a test on the page. Then rank your test ideas based on the cumulative score to see which should be prioritised.
For your first tests, it might be worth giving a greater weighting towards the ease of test build to make sure you get some simple up and running quickly. As your programme develops, increasing the scope of criteria used to prioritise your experiments is a good method to further remove bias that could creep into a simple model and will also ensure that you are running the tests best suited to your business and objectives.
Go kickstart your experimentation programme.
If you’re just starting off on your experimentation journey, it might seem like a daunting task to set up all the processes outlined in this article, when all you want to do is dive straight in to building your first tests! Investing time aligning your goals and building a cohesive structure to your programme at this early stage is crucial to success, and will help you to reap the benefits of experimentation and maximise ROI.
