As we continue our series of key takeaways from the VWO CONVEX sessions, we’re excited to share insights from a conversation between our CXO, Chris Gibbins, and Vicky Routely, CRO Manager at easyJet.
Together, they explored the airline’s unique customer-centric approach to experimentation.
Our first article in this series was Annette Rowson, Digital Optimisation Lead at Primark, discussing Primark’s bold step into digital experimentation.
If you missed it, have a read and get caught up!
Known for its budget-friendly flights and commitment to delivering exceptional value, easyJet operates two distinct websites: one for airline bookings and another dedicated to package holidays. They also offer an app, making it even more convenient for customers to plan and manage their trips on the go too.
These platforms allow customers to book everything from flights, hotels, and extras, all in one seamless experience that caters to both independent travellers and those seeking complete holiday packages.
While they have an overarching strategy for both sites, the experimentation programmes are run slightly differently on each. However, there is a clear intention to integrate these efforts further as the businesses become closer.
At Creative CX, we support the easyJet Holidays Experimentation and Product teams with fully embedded Experimentation and UX consultants, and on the airline’s side, with an ad hoc consultancy and regular Opportunity Discovery Research.
How Experimentation Works at easyJet
Chris mentioned in the interview that there is a lot of talk in the experimentation comminity about the different kinds of operating models, and he asked Vicky to explain a bit more about how easyJet are integrated with all of their product teams.
easyJet’s operating model wasn’t centralised, but also that it wasn’t yet a centre of excellence (hybrid) model. It was actually somewhere in between as they are still designing the actual experiments and developing them within the experimentation team but working closely with the product teams when it comes to the research, ideation and UX design.
This structure encourages strong collaboration, as the product teams are so invested in the experimentation process.
The experimentation team is self-sufficient, comprising developers, designers, and researchers, which allows for rapid delivery of value to customers without being hindered by the broader product development process.
Vicky also emphasised how great the whole of easyJet’s mindset is:
“They already have the mindset that they want to test everything. They want to know that what they’re delivering is valuable to their customers.”
Additionally, the experimentation team plays a vital role in educating the business overall about the benefits of experimentation and A/B testing.
On the airline’s side, easyJet has the fully in-house team of developers and designers who are directly linked to a product owner.
Although they don’t sit within the squads like a traditional product development function, the aim is for them to work specifically with a particular product owner to build that relationship, allowing them to focus on that product area consistently.
This means they are almost part of the product team, delivering outside the code base and release schedule.
By being linked to those product owners and working closely with them, the team maintains self-sufficiency with their designers and analysts within the experimentation team.
This structure encourages strong relationship building, enabling developers and designers to better understand the product direction. Vicky emphasised the importance of designers grasping what’s coming down the line from a product perspective, as it influences their approach to specific A/B testing.
She expressed satisfaction with experimentation being recognised as a function within the product team, highlighting that it significantly enhances the value they can deliver because they are aware of everything in the pipeline, ensuring nothing comes as a surprise.
The experimentation team is seen as a core piece of the digital puzzle, focusing on strong relationships with the leadership team and other departments.
Ideas can come from anywhere within the business, and the team is actively involved in discussions about digital initiatives.
All tests currently run on the client side, and while they do not run A/B tests at full scale due to the website’s high traffic, they aim to turn around changes as quickly as possible, typically within one to two months.
Current Velocity of A/B Testing at easyJet
In their discussion about A/B testing velocity, Vicky shared that on the holidays side, easyJet is currently running around 10 tests per month, a significant increase from just 2 or 3 tests earling in the year before working with Creative CX.
Creative CX has boosted their velocity significantly and helped to enhance their overall experimentation efforts. Vicky noted that this progress has led to noticeable changes across the business.
On the airline’s side, easyJet conducts about eight to ten tests per month as well. However, the nature of these tests is often larger in scale, such as significant page or funnel changes, which require more development time.
While this results in a slightly lower velocity compared to the holiday side, Vicky emphasised that testing these larger changes is essential for meaningful outcomes.
Chris pointed out the importance of balancing velocity with quality, noting that while increasing the number of tests is straightforward with smaller experiments, maintaining high standards is crucial, especially when taking bigger bets.
Vicky agreed, highlighting her focus on ensuring that the team prioritises the right experiments that deliver real value, rather than merely meeting a target number of tests.
She stressed the need for buy-in from leadership to support meaningful experimentation and to track the value generated from these efforts rather than just presenting a high volume of tests.
Problem-First Prioritisation
At Creative CX, taking a problem-first approach and specifically problem-first prioritisation is something that we’ve been developing over the last couple of years and which we are very passionate about because of the impact that it can have on an organisation and how they experiment.
We have a full dedicated article about problem-first prioritisation, which you can read here.
Vicky and Chris addressed the recent shift at easyJet towards a problem-first prioritisation approach. This concept is new to the organisation, having only been implemented in early 2024. With support from Creative CX, Vicky emphasised:
“You guys at Creative CX have really supported us on that, and this is how we’re now looking to work.”
Chris explained that “the traditional Prioritisation frameworks encouraged a kind of bad solution-first behaviour because if you have an idea, you obviously want to ramp it up the list by then cherrypicking data to back up the idea.
This is the FILTH (Forcing Insights & Learnings To Hypotheses) confirmation bias that’s so prevalent.
Vicky shared that when easyJet initially started their programme, they had a complex prioritisation model that inadvertently fostered this behaviour among product owners.
This approach often resulted in ideas being backed by limited data, and that the ideas did not truly address real customer problems.
Chris added that this solution-first mindset often overlooks larger opportunities elsewhere in the user journey, leading teams to focus on optimising their preferred areas of the website, instead of conducting comprehensive end-to-end research to identify the real problems in the first place.
By prioritising problems based on their seriousness, the reach (how many people it affects) and the confidence in understanding, crucially before ideation, senior leaders can effectively influence which problems to tackle in a given quarter.
easyJet are now prioritising quarterly problem statements, which helps shift the mindset across the business to address larger, more impactful challenges aligned with overall business objectives and customer needs.
Chris noted that this approach is particularly beneficial in larger organisations with product teams responsible for different aspects of the user journey, enabling a more structured assignment of problems and objectives to specific teams.
This method not only addresses customer problems, but also ensures that product managers see their challenges being actively solved, facilitating a more collaborative ideation process.
Chris then drew the audience’s attention to this graphic which illustrates easyJet’s problem-first approach. At the top of the timeline, he highlighted a quarterly process called Opportunity Discovery Research, which serves as the starting point.
This phase includes usability testing that often brings revealing and valuable insights into user needs and challenges. The next critical step in this timeline is triangulated analysis – a process of building problem statements by combining both qualitative and quantitative data.
Encouraging Diverse Perspectives and Wild Card Ideas in Ideation
Involving a broad mix of team members in ideation sessions is essential to uncovering impactful ideas. Vicky noted how inviting senior stakeholders to these sessions allows them to witness the challenges customers face directly, often leading to stronger buy-in.
For instance, seeing a customer struggle with booking a flight can offer senior leaders a fresh perspective, relieving issues that may go unnoticed by those who use the website daily.
These ideation sessions help shift the team away from relying on one single solution, encouraging “wild card” ideas instead.
Given easyJet’s high web traffic, they are well-positioned to conduct A/B/n tests with multiple variations, enabling the team to explore more creative options beyond the initial idea.
“It’s just about how we communicate thinking outside the box,” Vicky explained, “coming to ideation sessions and doing those sorts of things, because it puts you in an environment where you can come up with multiple ideas, you’re not just coming up with one, and that does help to broaden horizons.”
To ensure the focus remains customer centric, sessions are always grounded in clear, research driven problem statements based on actual data.
Vicky also emphasised the value of including the developers in these sessions, as they bring technical insights that can unlock ideas others might deem impossible.
“I might think, ‘Oh, I’m not even going to put that idea down because it’s going to be way too technical and way too difficult,” Vicky shared, “but we’ve had a scenario where I have done that in an ideation session, and I happened to be in a group with a developer who said it was definitely possible.”
Everyone is going to have a different view of how something might work and bringing together different perspectives helps create the best ideas and mitigates any bias.
Plans for Personalisation at easyJet
At the start of the interview, Chris pointed out some existing elements of personalisation on easyJet’s flights website: “You can see a bit of personalisation here; it remembers my last visit and, critically, the airport I use most often. One of my favourite features is that you can immediately see the price for different flight days.”
Building on this, Vicky highlighted easyJet’s focus this year on expanding personalisation efforts, especially as they look to integrate the flights and holidays businesses to convert customers into booking packages, and not just flights.
The goal involves understanding the distinct needs of their varied customer base:
“There’s a very different customer who might want a one-way flight for business to Milan versus a customer who might want to go on holiday with their family in the middle of August for two weeks. We need to make sure that the way we’re presenting our package holidays offering is targeted to the right customer base, and that is something we’re really working on this year.”
To meet the diverse needs of their customer base, easyJet is focused on creating a personalised journey for users across the entire website experience, extending beyond just those browsing package holidays.
A key initiative in this effort is session continuity, which allows users to seamlessly pick up where they left off without needing to re-enter data – an essential feature for mobile experiences. Chris encouraged organisations looking to start their personalisation efforts to consider session continuation as an initial step in their strategy.
Closing Insights: Leadership Buy-In and Personalisation
In closing, Vicky offered key advice for others in experimentation or CRO roles who want to scale their programmes and drive impactful improvements to increase value.
A primary focus, she offered, should be securing buy-in from leadership teams; the more that experimentation’s positive impact on the business is communicated, the easier it becomes to build and advance the programme.
Vicky also highlighted how strong senior support at easyJet has allowed their experimentation programme to flourish, making the process of scaling seamless.
As easyJet continues to expand its personalisation initiatives, it’s an exciting time for the company, with new strategies aligning their airline and holidays businesses.
Our series recapping VWO CONVEX 24 takeaways will wrap up with a final exploration into insights-led personalisation, where Chris’s workshop will explore the essential ‘whys’ and ‘hows’ of developing effective personalisation strategies. Don’t miss out!
