Measuring the success of product recommendations on the e-shop: How to optimize offer personalization
26. 6. 2024
In the digital era of sales, personalization has become a key element for successful online commerce. One of the most effective tools for personalizing product offerings on e-commerce sites are product recommendations. However, the success of this type of personalization cannot be merely estimated intuitively. To measure and optimize the effectiveness of product recommendations, various metrics tracking the performance of a given recommendation should be used. These can most often be accessed through precision measurement and AB-testing. However, there are a number of pitfalls here that e-shop owners do not initially realize.

Metrics
A relevant AB test is the first step to success. But when you're testing different forms of product recommendation, you need to have thought about what the purpose or long-term goal is. Does the retailer want to increase conversion rates or achieve more sales? Increase average margin? To boost private label? All of this can be measured quite objectively and relevantly, but one must know how to do it.
The first thing to remember is that some metrics can go against each other, while others can cannibalize each other. In practice, it may look like this.
For example, is it possible to increase both e-commerce sales and average margin at the same time? It sounds like every marketer's dream, but the reality is quite different. Because if a retailer goes for the average margin and starts pushing the highest margin products into the recommendation areas, it will often be complementary goods or private label, which in turn has a lower absolute price. So, as a result, you can achieve, for example, a 10% increase in average margin, but sales fall by 2%. Customer behaviour may also reveal that they don't actually 'like' such recommendations and both metrics will fall.
In another case, recommendation areas can be placed on the home page and throughout the product detail, for example. After deploying an external tool on these recommendation areas, some retailers can expect sales to automatically skyrocket by 30% due to the "magic" as the vendor promises this on their website. But it's not that simple. For example, the total traffic to the homepage and the bottom of the product detail area is only 20% of the entire site. So if an e-shop wants to lift sales and outperform the competition, it will need to add more space to maximise the reach of as many customers as possible, not just the top fifth.
So what are the relevant metrics and what can they mean for us, including the various pitfalls?
AB-testing
If the e-shop knows what metrics to measure and what it wants to achieve in the result, it's time to start AB-testing.
In general terms, AB-testing is a basic tool for comparing the results of two or more recommender variants. For example, it might look like an e-commerce store already has basic product recommendation functionality and is about to compare it with an external tool that offers higher performance. In practice, the tools may then differ in the effectiveness of different algorithms, parameters or recommendation strategies.
The principle is simple. You need to randomly divide e-shop visitors into two or more equally sized groups (A, B, etc.), with each group seeing a different product recommendation option. Then their behavior is compared against these variants and the selected metrics. For example, how many visitors purchased from each group, how long they stayed on the page, or what their average shopping cart value was.
However, what is important to keep in mind and elaborate further?
You can't test relevance on a small sample of data.
If the results seem too good or bad in favor of one of the options, there's a mistake somewhere!
You can't look at the results every day, AB-testing is like investing in the stock market.
Most importantly: If an e-shop really wants to know what is (best), it will cost time and (probably) money.
From our blog
Looking back to 2024: Zoe.ai's dynamic development and expansion plans
How does Zoe.ai Product Manager Michal Krňák assess the past year 2024? What AI tools for e-commerce personalization did Zoe.ai expand its portfolio with? And what plans does the team have for this year?
Should we be worried about our data being leaked when shopping online?
Is online shopping safe? When making online purchases, consumers are required to provide a range of personal information, including payment information, address and contact details. This information can be a valuable target for cyber criminals seeking to obtain sensitive data for nefarious purposes such as identity theft, fraud or extortion.
Let's talk
To arrange a demonstration with one of our experts, or to request a free consultation with one of our e-commerce specialists, please contact us. We will demonstrate how Zoe.ai can help you and your e-commerce store become more profitable.