In my previous posts, I've talked about how you can analyze website data, or A/B test data, and use it to identify winning segments of users.
However, your wonderful new test design may be an unfortunate loser. It may have shown dreadful results - all the KPIs you can think of are in the red. There's no way to salvage this one, every single metric shows a drop for Recipe B. And I suppose you have two options: learn from the data and try again, or segment the data and see why it lost.
And so begins a trip down a very dangerous rabbit-hole.
"If we look at new versus return visitors, we find that return visitors didn't perform as badly as new visitors."
"And if we look at return visitors who were visiting on a mobile device instead of a laptop or desktop, then we see that performance is actually slightly better."
"And if we look at return visitors who visited on a mobile device and were looking for our higher-price products, then we actually see an improvement."
Great. But after three rounds of filtering, targeting or segmenting (your choice of terminology), you've gone from 50% of traffic (the test population) down to 4.3% of traffic. Is it really worth spending time, effort and energy to provide 4.3% of your traffic with a unique experience? If you're a luxury brand like Rolls Royce, or Beaverbrooks, then possibly, yes. If you're selling discounted pet supplies, perhaps not.
But we can get into this level of detail with our targeting and personalisation campaigns too. I've previously talked about the challenges of setting up a personalisation campaign - the first is obtaining and analysing the data, the second is having the content to present to user segments. But assuming we can make a decent effort at both, we don't want to get into too much detail in our segments. For example:
What do you show in the homepage hero banner? Or what do you show in your "We think you'll like this..." module? Do you stand at the front of your virtual store, identifying customers based on data such as previous visits or previous purchases, and say, "I think you'd like to buy this Lego set. It's not discounted, there's no extra incentive to buy, but we watched you on your previous visit, and we think this Lego set is for you."
Is your targeting that good?
In my experience and conversations with other professionals, Netflix and Amazon are often cited as the leaders in targeting. "Because you watched Star Trek: Voyager" is a reasonable and transparent explanation of the recommendations that Netflix shows me. And sure enough, around half of the recommendations are actually interesting to me - some of them I've seen before, some of them aren't my cup of tea. And when you have the opportunity to present me with 42 options (the screen scrolls horizontally seven times) then you can show me specific examples. If I don't know what I'm looking for, this is a good place to start.
So you could stand at the front of your virtual toy store and say, "We can recommend these Lego models...." and show 42 from your catalogue. And why not?
If that's not feasible (perhaps due to challenges with obtaining stock values - there's nothing worse than actively recommending an item that's out of stock) then you can be less specific. Standing on the virtual front door of your virtual store, you could offer, "Would you like to see our Lego models? Please walk this way" instead of "We think you want this model." You're more likely to get a positive reaction, for a start; in a world where engagement metrics are the king of the KPIs, you're at least more likely to see better results from being a little less specific. You can certainly expect to be more successful with a broad recommendation than with no targeting at all. Compare, for example, "Welcome to our toy shop! These are our favourite toys!" with "We think you're interested in construction toys." The first is symptomatic of the "We want to sell you this" which pervades many home-page banners, instead of the notion that we find out what our customers want to buy, and show them that. I'll leave that one there for now, but at least some level of targeting is better than none (and probably better than over-targeting too).
If we can say with 75% chance that a visitor is looking for Lego models, but only a 23% chance that it's Lego Technic (the advanced, engineering-level Lego), and only a 5% likelihood that it's a Lego Technic Race Car, then perhaps leading with one specific model is too much. It would be better to suggest the Lego Technic range, and direct users to a category page and let them find their own way from there.
Your virtual store could be selling electronics; home appliances; books; streaming TV shows; or whichever, but Lego has the advantage of being widely known globally, and very visual and tangible. Insert your relevant product subcategory in here (I suppose if I had been paying attention to your browsing habits, I could have personalised the content of the blog to make it more relevant to you. Maybe next time!).
