Why Personalisation Programs Struggle

So why aren’t we living in a world of perfect personalisation? We've been hearing for a while that it'll be the next big thing, so why isn't it happening?

Because it’s hard.  There's just too much to consider, especially if you're after the ultimate goal of 1-to-1 personalisation.

In my experience, there are three areas where personalisation strategies come completely unstuck.  The first is in the data capture, the second is the classification and design of ‘personas’, and the third is in the visual design.

1. Data capture:  what data can you access?

Search keywords?
PPC campaign information?
Marketing campaign engagement?
Browsing history?
Purchase history?
Can you get geographic or demographic information?
Surely you can’t form a 1x1 relationship between each individual user and their experience? 
Previous purchaser?  And are you going to try and sell them another one of what they just bought?
Traffic source:  search/display/social?
What products are they looking at?
What have they added to basket?

2. Classification:  how are you going to decide how to aggregate and categorise all this data?  

Is it a new user?  Return user?

And the biggest crunch:  how are you going to then transfer these classifications to your Content Management System, or to your Targeting engine, so that it knows which category to place User #12345 into.  And that’s just where the fun begins.

And how do you choose the right data?  I'm personally becoming bored of seeing recommendations based on items I've bought:  "You bought this printer... how about this printer?" and "You recently purchased a new pair of shoes... would you like to buy a pair of shoes?" As an industry we seem to lack the sophistication that says, "You bought this printer - would you like to buy some ink for it?" or "You bought these shoes, would you like to buy this polish, or these laces?"

3. Visual Design

For each category or persona that you identify, you will need to have a corresponding version of your site.  For example, you’ll need to have a banner that promotes a particular product category (a holiday in France, the Caribbean, the Mediterranean, the USA); or you may need to have links to content about men’s shoes; women’s shoes; slippers or sports shoes. 

And your site merchandising team now needs to multiply its efforts for its campaigns. 

Previously, they needed one banner for the pre-Christmas campaign; now, they need to produce four, five or more instead.  This comes as they are approaching their busiest period (because that’s when you’ll get more traffic in and want to maximise its performance) and haven’t got time to generate duplicated content just for one banner.

Fortunately, there are ways of minimizing the headaches that you can encounter when you’re trying to get personalization up and running (or keeping it going).

Why not take the existing content, and show it to users in a different order?  Years ago, there was a mantra (with a meme, probably) going around that told us to 'Remember: There is no fold' but I've never subscribed to that view.  Analytics regularly shows us that most users don't scroll down to see our wonderful content lying just below the edge of their monitor (or their phone screen).  So, if you can identify a customer as someone looking for men's shoes, or women's sports shoes, or a 4x4, or a hatchback, or a plasma TV, then why not show that particular product category first (i.e. above the fold, or at least the first thing below it)?

4. Solutions

The flavour du jour in our house is Airfix modelling - building 1/72 or 1/48 scale vehicles and aircraft, so let's use that as an example, and visit  one of the largest online modelling stores in the UK, Wonderland Models.

Their homepage has a very large leading banner, which rotates like a carousel around five different images: a branding image; radio controlled cars; toy animals and figures; toys and playsets; and plastic model kits.  The opportunity here is to target users (either return visitors, which is easier, or new users, which is trickier) and show them the banner which is most relevant to them. 

The Wonderland Models homepage.  The black line is the fold on my desktop.

How do you select which banner?  By using the data that users are sharing with you - their previous visits, items they've browsed (or added to cart), or what they're looking for in your site search... and so on.  Here, the question of targeted content is simpler - show them the existing banner which closest matches their needs - but the data is trickier.  However, the banners and categories will help you determine the data categorization that you need to - you'll probably find this in your site architecture.

However, here's the bonus:  when you've classified (or segmented) your user, you can use this content again... lower down on the same page.  Most sites duplicate their links, or have multiple links around similar themes, and Wonderland Models is no exception. Here, the secondary categories are Radio Control; Models and Kits; Toys and Collectables; Paints, Tools and Materials; Model Railways and Sale.  These overlap with the banner categories, and with a bit of tweaking, the same data source could be used to drive targeting in both segments. 

As I covered in a previous blog about targeting the sequence of online banners, the win here is that with six categories (and a large part of the web page being targeted), there are thirty different combinations for just the first two slots, with six options for the first position, and five for the second.  This will be useful as the content is long and requires considerable scrolling.

The second and third folds on Wonderland Models.  The black lines show the folds.

Most analytics packages have an integration with CMS’s or targeting platforms.  Adobe Analytics has Target, which is its testing and targeting tool.  It's possible to connect the data from Analytics into Target (and I suspect your Adobe support team would be happy to help) and then use this to make an educated guess on which content to show to your visitors.  At the very least, you could run an A/B test.

5. The Challenge

The main reason personalization programs struggle to get going is (and I hate to use this expression, but here goes) that they aren't agile enough.  At a time when ecommerce is starting to use the product model and forming agile teams, it seems like personalization is often stuck in a waterfall approach.  There's no plan to form a minimum viable product, and try small steps - instead, it's wholesale all-in build-the-monolith, which takes months, then suffers a "funding reprioritization" since the program has nothing to show for its money so far...  this makes it even harder to gain traction (and funding) next time around.

6. The Start

So, don't be afraid to start small.  If you're resequencing the existing content on your home page, and you have three pieces of content, then there are six different ways that the content can be shown.  Without getting into the maths, there's ABC, ACB, BAC, BCA, CAB and CBA.  And you've already created six segments for six personas.  Or at least you've started, and that's what matters.  I've mentioned in a previous article about personalization and sequencing that if you can add in more content into your 'content bank' then the number of variations you can show increases exponentially.  So if you can show the value of resequencing what you already have, then you are in a stronger position to ask for additional content.  Engaging with an already-overloaded merchandising team is going to slow you down and frustrate them, so only work with them when you have something up-and-running to demonstrate.

Remember - start small, build up your MVP and only bring in stakeholders when you need to.  If you want to travel far, travel together, but if you want to travel quickly, travel light!

Zero Traffic

I've mentioned before that it's always a concern when any one of your success metrics is showing as zero. It suggests that any part of your tracking, calculation or reporting is flawed, and there's no diagnostic information on why.

I have had an ongoing tracking problem with this blog, but hadn't realised until several weeks ago.  I use Google Analytics to track traffic, and the tag is included in one of the right-column elements, after the article list and some of the smaller images.  All good, lots of traffic coming in on a weekly and monthly basis.

Except, as I realised last November, none of my posts from 2021 or 2022 were showing any traffic.  Zero.

I was getting plenty of traffic for my older posts (some of them even rank on the first page of Google for the right search terms) and this was disguising the issue.  Overall traffic was flat year-on-year, despite me keeping up a steady flow of new articles each month.  And then I discovered two gaps in my data:

1. Zero traffic on mobile phones

2.  Zero traffic from social media

Which is obvious in retrospect, since I share most of my posts on Facebook, and my friends comment on my shares.  

In order to tackle these two issues, I've taken a number of steps (not all of which have helped).

a. I've moved the tag from an element in the right column to the middle column, under the content of the post.  The right column doesn't load in mobile devices, due to thr responsive nature of Blogger, so the tag never loaded.  And hence I never saw any of my mobile traffic.

This has worked: I've tested the tag on my own mobile phone and I can see my own visit.  Yay! An increase from zero to one is an infinite increase, and it means the tag is working.

b. It turns out that Facebook's own in-app browser doesn't fire the tracking tag.  At all.  I am in the process of adding the Facebook user agent to my code, and in order to do this, have upgraded to Google Tag Manager.  I'm still not seeing Facebook-referred traffic, but it's an improvement.

And I'm looking at moving to a different platform for my blog.  I've had this one for over 10 years, and it pre-dates my Facebook account.  Maybe it's time for a change?

Other articles I've written on Website Analytics that you may find relevant:

Web Analytics - Gathering Requirements from Stakeholders

Analysis is Easy, Interpretation Less So
Telling a Story with Web Analytics Data
What is "Direct Traffic"?
Reporting, Analysing, Testing and Forecasting
Pages with Zero Traffic

The Parable of the 99 Sheep

The parable of the lost sheep/good shepherd is very well known, especially in church circles.  I learned the parable in Sunday school, and I've heard sermons on it many, many times from many different preachers and teachers.  It's a reminder to the church that God, the Good Shepherd, wants to gather as many people into the church as possible, and it's frequently used as part of what might be called a recruitment drive - time to evangelise to neighbours, friends and family who aren't part of the flock.  After all, the sheep are all equally valuable, and introducing more sheep to the flock is a good thing.

Here it goes:

"Suppose one of you has 99 sheep, and you decide you want to increase your flock by one.  Would you not leave the 99 sheep in the open country, and go out to find a hundredth sheep, and bring it home on your shoulders with rejoicing?  In the same way, there is more rejoicing over one sinner who repents than over 99 righteous people who do not need to repent."

As you can see if you check your Bible, that's not what the parable says.  

But when it's read out in church, the message that often accompanies the reading is the 'recruitment drive'.  It's as if the parable says we need to get the hundredth sheep because the hundredth sheep is equally important to the 99 already safe in the flock.  And: we the church need to get the extra sheep.  I'm not saying that evangelism is unimportant (it's crucial), but there's another way of reading this parable:  what does Jesus say?

Then Jesus told them this parable: “Suppose one of you has a hundred sheep and loses one of them. Doesn’t he leave the ninety-nine in the open country and go after the lost sheep until he finds it? And when he finds it, he joyfully puts it on his shoulders and goes home. Then he calls his friends and neighbors together and says, ‘Rejoice with me; I have found my lost sheep.’ I tell you that in the same way there will be more rejoicing in heaven over one sinner who repents than over ninety-nine righteous persons who do not need to repent.

Matthew 18:10-14; Luke 15:1-7; John 10

At the start of the parable, the shepherd already has 100 sheep in the flock.  And the truth is that sometimes, sheep wander off.  It's in their nature - they keep their nose to the grass, and they follow their instincts towards what they think is the next best grass... the next thing they know, they're not in the flock any more.  And we are just the same.  Yes we are.  We go to church every Sunday, we join house groups and we attend the prayer meetings.  We all think we're so clever, and we know what's going on in our lives, but we get comfortable, and then we get distracted from where we're supposed to be going.  And then suddenly, we're not sure where we are any more.

It happens.  Christians get criticised for being willing to identify with sheep, but we people aren't half as clever as we think we are, and we follow the crowd far more often than we'd like to admit. 

Sometimes, even members of the flock need to be found. And it's ok if that's you (it's been me).  If you read the parable, you'll notice that there is no blame attached to the sheep.  The sheep isn't berated or criticised, because sheep are sheep (that's why they need shepherds).  Also: it's the shepherd who goes to search and retrieve the 100th sheep, not some of the 99. 

The parable isn't described as the parable of the bad sheep, it's the parable of the good shepherd - who cares when a member of the existing flock wanders off and gets lost.  He doesn't shrug his shoulders and say, "Oh well, a 1% loss is acceptable for this year's performance, I still have the other 99 and that'll balance the books."  He's searching for even one who goes astray, just as much as for the ones who aren't yet part of the flock.  

Other articles I've written based on Biblical principles

10 things I learned from not quite reading the Bible in a year
Advent and a Trip to London
Advent: Names and Titles
Reading Matthew 1
My reading of Matthew 2
The Parable of the 99 Sheep
Why I Like Snow (it's not as crazy as you may think)

First Times of 2022

In what is becoming an annual tradition (here's the 2021 list), here's my list of 'things I did for the first time this year', which shows no sign of getting any shorter, even though I'm getting older.  

1. I taught someone how to tie a necktie.  It wasn't complicated, and my student was a very capable learner who simply hadn't needed to learn how to tie a tie before.  Took us about three minutes.

2.  I taught someone how to tie their shoelaces (because Velcro has been around for a while now, but hasn't taken over completely).  It took us five minutes: four minutes for me to analyse and work out how I've done it on autopilot for the last 40 years, and then one minute to demonstrate it step by step. 

3. I dressed up as Darth Vader.  Quite a few times, in fact, starting off with comic conventions initially, then a few charity events, but later in the year I also then attended a birthday party as an invited guest.  I mean, they weren't inviting me, they were inviting Darth Vader as part of my local Star Wars group and it would have been rude not to accept.  We attended a birthday party for a young man with cystic fibrosis, and for his friends, some of whom had other disabilities. 

And to complete the year, I attended a Christmas Carol service as Darth Vader, playing the role of King Herod, sharing the stage with a ballet dancer who played the role of the Angel Gabriel.  Not something I'd anticipated at the start of the year!

4. I spoke at my church on a Sunday morning.  I was invited to speak for 10 minutes on one of the Fruits of the Spirit, and chose patience (because people think I'm patient when I'm actually just very good at not speaking out). I've spoken at Sunday evening youth group meetings, but that was 20+ years ago, and this audience was a little more mature. I enjoyed it (I wrote and practised for weeks) and might get to do it again next year.

Here's the link to the video recording: Youtube Link

5.  I became a Trustee at my church.  This means I have responsibilities (legal responsibilities) for the finances and legal behaviour of the church.  I'm not the only Trustee - we have four in total, plus the senior minister- and the others have all been in the role for several years. I'm hoping to get up to speed pretty quickly.

6. I found, and completed, a South Korean "K-Drama" series which I selected.  As I mentioned last year, my wife introduced me to K-dramas in early 2021, and we watched "Crash Landing On You" which is one of the best drama series I've ever watched - her suggestion.  We also watched "My Love From The Star" which was good (after CLOY, the others tend to pale by comparison).  

This year, I found "Extraordinary Attorney Woo", which is set in a South Korean law firm. Attorney Woo has autistic spectrum disorder, and is a spectacular law student who's just graduated and started work practising law.  It's dubbed, but you soon forget that as the production and sound is exceptionally good.  I recommend it, it's heart-warming and life-affirming.

Here are some other K-dramas that we've watched and enjoyed on Netflix

Crash Landing On You - the original and best

My Love from the Star - an alien lands on Earth, and doesn't age.

Extraordinary Attorney Woo

7. I taught myself to solder.  Yes, I'm a late developer, but I taught myself out of necessity. I needed to fix my Darth Vader voice box, and didn't want to wait until I could get somebody else to do it.  I have since soldered a couple of other small electronics items, and only burned myself slightly once, on my thumb.

8. I taught Ben to play a piece of music on the piano. In early September, he came home from school with music homework: perform a piece of music to a backing track.  Ben won't mind me telling you that he has zero interest in music and hence no experience or musical knowledge.  However, by clever planning and careful choices, he learned to play firstly Twinkle Twinkle Little Star, and then the main theme from Star Wars on the piano, single-finger melody, in a weekend (and then practised it for the next two weeks ready for his music lesson).  

A piano will always play the same note when you hit the same key (unlike a violin, which requires skill just to extract a sound from it), irrespective of which finger you use or how close you get to the middle of the key.  This makes it ideal for a beginner who is going to play through a brute-force memorisation exercise. It was a triumph of perseverance and determination for both of us, but for Ben especially. Music is already one of his least favourite subjects at school (what other subject expects you to take the end-of-year test before teaching you the course material?).

I'm pleased and surprised at how I'm still learning, teaching and doing new things. I wish you a happy 2023 and many more new opportunities!

The Lists of Firsts

A first time for everything: 2018
2019 in reflection
First times in 2021 list
First times of 2022
First times in 2023
Things I did for the first time in 2024

The BBC Micro and sums of 2^n

Our first home computer was an Acorn Electron. Not as widely known as the Commodore 64 or the Sinclair Spectrum, the Electron is most easily described as a cut-down version of the BBC Micro (which was extremely common in schools).  The Electron ran BBC BASIC, which made it relatively user-friendly. There was a full BASIC manual in the box, along with a book containing some relatively simple games and programs, and using the two together it was relatively easy to enter and the customise the games.

In particular, it was possible to modify the games' graphics, and the character sprites.  Each component of the game (a gameplay character) was made either from a combination of simple ASCII characters >->  o<-< or from user-defined graphics.  And the way that you defined the sprites used some simple maths based on a grid and the powers of 2 (1, 2, 4, 8, 16, etc.).  The grid is 8x8 pixels, and each column has a value.  The sum of each row gives its overall value.

Here's an example:

In order to program one of the game sprites, you take each row one at a time. You identify which of the columns have been filled for that row, and then sum their values (based on the column heading).  The first row in the example above has the two central squares shaded, and these have the values 16 and 8; the sum is 24.  The next row has four squares shaded, and these are 32, 16, 8 and 4.  The next has the central six squares, where 64+32+16+8+4 = 126, and the fourth row has all eight squares, the sum is 255.

The heart shape has some new and some familiar combinations, but the same principle applies.

And this walking stick-man has an asymmetrical arrangement, but this is no problem as the same principles apply.

Admittedly I was 8-10 years old when I was programming these sprites, but it never occurred to me that each combination was represented ny a unique sum, and conversely, each total could only be shown by one combination of pixels.

Why? In base 10, each number can only be expressed as a unique sum of 10^0s, 10^1s, 10^2s... and so on.  We call these units, tens and hundreds, and each can hold a value between 0 and 9. So, 9 x 10^0 is 9, and if we want to show an additional unit, we have to add this as an increment to the next column, and have 1x10^1 + 0 x 10^0.  Each number can (and must) be represented by a unique combination of digits in the hundreds, tens and units columns.

The same applies in binary, base 2, which is the underlying basis of the pixel grid. Each pixel can be either 1 or 0, with 1 representing a pixel, and 0 representing a gap.  And every number can be represented by a unique combination of 1s and 0s - binary is just the same as decimal counting, its just the digits that are different!  This time, the values of each column aren't 10^0, 10^1, 10^2 and so on, but 2^0, 2^1, 2^2, 2^3, 2^4 and so on. 

Naturally, the Acorn Electron was also fluent in hexadecimal, but that's a whole other story.

Other similar posts that I've written

My Life in 10 Computer Games (including a few that I played on the Acorn Electron and BBC Micro)
Sums to Infinity and Refuelling Aircraft (based on a puzzle from a BBC Micro coding book)
And this post: The BBC Micro and Sums of 2^n

How many of your tests win?

 As November heads towards December, and the end of the calendar year approaches, we start the season of Annual Reviews.  It's time to identify, classify and quantify our successes and failures opportunities from 2022, and to look forward to 2023.  For a testing program, this usually involves the number of tests we've run, and how many recipes were involved; how much money we made and how many of our tests were winners.

If I ask you, I don't imagine you'd tell me, but consider for a moment:  how many of your tests typically win?  How many won this year?  Was it 50%?  Was it 75%?  Was it 90%?  And how does this reflect on your team's performance?

50% or less

It's probably best to frame this as 'avoiding revenue loss'.  Your company tested a new idea, and you prevented them from implementing it, thereby saving your company from losing a (potentially quantifiable) sum of money.  You were, I guess, trying some new ideas, and hopefully pushed the envelope - in the wrong direction, but it was probably worth a try.  Or maybe this shows that your business instincts are usually correct - you're only testing the edge cases.

Around 75%

If 75% of your tests are winning, then you're in a good position and probably able to start picking and choosing the tests that are implemented by your company.  You'll have happy stakeholders who can see the clear incremental revenue that you're providing, and who can see that they're having good ideas.

90% or more

If you're in this apparently enviable position, you are quite probably running tests that you shouldn't be.  You're probably providing an insurance policy for some very solid changes to your website; you're running tests that have such strong analytical support, clear user research or customer feedback behind them that they're just straightforward changes that should be made.  Either that, or your stakeholders are very lucky, or have very good intuition about the website.  No, seriously ;-)

Your win rate will be determined by the level of risk or innovation that your company are prepared to put into their tests.  Are you testing small changes, well-backed by clear analytics?  Should you be?  Or are you testing off-the-wall, game-changing, future-state, cutting edge designs that could revolutionise the online experience? 

I've said before that your test recipes should be significantly different from the current state - different enough to be easy to distinguish from control, and to give you a meaningful delta.  That's not to say that small changes are 'bad', but if you get a winner, it will probably take longer to see it.

Another thought:  the win rate is determined by the quality of the test ideas, and how adventurous the ideas are, and therefore the win rate is a measure of the teams who are driving the test ideas.  If your testing team is focused on test ideas and has strengths in web analytics and customer experience metrics, then your team will probably have a high win rate.  Conversely, if your team is responsible for the execution of test ideas which are produced by other teams, then a measure of test quality will be on execution, test timing, and quantity of the tests you run.  You can't attribute the test win rate (high or low) to a team who develop tests; in fact, the quality of the code is a much better KPI.

What is the optimal test win rate?  I'm not sure that there is one, but it will certainly reflect the character of your test program more than its performance. 

Is there a better metric to look at?   I would suggest "learning rate":  how many of your tests taught you something? How many of them had a strong, clearly-stated hypothesis that was able to drive your analysis of your test (winner or loser) and lead you to learn something about your website, your visitors, or both?  Did you learn something that you couldn't have identified through web analytics and path analysis?  Or did you just say, "It won", or "It lost" and leave it there?  Was the test recipe so complicated, or contain so many changes, that isolating variables and learning something was almost completely impossible?

Whatever you choose, make sure (as we do with our test analysis) that the metric matches the purpose, because 'what gets measured gets done'.

Similar posts I've written about online testing

Getting an online testing program off the ground
Building Momentum in Online testing
Testing vs Implementing Directly

What is the highest product of numbers that sum to 10?

 I'm always on the lookout for maths puzzles, especially ones which can be described simply but require some work to find the solution.  The most popular article on my blog is the one "Can you use the digits 1,2,3 and 4 to make the numbers from 1 to 50."  It's depressingly popular, when I have spent so much more time on many of the other posts here.  Ah, well, that's life.

A puzzle I found recently goes like this: "What is the highest product (i.e. multiply them all together) of numbers that sum to 10?"

Simple enough. 

I started with the pairs, 1+9, 2+8 and so on:

1 * 9 = 9
2 * 8 = 16
3 * 7 = 21
4 * 6 = 24
5 * 5 = 25

Well, there we are.  The answer so far is 25.  The highest value we can reach for x (10 - x) is when x = 5 and 5 * 5 = 25.  We can even prove this, using differention:

If y = x(10-x) = 10x - x^2  

then dy/dx = 10 - 2x 

And the maximum value is when dy/dx = 0 and if 10 - 2x = 0 then x = 5.  QED

However, there are multiple other ways of making 10 by summing numbers together; nobody said it had to be just two numbers.

For example, 4 + 4 + 2 = 10, and 4 * 4 * 2 = 32, which is higher than the 25 we achieved previously.

A few more examples:

2 * 4 * 2 * 2 = 32
2 * 3 * 5 = 30
3 * 3 * 4 = 36

The highest that can be achieved, it seems, is 36.

Let's try another number - let's try summing to 15, and see if we can find a pattern.

3 * 6 * 6 = 108
5 * 5 * 5 = 125
3 * 3 * 3 * 4 * 2 = 162
3 * 4 * 4 * 4 * 4 = 192
3 * 3 * 3 * 3 * 3 = 243

Summing to 15, the highest product we can achieve is 243.

Summing to 19 (if you fancy a challenge) is 972, where 3 * 3 * 3 * 3 * 3 * 4 = 972.

Proof (or general principle)

When forming the product with the highest value, the highest value can be achieved by using 2s and 3s.

Every larger number can be broken down to 2s and 3s which will multiply together to form a larger product.

For example, 9 can be be broken down to 5 and 4, which will multiply to form 20 (which is higher than 9).

9 --> 5 * 4 = 20

5 -->  3 * 2 = 6

9 --> 3 * 2 * 4 = 24

4 --> 2 * 2 = 4 (hence it does not matter if we choose a 4 or two 2s).

7 --> 3 * 4 = 12

6 --> 3 * 3 = 9

Any number n can be broken down to 2s and 3s which will multiply together to form a product which is greater than n.  Hence, the largest product we can obtain for 10 is 3 * 3 * 3 * 2 = 18, and the general principle is to break down the sum number into 3s (and 2s if the number is not a product of 3) to obtain the highest product.