Designing Personas for Design Prototypes

Part of my job is validating (i.e. testing and confirming) new designs for the website I work on.  We A/B test the current page against a new page, and confirm (or otherwise) that the new version is indeed better than what we have now.  It's often a last-stop measure before the new design is implemented globally, although it's not always a go/no-go decision.

The new design has gone through various other testing and validation first - a team of qualified user experience designers (UX)  and user interface designers (UI) will have decided how they want to improve the current experience.  They will have undertaken various trials with their designs, and will have built prototypes that will have been shown to user researchers; one of the key parts of the design process, somewhere near the beginning, is the development of user personas.

A persona in this context is a character that forms a 'typical user', who designers and product teams can keep in mind while they're discussing their new design.  They can point to Jane Doe and say, "Jane would like this," or, "Jane would probably click on this, because Jane is an expert user."

I sometimes play Chess in a similar way, when I play solo Chess or when I'm trying to analyze a game I'm playing.  I make a move, and then decide what my opponent would play.  I did this a lot when I was a beginner, learning to play (about 40 years ago) - if I move this piece, then he'll move that piece, and I'll move this piece, and I'll checkmate him in two moves!  This was exactly the thought process I would go through - making the best moves for me, and then guessing my opponent's next move.

It rarely worked out that way, though, when I played a real game.  Instead, my actual opponent would see my plans, make a clever move of his own and capture my key piece before I got chance to move it within range of his King.

Underestimating (or, to quote a phrase, misunderestimating) my opponent's thoughts and plans is a problem that's inherent with playing skill and strategy games like Chess.  In my head, my opponent can only play as well as I can. 

However, when I play solo, I can make as many moves as I like, but both sides can do whatever I like, and I can win because I constructed my opponent to follow the perfect sequence of moves to let me win.  And I can even fool myself into believing that I won because I had the better ideas and the best strategy.

And this is a common pitfall among Persona Designers. 

"Jane Doe is clever enough to scroll through the product specifications to find the compelling content that will answer all her questions."

"Joe Bloggs is a novice in buying jewellery for his wife, so he'll like all these pretty pictures of diamonds."

"John Doe is a novice buyer who wants a new phone and needs to read all this wonderful content that we've spent months writing and crafting."

This is something similar to the Texas Sharpshooter Fallacy (shooting bullets at the side of a barn, then painting the target around them to make the bullet holes look like they hit it).  That's all well and good, until you realize that the real customers who will spend real money purchasing items from our websites, have a very real target that's not determined by where we shoot our bullets.  We might even know the demographics of our customers, but even that doesn't mean we know what (or how) they think.  We certainly can't imbue our personas with characters and hold on to them as firmly as we do in the face of actual customer buying data that shows a different picture.  So what do we do?

"When the facts change, I change my mind. What do you do, sir?"
Paul Samuelson, Economist,1915-2009

My Longest Online Chess Game

 

While my search for my oldest recorded Chess game continues, I have digressed into finding my longest online Chess game.  This game was started on 25 September 2014, and again, I was Black.  Again, I faced e4 and replied ... e5.  I know this is very one-sided, but it's purely coincidental that I won this game too.  It's got a few interesting decisions throughout (I still think Chess is a game more than a logic puzzle) but a positive ending for me.  My opponent, New_Titanic, resigned, otherwise this long game could have been even longer.   

1. e4 e5
2. Nf3 Nc6
3. Bb5 a6

The way I usually play the Ruy Lopez as Black.  I'm happy to play 4. Bxf6 5. bxf6 Nxe5 6. Qd5 and then capture White's e-pawn.

4. Ba4 Nf6
5. d4 Nxe4
6. O-O b5

I'm not sure why I didn't play ... exd4, although I suspect that it would have lost to Bxf6, and hence I played ... b5 instead.  Not perfect, but playable.

7. Bb3 Bb7
8. dxe5 Be7
9. Qd5 Nxe5?

I don't think I was concentrating, because here is the first of my blunders.  Surely my move loses on the spot to Qxb7; even ...c6 is not going to be a sufficient response.  I'm not sure where my head was, because this was a poor series of moves from me.

10. Qxe5 O-O
11. Bd5 Bxd5
12. Qxd5 Nf6
13. Qd1 Ne4
14. Ne5 Bd6
15. Re1 Nxf2

I'm not sure if this was brave or stupid.  My thought was if Kxf2, then Qf6+ should win me the knight on e5.  Either way, I'm threatening the Queen which thought it was safe on d1.

16. Qd5 Qh4
17. Nf3 Bxh2+

I think I'm overcomplicating this, because Nxh2 is surely a safe move from white, leaving me material down.  Whichever it is, it seems like White is equally confused, and plays a safe move.

18. Kf1 Qg3
19. Nxh2 Nh1

Threatening checkmate, and moving my knight from the frying pan into the fire.

20. Qf3 Qxh2
21. Bf4 Qh4
22. Nd2 d6
23. g3 Nxg3+
24. Qxg3 Qh1+
25. Ke2 Qh5+
26. Qf3 Rfe8+
27. Kd1 Rxe1+

Forced, although I'd prefer not to trade (being a piece down).  My Queen is still en prise and my rook isn't looking great on e8.  My complications have got the better of me and I'm in trouble.  I'm not even well-developed.

28. Kxe1 Qh4+
29. Kd1 Re8

Two pieces down and trying to complicate as much as possible.  This is another transparent checkmate threat.

30. Bg3 Qd4
31. Kc1 Qg1+
32. Nf1 h6
33. Kb1 Re6
34. Bh4 Qd4

Re-centralising my Queen.  If White plays Qa8+ he will need to be aware of my possible move Qd1#.  I'm also threatening his Bishop on h4, so while I don't have much to do, I'm making the most of it.

35. Bf2 Qf6?
36. Qa8+ Kh7
37. Qg2? Re2!
38. Bh4 Rxg2
39. Bxf6 Rg1
40. b3? Rxf1+

Winning back all the material after two White blunders.  I'm now four pawns up, including three unopposed passed pawns on the kingside.  The long-term view (for me) is very positive; all I need to do know is shepherd them home.

41. Kb2 Rxf6
42. a4 b4
43. Re1 Re6
44. Rf1 f6
45. Rf4 c5
46. a5 g5
47. Rf1 h5
48. Rf2 h4
49. c3 bxc3+
50. Kxc3 Kg6
51. Kc4 Re5
52. Rd2 Re4+
53. Kc3 Rd4
54. Rh2 f5
55. Re2 Kf6
56. Re8 h3
57. Rf8+ Ke5
58. Re8+ Kf4
59. Kc2 h2
60. Rh8 Kg3
61. Rh5 Rh4

Blundering to the end.  Surely Rg4 was better, protecting the g-pawn.  But no, not me - I'm going to play this game my way!

62. Rxg5+ Kh3
63. Rxf5 h1=Q

Brute force.  Effective, but lengthy.

64. Kd2 Rd4+
65. Ke3 Qe4+ 

Centralised Rook and Queen against an isolated and defenceless King?  And I'm about to pick up White's Rook for good measure.

0-1

My Very Earliest Online Chess Game

 The search continues.  I thought my previous post on my earliest online Chess game was the earliest, at 2020.  I've beaten that, with a Chess game from 2014.  Given that my blog was launched in 2010, I think I might have to search even harder.

Anyway, here's my first Chess game on Chess.com, and this one is another long-term game, with a move every three days.

I played Black, against fergosa, on 20 June 2014.

1. e4 e5
2. Bc4 Nf6

It was a recurring theme of my earliest online Chess games that my opponents would try to mate me as quickly as possible, and would therefore play ill-advised or reckless moves.  Meanwhile, I would play the sensible replies and gain an advantage from a more stable position.
  
3. d3 Nc6
4. Bg5 Be7
5. Bxf6 gxf6

No, it's not great to double my pawns, and I'm sure this would be criticised by all the experts, but I want to open the g file in case my opponent castles short.

6. Qh5 O-O
7. Qg4+ Kh8

Ok, he got a check, but my king is safe in the corner and I've vacated the g-file for my rook.

8. Qd1 f5
9. exf5 d5

Well, goodbye to my doubled pawns, and hello to an activated bishop and a pawn break in the centre.

10. Bb5 Nd4
11. Ba4 Bxf5
12. Nf3 Bg4
13. Nbd2 e4

"Always attack pinned pieces."

14. h3 Bxf3
15. Nxf3 exf3

I wonder if I could have played this better, but in the end I shattered white's kingside pawns - before he can castle short - and was a knight up.  And it's an advanced knight, in the middle of the board.

16. gxf3 c6
17. O-O Qd7

He castled kingside?  Surely the end is nigh!  I played c6 to shore up my pawn, and also to give me the d7 square for my queen.  I'm now eyeing h3.

18. c3? Qxh3

Surely 18. Kh2 was better?  Oh well...

19. b3? Rg8# 

Fiddling with queenside pawns while I finally get to place my rook on the g8 square.  And deliver checkmate!

Not perfect, but a lot of fun! 

My Earliest Online Chess Game

 Way, way back in the dim and distant past, this blog was launched and called, "Dave's Chess Games", and was going to exclusively feature my Chess games - instructive, successful, disastrous, educational and bizarre.  I soon realised, however, that Chess blogs are two-a-penny and written by Chess players with far more skill in the game than me.  The blog was retitled Chess, Opinions, Maths and Puzzles; then it was changed to Web Analytics, Maths and Puzzles.  Looking at recent content, it could probably be called, "What is he going to write about next?" with a focus on mathematical, analytical and numerical content, and some science fiction thrown in for good measure.  Who knows?

What I do know, though, is that I've been playing Chess for a while, and playing online for about 10 years, starting on the Google Chess app, back when we had Google homepages. 

This may not be the oldest game I've played, but it's certainly a start: 7 September 2020.

The time control for this game is one move every three days, so blunders are to be criticised!

I was playing direct Chess back then, and this is what happened:  I was Black.

1.e4 e5
2.Bc4 Nf6 

All fairly standard.  I'm not going to let White play Qh5 and start making my life difficult.

3. d3 Be7

 4. Nf3 d6
5. O-O Bg4
6. h3 Bh5 

If and when White plays g4, I will almost certainly sacrifice one of my minor pieces to open up the files in front of his king.  I will then also advance my g and h pawns.  We'll see.

7. Nc3 Nc6
8. Be3 Qd7
9. Qd2 g5
10. Nxg5 Rg8
11. g4 

So, White has advanced his g pawn.  I've placed my queen on the c8-h3 diagonal, so it seems like the ideal time to sacrifice (especially since I have also place my rook on the g file).  I suspect this was as serious blunder by White - what else would I do?

11. ...   Nxg4
12. hxg4 Qxg4+
13. Kh2 Qh4+
14. Nh3 Bg4
15. Ne2 Qxh3+

There goes White's h pawn.

16. Kg1 Bf3+
17. Ng3 Qg2# 0-1

Whenever you join a new online Chess club or site, and start off as an unranked player, it seems like everybody wants to try and mate you as quickly as possible.  You're fresh meat and you're probably not very good (unlike the long-standing mediocre players you're about to face).  So a good plan against 1. e4 e5 2. Bc4 is probably a good idea.  You can play the Sicilian (1 ... c5) or like me, play Nf6.  That should slow them down a bit. 

Film Review: Transformers One

 NO REAL SPOILERS

I was there, back in 1984, reading the comics featuring Optimus Prime and Megatron.  I started with Issue 4 of the UK run (the second half of US #2) and that was origin enough for me - the Autobots and Decepticons had arrived on Earth and had the ability to transform.  I could work the rest out backwards.

Origin stories generally have the disadvantage that you know how they turn out (we know the characters from stories we've read or seen previously).  In the case of Transformers, the history of Optimus Prime has been covered in the cartoon series, in the episode "War Dawn", where we meet Orion Pax and Elita One.

With such a depth and wealth of history, how is the latest Transformers film, simply titled, "Transformers One" going to manage to tell a new story?  Partly by rewriting canon, and partly by focusing on the characters - this is the origin story for Megatron (and by changing him from a gladiator to a miner) more than for Optimus Prime.

Orion Pax (precursor to Optimus Prime) is written with a youthful and naive optimism throughout.  D-16 (precursor to Megatron) has a similar personality, at least initially.  Bumblebee is annoying, but far better written that in any of the other Michael Bay films (I was amazed to discover that Mr Bay had got his name on this film, there's far more character development and fewer explosions than a typical Bay outing).

D-16's character arc is far more interesting than Orion's - the two characters start as friends, and in order to become enemies, one of them has to go in a different direction to the other: it's D-16 who changes course.  Orion's development to Optimus is simply due to his relentlessly pursuing 'the right thing'; D-16 is more interested in forging a different path and doing things differently.  This is partly ironic, because Orion is the one who 'breaks protocol' in order to save one of his fellow miners near the start of the film.  

There are some key plot developments around halfway through the story, and the two characters respond differently to them.  This sets D-16 on a tangential course to Orion, and the further the story develops, the further D-16 moves away from Orion.  The plot also takes over - D-16 is captured by the enemy at one point, and his experience while he's separated from Orion change him further.  As I said, this is a character story and keeping the focus on a smaller number of characters works well - because they have such distinct personalities.

This doesn't mean that the planet Cybertron is uninhabited, or populated only by faceless drones.  On the contrary, you could rewatch this film and keep finding well-known G1 characters in the backgrounds of all the well-populated scenes.  G1 fans will thoroughly enjoy tracking them all down (for the record, I spotted Jazz, Brawn, Red Alert, Sideswipe, Ratchet, Wheeljack and some of the Coneheads while watching for the first time in the cinema today).

And when you've counted all the characters, listen out for the references:  I picked up, "You don't have touch and you don't have the power!"; "Transform and Roll Out" and "You're on the surface with these idiot Gobots!".

And then there are the visual references - my favourite was close to the end, where Optimus leaps into the air to attack the 'baddies'.  The film goes into slow-mo to make sure it's not missed, but if you ever saw Transformers The Movie, you'll recognise this sequence:

This is the best Transformers film since Dark of the Moon (2011), and possibly since the Transformers (2007).  It's aimed towards children and teenagers (and benefits significantly from not having the hormone-addled Sam Witwicky and his thoroughly annoying parents), and still has plenty to offer long-time Transformers fans who are the parents of those children and teenagers.  I took my teenage son to watch it and we both thought it was amazing, for very different reasons (although the first appearance of Megatron's weapon had us both wide-eyed and amazed).

My Life in 10 Computer Games

During lockdown, I participated in the social media trend of posting lists of ten, one a day for ten days:  there was the list of ten albums, and the list of ten computer games.  They were posted at the time without any explanation, but now (a few years later, and now that it's the summer holidays) here is my list of ten computer games that have taken up more hours of my leisure time than they should have done --- or perhaps they're just the ones I like the most:

1.  Snapper

The first computer game we bought (as a family), on our first home computer: the Acorn Electron.  We programmed a number of games but couldn't save them as we had our Electron several weeks before we had a cassette recorder.  Once we had the tape recorder, we could save our programs and - more importantly - load purchased games as well.  Snapper is a straight rip-off of Pacman, with a few cosmetic changes, and we spent far too many hours on it.  

2. Around the World in Forty Screens (Repton 3)

 

In the 1980s and 90s, Repton was not widely known outside Electron and BBC Micro owners, which suprised me because for those platforms it was a seminal series.  I didn't play Repton or Repton 2 until years after they were released, but Repton 3 and the various spin-offs were high on our playlist, and in our house, that meant Around the World in Forty Screens.  Repton 3 is a 2D scrolling maze game, where you have to collect diamonds, kill monsters and avoid getting trapped or hit by falling rocks. 

The main aspect of Repton 3, though, was the ability to edit the characters and the maps.  You could load up the existing levels and change them, or adapt the default sprites and devise your own maps.  And we spent weeks and weeks and weeks doing this: we had Repton in Space.  This was in the era before discs and before mice, so every character or level, or both, had to be edited with the cursor keys and the space bar, and then saved to cassette.  Then you had to restart the system, load the game, load your new levels, and playtest them.  No wonder it took us weeks - but creating the levels was almost as addictive as playing the existing ones, and that's why Repton is in this list.

3. F-29 Retaliator

Eventually, we moved from the Acorn Electron to a PC.  Our first PC was a 286 with 640KB RAM, and we set about upgrading it - and the first thing was the monitor.  A colour monitor and a 1.44MB disc drive opened up enormous possibilities, including the option to play two or three different games in an hour (no more tape cassettes).  I've listed F-29 Retaliator here as it was the first game we played head-to-head:  with a serial cable, it was possible to connect two PCs to each other and play against another human being on another PC (in an adjoining room).  This was earth-shattering at the time, and that's why F-29 is on this list - my brother and I used to demonstrate this tech to any visitors, friends or family who came to watch.  And it was our first decent flight sim.

4. Command and Conquer

I received this for Christmas 1995 (my choice) in the big-box version shortly after it released and it too expanded my game-playing horizons.  I had been playing Dune 2 (its precursor) for months beforehand, and had read about the new Command and Conquer in magazines - I bought it without trying a demo. Full motion video.  A campaign that made sense.  Good in-game visuals and sound effects (especially the sound effects and voice effects).  And the ability to play head-to-head against my brother (who was Nod, and was always better than me).  It defined the genre, and I was there at the start (although I never finished either campaign).  I bought all the follow-ups, including the First Decade, and finished C&C3 (as GDI), but I still enjoy the original.

5.  Worms

This is straightforward comedy - playing computer games for fun.  After F-29 and Command and Conquer, Worms took us back to sharing a computer and taking it in turns to play, but what a crazy game.  My abiding memory of my 19th birthday is playing Worms with my parents and brother in my university room.  And then, several years later, introducing it to my kids, who love playing it with me (including the time that one of my sons deliberately took my turn and walked one of my worms straight into the sea!).  Light-hearted and crazy fun - as gaming should be.  And for me, the sequels became too complicated - the original was the best.

6.  Dark Reign: The Future of War

Command and Conquer spawned a whole range of copies and clones, and defined the Real-Time Strategy (RTS) genre.  Some were reasonable, some were clunky, and some - like Dark Reign - were actually very good.  This one had a unique (at the time) approach to the fog of war, and had a wider range of units than most games.  The graphics were a step-change higher than the original C&C, and the animation was also much smoother.  I only ever played it single-player, but it did have a long-standing online community.  

7.  Wii Sports

This is where the chronological ordering breaks down, but in around 2008 we (as a young married couple) invested in a Wii.  I'm not sure why, but we did, and we had a lot of fun with it.  You might call it a team-building exercise, or you could just call it entertainment.  We specialised in Wii Sports, and were particularly good at doubles tennis.  Nowadays, it's bowling against my two sons, and two-against-one (them against me) at doubles tennis.  They also enjoy the sword-fighting in Wii Sports Resort, but that's a whole separate game!

8. Lego Star Wars PC (2006)

Not to be confused with the various spin-offs and sequels, this was (again) the original Star Wars Lego game for PC, and I played it on PC first.  Before it was playable on any of the other platforms with their optmized-for-gaming controllers.  Yes, I've played those versions, and they are just as funny - and as always, better played with two or more people.  I never completed it (it's a long-standing trend, by the looks of it) because I got stuck with a puzzle with R2D2 who wouldn't stand where he was supposed to.  This was before YouTube really took off, and it occurred to me to watch videos of how to win the game - and the written instructions in the walkthroughs said, "Solve the problem with R2."  Yes?  How?  With the Force?  Fun, but frustrating - and then a decade later, my son solved the same game on PS3.

  

9.  Lego Jurassic World PS3

I'm probably breaking my own rules here, because I've not played this game - but it has had an impact on my life (courtesy of my son, who spent most of Christmas Day 2017 playing this on our lounge TV).  He was heavily into Jurassic World and Lego, and this was the perfect intersection.  He had the sense and initiative to consult online support when he got stuck, and he finished the game by the end of the Christmas holidays.  He then went on to work his way through the Lego Star Wars games as well.  The games are well-designed, have cute graphics and clever stories, and have lasting appeal as you try to get all the points, all the bonuses and all the characters.

I think he let me play it once! :-)

10.  Zenon  (Acorn Electron cassette)

I'm not sure how I missed this one out on my first run through, because I think this is the first co-operative game we (my brother and I) played as kids.  Zenon was a sideways-scrolling shoot-'em-up game, where you had to run to the end of each level, destroying as many aliens as possible along the way - without getting hit by them.  It was all standard arcade action, but with the (then) added twist that two players could play co-operatively at the same time.

It made my Christmas list in the late 1980s or early 1990s, and had the advantage of being only half the price of a 'normal' computer game at the time - Impact Software were onto a good thing.  The review of the game mentioned the two-player option, and that's what swung the decision for me.

Gameplay was simple and straightforward; each player lost a life if they lost all their health through collisions with the aliens or taking blaster fire from them.  There was a wide variety of power-ups to collect, including health and weapons, and if both players got the best weapon (the red laser) then they could do considerable damage to the aliens very quickly.

Now:  we completed this game.  There were around 200 levels (I can't recall), and my brother sat down early one morning, loaded the game (via cassette) and blasted our way through all of them.  We even saw the 'Congratulations screen'. But - to my frustration to this day - it came with a 'press any key to continue' so we saw the message, but because the level ended and the message showed so abruptly, we were both still pressing our gaming keys - run, jump, fire, whatever, so the message appeared for a second and then we were returned to the main menu - Level One.  Come on, game designers, do better!

Anyway, we stopped playing after that.  It's a shame really, because it was one of the best games on the Electron at the time - the graphics were above average but the gameplay set it apart (in two player).  Yes it was difficult, yes there was a high element of randomness in the game and no, there weren't any patterns - it was just run and gun.

And that's my list of 'my life in ten computer games', covering the mid 1980s to the late 2010s.  I haven't included Star Wars Galactic Battlegrounds; Star Wars Battlefront 1 and 2, or Fortnite.  Maybe another time...

How not to Segment Test Data

 Segmenting Test Data Intelligently

Sometimes, a simple 'did it win?' will provide your testing stakeholders with the answer they need. Yes, conversion was up by 5% and we sold more products than usual, so the test recipe was clearly the winner.  However, I have noticed that this simple summary is rarely enough to draw a test analysis to a close.  There are questions about 'did more people click on the new feature?' and 'did we see better performance from people who saw the new banner?'.  There are questions about pathing ('why did more people go to the search bar instead of going to checkout?') and there are questions about these users.  Then we can also provide all the in-built data segments from the testing tool itself.  Whichever tool you use, I am confident it will have new vs return users; users by geographic region; users by traffic source; by landing page; by search term... any way of segmenting your normal website traffic data can be unleashed onto your test data and fill up those slides with pie charts and tables.

After all, segmentation is key, right?  All those out-of-the-box segments are there in the tool because they're useful and can provide insight.

Well, I would argue that while they can provide more analysis, I'm not sure about more insights (as I wrote several years ago).  And I strongly suspect that the out-of-the-box segments are there because they were easy to define and apply back when website analytics was new.  Nowadays, they're there because they've always been there,  and because managers who were there at the dawn of the World Wide Web have come to know and love them (even if they're useless.  The metrics, not the managers).

Does it really help to know that users who came to your site from Bing performed better in Recipe B versus Recipe A?  Well, it might - if the traffic profile during the test run was typical for your site.  If it is, then go ahead and target Recipe B for users who came from Bing.  And please ask your data why the traffic from Bing so clearly preferred Recipe B (don't just leave it at that).

Visitors from Bing performed better in Recipe B?  So what?

Is it useful to know that return users performed better in Recipe C compared to Recipe A?

Not if most of your users make a purchase on their first visit:  they browse the comparison sites, the expert review sites and they even look on eBay, and then they come to your site and buy on their first visit.  So what if Recipe C was better for return users?  Most of your users purchase on their first visit, and what you're seeing is a long-tail effect with a law of diminishing returns.  And don't let the argument that 'All new users become return users eventually' sway you.  Some new users just don't come back - they give up and don't try again.  In a competitive marketplace where speed, efficiency and ease-of-use are now basic requirements instead of luxuries, if your site doesn't work on the first visit, then very few users will come back - they'll find somewhere easier instead.  

And, and, and:  if return users perform better, then why?  Is it because they've had to adjust to your new and unwieldy design?  Did they give up on their first visit, but decide to persevere with it and come back for more punishment because the offer was better and worth the extra effort?  This is hardly a compelling argument for implementing Recipe C.  (Alternatively, if you operate a subscription model, and your whole website is designed and built for regular return visitors, you might be on to something).  It depends on the size of the segments.  If a tiny fraction of your traffic performed better, then that's not really helpful.  If a large section of your traffic - a consistent, steady source of traffic - performed better, then that's worth looking at.

So - how do we segment the data intelligently?

It comes back to those questions that our stakeholders ask us: "How many people clicked?" and "What happened to the people who clicked, and those who didn't?"  These are the questions that are rarely answered with out-of-the-box segments.  "Show me what happened to the people who clicked and those who didn't" leads to answers like, "We should make this feature more visible because people who clicked it converted at a 5% higher rate." You might get the answer that, "This feature gained a very high click rate, but made no impact [or had a negative effect] on conversion." This isn't a feature: it's a distraction, or worse, a roadblock.

The best result is, "People who clicked on this feature spent 10% more than those who didn't."

And - this is more challenging but also more insightful - what about people who SAW the new feature, but didn't click?  We get so hung up on measuring clicks (because clicks are the currency of online commerce) that we forget that people don't read with their mouse button.  Just because somebody didn't click on the message doesn't mean they didn't see it: they saw it and thought, "Not interesting," "not relevant" or "Okay, that's good to know but I don't need to learn more".  The message that says, "10% off with coupon code SAVETEN - Click here for more" doesn't NEED to be clicked.  And ask yourself "Why?" - why are they clicking, why aren't they?  Does your message convey sufficient information without further clicking, or is it just a headline that introduces further important content.  People will rarely click Terms and Conditions links, after all, but they will have seen the link.

We forget that people don't read with their mouse button.

So we're going to need to have a better understanding of impressions (views) - and not just at a page level, but at an element level.  Yes, we all love to have our messages, features and widgets at the top of the page, in what my high school Maths teacher called "Flashing Red Ink".  However, we also have to understand that it may have to be below the fold, and there, we will need to get a better measure of how many people actually scrolled far enough to see the message - and then determine performance for those people.  Fortunately, there's an abundance of tools that do this; unfortunately, we may have to do some extra work to get our numerators and denominators to align.  Clicks may be currency, but they don't pay the bills.

So:  segmentation - yes.  Lazy segmentation - no.