Niamh tag

Wednesday, 1 July 2015

Online Optimisation is a Game of Chess

It occurred to me recently that there are some similarities between playing a game of Chess and optimising a website through testing (apart from the considerable differences between them).

 Many years ago, when I started this blog, I envisaged it as a place where I would type up my chess games, with commentary, lessons and points for improvement.  A quick glance through my archive of blog posts will quickly show that the blog really hasn't matched its original plan.  It evolved and changed, in particular in 2011, when I shared my first few posts about web analytics and I realised that I could attract significantly more readers by blogging and sharing about web analytics than I have ever managed with Chess.  I figure that Chess is a much more mature subject, with many more people who are considerably more experienced than me; whereas web analytics (and especially my main area of interest, testing) is a much younger field, and there's scope for sharing ideas and experiences which are still novel and interesting.  However, there are some similarities between the two - here are a few thoughts.

Strategies and Tactics

Black prepares a tactic.
Plans in Chess can be broadly separated into strategies (long-term aims) and tactics (short term two-or-three move plans).  The long-term aim is to checkmate your opponent's king, and throughout the game that will become the more prominent goal.  In the short term, as you progress through the opening moves, you'll identify potential opportunities to capture your opponent's pieces, to put your pieces on good squares and to restrict your opponent's chances of beating you.  Some of these are short term, some are long term.  For example, it may be possible to win one of your opponent's bishops with a cunning trap (if your opponent is not vigilant).  This will make it easier to achieve your long-term goal of winning the game by checkmating your opponent.

Testing offers the same range of opportunities:  are you looking for quick wins (who isn't?) or are you planning to redesign an entire page, or even an entire site?  Will you be implementing wins as soon as you have confirmed results, or are you going to iterate and try to do even better?  Are you compiling wins to launch in one big bang, or are you testing, implementing and then repeating?  How far ahead are you planning?  Neither approach is necessarily better, as long as you're planning, and everybody is agreed on the plan!

Aims and Goals

White threatens checkmate.
As I've just mentioned, in Chess there's a clear goal:  checkmate your opponent's king, by threatening to capture his king and leaving him with no way to escape.  The aim for your online testing program may not be so clear cut, and if it isn't, then it might be time to find one single aim for it.  You might call it the mission statement for your optimisation program, but whatever you call it, it's important that everybody who's involved in your program understands the purpose of the testing team.  The same applies to each test, as I've mentioned before - each test should have a clear aim, that everybody understands and that can be measured in some clearly-defined way.

Values and KPIs

Each piece in Chess has a certain nominal value; the values aren't precise, but they provide a meaningful comparison of the strength and ability of each piece.  For example, the rook is worth five pawns, the knight and bishop are worth three pawns each, and the queen is worth nine.  This enables players to quickly evaluate a position in a game and say which player is winning, or if the position is roughly equal.  It's slightly more complicated than that, as you have to take into consideration the position of the pieces and so on, but a quick comparison of the total material value that each player has will give a good idea of who's winning.

Rooks are worth five pawns; bishops are worth three pawns and queens are worth nine.
This also means that it's possible to determine if a plan or a strategy is likely to win and is worth pursuing.  It may be possible to trap your opponent's rook (worth five pawns), but if doing so will mean losing two knights (each worth three pawns) and a pawn, then the trap is not really beneficial to you.  If, on the other hand, you could trap your opponent's king (winning the game) at the cost of two rooks and a knight, then that's definitely worth doing.

The key performance indicator for a game of Chess is your opponent's king, and if you can measure how close you are to capturing (or checkmating) your opponent's king, then you can see how close you are to winning the game.  You also need to keep your own king safe, but that's where the analogy breaks down :-)

White wins despite less material.
If online testing, your plan, strategy and tests each need to have KPIs.  Once you've established your long-term aim, you can set KPIs against each test which are connected to achieving the long-term aim.  If you want to improve the return on investment (ROI) of your online marketing, you could look at the landing page... improve the bounce rate and the exit rate, and encourage more people to move further into your site and view your products.  Alternatively, you could look at improving conversion of visitors from cart (basket) to checkout.  Or perhaps the flow of visitors through your checkout process.  Providing you can tie each of your tests and your tactics to the overall strategy: "Improve ROI for online marketing" then you can measure whether or not it's succeeding.

Classifying your KPIs in order of importance is also important - as we saw in Chess, if you can win your opponent's pieces but lose several of yours in the process, then it's probably not a good idea.  In testing, what would you do if your test recipe had a worse bounce rate but higher overall conversion (a situation that's not impossible)?  Which is the more important metric - conversion, or bounce rate?  Would your answer be the same if it was improved conversion but lower revenue (people not spending as much per order)?  Are you going to capture your opponent's knight but lose your queen?

Win, lose or draw?

Queen takes King: checkmate!
In Chess, there are clear rules that determine the outcome of a game.  Either one player wins (so the other loses), or it's a draw, and there are various ways of drawing:  including by agreement (both players decide neither can win); by stalemate (one player cannot make any legal moves) or a drawn position where it's clear that neither player has enough material to checkmate the other.

I hate losing at Chess.  However, the truth be told, I'm not much above average as a Chess player, and I'm the weakest player at my club (this has not deterred me, and I still play for fun).  This means that I get plenty of chances to analyse my losses, and see where I could improve.  Do I make the same mistakes in future games?  Not usually, no.  

However, I'm not satisfied to stay as the weakest player in my club - I just see this as an opportunity to do some giant-slaying in my future matches.  I read books, I visit Chess websites, I practice against other people and against computers, but I especially review my own games.  Occasionally, I win.  And do I analyse my winning games?  Absolutely - I may have already seen during the game that my opponent missed a chance to beat me, but did I also miss a chance to win more easily?

In online optimisation, the rules for calling a test a win, lose or draw are still up for debate, and they vary between companies.  And so they should.  Each company will have its own testing program with its own tactics and strategies, and its own requirements.  Do you want to have 99.9% confidence that a test will win, or do you want some directional test data to support something you already believed based on other data sources?  How quickly do you want to run the next test, or implement the winner?  Providing that the rules for calling the win, lose or draw are agreed in advance, I might even suggest that they could vary between tests. This is, of course, totally different from Chess, where the centuries-old rules of the game clearly state the requirements for a win or a draw.  Otherwise though, I think it's fair to say that KPIs, metrics and strategy have their approximate equivalents in pieces, pawns and plans - and that thinking and planning are definitely the way forward!

Chess cartoons taken from the 1971 printing of  Chess for Children, originally published 1960.

Wednesday, 24 June 2015

Mathematical Film Reviews

Here are some simple film reviews, expressed in mathematical form.  If you've seen some of the older films (or read the books), then you won't need to watch the newer ones (or perhaps they'll help you decide if you'd like to watch them).

Equilibrium = Fahrenheit 451 + 1984 + The Matrix

After Earth = Oblivion + Will Smith's son

Noah = Transformers + Merlin + Titanic -Bible

Live Die Repeat = Mission Impossible + Groundhog Day + 12:01 + Starship Troopers

White House Down = Olympus has Fallen

Earth to Echo = ET + Close Encounters + Batteries Not Included

Lucy = Limitless + Scarlett Johansson

The Philadelphia Experiment = Quantum Leap + Titanic + Time Tunnel

Looper = Back to the Future + Die Hard

X-Men: Days of Future Past = Back to the Future + The Incredibles

Guardians of the Galaxy = The Avengers Assemble + Star Wars IV

I'm not saying that all the films in Hollywood are derivative (in fact there are some very original films out there), I'm just pointing out some recent similarities I've found.  To be honest, many of the films I've mentioned here are actually favourites of mine.

On the subject of very original films, I'd like to share a brief word about 2001 A Space Odyssey.  It's widely regarded as a genre-defining ground-breaking film, so you might be surprised to learn that even though I'm a sci-fi fan, I didn't watch it until 2015 (even though it was released in 1968, years before I was born).

I don't know why I hadn't watched it until now.  Now, I wish I hadn't bothered.

For example, if you just want to watch a computer play chess, then watch Kasparov and the Machine.

If you like model spaceships and spacecraft, go to a hobby modeller shop
If you like actual spacecraft, watch the footage of the Sputnik launch and the early Apollo missions.
If you like listening to the Blue Danube Waltz, buy it on CD (or tape, or even vinyl).

If you like a murder mystery, go and watch (or read) Agatha Christie or Arthur Conan Doyle
If you like stories with a bizarre ending, watch The Italian Job or Planet of the Apes
If you like long drawn-out stories, watch Les Miserables (released 1925, at 5hr 59 mins), or the 1927 film Napoleon (at 5hr 30 mins).

All better in comparison, and all better for watching.  2001 A Space Odyssey's only claim to fame is that it did a lot of things first.  It's just a shame that it made so many mistakes at the same time, and while doing so many things first, it forgot about the basics like story-telling, pace, narrative, dialogue and just having things make sense.

Thursday, 18 June 2015

Can a 747 take off from a conveyor belt runway?

This question is currently going round on Facebook; one of my friends posted it, I answered it, and a few hours later entered a lengthy and circular debate in the comments section.  Here, with more space than a Facebook comments section, I'd like to pose the question, answer it and then address some of the misconceptions.

Here's the question:  Imagine a 747 is sitting on a conveyor belt, as wide and long as a runway.  The conveyor belt is design to exactly match the speed of the wheels, moving in the opposite drection.  Can the plane take off?
Shared originally by Aviwxchasers.Com a news and media site from the US.

My friend shared the post, and answered, "No - there's no way to generate lift."

My original answer:  

"I think you're assuming that the 747 will be moving because its wheels are being driven. The forward thrust of the aircraft comes from its engines, not from making its wheels turn (like a car). So the engines push the aircraft forward and the wheels just slide and skid along the conveyor belt , while the engines push the aircraft to take-off speed. The wheels don't have to turn to allow the aircraft to move."

However, this wasn't deemed sufficient by some other commentators, who (to summarise) posted the following questions or objections.

1. The conveyer would counteract the forward movement produced by the engines' thrust.  The conveyor matches the speed of the wheels in the opposite direction, so there can be no forward motion.

2. Lift is created by air flow over the wings. It doesn't matter how much thrust you have , if doesn't generate airflow over the wing, it won't fly.

3. The method of propulsion should be irrelevant. If the speed of the conveyor matches the speed of the wheels in the opposite direction, there can be no forward motion.

4. Take the vehicle out of the equation, it's all about the wheels and the conveyor. The faster the wheels turn, the faster the conveyor goes. Stick anything you want on top of the wheels, the principles are the same. Newton's third law
5. Indeed it's not about the wheels it's about forward movement (thrust from the engine) needed to take off but that forward movement is being counteracted by the conveyer    

Here, I propose to look at these arguments individually and collectively, and explain why the points which are raised aren't sufficient to stop the aircraft from moving and taking off.

1.  The conveyor would counteract the forward movement produced by the engines' thrust.  The conveyoy matches the speed of the wheels in the opposite direction, so there can be no forward motion.

1A.  The conveyor will only stop the wheels' rotation from generating forward movement.  However, it is not the wheels which are being driven - this is not a car or a truck.  The forward movement of the plane is not a result of the wheels successfully pushing against the conveyor belt; the forward movement of the plane comes from the push of the engines.  The result is that the plane will move forwards (the engine pushes against the air flowing through it, the air pushes back - Newton's Third Law) and the wheels will spin and skid down the track.

It's not "The plane moves forwards because the wheels go round", it's, "The wheels go round because the plane goes forwards [over a surface which has sufficient friction] ."

2. Lift is created by air flow over the wings. It doesn't matter how much thrust you have , if doesn't generate airflow over the wing, it won't fly.

True.  But there is thrust, and it is generating airflow over the wing.  The conveyor belt does not have the ability to resist the movement of the plane, only to counteract the turning of the wheels.  

The plane can move forwards -even along the ground - without its wheels turning.  The engine isn't driving the wheels.

3.  The method of propulsion should be irrelevant. If the speed of the conveyor matches the speed of the wheels in the opposite direction, there can be no forward motion.
4. Take the vehicle out of the equation, it's all about the wheels and the conveyor. The faster the wheels turn, the faster the conveyor goes. Stick anything you want on top of the wheels, the principles are the same. Newton's third law

3A and 4A.  Comment 3 was a response to my question, "What happens if we replace the 747 with a space rocket, aligned horizontally on the conveyor belt runway, on wheels?"  and the comment suggests a misunderstanding about what's causing the forward motion of the plane.  The thrust of a space rocket is completely independent of the any wheels (they normally fly fine without them) and the wheels will just get dragged along the conveyor belt, without turning. 

Yes, the faster the wheels turn, the faster the conveyor belt goes.  But the wheels don't have to turn for the plane to move (as I said in response to 2).  The conveyor belt does not have some property which prevents something from skidding along it, just from preventing any forward motion due to wheels turning on it. 

You can only take the vehicle out of the equation when you realise that the vehicle isn't a car, with the limitations that a car has.

Newton's third law applies to rotating wheels and conveyor belts.  It also applies to the aircraft engines and the air flowing through them, or to space rockets and the fuel burning inside them.  I wonder if the Starship Enterprise (on wheels) would have this problem?

5.  It's all about forward movement (thrust from the engine) needed to take off but that forward movement is being counteracted by the conveyer  

The forward movement is not counteracted by the conveyor.  The conveyor just stops rotating wheels from generating any forward movement.  But if the wheels aren't rotating (because they're not being driven, such as in a car) then there's no movement to resist.

In conclusion, I'd like to offer this video from "Mythbusters" which shows a light aircraft attempting to take off against a conveyor belt (which in this case is being pulled by a pickup truck to match the plane's speed).  To quote one of the engineers, "People just can't wrap their heads around the fact that the plane's engine drives the propellor, not the wheels."

Thursday, 4 June 2015

Numbers from 1,2,3,4 to 50

As an occasional diversion from work, I like to try maths puzzles, and to this end, I recently purchased a couple of high school maths textbooks (for use with students aged from about 14 to 16).  Not that I can't do the puzzles (I think my maths is okay), but because sometimes the questions and puzzles in them some interesting ideas about extension activities - for example, the recent question about the circle in the corner of a circle and a square is probably intended to be solved with trigonometry, and instead, I solved it in terms of square roots, using just Pythagoras' theorem.  That led to me solving the situation for the circle in the corner of a hexagon (which wasn't in the textbook, but which had an interesting solution too).

Anyway, I found a great question in the extension section at the end of one of the textbooks, and it goes like this:

Creating Numbers: a task requiring imagination

Your task is to create every number from 1 to 50.
You can use only the digits 1, 2, 3 and 4 once in each and the operations + - * / . 
You can use the digits as powers, and you must use all of the digits 1, 2, 3, 4

Here are some examples:

1 = (4-3) / (2-1)
20 = 42 +3 +1
68 = 34 * 2 * 1
75 = (4+1)2 * 3

So, here goes... 1 to 50, using only 1, 2, 3 and 4 and the basic maths operators.  Some of the answers seem a little repetitive or derivative (look at 38 through 43), and in some cases I found alternative answers afterwards.

1 = (4-3) * (2-1)
2 = (4-3) + (2-1)
3 = (4+2) / (3-1)  
4 = (4 * 3) / (2+1)  
5 = (2 * 4) – (1 * 3)  
6 = (2 * 4) – (3-1)
7 = (3+4) * (2-1)
8 = (3+4) + (2-1)  
9 = (3+4) + (2 * 1)  
10 = 1 + 2 + 3 + 4  
11 = (4 * 3) – (2 * 1)  
12 = (4 * 3) * (2-1)  
13 = (4 * 3) + (2-1)  
14 = (4 * 3) + (2 * 1)  
15 = (4 * 3) + 2 + 1
16= 4 ^ ((3+1)/2)  
17 = 3(4+1) + 2  
18 = 42 + (3-1)  
19 = 42 + (3 * 1)  
20 = 42 + 3 + 1  
21 = (4+3) * (2+1)  
22 = (4+1)2 – 3  
23 = 32 + 14  
24 = 1 * 2 * 3 * 4  
25 = 31 – (2+4)  
26 = 13 * (4-2)  
27 = 32 * (4-1)  
28 = 32 – (4 * 1)  
29 = 31 – (4 -2)  
30 = (4+1) * 3 * 2  
31 = 34 – (1+2)  
32 = 4(3-1) * 2  
33 = 34 – (2 -1)
34 = 34 * (2-1)  
35 = 34 + (2-1)  
36 = (4 * 3) * (2+1)  
37 = 34 + 1 + 2  
38 = 42 – (3+1)  
39 = 42 – (3 * 1)
40 = 41 – (3-2)  
41 = 43 – (1 * 2)  
42 = 43 – (2-1)  
43 = 41 + (3-1)  
44 = (14 * 3) + 2  
45 = 43 + (2 * 1)  
46 = 42 + 1 + 3  
47 = 41 + (3 * 2)  
48 = 24 * (3-1)  
49 = ((4 * 1) + 3)2  
50 = 41 + 32

As the logical extension, I attempted to carry on past 50.  It becomes increasingly difficult, since 1, 2, 3 and 4 are all small numbers, and the combinations of those small digits become less useful in making specific larger values (especially the prime numbers). 

However, if we expand the rules to allow ! (factorial) and decimal points, then this enables us to find solutions for 57 (for example).  I'd like to thank the free math help forum community (especially Denis for his initial suggestion to extend the rules), for the additional solutions, comments, corrections and suggestions.  They've been very friendly in quickly adopting my idea and sharing their comments and solutions.  An additional rule that's been introduced is the use of decimals - by doing this, we can include dividing by .2 (for example) as a way of multiplying by 5.

51 = (12 * 4) + 3  OR (42+1) x 3
52 = 43 - 12
53 = (1 + 4!) * 2 + 3
54 = (13 * 4) +2
55 = 34+ 21
56 = (1 + 3 + 4!)2
57 =(1+4)! / 2 - 3  OR (4 + 2) / .1 - 3
58 = (31 * 2) - 4
59 = (21 * 3) - 4
60 = 34 - 21
61 = 43 - (1 + 2)
62 = 43 - (1 * 2)
63 = 43 - (2 - 1)
64 = (2 - 1) * 43
65 = (2 - 1) + 43
66 = (2 * 1) +43
67 = (34 * 2) -1
68 = 34 * 2 * 1
69 = (34 * 2) + 1
70 = 43 + (1 + 2)!
71 = ((4! / 2) * 3!) - 1
72 = 24 * 3 * 1
73 = (3 * 4!) + (2 - 1)
74 = (3 * 4!) + (2 * 1)
75 = (4+1)2 * 3
76 = (41 * 2) - 3!
77 = ((4! + 1) * 3) + 2
78 = (4! + 2) * 3 * 1
79 = 34 - (2 * 1)
80 = 34 - (2 -1)
81 = (4! + 1 + 2) * 3
82 = 34 + (2 - 1)
83 = 34 + (2 * 1)
84 = 34 + 2 + 1
85 = (43 * 2) - 1
86 = 43 * 2 * 1
87 = (21 * 4) + 3
88 = (43 + 1) * 2
89 = (3!)! / (2 * 4) - 1
90 = (1! + 2!) * (3! + 4!)
91 = (23 * 4) - 1
92 = 23 * 4 * 1
93 = (23 * 4) + 1
94 = (1 + 3)4! - 2
95 = 3!*(24) - 1
96 = (12 * 4!) /3
97 = 4(3! - 2)! + 1
98 = (1 + 3)4! + 2
99 = 123 - 4!
100 = (3 / .12) * 4

The original textbook question asked only for the numbers from 1 to 50 with only powers and basic operators, but by expanding the rules, many more numbers have become achievable.  Dare we go above 100?

Monday, 1 June 2015

What is a "growth hacker"?

Image credit: Pinsoft Studios
Okay, I admit it: I'm confused.  I kept up to date with "experts", "gurus", "rock stars" and "ninjas", but I've reached the limit of my understanding. Why are we (as the online analytics community) now using the term 'hacker'?  When modems were dial-up, and going online meant connecting your computer to your telephone headset, hackers were bad people who illegally broke into (or 'hacked') networks

Nowadays, though, hackers are everywhere, and one of the main culprits (especially online) are the "growth hackers".  I'm just going to borrow from Wikipedia to set some context for what these new hackers actually are:
  • Hacker (term), is a term used in computing that can describe several types of persons
So we have, "excellence, playfulness, cleverness and exploration" in performed activities?  Really?  That's what a hacker is nowadays?  Can't we just be good at what we do?  We have to be playful and creative while we're at it?  Perhaps I'm an optimisation hacker and I never realised.

My research into growth hacking indicates that the term was first coined in 2010 by Sean Ellis.  Why he chose 'hacking', I'm not sure (especially given its previous connotations), but here's how he described growth hacking:

"A growth hacker is a person whose true north is growth.  Everything they do is scrutinized by its potential impact on scalable growth. ... I’ve met great growth hackers with engineering backgrounds and others with sales backgrounds; the common characteristic seems to be an ability to take responsibility for growth and an entrepreneurial drive.  The right growth hacker will have a burning desire to connect your target market with your must have solution ...  The problem is that not all people are cut out to be growth hackers."

So:  a growth hacker is a marketer whose key performance indicator is growth.  So why 'hacker'?  Perhaps it's about cracking the code for growth and finding a short cut to success?  Perhaps it's about carving a way through a jungle filled with bad ideas for growth, with an instinctive true north and a sharp blade to cut through all the erroneous ideas?

Image credit:
It's an imaginitive either way.  And now, five years after Mr Ellis's post, it seems we have growth hackers everywhere (ironic, considering the URL for the original blog post is /where-are-all-the-growth-hackers/  - they now have multiple websites and Twitter accounts ;-)

So - my curiousity has been satisfied: a [good] growth hacker is a marketer who will help rapidly accelerate growth for a small or start-up company by rapidly analysing what's working for its audience and focusing on those strategies with agility and velocity.  Why are they so popular now?  Because - I'm guessing - after the global financial issues of 2008-2009, there is now much more interest and emphasis in start-ups and the entrepreneurial spirit - and every start-up needs a growth hacker to crack the code to accelerated growth rates.