Numbers from 1,2,3,4 to 50

As an occasional diversion from work, I like to try maths puzzles, and to this end, I recently purchased a couple of high school maths textbooks (for use with students aged from about 14 to 16).  Not that I can't do the puzzles (I think my maths is okay), but because sometimes the questions and puzzles in them some interesting ideas about extension activities - for example, the recent question about the circle in the corner of a circle and a square is probably intended to be solved with trigonometry, and instead, I solved it in terms of square roots, using just Pythagoras' theorem.  That led to me solving the situation for the circle in the corner of a hexagon (which wasn't in the textbook, but which had an interesting solution too).

Anyway, I found a great question in the extension section at the end of one of the textbooks, and it goes like this:

Creating Numbers: a task requiring imagination

Your task is to create every number from 1 to 50.
You can use only the digits 1, 2, 3 and 4 once in each and the operations + - * / . 
You can use the digits as powers, and you must use all of the digits 1, 2, 3, 4

Here are some examples:

1 = (4-3) / (2-1)
20 = 4² +3 +1
68 = 34 * 2 * 1
75 = (4+1)² * 3


So, here goes... 1 to 50, using only 1, 2, 3 and 4 and the basic maths operators.  Some of the answers seem a little repetitive or derivative (look at 38 through 43), and in some cases I found alternative answers afterwards.


1 = (4-3) * (2-1)
2 = (4-3) + (2-1)

3 = (4+2) / (3-1)  
4 = (4 * 3) / (2+1)  
5 = (2 * 4) – (1 * 3)  
6 = (2 * 4) – (3-1)
7 = (3+4) * (2-1)
8 = (3+4) + (2-1)  
9 = (3+4) + (2 * 1)  
10 = 1 + 2 + 3 + 4  
11 = (4 * 3) – (2 - 1)  
12 = (4 * 3) * (2-1)  
13 = (4 * 3) + (2-1)  
14 = (4 * 3) + (2 * 1)  
15 = (4 * 3) + 2 + 1
16= 4 ^ ((3+1)/2)  
17 = 3(4+1) + 2  
18 = 4² + (3-1)  
19 = 4² + (3 * 1)  
20 = 4² + 3 + 1  
21 = (4+3) * (2+1)  
22 = (4+1)² – 3  
23 = 3² + 14  
24 = 1 * 2 * 3 * 4  
25 = 31 – (2+4)  
26 = 13 * (4-2)  
27 = 3² * (4-1)  
28 = 32 – (4 * 1)  
29 = 31 – (4 -2)  
30 = (4+1) * 3 * 2  
31 = 34 – (1+2)  
32 = 4^(3-1) * 2  
33 = 34 – (2 -1)
34 = 34 * (2-1)  
35 = 34 + (2-1)  
36 = (4 * 3) * (2+1)  
37 = 34 + 1 + 2  
38 = 42 – (3+1)  
39 = 42 – (3 * 1)
40 = 41 – (3-2)  
41 = 43 – (1 * 2)  
42 = 43 – (2-1)  
43 = 41 + (3-1)  
44 = (14 * 3) + 2  
45 = 43 + (2 * 1)  
46 = 42 + 1 + 3  
47 = 41 + (3 * 2)  
48 = 24 * (3-1)  
49 = ((4 * 1) + 3)²  
50 = 41 + 3²

As the logical extension, I attempted to carry on past 50.  It becomes increasingly difficult, since 1, 2, 3 and 4 are all small numbers, and the combinations of those small digits become less useful in making specific larger values (especially the prime numbers). 

However, if we expand the rules to allow ! (factorial) and decimal points, then this enables us to find solutions for 57 (for example).  I'd like to thank the free math help forum community (especially Denis for his initial suggestion to extend the rules), for the additional solutions, comments, corrections and suggestions.  They've been very friendly in quickly adopting my idea and sharing their comments and solutions.  An additional rule that's been introduced is the use of decimals - by doing this, we can include dividing by .2 (for example) as a way of multiplying by 5.

51 = (12 * 4) + 3  OR (4²+1) x 3
52 = 4³ - 12
53 = (1 + 4!) * 2 + 3
54 = (13 * 4) +2
55 = 34+ 21
56 = (1 + 3 + 4!)2
57 =(1+4)! / 2 - 3  OR (4 + 2) / .1 - 3
58 = (31 * 2) - 4
59 = (21 * 3) - 4
60 = 3⁴ - 21
61 = 4³ - (1 + 2)
62 = 4³ - (1 * 2)
63 = 4³ - (2 - 1)
64 = (2 - 1) * 4³
65 = (2 - 1) + 4³
66 = (2 * 1) +4³
67 = (34 * 2) -1
68 = 34 * 2 * 1
69 = (34 * 2) + 1
70 = 4³ + (1 + 2)!
71 = ((4! / 2) * 3!) - 1
72 = 24 * 3 * 1
73 = (3 * 4!) + (2 - 1)
74 = (3 * 4!) + (2 * 1)
75 = (4+1)² * 3
76 = (41 * 2) - 3!
77 = ((4! + 1) * 3) + 2
78 = (4! + 2) * 3 * 1
79 = 3⁴ - (2 * 1)
80 = 3⁴ - (2 -1)
81 = (4! + 1 + 2) * 3
82 = 3⁴ + (2 - 1)
83 = 3⁴ + (2 * 1)
84 = 3⁴ + 2 + 1
85 = (43 * 2) - 1
86 = 43 * 2 * 1
87 = (21 * 4) + 3
88 = (43 + 1) * 2
89 = 1+ 2^(3!) + 4! 
90 = (1! + 2!) * (3! + 4!)
91 = (23 * 4) - 1
92 = 23 * 4 * 1
93 = (23 * 4) + 1
94 = (1 + 3)4! - 2
95 = 3!*(2^4) - 1
96 = (12 * 4!) /3
97 = 4(3! - 2)! + 1
98 = (1 + 3)4! + 2
99 = 123 - 4!
100 = (3 / .12) * 4


The original textbook question asked only for the numbers from 1 to 50 with only powers and basic operators, but by expanding the rules, many more numbers have become achievable.  Dare we go above 100?

Further posts:  Numbers 101-150 and Numbers 150-200.

If you enjoyed this puzzle, you may also be interested in some of the other puzzles I've explored more recently:

Snakes and Ladders (the Collatz Conjecture)
Crafty Calculator Calculations (numerical anagrams with five digits)
More Multiplications (numerical anagrams, four digits)
Over and Out (reduce large numbers to zero in as few steps as you can)
Calculator Games: Front to Back
Calculator Games: Up, up and away with Ulam sequences
Calcualtor Games: The Kaprekar Constant

What is a "growth hacker"?

Okay, I admit it: I'm confused.  I kept up to date with "experts", "gurus", "rock stars" and "ninjas", but I've reached the limit of my understanding. Why are we (as the online analytics community) now using the term 'hacker'?  When modems were dial-up, and going online meant connecting your computer to your telephone headset, hackers were bad people who illegally broke into (or 'hacked') networks

Nowadays, though, hackers are everywhere, and one of the main culprits (especially online) are the "growth hackers".  I'm just going to borrow from Wikipedia to set some context for what these new hackers actually are:

• Hacker (term), is a term used in computing that can describe several types of persons
  • Hacker (computer security) someone who seeks and exploits weaknesses in a computer system or computer network
  • Hacker (hobbyist), who makes innovative customizations or combinations of retail electronic and computer equipment
  • Hacker (programmer subculture), who combines excellence, playfulness, cleverness and exploration in performed activities

Hacker image credit: Pinsoft Studios
So we have, "excellence, playfulness, cleverness and exploration" in performed activities?  Really?  That's what a hacker is nowadays?  Can't we just be good at what we do?  We have to be playful and creative while we're at it?  Perhaps I'm an optimisation hacker and I never realised.

My research into growth hacking indicates that the term was first coined in 2010 by Sean Ellis.  Why he chose 'hacking', I'm not sure (especially given its previous connotations), but here's how he described growth hacking:

"A growth hacker is a person whose true north is growth.  Everything they do is scrutinized by its potential impact on scalable growth. ... I’ve met great growth hackers with engineering backgrounds and others with sales backgrounds; the common characteristic seems to be an ability to take responsibility for growth and an entrepreneurial drive.  The right growth hacker will have a burning desire to connect your target market with your must have solution ...  The problem is that not all people are cut out to be growth hackers."

So:  a growth hacker is a marketer whose key performance indicator is growth.  So why 'hacker'?  Perhaps it's about cracking the code for growth and finding a short cut to success?  Perhaps it's about carving a way through a jungle filled with bad ideas for growth, with an instinctive true north and a sharp blade to cut through all the erroneous ideas?

Image credit: Recode.net

It's an imaginitive either way.  And now, five years after Mr Ellis's post, it seems we have growth hackers everywhere (ironic, considering the URL for the original blog post is /where-are-all-the-growth-hackers/  - they now have multiple websites and Twitter accounts ;-)

So - my curiousity has been satisfied: a [good] growth hacker is a marketer who will help rapidly accelerate growth for a small or start-up company by rapidly analysing what's working for its audience and focusing on those strategies with agility and velocity.  Why are they so popular now?  Because - I'm guessing - after the global financial issues of 2008-2009, there is now much more interest and emphasis in start-ups and the entrepreneurial spirit - and every start-up needs a growth hacker to crack the code to accelerated growth rates.

A circle in the corner of a hexagon

In my last post, I calculated the radius and diameter of a circle drawn within the external 'corner' of another circle - diagram below.  The final ratio indicated that the diameter of the small circle is 10.8% of the larger circle.  In this post, I'm going to extend this to look at circles in hexagons (something that I discussed a few years ago when I looked at square or cubic packing and hexagonal packing).  I'll also then look to apply my findings to real life, by reviewing the sizes of atoms in alloys - how do they fit together?

Firstly, here's a reminder of the result from last time:

For a circle with radius DE = EF = 1 unit, then the radius of the smaller circle "in the corner" BC = CD = 0.108... units.  The smaller circle has a radius of 11% of the larger one.  The length AD is 40% of the radius of the larger circle.

In this blog, I'd like to study the relationship between a circle and the hexagon surrounding it (the circle is inscribed in the hexagon). The diagram below shows three such hexagons.  The circles have radius r, and the distance from the centre of the circle (O) to the corner of the hexagon is h (the hypotenuse of the triangle formed by the radius and the tangent).  The angle between the radius and the hypotenuse is 30 degrees.

Given that r = 1 unit, what's the length h?

Trigonometry tells us that cos 30 = r/h therefore h = r / cos 30 = r / 0.866
h = 1.154 r

This means that the additional distance ( h - r ) = 0.154 r or 15% of the radius of the larger circle.  This additional distance is the maximum radius of a circle that would fit in the space between the three circles shown in the diagram.  For example, if the larger circles have a radius of 1 metre, the radius of the smaller circles would be 0.15m or 15 cm.  As I've calculated previously, there isn't much space between hexagonally-packed circles.


Atoms in metallic solids are typically hexagonally packed, and alloys are formed when other atoms are mixed with a pure metal.  It occurred to me that it may be possible for these atoms to be small enough to fit into the gaps between the atoms - using the mechanism I've indicated above.


Here are the radii of some metal atoms, for reference, in picometres.  1 picometre, = 1 x 10 -12 metres

Fe (iron)  156
Sn (tin)  145
Al (aluminium) 118
Co (cobalt) 152

15% of these values gives us 24, 21, 17, and 22.    The atomic radius of carbon (which is alloyed with iron to make steel) is 67 pm.  So, there is no way that carbon will fit neatly into the gaps in the matrix of hexagonally-packed iron atoms.   [Indeed, helium has the lowest atomic radius, and that's 31 pm, still too large].

Instead, in interstitial alloys, the smaller atoms in an alloy distort the close-packed arrangement of the metal and that's what affects its physical properties.

Further reading

Previous post on the radius of a circle in the corner of a circle
Space filling calculation 3D - sphere in cube, sphere in hexagonal prism
Hexagonal close packing - 2D-filling calculation

 

Geometry: A Circle in the corner of a circle

This article is specifically written to answer the geometry question:  "What is the radius of a circle drawn in the space between a circle of radius 1 unit, and the corner of the enclosing square?" To better explain the question, and then answer it, I have drawn the diagram shown below.  The question states that the radius of the larger circle is 1 unit of length, and that it is enclosed in a square. What is the radius of the smaller circle drawn in the space in the corner region?  (The question was asked in a GCSE workbook, aimed for children aged 14-16, although the geometry and algebra became more complicated than I had expected).

The diagram isn't perfect, but I'm better at using a pencil and compasses than I am with drawing geometric shapes in Photoshop.  The larger circle has centre E, and DE = EF = 1 unit.  What is the radius of the smaller circle, with centre C (BC = CD = smaller radius).

By symmetry, the angle at A = 45 degrees, so triangles ACG and AEH are right-angled isosceles triangles.
AH and EH are equal to the radius of the larger circle = 1 unit
By Pythagoras' Theorem, length AE = √2

Length AD = AE - DE = √2 -1
However, length AD is not the diameter of the smaller circle.  The diameter of the smaller circle is BD, not AD.  We are still making progress, nonetheless.

Next, consider the ratio of the lengths AD:AF.
AD = √2 -1 as we showed earlier
AF = AD + DF = (√2 -1) + 2 (the diameter of the circle) = √2 + 1

So the ratio AD:AF =  √2 -1 :  √2 + 1

And the fraction AD/AF = √2 -1 /  √2 + 1

What is the remaining distance between the circle and the origin?

Look again at the larger circle, and the ratio of the diameter to the distance from the corner to the furthest point on the circle?

The fraction AB/AD is equal to the fraction AD/AF.  This fraction describes the relationship between the diameter of a circle and the additional distance to the corner of the enclosing square.  The diameter of the circle is not important, the ratio is fixed.

So we can divide the shorter length AD in the ratio AD:AF, and this will give us the length AB and (as we already know AD) the diameter of the smaller circle, BD.

To express it more simply and mathematically:  AD/AF = AB/AD
Substituting known values for AD and AF, this gives:

AD/AF = AB/AD

(AD^2) / AF = AB

(√2 -1)^2 /  (√2 + 1) = AB


 Evaluating:
(√2 -1)^2 = 3 - 2 √2 = 0.17157...

and:
(√2 + 1) =  2.4142...

Now:  BD (diameter or circle) = AD ('corner' of larger circle) - AB ('corner' of small circle)
Substituting values for AD and AB, and then combining terms over the same denominator, we get:

 Having combined all terms over the same denominator, we can now simplify (√2 -1)(√2 +1), since (a+b)(a-b) = a^2 - b^2

BD is the diameter of the smaller circle, BD = 0.216...  Comparing this with the diameter of the larger circle, which is 2.00, we can see that the smaller circle is around 10% of the diameter of the larger one.  This surprised me - I thought it was larger.

In future posts, I'll look at other arrangements of circles in corners - in particular the quarter-circle in the corner (which, as a repeating pattern, would lead to a smaller circle touching four larger circles in a square-like arrangement), and a third-of-a-circle in the corner of a hexagon.  I'll then compare the two arrangements (in terms of space filled) and also check against any known alloys, looking at the ratios of diameters to see if I can find a real-life application.

Other 'circle' posts:

A Circle in the Corner of a Circle
A Circle in the Corner of a Hexagon
Close-packed Circles - calculating the space occupied

Close-packed Spheres - calculating the volume occupied

Pitfalls of Online Optimisation and Testing 3: Discontinuous Testing

Some forms of online testing are easy to set up, easy to measure and easy to interpret.  The results from one test point clearly to the next iteration, and you know exactly what's next.  For example, if you're testing the height of a banner on a page, or the size of the text that you use on your page headlines, there's a clear continuous scale from 'small' to 'medium' to 'large' to 'very large'.  You can even quantify it, in terms of pixel dimensions.  With careful testing, you can identify the optimum size for a banner, or text, or whatever it may be.  I would describe this as continuous testing, and it lends itself perfectly to iterative testing.

Some testing - in fact, most testing - is not continuous.  You could call it discrete testing, or digital testing, but I think I would call it discontinuous testing.

For example:
colours (red vs green vs black vs blue vs orange...)
title wording ("Product information" vs "Product details" vs "Details" vs "Product specification")
imagery (man vs woman vs family vs product vs product-with-family vs product alone)

Both forms of testing are, of course, perfectly valid.  The pitfall comes when trying to iterate on discontinuous tests, or trying to present results, analysis and recommendations to management.  The two forms can become confused, and unless you have a strong clear understanding of what you were testing in the first place - and WHY you tested it - you can get sidetracked into a testing dead-end. 

For example; let's say that you're testing how to show product images on your site.  There are countless ways of doing this, but let's take televisions as an example.  On top right is an image borrowed from the Argos website; below right is one from Currys/PC World. The televisions are different, but that's not relevant here; I'm just borrowing the screenfills and highlighting them as the main variable.  In 'real life' we'd test the screenfills on the same product.

Here's the basis of a straightforward A/B test - on the left, "City at Night" and on the right, "Winter Scene".  Which wins? Let's suppose for the sake of argument that the success metrics is click-through rate, and "City at Night" wins.  How would you iterate on that result, and go for an even better winner?  It's not obvious, is it?  There are too many jumps between the two recipes - it's discontinuous, with no gradual change from city to forest.

The important thing here (I would suggest) is to think beforehand about why one image is likely to do better than the other, so that when you come to analyse the results, you can go back to your original ideas and determine why one image won and the other lost.  In plain English:  if you're testing "City at Night" vs "Winter Scene", then you may propose that "Winter Scene" will win because it's a natural landscape vs an urban one.  Or perhaps "City at Night" is going to win because it showcases a wider range of colours.  Setting out an idea beforehand will at least give you some guidance on how to continue.

However, this kind of testing is inherently complex - there are a number of reasons why "City at Night" might win:
- more colours shown on screen
- showing a city line is more high-tech than a nature scene
- stronger feeling of warmth compared to the frozen (or should that be Frozen) scene

In fact, it's starting to feel like a two-recipe multi-variate test; our training in scientific testing says, "Change one thing at a time!" and yet in two images we're changing a large number of variables.  How can we unpick this mess?

I would recommend testing at least two or three test recipes against control, to help you triangulate and narrow down the possible reasons why one recipe wins and another loses. 

Shown on the right are two possible examples for a third and fourth recipe which might start to narrow down the reasons, and increase the strength of your hypothesis.  
 
 If the hypothesis is that "City at Night" did better because it was an urban scene instead of a natural scene, then "City in Daylight" (top right) may do even better.  This has to be discontinuous testing - it's not possible to test the various levels of urbanisation; we have to test various steps along the way in isolation.

Alternatively, if "City at Night" did better because it showcased more colours, then perhaps "Mountain View" would do better - and if "Mountain View" outperforms "Winter Scene", where the main difference is the apparent temperature of the scene (warm vs cold), then warmer scenes do better, and a follow-up would be a view of a Caribbean holiday resort. And there you have it - perhaps without immediately realising, the test results are now pointing towards an iteration with further potential winners. 

By selecting the test recipes carefully and thoughtfully and deliberately aiming for specific changes between them, it's possible to start to quantify areas which were previously qualitative.  Here, for example, we've decided to focus (or at least try to focus) on the type of scene (natural vs urban) and on the 'warmth' of the picture, and set out a scale from frozen to warm, and from very natural to very urban.  Here's how a sketch diagram might look:

Selecting the images and plotting them in this way gives us a sense of direction for future testing.  If the city scenes both outperform the natural views, then try another urban scene which - for example - has people walking on a busy city street.  Try another recipe set in a park area - medium population density - just to check the original theory.  Alternatively, if the city scenes both perform similarly, but the mountain view is better than the winter scene (as I mentioned earlier), then try an even warmer scene - palm trees and a tropical view.

If they all perform exactly similarly, then it's time to try a different set of axes (temperature and population density don't seem to be important here, so it's time to start brainstorming... perhaps pictures of people and racing cars are worth testing?).

Let's take another example:  on-page text.  How much text is too much text, and what should you say? How should you greet users, what headlines should you use?  Should you have lengthy paragraphs discussing your product's features, or should you keep it short and concise - bullet points with the product's main specifications?

Which is better, A or B?  And (most importantly) - why?  (Blurb borrowed and adapted from Jewson Tools)

A: 
Cordless drills give you complete flexibility without compromising on power or performance.  We have a fantastic range, from leading brands such as AEG, DeWalt, Bosch, Milwaukee and Makita.  This extensive selection includes tools with various features including adjustable torque, variable speeds and impact and hammer settings. We also sell high quality cordless sets that include a variety of tools such as drills, circular saws, jigsaws and impact drivers. Our trained staff in our branches nationwide can offer expert technical advice on choosing the right cordless drill or cordless set for you.

B:
* Cordless drills give you complete flexibility without compromising on power or performance.
* We stock AEG, DeWalt, Bosch, Milwaukee and Makita
* Selection includes drills with adjustable torque, variable speeds, impact and hammer settings
* We also stock drills, circular saws, jigsaws and impact drivers
* Trained staff in all our stores, nationwide


If A was to win, would it because of its readability?  Is B too short and abrupt?  Let's add a recipe C and triangulate again:

C:
* Cordless drills - complete flexibility
* Uncompromised performance with power
* We stock AEG, DeWalt, Bosch, Milwaukee and Makita
* Features include adjustable torque, variable speed, impact and hammer settings
* We stock a full range of power tools
* Nationwide branches with trained staff

 C is now extremely short - reduced to sub-sentence bullet points.  By isolating one variable (the length of the text) we can hope to identify which is best - and why.  If C wins, then it's time to start reducing the length of your copy.  Alternatively, if A, B and C perform equally well, then it's time to take a different direction.  Each recipe here has the same content and the same tone-of-voice (it just says less in B and C); so perhaps it's time to add content and start to test less versus more.


D:
* Cordless drills - complete flexibility with great value
* Uncompromised performance with power
* We stock AEG, DeWalt, Bosch, Milwaukee and Makita
* Features include adjustable torque, variable speed, impact and hammer settings
* We stock a full range of power tools to suit every budget
* Nationwide branches with trained and qualified staff to help you choose the best product
* Full 30-day warranty
* Free in-store training workshop  

E: 
* Cordless drills provide complete flexibility
* Uncompromised performance
* We stock all makes
* Wide range of features

* Nationwide branches with trained staff

In recipe D, the copy has been extended to include 'great value'; 'suit every budget', training and warranty information - the hypothesis would be that more is more, and that customers want this kind of after-sales support.  Maybe they aren't - maybe your customers are complete experts in power tools, in which case you'll see flat or negative performance.  In Recipe E, the copy has been cut to the minimum - are readers engaging with your text, or is it just there to provide context to the product imagery?  Do they already know what cordless drills are, what they do, and are they just buying another one for their team?

So, to sum up:  it's possible to apply scientific and logical thinking to discontinuous testing - the grey areas of optimisation.  I'll go for a Recipe C/E approach to my suggestions:

*  Plan ahead - identify variables (list them all)
*  Isolate variables as much as possible and test one or two
*  Identify the differences between recipes 
*  Draw up a continuum on one or two axes, and plot your recipes on it
*  Think about why a recipe might win, and add another recipe to test this theory (look at the continuum)

The  articles in the Pitfalls of Online Optimisation and Testing series

Article 1:  Are your results really flat?
Article 2: So your results really are flat - why?  
Article 3: Discontinuous Testing

Pitfalls of Online Optimisation and Testing 2: Spot the Difference

The second pitfall in online optimisation that I would like to look at is why we obtain flat results - totally, completely flat results at all levels of the funnel.  All metrics show the same results - bounce rate, exit rate, cart additions, average order value, order conversion. There is nothing to choose between the two recipes, despite a solid hypothesis and analytics which support your idea.

The most likely cause is that the changes you made in your test recipe were just not dramatic enough.  There are different types of change you could test:
 

*  Visual change (the most obvious) 
*  Path change (where do you take users who click on a "Learn more" link?)
*  Interaction change (do you have a hover state? Is clicking different from hovering? How do you close a pop-up?)

Sometimes, the change could be dramatic but the problem is that it was made on an insignificant part of the site or page.  If you carried out an end-to-end customer journey through the control experience and then through the test experience, could you spot the difference?  Worse still, did you test on a page which has traffic but doesn't actively contribute to your overall sales (is its order participation virtually zero?)?

Is your hypothesis wrong? Did you think the strap line was important? Have you in fact discovered that something you thought was important is being overlooked by visitors?

Are you being too cautious - is there too much at stake and you didn't risk enough? 

Is the part of the site getting traffic? And does that traffic convert? Or is it just a traffic backwater or a pathing dead end?  It could be that you have unintentionally uncovered an area of your site which is not contributing to site perofrmance.

Do your success metrics match your hypothesis? Are you optimising for orders on your Customer Support pages? Are you trying to drive down telephone sales?

Some areas of the site are critical, and small changes have big differences. On the other hand, some parts of the site are like background noise that users filter out (which is a shame when we spend so much time and effort selecting a typeface, colour scheme and imagery which supports our brand!). We agonise over the photos we use on our sites, we select the best images and icons... And they're just hygiene factors that users barely glance at.  On the other hand, there are some parts that are critical - persuasive copy, clear calls to action, product information and specifications.  What we need to know, and can find out through our testing, is what matters and what doesn't.

Another possibility is that you made two counter-acting changes - one improved conversion, and the other worsened it, so that the net change is close to zero. For example, did you make it easier for users to compare products by making the comparison link larger, but put it higher on the page which pushed other important information on the page to a lower position, where it wasn't seen?  I've mentioned this before in the context of landing page bounce rate - it's possible to improve the click through rate on an email or advert by promising huge discounts and low prices... but if the landing page doesn't reflect those offers, then peopl will bounce off it alarmingly quickly.  This should show up in funnel metrics, so make sure you're analysing each step in the funnel, not just cart conversion (user added an item to cart) and order conversion (user completed a purchase).


Alternatively:  did you help some users, but deter others?  Segment your data - new vs returning, traffic source, order value...  did everybody from all segments perform exactly as they did previously, or did the new visitors benefit from the test recipe, while returning visitors found the change unhelpful?

In conclusion, if your results are showing you that your performance is flat, that's not necessarily the same as 'nothing happened'.  If it's true that nothing happened, then you've proved something different - that your visitors are more resilient (or perhaps resistant) to the type of change you're making.  You've shown that the area you've tested, and the way you've tested it, don't matter to your visitors.  Drill down as far as possible to understand if you've genuinely got flat results, and if you have, you can either test much bigger changes on this part of the site, or stop testing here completely, and move on.

The  articles in the Pitfalls of Online Optimisation and Testing series

Article 1:  Are your results really flat?
Article 2: So your results really are flat - why?  
Article 3: Discontinuous Testing

Reviewing Manchester United Performance - Real Life KPIs Part 2

As a few weeks have passed since my last review of Manchester United's performance in this year's Premier League.  An overview of the season so far reveals some interesting facts:

Southampton went to third position in mid-January, following their win at Old Trafford.  Southampton finished eighth last season, and 14th in the season before that.  This is their first season with new manager Ronald Koeman.  Perhaps some analysis on his performance is needed, another time perhaps. :-)

Southampton enjoyed their first win in 27 years in the league at Old Trafford on 11 January.  Their fifteen previous visits were two draws (1999, 2013) and thirteen wins for Manchester United. Conversely, United had won their last five at home and missed out on the chance for a ninth win in the league – which was their total for home wins in the whole of last season.

So let's take a look at Louis Van Gaal's performance, as at 9 February 2015, and compare it, as usual, with David Moyes (the 'chosen one'), Alex Ferguson (2012-13) and Alex Ferguson (1986-87, his first season).

Horizontal axis - games played
Vertical axis - cumulative points
Red - AF 2012-13
Pink - AF 1986-87
Blue - DM 2013-2014
Green - LVG 2014-15 (ongoing)

The first thing to note is that LVG has improved his performance recently, and is now back above the blue danger line (David Moyes' performance in 2013-14, which is the benchmark for 'this will get you fired').

However, LVG's performance is still a long way below the red line left by Alex Ferguson in his final season, so let's briefly investigate why.

Under LVG, Manchester United have drawn 33% of their league games this season, compared to just 13% for Alex Ferguson's 2012-13 season.  This doesn't include the goal-less draw against Cambridge United in the FA Cup, which is a great example of Man Utd not pressing home their apparent advantage (Man Utd won the rematch 3-0 at Old Trafford). Yesterday (as I write), Manchester United scraped a draw against West Ham by playing the 'long-ball game', criticised after the match by West Ham's manager, Sam Allardyce.  West Ham are currently eighth in the table, four places behind Man Utd.

Interestingly, Moyes and Van Gaal have an identical win rate of 50%.  It might be suggested that Van Gaal's issue is not converting enough draws into wins; this is a slightly better problem to have compared to Moyes' problem, which was not holding on to enough draws and subsequently losing.  In football terms, Van Gaal needs to teach his team to more effectively 'park the bus'.

Is Louis Van Gaal safe?  According to the statistics alone, yes, he is, for now.  He's securing enough draws to keep him above the David Moyes danger line, and he's achieving more wins that Alex Ferguson did in his first season.  However, his primary focus must be to start converting draws into wins.  I haven't done the full match analysis to determine if that means scoring more or holding on to the lead once he has it - perhaps that will come later.

Is Louis Van Gaal totally safe?  That depends on if the staff at Man United think that a marginal improvement on last season's performance is worth the £59.7m spent on Angel Di Maria, £29m on Ander Herrera, and £27m on Luke Shaw (plus others).  £120m for a few more draws in the season is probably not seen as good value for money.