Analysis is Easy, Interpretation Less So

Every time we open a spreadsheet, or start tapping a calculator (yes, I still do), or plot a graph, we start analysing data.  As analysts, it is probably most of what we do all day.  It's not necessarily difficult - we just need to know which data points to analyse, which metrics we divide by each other (do you count exit rate per page view, or per visit?) and we then churn out columns and columns of spreadsheet data.  As online or website analysts, we plot the trends over time, or we compare pages A, B and C, and we write the result (so we do some reporting at the end as well).

Analysis. Apparently.

As business analysts, it's not even like we have complicated formulae for our metrics - we typically divide X by Y to give Z, expressed to two decimal places, or possibly as a percentage.  We're not calculating acceleration due to gravity by measuring the period of a pendulum (although it can be done), with square roots, fractions, and square roots of fractions.

Analysis - dare I say it - is easy.

What follows is the interpretation of the data, and this can be a potential minefield, especially when you're presenting to stakeholders. If analysis is easy, then sometimes interpretation can really be difficult.

For example, let's suppose revenue per visit went up by 3.75% in the last month.  This is almost certainly a good thing - unless it went up by 4% in the previous month, and 5% in the same month last year.  And what about the other metrics that we track?  Just because revenue per visit went up, there are other metrics to consider as well.  In fact, in the world of online analysis, we have so many metrics that it's scary - and so accurate interpretation becomes even more important.

Okay, so the average-time-spent-on-page went up by 30 seconds (up from 50 seconds to 1 minute 20).  Is that good?  Is that a lot?  Well, more people scrolled further down the page (is that a good thing - is it content consumption or is it people getting well and truly lost trying to find the 'Next page' button?) and the exit rate went down.  

Are people going back and forth trying to find something you're unintentionally hiding?  Or are they happily consuming your content and reading multiple pages of product blurb (or news articles, or whatever)?  Are you facilitating multiple page consumption (page views per visit is up), or are you sending your website visitors on an online wild goose chase (page views per visit is up)?  Whichever metrics you look at, there's almost always a negative and positive interpretation that you can introduce.

This comes back, in part, to the article I wrote last month - sometimes two KPIs is one too many.  It's unlikely that everything on your site will improve during a test.  If it does, pat yourself on the back, learn and make it even better!  But sometimes - usually - there will be a slight tension between metrics that "improved" (revenue went up), metrics that "worsened" (bounce rate went up) and metrics that are just open to anybody's interpretation (time on page; scroll rate; pages viewed per visit; usage of search; the list goes on).  In these situations, the metrics which are open to interpretation need to be viewed together, so that they tell the same story, viewed from the perspective of the main KPIs.  For example, if your overall revenue figures went down, while time on page went up, and scroll rate went up, then you would propose a causal relationship between the page-level metrics and the revenue data:  people had to search harder for the content, but many couldn't find it so gave up.

On the other hand, if your overall revenue figures went up, and time on page increased and exit rate increased (for example), then you would conclude that a smaller group of people were spending more time on the page, consuming content and then completing their purchase - so the increased time on page is a good thing, although the exit rate needs to be remedied in some way.  The interpretation of the page level data has to be in the light of the overall picture - or certainly with reference to multiple data points.  

I've discussed average time on page before.  A note that I will have to expand on sometime:  we can't track time on page for people who exit the page.  It's just not possible with standard tags. It comes up a lot, and unless we state it, our stakeholders assume that we can track it:  we simply can't.  

So:  analysis is easy, but interpretation is hard and is open to subjective viewpoints.  Our task as experienced, professional analysts is to make sure that our interpretation is in line with the analysis, and is as close to all the data points as possible, so that we tell the right story.

In my next posts in this series, I go on to write about how long to run a test for and explain statistical significance, confidence and when to call a test winner.

Other articles I've written on Website Analytics that you may find relevant:

Web Analytics - Gathering Requirements from Stakeholders

Analysis is Easy, Interpretation Less So
Telling a Story with Web Analytics Data
What is "Direct Traffic"?
Reporting, Analysing, Testing and Forecasting
Pages with Zero Traffic

Maths Puzzle: Sugar Cubes (orders of magnitude)

Continuing the current series of puzzles taken from Math-E-Magic, here's one which tests understanding of orders of magnitude.

Sugar Cubes
The Big Sugar Corporation wants to persude people to use lumps of sugar, or sugar cubes; so they run a puzzle competition and the first person to achieve correct answers to all three questions wins a lifetime supply of sugar.  It's not the healthiest competition, but here we go.

You have been sent one million cubes of sugar; each cube is just half an inch long, wide and high.

1.  Suppose the 1,000,000 sugar cubes arrive packed into one giant cube.  Where would you store it?  Garage? Warehouse? Under the table?

2.  Suppose you lay the 1,000,000 sugar cubes out on the ground in a single square layer.  How much area would you need?  A tennis court?  A football field?  An entire car park?


3.  Suppose you stacked the sugar cubes (if you were able to play this sweet Jenga) into a single tower 1,000,000 cubes high.  How far would it reach?  As high as a house? Skyscraper? Mountain?  To the Moon?

Let's do some maths.

1.  If we form a cube from the little sugar cubes, then each side of the cube will need to be the cube-root of 1,000,000 cubes long.  The cube root of a million is 100, since 100 cubed = 100 x 100 x 100 = 1000000

Each cube is a quarter of an inch long (we started in imperial measurements, so we'll stay there for now), and since 100 x 0.25 = 25, then the cube will be 25 inches long, wide and high.  Under the kitchen table will do fine.  (25 inches is approx. 60 cm.).

2.  To form a square from the sugar cubes, each side will have a length equal to the square root of 1,000,000 and that's 1,000.  1,000 x 0.25 inches = 250 inches, or nearly 21 feet.  A square with sides of 21 feet would fit into a medium-sized garden, or about six car park spaces.  A tennis court is 27 feet wide (for singles) and 63 feet long, so it would fit comfortably in half of a tennis court (from the baseline to the net).  In fact, the service line (the area which marks the two service boxes) is exactly 21 feet from the net and parallel to it.

3.  One million sugar cubes, eventually stacked one on top of the other, would be 250,000 inches (a quarter inch multiplied by a million).  250,000 inches is 20,800 feet, or 6,944 yards, which is just under four miles.  That won't reach the moon (250,000 miles away), it's more like a small mountain.

Gobally, the top 100 mountains are all above 23,000 feet, but we are still talking around the scale of a mountain.

The highest mountain in the UK is Ben Nevis, which stands at 4,411 feet.
The tallest in Europe is Mount Korab in Albania, which is 9,068 feet.
Sugarloaf Mountain in Brazil is 15,000 feet (396 metres).
Our sugar mountain, at 20,800 feet, is certainly a respectable mountain.

--
Incidentally, if you were (in some way) able to stack the sugar cubes at the rate of one a second for a million seconds, that would be 277 hours, or 11.5 days.  If you stacked them at the rate of one a minute (which it may be as you start climbing an adjacent mountain), then you'd be going for almost two years.

Other articles in the 'A Puzzle A Day' series:

Three Horse Race
How Old is Aunt Tabitha
Big Ben Strikes Twelve

Round Goes The Gossip
Sugar Cubes (Orders of Magnitude)
Add Nine, Reverse Digits
Flowers and Venn Diagrams
Sum of Three Numbers Equals Their Product

Another random puzzle:

Front to back (palindromes in numbers)

Maths Puzzle: Cookie Jars

These puzzles are the second batch I'm taking from Math-E-Magic by Raymond Blum, Adam Hart-Davis, Bob Longe and Derrick Niedermann.  The first was a geometric question; these are based on algebra.
These puzzles are entitled Cookie Jar and Fleabags, but they are very similar to a wide range of puzzles (typically related to the relationships between people's ages).

Cookie Jar
Joe and Ken each held a cookie jar and had a look inside them to see how many cookies were left.  
Joe said, "If you gave me one of yours, we'd both have the same number of cookies."
Ken replied, "Yes, but you've eaten all of yours - you have none left!"
How many cookies does Ken have?

This is a relatively straightforward puzzle, helped by the fact that Joe has zero cookies, and there's only one other constraint - if Ken gives Joe a cookie, they'll have the same number (one).  So, if Joe will have one cookie after the transaction, then so will Ken.

But that isn't the answer.  We have to remember that Ken has one cookie after the transaction, but that he also had the one he would give to Joe - so he has two.

Fleabags

Two shaggy old dogs were walking down the street.
Captain sits down and says to Champ, "If one of your fleas jumped onto me, we'd have the same number."
Champ replies, "But if one of yours jumped onto me, I'd have five times as many as you!"
How many fleas are there on Champ?

This one is going to take a little more work - and we can use algebra to help solve it.

Let's have the number of fleas on Captain as A, and the number of fleas on Champ as H (taking the second letter of the two dogs' names).

If one flea jumps onto Captain, he will have A+1.  And if that flea has come from Champ, then he will have H-1.  And these numbers are the same, so A+1 = H-1  (1)

Now, if one flea jumps from Captain, he will have A-1.  And this number is five times greater than Champ's new total H+1.  So 5(A-1) = H+1    (2)

If A+1 = H-1 then A+2 = H (from 1)

And we can use this new value of H in (2), to give us 5(A-1) = (A+2) + 1

Expanding and simplifying:
5A - 5 = A + 3
4A = 8
A = 2

Captain has two fleas.

And since A+2 = H, Champ has four fleas.

A few other articles in the Mathemagic Series:

Arrange nine coins into ten straight lines
Solve 1/a + 1/b + 1/c = 1  for unique a, b, c
Solving Magic Triangles
and the slightly more complex Magic Hexagons

Maths Puzzle: Arrange Nine Coins into Ten Straight Lines

This puzzle is taken from Math-E-Magic by Raymond Blum, Adam Hart-Davis, Bob Longe and Derrick Niedermann.  I've owned my copy of this book for a number of years and have referered to it in the past top help with problems I've been working on separately.  More recently, during a few idle moments (when there's not been enough time, energy or enthusiasm to do anything bigger) I've started solving some of the the puzzles it poses.  I've even (horror of horrors) started writing IN the book (but only in pencil).

Here's the first puzzle I looked at:

Nine Coins (page 29)

Wendy got into trouble in her math class.  She was sorting out money she planned to spend after school, and accidentally dropped nine coins onto the floor. The teacher was so upset that she told Wendy to stay at school until she could arrange the nine coins into at least six lines with three coins in each line.  Can you do it?  Wendy did, and in fact she arranged her nine coins into ten lines, with three in each line.  How?
The first question - can you get nine coins into six straight lines, is fairly straightforward, especially if you realise that nine is a square number.  If you arrange the nine coins into a 3x3 rectangle you can achieve six lines (three horizontal and three vertical).  

If you're more careful, you can arrange them into a symmetrical rectangle, or even a square, so that the diagonals form two extra lines, bringing the total to eight.  It's not ten, but it's getting us closer.

Working on the principle of increasing diagonal lines, in order to reach ten straight lines, we need to adapt the central row (or column) of coins so that they can provide the extra lines we need.  By moving the two coins at the end of the central row, we can achieve the addition more diagonal lines - see below...

The new diagonal lines are shown in the paler grey colour. We have lost the vertical lines at the edges of the square, but we have gained four diagonal lines, bring our total up from eight to ten.  The diagram below shows the solution with all ten lines shown:  three horizontal, one vertical, two long diagonals and four short diagonals.

One KPI too many

Three hypothetical car sales representatives are asked to focus on increasing their sales of hybrid cars for a month. They are a good cross-section of the whole sales team (which is almost 40 sales reps), and they each have their own approach. The sales advisor with the best sales figures for hybrid cars at the end of the month will receive a bonus, so there's a clear incentive to sell well.  At the end of the month, the sales representatives get together with management to compare their results and confirm the winner.

Albert

Albert made no real changes to his sales style, confident that his normal sales techniques would be enough to get him through top sales spot. 

Albert is, basically, our "control", which the others will be compared against. Albert is a fairly steady member of the team, and his performance is ideal for judging the performance of the other individuals.  Albert sold 100 cars, of which 20 were hybrids.

Britney 

Britney embraces change well, and when this incentive was introduced, she immediately made significant changes to her sales tactics.  Throughout the incentive period, she went to great lengths to highlight the features and benefits of the hybrid cars.  In some cases, she missed out on sales because she was pushing the hybrids so enthusiastically.

While she doesn't sell as many cars as Albert, she achieves 90 sales, of which 30 are hybrids.

Charles

Finally, Charles is the team's strongest salesman, and throughout the sales incentive month, he just sells more cars.  He does this by generally pushing, chasing and selling harder to all customers, using his experience and sales skills.  He doesn't really focus on selling the hybrids in particular.

Consequently, he achieves an enormous 145 sales, which includes 35 hybrid sales.   

Let's summarise, and add some more metrics and KPIs (because you can never have too many, apparently...).

	Albert	Britney	Charles
Total car sales	100	90	145
Hybrid car sales	20	30	35
% Hybrid	20%	33.3%	24.1%
Total revenue	$915,000	$911,700	$913,500
Revenue per car	$9,150	$10,130	$6,300

Who did best?

1. Albert achieved the highest revenue, but only sold 20% hybrid cars.
2. Britney achieved 33% hybrid sales, but only sold 90 cars in total.  She did, however, achieve the highest revenue per car (largely due to sales of the new, more expensive hybrids).
3. Charles sold 35 hybrids - the most- but only at a rate of 24.1%.  He also sold generally cheaper cars (he sold 110 non-hybrid cards, and many of them were either discounted or used cars)

So which Key Performance Indicator is actually Key?

This one is often a commercial decision, based on what's more important to the business targets. Is it the volume of hybrid cars, or the percentage of them? How far could Britney's drop in overall sales be accepted before it is detrimental to overall performance? And how far could Charles's increase in overall sales be overlooked?

Sometimes, your recommendation for implementing an optimisation recipe will run into a similar dilemma. In situations like these, it pays to know which KPI is actually Key! Is it conversion? Is it volumes of PDF downloads, or is it telephone calls, chat sessions, number of pages viewed per visit or is it revenue? And how much latitude is there in calling a winner? In some situations, you won't know until you suddenly realise that your considered recommendation is not getting the warm reception you expected (but you'll start to get a feel for the Key KPIs, even if they're never actually provided by your partners).

Image credits:
Albert:  https://www.victorylaynechevrolet.com/MeetOurDepartments 
Britney:  https://windsorstar.com/news/local-news/fcas-best-month-ever-as-overall-canadian-auto-sales-hit-new-record
Charles:  https://lifestyle.clickhole.com/beautiful-this-car-salesman-shaved-1-000-off-the-stic-1825120441 

Car: https://www.turbosquid.com/3d-models/3d-model-of-generic-hybrid-car-simple/942292 

Bumblebee - Movie Review

After a slow and unfortunate deterioration in the quality of the Transformers movies, the franchise was in serious need of a good reboot, and with Bumblebee the series has transformed.

I saw Bumblebee on New Year's Eve 2018, a week or so after it was released. Early reviews had been positive and the trailers showed a refreshed look and a more grown-up approach to plot and story (even the first Transformers film had "Sam's Happy Time" which somehow escaped the cutting room floor). And the trailer where Soundwave ejects a transforming Ravage... sign me up for tickets, now!

The film opens on Cybertron - yes, really - and I very quickly identified Prowl, Wheeljack,  Brawn, Optimus and others. The Decepticons are also well represented, with later scenes showing the Seekers, Shockwave and Soundwave - who sounds like the original G1 Soundwave too (none of the whispering and growling of the recent films). 

Bumblebee is dispatched to Earth to set up a base and ensure the Decepticons don't establish themselves there.  However, Bumblebee's arrival on Earth is shortly followed by Starscream's and Bumblebee is massively outgunned.  I assume it was Starscream as he transformed and looked and even sounded like Starscream,  even though I don't think he was named specifically.

The battle with Starscream leaves Bumblebee without his voice or memory. I detect the hand of Mr Spielberg in delivering the "alien must have no voice and learn to communicate" plot which he has been pushing - albeit successfully - since ET The Extra Terrestrial.  Losing his voice and his memory, Bumblebee subsequently goes into hiding.

The story is well written, and I'm pleased to say that the robots get plenty of screen time. One of my main criticisms of the later films was the excessive focus on the humans in the story, to the detriment of the robots. I fully get that humans are cheaper to film than robots, but the over-reliance on the human back-story has been an ongoing issue for me.  That's not the case here - the robots feature heavily in the story and it feels like Bumblebee is on screen around 70% of the time, and it certainly seems like the human-only scenes last barely five minutes before there's a robot back on screen again.

And I'm pleased to report that the humans are the best written in Transformers history, by far. The parents are far better than the Witwicky parents, who were consistently dreadful - the only thing that could be done with them was to limit their screen time.  Here, though, the humans - the military and civilian - are all credible and make genuine, believable contributions to the plot. We see an early Sector 7 and the basis of the Internet, all handled well.  This is the kind of writing we saw in the better parts of Dark of the Moon, with the Transformers actively involved in human history.

The story revolves around two Decepticons trying to track down Bumblebee, firstly across space  (dispatching a G1 Cliffjumper along the way) and then on Earth, while Bumblebee and his human companion Charlie learn to work together to keep him safe until he recovers his memory.  There are some fantastic touches from Transformers history - The Touch by Stan Bush is a great example - and the final showdown features a neat little sequence where one of the Decepticons throws Bumblebee to the ground; Bumblebee transforms to car mode before turning 180 degrees, driving towards the Decepticon, jumping and transforming back into robot mode - exactly as Jazz does in the title scenes of the series one cartoon.

The ending features Bumblebee leaving Charlie, his mission now accomplished. After exchanging his vehicle mode from a Volkswagen to a Camaro, he drives off across the Golden Gate Bridge and to my delight, joins a red articulated lorry with grey trailer.  There isn't a single word of dialogue or explanation, but the sight of a genuine G1 Optimus Prime on the big screen made me realise what has been missing from all the previous Transformers movies.

I was surprised - almost astonished - to discover that many of the film crew for Bumblebee are the same as the previous movies, with Michael Bay now as a producer instead of director, along with Lorenzo di Bonaventura, Tom deSanto and Don Murphy.  This film feels, looks and sounds so different from all of the previous films that credit must go directly to the writer, Christina Hodson, and the director Travis Knight. There are fewer explosions and out-and-out gun battles, and instead the focus is on the story - which is believable (as far as these stories can be) and written with three-dimensional characters who aren't written just for awkward gags.  There is still plenty of action and robot vs robot combat, on Cybertron and on Earth, but less of the widespread explosions and overly lengthy battle scenes.

On a side note, this film plays fast and loose with the continuity established in the first film.  For example, Optimus is already on Earth in 1987, and in his G1 form, and at the end of the movie we see seven new Autobots about to land on Earth (observed by Bumblebee and Prime). It seems that the new movie is rebooting the continuity - given the strength of this story, that's fine with me.

Overall, this film is far and away the best of the series, even exceeding the first and third films.  Even taking the excellent Cybertronian footage out of the equation, the film is still outstanding. The focus on a limited number of characters works (as it did in the first movie) and the human characters are written with depth, care and real feelings - not the stereotypes we have been served recently.  There is also - most importantly - a significant proportion of screen time dedicated to the Transformers, which is ironic since this is the first film in tbe franchise without "Transformers" in the title.

I highly recommend this film, for original Transformers fans who are old enough to have children and for their children too (who will enjoy the movie just as much, but for entirely different reasons).  10/10.

2018: A first time for everything

Having recently passed 40, I thought I was done with firsts. I was wrong; this year has been a year of many firsts. You won't find my highlights on Facebook (I gave up Facebook for Lent, and all social media for September), so I've compiled some firsts here.

2018 saw the first time I played with my church's music group at a funeral. I've played at a number of weddings and christenings, which were joyful occasions. It was a challenge and an honour to be asked to play at a funeral - I had to "put my feelings in my pockets" for the duration of the service, and let them out afterwards. Not only was 2018 the first time I played at a funeral; it included the second time too.

I owned a BigTrak for the first time ever in 2018.  It was a very short association; the new 21st century BigTrak is actually underpowered compared to its 1980s predecessor (it runs off one less battery) and thus means it doesn't turn with the same level of accuracy or reliability. An instruction to turn 90 degrees ends up somewhere between 65 and 75 degrees, so there's no way to accurately program a square path. I sent it back barely 30 minutes after opening it. They say you should never revisit your childhood heroes; perhaps they were right.

A major highlight for me was seeing Michael W Smith live for the first time, at the Festival of Hope in Blackpool in September.

I've enjoyed his music and bought his albums (on cassette, even) for over 20 years, but never previously seen him live, since he doesn't cross the Atlantic all that often and I've never seen the dates in advance.  Seeing him live was well worth the wait; they say you should never meet your heroes - they are totally wrong.

 

Earlier in the year, spring 2018 saw the first time taking the family to see the Red Arrows.  I've seen them dozens of times before, but the Armed Forces Day in Llandudno was the first occasion where the Leese family en masse attended an air display. 

The trip was very successful, even if it didn't go entirely as planned: one of our children got bored partway through the the Red Arrows' display and opted to walk down to the waterline and throw stones into the sea instead, while another fell asleep in the lull between the Red Arrows and the Typhoon display, and had to be provided with ear defenders to help shut out the noise and stay asleep.

I created an account on Soundcloud for the first time this year. It isn't getting much attention, but I am uploading a few miscellaneous tracks to it (all my own work). It's 20 years since I bought my first keyboards, and I've finally decided it's time to upload some of the music I've produced for the wider world.  I've included some additional voice talent in my most recent recordings, which means there's a whole list of firsts: first shout-out to find voice actors; first online auditions; first recordings; and so on.

Unfortunately, 2018 has seen me start taking immuno suppressant drugs, as I have been diagnosed with psoriatic arthritis  (a form of rheumatoid arthritis). Back in January, I had severe pain in my left foot, at the base of my toes. After an initial diagnosis of tendonitis that didn't improve, I was eventually referred for blood tests and now attend the Rheumatology department of my local hospital every month for progress checks. The drug I'm on - methatraxate - seems to be working very well, with very few side effects (except occasional bouts of can't-be-bothered, and sometimes one day a week where I have almost no energy). I've also had the chance to see ultrasounds on my hands and feet, which were fascinating.

Summer 2018 was the first time I've volunteered to help at my church's summer club.  In fact, it's the first time I've volunteered to help at any summer club ever.  It was an exhausting three-day club, and somehow I managed to fit my normal work around it as well!  It was a great opportunity to 'give more than I receive' - I was completely drained by the end of it. It was a great experience, volunteering alongside a team of amazing people and sharing the good news of Jesus with dozens of children, and I'm already looking forward to next year's club!

I am not as 'technical' as people think, and I really don't know how to fix your computer. Truth be told, I'm not sure how to fix my own. But when our laptop started beeping incessantly, it fell to me as the most technical member of our household, to fix it. Short answer: I had to dismantle most of the laptop to get to the CMOS battery (roughly the size of a 10p) and replace it. Successfully. On the second attempt. First time performing laptop surgery - check!

All in all, 2018 has been an interesting year. I've done many new things; some new things have happened to me, and it's been a surprising year of firsts.  I intend to keep on growing and doing more firsts next year.

The Lists of Firsts

A first time for everything: 2018
First times in 2021 list
First times of 2022
First times in 2023
Things I did for the first time in 2024