Thursday, 22 June 2017

The General Election (Inferences from Quantitative Data)

The Election

The UK has just had a general election: all the government representatives who sit in the House of Commons have all been selected by regional votes.  The UK is split into 650 areas, called constituencies, each of which has an elected Member of Parliament (MP). Each MP has been elected by voting in their constituency, and the candidate with the highest number of votes represents that constituency in the House of Commons.

There are two main political parties in the UK - the Conservative party (pursuing centre-right capitalist policies, and represented by a blue colour), and the Labour party (which pursues more socialist policies, and represented by as red colour).  I'll skip the political history, and move directly to the data:  the Conservative party achieved 318 MPs in the election; the Labour party achieved 262; the rest were spread between smaller parties. With 650 MPs in total, the Conservative party did not achieve a majority and have had to reach out to one of the smaller parties to reach the majority they require to obtain a working majority.

Anyway:  as the results for most of the constituencies had been announced, the news reporters started their job of interviewing famous politicians of the past and present.  They asked questions about what this meant for each political party; what this said about the political feeling in the country and so on.

And the Conservative politicians put a brave face on the loss of so many seats.  And the Labour politicians contained their delight at gaining so many seats and preventing a Conservative majority.

The pressing issue of the day is Brexit (the UK's departure from the European Union).  Some politicians said, "This tells us that the electorate don't want a 'hard' Brexit [i.e. to cut all ties completely with the EU], and that they want a softer approach." - views that they held personally, and which they thought they could infer from the election result.  O
thers said, "This shows a vote against austerity,"; "This vote shows dissatisfaction with immigration." and so on.

The problem is:  the question on election day is not, "Which of these policies do you like/dislike?" The question is, "Which of these people do you want to represent you in government?"   Anything beyond that is guesswork and supposition - whether that's educated, informed, biased, or speculative.

Website Data

There's a danger in reading too much into quantitative data, and especially bringing your own bias (intentionally or unintentionally) to bear on it.  Imagine on a website that 50% of people who reach your checkout don't complete their purchase.  Can you say why?

- They found out how much you charge for shipping, and balked at it.
- They discovered that you do a three-for-two deal and went back to find another item, which they found much later (or not at all)
- They got called away from their computer and didn't get chance to complete the purchase
- Their mobile phone battery ran out
- They had trouble entering their credit card number

You can view the data, you can look at the other pages they viewed during their visit.  You can even look at the items they had in their basket.  You may be able to write hypotheses about why visitors left, but you can't say for sure.  If you can design a test to study these questions, you may be able to improve your website's performance.  For example, can you devise a way to show visitors your shipping costs before they reach checkout?  Can you provide more contextual links to special offers such as three-for-two deals to make it easier for users to spend more money with you?  Is your credit card validation working correctly?  No amount of quantitative data will truly give you qualitative answers.

A word of warning:  it doesn't always work out as you'd expect.

The UK, in its national referendum in June 2016, voted to leave the EU.  The count was taken for each constituency, and then total number of votes was counted; the overall result was that "leave" won by 52% to 48%.  

However, this varied by region, and the highest leave percentage was in Stoke-on-Trent Central, where 69% of voters opted to leave.  This was identified by the United Kingdom Independence Party (UKIP) and their leader, Paul Nuttall, took the opportunity to stand as a candidate for election as an MP in the Stoke-on-Trent Central constituency in February 2017.  His working hypothesis was (I assume) that voters who wanted to leave the EU would also vote for him and his party, which puts forward policies such as zero-immigration, reduced or no funding for overseas aid, and so on - very UK-centric policies that you might imagine would be consistent with people who want to leave a multi-national group.  However, his hypothesis was disproved when the election results came in:

Labour Party - 7853
UKIP (Paul Nuttall) - 5233

Conservative Party - 5154
Liberal Democrat Party - 2083

He repeated his attempt in a different constituency in the General Election in June; he took 3,308 votes in Boston and Skegness - more than 10,000 fewer votes than the party's result in 2015.  Shortly afterwards, he stood down as the leader of UKIP.

So, beware: inferring too much from quantitative data - especially if you have a personal bias - can leave you high and dry, in politics and in website analysis.

Wednesday, 31 May 2017

Collatz Conjecture: 3n+5






Friday, 28 April 2017

Collatz Conjecture Revisited (part 2): 3n+3

I've previously looked at the Collatz conjecture, (3n+1) and I have revisited it before, too (5n+1).  Now, I would like to revisit it again.

The Collatz Conjecture states that when you take a number, and if it's even then divide by two, or if it's odd then multiply by three and add one, then you will eventually reach 1.  There's no proof (yet), but it holds for all numbers that have been tested.

I extended this in a previous post, and looked at the case of multiplying by five (instead of three) and adding one, and identified two loops and a growing series.

In this post, I will share my findings on another alternative, which is "3n+3".  [3n+2 doesn't work, since if n is odd, then 3n + 2 will also be odd].


3n+3 has one loops which covers all numbers I have tested.

The simple loop/termination is [1], 6, 3, 12, 6, 3, 12.

There are various ways into this loop, in particular,

10, 5, 18, 9, 30, 15, 48, 24, 12, 6, 3, 12 etc.
7, 30, 15, 48 etc.
11, 36, 18, 9, 30, 15, etc.
13, 42, 21, 66, 33, 102, 51, 156, 78, 39, 120, 60, 30, 15, etc.

Interestingly, many of the starting numbers reach a common maximum value of 27696 before coming back down to 1.    This is first seen for an initial n=53.

For larger values of n, there is a longer sequence.  The graph below shows the maximum value of n (vertical axis) for different start values of n (between 101 and 241, as example material).  Note how 27696 predominates as the largest value reached.

The sequence from 27696 is:

27696, 13848, 6924, 3462, 1731, 5196, 2598, 1299, 3900, 1950, 975, 2928, 1464, 732, 366, 183, 552, 276, 138, 69, 210, 105, 318, 159, 480, 240, 120, 60, 30, 15, 48, 24, 12, 6, 3

27696 is seen for the following initial values of n:

53, 61, 81, 93, 107, 109, 123, 125, 141, 145, 163, 165, 181, 187, 189 (and others).

For values above 27696
I have not explored extensively above 27696, but there is a cluster of initial values that have the same new peak.  The cluster is around 27754:  27754, 27755 and 27757 all have the same maximum, which is 2026128.  The highest peak I have observed so far is for 27729, which reaches a height of 2698752.

To close, the full sequence for 27729 is:

27729, 83190, 41595, 124788, 62394, 31197, 93594, 46797, 140394, 70197, 210594, 105297, 315894, 157947, 473844, 236922, 118461, 355386, 177693, 533082, 266541, 799626, 399813, 1199442, 599721, 1799166, 899583, 2698752, 1349376, 674688, 337344, 168672, 84336, 42168, 21084, 10542, 5271, 15816, 7908, 3954, 1977, 5934, 2967, 8904, 4452, 2226, 1113, 3342, 1671, 5016, 2508, 1254, 627, 1884, 942, 471, 1416, 708, 354, 177, 534, 267, 804, 402, 201, 606, 303, 912, 456, 228, 114, 57, 174, 87, 264, 132, 66, 33, 102, 51, 156, 78, 39, 120, 60, 30, 15, 48, 24, 12, 6, 3

I shall continue to explore 3n+3, and also too compare the data and sequences with 3n+5.

Thursday, 16 March 2017

Average Time Spent on Page

The history of Web analytics tools has left a legacy of metrics that we can obtain "out of the box" even if they are of no practical use, and I would argue that a prime candidate for this category is time spent on page, and its troublesome partner average time spent on page. It's available because it's easy to obtain from tag-fires (or server log files) - it's just the time taken between consecutive page loads.  Is it useful? Not by itself, no. 

For example,  it can't be measured if the visitor exits from the page. If a user doesn't load another page on your site, then there are no further tag-fires, and you don't get a time on page.  This means that you have a self-selecting group of people who stayed on your site for at least one more page.  It entirely excludes visitors who immediately tell they have the wrong page and then leave. It also, sadly, excludes people who consume all the content and then leave. No net benefit there, then.

Worse still, visitors who immediately realise that they have the wrong page and hit the back button are included.  So, is there any value to the metric at all?  In most cases, I would argue not, although there can be if handled carefully. For example, there is some potential benefit in monitoring pages which require data entry, such as checkout pages or other forms. In these circumstances, faster is definitely better, and slower suggests unnecessarily complicated or lengthy. For most shopping pages, though, you will need a much clearer view of whether more time is better or worse. In an international journey, four hours on an airliner is very different from three hours in an airport.

I mentioned that time on page is not helpful by itself: it can be more informative in conjunction with other metrics such as exit rate, bounce rate or revenue participation. For example, if a page has a high exit rate and high time on page, then it suggests that only a few people are finding the content helpful and are prepared to work through the page to get what they want - and to move forwards. Remember that you can't draw any conclusions about the people who left - either they found everything they needed and then left, or they gave up quickly (or anything in between).

So, if you use and quote average time on page, then I suggest that you make sure you know what it's telling you and what's missing; that you quote it in conjunction with other relevant metrics, and you have decided in advance if longer = better or longer = worse.

Friday, 24 February 2017

Ten Things I learned from not quite reading the Bible in A Year

My 2016 sort-of New Year's Resolution was to read the Bible in a year. I've not tried it before - and it wasn't until the start of February that I decided to go for it. I'd read all of Genesis in 2015, so in order to catch up, I started with Exodus. The church I attend was providing reading plans showing that it was possible, and what to read each day. I started February by keeping track and ticking boxes, but then I lost my reading plan in the middle of May 2016 - and carried on anyway. I didn't quite manage it (it's late January 2017 and I'm still in 1 Corinthians), but here's what I learned.

1. Reading the Bible in a year is a lot like running the four-minute mile.  It's likely that you'll be so busy trying to read your daily quota and trying to keep up the pace that you won't have much time to think and you certainly won't get chance to smell the roses.  It's relentless, and if you start slowing down, you'll need to up your pace for the following days just to keep up. If you want to read, consider, ponder and meditate on the Bible, then you're going to have to go more slowly.

2.  If you read the Bible chronologically in a year, you're in for a tough ride. If you read it sequentially (from cover to cover, like I did) it's going to be challenging.  For a start, you won't meet Jesus in the flesh until September or October.  That's a long time.  You'll get plenty of hints and clues about him, but he doesn't arrive until the last quarter of the year. Hang on in there.

3.  It's more fun reading it when we're winning.  Moses, Joshua, Saul, David, Solomon, Gideon and so on - all big winners, and all interesting and easy to read.  It's definitely easier than some of the later stuff, when it's doom and gloom; exiles and punishment; warnings, wrath and judgement.  The narrative of the early Old Testament is more straightforward than some of the symbolic stuff that comes later on.

4.  April will feel like a repeat of March as you read 1 and 2 Chronicles after 1 and 2 Samuel and 1 and 2 Kings.

5.  There are spoilers all the way through the Old Testament.  The prophets persistently warn about desolation, destruction and devastation, but they almost all break off briefly to say, "God's going to send somebody to fix all your mess."  Keep reading - it's worth it to find these snippets, and they'll keep you going through Lamentations, and the last few minor prophets.

6.  Some of it is downright confusing (even the Old Testament).  When they aren't warning God's people, or telling them to repent, or reassuring them that God will send somebody to help, some of them even have visions of heaven, or of the distant future.  You thought the living creatures and the elders were only in Revelation?  So did I, until this year.

7.  Your speed through the Bible will probably change dramatically, depending on where you are.  It's possible to "read" large parts of Numbers very quickly - 132,000 of this tribe, 89,000 of that tribe, and so on. Remember how I said reading the Bible in a year is like running a four-minute mile?  Well, this mile-long track is not flat; you'll definitely run different parts of it at different speeds.

8.  When you do reach the start of the New Testament, it'll feel like a breath of fresh air.  Yes, there's the genealogy stuff that we had in the early parts of the Old Testament, but it gets going again - and you'll slow down again. There's just so much happening in each chapter. And when I finally started picking up the pace (through Acts), I slowed down again (Romans, with its exceptionally long, parenthetical, phrases, and therefore I couldn't go as quickly - or as rapidly, therefore - as I could through some of the other books).

9.  On the subject of the New Testament: one advantage to reading sequentially is that you start to notice themes and patterns that you wouldn't spot if you were just reading sections or passages.

For example:  Matthew's gospel frequently refers to "your heavenly Father", which is a very welcome change after God Above in the Old Testament.  There really is a massive change of tone between Jesus and the prophets who came before him - Jesus really knows about heaven.  Mark - immediately.  Jesus is quite clearly in a hurry.  Everything is immediately.  Matthew spends a few chapters with Jesus' genealogy, birth and childhood.  But Mark?  "This is the good news about Jesus: this is what he did first", and Jesus has carried out miracles by the end of the first chapter.   Luke - I didn't find anything specific in Luke, but John is all about "eternal life."  Inheriting it, getting it, having it.  I don't remember reading much about eternal life in the other gospels, so John is clearly compensating :-)

10.  Regular reading works.  I try to read at the same time each day, after the children have gone to bed and are settling down to sleep.  This works providing they sleep at the same time, and the rest of the daily routine works.  I didn't read as much when we were on holiday, and I didn't read as quickly when they stayed up late or didn't go to sleep as usual.  If you can find a fixed, regular time, that'll probably work better.

It's now late January, and as I said, I'm working through 1 Corinthians, with a view to completing the New Testament by the end of February... the finish line is in sight!  This year, I'm taking a far more measured approach - when I've finally completed the New Testament, I'm just going to loop round the gospels for a bit.  The aim of reading the Bible in a year isn't really to read all the pages in 365 days.  It isn't even to read all the pages eventually (although it's a worthwhile aim).  It's to learn to read the Bible regularly.  It's not about running a four-minute mile, it's about building strength and keeping fit.

Film Review: Wreck-It Ralph

"My name's Ralph. And I'm a bad-guy. I'm 9' tall, weigh 643 lbs... And I'm a wrecker. I wreck things. Professionally.  The problem is, fixing things is the name of the game.  Literally: Fix-it Felix Jr. When Felix does a good job, he gets a medal.  But are there medals for wrecking stuff really well? To that, I say "Ha!""

Ralph is a bad-guy with a good heart. Or, as he and his bad-guy friends point out: "I am a bad-guy, but that does not make me a bad guy."  However, in arcade world, computer game villains are stereotyped as evil and dangerous, and Ralph has to overcome this prejudice to become a hero and earn a medal - this becomes his mission in life. He wants to be accepted by the other characters in his game, and show everyone that there's more to bad guys than being bad.  An early exchange between Ralph and the other characters in his game makes it clear that they tolerate him, but nothing more.

"There's no room for you up here... 
Only good guys win medals. And you are no good guy. You're just a bad guy who wrecks the building!"
"No, I'm not."

Having realised his need to win a medal in order to be seen as equal by his contemporaries,  Ralph quickly works out how to get a medal, and successfully wins one in Hero's Duty - another game in the arcade, a first-person shooter set on an alien planet  [
"I thought this was going to be like Centipede. When did computer games become so violent and scary?"].  However, shortly after his heroic achievement, he's accidentally launched into another game in the arcade, where he has to win his medal all over again.

The story is very good, and moves along well, but the best parts of Wreck-It Ralph are the arcade game references (there are many) and the amazing visuals.  Each game and each group of characters have their own visual identity and their own soundtrack. The characters in the arcade classic Fix-It Felix Jr are rendered as slightly jerky 3D versions of chunky square sprites, which jump and bounce around to a classic 8-bit synth tune (and Fix-It Felix's steps and jumps are accompanied by quick riffs, exactly as you'd have heard them in the arcade, even outside his own game).  Hero's Duty is a darker, modern game with sharply-rendered 3D characters, while Sugar Rush Speedway has its own happy soundtrack. 

Wreck-It Ralph takes on an almost completely unique perspective of an arcade game character who knows he's an arcade game character. The one exception I can think of is a book I read at primary school, called Colin's Fantastic Video Adventure... back in the 1980s, but that's nothing compared to this.  The story makes full use of its premise of self-aware game characters, populating its universe with a wide array of game characters - have fun spotting them all - and the interactions between the characters from the different games is what really makes the movie.

Like a 3D view of its 2D characters, Wreck-It Ralph has surprising depth.  I bemoaned and criticised Pixels for its lack of depth and its poor treatment of women; I have no such complaints about Wreck-It Ralph. The two main female characters are both well written and portrayed with surprisingly well thought-out personalities and histories  (presented very quickly in flashbacks).  They contribute significantly to the plot and to Ralph's story, and the relationship between Fix-It Felix Jr and Sergeant Calhoun is thoughtfully written ("Your face is red, you might want to hit it with your hammer."  "Oh, that's not blunt force trauma, ma'am. That's just the honey glow in my cheeks.").

So: I genuinely like Wreck-It Ralph. It's a good take on the computer game theme; it knows its source material, respects it and expands on it. It has great visuals, clever plot and is very funny; I highly recommend it. 

Wednesday, 1 February 2017

Film Review: Pixels

Image credit: IMDB
Loud; brash; subtle-as-a-sledgehammer: 1980s arcade games. And, to be fair, "Pixels", the Adam Sandler movie. If, like me, you grew up on classic video games (not Halo or Call of Duty) like Defender, Pac-Man and so on, then you'll enjoy this film.  The content is 1980s video games, made larger-than-life and blasted all over the planet, and the style is also very much 1980s video games.
The story is full of all the clich├ęd characters, including - the cluster of nerds, the "snobby" love interest, the British military officer, the cannon fodder, the wizened old general who represents "the way things have always been done" and who doesn't get this new-fangled technology.  The film even goes so far as to actively introduce them.  For example, the Adam Sandler character, Brenner, is a nerd herder in the style of Chuck Bartowski, but lacking any of Chuck's charisma:

"Hello.  I am a nerd from the Nerd Brigade.  Here to nerd out on all your audio and visual needs."
"Do you have to say that every time you turn up at a house?"
"If I want to get paid, yes."
"Isn't that kind of demeaning?"
"Only if someone brings it up."

The love interest is in the form of one of Brenner's customers, Violet Van Patten, who - it transpires - is also a senior officer in the US army.  She doesn't fully understand 1980s computer games, but she is prepared to go along with him anyway - eventually.  There isn't much plot or character development to speak of: this is basically a Hollywood mash-up of the best computer games of the 80s exploded into the real world, and viewers will watch in order to identify all the old games blasted across the big screen, and to get all the references.  The story that supports the video-gaming roadshow is very straightforward:  arcade game experts (rendered as has-beens, with the exception of the President of the US - who for his part is inept at his job and can't read or count properly) take on a bunch of aliens who misunderstood 1980s video game footage as a declaration of war and have decided to retaliate.  Or start a war.  Their motivation is a little unclear. 

Anyways, after the declaration of war, the arcade experts have to outgeek the aliens, and in so doing, become recognised as national - no, global - heroes.  After all, all geeks are actually really sensible people who just want their chance at success and fame, even if it takes interstellar war to achieve.  In the real world, playing arcade games probably won't make you famous, but this is Hollywood, and hey, it's harmless enough fun to try and be the best at the latest version of electronic entertainment on the off-chance you might get recognised in the wider world for it.  (Back in the 1980s, the way to get famous with electronic entertaionment was by was high-scoring on video games; nowadays it's blogs, social media or YouTube channels.  Like and subscribe, folks ;-) ).

The unprepared humans, their hardware and buildings get beaten into cubic pixels by the first few waves of alien attacks - Galaga and Breakout - but get their act together and start repelling the next attack in the form of Centipede.  The nerds have been training the regular military, but this approach is not working, and Brenner has to take a light cannon and show the army how it's done.  He tells his friend Ludlow to join him in defending the planet, but the army refuse to give weapons to civilians.  Violet refers the question to the President:

Violet:  "Mr President?"
President:  "Let the nerds take over."
Violet (to the General): "Let the nerds take over!"

And they do.  The Centipede scene is where the film got my attention - after all, nobody has ever previously taken 2D sprites and made them several hundred feet tall, close up and in full glorious HD colour and in 3D.  The humans have weapons that are basically real-world equivalents of the weapons in the original games, and the nerds acquit themselves well (who knew that they were practising for this all those years ago in the arcades every weekend).
The rest of the film is a series of games played in various cities - the Pacman in New York is a highlight for me - with the establishment giving them increasing support, and the love interest slowly warming to our hero.  As the situation with the aliens escalates, our heroes face greater challenges and receive greater adulation from the rest of the world.  Are you ready for Level 2?

I have to say that the treatment of women in this movie is a reflection of the treatment of women in 1980s computer games:  arcade games then were predominantly played by boys, and the characters in the games were almost always men. There are two main female characters:  Brenner's love interest, Lt Colonel Violet Van Patten (who we first meet having an emotional breakdown in her wardrobe), a senior military advisor to the president (played straight-laced but fragile), and later we also meet the love interest for Ludlow (one of Brenner's friends, played by Josh Gad).  Ludlow has been fantasising about a female character from one of the 1980s games, named "Lady Lucy".  Lady Lucy wears a revealing, bright red costume and wields two swords.  Sadly, I think she only wields two lines of dialogue in the whole film, too; this may be the 21st century, but the source material is definitely late 20th.

Image credit: Pixel Dynamo
There are a few twists in the plot along the way - one of the nerds is exposed as a cheat - but as I said earlier, you're probably not going to watch this film for the main plot points (there are some, and they carry the story along well in between the various battles).  You'll be watching for the special effects; the re-interpretation of classic computer games on a huge scale.  Think Independence Day but with lasers on both sides.  

Pixels is a lot of fun - I honestly enjoy watching it, (the flaws in the film and the chronic stereotyping only really occurred to me after a few viewings) but watch it most for the visuals and to try and spot some more arcade game references.  If you're looking for a more thoughtful treatment of retro arcade games (and, to be fair, you probably won't find one that's less thoughtful), then I can recommend reading Ready Player One (there's a film in planning) or Armada by Ernest Cline, or even the Disney movie Wreck-It RalphRalph may not be any more intelligent, but it's more sensitive and has a moral that isn't "One day, playing computer games might save the planet and get you the girl", which is mostly what we have here.  I like Pixels - it's in-your-face, loud, bright and brash - but it could've been much better if it had been slightly more intelligently written.