Thursday, 6 October 2011
The Emerging Role of the Analyst
It was not that long ago that Internet Profiles Corporation (I/PRO) launched the first ever commercial log file analyser, LogAnalyzer back in June 1994, and the web analytics industry was born. Since then, technology has been advancing at a phenomenal pace. Finding the software and the staff to keep pace with it, and to keep measuring it, has become increasingly challenging. After all, software designers, web designers and JavaScript programmers rarely think about how usage of their technology can be measured when they develop their products; they’re far more interested in beating the competition and providing the best visitor experience (hopefully doing both at the same time).
This means that the role of the analyst has had to come a long way, and there have been various pitfalls to avoid along the way. For example, I/PRO went through mergers and takeovers along its way, and almost exactly 10 years after it was founded, I/PRO was declared insolvent. Hopefully most analysts will last longer than that (I/PRO was saved by another buyout, and its history has become lost in various name changes and mergers). One of the key pitfalls for current website analysts is to avoid being just a counter and a report.
Back in the mid-1990s, when website analysis was in its infancy, the analyst was probably just a counter and a reporter, and that was all that was required of the role - in itself, converting log files into meaningful data, and then information, was hard enough. The ‘analysis’ came from making sense of log files and translating them into counts and numbers, and was much more connected to computer analysis. To an analyst in the 1990s, qualifications in computer science would be more beneficial than those in marketing or mathematics, and the ability to assimilate large volumes of computer reports would be more use than intuition and curiosity.
However, with time, other players entered the website analytics software market, and the skill set required to be a website analyst started to shift, as the mathematical and log-file heavy lifting was moved from staff to software. Staff – i.e. people – were hired to translate the numbers into words and pictures (as business will always love a good graph) instead of converting the bits and bytes into numbers. More demands were placed on the staff, who became able to do the number-crunching instead of the log-file crunching.
We’ve seen a technological revolution, where the software providers have had to keep pace with business requirements, and business requirements have expanded to exploit the expanding capabilities of new web analytics software. Log file analysis was replaced with 1x1 transparent gifs, then JavaScript tags. We used to put counters at the bottom of our pages, and we would count hits and page views, now we count visits, visitors, and analyse click paths, and following integration with other data sources, we even measure revenue and sales.
Depending on your perspective, the role of the website analyst has now emerged from the shadows of a counter and reporter to a business-critical source of data and recommendations (in terms of business role); or from the computer science engineer to a data-savvy marketer and key decision influencer. We’re able to represent the website data in an understandable way – either by writing reports, e-mailing recommendations or presenting our analysis face-to-face. We can up with ideas for testing, with supporting hypotheses. The role of the analyst has emerged from a role of supporting existing projects (“This is what we’re going to do, see if it works”) to being a key consultant at the planning and delivery stages (“What do you think we should do here?”). The questions have moved from closed questions to open questions, as web professionals have become more open-minded, and analysts have been able to get onto the front foot more often.
It reminds me of a scene in an episode of Star Trek Voyager (“Relativity”). One of the characters (“Seven”) consults the medical database and decides, after reviewing her syptoms, that she has a serious, degenerative illness, and consults the Doctor to explain her reasoning. The Doctor is a hologram, programmed to identify and cure sickness and disease in the crew, based on the data in the medical database. Seven and the Doctor then go through her symptoms, and the Doctor explains what he finds to be the problem, and gives her the necessary treatment, with an immediate improvement. He returns to his sickbay, and, as he leaves, says, “Remember, next time your human physiology fails, you don’t consult the database; just call me.” “You are the database.” “With two legs and a bedside manner!” As analysts, we’re often called on to represent our data, to make sense of it and to turn it into words. That, for me, is the emerging role of the analyst – from the number cruncher who follows along behind the web team, to a leading position in informing future decisions about a website.
This article has been written as part of the first ever web analytics Blogarama, with analysts sharing their views on 'the emerging role of the analyst'.
Tuesday, 4 October 2011
Film titles: X of the Y
X of the Y film titles
I watch a lot of films. Not vast numbers, and I'm not a film expert, or a film critic (despite writing articles full of criticisms of films - that's another story), but I've noticed an interesting or peculiar trend in the names of some films.
Return of the Jedi
Revenge of the Fallen
Revenge of the Sith
Dark of the Moon
Planet of the Apes
The Night of the Hunter (I haven't seen it, but apparently it's a thriller)
Pirates of the Caribbean
Even, historically, "Night of the Living Dead". I haven't seen it - nor do I want to - but it's included in this rather strange list of films which has the title form "X of the Y". I started thinking of the films I've seen, and then, in addition I've had a glance through various lists of top 100s or top 500s, to see if I could find many more. Funnily enough, most of them are science fiction - the majority of "normal" films (as various people would call them) have "normal" titles. So, my question is: Why do they have the title format "X of the Y" and not "The Y's X". Why not have, "The Apes' Planet" and "The Caribbean Pirates" or "The Sith's Revenge"? I suppose there are various reasons, and I'm considering the following. It could be:
* a bad case of apostrophobia (people not knowing where to put the apostrophe),
* creation of uncertainty in the title (is there one Y, or more than one?)
* a literal translation from a foreign language,
Is it a bad case of apostrophobia? No, it's not a real word, but it appears to me that an increasing number of people don't know how to use an apostrophe, and so they leave it out completely, or, in the case of film title writers, they rewrite their prose to avoid the need for one. How do you punctuate the straight version of "Revenge of the Fallen"? Is it "The Fallen's Revenge" or "The Fallens' Revenge" or do you go for "The Fallen Revenge" and completely lose the 's' and the apostrophe? In each case, the meaning is slightly different - it's also explicit in each title that there is either one Fallen, or more than one, or in the last case that the Fallen has moved from being a noun (stating what it is) to an adjective (describing the type of revenge). In this case it's also a little clumsy. As far as the film goes, the accurate title would be first one (there's only one Fallen), and the second one looks clumsy.
The Star Wars films benefit from having Jedi and Sith which are single and plural terms - one Jedi, many Jedi, one Sith, many Sith - so there's still a degree of uncertainty in how many Jedi and Sith are involved when you move to the straight form: "The Jedi's Return", "The Sith's Revenge" and in fact "The Jedis' Return" would be incorrect. However, "The Night of the Living Dead" is another where the Y is a plural term (I am assuming there's more than one of them, having successfully dodged the film so far), but "The Living Dead's Night" sounds equally menacing to me.
So perhaps it really is a case of developing uncertainty in the title, rather than explicitly stating that there's one or more of the Ys, which is my second suggestion. The Revenge of the Sith - we really don't know how many there are - is it one, two, six or ten? The Revenge of the Fallen has this benefit - as mentioned above, is it one character called the Fallen, or is there a group of fallen characters (which would be more likely)?
"The Dark of the Moon" is a good example of this: the straight version would be "The Moon's Dark" which sounds like the obvious statement that the Moon is dark. The form used in the film title makes it clear that it's the Dark belonging to the Moon; this film title is unusual because dark isn't usually a noun, but an adjective - more mystery and uncertainty, which works well for the film.
The other option - a literal translation from a foreign language - is unlikely, since most films aren't taken from e.g. French storylines, although it does fit. For example, "le jardin des enfants" translates literally as The Garden of the Children, but would be written as the Children's Garden (another plural). Or how about "La maison de l'homme grand" which is The House of the Large Man... okay, it's a poor example, but it also shows the power of the plural group - compare that to the House of the Large Men. No? Perhaps not.
So, it seems we're going to be stuck with X of the Y - especially, as my list shows, in the science fiction genre, where the deliberate ambiguity that's created by the long form outweighs the simplicity of the Y's X form. An extension of this is to sequels, where you could have Z of the X of the Y, for example, The Rise of the Planet of the Apes, which is worth double points.
Are there any others? Are there any more Z of the X of the Y in particular?
Thursday, 8 September 2011
Probabilities and Free Toys, Part 2
Last time, I looked at solving a probability question: for a set of 3, 4, 5 or n toys which are given away free inside a cereal packet, what's the probability of obtaining the full set of toys after buying the same number of packets (e.g. for 5 toys, getting all 5 after buying 5 packs).
This time, having solved the easier question, let's take a look at the harder question: with 10 toys (or a number larger than three or four), how many packs do I have to buy to be 50%, 70% or 90% sure of having the full set?
Once again, let's start with two toys (start small!):
After buying two packs, the probability of success is 0.5. The two successful combinations are AB and BA, and the two unsuccessful are AA and BB (i.e. I get the same toy twice).
After buying three packs, the probability of success rises to 6/8, which is 0.75 or 75%.
There are eight total combinations (from AAA, AAB through to BBA and BBB), but only two are unsuccessful (AAA and BBB are unsuccessful), leaving six successful combinations.
After buying four packs, the probability of success rises to 7/8. There are 16 total combinations, but the number of unsuccessful combinations remains at two, and the number of successful combinations rises to 14.
Generally, for the two-toy problem, the probability of getting a successful combination after t turns is equal to (2^t - 2) / 2^t or, in words, the total number of combinations minus unsuccessful, divided by the total number of combinations.
This, however, is where it gets tricky. With three toys and three packs, there are just six successful combinations (the exact permutations, in alphabetical order, are ABC, ACB, BAC, BCA, CAB and CBA) and 27 total combinations (which confirms what I proved above for the three toys and three packs case). With three toys and four packs, the number of successes rises to 36, and the total number combinations is 81. The success probability is 36/81 which is 4/9, coincidentally double the probability after three packs. After five packs, the number of combinations rises to 243.
Looking at the number of combinations I've seen so far, for the three-toy problem, it's risen from 9 to 81 to 243, rising by a factor of 3 each time, and n=3, the number of toys squared. In fact the denominator, the total number of permutations, is simply n^t (number of toys raised to the power of the number of turns or packs). This is also known as the formula for 'permutations with repitition'.
Now I need to identify the number of successful permutations - those that contain one of each of the toys.
For n=3 toys and t=3 turns, there are 6 successful permutations (as listed above).
For n=3 toys and t=4 turns, there are 36 successful permutations
Now, the question becomes - how many will there be after five turns?
Let's go back to four turns and look closely at each of the permutations we have. We have the 36 successful ones, ABCA, ABCB, AABC and so on. For these 36 successes, it doesn't matter what we pick next, we'll still have a successful combination; there are three options for each of these, so that's 36x3=108 successes for each of them.
There are also the 3 combinations AAAA, BBBB and CCCC which will not produce a success, irrespective of what we pick next, so that's 3x3=9 fails.
This leaves the rest, which logically must contain two different toys. We've covered the ones which already contain three different toys, and we've looked at the ones which contain only one toy. Since there were 81 combinations in total after four goes, there must be 81-(36+9) = 36 combinations which contain two different toys. There is a probability of 1/3 of the next choice being the correct one, which equals 36 x 1/3 = 12.
So, after five turns, we have 108 + 12 = 120 successful permutations.
Let's review:
After three turns: 6 successful permutations
After four turns: 36 successful permutations
After five turns: 120 successful permutations
Let's take a look at six turns, based on the process we used for five turns.
After five turns, we have 120 successes which will each yield three more successes, so 3x120 = 360
We also still have the combinations which have just one toy in - AAAAA, BBBBB and CCCCC, and these will each produce three more unsuccessful combinations, irrespective of the next choice. 3x3=9 unsuccessful combinations.
There are 3^5 total combinations after five turns, 243 in total, which means that we have 243-(120+9) = 114 other combinations which have two toys. A third of these will become successful with the sixth turn, which is 114/3 = 38.
So, after six turns, we have 360 + 38 = 398 successful permutations. I've deduced the formula for working out successful permutations in an iterative manner, but I don't have the computing power to determine the 10th, 15th or 115th term without knowing each one before. Furthermore, this method won't easily expand to cover five, six, or ten toys. It's all about knowing how many successes you've had previously, and how many certainly won't become successful (because they have n-2 different toys and will require at least two more goes to become successful).
Three toys is an easy case - you either have a successful combination, a combination with only one toy repeated, or a two-toy combination with a 1/3 chance of becoming successful. With four toys, you may already have one, two or three different toys, or be successful, and I don't quite see how to sort all that out.
So, next time, it's on to spreadsheet modelling. I'm going to write a macro that simulates buying the cereal packets and examining the toys, and determining whether or not the combination is a success. If maths fails, use sampling!
Part 2: Solving for "How many packs do I need to buy to be 50%, 70%, 90% sure of getting all the toys?"
Once again, let's start with two toys (start small!):
After buying two packs, the probability of success is 0.5. The two successful combinations are AB and BA, and the two unsuccessful are AA and BB (i.e. I get the same toy twice).
After buying three packs, the probability of success rises to 6/8, which is 0.75 or 75%.
There are eight total combinations (from AAA, AAB through to BBA and BBB), but only two are unsuccessful (AAA and BBB are unsuccessful), leaving six successful combinations.
After buying four packs, the probability of success rises to 7/8. There are 16 total combinations, but the number of unsuccessful combinations remains at two, and the number of successful combinations rises to 14.
Generally, for the two-toy problem, the probability of getting a successful combination after t turns is equal to (2^t - 2) / 2^t or, in words, the total number of combinations minus unsuccessful, divided by the total number of combinations.
This, however, is where it gets tricky. With three toys and three packs, there are just six successful combinations (the exact permutations, in alphabetical order, are ABC, ACB, BAC, BCA, CAB and CBA) and 27 total combinations (which confirms what I proved above for the three toys and three packs case). With three toys and four packs, the number of successes rises to 36, and the total number combinations is 81. The success probability is 36/81 which is 4/9, coincidentally double the probability after three packs. After five packs, the number of combinations rises to 243.
Looking at the number of combinations I've seen so far, for the three-toy problem, it's risen from 9 to 81 to 243, rising by a factor of 3 each time, and n=3, the number of toys squared. In fact the denominator, the total number of permutations, is simply n^t (number of toys raised to the power of the number of turns or packs). This is also known as the formula for 'permutations with repitition'.
Now I need to identify the number of successful permutations - those that contain one of each of the toys.
For n=3 toys and t=3 turns, there are 6 successful permutations (as listed above).
For n=3 toys and t=4 turns, there are 36 successful permutations
Now, the question becomes - how many will there be after five turns?
Let's go back to four turns and look closely at each of the permutations we have. We have the 36 successful ones, ABCA, ABCB, AABC and so on. For these 36 successes, it doesn't matter what we pick next, we'll still have a successful combination; there are three options for each of these, so that's 36x3=108 successes for each of them.
There are also the 3 combinations AAAA, BBBB and CCCC which will not produce a success, irrespective of what we pick next, so that's 3x3=9 fails.
This leaves the rest, which logically must contain two different toys. We've covered the ones which already contain three different toys, and we've looked at the ones which contain only one toy. Since there were 81 combinations in total after four goes, there must be 81-(36+9) = 36 combinations which contain two different toys. There is a probability of 1/3 of the next choice being the correct one, which equals 36 x 1/3 = 12.
So, after five turns, we have 108 + 12 = 120 successful permutations.
Let's review:
After three turns: 6 successful permutations
After four turns: 36 successful permutations
After five turns: 120 successful permutations
Let's take a look at six turns, based on the process we used for five turns.
After five turns, we have 120 successes which will each yield three more successes, so 3x120 = 360
We also still have the combinations which have just one toy in - AAAAA, BBBBB and CCCCC, and these will each produce three more unsuccessful combinations, irrespective of the next choice. 3x3=9 unsuccessful combinations.
There are 3^5 total combinations after five turns, 243 in total, which means that we have 243-(120+9) = 114 other combinations which have two toys. A third of these will become successful with the sixth turn, which is 114/3 = 38.
So, after six turns, we have 360 + 38 = 398 successful permutations. I've deduced the formula for working out successful permutations in an iterative manner, but I don't have the computing power to determine the 10th, 15th or 115th term without knowing each one before. Furthermore, this method won't easily expand to cover five, six, or ten toys. It's all about knowing how many successes you've had previously, and how many certainly won't become successful (because they have n-2 different toys and will require at least two more goes to become successful).
Three toys is an easy case - you either have a successful combination, a combination with only one toy repeated, or a two-toy combination with a 1/3 chance of becoming successful. With four toys, you may already have one, two or three different toys, or be successful, and I don't quite see how to sort all that out.
So, next time, it's on to spreadsheet modelling. I'm going to write a macro that simulates buying the cereal packets and examining the toys, and determining whether or not the combination is a success. If maths fails, use sampling!
The Probabilities and Free Toys Series
Part 1: Solving for "What's the probability of getting 10 toys in just 10 packs?"Part 2: Solving for "How many packs do I need to buy to be 50%, 70%, 90% sure of getting all the toys?"
Part 3: Spreadsheet modelling the toys and probability question
Part 4: An elegant solution to 'what's the average number of boxes you'd need to open to get all the toys'?
Part 4: An elegant solution to 'what's the average number of boxes you'd need to open to get all the toys'?
Tuesday, 6 September 2011
A Beginner's Social Media Strategy
I say 'for beginners' as I don't feel particularly qualified to discuss it in much detail (as you'll soon see), and this is more an explanation of my background and experience so far.
A few years ago, I set up my own website; you may have seen me refer to it previously. It was set up entirely as an exercise in website-building and tagging - it's all hard-coded HTML. It's tagged with Google Analytics, and I've been monitoring traffic to it since then, doing a little SEO and making changes (and hopefully improvements) to the content here and there. As analysts, we're usually charged with analysing, understanding and reporting stats, and then generating recommendations from them; we're not usually given a logon to a CMS and given free run of a website. By building my own website, I got to play both sides (and realised how time-consuming content generation and site maintenance can be). Better still, I've even been running A/B/C tests on it, and finding out how easy it is to set up (providing you've got multiple content ready to serve).
Anyway, with time, I moved from focussing on the website to a blog. Blogs are, for this JavaScript beginner, much easier than HTML websites, especially when I can't use server-side includes. So, even with a WYSIWYG HTML editor, blog posts are much easier than HTML pages. Once again I found out how to tag my blog by putting javascript includes in my posts and inserting GA tags in my posts, and I've monitored the traffic. I also discovered how many analysts there are out there who are reading this blog (it's not a huge number, but compared to the single digits I was experiencing before I started writing about web analytics, it's a significant uplift). The blog has an 'about me' page which includes my Facebook and Twitter details, so that people can follow me, and I post updates about my blog on Twitter or Facebook or both, and occasionally on the Yahoo Web Analytics forum.
All of which means that I've built up a cyclical path between my social media accounts (where you can find links to my blog) and my blog (which explains how to follow me on social media).
That's not a strategy, that's just a circle. Which brings me to one key question: What am I actually trying to achieve with all this online presence?
Am I trying to get Twitter followers? Am I doing this for my ego, or for PR, or something like that? Maybe, but probably not. Am I trying to get Facebook friends? No - I've got enough friends (and I've met 99% of them in person) and I've successfully tracked down my best friends from primary school, high school and university. Am I trying to get more people to read my blog? Now then - that seems more likely. What I'm trying to do is to share what I know about various subjects (maths, chemistry, chess, web analytics) and hopefully build up an online reputation as a reliable source of useful, accurate information - to be regarded as a specialist in my field (and possibly even, one day, an expert).
People aren't going to get that level of knowledge about me from my Twitter feed (which, even with my best intentions, is very clouded up with links to miscellaneous stuff I find interesting). They are most certainly not going to get that from my Facebook updates, which are much more personal and include family updates, photos from days out and the like, and are very much about my views and opinions and general chatter. Hopefully, though, my peers and friends will read my blog, where I write my more considered opinions and views, and share what I hope will be useful insights into areas that specifically interest me - as I said, Chess, maths, chemistry and web analytics. The blog also has a Google Analytics goal set up - visitor views the About Me page - and this means that my social media strategy not only has an aim (to get people to read the blog) but a specific goal (find out more about me and my professional skills and experience).
I could go on and build KPIs about blog traffic levels and so on, and on to Twitter followers (excluding the spam accounts) but these will be secondary to getting people to view my profile page on my blog. I can THEN use analytics to tell me which blog post they read before reading my profile, and also where they came from... and then write more blogs on similar topics and post links on similar sources.
And that, in a nutshell, is my social media strategy. I can't say that it's scalable to a company level, but I think the main points (which are probably covered elsewhere) are:
What am I actually trying to achieve with all this online presence (answer in English words)?
What is my actual aim?
What does this look like as an online event (in page terms)? Make this an online 'goal' or 'event' in analytics package.
What type of visitor carries out a success event? Which social media site did they come from?
How do I get more of them? What sort of content do they look at?
I'm not sure if this is a social media strategy, or just a reiteration of a normal online strategy. Like I said, I'm a beginner on social media strategies (despite having a blog, Facebook and Twitter accounts for years) so I'm open to other suggestions!
A few years ago, I set up my own website; you may have seen me refer to it previously. It was set up entirely as an exercise in website-building and tagging - it's all hard-coded HTML. It's tagged with Google Analytics, and I've been monitoring traffic to it since then, doing a little SEO and making changes (and hopefully improvements) to the content here and there. As analysts, we're usually charged with analysing, understanding and reporting stats, and then generating recommendations from them; we're not usually given a logon to a CMS and given free run of a website. By building my own website, I got to play both sides (and realised how time-consuming content generation and site maintenance can be). Better still, I've even been running A/B/C tests on it, and finding out how easy it is to set up (providing you've got multiple content ready to serve).
Anyway, with time, I moved from focussing on the website to a blog. Blogs are, for this JavaScript beginner, much easier than HTML websites, especially when I can't use server-side includes. So, even with a WYSIWYG HTML editor, blog posts are much easier than HTML pages. Once again I found out how to tag my blog by putting javascript includes in my posts and inserting GA tags in my posts, and I've monitored the traffic. I also discovered how many analysts there are out there who are reading this blog (it's not a huge number, but compared to the single digits I was experiencing before I started writing about web analytics, it's a significant uplift). The blog has an 'about me' page which includes my Facebook and Twitter details, so that people can follow me, and I post updates about my blog on Twitter or Facebook or both, and occasionally on the Yahoo Web Analytics forum.
All of which means that I've built up a cyclical path between my social media accounts (where you can find links to my blog) and my blog (which explains how to follow me on social media).
That's not a strategy, that's just a circle. Which brings me to one key question: What am I actually trying to achieve with all this online presence?
Am I trying to get Twitter followers? Am I doing this for my ego, or for PR, or something like that? Maybe, but probably not. Am I trying to get Facebook friends? No - I've got enough friends (and I've met 99% of them in person) and I've successfully tracked down my best friends from primary school, high school and university. Am I trying to get more people to read my blog? Now then - that seems more likely. What I'm trying to do is to share what I know about various subjects (maths, chemistry, chess, web analytics) and hopefully build up an online reputation as a reliable source of useful, accurate information - to be regarded as a specialist in my field (and possibly even, one day, an expert).
People aren't going to get that level of knowledge about me from my Twitter feed (which, even with my best intentions, is very clouded up with links to miscellaneous stuff I find interesting). They are most certainly not going to get that from my Facebook updates, which are much more personal and include family updates, photos from days out and the like, and are very much about my views and opinions and general chatter. Hopefully, though, my peers and friends will read my blog, where I write my more considered opinions and views, and share what I hope will be useful insights into areas that specifically interest me - as I said, Chess, maths, chemistry and web analytics. The blog also has a Google Analytics goal set up - visitor views the About Me page - and this means that my social media strategy not only has an aim (to get people to read the blog) but a specific goal (find out more about me and my professional skills and experience).
I could go on and build KPIs about blog traffic levels and so on, and on to Twitter followers (excluding the spam accounts) but these will be secondary to getting people to view my profile page on my blog. I can THEN use analytics to tell me which blog post they read before reading my profile, and also where they came from... and then write more blogs on similar topics and post links on similar sources.
And that, in a nutshell, is my social media strategy. I can't say that it's scalable to a company level, but I think the main points (which are probably covered elsewhere) are:
What am I actually trying to achieve with all this online presence (answer in English words)?
What is my actual aim?
What does this look like as an online event (in page terms)? Make this an online 'goal' or 'event' in analytics package.
What type of visitor carries out a success event? Which social media site did they come from?
How do I get more of them? What sort of content do they look at?
I'm not sure if this is a social media strategy, or just a reiteration of a normal online strategy. Like I said, I'm a beginner on social media strategies (despite having a blog, Facebook and Twitter accounts for years) so I'm open to other suggestions!
Probabilities and Free Toys, Part 1
Once upon a time, a long time ago, my high-school maths teacher set an extension problem: I never got chance to tackle it, and I've never tried to since. I've remembered it through the years, as a friend of mine was able to solve it elegantly, and I never saw his answer. So, it's time for some closure again.
Here's the question:
A certain breakfast cereal manufacturer is giving away a free toy inside each pack. There are ten toys in the series, and I'd like to collect them all. However, I can't tell which toy I'm going to get when I buy the pack. The original question was: what's the probability of getting all ten toys after opening ten packs? And the follow-up question I'd like to look at is: assuming that each toy is distributed equally, how many packs would I have to buy to be 50%, 70% or 90% sure of having all ten toys?
Now, bearing in mind that this is a high-school maths problem, it shouldn't take any advanced maths to solve the first question. The follow-up question is one of my own, and could take me anywhere.
So, let's look at the first question, and let's start with two toys and build up to ten.
After buying two packs, the probability of success is 0.5. The two successful combinations are AB and BA, and the two unsuccessful are AA and BB (i.e. I get the same toy twice). But let's look at that as a step-by-step process. When I buy the first pack, I am certain of getting a toy I haven't got before. There are two alternatives, A and B, and two successes (either of them). So the probability is 2/2. The probability of getting a successful toy with my second pack is 1/2. There's now only one successful toy (the one I haven't got), but there are two toys available. To calculate the probability of getting the first toy and the second toy in two packs is 1/2 x 2/2 = 1/2.
This can be expanded to three toys, A, B and C:
Probability of success with first pack is 3/3 (any of the toys is a success)
Probability of success with the second pack is 2/3 (I need to avoid getting a duplicate, so there are now only two successes. If I have A, then I only need B or C).
Probability of success with the third pack is 1/3 (I now need one specific toy as I have the other two).
So, the probability of success after three packs = 3/3 x 2/3 x 1/3 = 6/27 = 2/9 = 22%
I'll do the case for four toys, before moving to a general expression:
p(success with first pack) = 4/4 as any of the four toys is a success
p(success with second pack) = 3/4 as I already have one toy, and need one of the other three
p(success with third pack) = 2/4 as I have two toys and only two are now successes
p(success with fourth pack) = 1/4 as I only need one of the four toys to complete my set.
So, probability of success with four toys and four packs =
4/4 x 3/4 x 2/4 x 1/4 = 24/256 = 3/32 = 9.375%
There's a clear pattern developing. For five toys, the numerators will be 5, 4, 3, 2, 1 and the denominators will be 5, 5, 5, 5, 5. The numerators are multiplied together, 5x4x3x2x1 which is called 5 factorial, and written 5! while the denominators are 5x5x5x5x5 which is 5^5. Looking back, the same rule applies to four toys, three toys and two toys, and will apply going upwards.
So, the probability of getting all n toys with n packs is n! / n^n
n! is an expression that increases very quickly with n (1, 2, 6, 24, 120, 720, 5040 and so on) but the denominator n^n increases even more quickly (1, 4, 27, 256, 3125, 46656 and so on). The table below shows n, n!, n^n and the ratio n! / n^n which is the probability of getting n toys with n packs. For the original question - what's the probability of getting 10 toys in 10 packs, the answer is 10! / 10^10 which is 0.036% (less than one in a thousand).
Next time, I'll look at the harder question: with 10 toys (or a number larger than three or four), how many packs do I have to buy to be 50%, 70% or 90% sure of having the full set?
Part 2: Solving for "How many packs do I need to buy to be 50%, 70%, 90% sure of getting all the toys?"
Here's the question:
A certain breakfast cereal manufacturer is giving away a free toy inside each pack. There are ten toys in the series, and I'd like to collect them all. However, I can't tell which toy I'm going to get when I buy the pack. The original question was: what's the probability of getting all ten toys after opening ten packs? And the follow-up question I'd like to look at is: assuming that each toy is distributed equally, how many packs would I have to buy to be 50%, 70% or 90% sure of having all ten toys?
So, let's look at the first question, and let's start with two toys and build up to ten.
After buying two packs, the probability of success is 0.5. The two successful combinations are AB and BA, and the two unsuccessful are AA and BB (i.e. I get the same toy twice). But let's look at that as a step-by-step process. When I buy the first pack, I am certain of getting a toy I haven't got before. There are two alternatives, A and B, and two successes (either of them). So the probability is 2/2. The probability of getting a successful toy with my second pack is 1/2. There's now only one successful toy (the one I haven't got), but there are two toys available. To calculate the probability of getting the first toy and the second toy in two packs is 1/2 x 2/2 = 1/2.
This can be expanded to three toys, A, B and C:
Probability of success with first pack is 3/3 (any of the toys is a success)
Probability of success with the second pack is 2/3 (I need to avoid getting a duplicate, so there are now only two successes. If I have A, then I only need B or C).
Probability of success with the third pack is 1/3 (I now need one specific toy as I have the other two).
So, the probability of success after three packs = 3/3 x 2/3 x 1/3 = 6/27 = 2/9 = 22%
I'll do the case for four toys, before moving to a general expression:
p(success with first pack) = 4/4 as any of the four toys is a success
p(success with second pack) = 3/4 as I already have one toy, and need one of the other three
p(success with third pack) = 2/4 as I have two toys and only two are now successes
p(success with fourth pack) = 1/4 as I only need one of the four toys to complete my set.
So, probability of success with four toys and four packs =
4/4 x 3/4 x 2/4 x 1/4 = 24/256 = 3/32 = 9.375%
There's a clear pattern developing. For five toys, the numerators will be 5, 4, 3, 2, 1 and the denominators will be 5, 5, 5, 5, 5. The numerators are multiplied together, 5x4x3x2x1 which is called 5 factorial, and written 5! while the denominators are 5x5x5x5x5 which is 5^5. Looking back, the same rule applies to four toys, three toys and two toys, and will apply going upwards.
So, the probability of getting all n toys with n packs is n! / n^n
n! is an expression that increases very quickly with n (1, 2, 6, 24, 120, 720, 5040 and so on) but the denominator n^n increases even more quickly (1, 4, 27, 256, 3125, 46656 and so on). The table below shows n, n!, n^n and the ratio n! / n^n which is the probability of getting n toys with n packs. For the original question - what's the probability of getting 10 toys in 10 packs, the answer is 10! / 10^10 which is 0.036% (less than one in a thousand).
Next time, I'll look at the harder question: with 10 toys (or a number larger than three or four), how many packs do I have to buy to be 50%, 70% or 90% sure of having the full set?
The Probabilities and Free Toys Series
Part 1: Solving for "What's the probability of getting 10 toys in just 10 packs?"Part 2: Solving for "How many packs do I need to buy to be 50%, 70%, 90% sure of getting all the toys?"
Part 3: Spreadsheet modelling the toys and probability question
Part 4: An elegant solution to 'what's the average number of boxes you'd need to open to get all the toys'?
Part 4: An elegant solution to 'what's the average number of boxes you'd need to open to get all the toys'?
Thursday, 1 September 2011
My X Factor predictions for 2011
Now, I reckon I'm a fairly optimistic person. I look for the best in people and in situations, as a rule, and I will try to give people the benefit of the doubt. However, I can also be quite cynical. I don't see it as cynical, other people do, I see it as identifying trends and patterns and expecting them to be repeated - even though I hope for the best.
There is, however, one area where I am just plain cynical - or, alternatively, very good at spotting patterns and trends - and that is with the Saturday evening television monstrosity known as the X-Factor (which I do call the X-Factory given its aim of mass production of pop music and cardboard cut-out pop stars).
Here, then, based on previous years' viewing (despite myself) are my predictions for what we can expect from the Simon Cowell juggernaut this year.
* At least one finalist to have estranged parent or sibling - I appreciate I'm late with this, given that immediately after the first episode, one of the judges discovered a brother she never knew she had.
* Gary Barlow to have one of his Take That mates at the judges' house stage (and it won't be Robbie)
* One of finalists to have been bullied at school
* There will be the formation of a boy group and girl group, made up of the boy dregs and girl dregs at the end of the boot camp stage. "We want to put you together into a group [because we haven't got enough groups already]."
* These synthetic dregs-groups to go through to the live shows (you didn't think the judges would put them together and not let them go through, did you?).
* These synthetic groups to get eliminated in first two weeks - first the girls (who will dress inappropriately) and then the boys (who can't sing as a team)
* Simon Cowell to make a guest appearance, to much fanfare and flashing lights
* Last year's winner (whoever that was) to release album just in time for Christmas
* Louis Walsh to pick a wildcard act (or just a wild act) which is no good, but which secures the votes of those who deliberately vote for the worst (Jedward, Wagner).
* There will be extensive media coverage of an apparent spat between two of the judges, probably the two ladies, but possibly the two blokes
* One of the acts to suffer with a cough/cold/laryngitis/glandular fever part way through the TV shows
* Two of the acts to form a 'secret' relationship, again with much media coverage
Print out the list, and tick them off. If there any left by Christmas, I think I'll genuinely be surprised. In fact, give me a week or two, and I'll probably have some more predictions.
There is, however, one area where I am just plain cynical - or, alternatively, very good at spotting patterns and trends - and that is with the Saturday evening television monstrosity known as the X-Factor (which I do call the X-Factory given its aim of mass production of pop music and cardboard cut-out pop stars).
Here, then, based on previous years' viewing (despite myself) are my predictions for what we can expect from the Simon Cowell juggernaut this year.
* At least one finalist to have estranged parent or sibling - I appreciate I'm late with this, given that immediately after the first episode, one of the judges discovered a brother she never knew she had.
* Gary Barlow to have one of his Take That mates at the judges' house stage (and it won't be Robbie)
* One of finalists to have been bullied at school
* There will be the formation of a boy group and girl group, made up of the boy dregs and girl dregs at the end of the boot camp stage. "We want to put you together into a group [because we haven't got enough groups already]."
* These synthetic dregs-groups to go through to the live shows (you didn't think the judges would put them together and not let them go through, did you?).
* These synthetic groups to get eliminated in first two weeks - first the girls (who will dress inappropriately) and then the boys (who can't sing as a team)
* Simon Cowell to make a guest appearance, to much fanfare and flashing lights
* Last year's winner (whoever that was) to release album just in time for Christmas
* Louis Walsh to pick a wildcard act (or just a wild act) which is no good, but which secures the votes of those who deliberately vote for the worst (Jedward, Wagner).
* There will be extensive media coverage of an apparent spat between two of the judges, probably the two ladies, but possibly the two blokes
* One of the acts to suffer with a cough/cold/laryngitis/glandular fever part way through the TV shows
* Two of the acts to form a 'secret' relationship, again with much media coverage
Print out the list, and tick them off. If there any left by Christmas, I think I'll genuinely be surprised. In fact, give me a week or two, and I'll probably have some more predictions.
Friday, 26 August 2011
Film review: Cloverfield
Cloverfield
I've been considering add this to my Lovefilm list, although I was never completely convinced. The trailer for the film made it look a lot like another Godzilla movie - and yes, I've enjoyed the Jurassic Park series, and I've watched bits of the new Godzilla film, but the genre has never really appealed to me. However, when Cloverfield showed up on the TV listings, I set the recorder and figured I'd pick it up when there was nothing on the TV.
Cloverfield goes for the innovative approach of filming everything from the first person, with a hand-held camcorder (or at least making it look that way). The film starts steadily, as one night a huge monster starts ripping up a city. Which city? I'll give you a clue: it's the main city in Independence Day, The Day The Earth Stood Still, Spiderman... yes, once again, New York is the shortcut for 'any main Earth city'. Normally, I'd include a summary of the plot in my review, but I'm going to struggle for this film. There isn't much plot. Monster rips up city; army arrives eventually; conventional weapons are utterly useless; government authorises use of nuclear weapons. The remaining story revolves around a group of civilians (including our cameraman) who aren't intelligent enough to run away, and instead, insist on 'documenting' everything. Their motivation for this isn't clear, and as I watched our characters head towards the danger, while crowds of intelligent people started felling, I lost any interest or sympathy for them.
The other main character - the monster - had no character at all. It was extremely difficult to feel anything for it - was it an evil enemy bent on destruction and conquest? Was it from outer space or under the ground? Was it - as one of the characters suggested - from under the sea? Was it lost? Was this a misunderstood first contact going badly wrong? It was part - and an unfortunate part - of the film's set-up that it's almost entirely filmed from one perspective; perhaps the film was going for the idea of portraying the details of a monster attack (I refuse to call it an alien invasion) from an individual's point-of-view. If it was, then it didn't work, for three key reasons: the cameraman and his associates were not particularly well drawn as characters (despite the occasional flashbacks); they kept running towards trouble, instead of away from it, and the film failed to answer one question sufficiently for me: why didn't he just ditch the camera, which was slowing him down and took up one of his hands, and run?
I know I'm slating this film, but, apart from the criticisms I've levelled against it already, there are a few good points. There's a section in the film where our characters are walking along a tunnel to ... well, I think they were looking for a safe way to get closer to the monster to look at it; I don't think they were trying to escape. I must have missed the line that explained this entry in their list of poor decisions. Anyway, they're traipsing along in the dark, when suddenly, a couple of mini-aliens (looking like a smaller, dressed-down version of the bugs in Starship Troopers) start attacking them, snapping and biting and scratching and generally causing lots of trouble. One of the cast suffers a scratch or a bite to her shoulder, and, when the team take a proper look at it, it's infected and looking decidedly alien. I surmised at this point that she was going to turn into an alien (something like District 9), but I was wrong. The team make their way back to the surface, and are found by the army, with one soldier delivering the best line of the film, "We're not sure what it is, but we know one thing: it's winning."
However, despite one of the team being seriously wounded, and is desperate for treatment, the characters decide they still want to go and rescue their friend (they assume that she's as daft as they are, and hasn't made a run for it). Not, "Please will you treat our friend's shoulder," but, "We know we're unarmed, but want to go risk our necks too, where's the way out?" Fortunately (for the story, not for the characters), one of the medical staff in the army's triage centre spot the wound, and with a shout of, "Bite!" they whisk the female lead (now bleeding from the eyes) out behind a screen, where she dies a very swift and dramatic death. Still undeterred, our characters (I'm not calling them heroes, sorry) manage to get back outside, with the promise of a helicopter pick-up in the following morning. They find their friend's apartment block, toppled over and leaning on another adjacent tower. Has the friend run away? Has she died? Do we care? Do the characters? No. They decide that it makes sense to go up the adjacent block and then jump across. Oh dear.
At least the army and military have the right idea, as we see fewer armoured vehicles on the ground, and more aircraft firing missiles and dropping bombs, as our characters go on their crazy mission.
I think I can summarise my disappointment or dislike of this film with one of the final scenes; the crew enter the tower block, and try the lifts. They don't work, so they decide to take the stairs, and I suddenly realised the stupidity of what I could be about to witness: a trio of people climbing all the stairs of a skyscraper. Fortunately for me, Mr Cameraman pressed the pause button a few times, to save me from total boredom (but highlighting how bored I was at this point) and during one conversation on the stairs, he and his friend say, "What are you talking about?" "I'm just talking. I don't know why I'm talking." Sadly, sir, you're the only narrative to this story. Otherwise, if you don't know why you're talking, then, even more sadly, neither do I.
Some of my other film reviews:
Cloverfield
Inception
The Green Hornet
Transformers 2: Revenge of the Fallen
Transformers 3: Dark of the Moon
Transformers: Bumblebee
Transformers: One
Tron
Subscribe to:
Posts (Atom)