Friday, 23 September 2016

Premier League Excitement - Further Analysis

In my last post I looked at 'How exciting is the Premier League' and produced the interesting data point that less than 10% of Premier League games are goal-less.  This may be interesting, and it might even count as insight, but it's not very actionable.  We can't do anything with it, or make any decisions from it.  I suppose the question is, "Is that a lot?" and I'll be looking at that question in more detail in future.

So, my next step is to look at how the different teams in the Premier League compare on some of the key metrics that I discussed - goals per game (total conceded plus scored), percentage of goalless games and so on.

Number of goals per game (conceded plus scored)

Firstly, I segmented the data per team:  how many goals were there per game for each team in the Premier League.  This is time-consuming, but worthwhile, and a sample of the data is shown below.  I have data as far back as the 2004-5 season, but the width wouldn't fit on this page: 
Club
Y2010
Y2011
Y2012
Y2013
Y2014
Y2015
Y2016
Arsenal
        2.58
        3.03
        3.24
        2.87
        2.87
        2.82
        2.66
Aston Villa
        2.21
        2.82
        2.37
        3.05
        2.63
        2.32
        2.71
Birmingham

        2.50





Blackburn
        2.79
        2.76
        3.32




Bolton
        2.61
        2.84
        3.24




Charlton
        2.47






Chelsea
        2.32
        2.68
        2.92
        3.00
        2.58
        2.76
        2.95
Crystal Palace




        2.13
        2.58
        2.37
Everton
        2.32
        2.53
        2.37
        2.50
        2.63
        2.58
        3.00
Fulham
        2.58
        2.42
        2.61
        2.89
        3.29


Liverpool
        2.21
        2.71
        2.29
        3.00
        3.97
        2.63
        2.97
Man City
        1.92
        2.45
        3.21
        2.63
        3.66
        3.18
        2.95
Man United
        2.89
        3.03
        3.21
        3.39
        2.82
        2.61
        2.21
Middlesbrough
        2.45






Newcastle
        2.24
        2.97
        2.82
        2.97
        2.68
        2.71
        2.87
Norwich


        3.11
        2.61
        2.37

        2.79
Portsmouth
        2.29






Southampton



        2.87
        2.63
        2.29
        2.63
Tottenham
        2.92
        2.66
        2.82
        2.95
        2.79
        2.92
        2.74
West Brom

        3.34
        2.55
        2.89
        2.68
        2.34
        2.16
Wigan
        2.53
        2.66
        2.74
        3.16



Season Average
2.77
2.80
2.81
2.80
2.77
2.57
2.70

Blank columns indicate a season where a team was not in the Premier League.  
Bold figures show where a team achieved over 3 goals per game for the season.
Y2008 indicates the season 2007-2008.
Firstly:  sorting alphabetically makes sense from a listing perspective, but for comparison the data is best sorted numerically (from highest to lowest). 

Secondly:  There's a lot of data here, and clearly a visualisation is needed:  I'm going with a line graph.  And to avoid spaghetti, I'm going to highlight some of the key teams - the team with the highest average number of goals per game; the team with the lowest, and the average.

Thirdly:  to identify the overall highest- and lowest-goal teams, I'm just going to take the totals of the averages for the last nine seasons, and sort them from the list.  Teams that were not in the Premier League for one or more seasons are included based on their performance while they were in the Premier League.

Premier League Teams:  Average number of goals per game over the last 12 seasons:


Club

Club
Average
Arsenal
      2.842
Tottenham
      2.833
Man City
      2.825
Blackburn
      2.816
Man United
      2.807
Liverpool
      2.781
Newcastle
      2.751
Norwich
      2.717
Bolton
      2.705
Overall Average
      2.702
Birmingham
      2.671
Chelsea
      2.670
West Brom
      2.669
Aston Villa
      2.667
Fulham
      2.613
Southampton
      2.605
Wigan
      2.566
Everton
      2.518
Charlton
      2.474
Middlesbrough
      2.404
Portsmouth
      2.368
Crystal Palace
      2.360

Key takeaways:  
- Arsenal have had the most total goals per game over the last nine seasons (2.842 goals per game)
- Everton have the lowest average number of goals per game for teams which have been present in all 12 seasons (2.518 goals per game).
- Put another way:  Arsenal fans have seen 1296 league goals in the last 12 seasons, compared to 1148 for Everton fans (148 fewer).


Theo Walcott, celebrating during Arsenal's win over Hull, Sept 2016  Image credit

Time for some graphs!

Firstly, average goals per season, for the last 12 seasons, for Arsenal, Everton, the league average, Liverpool (who achieved an average of 3.97 in 2013-14) and Man United (because they're always worth comparing).



This shows clearly that Arsenal (green line) have consistently exceed the league average, falling below it only twice in the last 12 seasons.  Everton (blue) have only once exceeded the average, and that was in the most recent season.  Liverpool have exceeded the average over the last four seasons, but prior to that were consistently below (and similar to Everton).

Connecting this to 'real life' events:

- Everton moving from David Moyes to Roberton Martinez in August 2013 did not make any difference to their 'excitement' factor until the 2015-16 season.

- Arsenal, and Arsene Wenger, could not be called 'boring' based on their goals per game. 

- Brendan Rogers had an interesting time at Liverpool, when they hit the highest goals-per-game for the season for any club in the last 12 years (3.97).  Note that this does not discriminate between goals scored or conceded.

Secondly, adjusting the data to show the difference between each team and the overall average (so that the data shows a delta versus the average).



To give you an indication of Liverpool's remarkable 2013-4 season:  their games had more than one goal per game more than the season average.  Brendan Rogers had an eventful time at Liverpool.

Fulham also had an 'exciting' season in 2013-4, achieving 3.29 goals per game (average was 2.77) - but were subsequently relegated.

In summary:
- Arsenal have had the highest average goals per game over the last nine seasons (2.842 goals per game), while Everton have the lowest, at 2.518 goals per game.
- Arsenal have exceeded the league average goals per game in 10 out of the last 12 seasons, and have the highest average overall.
- Man United have achieved above-average goals per game in nine of the last 12 seasons; however the 2015-16 season was the least 'exciting' they've recorded in that period.

Review
Segmenting the data by team is proving more useful.  It's now possible to make predictions about the 2016-17 season:
- Arsenal to remain most 'exciting', closely followed by Tottenham and Man City.
- Everton to remain the least 'exciting', with 1-1, 2-1 and 2-0 results dominating.
- Man United are extremely unpredictable, especially as they have a new manager this season (although nobody could have predicted the dreadful start they've made to the current season).

The raw data used in this analysis is available from the football data website, among others.


Thursday, 1 September 2016

How Exciting Is the English Premier League?

So, it's the start of the English Premier League (EPL) season. Sport generates vast amounts of data, all available for analysis and insight, and in this post (and probably a couple of following posts), I will be looking at the English Premier League (football, aka soccer) for recent years and reviewing how the game has changed.  This will form a practical look at data, reporting, analysis, insights and recommendations - as you'll see.



Photo: mine (Stoke City vs Sunderland, April 2016)


Firstly, a quick mathematical puzzle that will help with our analysis:  how many games are played in each EPL season?  Each team will play each other team - twice (once at home and once away). And not forgetting that teams don't play themselves ;-)

If we consider placing all the teams on the horizontal and vertical axes of a table, then each cell within that table will represent a game (the team on the horizontal axis can be the home team, for the sake of argument).

Team Arsenal Bournemouth Chelsea
Arsenal x - -
Bournemouth - x -
Chelsea - - x

So each team plays each other team twice (once at home, and once away - or once above and below the leading diagonal), and the total number of games is equal to the number of teams squared, minus the number of teams (as shown by the leading diagonal - the teams cannot play themselves).

So, for a league with n teams where each team plays the others twice, the total number of games, G = n2-n = n(n-1). And for the UK Premier League, where n=20, the number of games = 380.


That was easy enough:  now to use this to answer some other questions about the English Premier League over recent years, and potentially provide a basis for comparisons against other leagues.  The main question that I'd like to ask (and answer) is How exciting is the English Premier League?

There's no metric for excitement (after all, excitement is an emotional response, and we have only numerical data to work from).  But let's look at some potential numbers we can use to get a handle on how exciting each season has been:

KPIs
- What's the average number of goals per game in the league
- How many games (number and percentage) are 0-0 draws?  

- How many goals are won with a one-goal margin?
- What's the largest number of goals scored in one game (for each season)?
- What's the spread of total points at the end of the season?

This will help us to move on to other questions, such as:

- People say that the lower leagues are more exciting and more interesting - is this true?
- Is it worth getting a Sky Sports subscription to watch the Premier League games?  Or should I just watch the lower leagues on a less expensive subscription?
- Which pairings (Club A vs Club B) tend to product the most games?  (This question is going to require more detailed work)  Is it Manchester United vs Chelsea, or is it Liverpool vs West Ham (for example)?

So:  ANALYSIS

  
1. What's the average number of goals per game?
If more goals equals more excitement, then this is a good data point to start with.

The highest number of goals per game was achieved in 2012, with an average of over 2.8 goals per game (1066 total goals).  The lowest was in 2007, when the average fell to less than 2.5 goals per game (931 in total).





This was achieved through a high number of high-scoring games; comparing the highest-scoring and lowest-scoring games, we have a clear indication of the increase in games with 5-8 goals, and a smaller proportion of 0-0 and 1-0 games.
 
2.  How many games are 0-0 draws?

Surely the most boring games are goal-less draws, so a high proportion of 0-0 matches indicates a dull season.  As we saw above, 2007 had a large number of 0-0 games, but the worst offender was 2009.  2007 had 34 goalless draws, while 2009 had 42.  The fewest 0-0 draws came in 2011, when there were just 25.

 
















This is interesting data, and interesting analysis too, but there's nothing here that says, "Oh, now I should do this...".  What it does tell us is that the 2015-16 season was considerably better than 2015 (which had an unusally low number of goals per game).
3.  How many goals are won with a one-goal margin?Tense, nail-biting finishes where one team wins by just one goal... it's gripping stuff (as far as football can be described as gripping).  One team scores a goal and successfully 'parks the bus', or grabs a late winner.  Either way, one-goal-margins (whether that's 1-0, 2-1, 3-2 etc) tend to make for exciting games.  Here's the breakdown:

So perhaps 2015 was the year that teams learned to "park the bus"- get a one goal advantage and then hold the game - and 2016 was a year that they started to forget.  It's something LVG tried to do at Manchester United (but we all know how that turned out).

4.  What's the highest number of goals scored in one game?


2008 saw Portsmouth beat Reading 7-4.
2010 had Tottenham 9 Wigan 1
In 2012 the high-scoring game was Man United 8 Arsenal 2
2013 had two high-scoring games: Arsenal 7 Newcastle 3 and West Brom 5 Man United 5

Interesting, but not helpful.  This shows us the outliers - the maximum number of goals is rarely achieved in more than one game.  It doesn't tell us which teams are most likely to achieve them, and which games we should watch to get more goals.  It does tell us that when a game has reached nine goals, that's probably the most we can get, and it's time to go home.

Review
So far, this is all data and analysis, there's not much insight, and there's certainly nothing actionable.   Trending data has had limited use, and that's partly because it's not been segmented in any way.  There are a few interesting headlines, which can be used to make the data sound interesting, but don't really add anything (sometimes, web managers like this kind of stuff):

How exciting is the English Premier League? 90% of Premier League games have at least one goal.Each week, the EPL sees over two goals per game, on average.
This is an example of reporting.  It's good reporting, but nothing more.  It tells us what's happened, but doesn't tell us what to do about it.  It may have taken hours to produce (and trust me, it has), but it's not adding any value.  In order ot move from reporting to insight and recommendations, we need to start doing two additional things:  1. Segmenting the data (to see, for example, which team is the most exciting) and 2. Comparing with external data (for example, the Spanish or Italian leagues) to see how the English League compares.


More next time... 
The raw data used in this analysis is available from the football data website, among others.