I enjoy analytics and testing, and applying them to online customer experience - using data to inform ways of improving a website. Occasionally, it occurs to me that life would be great if we could do 'real life' testing - which is the quickest way home; which is the best meal to order; which coat should I wear today (is it going to rain)? Instead, we have to be content with before/after analysis - make a decision, make a change, and see the difference.
One area which I also like to look at periodically is sport - in particular, football (soccer). I've used football as an example in the past, to show the importance of picking the right KPIs. In football, there's no A/B testing - which players should a manager select, which formation should they play in - it's all about making a decision and seeing what happens.
One of my least favourite football teams is Manchester United. As a child, my friends all supported Liverpool, and so I did too, having no strong feeling on the subject at the time. I soon learned, however, that as Liverpool fans, it was traditional to dislike Manchester United, due to their long-standing (and ongoing) rivalry. So I have to confess to slight feeling of superiority whenever Manchester United perform badly. Since the departure of their long-serving manager, Alex Ferguson, they've seen a considerable drop in performance, and much criticism has been made of his two successors, first David Moyes, and now Louis van Gaal. David Moyes had a poor season (by Man Utd's standards) and was fired before the end of the season. His replacement, Louis van Gaal, has not had a much better season this far. Here's a comparison of their performance, measured in cumulative points won after each game [3 points for a win, 1 for a draw, 0 for a loss].
So, how bad is it?
Well, we can see that performance in the current season (thick green line) is lower than last season (the blue line). Indeed, after game 10 in early November 2014, the UK media identified that this was the worst start to the season since 1986. But since then, there's been an upturn in performance and at the time of writing, Manchester United have won their last two matches. So perhaps things aren't too bad for Louis van Gaal. However, the situation looks slightly different if we overlay the line for the previous season, 2012-2013, which was Sir Alex Ferguson's final season in charge.
You can see the red line indicating the stronger performance that Manchester United achieved with Sir Alex Ferguson, and how the comparison between the two newer managers pales into insignificance when you look at how they've performed against him. There's a message here about comparing two test recipes when they've both performed badly against the control recipe, but we'll move on.
There have been some interesting results for Manchester United already this season, in particular, a defeat by Leicester City (a much smaller team who had just been promoted into the Premier League, and were generally regarded as underdogs in this match). The 5-3 defeat by Leicester rewrote the history books. Among other things...
- It was the first time Leicester had scored five or more goals in 14 years- It was the first time Man Utd have ever conceded four or more goals in a Premier League game against a newly-promoted team
- It was the first time Leicester City have scored four or more goals against Manchester Utd in the league since April 1963
But apart from the anecdotal evidence, what statistical evidence is there that we could point to that would highlight the reason for the recent decline in performance? Where should the new manager focus his efforts for improvement -based on the data (I haven't watched any of the matches in question).
Let's compare three useful metrics that show Manchester United's performance over the first 10 games of the season: goals scored, goals conceded and clean sheets (i.e. matches where they conceded no goals). Same colour-scheme as before:
This graph highlights (in a way I was not expecting) the clear way that Sir Alex Ferguson's successors need to improve: their teams need to score more goals. I know that seems obvious, but we've identified that the team's defence is adequate, conceding fewer or the same number as in Alex Ferguson's season. However, this data is a little-oversimplified, since it also hides the 5-3 defeat I gave as an example above, where the press analysis after the match showed 'defensive frailties' in the Manchester United team. Clearly more digging would be required to identify the true root cause - but I'd still start with 'How can we score more goals'.
Disclaimers:
- The first ten games for each season are not against the same teams, so the 2012-13 season may have been 'easier' than the subsequent seasons (in fact, David Moyes made this complaint before the 2013-14 season had even started).
- Ten games is not a representative sample of a 38-game season, but we're not looking at the season, we're just comparing how they start. We aren't giving ourselves the benefit of hindsight.
- I am a Liverpool fan, and at the time of writing, the Liverpool manager has had a run of four straight defeats. Perhaps I should have analysed his performance instead. No football manager is perfect (and I hear that Arsenal are also having a bad season).
So: should Manchester United sack Louis van Gaal? Well, they didn't sack David Moyes until there were only about six matches left until the end of the season; it seems harsh to fire Louis van Gaal just yet (it seems that the main reason for sacking David Moyes was actually the Manchester United share price, which also recovered after he'd been fired).
I whole-heartedly endorse making data-supported decisions, but only if you have the full context. Here, it's hard to call (I haven't got enough data), especially since you're only looking at a before/after analysis compared to an A/B test (which would be a luxury here, and probably involve time travel). And that, I guess, is the fun (?) of sport.
One area which I also like to look at periodically is sport - in particular, football (soccer). I've used football as an example in the past, to show the importance of picking the right KPIs. In football, there's no A/B testing - which players should a manager select, which formation should they play in - it's all about making a decision and seeing what happens.
One of my least favourite football teams is Manchester United. As a child, my friends all supported Liverpool, and so I did too, having no strong feeling on the subject at the time. I soon learned, however, that as Liverpool fans, it was traditional to dislike Manchester United, due to their long-standing (and ongoing) rivalry. So I have to confess to slight feeling of superiority whenever Manchester United perform badly. Since the departure of their long-serving manager, Alex Ferguson, they've seen a considerable drop in performance, and much criticism has been made of his two successors, first David Moyes, and now Louis van Gaal. David Moyes had a poor season (by Man Utd's standards) and was fired before the end of the season. His replacement, Louis van Gaal, has not had a much better season this far. Here's a comparison of their performance, measured in cumulative points won after each game [3 points for a win, 1 for a draw, 0 for a loss].
Well, we can see that performance in the current season (thick green line) is lower than last season (the blue line). Indeed, after game 10 in early November 2014, the UK media identified that this was the worst start to the season since 1986. But since then, there's been an upturn in performance and at the time of writing, Manchester United have won their last two matches. So perhaps things aren't too bad for Louis van Gaal. However, the situation looks slightly different if we overlay the line for the previous season, 2012-2013, which was Sir Alex Ferguson's final season in charge.
You can see the red line indicating the stronger performance that Manchester United achieved with Sir Alex Ferguson, and how the comparison between the two newer managers pales into insignificance when you look at how they've performed against him. There's a message here about comparing two test recipes when they've both performed badly against the control recipe, but we'll move on.
There have been some interesting results for Manchester United already this season, in particular, a defeat by Leicester City (a much smaller team who had just been promoted into the Premier League, and were generally regarded as underdogs in this match). The 5-3 defeat by Leicester rewrote the history books. Among other things...
- It was the first time Leicester had scored five or more goals in 14 years- It was the first time Man Utd have ever conceded four or more goals in a Premier League game against a newly-promoted team
- It was the first time Leicester City have scored four or more goals against Manchester Utd in the league since April 1963
But apart from the anecdotal evidence, what statistical evidence is there that we could point to that would highlight the reason for the recent decline in performance? Where should the new manager focus his efforts for improvement -based on the data (I haven't watched any of the matches in question).
Let's compare three useful metrics that show Manchester United's performance over the first 10 games of the season: goals scored, goals conceded and clean sheets (i.e. matches where they conceded no goals). Same colour-scheme as before:
This graph highlights (in a way I was not expecting) the clear way that Sir Alex Ferguson's successors need to improve: their teams need to score more goals. I know that seems obvious, but we've identified that the team's defence is adequate, conceding fewer or the same number as in Alex Ferguson's season. However, this data is a little-oversimplified, since it also hides the 5-3 defeat I gave as an example above, where the press analysis after the match showed 'defensive frailties' in the Manchester United team. Clearly more digging would be required to identify the true root cause - but I'd still start with 'How can we score more goals'.
Disclaimers:
- The first ten games for each season are not against the same teams, so the 2012-13 season may have been 'easier' than the subsequent seasons (in fact, David Moyes made this complaint before the 2013-14 season had even started).
- Ten games is not a representative sample of a 38-game season, but we're not looking at the season, we're just comparing how they start. We aren't giving ourselves the benefit of hindsight.
- I am a Liverpool fan, and at the time of writing, the Liverpool manager has had a run of four straight defeats. Perhaps I should have analysed his performance instead. No football manager is perfect (and I hear that Arsenal are also having a bad season).
So: should Manchester United sack Louis van Gaal? Well, they didn't sack David Moyes until there were only about six matches left until the end of the season; it seems harsh to fire Louis van Gaal just yet (it seems that the main reason for sacking David Moyes was actually the Manchester United share price, which also recovered after he'd been fired).
I whole-heartedly endorse making data-supported decisions, but only if you have the full context. Here, it's hard to call (I haven't got enough data), especially since you're only looking at a before/after analysis compared to an A/B test (which would be a luxury here, and probably involve time travel). And that, I guess, is the fun (?) of sport.
No comments:
Post a Comment