Lawro vs. Data

After the Nate Silver talk (see previous blog post), there was some discussion around TV pundits and their predictions. Naturally, the subject drifted to football pundits, and in particular Mark Lawrenson of the BBC. Mark Lawrenson is a former professional football player, and now long time BBC pundit who regularly appears on the famed Match of the Day programme. For several years, he’s been making predictions for each EPL matchday, and pitting himself against a guest. We thought it might be fun to do a little analysis of his results, and investigate how expert this expert really is.

For each matchday, Lawro makes predictions of scores for each match to be played. A tally is kept of his correct results and correct scores. His predictions can be found at the BBC site: http://www.bbc.co.uk/sport/, or compiled at the site: http://www.myfootballfacts.com/.

For now, our analysis has focussed on match outcomes rather than specific scores. Predicting scores is much harder; think of all the combinations, versus three possible match outcomes.

Let’s get into some analysis. Here are the statistics for the last three EPL seasons, including the 2012-2013 season that finished recently. Data grabbed from: www.football-data.co.uk/.

Result

2010-2011

2011-2012

2012-2013

Average

Home Win

179 (47.11%)

171 (45%)

166 (43.68%)

45.26%

Away Win

90 (23.68%)

116 (30.53%)

106 (27.89%)

27.37%

Draw

111 (29.21%)

93 (24.47%)

108 (28.42%)

27.37%

The first thing you’ll notice, and may well be surprised by, is the just how strong the Home bias is. Recent analysis in the book ‘Scorecasting’ by Moskowitz and Wertheim claims this is due to (unintentional) referee bias. Also, you’ll notice that the averages across seasons for Away Wins and Draws are equal. Interesting.

Now, here are Lawro’s results for the last three seasons.

2010-2011

2011-2012

2012-2013

Average

Correct Results

168 (44.21%)

179 (47.11%)

201 (52.89%)

48.07%

Correct Scores

41 (10.79%)

41 (10.79%)

44 (11.58%)

11.05%

You’ll immediately observe that Lawro is getting better at predicting results, and that his correct scores’ percentages are pretty steady.

These results really beg the question: How is Lawro getting so much better? Is he using odds or modelling to make his predictions?

Back to it. Given these results, if you were to predict a Home Win for all games (all 380 games in an EPL season), you would, on average, get 45.26% results correct. Not too bad. Better than Lawro in 2010-2011. But he has got a lot better since then.

Based on this analysis, and our discussions, we decided to develop our own strategy for predicting outcomes. We thought we would try using Home Wins as a basis.

Our simple strategy is as follows:

  • choose Home Win, unless home team in current bottom 3, then result is Away Win;

  • or pick Away Win if away team is in current top 4, and home team not in current top 4;

  • if no games played, i.e. beginning of the season, then pick Home Win.

The intuitive idea being that, particularly at the top and bottom, current league position is a pretty good indicator of the strength of a team, and so we combine it with Home Wins (given the enormous Home bias). We predict no draws.

We evaluated our strategy, simulating each season’s games and generating league tables as we simulate, and here are the results:

2010-2011

2011-2012

2012-2013

Average

Correct Results

177 (46.58%)

190 (50%)

182 (47.89%)

48.16%

The results show that we are better than Lawro for the 2010-2011 and 2011-2012 seasons, and slightly better on average across the three seasons. But Lawro had a great season this year!

But how good are Lawro’s and our predictions really? We compared the results with pre-match odds for game outcomes for two high-street bookies. We grabbed the data from: http://www.football-data.co.uk.

We transformed the provided decimal odds to probabilities (1/odd), then picked the most likely outcome (highest probability) as the match outcome. If the odds reflect a good estimate of the true probability of the outcomes, we would expect them to predict results pretty well in the long run.

The results for William Hill and Ladbrokes are:

2010-2011

2011-2012

2012-2013

Average

William Hill

191 (50.26%)

199 (52.37%)

201 (52.89%)

51.84%

Ladbrokes

191 (50.26%)

198 (52.11%)

204 (53.68)

52.02%

Observe that the bookies only predict just above 50% of the results correctly. And, yes, you are reading that right: this season Lawro did as well as William Hill!

To conclude, on the one hand, we’re impressed that our simple strategy appeared to work better than Lawro. But on the other, we’re intrigued by how much better he was this season. When does next season start?

Final note: our strategy could probably be improved through a better initialisation. Currently, we pick Home Win if no games have been played yet. It would be possible to use the previous seasons’ final league table instead, plus some exceptions for promoted teams. Exercise for the reader.

Our analysis and tools can be found at: https://github.com/tfrench/lawro/.

Comments are closed.