A comparison of the pre-season prediction models

Prior to the season a bunch of predictions were made public as to how the final Premiership standings would look. Unfortunately there were only a few that put points along with their positions. Of those that have done so now is the time to check how each model performed.

So, my predictions are all available if you go back and check the previews for each team, which can be found here, a copy of the Euro Club Index predictions has been e-mailed to me by Simon Gleave (I also witnessed these before the start of the season and can verify their accuracy), and the Pinnacle sports predictions can be found here (h/t Ted Knutson).

As with last seasons review I’ll also include two further categories. ‘Same’ means that teams this year score an equal points to last year, with the three promoted clubs projected to score 38 points apiece, and equal means all clubs are assumed to be equal, and will score 52 points apiece.

So onwards we go. In the table below is a summary of the actual number of points each team scored, along with the number predicted by each of the models mentioned above.

Screen Shot 2013-05-27 at 4.00.13 PM

Now I’m not going to go through club by club but a couple of things stick out. The first is that every model underestimates United (and they were a bad possession team again this year, they’ll be predicted to do poorly again next term), and the second is that every model correctly predicted Newcastle to drop off, but not as drastically as they did. There’s also some interesting divergences but they’re things for me to look into at a later date.

So that’s all well and good, but lets put some numbers to the discrepancies between how the models predicted the league to look, and how it finished. I’m going to do this by looking at the standard deviation of the differences between the number of points predicted and the number of points earned for each model:

Screen Shot 2013-05-27 at 4.06.24 PM

There’s a stand out winner here. The Euro-club Index is significantly ahead of the field. And whilst two seasons isn’t definitive it was also better than the shots models last time around too. Thankfully each and every model beat the ‘equal’ model, so we’re doing something right.

It’s worth noting that when I compare this season to how each of my models have performed since ’00-01 it looks like a pretty average year in terms of how difficult it was to predict the league. So how impressive is the performance of the Euro-club Index? Well not so long ago I suggested that even the best possible predictive models would still have a standard deviation of ~8 points due to random variation alone. We’d need more data to be sure, but on the surface the Euro-club Index has returned a result lower than 8, so it’s not only been very good this year, but has also benefited from some luck too.

Finally, let’s compare these numbers with those of the bookies as a collective, who were predicting the result of each game. The numbers are taken from the right hand side of this helpful tableau graphic put together by Simon Gleave:

Screen Shot 2013-05-27 at 4.23.58 PM

So the bookies end up with a standard deviation of 6.34 points. Why so low? Well they make predictions on a game by game basis, and so have more knowledge to build into their system. I’m not sure how large a number we’d expect due to random variation, but it’d certainly be lower than the 8 points mentioned above. I’m actually pretty impressed that these models can bridge 60-80% of the knowledge gap between ‘no knowledge whatsoever’ and ‘bookies compensating for events that happen between every game’ before the season even begins.


3 thoughts on “A comparison of the pre-season prediction models

  1. Pingback: Premiership predictions – ’13-14 edition | James' Blog

  2. Pingback: 26 Predictions: English Premier League forecasting laid bare | Scoreboard Journalism

  3. Pingback: 2014-15 Premiership Predictions | James' Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s