Moneyball: Sports Analytics in Soccer to Predict Performance and Outcomes

There is no doubt that soccer is the most popular sport in the world, and its popularity is growing in the US. Over 25 million fans watched U.S. Women’s FIFA World Cup 2015, and earned Fox over $40 million in ad revenue. Similarly, the United States’ 2-2 draw with Portugal in the 2014 World Cup was seen by an average of 24.7 million viewers on Univision and ESPN.
What's Sports Analytics?

Sports analytics is the processes that identify and acquire the knowledge and insight about potential players’ performances based on the use of a variety of data sources such as game data and individual player performance data. These advanced and sophisticated type of analytics should be able to extract valuable actionable insights for the coaches and managers to utilize.

Sports analytics can be utilized in various domains including:
Predicting the outcome of a game
Predicting the performances of teams or individual players
Building new strategies for upcoming competitions
Deciding the price of a player if a club was to rent/sell/buy him or her
Connecting players to brands and sponsors

Of course, not all teams use analytical tools. In addition to the costs involved, there’s also the problem of explaining complex analytical methods to coaches in ways they can understand. Thus, soccer analytics is more widespread in big clubs where they have the necessary financial power to utilize these methods.

But that should not necessarily be the case. Since traditional sports analysts do not reveal the logic behind their methods, we’ve decided to give you a flavor of what can be done with soccer data. While this post will not answer all the questions you have about soccer analytics, it can help you understand how to get started.

In this project, we have used a very limited number of player’s attributes (See the section under Player’s Features/Attributes) that are easy and not expensive to gather, to both reverse engineer the most advanced Rating and Performance index (The results are presented in Figures 1-12) and then to propose a more robust and easy model for future player ratings and performance prediction (See the section under Machine-Learning and AI Models). The program runs on Spark and Cloud Environment, and can be used for Terra-Petta scale of data, from multiple years, with thousands of players, with 100s of attributes.
What's Soccer Analytics?

Soccer Analytics is the art of creating insights and actionable decisions using soccer related data. While predictive analytics uses big data to determine the probability or the likelihood of a certain outcome, intelligent descriptive analytics looks at big data and analyzes it using machine learning and artificial intelligence methods to come of with suggestions that will improve the likelihood of a desired outcome.

Some important concepts to know while conducting this analysis are:
Game Modeling: Modeling of the game before, during and after the game using scientific techniques to match or predict a set of outcomes.
Expert Player Rating: Players ratings given by an expert. These ratings take a black-box approach, and they vary according to the prior knowledge of the expert.
Soccer Performance Analytics: Is a tool to help players, coaches and managers to quantitatively assess the players and team performance and help to improve both players and team performance and design a set of wining strategies for upcoming game(s).

Here are some questions to ask when running analysis on soccer players:
How does expert rating differ from ratings generated by data-driven Machine Learning ratings?
What are the most important players’ attributes linked to their performance?
What criteria do experts use when they evaluate Players? Is there a way to reverse engineer their criteria?
Which attributes are important for each specific position?
Can we use ratings and players’ attributes to predict the outcome of a game?
Can we aggregate the players’ rating to come up with a team rating?
Is there a way to correlate the team rating to the outcome of the game?
Do the outcomes of games influence expert ratings more than individual performance indicators do?
Can we predict the outcome of a new game given the past performance of the players?

Using advance analytics and visualization tools such as Machine Learning and network analytics to predict the outcomes of soccer games is becoming more and more popular as these methods continue to move into the mainstream with the help of tools that make it easier to conduct these advanced analytical methods.
Some insights to our approach:

In this project, we used our selected set of clustering and classification techniques and the best model was selected and ranked based on Train-Validation-Test process.
Companies such as OPTA, Prozone, Amisco, and WhoScored are now collecting rich soccer data. These can be utilized to conduct accurate assessments.
For our project, a rich data set containing more than 210 attributes of players including 198 performance statistics were used. To calculate the overall performance and ratings of the players, some or all of the attributes were being used. Some of the very advanced Expert Ratings include: Caapello Index, Castrol Index, and WhoScored.com. These Ratings include each player’s cumulative ratings and game-based ratings.
For classification-regression and clustering, there are many Machine learning models that can be used. For classification-regression model, you can use Machine Learning models (SVMs, logistic regression, linear regression), naive Bayes, Regression by Discretization using J48, Additive Regression with Decision Stump, decision trees, ensembles of trees (Random Forests and Gradient-Boosted Trees), isotonic regression, Multilayer Perceptron, RBF Network. For Clustering, you can use k-means, clustering using affinity propagation, Agglomerative Clustering (Ward, Average, and Complete), Gaussian mixture, power iteration clustering (PIC), latent Dirichlet allocation (LDA). Furthermore, you can use dimensionality reduction such as singular value decomposition (SVD) and principal component analysis (PCA) to reduce the feature space. In our case study, we tested all the models and the best results combined and are presented in Figures 1-12, without elaborating about specific model and how they can be aggregated.
Players' Features/Attributes:

In this project, we have used a subset of the Player’s Features/Attributes from the following list. Keep in mind that your model should be able to select and rank these attributes based on their importance. At this blog, we are not elaborating on what features have been selected by models, as these will depend on the specific approach you choose to take when building your model.
Nationality, Club, League, Age, Height, String Foot, Position (GK, CB, RB, LB, DM, CM, RM, LM, AM, RW, LW, SS, CF)
Attacking Prowess, Ball Control, Dribbling, Low Pass, Lofted Pass, Finishing
Place Kicking, Swerve, Header, Defensive Prowess, Ball Winning, Kicking Power, Speed, Explosive Power, Body Balance, Jump, Stamina, Goalkeeping, Saving, Form, Injury, Resistance, Weak Foot Use, Weak Foot Accuracy, Trickster, Mazing Run, Speeding Bullet,, Incisive Run, Long Ball Expert, Early Cross, Long Ranger
Scissors Feint, Flip Flap, Marseille Turn, Sombrero, Cut Behind & Turn, Scotch Move, Long Range Drive, Knuckle Shot, Acrobatic Finishing, First-time Shot, One-touch Pass, Weighted Pass, Pinpoint Crossing, Outside Curler, Low Punt Trajectory, Long Throw, GK Long Throw, Man Marking, Track Back, Captancy, Super-sub, Fighting Spirit

While, we do not have intention to disclose the final optimized features selected by our optimized model and strategy, we used the following ML-AL based aggregator operator model (Figure 13 shows the Structure of our Committee Machine).
Figures:

Finally, here's a list of the plots and figures created as a result of this analysis:



Figure 1. Clustering (similarities of players’ clusters) and prediction of the rating (plot: prediction vs. actual) for the Forward Players. Train, test and Validation. The similarities of the individual players are shown by the lines on clustering plots.



Figure 2. Clustering (similarities of players’ clusters) and prediction of the rating (plot: prediction vs. actual) for the Goalkeepers. Train, test and Validation. The similarities of the individual players are shown by the lines on clustering plots.



Figure 3. Clustering (similarities of players’ clusters) and prediction of the rating (plot: prediction vs. actual) for the Defensive Players. Train, test and Validation. The similarities of the individual players are shown by the lines on clustering plots.



Figure 4. Clustering (similarities of players’ clusters) and prediction of the rating (plot: prediction vs. actual) for the MidField Players. Train, test and Validation. The similarities of the individual players are shown by the lines on clustering plots.



Figure 5. Clustering (similarities of players’ clusters) using advanced Visual-Analytics-Clustering for Forward Players. The similarities of the individual players are shown by the lines and their strength with the width of lines on clustering plot.



Figure 6. Clustering (similarities of players’ clusters) using advanced Visual-Analytics-Clustering for Forward Players. Similar clusters are closer to each other.



Figure 7. Typical performance of our Regression model’s Prediction for the rating of the Forward players (Prediction Vs. Actual). Train data for Forward Players.



Figure 8. Typical performance of our Regression model’s Prediction for the rating of the Forward players (Prediction Vs. Actual). Validation data for Forward Players.



Figure 9. Typical performance of our Regression model’s Prediction for the rating of the Forward players (Prediction Vs. Actual). Test data for Forward Players.



Figure 10. Typical performance of our Regression model’s Prediction for the rating of the players (Prediction Vs. Actual). Test data for Forward-MidField-Defensive Players.



Figure 11. Typical performance of our Regression model’s Prediction for the rating of the players (Prediction Vs. Actual). Validation data for Forward-MidField-Defensive Players.




Figure 12. Typical performance of our Regression model’s Prediction for the rating of the players (Prediction Vs. Actual). Test data for Forward-MidField-Defensive Players.



Figure 13. Committee Machine, and Intelligent Multi-Level-Aggregator Tree (MAT) platform.

HOW THE MODEL WORKS

At a high level, our approach is as follows. First, we estimate a regression model to predict the number of goals scored by a particular team (“team i”) against a particular opponent (“team j”) using the entire history of mandatory international matches since 1958, when the first European championship was played (a total of 4,719 matches).[1] Following the literature on predicting football matches, we assume that the number of goals scored by team i is described by a so-called Poisson distribution and explained by the following statistical factors:[2] 1. The difference in team performance as reflected in Elo ratings prior to the match. The Elo system was originally devised to rank chess players. It is a composite measure of national football team success that evolves depending on a team's results and the strength of its opponents. 2. The number of goals scored by team i in the last 10 competitive matches. 3. The number of goals conceded by team j in the last 2 competitive matches. 4. A home dummy. 5. A European Championship dummy to capture whether a team does systematically better at European Championships than in other competitive matches Second, we use these regression estimates and our assumed Poisson distribution in a Monte Carlo simulation with 100,000 draws to generate a distribution of outcomes for each of the 52 matches, from the opener between France and Romania on June 10 to the final on July 10. We use the rounded prediction of the goals scored to determine the outcome of each match during the group stage and the unrounded prediction to pick the winner in the knockout stage. Third, we use the estimation results to generate both a set of probabilities that a particular team reaches a particular stage of the tournament, up to and including the championship, and a modal—that is, single most likely—forecast for the outcome of each match, which we then run forward through the tournament until the final.


A SUMMARY OF PREDICTIONS
Our probabilities are shown in Exhibit 1. The model says that France has a 23% probability of winning the trophy, followed by Germany at 20%, Spain at 14%, and England at 11%. Although Germany has the highest Elo rating, France is favored because of its home advantage.




Exhibits 2 and 3 provide a different perspective by showing the modal prediction for the entire tournament. There are some interesting contrasts with the probabilities in Exhibit 1. For example, Exhibit 1 says that Germany is more likely than Spain to win the tournament because it is more likely to succeed across the entire range of possible tournament configurations. But Exhibit 3 says that in the single most likely case, Spain beats England in Semifinal 1, France beats Germany in Semifinal 2, and France then wins the final—i.e., Germany finishes behind Spain. Which approach is better, the probabilistic one in Exhibit 1 or the modal one in Exhibits 2 and 3? A modal forecast does have the advantage of being more “crisp.” The sentence “Goldman Sachs says France will win” has a better ring to it than “Goldman Sachs says France has a 23% probability of winning, with Germany close behind.” Nevertheless, we think that a probabilistic approach is more useful—for predicting the outcome of football tournaments and, increasingly, for our day-to-day work on economic forecasting.



Exhibit 4 provides more insight into the results by breaking down the probabilities of winning for the top four teams in a “waterfall chart” format. It shows that the most important factor is the Elo score, followed by home advantage and the European Championship dummy. The chart illustrates that the front-runner position for France derives largely from its home advantage, as its Elo rating is well below Germany’s and also a bit below Spain’s. Meanwhile, Germany benefits from the European Championship dummy, which picks up its historically strong tournament performance. 





HOW CONFIDENT CAN WE BE?

It is difficult to assess how much faith one should have in these predictions. On the plus side, our approach carefully considers the stochastic nature of the tournament using statistical methods, and we do think that the Elo rating—the most important input into our analysis—is a compelling summary of a team’s track record. On the minus side, we ignore a number of potentially important factors that are difficult to summarize statistically, including the quality of the individual players unless they are reflected in the team’s recent track record.[4] And there is no room for human judgment (which may not be such a bad thing given that none of us are really football experts but some are enthusiastic Germany supporters).[5]



One useful cross-check is to compare our results with bookmakers’ odds. Exhibit 5 plots our estimated championship probability against the average probability implied by the odds offered by five different bookmakers. The basic result is clear. Even though our model does not include bookmakers’ odds in any way, the probabilities are quite similar. A possible reason is that professional betting firms use many of the same inputs—such as Elo ratings—in their analysis and that they process the information in ways that are ultimately similar to ours. Another useful check is to evaluate the performance of our model for the 2014 World Cup, which followed an essentially identical approach to the one presented here.[6] It is safe to say that we had our hits and misses. First, performance in the group stage was not great. The model only identified 9 of the 16 advancing teams and failed to predict the elimination of heavyweights Spain and Italy, although it correctly anticipated that England would fly home early. Second, the model gave Brazil a 48% probability of winning the trophy, by far the highest of all the contestants. That failure illustrates a certain lack of imagination that is inherent in our approach. If the greatest football nation on earth—in terms of both past victories and its 2014 Elo rating—plays a World Cup at home, we are bound to project success. At least the probability was below 50%! Third, the model did correctly identify three of the four semifinalists before the start of the tournament, namely Argentina, Brazil, and Germany, although it incorrectly picked Spain over the Netherlands. Fourth, the fully updated version of the model—that is, the projection we sent out before each day of play on the basis of updated Elo ratings and other performance measures—was remarkably accurate during the knockout stage. It correctly predicted the winner of every match except the 7-1 semifinal between Germany and Brazil. But that was, by one estimate, the single most surprising result in World Cup history.[7] Ultimately, this last predictive failure might best capture the spirit of the exercise. As we said in our comment at the time: “Speaking as forecasters, we regret the miss. But, speaking as Germans, we would note that there are more important things than being right.” May the best team win and let’s hope that watching Euro 2016 is as much fun as it was to write this article!

THE ECONOMETRICIAN’S TAKE ON EURO 2016

We present a model for predicting the outcome of the 2016 European Football championship in France from June 10 to July 10. It is similar to our model for the 2014 World Cup. Using historical performance data for each team—most importantly the Elo rating system originally devised to rank chess players—we estimate a set of probabilities that a particular team will reach a particular round, up to and including the championship. We also provide a modal “most likely” case for how the tournament will unfold (although “most likely” does not mean “likely”). The model says that France has a 23% probability of winning the trophy, followed by Germany at 20%, Spain at 14%, and England at 11%. Although Germany has the highest Elo rating, France is slightly favored because of its home advantage. After each day of play, we will re-run the model using updated historical performance data in order to generate new probabilities and a new modal forecast. How much faith should we have in these predictions? On the plus side, our approach carefully considers the stochastic nature of the tournament using statistical methods; also, the predictions are not far from bookmakers’ odds. On the minus side, the environment is “stochastic” indeed, i.e., football is quite an unpredictable game! That charming unpredictability was on full display two years ago, when our model failed to anticipate the elimination of heavyweights Spain and Italy in the group stage and gave Brazil a 48% probability of winning the trophy. More encouragingly, it identified three of the four semifinalists before the start of the tournament, and the fully updated version predicted the winner of every match in the knockout stage except for the 7-1 semifinal between Germany and Brazil. Below we will introduce our statistical model for predicting the outcome of the 2016 European Football Championship in France from June 10 to July 10.