With Paul the Octopus, Nelly the Elephant and now Achilles the Cat predicting match results, animal intuition has been reigning popular for forecasting game outcomes. However, predicting match results in a dynamic sports field poses an interesting challenge for data scientists and experts in machine learning. A wide range of variables could swing the match either way. The direction of the wind, for example, could shift the trajectory of a shot enough to reduce the chances of scoring. Subtle changes in humidity could have an impact on the player’s speed, the striker’s chances of scoring could decline over time, and the team’s formation may have an impact on defensive strategies.

During the 2014 World Cup, Bing users searching for updates on their favorite teams were greeted with a surprising result - predictions on the winning team. This was Microsoft’s first foray into an Artificial Intelligence (AI) powered in-game prediction engine - Bing Predicts. Ever since, Bing has predicted the results of various big-ticket sporting events.

Backed by a machine learning Model, Bing Predicts envisages the winning chances of teams in various sporting tournaments. Bing accurately predicted the winners of 6 out of 8 knockout stage games so far, in the ongoing 2018 World Cup, including the victory of France over Argentina. Its prediction of Belgium’s win over England in the group stages was against odds but Bing’s pick persevered.

Today, Bing Predicts can envisage possible outcomes for sports events across the world be it football, cricket, tennis or individual sports like track and field events.

Creating the prediction engine

Predicting the outcome of live competitions is a breakthrough in predictive analysis and machine learning. With many variables, live events like football matches and reality TV shows are rich in data but challenging to predict. Creating a model for accurately forecasting these dynamic events was an interesting experiment for our researchers. Here’s how the model was created and improved over the years.

Every sport is unique and has different parameters. However, all competitions can be divided into two broad groups - individual and team-based. Individual sports like sprints and motor races have a ranking system for contestants whereas team-based sports like tennis and football matches are zero sum games where one side must come out on top.

Our objective with the prediction model was to analyze extensive historical data to predict the winners in team-based matches and to rank the winners in individual sports. For each sport we followed four steps to create a unique and accurate prediction model.

Step 1:  Gathering data

Every variable has an impact on the outcome of each match. To make the model as accurate as possible, we gather data on the minutest aspects of historical matches. Data on the structure of the team, age of players, performance records, strength of the schedule, margin of past victories, tendency for home-field advantage, weather conditions, and the texture of the playing surface are all considered for every match.

Step 2:  Creating features

Features add further dimensions to the raw data. We apply comparative features such as how well a certain team has performed against another team in the past. Aggregate features like the number of games a team has played helps determine experience. Unique features like the overtake friendliness of a race circuit are also considered. Official rankings of players and teams are a separate feature. The final step is to give each feature a weightage based on the page rank algorithm. The data enriched with game-specific features enables the model to build accuracy.

Example

To predict the outcome of any match, we gather data on each player from both teams. This could include the number of matches played, the goals scored, the goal attempts blocked, the average speed of sprints, and the general position of players. This data is overlaid with features like the percentage of wins one team has experienced against the other in the past, adjusted for the players who have never faced each other before, and weighted by the official ranks of each team published by the official member association. The ranks establish the performance of each team over the course of the tournament in terms of recent competency and fitness. Ranks are normalized for time decay and comparative performances in different tournaments for accurate results.

Step 3: Training the machine

We train the model with feature-enriched data to classify teams and individual players. This classification can be either ‘winner’, ‘loser’ or ‘draw’ in a team-based sport or ‘top three’ or ‘bottom three’ in an individual ranked sport. The model runs analysis on all the data to determine the margin of victory for each match. The choice of the exact model depends on the sport being predicted. The features and data are analyzed multiple times to arrive at a series of different outcomes. The model then creates a prediction based on the aggregate of these outputs.

Step 4: Checking for accuracy

Measuring accuracy is an essential part of the predictive model. The ratio of correct outcomes to incorrect ones is good enough for team-based sports. For individual sports, the accuracy measurement is more complex. We apply Normalized Discounted Cumulative Gain (NDCG) to see how accurate the model is. This gives us a clear indication of the predictive capacity for each rank in the individual sport. For motor racing, we focus on the top three ranks to make the predictions precise since the racers who end up on the podium are crucial.

Cues from ‘wisdom of the crowds’

Statistical data is not enough to accurately predict live games. Bing Predicts applies this model with two more layers of data - anonymized crowd sentiments and real-time updates. Anonymized web activity helps apply the ‘wisdom of the crowds’ to this model while real-time updates on player injuries and unforeseen suspensions help augment the prediction engine.

During our research, we have noted that web activity is less biased than polling. Web trends analysis is hence, more insightful. In fact, listening to the ‘wisdom of the crowd’, can enhance the accuracy of Bing Predictions by 5%, a significant number for accuracy!

The algorithm derives sentiments and real-time data from web activity and public sentiments across social media. This part of the model typically picks up fans’ views and the latest happenings on controversies, injuries, changes in line-ups, suspensions and more. For instance, Germany and Brazil entered the semifinals of the 2014 football tournament as the favorites to win – with Germany ranking second and Brazil ranking third in world rankings. However, just before the crucial match, Brazil lost two key players – a key striker to an injury and its captain and defender to accumulation of yellow cards. This was the key determinant prompting the model to choose Germany as the winner in the pregame predictions.

Deeper insights

Over the years the engine has been modified and tweaked to increase precision and provide deeper insights into upcoming events. These come directly from the statistical features that the model uses and crowd sentiments. For example, predictions for the knockout stages of the current football World cup show the strength and weakness of each team and what it would take for any of the teams to win.

Further, facts from their past matchups reveal interesting insights.

Foretelling the future

Sports predictions are different from forecasts of events based on popularity and voting. Indeed, sports are fun because of their unpredictability. The actions of a handful of players can affect the outcome of games and tournaments. The uncertainty makes forecasting sporting events more challenging and fun. Today, our engine provides users predictions for who will win, the percentage of our prediction confidence, and the reasons for each victory across the most popular sporting events.

You can view the experience at https://www.bing.com/search?q=fifa. Bing users can follow our model’s prediction closely to see whether the engine can surpass its own record.