What do Ireland, France, England, Fiji and the All Blacks all have in common? A new advanced analytics tool, one that will sicken fans of those countries, has emerged as a uniting force. According to expected points (xP), a statistic published by data company Opta, all of these teams were knocked out of the World Cup in matches which they should have won.
Soccer fans will be familiar with the idea of expected goals, or xG. A metric used to analyse the quality of goal-scoring opportunities for individual players and teams, the stat has become so ubiquitous that Match of the Day incorporates it into its post-match graphics.
A decade on from xG’s introduction to soccer, rugby is starting to catch up. Expected points (as they are in rugby, rather than goals), has been shared with rugby fans since 2013 – the probability of success when a kicker is placing the ball on a tee is a regular feature on TV. Yet it was only during the recent World Cup that xP for entire rugby matches was made available to a wider audience.
An easy knock against expected stats is that they are almost always wrong. During their quarter-final defeat, Ireland were nearly seven points better than the All Blacks according to Opta, but still lost by four.
“I would always say that xP is never supposed to equal points scored,” explains Will Kane, product manager for Opta Rugby, the company that provided xP figures for World Cup broadcasts. “It’s supposed to be a way of showing how many points we expect someone to score from the quality of possession and the quality of game they played.
“Understanding that you have a high xP but are not hitting it doesn’t necessarily mean you’re attacking badly, the other team is just defending really well. Just like in football, not matching a high xG can mean the goalkeeper is having a really good game.”
France, England and New Zealand all lost to South Africa in the knockouts despite in some cases comfortably outperforming them on xP. Praise of the Springboks’ defence has been widespread across both of their back-to-back World Cup triumphs. Here is a way of showing just how good it is statistically – they regularly force teams to underperform their xP.
While matches are being played, researchers work out the percentage of a team scoring a try from any given possession based on a number of factors.
Some of them are obvious. “Are we expecting a low number of points to be scored on a wet day?” explains Kane. “Does the team that has the ball have a high propensity to score? Does the defence have a high propensity to concede? Player advantage, is there a yellow card and is that player in the forwards? Is it multiple players?”
Other factors are more granular. “We’re looking at where they started the possession,” says Kane. “How the game has gone so far, are they tackling well, are they attacking well? Gainline success rate, retention of ball, distance made in possessions, lots of bits and pieces go into it.
“The bigger thing is the historical side of it. If a team is historically poor or strong in these areas then we assume there will be some sort of an effect. We would expect a good scrummaging team is going to stay consistent as a good scrummaging team.”
Given all these factors, a percentage is fixed to each possession indicating the likelihood a try will be scored from that position at that time. If that figure is 50 per cent, then in theory the xP of that attack is 2.5 points, half of the value of the five awarded for a try.
Only it’s not that simple. Opta also looks at the historical likelihood of a team taking a drop-goal during that possession, or of winning a kickable penalty. “If England has the ball we know there’s a high chance of them going for a drop-goal compared to New Zealand,” explains Kane. “It wouldn’t be just 2.5 xP if there is a 50 per cent chance of scoring. It would be slightly lower or slightly higher depending on the team.”
The sum of the xP of all a team’s possessions equals the final xP figure over 80 minutes.
The model is not perfect. When, in the semi-final, South Africa infamously hooked their running outhalf Mannie Libbok after 30 minutes in favour of Handré Pollard, a stronger tactical kicker, that should have lowered their chances of scoring from attacks starting in their own half. Opta’s modelling did not allow for such a stylistic change in personnel.
“We are planning to, because we realise how important that is to their game plan,” says Kane. “Comparing how differently the two of them play on the field, it’s like chalk and cheese.”
The parochial question nagging at the back of the mind throughout this conversation inevitably brings us back to Ireland. How did they outscore the All Blacks by a converted try on xP but still lose in real life?
“The maul being held up was a big one,” says Kane of Jordie Barrett’s ability to deny Rónan Kelleher in the second half. Given the location of that possession deep inside the 22 and the recent historical success – Ireland’s maul earned a penalty try minutes earlier – that passage had a very high xP which Ireland underperformed.
Similarly, Ireland’s xP during the two sin-bin periods endured by the All Blacks was high, but they undershot the figure.
“In the first half, there were four times where Ireland’s attacking threat and xP was significantly higher than New Zealand’s but they weren’t able to take advantage,” explains Kane. “Then in the second half, between the 55th and 70th minute we’re saying Ireland should have scored 12 points and they were quite a bit under that. There was the penalty try and that was it. As far as we’re concerned that’s the difference in the game.”
“For the last possession of the game, [Ireland launched a 37-phase attack from deep] that xP is quite low,” says Kane. “It’s not just starting quite a way out, but the pressure. We don’t regard it that highly, we don’t have enough data but it is taken into account slightly that teams aren’t as good chasing a score late in the game.”
Expected points can go even further. Opta is currently working on a way of calculating the xP contributed by each individual player. They’re not ready to release that yet.
One data analyst who has shared such information publicly is Simon Chi. Based in Canada, Chi, who coaches the University of Calgary’s rugby team and has also worked as an analyst with the Canada women’s sevens, published a team of the pool stages based on a metric he calls expected points added (EPA). A value is assigned to an individual that reflects all the positive things they do minus the negative.
Calculations are also based on the percentage of scoring a try in any given situation at a given point on the field. Chi looks at every contribution made by a player – tackles, tackles missed, ruck arrivals, passes, penalties conceded (plus countless others) – assigns a value over average and works out the EPA, accordingly.
“If you do a goal-kick, let’s say you have a 67 per cent chance, two out of three” he explains. “If you make it you gain an xP of one since that’s the additional value [added by the player], but if you miss it you get minus two. There are a lot of events. If you make a positive tackle, the impact on the probability of the opposition scoring goes into your EPA.”
In a shock to more parochial tendencies, Chi ranked Anton Lienert-Brown (32.2) higher than Bundee Aki (29.21) after the pool stages. Granted, the former was part of an All Blacks side that put 73 points on Uruguay and 96 on Italy. EPA is proportional to performance, so high individual scores should translate to the scoreboard. Chi backs his model because, when creating team rankings in the English Premiership from adding up individual player EPA, he mirrored the final league table.
Intriguingly, Chi gave Garry Ringrose (30.40) a higher EPA score than Aki during the pools. “Ringrose did slightly better in kicking and as a playmaker scored much higher,” explains Chi. Yet of all the players who earned a high EPA score, Italian second row Federico Ruzza was the most surprising. The Benetton forward scored higher than any other lock during the pool stages (19.93).
“Ruzza is someone you’d want to look at because he’s not in a top team yet he’s performing with the top players in the tournament,” says Chi. “If you’re in the All Blacks in the second row you’re probably not expected to do as much for your team as Ruzza is for his, but if you’re building a squad he’d be someone you’d want to look at.”
Ruzza was only fifth among locks for total tackles in the pool stages, but he was highest for tackle EPA. If you looked at just the quantity of tackles, he wouldn’t stand out. But this metric shows that Ruzza’s tackling had a bigger impact on his team’s chances of winning, showing the limitation of the surface-level stat.
Recruitment appears to be where Chi’s metric has the most untapped value. It is potentially rugby’s answer to the Moneyball phenomenon, one of finding previously unseen value in a player.
“In 2014, I was doing an evaluation for a team,” says Chi. “This was for hookers and this guy was right at the top. I was like, ‘Who’s Harry Thacker?’ You do research on him and he doesn’t get a look because he’s small.
“At the time he was fourth choice at Leicester. It took a guy like Pat Lam to take him to Bristol and now he’s still playing at a high level. Does this happen all the time? No. It’s not easy.”
Recruitment is clearly a space where xP and EPA can be expanded – some teams are using it already. Yet in terms of data available for fan consumption, rugby is “20, 30 years behind most other sports,” says Chi, especially North American ones.
Both Chi and Kane at Opta see expected points to lead the way in making rugby fans more data-literate. “The way World Rugby uses it is good, but there’s a lot more that can be done in that broadcast space, the storytelling of a match,” says Kane. “That’s going to be interesting to see how that happens.
“It’s meant to help entertain and explain the game a little bit, have a bit of fun with it and try an innovate where we can.”
