Outs Above Average: Why Do We Care?
Analyzing the prevalence and validity of Savant's premiere defensive metric.
Similar to batting and pitching metrics, the ways in which a player is evaluated in modern baseball has shifted drastically, even from the game just twenty years ago. Fielders used to be measured by the number of assists, putouts, and errors they were responsible for in a season, but this has proven to be unreliable for a number of reasons. Errors do not account for a fielder who has a higher range than an average fielder at that position. Since the fielder is capable of a higher range within their position, of course they would have more errors, since they were put in a position where they can make a play more often. All this to say…errors, and therefore, fielding percentage, mean nothing in evaluating a player’s defensive ability.
1. OAA for Outfielders
Outs above average (OAA) is a relatively new metric, introduced by Baseball Savant first in 2015 as a metric to evaluate outfielders. In 2020, Savant updated the formula to include infielders as well, and became one of the prime metrics to evaluate the capability of a fielder. For outfielders, OAA is primarily based on the average catch probability of a ball in play, which is determined by the hang time and the fielder’s distance from the ball. As MLB.com writes:
“For example, if an outfielder has a ball hit to him with a 75 percent Catch Probability -- that is, one an average outfielder would make three-quarters of the time -- and he catches it, he'll receive a +.25 credit. If he misses it, he'll receive -.75, reflecting the likelihood of that ball being caught by other outfielders.”
This separates balls into automatic outs, automatic hits, and a space in-between called the “opportunity space”, which allows a fielder to determine the destiny of a semi-catchable ball. The amount of balls caught in the opportunity space is essentially what determines a fielder’s OAA. Let’s look at Kevin Kiermaier’s 2023 season thus far, in which he leads all centerfielders with 7 OAA.
The deepest teal before the cutoff here represents the statistical limit of the opportunity space, or what a fielder is capable of before a ball is considered an automatic hit. Red represents an out, gray represents a hit, and a green border around a data point represents a play at the wall, which Savant assigns a lower catch probability. Savant considers a number of factors in their formula for this, such as the player’s route to the ball and his jump.
Baseball Savant, Kevin Kiermaier Opportunity Space
What is valuable in a defender’s OAA is their quick reaction, their route to the ball, and their sprint speed. Sprint speed is especially important in creating elite defensive outfielders; correlation does not equal causation, but the top five defensive centerfielders all have sprint speed percentiles above 80%. While an outfielder with non-elite speed can overcome this with good routes and good reaction to the ball, having speed in his game will undoubtedly enhance his ability to track down balls.
2. OAA for Infielders
In certain regards, how OAA is calculated for infielders is similar to its outfield formula. OAA for infielders considers the fielder’s distance from the ball, how much time he has to get to the ball, how far he is from the base the runner is running to, and how fast the runner is. As mentioned earlier, OAA for infielders is a relatively new metric, but it has already proven to be one of the premiere metrics in evaluating infielders.
While not as relevant anymore considering the shift ban that took place following the 2022 season, OAA’s formula considers defensive positioning of the infielders at the time of the ball being hit. MLB.com uses Nolan Arenado’s 2019 season as an example, where Arenado was +17 OAA at third base. However,
“Arenado's +17 actually breaks down into +14 OAA where third basemen typically play, and +3 OAA where shortstops play.”
Obviously this is less relevant in a post-shift world, but considering that MLB teams have still found ways to implement defensive shifts, it is good to know that OAA’s formula will compensate for this. However, if there’s a fault to be had with OAA, it’s the relative inaccessibility of the formula. Nowhere online has Savant posted exactly how catch probability or play probability calculates OAA. While Fangraphs has attempted to recreate MLB’s formula for OAA, the author even writes,
“I’m not going to go into details of how I computed this metric; it’s standard machine learning stuff.”
It may be “standard machine learning stuff”, but even to modern fans of the game this is still relatively inaccessible. Unfortunately, there are times when you have to let sabermetricians be themselves and hope that their metrics match the eye test. All of these stats exist to justify the defensive eye test, where it it easy for a viewer to tell that Ke’Bryan Hayes and Nolan Arenado are the best third baseman in the sport. While all statisticians have an implicit bias in what they may think is valuable in a sample, it is up to the viewer to communicate whether a statistic is valuable. OAA as a statistic possesses significant staying power because it is more or less, accurate regarding defensive ability.
A favorite quote of mine comes from British economist Ronald H. Coase, who in his 1994 book Essays On Economics And Economists, writes,
“If you torture data long enough, it will confess to anything.”