WAR is Wrong: Top Relievers Are Worth Double their WAR - So D-Rob Easily Merits $12-15m

Relievers are weird creatures the standard wins-above-replacement (WAR) stat doesn't evaluate well. I'll give the punch line first: relievers pitching high-leverage innings are worth at least double what WAR claims. I'll get back to David Robertson, but let's start with Greg Holland, a nice example of a consistently true-elite reliever – but you could substitute prime-years Mariano Rivera, who averaged only 3.0 WAR/yr as a closer. In 2013 and 2014, Holland logged low-1s ERAs and converted 95% of about 50 save opportunities (2-3 blown saves (BS) a year) – yet BBREF and Fangraphs call him only a 2-3 WAR player. Wouldn't a replacement-level pitcher giving up over a run every two innings – call him "Kawn Shelley" – blow at least 20% of save chances, or easily 10 out of 50 rather than Holland's 2-3 out of 50 or Robertson's 5 out of 44? Not all blown saves are fatal – lost leads may be recoverable – but about 60% of blown saves become losses. (That's my rough estimate from looking up the game logs for Yankee closers in 2012 (Soriano), 2013 (Rivera), and 2014 (Robertson): each blew 5-7 saves; the team lost 57-60% of those each year.)

Comparing to Holland: the 8 extra BS of a replacement-level pitcher yield about 5 more losses – making Holland a 5-WAR pitcher in just his save-opportunity games. But closers pitch in non-save settings too, typically 15-20 times a year; some of those are still (or even more) high-leverage innings, like ties in innings 9-10 or being down 1 run against a rival. If Holland is worth 5 WAR in 50 save-situation innings, he easily could add at least another 1 WAR in 15-20 non-save-situation innings.

So an elite reliever like Holland is worth about 6 WAR – which is over double the BBREF and Fangraphs WAR, because of an inherent flaw in WAR: it assumes all runs saved are created equal, that "every ten runs a player adds or subtracts adds about one win," with the 10run/1win ratio tweaked for high- or low-scoring parks (e.g., Coors vs. Safeco) or seasons (e.g., 1997 vs. 2014).

Ten runs per win is accurate on average - that's why WAR was a revolutionarily useful stat - but there's one hole in that assumption: runs have more value in close games, and top relievers systematically pitch when the score is close. Sure, some "saves" come with 3-run leads in the ninth, which isn't exactly a nail-biteworthy situation. The key point remains: closers almost never enter with the 4-run (or greater) leads or deficits starters or (especially) mop-up relievers see; so on average closers appear when each run matters more. Any HR off a closer very likely turns a narrow lead into a tie or loss; a HR off a starter is less certain to be a difference-maker; a HR off a mopper-upper like David Huff is especially unlikely to change the game outcome, given the large lead or deficit necessary for a Huff to be trusted with the mound.

Various number-crunchings all say that just as Holland is a 6-WAR reliever, Robertson is about 3-4 WAR. He isn't Holland or Rivera, but as Brad and a gang of commenters noted, he's still very good; his 2014 ERA of 3.08 was good enough (126 ERA+), but his 2.68 FIP was more like his usual mid-2s ERA, showing he's still a mid-2s talent. His FIPs are about a run higher than Holland's, so over 70 innings, you can expect about 8 more runs, which WAR thinks is a difference of less than 1 win. But Robertson had 3 more BS than the 6-WAR Holland, which would normally yield about 2 wins fewer, making Robertson about a 4 WAR pitcher. Or look at Robertson's 5 BS and compare it to the 10+ BS of a replacement-level reliever: Robertson is 5 BS above-replacement; given the 60% loss rate of blown saves, that's 3 WAR in his roughly 50 save-opportunity innings; add another 1 WAR for 15-20 non-save-situation innings, and again you get 4 WAR for Robertson.

I'm a little sheepish about peddling armchair math to declare an established stat wrong, but others agree: "evaluating relievers is tricky, and WAR doesn't always tell the whole story," because relievers may pitch higher-leverage innings. And when a few different ways of eyeballing numbers yield the same result, you start to believe those eyeballs – especially when the alternative is, frankly, silly. I think Robertson is 3-4 WAR, but the official WAR estimate of 1.2-1.7 almost exactly matches David Freese or some other mildly subpar regular. Ignore contract differences: would anyone sane trade Robertson for Freese? Of course not, which is to say nobody really thinks Robertson is a sub-2-WAR player. WAR of 3-4 for a very good reliever like Robertson, and 5-6 for an elite one like Holland, matches about what everyone thinks: an elite closer like Holland isn't quite in Mike Trout's 8-10 WAR territory yet easily equals a non-MVP but all-star-caliber player like Dustin Pedroia (5-7 WAR) or Adam Jones (4-5 WAR); and a good closer like Robertson far exceeds the 2 WAR of a dead-average player, easily equaling a Brett Gardner (4.0 WAR), Chase Headley (3.5), or Martin Prado (3.4).

If Robertson is 3-4 WAR, is he worth $12-15 million? Of course; several analysts looking at the past two years of salaries all conclude that the free-agent going rate is about $5.5-6 million per WAR. A look at recent Yankee signings confirms this. Some 3-4 WAR folks considered good deals still cost $10-13m: Gardner (4 WAR, $13m); Headley (3.5 WAR, $10.5m); Prado (3.4 WAR, $11m). But you can't look at just the deals that worked out; trying to buy a 3-5 WAR player yields not only a few massive overpays due to injuries (like negative WARs from Beltran and Sabathia's combined $38m), but plenty of middling-to-good performances for eight-figure salaries, like Jacoby Ellsbury (a solid 3.3 WAR, but for $21.3m), Brian McCann (1.8 WAR, $17m), Brandon McCarthy (1.1 WAR, thanks to his terrible first half, for $10.5m), and Hiroki Kuroda (whose middling 2.4 WAR was his worst in five years, for $16m). So $12-15m for Robertson's 3-4 WAR is somewhere between slightly sub-market and a real bargain.

The big caveat: multi-year deals can be nail-biters for relievers, who can flame out quickly. Even the best relievers typically got that role by lacking the broader-based skill set to be starters. Many have barely-adequate control, and most rely on the velocity or sharp break of only 1-2 pitches, not a portfolio of offerings they can adjust, or locate better, when the fastball loses fastness, the cutter loses cut, etc. But Robertson seems safer than some recent crash-and-burn reliever free agent signings. He's 29, not Joe Nathan signing a 2-year deal at age 38, and not even Jonathan Papelbon signing a 4-year deal for his 31-34 years. Unlike the wild early-20s Robertson, late-20s Robertson isn't the high-wire act of inconsistency you saw with, say, prime Papelbon, whose walk rates meandered from 1.0 to 3.8 BB/9. Robertson's last three years have shown remarkable consistency: FIPs of 2.49 to 2.68; BB/9 of 2.4-3.2, with amazingly similar K/BB rates of 4.17-4.28; and through age 29, no drop in velocity, according to Fangraphs PitchF/x.

In short, a fair look at the stats shows not only that Robertson is worth over double the official WAR stat that badly understates the value of top relievers, but also is a pretty consistent performer – the sort more likely to age well than the recent free-agent closer flops who are legitimate, but distinguishable, cautionary tales.