Warning: Creating default object from empty value in /home4/derok6/public_html/dbacksvenom/components/com_sh404sef/shInit.php on line 37
The Diamondbacks' Outfield Defense is Very Good, but Can Their Outstanding UZR Numbers be Trusted? | September

There is no question that the Diamondbacks' outfield defense has very good range, and has been a big part of the team's success this year. According to the Ultimate Zone Ratings (UZR) at FanGraphs, the most valuable defensive player in the National League this year has been Gerardo Parra, with Chris Young #2 and Justin Upton #6. In terms of just range (excluding arms and errors), the Diamondbacks have 3 of the Top 5, with Upton the NL leader by far. At the team level, the Diamondbacks' UZR of +52.6 Runs gives them a 12.8 Run advantage over the #2 Reds (+39.8), and a whopping 39.2 runs over the #3 team, the Padres at +13.2. Let's take a closer look at the Diamondbacks' outfield defense and how the various defensive metrics rate them.

At the most basic level, the job of a team's defense is to turn batted balls into outs. The stat Defensive Efficiency Rating, first developed by Bill James, measured exactly that - what percentage of batted balls are turned into outs. In this stat, the Diamondbacks rank right in the middle of the pack, #13 out of 30 teams at a DER of .712 (Or inversely, a BABIP of .288).  

1 Rays .734 .266
2 Giants .721 .279
3 Rangers .720 .280
4 Mariners .720 .280
13 D'Backs .712 .288
27 Twins .698 .302
28 Astros .698 .302
29 Orioles .697 .303
30 Cubs .696 .304

But one problem with evaluating a team defense by looking at DER or BABIP is that some types of batted balls are more likely to turn into outs than others. For example, fly balls are much more likely to turn into outs than ground balls, and infield flies are almost a certain out.

Batted Ball Type Approximate DER Approximate BABIP
Ground Ball .750 .250
Fly Ball .880 .120
Line Drive .260 .740
Infield Popup .990 .010

So a pitching staff that gets more flyballs will have a higher team DER and a lower BABIP (although they will also give up more homeruns and extrabase hits, but that's not the point of this exercise). The Diamondbacks' pitching staff in 2011 gets a lot of fly balls - their FB% rate of 37.7% is the highest in the NL and #4 in the Majors. So this would suggest that the D'Backs should have an even better DER, but they don't. So the question remains, if the team's DER is not exceptional, especially when the fly ball nature of the staff is considered, why does UZR like them so much?

Unfortunately, the raw data used for FanGraphs' UZR calculations is not available (as far as I can tell). So it's not clear why Upton's RngR (runs saved due to range) is the best in the NL.  However, FanGraphs does post some of the raw numbers from The Hardball Times' Revised Zone Ratings (RZR). RZR is a simpler, but in many ways more intuitive, defensive metric. RZR divides the field into discrete zones, and just counts how many balls are hit into each zone, and how many are turned into outs. A fielder's "Zone" is defined as those areas where the average fielder at that player's position turns at least 50% of balls into outs.  Here are the 32 qualifying NL outfielders, ranked by RZR:

Rank Name Team Inn BIZ Plays RZR OOZ
1 Jon Jay STL 833 171 163 0.953 69
2 Dexter Fowler COL 918 213 203 0.953 58
3 Shane Victorino PHI 984 189 180 0.952 67
4 Corey Hart MIL 951 182 173 0.951 64
5 Andre Ethier LAD 1091 184 173 0.940 70
6 Drew Stubbs CIN 1198 281 264 0.940 66
7 Jason Heyward ATL 863 146 137 0.938 45
8 Chris Young ARI 1225 267 250 0.936 99
9 Mike Stanton FLA 1100 196 183 0.934 73
10 Justin Upton ARI 1241 208 194 0.933 112
11 Matt Holliday STL 917 146 136 0.932 48
12 Cameron Maybin SDP 1010 227 211 0.930 90
13 Andrew McCutchen PIT 1215 294 273 0.929 93
14 Matt Kemp LAD 1213 267 247 0.925 66
15 Ryan Braun MIL 1109 179 165 0.922 63
16 Angel Pagan NYM 938 222 204 0.919 69
17 Carlos Gonzalez COL 1022 160 147 0.919 62
18 Marlon Byrd CHC 857 194 178 0.918 64
19 Lance Berkman STL 844 130 119 0.915 39
20 Jason Bay NYM 968 161 147 0.913 61
21 Jay Bruce CIN 1205 218 199 0.913 65
22 Michael Bourn - - - 1195 286 260 0.909 79
23 Hunter Pence - - - 1179 199 180 0.905 79
24 Carlos Beltran - - - 1002 154 139 0.903 60
25 Seth Smith COL 921 155 139 0.897 57
26 Jayson Werth WSN 1178 238 213 0.895 91
27 Gerardo Parra ARI 944 181 161 0.890 96
28 Ryan Ludwick - - - 1002 139 123 0.885 68
29 Cody Ross SFG 894 126 110 0.873 56
30 Logan Morrison FLA 916 162 141 0.870 48
31 Alfonso Soriano CHC 897 143 124 0.867 45
32 Raul Ibanez PHI 1040 158 137 0.867 43

Upton and Young are both in the upper third in turning "Balls in Zone" into Outs, but Parra is near the bottom. Where all three outfielders stand out, however, is in "Out of Zone" (OOZ) catches. In fact, Upton, Young, and Parra rank #1, 2, and 3 in the NL in making OOZ plays. Keep in mind that OOZ is a counting stat, not a rate stat, so all three are helped by being behind a fly-ball heavy pitching staff. Upton and Young have also played the most innings, although Parra and Upton rank #1 and #2 in OOZ plays per inning.

What are the differences between RZR and UZR? Well, there are a lot. First, UZR does not use discrete "Zones", but a more continuous distribution method. Second, and probably most important, UZR considers the time the ball takes to reach the zone (equivalent to a measure of how hard the ball was hit). Then, UZR gives a player positive or negative points based on how often a play is made on that type of batted ball compared to that of an "average" fielder at that position.  Finally, UZR makes many adjustments to the data, on things like park effects, the handedness of the pitcher, the base/out situation, and the batted-ball nature of the pitching staff. While all of these are important effects, it does make the end result of UZR much harder to follow.

In addition to UZR, there are at least three other defensive metrics worth examining - John Dewan's Plus-Minus (also called Defensive Runs Saved, or DRS), Baseball Reference's Total Zone, and Baseball Prospectus' Fielding Runs Above Average. DRS is the closest in concept to UZR, as both break down batted balls by type, location, and velocity, and then estimate the probability of the play being made. The two systems even use the same database. Yet the two systems often produce very different results, which is one of the most disconcerting aspects of fielding stats. An excellent discussion of the differences between the two systems is here. The basic differences are:

  • UZR uses multi-year samples to estimate out probabilities, while DRS uses data over one year.
  • DRS uses smaller buckets to divide plays.
  • UZR uses a blanket park effect for the outfield and another for the infield, while DRS applies park factors to more buckets.
  • They have slightly different techniques in assigning run values to each play made or not made.

While these don't seem like major differences, the discrepancies between UZR and DRS can occasionally be large. The creators of these stats acknowledge that sample sizes are a problem for all defensive stats, and suggest using multiple years of data to get a valid sample, or even regressing the UZR numbers halfway to toward zero. BR.com's Total Zone uses play-by-play data (i.e., fly ball to CF, or ground ball to 2B), rather than the direct observation of UZR and DRS. Baseball Prospectus' latest version of Fielding Runs is described here, and it uses neither direct observation or play-by-play data. Rather than evaluate the pros and cons of each method, let's just see how all four rate the Diamondbacks' outfielders.

Gerardo Parra 14.2 17 11 15.7
Chris Young 12.2 3 17 -6.3
Justin Upton 10.3 5 -7 2.1

All four systems score Parra highly, but he is the only one of the three with a consistent rating. What's most troubling for me is the large discrepancy between UZR and DRS, since both are based on the same dataset. Clearly, the adjustments and weights given to the raw data can make a huge difference. FRAA likely rates Young low because the outfield just doesn't make as many plays as would be expected by a fly-ball heavy staff. It is not at all clear which metric is the most correct, since they all differ so much. To me, the methodology of UZR seems the most sensible, but their results are also more likely to fail the "eye test", especially when compared to DRS (See for example, the great UZR scores this year for players like Nick Swisher or Pablo Sandoval, or for Pat Burrell last year).  Also, the UZR variations of players who change teams (i.e., Carl Crawford from Rays to Red Sox or Kelly Johnson from Braves to D"Backs) makes me question if UZR is really measuring the player's defense.

One other UZR issue that has come up recently is the possible correlation between Outfield UZR and a team's fly ball rate. Over the last two seasons, teams with a high fly ball rate have been scoring very high in UZR. 

2011 OF UZR (Rank) FB%
D'Backs +36.2 (1) 37.7% (4)
Red Sox + 30.9 (2) 40.0% (1)
Yankees +28.1 (3) 35.0% (23)
Rays + 16.0 (4) 39.0% (2)
2010 OF UZR (Rank) FB%
Giants +40.7 (1) 40.7% (2)
Rays +31.6 (2) 40.8% (1)
D'Backs + 30.1 (3) 39.2% (5)
Yankees +28.6 (4) 39.1% (7)

Of course, teams that have a lot of fly ball pitchers would want to have a good defensive outfield. But this link between fly ball rate and Outfield UZR does look suspicious. And a look at some of the individual outfielders on these teams adds to the worry. The Yankees, for example, have a +11.6 from Nick Swisher this year. In 2010, the Giants scored a +6.1 from Pat Burrell, +1.5 from Aubrey Huff, and even a +1.0 from Jose Guillen. It almost seems like the adjustments used in UZR are not properly accounting for the effects of a strong fly ball staff.

In previous years, I think Tango's Fan Scouting Report was a great way to study a player's defense. Unfortunately, I think people's evaluations are being influenced more and more by the ubiquity of the UZR data, and the FSR is not as independent as it used to be.

Even with the recent improvements in defensive metrics, it is still very difficult to evaluate a player's defense. While UZR and DRS both seem like a logical way to measure defense, the answers spit out by these two methods differ more than one would expect. And the correlation between outfield UZR and the pitching staff's fly ball rate suggests that there could be a problem in the way UZR adjusts for the style of the pitching staff. 

Calculations like RZR and OOZ are more straightforward and easy to understand, since they are basically just counting stats, and do not make the extra step in trying to decide if an "average" fielder would have made the play. They at least show that the Diamondbacks' outfielders make a lot of "Out of Zone" plays. There is still a question as to why the team turns so few batted balls into outs. The team ranks #13 in DER, which is not a very good number for a team with a fly ball pitching staff and seemingly several good defensive players.

So are Parra, Young, and Upton the three most valuable defensive outfielders in the National League? I'm not sure if I would go that far. All of the metrics do agree on Parra, and most observers feel that he has played well in LF. In addition to good range, Parra has a strong and accurate arm too. Young's defensive metrics are all over the place - my visual evaluation rates him as above average for range (but not throwing), but I wish the metrics were more consistent. As for Upton, there is no doubt that he gets to a LOT of fly balls, and he also makes a few obvious errors. Overall, I think Upton is still a pretty big positive in the outfield. As a group, I think the trio of Parra, Young, and Upton are one of the best in baseball. I just wish I could trust the UZR numbers a little more.