Monday, January 4, 2010

Is Javier Vazquez Unclutch?

Throughout his career it has been intimated that Javier Vazquez has the stuff of an ace but the track record of a back of the rotation starter. He has excellent peripheral numbers (3.45 K/BB), a better than league average ERA (107 ERA+) and averages well over 200 innings per season. Above average performance and lots of innings should be a winning combination, but Vazquez has always seemed to underachieve when it comes to his won-lost record. His four postseason appearances haven't been pretty either.

Plenty of Yankee fans are worried that he doesn't have the make up to pitch in New York. Vazquez bombed in the second half of his only season with the Yankees and served up perhaps the most infamous home run in Yankee history. But all of that was more than 5 years ago and the latter was the result of one pitch. Ozzie Guillen ran him out of town in Chicago after calling him out for not being a big game pitcher, but Guillen was supposedly responsible for ousting Nick Swisher too and that worked out pretty well for the Yankees.

Does Javier Vazquez's performance get demonstrably worse under pressure, or has he been a victim of marginal teams and bad luck? Is his reputation as someone who shrinks as the expectations grow deserved, or is it the conflation of a few unrelated events?

As I attempt to take a deeper look into these questions, I'm going to go back to something Tom Tango said during his Q & A with Mike Silva last week. In the process of justifying FIP as a useful statistic, Tango pointed out that there is a strong correlation between the best pitchers in the league over a long period of time when ranked by FIP and ERA.

Those familiar with the two stats might take that for granted at this point, but it's fairly remarkable considering all the things that FIP totally disregards that factor into ERA: singles, doubles, triples, runs allowed, etc. What's more intriguing, he contends, is what the significant gaps between FIP and ERA over the long term can tell us:
Tom Glavine’s career FIP is about 0.50 runs worse than his career ERA. This signals that Glavine does something extra, either he can sequence his events better (leaves alot of runners on base for example), or he has better control on his balls in play. And Javy Vazquez’s ERA is about 0.30 runs worse than his FIP, which signals something different, that perhaps he gives up alot of doubles, or doesn’t sequence his events well, etc.

Overall, two-thirds of pitchers will have their FIP and ERA be within 0.20 runs of each other, and almost all will be within 0.40 runs of each other.

That’s the power of FIP: that it’s designed to tell you one specific thing, and it tells you a second, perhaps even more important, thing.
So what's that thing? That's the million dollar question.

One interesting fact is that Vazquez has been extremely consistent in under performing his FIP throughout his career. He's only had an ERA lower than his FIP twice in his 12 professional seasons and only then by the slimmest of margins.

Even in his career year with the Braves last season, his FIP was lower than his ERA (his low FIP was one of the reasons Keith Law included him on his Cy Young ballot). I think we can agree that this not just the result of random chance: there is something about the way that Vazquez pitches that causes him to have an FIP lower than his ERA.

So what is it about Vazquez's pitching that could be causing this? Let's look at the career FIP/ERA differentials for Vazquez, Glavine as well as Andy Pettitte:

Above, Tango says that almost all pitchers will have an FIP and ERA within .4 of each other, so keep in mind that we are looking at two outliers extreme outliers here in Vazquez and Glavine. This is the main reason I chose to include Pettitte, who is closer to the norm.

If you're a Yankee fan, I bet you're a little surprised that Pettitte's differential is closer to Vazquez's than Glavine's. Pettitte has the reputation of being able to bear down and pitch better when it matters and seems to induce a lot of double plays, both of which would lower his ERA but not FIP. But the numbers indicate that he's not as great at controlling his outcomes as many assume.

As Tango mentioned above, one thing that might inflate ERA independently of FIP is giving up a lot of doubles. (Numbers are normalized over 200 IP to account for different career lengths):

Vazquez and Pettitte give up more two-baggers than Glavine does, but four over the course of 200 innings doesn't seem like enough to account for a shift in 3/4 of a run in ERA. Furthermore, Vazquez and Pettitte give up doubles at a nearly identical rate, but the latter has an FIP significantly closer to his ERA. There's something else at work here.

The other possibility that is mentioned above is "sequencing". Maybe Vazquez gives up hits in a way that hurts him. For example, starting an inning with a walk before a double often results in a run scored and a man on second. Conversely, a double before a walk likely means men on first and second without a run scoring. Vazquez gives up his fair share of home runs as well, and they are obviously more harmful with men on base (something that doesn't show up in FIP but does in ERA).

A quick and dirty way of teasing this out of the data is to look at a pitcher's stats with men on base. The numbers below are shown in tOPS+, which compares a pitcher's performance in a given scenario to his performance in other situations. (A score of 100 means is a pitcher is average in that situation while anything lower means they were better and higher means they were worse)

One caveat: the bases loaded data is by far the smallest sample size of the bunch. Vazquez has faced 163 batters with the bases loaded while Glavine and Pettitte have 428 and 237, respectively. That said, there is a major difference between Vazquez and the other two (particularly Glavine) when it comes to their performance with men on base and especially with the bags packed. And those situations are the ones that separate FIP from ERA the most.

Now let's look at how each pitcher performed based on the score of the game, again using tOPS+:

The smallest sample size in this case is "> 4R", but even for Vazquez that includes 811 plate appearances. Pettitte has the most favorable distribution, pitching his best when the game is close and his worst when it is out of hand. Vazquez, however, is just about average in tight games but much better when the lead or deficit is greater than 4.

Now let's put the last two items together. Baseball-Reference's Leverage stats take into account the occupancy of the bases as well as the score of the game, as does FanGraphs Clutch stat:

This tells a slighty different story about Pettitte but affirms what's becoming a trend for Vazquez - he doesn't perform as well in high leverage situations as he does in lower ones.

Does WPA agree?

Yes. It appears that Vazquez has not given his team as good of a chance to win as either Pettitte over Glavine over the course of his career, but this is to be expected to a certain degree because WPA is tied more closely to ERA then FIP. In a sense, we already knew this.

So what do we make of all this?

It's tempting to use these stats to claim that Vazquez can't handle pitching in New York. His data seem to trend very uniformly (remarkably so) from good to poor as the leverage increases. However, why didn't he cripple under the weight of the New York media in the first half of 2004 when he had a 3.56 ERA in 118 2/3 IP and won 10 games?

To say that he can't handle New York not only gives too much weight to a small sample size but requires a jump that conflates the pressure of in-game situations to be analogous to the demands of pitching for one franchise or another. Does the weight of overall expectations have the same effect on performance that increases in in-game leverage do?

Well, maybe. As 'Duk from Big League Stew pointed out in September, Vazquez's best seasons in terms of ERA+ have come with teams that were out of contention and his worst years came with teams in playoff races:
Three of his top ERA+ years came in the anonymity of Montreal and one came for the 2007 White Sox, who went 72-90. This year's ERA+ of 139 equals his career-best with the 2003 Expos, but while the Braves stuck around as a potential contender for longer than expected, they didn't occupy striking distance space for long.

Meanwhile, Vazquez's worst ERA+ years — with the exception of his first two seasons — all came with contenders: the '04 Yankees, the '05 D'Backs and the '06 and '08 White Sox.
So what is he doing wrong? As we saw when trying to identify some differences between A.J. Burnett's good and bad starts last week, it's extremely difficult isolate any one underlying factor or find a specific reason that Vazquez's numbers go bad as the leverage of the game increases. Javy's K/BB ratio slips from 4.34 to 3.26 to 2.57 as the leverage rises from low to medium to high. His batting average, on-base and slugging percentages all ascend with the gravity of the situation as well.

Even if you grant that Vazquez gets worse under pressure and will pitch worse just by virtue of being a Yankee, he's still likely to be better than league average and throw more than 200 innings. It would be extremely difficult to do that and not add significant value to a team regardless of how his performance is distributed by leverage.

And of course, there's a big difference between "hasn't" and "can't". I'm willing to say that Vazquez certainly hasn't pitched well under pressure in his career, but not that he can't. He clearly had a great year in Atlanta and some of that has been attributed to an improved change up, giving him a second pitch to miss bats with in addition to his curveball. FanGraphs shows that his curveball was what stood out last year, but his change up looked to be improved as well.

It's certainly not impossible that Javy has a good year and surprises his doubters (and even some of his supporters). Vazquez might be frustrating to watch at times in 2010, but the beauty of the situation is that our expectations for him shouldn't be that high to begin with. Not too many teams have the luxury of acquiring a very good pitcher and hoping that he's just average.


  1. Seriously??? I've been working on practically the same post, built off the same quote, on and off for the last few days. Same analysis, same conclusions. Damn it.

    Oh, and great work, really thorough. I do think that Vazquez may have turned a bit of a corner last year. Eric at TYU looked at what might have changed, and it seems Vazquez threw significantly fewer fastballs last year. There is no way to check this without access to the data, but I wonder if he was throwing significantly more of those added breaking balls with runners on.

  2. Moshe - Sorry man. I've been slowly piecing this thing together since last Wednesday but decided to hold it back until today so it wouldn't get lost over the holiday weekend. I know the feeling of getting scooped on something you spent a long time on - it's not pleasant. Such has become the nature of being a Yankee blogger, it seems.

    It would be interesting to look at the PITCHf/x data with runners on base, but even if you could get your hands on that data, it would be extremely tedious.

    One thing that makes this kind of analysis tough is that the stuff that is supposed to tell you the most comes with the smallest sample sizes. Still, the numbers speak for themselves.

  3. "Such has become the nature of being a Yankee blogger, it seems."

    Pretty much. I'll link your post today and add some of my own thoughts.

    "It would be interesting to look at the PITCHf/x data with runners on base, but even if you could get your hands on that data, it would be extremely tedious."

    I'll bet the Yankees have access to a PBP database that they can easily sort. It should be interesting to see whether they encourage fewer FB from Javy, particularly with men on base. I might try and keep track of it over the first few weeks.

  4. I hope this leads to the same (playoff) results as your analysis of CC's clutchiferousness did last year. ;)

  5. Less FB with men on from Javi would probably decrease his HR rate with men on base, which could help stop him from giving up runs in high leverage situations.

  6. I'm a White Sox fan and Vazquez was fascinating/frustrating to watch when he was here. His stuff is terrific and he'd have stretches where he was blowing hitters away and then just suddenly start getting lit up. Some guys can have a rough inning, pitch through it, contain the damage, and put it behind them. With Vazquez things just seem to get worse. He got a reputation as someone who, to put it delicately, was not mentally tough.

  7. Great analysis. I have a question: If Vazquez is getting his run support from the league's best lineup, won't he have big leads more often, decreasing his ratio of high-lev innings to total innings? Intuitively it seems like that should help him, but I don't know how to quantify it.

  8. Last Anon - It's hard to say. Those margins are calculated to include both leads and deficits. The Yankees should blow teams out more often, but their good offense should also prevent them from being blow out too often as well. It cuts both ways.

    The bottom line is that their offense is likely to be one of the best he's ever pitched in front of and regardless of what that means for leverage, run support is a positive.

  9. Vazquez's biggest problem is mental toughness, indeed. When things start to go wrong he seems to wallow in frustration and loses his cool completely, which is why he gets hammered even with the stuff he has. It won't matter much if he throws less fastballs in certain situations, he has to learn to suck it up and he will be fine.

  10. Going big time:

  11. Nice post. One other thing I think would be important to look at with regard to the difference between Vazquez' ERA and FIP is the quality of his teams' defenses, because he has pitched for teams with fairly poor defenses overall for his career. Usually, you'd expect that to even out over as long as Javy's pitched, but it hasn't for him. Total Zone has his teams averaging about -21 to -22 runs (relative to average) per year on defense over his career, which is bad enough to add roughly .12 runs to his ERA. If you take that out of the difference between his ERA and FIP, the difference isn't quite so extreme. Also, his WAR from, which uses RA with an adjustment for team defense instead of FIP, is still really good. So I think that's good news if you're a Yankee fan.

    I also think the bases loaded split can be somewhat encouraging in this context. You know that his ERA is worse than his FIP, and you know that a significant part of that is that he pitched like crap over 163 PAs with the bases loaded, which adds a ton of runs to his ERA over his FIP. The good news is that a sample of 163 PAs, even if it is that bad, probably has little predictive power, so that is a good chunk of the difference between his ERA and FIP that is explained away by something unlikely to be repeated going forward (which is what a Yankee fan would care about).

    The bad news would seem to be those leverage splits. Those probably are big enough samples that they are picking up on a real issue with Vazquez beyond just the difference between his ERA and FIP.