The Fine Line Between Awesome & Awful

Monday at The Hardball Times, Nick Steiner attempted to figure out what stats (particularly Pitch f/x) could tell us about the difference between a pitcher at their best and at their worst. We continually lean on clichés like "He didn't have his best stuff today" to explain why a pitcher has a bad outing. It seems apparent fairly early in a game, at least in hindsight, when a starter is dealing or is not.

But how much of that is confirmation bias? In other words, how much does the outcome of the start effect how we remember our perceptions of the beginning of the game? Maybe the difference between a 7 inning shutout and a 7 run disaster isn't "stuff". Perhaps, from the pitcher's perspective, there isn't much difference at all. Is it possible that Joba Chamberlain really did "throw a lot of good pitches" in some of his poor outings?

For a subject, Steiner chose A.J. Burnett, because of the stark difference between his best and worst outings. When sorted by Game Score, Burnett's 10 best starts in 2009 added up to an ERA of 1.06 while his 10 worst came out at 9.13. He went (6-2) in his top 10 and (0-6) in his bottom 10.

There should be some major differences between these two groups of starts. You'd expect to see some patterns emerging in terms of velocity, or movement, or location or pitch selection, right?

In short, no. There was almost no difference at all.

Steiner dug through all of Burnett's Pitch f/x data for this year, painstakingly categorizing it by pitch type (4-seam fastball, 2-seam fastball, change up, curveball, slider), movement (horizontal, vertical), location (outside, border out, border in, middle), batter (lefty, righty) and count (pitcher's, hitter's, neutral).

He sliced the data in lots of intuitive ways but found almost no significant differences between Burnett's good and bad starts. And for every directional variation which might explain his better starts (fewer pitches down the middle in good starts), there is another which runs counter to what is expected (better velocity in bad starts). In my own look at the numbers, I found that Burnett actually walked fewer batters (29) is his bad starts than he did in his good ones (32).

So what separates a great start from a terrible one, if not for pitch selection, movement and location?

For one thing, there is a whole lot more luck involved with pitching than we realize. In a span of three starts this year, Burnett bookended a 4 2/3 inning, 7 run outing against the White Sox with two shutouts against the Rays and Red Sox, each at least 7 innings. Three starts, two absolutely brilliant ones and one that a AAA call-up would be ashamed of. (Relax conspiracy theorists, Jorge Posada caught all three of them.) There are few other professions where such wild variations between success and failure are common at such a high level.

Part of this is the fact that it only takes one pitch to alter the outcome of a game. One three run home run can change the complexion of a start entirely. And the difference between it ending up as a round tripper and a fly ball on the warning track is a matter of a fraction of an inch on the bat. That's just one pitch out of 100 or more.

If you look beyond Pitch f/x, some other things turn up in Burnett's starts. While his percentage of strikes looking was almost exactly the same regardless of the type of outing (18.7% to 18.5%), the occurrence of strikes looking was much higher in his better starts (10.4% to 6.6%). He also allowed almost twice as many fly balls and line drives in his 10 worst starts while ground balls were between 7% and 8% in both.

Are we to believe that he is throwing the same quality of pitches in both groups of outings and getting wildly different results just based on luck? If it was a random chance, the swings and misses, line drives and fly balls would be more evenly distributed. I think it's more likely that there is something that Pitch f/x isn't capable of telling us.

Especially in a broad analysis like the one Steiner conducted, it's difficult (maybe impossible) to zero in on the things that separate a curve ball that induces swings and misses from one that results in an opposite field single. It would have a hard time telling a fastball down the middle in a 3-0 count (unlikely to be swung at) from one when the batter was ahead 2-1. It can't tell which locations are preferable to which hitters, given that some like the ball inside while others favor it out over the plate, for example. Mistakes made with men on base are most costly than ones with the bags empty. What about pitch sequencing, or the amount of pitches hitters saw, how often Burnett was working from behind in the count and so on and so on...

One of the great things about baseball is the amount of data available, but it's a double-edged sword. It makes general questions like this one almost impossible to answer because of the endless number or variables. No two outings are exactly alike and something tells me that even if there was a parallel universe where two of the same games began at the same time, they would probably turn out completely differently anyway.