Hafner, Hope and Small Samples

The brightest light in what has been a dark beginning to the Tribe's season has been the performance of Travis Hafner.  As has been well chronicled, Hafner was a major question mark coming into this season following a regression in 2007, a dismal 2008, and off-season shoulder surgery.  The question of Hafner is made many times larger by the $57M the Indians are on the hook for him through 2012.  A repeat of last year would seemingly assure the Indians of the largest contractual albatross in club history.  A return to pre-2007 levels would alternatively be a huge boost to what should be a good offensive lineup even in Pronk's absence.

As we are pretty much all aware, Hafner has begun the season with 3 HRs, a double and two singles in his first 24 plate appearances (spanning 5 games).  This is great.  But does it give us any indication if Pronk is back?  One of the things I found most striking about Jay's interview with Antonetti was the repeated references to the need to get a large enough sample to assess performance and the difficulties of performance analysis in the context of small samples (hello, bullpen).  This is a fairly well addressed topic in sabermetric research, with several studies examining how many plate appearances are necessary for current season performance to be reliably predictive of total season performance (i.e. how many ABs do you need before something like SLG% or OBP% to become significant?).  24 plate appearances is about an order of magnitude too small to predict what Hafner will do the rest of the season.

But it doesn't mean we can't ask questions about his performance thus far.  As it turns out, my real life profession regularly involves me asking questions strongly constrained by sample size (morphological variability in the fossil record).  And while sample size is a major obstacle, it is possible to ask simple questions even given the limited information available to us.  

So that's what I want to do, ask a simple question about Hafner 2009.  And my question is this - what is the likelihood of observing Hafner's 2009 performance across 24 plate appearances in the 2008, 2007, or 2006 Hafner seasons?

To do this, I did a few things.  First, I coded Hafner's plate appearances over the past three years based on result; how many outs, walks, singles, doubles, triples and HRs (for simplicity sake, I'm excluding sacrifices, HBP, etc).  I then created a distribution of performance based on 24 plate appearances for each of Hafner's past three seasons by randomly drawing 24 plate appearances out of the entire sample of his season and repeating this process 10,000 times.  For example, in 2006 Hafner had 563 plate appearances, 100 of which were BBs, 66 of which were singles....42 of which were HRs.  What I did was randomly draw 24 of those plate appearances against which I can compare Hafner's actual 24 plate appearances this season.  To compare them, I simply calculated the OPS across those 24 plate appearances.  I then repeated this 10,000 times for each season.

For those worried about my sanity, this only takes about 5 minutes and 10 lines of code.  This is essentially a "bootstrap" test for those with a little stats background.

As a reminder, I'm interested in what the likelihood is of observing Hafner's 2009 performance in 24 plate appearances based on his 2008, 2007 or 2006 performance.

Here are the results:

Compared against his 2008 performance (for which he put up a stellar .628 OPS in 233 PAs), Hafner's 2009 performance was better than all but 111 of the 10,000 trials.  In other words, his numbers thus far would put him in the 99th percentile of expectations based on his 2008 performance (see distribution below).  At the 1% level we can make a pretty strong claim that Hafner's performance is better than you would expect based on his 2008 numbers.

Compared against his 2007 numbers, Hafner's 2009 totals exceed the simulated trials in 9,065 of the 10,000 trials.  Being conservative, although this is pretty far onto the right half of the distribution (see below) we can't say that his performance thus far has been different than what you would expect from his 2007 numbers.

Finally, compared against the 2006 season, Hafner's 2009 totals exceed the simulated trials in 6,570 of the 10,000 trials.  This is pretty much smack in the middle ground of the simulated distribution.

So what does this mean?  It provides some evidence, even given the limited data available to us, that Pronk is back.  This in no way suggests he is going to continue on this path.  This in no way predicts what he will do the remainder of the season.  But it suggests his performance this far is pretty inconsistent with his performance last season - which since he was terrible, is reason for hope.