SONAR Scores – The pitchers

By now I’m going to assume that you’ve seen my introductory piece on this new statistic I’ve created, which I’ve dubbed the SONAR Score. Because I don’t have to do a huge setup here explaining the premise of the entire project, I’m going to address a few quick points before getting to the pitching scores, then just dig right in. I won’t comment on everyone, obviously, because there are over 80 players on the sheet. Not all of these guys are legit prospects, obviously, but they are all scored on the same scale, all of the adjustments are applied, so including them does no harm. I also left Jason Knapp and Carlos Carrasco on the list as a point of reference, I’m well aware that they are no longer in the Phillies system. So, with that intro, lets get down to it.

* I hope everyone had a chance to go back and read my intro piece and then the comments section as well. I tried to address a few of the key questions briefly, and there’s some good information in there, especially with regard to my devaluing of the stolen base. If you still have questions on the hitting aspect, please post your questions in the hitting discussion, not here.

* Someone brought up the idea of looking at the score in season. I’m here to tell you that its not practical from my end. Creating these scores involves me copying/pasting data into an Excel sheet that I spent a lot of time building, and then after all of the data is copied/pasted, combining lines for players that played at multiple levels. Its very time consuming, there’s a lot that goes into it, and its just not something that I will be doing in season. The point of the score is kind of to look at the season that was and try to figure out if we can learn something that might go against the popular belief or prevalent notion. As I discussed in the other post, I’m not a big believer in minor league splits. I just don’t really think pulling 75 AB or 20 IP from any pitcher’s sample and analyzing it will give you something useful. Could the guy have really improved a core skill that changes his prospect outlook? Sure. Or he could have feasted on weaker competition after 10 of the best prospects in the league were promoted midseason. There are any number of explanations for hot and cold streaks, and its just not something I see the value in. As I mentioned before, this score isn’t some type of magic, there will be guys who score low but then suddenly break out, and guys who have a great year then fall flat. My hope is that by looking at the score, and then the confirming indicators (SecA, ISO, DICE) that we can maybe put a player’s season in perspective and see if hes maybe overrated/underrated at the time and what to expect going forward. It still requires a lot of analysis, you still need the scouting reports, and even then you really won’t ever be right 100% of the time, or 90% of the time, or probably even 75% of the time. Its the unpredictable nature of baseball.

* The pitching version of the SONAR score is built on the same basic principle as the hitting version. My goal was to target the aspects of pitching that the pitcher has control over. I’ve mentioned the DICE statistic before, there is also DIPS, and you can read all about it here and elsewhere. The three true outcomes for a batter are a walk, a strikeout and a home run, as none of the three are dependent on a ball being put in play. For a pitcher, the concept is similar. A pitcher can control the guys he strikes out, the guys he walks, and to a lesser extent, the number of home runs he gives up. The latter is more controversial, but I do believe HR rate is a predictor of future success/failure to some degree. I don’t weight that as much as I do the other two factors, but I think it is important. Obviously if you read into the last few sentences literally, you’ll say pitchers can’t control whether an umpire has a consistent strike zone, and things of that nature, and I agree, but that isn’t really the argument. Guys who strike out lots of hitters while walking very few batters are the guys who generally lead the league in pitching categories. When the ball is put in play, you’re at the mercy of your defenders and the luck that comes with batted balls in play. A pitcher with mediocre stuff on a great defensive team could have a shiny ERA, but then go to a team with a bad defense, have the same mediocre peripherals, but have an ERA 2 runs higher. By not allowing the ball to be put in play, as a pitcher you take the defense out of the equation. That’s the aim of DICE and FIP. Its also the foundation for my pitcher ratings, looking at a pitcher’s control, his domination, and how well he keeps the ball in the park. With hitters, the “confirming” statistic was SecA, and with pitchers the confirming stat is DICE. A guy who posted great peripheral statistics but pitched in a very hitter friendly park, or with a really bad defense behind him, could have a pedestrian/ugly ERA, but still have the core skills needed to thrive. So, a pitcher’s SONAR score will not be impacted by his ERA, instead by his core skills on the mound.

The scale for lining up all the prospects is the same for pitchers as it is for hitters. A refresher

Score of 110+ = Elite [80]
Score of 95-110 = Well Above Average to Elite [75]
Score of 80-95 = Well Above Average [70]
Score of 65-80 = Above Average to Well Above Average [65]
Score of 50-65 = Above Average [60]
Score of 35-50 = Average to Above Average [55]
Score of 20-35 = Average [50]
Score of 5-20 = Average to Below Average [45]
Score of 5 to -10 = Below Average [40]
Score of -10 to -25 = Below Average to Well Below Average [35]
Score of -25 to -40 = Well Below Average [30]
Score of -40 to -55 = Well Below Average to Terrible [25]
Score lower than -55 = Terrible [20]

Some of you complained that the image was not visible for you, or you couldn’t see it. If you are using a Windows based operating system, right click on the image/chart then select “view image”, or copy the image URL and then paste it into another window. I’m also going to post a URL link where you can click on and it should open the image in a new window/tab. If you continue to have issues with this, let me know. The far right column, labeled “20-80 scale” is again a reference to the numbers in brackets above, which I explained in the hitters post. Its just a frame of reference. Also, the red and green highlighted boxes are not scientific, just rates that I viewed as either positive or negative. I might not have gotten all of them, but I think it gives you a general idea.

So, lets get to the positive value pitchers.

phillies-pitchers1

If you cannot see the image, click here.

Any big surprises here? No real surprise that Drabek tops the list. I suspect most people would have expected a higher score, but his score was brought down slightly by his performance in his last few starts at Reading. His performance at Clearwater was excellent, and he was age appropriate for A+. He was young for Reading, but his performance dropped off a tad. He lost about 3.5 K per 9 and allowed 9 HR in 96 IP after allowing 0 in 61 IP at Clearwater. The FSL is a much tougher hitting environment, whereas Reading is one of the more hitter friendly parks in the Eastern League. Still, Drabek’s score is fine, and I suspect that when I compile all the numbers from every team, he’ll have one of the 20 or so best scores for pitchers. Nothing to be ashamed of.

Maybe the next two are kind of surprising, but not really. De Fratus has posted excellent peripherals at every stop. His walk rate is outstanding, he doesn’t give up many HR, and he’s still missing bats. Kevin Goldstein indicated that he has good raw stuff, not elite stuff, but good enough stuff to make it to the bigs. Right now the Phillies seem undecided about his role, and while he may ultimately end up a reliever, I’d give him a chance to start again in 2010 at High A and see what happens. Bastardo’s rapid ascent continued, as he reached the majors, blew away the Padres, then came back to Earth a bit, then vanished with another arm injury. At this point, that’s my biggest knock on him. Can he stay healthy enough to log 120 innings in a minor league season? He looks like a reliever to me, but a power reliever who can pitch late in games and provide great value in his league minimum MLB seasons.

Our resident guest contributor Michael Schwimer comes in just behind former Phillies prospect Jason Knapp and just ahead of former Phillies prospect Carlos Carrasco. Schwimer’s season is even more astounding because he threw only 64 innings. As I mentioned in my intro, guys with lower innings totals/plate appearance totals are docked, because the same is smaller, and thus a bit more unreliable. If I hadn’t had this type of adjustment in, Schwimer might have scored higher than Drabek. Almost 12.5 K/9 is ridiculously good, and just under 3 BB/9 is just fine. He keeps the ball in the park, and his 1.35 DICE backs up the fact that he had ridiculous stuff working this year. How will he do at the higher levels and beyond? We’ll see, but so far so good.

Touching on a few more in brief. Matt Way ranks the highest of the 2009 draft class, a stellar debut and a ridiculous K/BB rate. His stuff is more finesse, so it remains to be seen how he’ll fare against better competition. Futures Game rep Yohan Flande, who came out of nowhere, had a decent but not amazing season. His peripherals aren’t amazing, just solid, and he’s probably a reliever. Two guys who maybe no one is thinking about, Ebelin Lugo and Nicholas Hernandez, posted really nice scores. Lugo had a fairly pedestrian debut last year in the GCL, but all of his peripherals greatly improved. I wish I knew more about him. Hernandez, a 12th round pick this past June, was only 20 in the NYPL, not out of line, and posted good numbers, especially with his command. Hernandez, if you remember from the draft writeup info, was a good prospect (possibly top 100-150 picks) entering the spring but had a bad season and then fell down draft boards. He’s got an 88-91 mph fastball, a great changeup, but BA noted his curve was suspect. So he is basically the same type of pitcher as Matt Way. Same caveat, wait and see how they progress against better competition, but hes definitely a name to watch.

Jesus Sanchez, the last remnant of the Abreu trade (I think), continues his conversion from catcher to pitcher, had a solid year at Lakewood. He logged 136 IP, a pretty staggering number all things considered, and had solid peripherals. Jarred Cosart, who got a lot of positive juice in the BA Top 10, sits among an interesting group including fellow 2008 draftee Jon Pettibone. Austin Hyatt, who also got some love among NYPL observers, comes in slightly lower, but this is again an innings pitched adjustment, as his sample size is much smaller. Jumping further down is Andrew Carpenter, who has kind of stagnated over the last 2 seasons. He has some troubles with the long ball, he’s hittable, and at this point I’m not sure what his role is going forward.

The big surprise here is probably Trevor May. As you can see, May is kind of a feast or famine type right now. He struck out an eye popping 95 in 77 IP, but he also walked 43. He did keep the ball in the park, and he was really humming along by the end of the season. I’m much higher on May than the score represents, but again, that’s kind of the point, to have something objective to look at. May will rank near the top of my prospect list, but I think this is one of those things that merits watching. Brody Colvin obviously threw only 2 innings, you learn nothing from 2 innings.

Lets move along to the next group of guys

phillies-pitchers2

If you cannot see the image, click here.

Lots of organizational guys here, so not a whole lot to discuss in detail. Sergio Escalona is mildly interesting, and his strikeouts and walks are fine, but he was somewhat homer prone, which is a concern. Tyler Cloyd, a favorite of some here, has underwhelming strikeout totals, which brings his score down, but his control is good. Mike Cisco is an interesting guy. He showed excellent control but really didn’t miss many bats, and just as troubling, he was very homer prone at Reading. Vance Worley got killed over a 2 month stretch, which really brought down his score. Was he just tired? Did hitters adjust? Whatever it was, his season as a whole looks pretty ugly, but hes not someone I’d give up on.

And the final batch of guys

phillies-pitchers3

If you can’t see the image, click here.

Some interesting guys end up here, unfortunately. Mike Stutes had just an ok season and was vulnerable to the long ball at Reading. He didn’t really miss a ton of bats and also issued a few too many walks. Heitor Correa also shows up here. As you know, he missed all of 2008 due to a team suspension, and its pretty likely that he was just shaking off the rust in 2009. He’s still very young, he can repeat Lakewood and still be age appropriate, so I wouldn’t give up on him. It pains me to see Julian Sampson down here, but was there any other place for him? Its almost shocking that his score wasn’t worse. His K/9 is the worst of anyone on this entire list, which really says something, his walk rate wasn’t great, and for a sinkerball pitcher he gave up a ton of HR. Just an all around disaster. Another guy I’ve defended to no end, Joe Savery, also comes through with a very poor showing. His peripherals across the board really just were not good. He only struck out 1 more per 9 than he walked, which isn’t going to get it done. Your guess is as good as mine on him at this point. Its foolish to say he never had talent, I’ve outlined this countless times before, he was an elite prospect after his freshman year at Rice. The Phillies gambled that he’d come all the way back from his arm injuries, and he obviously didn’t. Took a gamble, lost the bet, and you move on. He might make it as a LOOGY, maybe he decides to take the Ankiel route and start hitting again, but its obvious that his prospect star has dimmed considerably at this point.

There ya go. Feel free to discuss any of the guys here, some I didn’t touch on just because I can only write the same types of descriptions so many times. If you have questions that I can answer about the system and the scores, ask away. I have to do battle with the DMV tomorrow, so I may not have a chance to address the issues until Friday. Enjoy.

27 thoughts on “SONAR Scores – The pitchers

  1. Carlos Monasterios also came over in the Abreu trade. Kinda nice to see him and Sanchez both grading out fairly well… maybe we’ll end up getting *something* out of that deal in the end.

  2. I’ve spoken with someone very close to De Fratus and was told that he is being moved to the bullpen full-time.

  3. Thanks for that info, Mike. I still think he has a shot to make it as a starter, but his stuff should really play up out of the bullpen, and he could move quickly at this point.

    Disappointing, though not entirely surprising, to see guys like Worley, Stutes, and Savery at the low end of things. Of the three, at least Worley still has age on his side, though his next go around at Reading is going to have to be much better.

    One question, James: how big of a role do you see league and park playing in the SONAR score, specifically with respect to HR/9? Homers are obviously a big portion of the consideration here — e.g., Julio Rodriguez has fantastic K and BB numbers, but presumably his HR and low innings totals drag his SONAR down — and someone like Worley, who pitched the entire year in a hitter’s park, is at a distinct disadvantage compared to someone like Drew Naylor, who spent the whole year in the FSL.

  4. A couple of queries:

    Personally, I’d advise you to be wary of leaning too heavily on minor league DIPS numbers. Just because those numbers work on the MLB level doesn’t mean they will have the same efficacy in the minors. There is also evidence that minor league BABIP can be used as one indicator of minor leaguers who will succeed in the majors:

    http://www.baseballprospectus.com/article.php?articleid=3946

    “Overall, the results are clear. The pitchers who made the major leagues are, not surprisingly, better than their counterparts who did not, by every measure of pitching you may desire–including giving up fewer hits per ball in play.”

  5. Honestly, I think this is a fool’s errand. There have been other analysts trying to make sense of minor league statistics but they are using far more advanced data. GO/FO ratio for one. It would take a lot of work and proofing to get this up to snuff and even so, there is only so much you can do with minor league statistics.

  6. I honestly think once our Bias and hope is taken away these lists seems fairly accurate. We like Stutes and Cisco and think they can succeed but they did put up some sub-par numbers this season.

  7. I don’t understand why Bastardo is so far below Drabek on your list. I don’t say the order is wrong just what is the justification, please.

  8. Alan: SONAR uses plenty of advanced data, and GO/FO numbers, like all minor league batted ball data, are flawed and unreliable. Perhaps incorporating ground ball percentage (the most trustworthy ball in play data available) would be useful, but I think you’re mistaking what phuturephillies is trying to do here. He isn’t trying to build an all-encompassing system that will magically spit out the best prospects; he has constructed a system that is meant to serve as a data point in analyzing prospects, so that we can use SONAR along with scouting reports, PECOTA, tRA, FIP, and whatever other analytical tools you want.

  9. Nowheels, I’ll bet that Bastardo received “deductions” for two reasons: (1) age relative to level; and (2) sample size. And both of those seem fair to me.

  10. Thanks for the hard work of compiling the data and building the model. It seems to me that you have provided a pretty accurate, unbiased picture of how well players have actually performed. In terms of eventual MLB viability the only thing you don’t capture is the random “off” or “on” year that a player might have. But that’s the point of scouting reports and additional years of data should even that out.

    I did have a thought on the random “everything clicks” type of event that happens with many stars. You may have already thought of this, but as you gather more data over the coming years, you may want to note or look for commonalities among players that have “everything clicks” events. For instance maybe 90% of those players have the pieces fall in place between the ages of 22-23.

    Anyway I love this stuff so thanks for sharing.

  11. Very interesting results, thanks again for putting all this out there, really enjoy the thought exercises and the chance to become more familiar with the entire farm system.

    I think there is something weird with mixing the results of starters and relievers together, but this is partly because of my personal biases. Specifically, I wouldn’t compare guys like schwimmer and rosenberg with guys like drabek that pitch 3x as many innnings. In terms of comparisons, I would prefer seeing the guys that are strictly 100% relievers only be compared to only other relievers, so as everyone gets the same type of deductions for sample sizes. I think starters are inherently more valueable, and agree that they should have more weight and that their results from a system like this would have less variance to the their actual performance levels. Relievers are subject to much noise simply because they get less chances to pitch.

  12. As always, very interesting data. I’m surprised that BJ wasn’t higher after the year he had. The list seems to have more disconnects to me that the hitters list had but I appreciate that its based on actual results. As far as DeFratus relieving, I’m surprised but there has to be some compression on the starters because there aren’t enough spots since most of the Reading staff is staying intact.

  13. One question, James: how big of a role do you see league and park playing in the SONAR score, specifically with respect to HR/9? Homers are obviously a big portion of the consideration here — e.g., Julio Rodriguez has fantastic K and BB numbers, but presumably his HR and low innings totals drag his SONAR down — and someone like Worley, who pitched the entire year in a hitter’s park, is at a distinct disadvantage compared to someone like Drew Naylor, who spent the whole year in the FSL.

    Its a good question. HR rate is weighted less than K rate and BB rate factors. I went back and looked at Rodriguez specifically and I can tell you why his score was so low. He gave up 6 HR in only 49 IP, and the GCL is one of the toughest leagues in all of the minors to hit HR. The average HR rate in the GCL for a pitcher was 0.35/9, Rodriguez had a rate of over 1.0. Thats a big red flag for me. And his score was reduced because he only threw 49 innings.

    As for park and league adjustments, they are all in there. Minor league park factors are shaky, but I used 3 year averages, not just 1 year, so it hopefully removes some of the noise. League strength is also factored in. That means that a prospect pitching in Reading (more of a hitter’s league, a very hitter friendly park) will have his score adjusted upward compared to a guy who pitches in a very pitcher friendly league in a very pitcher friendly park.

    Whats important to remember is that some leagues are very hitter friendly, but they also do contain pitcher friendly parks. The Cal League, which is an advanced A league, is considered the best hitting environment in the minors. High Desert and Lancaster are launching pads, with winds of 30+ mph blowing straight out on some nights. But on the opposite side of Lancaster and High Desert are San Jose and Inland Empire, which are both very strong pitcher’s parks. So guys who play in those two parks aren’t treated the same was as Lancaster and High Desert guys just because they play in the same league. Their baselines are the same (ie, the league average hitting stats for the entire Cal League), but they are definitely adjusted differently with regard to their parks.

    The league adjustment is the most critical. Because there is a huge difference between, say, the GCL and the Texas League, or the Midwest League and the Cal League. And then within each league, players are adjusted accordingly based on their park.

    I think there is something weird with mixing the results of starters and relievers together, but this is partly because of my personal biases. Specifically, I wouldn’t compare guys like schwimmer and rosenberg with guys like drabek that pitch 3x as many innnings. In terms of comparisons, I would prefer seeing the guys that are strictly 100% relievers only be compared to only other relievers, so as everyone gets the same type of deductions for sample sizes. I think starters are inherently more valueable, and agree that they should have more weight and that their results from a system like this would have less variance to the their actual performance levels. Relievers are subject to much noise simply because they get less chances to pitch.

    Yeah, and you can basically do this by just looking at the list and determining who is a reliever. I could have gone through and added that, but I didn’t think it was necessary. Reliever performance all the way up to the big league level is subject to quite a bit of variance from year to year, and in the minors its no different. That’s essentially why I built in the adjustment based on innings pitched. Starters are more valuable than relievers, and that should be reflected here. But Schwimer is a great example. He was much more dominating than Drabek when you stack up their peripherals, but Drabek is a starter, is a good bet to remain a starter, and thus will have more longterm value. However, Schwimer has dominated and right now he might be more valuable than a fringe starting pitching prospect. I think it kind of works itself out.

  14. James, have you taken note of baseball-reference’s franchise depth charts? They’ve added options now where you can format the list into a table and presumably grab it and plug it right into a spreadsheet. It could make raw tabulation a cinch.

  15. hey…looks like you finally agree with me on Savery :)

    but in all seriousness, i would love to see you do this ranking system verses other players that are professionals now. it is really hard to validate it otherwise. my gut was, drabek is scored too low. as were brown and taylor. but maybe they are just over-hyped. or maybe the system deflates value and just needs to re-adjust itself.

    how hard is it to run 10-15 mlb players through the system based on their minor league stats?

  16. To do that, I’d have to go back and collect the league averages for the leagues those players played in, plus try and figure out the park factors for their parks. If its guys who haven’t been in the minors for 15 years, theres a chance that the park they played in isnt even being used anymore or the affiliate is gone.

    Ill tell you what though, if you email me a list of 10 guys who you most want to see, I’ll try and see if I can’t throw something together. Its going to take some time though.

  17. Perhaps Bastardo is so much lower on the list than Drabek because Bastardo is expected to end up as a LOOGY/reliever while Drabek’s high end is a #2 starter?

  18. Phuturephillies,

    My apologies if this was asked in the first batch of SONAR scores, but does the score it self scale linearly (i.e. is a score of 24 twice as “good” as a score of 12)? Or is there a specific gap in performance or some sort of curve?

    I can’t help but look at the numbers and think the range of each data set would create a skew in the observed SONAR. Perhaps I think too plainly to believe that one man with a humble website created a player-value metric without flaws? I dunno, any answer would be much appreciated.

  19. Are you using the rate stats and DICE in your final calc? Because DICE already takes into account rate stats, so counting both DICE and the rate stats seems like you’re double rewarding guys with good rates and double punishing guys with bad rates.

  20. No I’m not, its only a confirmation statistic.

    Kevin, I mentioned in the hitter wrapup/intro that the scores are basically distributed in a bell-shaped type curve. the largest chunk of the scores come in the 35-50 range in terms of the equivalencies to the 20-80 scouting scale.

  21. NEPP Says:

    Perhaps Bastardo is so much lower on the list than Drabek because Bastardo is expected to end up as a LOOGY/reliever while Drabek’s high end is a #2 starter.

    I was asking in regards to this system of statics not expectations

  22. James, have you considered crowdsourcing your data entry?

    It seem to me that the most valuable additional data to look at would be results from a couple of years ago. We know what the results are, more or less, who washed out and who made the show, and you could validate (or invalidate) the stat and it variants- see what data has predictive value.

  23. Phuturephillies,
    I respect your efforts intellectually and physically in producing your SONAR system of analysis. It is exciting to see a passion pursued with such diligence, much like a minor leaguer. Good luck with it.

    Minor league baseball is about development and there are always components of the development process that unless part of the inside process is not always given consideration by the accounting of statistics or the fan following either at the stadium or the reading of boxscores.

    During the course of the year a batter may be asked to work on a new grip or stance. He may be asked to work counts or only swing after he has received a called strike. He may be told to hit opposite field during every at bat no matter the situation or he may be working on pulling the ball during all at bats. He might be told to offer only at a certain pitch or be told to approach his at bat as though a certain situation exists even though it does not. A pitcher may be asked to work a particular part of the plate or a particular pitch no matter the situation. Pitchers may be asked to work on grip or foot set up on the rubber. They may be working a new arm slot or leg drop.

    In the majors the mission is to win and the analysis of statistical data on a given player is representative of an approach geared to that end. In the minors the mission is to develope and statistical data will be skewed because of the effort towards specific developmental tasks.

  24. I think I pretty much addressed everything you brought up in my intro. I understand what the minor leagues are for. I understand that guys will sometimes post bad numbers because they’re trying to overhaul their approach, add a new pitch, tweak something, or whatever else. Which is why I said that this system should just be taken as another data point, its not meant to replace scouting reports.

Comments are closed.