I present to you my new statistical toy: the Prospect Power Rating (PPR)

So here’s the thing. I get lots of ideas in my head, and most of them I either discard, or I begin working on it, then give up half way through and move on. The SONAR project, something I started years ago, is somewhere in category 2, because I have a lot of ideas for it, but its incredibly in depth, it takes a ton of time, and its imperfect. Until I figure out the ways to remove most of the imperfections, it will remain dead. But in the meantime, I had an idea that kind of comes out of that semi-failed project. I was thinking it would be fun to have a way to take a quick and dirty look at how our prospects are performing, but without doing a ton of heavy lifting and adjusting. I figured once I built the spreadsheet, I could do a lot of copy/paste work, and it would basically update itself. And I think I was right. Which means I should be able to maintain this with little effort, which means I won’t get bored with it and discard it. Maybe. Anyway, I figured I’d unveil it here, and we’ll see how it goes. Full explanation below the fold, as well as the inaugural rankings

I figure the best way to introduce this is to answer a few basic W questions

What: What is this? Its just a toy. Its not meant to be statistically significant, because it takes raw data and runs it through a formula. I came up with the weightings, which I’ll get to in a minute, and its based on how I evaluate prospects. The whole formula will be outlined, I’m not hiding anything or trying to rig the system to produce results. You shouldn’t take the results as some sort of written in stone reading on a prospect. That’s why its called a toy. Its fun.

Why: Well, why not? I think that’s easy enough.

When: I plan to update this every 2 weeks. Hopefully. Remind me if I forget.

Where: On this site, silly. I’m going to create a page to keep all of the results. It will be at the top. Its not there now, so don’t panic.

So I don’t lose most of you with the math (even though its basic, because I suck at math), I’m going to outline this in really simple terms.

The actual score

The score is represented as a percentage. The higher the percentage, the higher the score. It will make sense when you see it.

The study population

I made very simple cutoffs in terms of who to include. The following minor leaguers were omitted from the results

3A: Age 27 and older players
2A: Age 26 and older players
A+: Age 25 and older players
A: Age 24 and older players

If you’re still at Lakewood and you’re 24, your prospect ship has probably sailed. Probably.

The metrics

As I mentioned in the intro, the point is for this to be quick and dirty. For hitters, I chose 2 stats, very easy to calculate, and they are

Secondary Average – This stat does a lovely job of incorporating a player’s power, his ability to draw walks, and his stolen base capability.
Strikeout Rate – Or better expressed, contact rate. Guys who rack up huge strikeout numbers against bad pitching in the low minors are generally going to struggle to make contact as they climb to tougher levels. This is important. Calculated by dividing strikeouts by plate appearances

For pitchers, I used 3 metrics:

Strikeout Rate – The ability to miss bats is hugely important for a pitcher
Walk Rate – Wildness may scare hitters, but they’ll just take their base. Walking guys isn’t great
Home Run Rate – Pitchers have some control over this, less than the first two, but it is an important indicator

These are the “three true outcomes” for a pitcher. They will be weighted differently, I’ll get to that in a minute.

Hitters and pitchers also have two more adjustments made to their performance, and they are:

Playing time – If you just adjust a player’s performance relative to league average (which I’m doing, more on that in a minute) and don’t consider the sample size, you’ll get some really wild results.
Age – I harp on this constantly, but its here as well. A .300/.400/.500 line in A ball means something completely different if an 18 year old does it, compared to, say, a 23 year old. Context baby, it matters!

The Weight

Its a great song by The Band, but in this case, I’m talking about the weights given to each statistic, and the adjustments made for playing time and age. Here they are

For hitters, SecA and K%:

SecA = 75%
K% = 25%

Simply put, SecA factors in power, walks, and speed, while K% factors in just contact skills. Makes sense, I think. Again, quick and easy

For pitchers, K/9, BB/9, HR/9

K/9 = 50%
BB/9 = 35%
HR/9 = 15%

Strikeouts are really important. If you can’t miss bats in A ball, the odds of you missing bats as you climb the ladder isn’t great. A guy with sub-par control but great swing and miss stuff might be able to learn some control, but a guy with only fringy stuff, even if hes a strike thrower, is gonna struggle against better hitters. HR rate is the weakest of the indicators, because there is the normal batted ball luck, but generally, if you’re giving up a lot of home runs, either your stuff just isn’t great, or more likely, your command isn’t good and you’re leaving pitches in the fat part of the plate. Again, its less reliable as an indicator, but it still has at least some analytical value, moreso than hits/9. And its easy to calculate. Again, quick and dirty.

Age adjustments are tough, but this is the scale I use to weigh a player’s score

3A

Age 21-22 = 1.5
Age 23-24 = 1.0
Age 25-26 = 0.5

2A

Age 20-21 = 1.5
Age 22-23 = 1.0
Age 24-25 = 0.5

A+

Age 19-20 = 1.5
Age 21-22 = 1.0
Age 23-24 = 0.5

A

Age 18-19 = 1.5
Age 20-21 = 1.0
Age 22-23 = 0.5

I hope this makes sense. The “average” legit prospect age in A ball is 20-21. So they are the baseline, at 1.0. If you are younger, you get a bump (1.5) and if you are older, you get docked (0.5). If a player has a “negative” score, his age factor is flipped. For example, if a player is 22 in A ball with a positive raw score of 10, his age adjusted score is 5. 10 * .5 = 5. On the flip side, if a 22 year old player has a raw score of -10, his adjusted score is -15 (-10*1.5) = -15). Your age either helps you or hurts you. If you are old for the level, it hurts you either way. If you are young for the level, it helps you either way. I hope this makes sense, it does to me because I’ve been working on versions of this formula for a long time.

Playing time is trickier. To adjust for position players, I took the total number of games the team has played, and multiplied it by 3, 3 being the number of plate appearances. So, if a team has played 50 games, I multiply by 3 and get 150 PA, which would be the baseline. If you have fewer than 150 PA, your weighted score is reduced. If you have more than 150 PA, your weighted score gets a boost. For pitchers, I assumed a baseline of 120 innings pitched in a minor league season. That’s essentially 24 innings per month, for 5 months. If a starter begins the year in the rotation and stays there all year, he’ll surpass that. If not, he probably won’t. But you have to “punish” relievers, because they pitch fewer innings, and their ratios are subject to more noise than a starter, and well, I value relievers less than starters as a general philosophy. Again, quick and dirty, not really scientifically significant, but its a toy!

The formula

Edit, I forgot the formula! Each category (SecA, K%, K/9, BB/9, HR/9) is calculated based on the league average. So, if the player is in AAA, its his numbers against the league average for all players in the International League. The same applies to all leagues. The hitter’s performance, after adjusted relative to the league average, is adjusted based on age, and then finally the playing time adjustment is made. Playing time adjustment is based on a 100% scale. If the baseline is 150 PA, and the hitter has exactly 150 PA, then his playing time adjustment is 100%. If the player has more than 150 PA, his PT adjustment will be greater than 100%, if he has less, it will be less than 100%. This is to keep it a positive number. For pitchers, the weighted sub-total against the league average is multiplied by playing time and age adjustments.

The result

The result, for both pitchers and hitters, is a score that is scalable and comparable. 0% represents a perfectly league average performance, adjusted for age and playing time. A positive percent represents above league average performance, adjusted for age and playing time. A negative percentage represents a below league average performance, adjusted for age and playing time. Because of the weights (adding up to 100%) for both pitchers and hitters, the scores are comparable. So 30% for a hitter is the same as 30% for a pitcher.

What is missing?

Well, a lot, but for the 10th time, its a toy!

For position players, defense is not counted. For pitchers, nothing outside of the 3 true outcomes is counted. Also, none of the numbers are park adjusted. Because minor league park factors are really wonky, and to be honest, I don’t trust them, even the 3 year weighted numbers. And that would take a lot more time to compile. Lakewood is a pitcher friendly park, Reading a hitter friendly park, so you can take those players with a pinch of salt, and conversely, adjust the hitters and pitchers respectively in your head. Since all players are evaluated based on their league averages, the performances don’t have to be further adjusted to take league strength into account.

The initial standings!

I’m going to post these as image files, because google documents is kind of a pain, and its easier to just post the image so you don’t have to click a bunch of links. As I mentioned, I’m going to incorporate this into the site at the top, but I haven’t done it yet. There are a number of things I want to do with regard to the site layout/navigation, but its gonna take some time. I’m going to give you the full chart of hitters, the full chart of pitchers, and then the consolidated rankings with just their PPR score. You can’t see the formulas, because its just EXCEL values, but I explained all of the weights above. Players highlighted in YELLOW have played at multiple levels, so I had to create a separate sheet to calculate their totals, which is why their weights are blacked out. I’m not hiding them, I just have to calculate it on a separate worksheet. All stats are current as of May 30th and taken from baseball-reference.com. Its late, if something looks wrong, tell me so I can correct it later. Discuss/Enjoy!

(click the images if they are too small, they should open in a new tab!)

Hitters

Pitchers

Consolidated Rankings


The Conclusion?

This is not a re-ranking of my top 30 and isn’t meant to be. This is simply evaluating the statistical performance of the player, in the context of his league, with considerations given to his age and his playing time sample size. That’s it. Its a toy. Its fun. Don’t get carried away!

Edit 1 —> I knew something didn’t look right. I forgot to add one of the cells in the formula. Fixed!

109 thoughts on “I present to you my new statistical toy: the Prospect Power Rating (PPR)

  1. I like it. Don’t know what the overall value is but it is a toy (11th time). Looks like the 2A guys except for Galvis strike out a lot. Swinging for the fences?

  2. Needs some work. The age factors are incorrect on a lot of players, Singleton and Valle jumped out right away.

  3. I appreciate that the playing time factor was put in to mitigate sample size, but I still don’t think it helps when analyzing Starters vs Relievers. Starters have to pace themselves, have to navigate lineups multiple times and face the best hitters every game. A good reliever should always have a better K/9 and HR/9 than a good starter, because he is able to go all out to hitters who haven’t seen him multiple times. A reliever’s 1 inning of work is not equal to a starter’s 1 inning of work, IMO. Relievers and Starters should be ranked separately.

  4. Also, unless I have the wrong interpretation of the age factor, it looks like many of the pitcher’s age factors are backwards. Example: Fierebend, Shreve and Biddle.

  5. Let’s see El Stumpo only gets into one game all season, goes 3 for 4, and still comes out a minus (2). Maybe if he ever gets in another game he can go 4 for 4.
    Ruf the hitter comes out ahead of Ruf the pitcher. As I recall, Ruf the pitcher didn’t give up a run.
    Ages look all right, weighting formula unknown. People object because it doesn’t fit their screed.

  6. Great, but one big request: List former prospects! Both the ones who worked out and the ones who didn’t, that would be fun, too!

  7. Actually, I do think the list is useful as something more than a toy–it’s reflective of our prospects’ actual performance, independent of our expectations/hopes/projection. In that sense, it’s interesting to look at. No one should get too excited about Rizzotti or Overbeck sitting at or near the top of the list, or too down on Singleton for being in negative territory, but it does give you an idea of how they’re doing right now. Also, at the extremes, it might highlight cases where guys are being underappreciated/overappreciated due to personal biases. Rizzotti really is far and away the best hitter in the system right now, according to this, and JC Ramirez’s reliance on BABIP luck looks like a big red flag. That seems like useful info–stuff we knew already, sort of, but the list really highlights it.

  8. Fun stuff. Maybe most useful in picking out surpsises – Mendez’ name leaps out in this regard. Note to some: the age “errors” aren’t errors; in order for the system to work, the weights need to be reversed for below average players.

    While just a toy, I have 2 thoughts:
    (1) The age thing needs to be worked differently. It shouldn’t be a percentage, it should be a lineat scaling of some sort, say maybe addeding/subtracting from the weighted subtotal. For example: a young guy in a league with average numbers should have an above average rating.
    (2) Position adjustments.

  9. Okay, a note on the age factor. I thought I explained this above.

    A player with a negative weighted score who is young for his league is helped with the adjustment, while a player with a negative weighted score who is old for the league is punished.

    Example from above.

    Biddle is young for Lakewood (19) so he will get an age boost. His peripherals are negative (-3.4%), so instead of his multiplier being 1.5 (a boost for a prospect with positive peripherals) it’s only 0.5

    On the opposite end of the spectrum, Feierabend has negative peripherals (-36.2%) and he gets punished at 1.5 for age because he is old for AAA. If his peripherals were positive, his age multiplier would only be 0.5

    Hope that clears it up.

    1. Clear as a bell for me. I think the problem is that the scale tha you use for age makes the system fairly unuseable except at the extremes. For players around the league average in the weighted stats, it massively overestimates the value of older players and underestimates the value of younger players. A 19 year old in A+ ball who is a league average hitter is a top prospect; a 24 year old in A+ ball who is a league average hitter isn’t a prospect at all. Yet under your system* they are given precisely the same rating.

      *Yes, I know it is just a “toy.” But even so if the toy is to have value, it needs to treat age better. I woudn’t bother commenting if I didn’t think the system otherwise had a lot of potential.

      1. I ran in to this problem with SONAR. Here is an example.

        A 17 year old prospect and an 18 year old prospect are likely separated by less than 12 months in age. There isn’t a TON of difference here. A 17 year old prospect and a 19 year old prospect is a wider gap.

        By cutting the age gaps where I did, I think its a pretty significant shift.

        In Lakewood, 18-19 year olds are scaled at 1.50 for the age factor, while 20-21 year olds are scaled at 1.00 and 22-23 year olds are scaled at 0.50. Thinking in percentages, a 19 year old prospect’s line is given a 50% bump.

        If prospect A is 20 and has a rating of 20%, his score is 20%. If prospect B is 19 and has a rating of 20%, his score is 30%, which is pretty significant.

        The risk of assigning one age weighting to each individual age is that it can create huge gaps in value, even when the age difference is only 2-3 years. For SONAR, I tried to break it down with one age weighting for year or age, and I found a lot more noise in the ratings.

        I could theoretically break the weighting down like this, using Lakewood as an example

        Age 17 = 1.75
        Age 18 = 1.50
        Age 19 = 1.25
        Age 20 = 1.00
        Age 21 = 0.75
        Age 22 = 0.50
        Age 23 = 0.25

        But this is much more rigid, and not every prospect, or even the majority of legit prospects, follows this pattern, especially players drafted out of college. I think dividing ages into 3 categories instead of 6 keeps things simpler. And that was kind of my aim here.

        1. I don’t think that addresses the real problem. And honestly I am pretty sure that I am not explaining my objection well.

          Try this: your system – even if you break age down more – puts more signficance on age the further you get from zero (average) in either direction – and NO signficance to age at the zero bound. Take a guy like Singleton – just about average, so that the age adjustment barely moves his rating. A 24 year old with his numbers would rate almost the same, just a little lower. A 19 year old in A+ holding his own should rate much higher on your scale. That’s a flaw, and breaking ages down further doesn’t address it.

          Or this. Under your system, Singleton gains only about 5 rating points for age. But Pettibone gains about 10 points from age (actually more because of the compounding effect of the PT adjustment) because he is further from the zero bound. Now, Pettibone should be rated higher, but he should gain the same amount from age, not more (or arguably less, 19 versus 20, but you address that issue).

          It’s not easy to address, unfortunately. The easiest way to do it would be to make the age adjustment additive rather than multiplicative. But there are problems with that approach as well.

    2. Thanks for explaining it again. I didn’t get it when reading the first time.
      I also automatically thought it had to be wrong due to a cursory glance at Biddle’s rank in comparison to Buchanan and Hollands. Just a couple of days ago, I noticed how close Holland, Buchanan and Biddle’s FIP are. My incorrect assumption was that due to age and similar FIP, Biddle would rank higher than either Hollands or Buchanan.

  10. It would be cool if in future rankings there is a /- next to the name of the player that indicates if they are moving up or down.

  11. Any system that gives a result of Valle being trash and the Rizz being the 2nd best prospect needs to be reworked.

    1. Yeah, see it penalizes Valle because he doesn’t walk at all, has no speed, and isn’t an elite contact hitter. A .164 SecA is below average. That doesn’t mean he isn’t a better “prospect” than Rizzotti, because he certainly is, it just means that statistically, Rizzotti is having a much better season. Which he is.

      I knew that even with a detailed and lengthy explanation, many people will miss the point of this. Its not complex. It takes 2 stats which I think are pretty important, looks at how the prospect performed in those 2 statistics, and spits out a score. If you don’t think Secondary Average is a good statistic, you obviously won’t find much use to this. But if you calculate the SecA of the top 50 or so big league hitters (ESPN offers it as a sortable stat under “sabermetric stats”) you’ll see that the leaders in this category are likely the best hitters in the league, or the best performers this season.

      And again, for the final time, this is not meant to be a determining factor on a prospect’s worth. It’s a data point. A fun toy. It takes a few metrics I consider really important, makes a few basic adjustments, and provides an output. This isn’t a re-ranking of my top 30. Its a snapshot as to how players are actually performing based on peripheral stats, not just three slash lines and ERA/WHIP.

      1. Then you need to adjust the age ratings or something. Or incorporate prior years performance. Or toss out the playing time factor since Valle is getting harshly punished for missing games due to injury. Rizz’s K rate is virtually identical, he has no speed.

        Not sure what the issue is, but the toy isn’t working if it shows Rizz as superior, no matter what secondary average is. 4.5 years and similar production one level apart is bigger than secondary average.

        No reason to get your back up.

        1. You’re missing the point.

          * its not meant to assess data from last year. Its for this year. Its not broken.

          * the age factor is fine. If age wasn’t factored in, Rizzotti’s score would be higher, Valle’s would be even lower.

          * this is only measuring statistical performance in key areas, it doesn’t evaluate their raw tools. That should be clear.

          Rizzotti has a .382 SecA and a 21.3% K rate.
          Valle has a .162 SecA and a 20.6% K rate.

          Even when you adjust for their age and league, Rizzotti’s statistical performance is miles ahead of Valle’s.

          1. Explain to me how a 20 year old hitting .326/.341/.477 in A+ ball should rank in the lower half of a ranking system. Valle doesn’t walk, we all know that, but he is way age advanced and has good ISO.

            Perhaps the age factor isn’t fine if it doesn’t come closer to normalizing between the two of them.

            1. I’m just saying that you are putting too much emphasis on one stat. Doesn’t mean that sec average isn’t a good stat.

              You don’t explain your age adjustment factor.

              Tools should give results that make sense. This result doesn’t make sense

            2. The age factor was explained in the article.

              Average prospect age

              A Ball = 20/21
              A+ = 21/22
              2A = 22/23
              3A = 23/24

              Its adjusted up or down from there. Its all explained in the setup above.

          2. See prior responses on the age factor PP. It isn’t fine for players like Valle. I agree it is fine for Rizzotti, who, gievn the nature of the system, should legitimately score very well.

            Note that even a “better” age system would place Rizz over Valle. But the gap would be narrower.

            1. I also think that arguably Valle is particularly tough guy for the system to rate. The BBs are so insanely low that they hurt his rating maybe more than they should. But that’s not a fault of the system; he’s just a somewhat hard guy to rate.

        2. Valle needs to start walking or he’ll never be an MLB player. Perhaps this tool is simply illustrating that fact?

        3. Your instinct is right but you are focusing on the wrong issue.

          The system gets it wrong for Valle – and it does – mainly because it does not adjust appropriately for age. (And secondarily it is not position adjusted, but that is arguably a valid choice, as opposed to current age rating system which is simply fubarred.)

          That is, PP’s response to you is almost entirely correct on the particulars – except that you’re correct that even given all of that Valle ranks lower than he should. Valle is a little below average as a hitter for his league – but given his age and position, he is a heck of a prospect anyway. The system intentionally doesn’t consider his position, and underestimates the value of his young age.

          Note that ANY system like this, which does not factor in defense or position, and looks at this year only (all valid choices) will necessarily rate Rizzotti very high. He is having a heck of a year as a hitter, any way you look at it. The problem isn’t Rizz’ rating, but Valle’s.

        4. It is partly what larrym said — by using the age factor as a multiplier, you give it almost no impact for an average guy. The age factor likely would work better as an add or subtract. That way it would be a significant factor anywhere along the performance spectrum.

          Second, I suspect that this tool weights AB and IP too heavily.

          Finally, and this is not a problem at all with the tool, but how you look at the result — this only measure offense. You have to combine the defensive side to get a prospect’s overall value and ranking. Thus, it is not at all surprising that Rizz ranks at the top — he does great offensively. His problem is poor D and that at a low value position. Similarly, Galvis gets a big boost for quality D at a primo positon. Valle gets a boost for at least OK D at a primo position. You almost need to plot the guy on two axes. This tool, with a better treatment of age, can work as the offensive axis. Now plot out the other axis with your best guest of defensive ability and value of the position played.

          1. The interesting thing is that if you make all the adjustments that we are calling for, Galvis would shoot near the top. I’m a Galvis skeptic, but that did get me to take a second look. And you know what? After over 200 PA, he is improved pretty much across the board. K rate down, BB rate up, power up. I was saying six weeks ago that his chance of develpoing into a regular was maybe 5%. I’m beginning to think that that was low, maybe way low. (Yes, he looks MUCH more impressive to me now batting .258 than he did early in the season when he was batting close to .300.)

            I’m still not nearly ready to annoint him the Phillies SS in 2012 – but I think the chance of him eventually filling that role has increased markedly.

            And that’s why I hope that PP fixes the age factor problem (which, by the way, is even more apparent when looking at the pitchers). This system is great for focusing attention on guys who are flying a bit under the radar.

    2. Hewitt has a positive score. This thing must be garbage!

      Seriously, I do like this. It’s a quick and dirty way to compare how guys are performing. Similar to SONAR, it is just one more data point to add to everything else we know (or think we know) about a player. I wish I had the imagination/energy to come up with stuff like this.

      1. Well, had we never seen Hewitt before and he was posting these numbers in Lakewood, we’d be pretty excited about his chances. Had he been, say, a JC guy picked in the 2010 draft, we’d be talking about his great defense and arm in CF along with his surprising pop…and all that other toolsy crap we say about every athletic OF we get.

        1. I had the same thought when reviewing his numbers this past week. Take it a step further: If Hewitt went to college he would have been a draft eligible sophomore in 2010. His placement in Lakewood this year would have been standard and he would be headed for the top 30 prospects list. There still would be concerns about his K/bb% though.

  12. One factor that should be incorporated is some weighting for current level. We scoff when someone looks at a guy’s numbers in AA and says that means he would be better than player X in the majors.

    The same holds true in the minors. Comparing the numbers of a 22-year old in A-ball should not be equivalent to a 22-year old’s performance in AA-ball but this system gives them the same age weighting factor. Even a 23 year-old putting up numbers in AA-ball is more impressive than those same numbers from a guy 2 years younger and 2 levels lower.

    Everyone agrees that the jump from A+ to AA is significant but nothing is done to reward that successful jump it it happens when the player is “age appropriate” for that level..

      1. OK PP, try this one on for size.

        Part 1:
        Modify the age scale to have 5 levels

        .5 – 2+ years old for level
        .75 – 1 year old for level
        1 – age appropriate
        1.25 – 1 year young for level
        1.5 – 2+ years young for level

        Part 2:
        Add weighting for current level
        .8 – low-a
        1 – A+ (used high-A as the baseline since most prospects achieve this level at some point)
        1.4 – AA (gave additional jump since this is the most significant level jump for prospects and greatest failure point)
        1.6 – AAA

        Part 3:
        Add weighting for position spectrum
        .7 – 1b
        .8 – LF
        .9 – RF
        1 -3B
        1.1 – CF
        1.2 -2b
        1.3 – SS
        1.4 – C

        For calculation purposes, these 3 factors are multiplied against each other to get a final weighting that is then used to calculate against their performance numbers.

        Using this approach on several of the players frequently mentioned gets the following weighting

        Rizzotti – .75 (age)*1.4(AA)*.7(1b) = .735 weighting factor
        Valle – 1.25 (age)*1(A+)*1.4(C) – 1.75 weighting factor
        Galvis – 1.5(age)*1.4(AA)*1.3(SS) – 2.73 weighting factor
        Singleton – 1.5(age)*1(A+)*.8(LF) – 1.2 weighting factor

        Unfortunately, I’m not able to apply this weighting to see how the numbers affect the ratings since I’m not going to attempt to recreate all of your work but would be interested to see how it shuffles the deck.

        1. A few comments:

          * I think adjusting for position is dangerous. For instance, you could theoretically consider Cody Overbeck at 3B, which would give him a big positional adjustment edge over considering him at 1B prospect. Then the question becomes “do we evaluate what position he plays now, or where we think he plays in the pros?”

          Same with Rizzotti, who you’d have to weigh as a DH, which has much less value than a 1B.

          The level adjustment I can definitely get with, and that’s a simple fix. I’d have to think about the weightings.

          See my comments above to LarryM about the age adjustment.

          1. I really enjoy your work, thanks!

            As you pointed out, this isn’t meant to be used as a prospect ranking, it’s just a snapshot of how they’re doing relative to how we might expect them to do based on certain underlying stats, adjusted for age. For this reason I don’t think adjusting for defensive position is helpful, especially since you are not comparing defensive stats.

            The underlying stats you chose are fine, though I’m partial to OPS and WHIP in almost everything I do. SecA does have the advantage of including stolen bases, though. Any stats will work, as you say, it’s preference, that’s all.

            What stands out here for me is your use of an age vs. level multiplier. I think that’s the greatest value in this new toy. The increments between .5 to 1 to 1.5 is hugely significant but it does highlight a point that is often missed by fans. Age vs. level really IS quite important.

            I agree with 3up that directly comparing stats across levels is undesirable. However, if you adjust for level on an absolute basis as he suggests, it still won’t work. Why should someone at Low A ball be considered .8 of a prospect just because he is farther away? As long as he is age appropriate he shouldn’t be penalized at all. He could still be more of a prospect than another age appropriate guy at AAA, the only difference is the AAA guy is farther along his progression. Divergences in the underlying stats from level to level can’t be summed up in a single multiplier, there really are too many factors to consider. Besides, this isn’t meant to be an absolute ranking of prospects, it’s just a snapshot of how they’re doing relative to how we might expect them to do based on certain underlying stats, adjusted for age.

            Maybe instead of comparing a prospect’s stats across leagues and levels, it might be better to compare how prospects at each level compare to their peers.

            In other words, you can take his stats and compare them to league averages and team averages (which will also crudely incorporate ballpark factors) rather than using an absolute number like .8 for Low-A or 1.4 for AA.

            So, for the Low-A age appropriate prospect, his .362 SecA might be quite a lot more impressive than the .362 SecA of an age appropriate prospect in AAA in a hitter’s league in a hitters park.

            You could know this by comparing the Low-A prospect’s .362 not to the AAA guy but to how that .362 SecA stacks up against his Low-A peers, both league and team.

            Again, was that Low-A guy’s .362 SecA even best on his team? Where does it rank in the league? That’s more important and accurate than comparing it to the guy in AAA. So I think you should adjust a prospects stats against his peers and THEN you can adjust that result by an age/level multiplier.

            That way you are not comparing apples to oranges, rather you’re comparing how Low-A apples relate to other Low-A apples vs. how AAA oranges relate to other AAA oranges. In other words, you’d be comparing two relationships, not two totals.

            It’s kind of like comparing stats across eras. You have to normalize for context and it can’t be done with a single static multiplier. How does 25 HR look in 1968 vs. 30 HR in 1999? I might be more impressed with 25 HR in 1968, but the only way to know is to compare relative HR totals against their peers within the year rather than comparing them across years. 25 HR by an Astros player in 1968 (Astrodome, pitcher’s year) is more impressive than 30 HR by a Ranger in 1999 (Arlington, hitter’s year) and with this method, you’d see it in the results.

            Anyway, great stuff, thanks again!

          2. You’re correct that determining a position for some players isn’t always simple but then again, since this is being offered as a snapshot of current performance, it would see to be as simple as putting the person into the position they currently play as their primary position (Or in the case of a guy like Overbeck, into the highest rated position he plays regularly)

            Once they have to be moved to a position down the defensive spectrum their rating would be adjusted.

            I get the goal is to make something simple to maintain but if the results don’t make sense then it doesn’t matter how simple it is..

  13. Hmmm. Very interesting stuff. I’ll need to study it a bit more – especially since it seems to be all over the place regarding the placement of our prospects (i.e. Our top 30 are scattered all throughout), but IIRC SONAR was kinda that way too.

    Of course, I must confess to not being overly familiar with SecA – to me is seems similar to OPS, except that it factors in net stolen bases. Is that a reasonable assumption?

    – Jeff

    1. SecA basically factors in everything except singles. The formula is

      TB – H + (BB+SB) – Caught Stealing / AB

      TB – H isolates all extra base hits, and caught stealings lower the numerator and the SecA. So speed is important, but not getting caught stealing is important too.

      1. I admit I am not terribly familiar with SecA as well. Looking at it however I like it a lot as I always thought that SB’s should be factored in to help show the true value of speedier guys with less pop vs station to station sluggers in OPS.

        However I have one question about it. Why are At Bats used as the denominator instead of Plate Appearances?

  14. I’ve been thinking about how you’re doing this age rating thing… the problem is this… (if i’m understanding you correctly) if a player scores a “zero” on your scale the age adjustment will be zero. And the larger or smaller his number, the larger the affect. Perhaps using a constant would be a similiarly simple, yet more effective method of managing this? -15 for 3 years older, -10 points of 2 yrs older, -5 for 1 year, 0 for appropriate, +5 for 1 year younger, +10 for 2 years younger, +15 for 3 years younger… this is overly simplified, perhaps it should be more like -18,-9,-4,0,+4,+9,+18 (becuase the impressiveness is not linear from 1 to 3 years young or vice versa for older).

  15. I think you guys are thinking too hard about this. Ultimately, you’re never going to be able to come up with a statistical system that perfectly adjusts for age, level, park dimensions and so on–the number of factors involved make the comparison far too complex to be described by any single formula. We all may feel strongly that Valle is a better prospect than Rizzotti, but if we tweak the formula until it spits out the result we want it kind of defeats the purpose of creating the formula in the first place. What this is a very crude measure of actual performance across levels, and it’s useful as that–a crude measure, a starting point for conversation, and another scrap of data we can use to chart how well our prospects have actually been performing.

    1. Yeah this kind of sums up my feelings.

      The point was to make it quick and easy. Objectively, if you compare Rizzotti and Valle in terms of their secondary skills, Rizzotti’s season is far more impressive. Valle’s holes in his game are obvious. This was only meant to be a quick snapshot, not splitting the atom level calculus.

      1. I understand that it’s extra work, if you want to send me the spread sheet, i’ll update it with the formulas using a constant rather then then the % approach, becuase otherwise it’s really hard for everyone to compare someone who is proforming at the ends of the graph vs those in the middle. I’ve included my email in my post.

        Thanks,
        Aaron

        1. I wanted to echo Aaron’s comments.

          First of all bravo PP, I spend my days working with numbers for a living and creating statistics is no simple feat.

          I do agree with many others however that you probably want to re-think a multiplicative factor for the age. While not make the excel publicly available and people can try their hand at improving your system? (… rather than complaining ; )

          Keep up the good work.

      2. It would be equally easy — even easier as well as more accurate — if your corrections were additive rather than multiplicative. Also, using a stat like SecA that counts power as much as it does, there certainly is a significant development difference between age 19 and 20.

    2. If PP or anyone else could come up with that perfect statistical system, they probably would not be posting the formula on a blog. That would be pretty valuable info for most MLB clubs.

      1. It is a system with promise. By saying that the age part of the system is broken – it is – we aren’t trying to insult PP who has done some darn good work here – we are trying to make the system better.

        Just look at the clustering of pitchers with near average performance. Age adjustments should be sorting the prospects from the non-prospects. They aren’t, because the age adjustments don’t work – CAN’T WORK – for near average players.

        It doesn’t have to be perfect. But the age adjustments are one of the most important factors here. And those adjustments aren’t just imperfect, they don’t work properly at all.

  16. I think this is terrific “toy”. Not meant to project whether any one player is a legit prospect or not, but gives an interesting look at current year performance. Thanks in advance for your willingness to take suggestions and tweak the tool. I do agree that a different scale by position could be beneficial but appreciate the challenges that brings.

  17. I don’t mean to sound like I’m continuing to jump on the system, as I said, I’m really trying to improve it, but I think the PT adjustments are working wrong on players with sub zero weighted subtotals – bascially, players with low PT & sub zero weighted sub totals get better scores than players with high PT & sub zero weighted sub totals. See, e.g., Biddle versus Ruf.

    1. Its working properly.

      The playing time factor really is almost like a reliability of the sample factor. A larger sample (good or bad performance) should count more than a very small sample subject to a lot of noise.

      1. You can be a little maddening when you respond like this. The question isn’t to the sample size issue, it’s to doing a multiplicative rather than an additive correction for age relative to level. That really isn’t working well.

        1. No, in this case he was responding to a seperate objection. And while I would have handled PT differently, his response makes sense on its own terms. Unlike the age issue – which IMO needs to be fixed if the system is to work at all – this choice on the PT issue is a defensibe one that doesn’t break the system. Biddle’s unreasonably low ranking is more a function of the age issue than off the PT issue.

  18. I think the easiest fix would be to change the zero point of the weighted subtotal from “average” to … well, zero. This would indirectly fix the age issue – not entirely, but mostly; no longer will average players not be effected by the age modifiers. It will also “fix” the playing time problem (I know PP doesn’t see this as a problem, but it should still work as he intended it, without the anomalous effect of penalizing below average starting pitchers). And it will make the age factor easier to implement, as the same values can be used for below and above average players.

    PP, could you please provide a spreadsheet so we can play around with the system?

    1. No, I think I’m going to hold off.

      Feel free to create your own system and post it on your website, and I’ll definitely check it out. I outlined why I designed things the way I designed them. The parameters I outlined are simplistic, which is the point of the entire exercise. I don’t think the age of a 20 year old in Lakewood should be weighted all that differently from a 21 year old, so I’m not in favor of a big overhaul and changing it. It was meant to be a simple little toy, which it is, and it was meant to look at very basic measures, in a very basic way.

      If you want to take what I did and do something different, I already outlined the formulas above, so you’re free to do it, but I’m not making any changes. And based on all the gnashing of the teeth over it, I’m not going to update it again and post results. I’ll use it for my own benefit, but it just doesn’t belong here, I suppose. And there’s no harm in that.

      1. We all appreciate what you do here and the value that you bring. Your presence is why visit this site.

        But when you continually respond with posts like this you look like a petulant teenager. Nobody is calling you an idiot or anything, yet you act like we are. Grow some thicker skin.

        1. I wasn’t being petulant. I stated that the way I designed it was the way I was comfortable using it, and if someone else wanted to do something different, they were free to take it, tweak it, and then publish it on their own site for all to see and use if they feel it is superior to what I came up with.

          I explained my rationale for all of the adjustments I made to the raw stats. There is always going to be differing opinions on how to do anything. My point wasn’t to design an air tight system that spits out one number, and that is the number I use to determine how good a prospect is. The point of this exercise, and toy, was to look at a player’s statistical performance, compare how that rates for his age and level, adjust for small sample sizes, and spit out a result.

          I understand the critique of the age adjustment. Again, however, I outlined why I chose 1.5, 1.0, and 0.5. The groupings I used I felt were more effective, because in terms of each level, the word is “appropriate”…21 is appropriate for Lakewood. 20 is appropriate. 19 is young, 18 is young. An 18/19 year old prospect may be separated by just a few months or even days in age. I don’t think it has to be broken down in even more micro form.

          As for comparing A ball players to AA players, I also see why some might see the value in adding an adjustment there, but again, players are compared to their league, and they are adjusted for age. A 23 year old in A ball’s numbers aren’t directly comparable to a 23 year old in AA’s numbers, which is why the age factor adjusts their numbers up/down as needed. The age factor is “rough” by design, because again, I think the separation needs to occur not from 20/21 but from the 18/19 group to the 20/21 group.

          Its not perfect, but the intention wasn’t for it to be perfect. If I was going to try and design a system to spit out a number that I would place a ton of importance on (I tried that before, it failed) I’d spend a lot more time on it, and I wouldn’t just use 2 or 3 statistics, I would go in to much more detail. That said, this is meant as just a brief overview.

          And the fact that it doesn’t provide results “you expect it to produce”, like you expect Valle to be ranked above Rizzotti, isn’t the issue. Because this is a 1 year snapshot, actually its only a 2 month snapshot now, and its not an evaluation of their tools or projectability, its based on what they’ve actually done on the field. Scouts views are still incredibly important, and no one statistic should trump a detailed first person scouting report.

          I think you’ve mistaken my indifference for petulance. I stated at least 10 times that this was only meant to be a toy, not a statistically significant study. I’ve spent probably close to 1,000 hours working on an algorithm to try and evaluate minor league numbers, considering every context and variable, and the end result is “its just not possible”, at least right now. The minor leagues have way too many unknowns and variables that can’t be accurately assessed. Not to mention, prospects are encouraged to work on new things, learn new pitches, learn new skills (switch hitting, for instance) which impacts their surface and peripheral numbers. As I said, assessing a scouting report is still extremely important.

          If someone wants to take the work I did above and make it in to something different, I’ve outlined everything to give them a head start, I’m just not going to utilize it here. If people still want me to update things with the formula I used, I will do that, but its not a big deal to me. Again, this was just a toy.

      2. PP, I don’t think I speak only for myself when I say that we LIKE what you were doing with this system, we just want to make it better.

        IMO, the criticism basically was of three kinds:

        (1) Spurious criticism which you answered.
        (2) Arguably legitimate criticism which would have required a more complex system which you quite reasonably rejected on simplicity grounds.
        (3) Criticism about the age adjustment – a scaling problem – in that average players get no adjustment and near average players don’t get enough of an adjustment. Contrary to my first thoughts, this could be fixed fairly easily, and is the difference between a promising but flawed toy and a tool with some real value.

        I’m not even sure why you rejected the criticism regarding the age adjustment. Initially you misconstrued it as a complaint that there were only 3 age categories, rather than different adjustments for every age. That was probably because I didn’t explain it well. But then I clarified the issue, and a couple other people independently made the same point, and you didn’t respond further.

        Fix that one problem and this is a really great tool.

      3. It’s your blog and your system, so you are of course free to ignore any and all suggestions. With all the work you put into the system, I would think you would be open to small changes that might make the system a lot better.

        And yes, we do appreciate what you do here. If we didn’t, we wouldn’t comment. 🙂

        1. I actually lost track of the discussion half way through, because I’ve been tied up with other projects. My issue is, an age appropriate prospect should neither be punished nor boosted for being age appropriate. To use the Lakewood example, a 20/21 year old prospect at Lakewood is at the right level, so he shouldn’t get credit for being where he belongs.

          If you can condense your feedback on the age issue and email it to me, I’ll review it and see if it makes sense to fix. I just posted a long comment upstream regarding why I’m hesitant to make changes.

  19. Just playing around with one possible fix – additive age adjustments, + .1 for younger players, -.1 for older players. Hollands goes from 5% ahead of Biddle to 10% below Biddle. And that conforms with our intuitions, doesn’t it? Even disregarding scouting reports, prior performance and prior expectation, just eyeballing the numbers and figuring that they are at the same level, but separated by 3 years in age, Biddle is having the more promising season. The change doesn’t make much difference for most players (though it actually helps Rizz), but it properly separates the prospects from the non-prospects for players near the average.

    1. Finally, to address this point.

      When designing something like this (or the failed SONAR project, discussed above), you can’t say “well, the system needs to say Player X is better than Player Y, because that’s what I think to be true, so I can shift things around until I get that result”…thats bad social science. The fact that Biddle would rank lower here than a lot of guys we would rank below him in an evaluation of the top 30 in the system, for instance, maybe just means that underneath the surface, Biddle hasn’t pitched as well as some of those other guys.

      When I sit down to write my top 30, I look at everyone’s numbers, both surface numbers and peripherals, but my evaluations are based a lot on scouting reports and then supplemented with stats and then kind of a gut feel on a guy. I ranked Valle where I did heading in to 2011 even though his peripherals weren’t great last year, because I liked his improvements defensively, and I believed in his raw power, despite the warts in his game. Those warts are still there, its evident in his SecA, his BB%, and his K%, and its evident in the PPR score. That doesn’t mean I’m downgrading him as a prospect in my top 30. And it doesn’t really matter who ranks above him in the PPR chart, its just a data point.

      So in summary, when I see the results, they aren’t “counter intuitive” to me, because its a straight performance metric. I love Biddle because of his pitcher’s frame, his intelligence, his mechanics, and his future as a solid middle of the rotation guy with a chance for more. I knew he’d have struggles this year, which would mean his stat line wouldn’t be fantastic. So I didn’t try and create a system that would prop him up, even if his performance didn’t merit it. Conversely, I don’t consider Rizzotti our best offensive prospect, far from it, but that didn’t mean I’d add in qualifiers or multipliers to punish him and drive his score down, just because I know I’m not going to rank him in my top 5 or 10 this winter.

      When I try to look at statistics on any level, whether it be the minors or majors, I try to leave my pre-conceived notions behind.

      1. “well, the system needs to say Player X is better than Player Y, because that’s what I think to be true, so I can shift things around until I get that result”

        Great point PP, If every Metric told what you expected to hear there would be no point in using them.

      2. Just to be clear, PP, the issue ISN’T simply that some players are not where we would “expect” them to be. It’s that a clear flaw with the scaling of the age adjustment INHERENTLY penalizes young players who are near average. So you get the bizzare result that two players who are having essentially the same season in the same league (with a slight edge to the older player) end up with predictably bizzare ratings relative to each other because of an inherent flaw in the system.

        That is NOTHING AT ALL like Rizzotti/Valle situation. The system ignores position, defense, scouting, and prior performance (all of which is fine), so OF COURSE Rizzotti ranks higher. In that regard, it is working as designed. But in the Hollands/Biddle situation, your system is FAILING TO DO WHAT IT WAS designed to do. Hollands ranks higher because both players are near average, with a tiny edge to Hollands, and thus THE SYSTEM MAKES NO MEANINGFUL AGE ADJUSTMENT for the two players. That’s not what you want the system to do. You want the system to make distinctions on age, and for players near the league average, it isn’t doing that.

        1. And really I’d urge you to go back up thread and read the relevant comments on the issue. This is an important issue that will continue to be problematic if you design similar systems in the future. And while I don’t want to sound obnoxious, it isn’t a “reasonable minds can differ” kind of argument. Using a multiplier for the age factor breaks the system.

        2. Sorry, I couldn’t read this because it appears your caps lock is broken. Or selectively broken.

          I think maybe you’re understating the difference between Hollands’ and Biddle’s raw performance. Biddle’s walk rate is well worse than league average. That negative (and its a big negative) greatly lowers his score. The current age adjustment still helps Biddle and hurts Hollands.

          If there were no age adjustment, Hollands ends up around 6%, using the current scale, and Biddle ends up at about -4%.

          Which I think might be getting lost in translation. Which is why I asked you to email me your consolidated comments on the age issue, because I don’t have time to parse through 30 comments talking about it. If you find a way to “tweak” the age factor, and it shows Biddle is ranked ahead of Hollands, then you’ve circumvented the purpose of the tool. Because considering their peripherals, Hollands has pitched better, and the sum of Biddle’s peripherals have been below league average for the SAL.

          1. PP – if I have time, I’ll e-mail you. But it’s not like the argument was already laid out by myself and others, and it’s not that complex. In the simplest terms, an average player will have no age adjustment, and a near average player will have only a tiny age adjustment, whereas a player much above or below average will have a huge age adjustment. Is that what you intended?

            Hollands/Biddle. 5.6% above average versus 3.4% below average. That’s not THAT far apart. And, since they are not that far from the average, the age adustment is virtually nil. Is that what you intended? Ask youself – if all you knew about these guys was age/level and the seasonal stats considered by your system, would you say that Hollands was the better prospect? Of course not.

            For the system to work properly, it should generate results that make sense given the relevant inputs. Valle/Rizzotti makes perfect sense considering the inputs of the system. Hollands/Biddle does not.

  20. I’ve been motivated to try and put togeather a sheet on this doing what we’re all talking about it, i’ll post a link to it on here, using google doc share. Gimme a couple days.

    1. I pulled everything from baseball-reference.com

      I just created a spreadsheet, pasted all of the league totals in, then added the formulas for SecA and K% and had it auto calculate it.

      1. Thanks PP, another question, your Adjusted SecA and Adjusted Contact Rate, what formula are you using to calculate these columns?

        I read your post where you say:

        “Edit, I forgot the formula! Each category (SecA, K%, K/9, BB/9, HR/9) is calculated based on the league average. So, if the player is in AAA, its his numbers against the league average for all players in the International League. The same applies to all leagues.”

        I just don’t get how you’re arriving at the final number after taking the league average.

        1. Like this

          (Player’s SecA – League Average SecA) / League Average SecA

          Example: .382 SecA – .250 Sec A / .250 Sec A = .528 adjusted

          In other words, a .382 SecA, when the league average is .250, is .528 (or 52.8%) better than league average. I don’t remember the league averages for the above, I just made up those numbers in the example to show the formula.

          For K%, its the opposite, because a lower K% is better

          (League Average K% – Player’s K%) / League Average K%

          So an example

          20% (LA) – 12% (player) / 20% = .400 (or 40% better than league average)

          Does that make sense?

          For pitchers, its the same principle

          For K rate: (Player’s K/9 – League Avg K/9) / League Avg K/9
          For BB rate: (League avg BB/9 – Player’s BB/9) / League Avg BB/9
          For HR rate: (League avg HR/9 – Player’s HR/9) / League Average HR/9

  21. I don’t post often, but after reading every single one of the comments I couldn’t take it any more.

    First I must say THANKS PP!! You do great work and I love your passion for this.

    Second and only slightly less important….To all of the people who are criticizing the age appropriate thing (yes LarryM, you’re in that group), shut up!! It’s his tool. He can do with it as he wants. Some of us see it for what it was explained to be. Something that tells us how a prospect has been doing. Nothing more. I for one, appreciate it. Not just the effort, but the result.

    But for all the harping and complaining and “suggestions”, the posters have successfully ruined my trip to this site. And like usual it’s always the same crotchety folks. Please. Show the man some respect. Make your point once and move on. Your complaining isn’t going to help your cause, and it only irritates the rest of us.

    I sincerely hope that PP continues to update this so that those of us who understood him and his goals with it can continue to follow how the guys are doing. I know, I know. It might be hard for some (you know who your are) to be quiet and allow the rest of us to appreciate PP’s work. All you have to do, is not click on the link to the tool. Then you won’t have to be annoyed that the rest of us are enjoying something that you feel is inaccurate. The rest of us just get tired of the predictible single minded responses that have no interest in dialogue. If that’s how you feel you should be heard, just go to a politic blog and have at it. This supposed to be fun and light-hearted.

    1. “no interest in dialogue”

      Okay, THAT’S rich. That’s really rich. I honestly couldn’t care less what you think, but THIS needed a response. If you had the reading comprehension of a first grader, you would know that a dialog is exactly what I wanted. And many of the other commenters as well. A dialog to MAKE A GOOD TOOL BETTER.

      PP unfortunately choose to completely ignore the ONE REALLY VALID CRITICISM in the thread. It’s his site, that’s his call. But god forbid that a few of us try to get him to understand that (unlike virtually every other criticism in the thread) our point isn’t “we want a different system,” but “your system isn’t doing WHAT YOU INTEND IT TO DO because of an unintentional flaw in the scaling of the age factor.

      Now I GET that 60% of the people who visit this site don’t have the math aptitude to understand the problem – you probably included. But PP DOES have that aptitude, and it’s a little frustrating that, about half way through the thread, he just started ignoring the comments and still doesn’t get that the age part of the system isn’t working as HE intended it to work.

      1. Two things:

        * stop typing in caps, its really annoying.
        * I’ve now asked you 3 times, if you have specific feedback, send it to me in an email. I have a job, I have other priorities, I can’t sift through all of the comments and analyze the argument. If you’re genuinely interested in “fixing” this, then stop typing IN ALL CAPS LIKE THIS and send me a simple email with the simple explanation of why the scale for age doesn’t work and the fix for it.

        And read my comment above. I still think with your assertions that everything is broken because of the age, you’re still missing the big picture, which is that Biddle has not performed as well as Hollands, relative to the league average performance of pitchers in the SAL, and that his walk rate is a big issue that you can’t calculate away by jiggering the age adjustment.

        Again, if you feel so passionately about it, which it appears you do, take 3 minutes and email me. Because I don’t have more time to devote to scouring the comments right now

        1. Never thought I’d see the day. LarryM can even get you tuned up. Good luck with that.

      2. I’ve been wanting to avoid this issue, but it seems that some won’t let it die, so I’m going to chime in…

        A multiplicative age factor makes perfect sense to me. If a younger player is doing really well, of course we would want to give him extra credit for his results. If an older player is doing really well, you would want to scale that back proportionally. By focusing an argument on the “average” player and saying that all players need to have a additive age factor, you “fix” one problem by creating another – not giving enough credit to those players who are performing well compared to average.

        In the specific example that LarryM sites, Hollands v. Biddle, Biddle is so much worse when it comes to bb/9 that it makes sense he should be ranked lower. Of course he has more potential, but right now that may be all he has.

        As a quick and dirty method, this seems rather intuitive. That is not to say that I could ever come up with it, but…

        1. You make a good point here regarding using a multiple rather then an additive factor, and i’m in email discussions with PP on this topic as I’ve created an updated sheet on my own (i don’t know if he’ll use it but the first draft is done). The problem is this… how do you compensate for the say 18 year old AAA player who puts up league average numbers. Using the current multiplicative method you give him ZERO extra credit for being 18. I agree that my additive method is not fool proof but atleast it addresses both the average stat line and extreme stat lines.

          If you can create a multiplicative formula that won’t screw the average stat line i’d be all about incorporating it in what I have in my email string to PP.

          1. Oh and the only thing I could really come up with btw on this is to say anyone within 15% of the league average uses a constant, and those outside of 15% use multiplicative, (if/then statement) which i’m not crazy about.

  22. I’m not sure if maybe the core of this task got lost in the shuffle.

    If a player’s peripherals result in him being below the league average in terms of performance, no matter how you adjust his age, he should still have a negative score. Because at the root, the metric is measuring performance relative to the league average.

    If Biddle were 16 in the SAL and had the same numbers, his score would still be negative, because the sum of his parts (K/9, BB/9, HR/9) would still be below league average. The age factor is a way to either amplify positive or amplify negative if the player’s age is not “league appropriate”

    Using the Biddle/Hollands comp. If Biddle had the same stats as Hollands, I’m talking the exact peripherals, his score would be much higher than Hollands. And if Hollands was the same age as Biddle, his numbers would be much higher.

    When you have a negative score (below league average performance, based on peripherals), no other adjustment is going to turn your score positive, no matter how you handle the age. If it does that, then you’re looking at something different than what I’ve aimed to capture. This metric is an evaluation of performance relative to the league you are in. The age and playing time factors are meant to help calm down some of the statistical noise you see in really small samples. For example, if a pitcher throws 1 inning and strikes out 3 guys, his K/9 is 27.00 So yeah, that would be an issue.

    Again, I want to state it clearly, Biddle’s peripherals are worse than league average, when weighted in the manner I talked about above (K’s more valuable than BB rate which is more valuable than HR rate), and thus, he has a negative score. If you change that and somehow make his score positive, then you’ve changed the toy all together and created something new.

    1. PP, now I understand why you did what you did. I was opperating under the assumption that a AAA 18 year old with a very slightly below league average SECA should have a positive number, i didn’t realize the purpose was normalizing.

      Ok, I guess I’ll leave it at, enough said.

    2. That’s exactly what I was trying to say. PP only put it much more elegantly…as per usual.

    3. This seems confused. If the goal of the metric is simply to measure performance against the league average, then why make any adjustment at all for age? You seem to start with the assumption that it is always a bad indicator for a player to be performing worse than league average. Is this really the case. When you have an 18-year old, like Santana, performing close to league average and you list the appropriate age for that league as 20-21, isn’t it fair to say that Santana is doing awfully well? Shouldn’t the age-adjusted rating reflect that? By your metric, an 18 year old and a 23 year old who perform at the exact average for the league in Sec-A and K/9 are going to end up with the exact same adjusted rating. Does that make any sense?

      1. I understand your point. But I don’t think there is a way to do this in a simplistic, easy manner. And that was the goal of this entire mission. Because coming up with one statistical number is only ever going to tell you half the story, and that’s optimistic.

        I don’t think there is a formula that will ever take everything in to account and weigh it properly, because of the nature of minor league baseball.

  23. [James edit]

    DymondDirt, thanks for trying to step in and defend me, but I don’t want the negative discourse to go any further. Larry feels passionately about this, probably more than he should feel, considering this wasn’t meant to be a deep experiment. And I think maybe he missed the key aspect of this, which is why he got off the rails about the age adjustment. I think I’ve explained it all above.

    I don’t want any more sniping from either side, so I’m editing this and removing the harshness.

    Lets all be friends.

  24. I’m cancelling my subscription to PhuturePhillies…

    What’s that? Its free and done on a completely volunteer basis?

    Oh, nevermind then. Carry on.

  25. I think this is pretty neat. It would be really cool if somebody who really knows statistics figured out how to empirically derive all the weights and so forth that PP guessed at (not that I’m criticizing, mind you; I’d have done the same thing), because I think this could go from being a fun toy to a serious evaluative tool.

    However, one nitpick I have is that I would call it something else. It seems to me that the question that power rankings tend to answer is “who’s hot (or not) *right now*?” This system, being an aggregate of the whole season to date, doesn’t quite do that, as early April counts as much as yesterday.

    Elsewhere, some months ago, I proposed a statistically-based “objective power ranking” (actually, it would be more precise to call it “power rating” but it could be used for rankings as well) system, which basically was a weighted average of whatever stats you wanted to look at for a team or a player with the most recent games weighted the highest. Seems to me this would work for prospects as well as major leaguers, though it would of course be labor-intensive as it would require inputting the results of every single game into your database (unless you’re an evil genius who can write a web bot to automatically parse the data from all the minor league box scores every day). A player who has a big game is going to get a little bump, but not too big, because even though more recent games are weighted more highly, early games are still counted – and this is as it should be, because one game doesn’t make a player “hot”. But a player who has a great week is going to get a sizeable bump, because those six or seven games are going to represent a much more substantial percentage of the total because of the weighting.

    1. Great Idea, I nominate you! Go Forth Young Man Go Forth!

      But no… in all seriousness, like the idea, but put it into action!

      1. I’d love to, but it really is pretty labor-intensive.

        I conceived this late last season actually to try to determine who the best pitcher in the NL at that particular moment, when Hamels and Oswalt were tearing it up, while Halladay had faltered a bit, Josh Johnson was on the shelf and Lincecum looked mortal. It took me most of a couple of evenings on the couch while watching games to figure out the Excel formulas and quite a bit more time to copy and paste the line scores from bb-ref. And that was just for a few players.

        To be really useful it would have to be someone’s full-time job, or it would have to be automated somehow.

  26. I just got to reading this page. Just to summarize, I hope PP’s June 2, 2:59pm comment answers the most common question that performing relative to league average is the center point of the metric. (No bonus points for being league average, even if younger.)

    Although I understand the commenters desire to ‘bump’ younger league average players, any additive factor would be arbitrary and created ‘to make the numbers fit’. On that point I agree that multiplicative factors are better. Also, I like the idea of creating the formula based on your reasonable weighting factors and THEN plugging in the players. Trying to then re-adjust the formula for specific cases defeats the purpose of the evaluation. As you said, it is a tool and like any statistical analysis there may be outliers.

    When I read the formula I actually thought you might be giving too much weight to age, though I realize it is very important. Then I read the comments it appears age factors want to be magnified. Doing my own litmus test, I actually like your age groups and weighting factors. Personally, I think Jiwan James is a ‘young’ 22 and Galvis an ‘old’ 21 but that is why there is a mathematical formula to give me data (which I will then have to spin as needed for my argument).

    I am continually impressed at your ability to compare players across multiple levels, leagues, ages, and positions (pitchers/hitters) to rank them in one list. As I am sure you expected, it is nearly impossible, but provides us with another educated data point. I’d like to see it updated monthly or whatever, just like I enjoy gketch’s daily posting to see if my ‘eyes’ are deceiving me.

  27. I don’t know quite what I think of PP’s new system for rating how pitchers are doing, but the more I think of it, the less I like it for hitters. Part of my problem is that the sabremetricians have been telling us that strikeouts don’t matter for hitters, yet Ks are one of the two metrics used to calculate the raw performance score for hitters. I also think the 1 – 1 setoff of steals and CS in the SecA overvalues steals. Sabremetricians have looked at this and tell us that a 1 -2.5 or so is a better offset. Finally, I think for a good all-around raw performance stat for hitters, that OPS works just fine. Admittedly it’s not league or park adjusted, but PP’s measure isn’t park adjusted either. If I had my druther’s, I’d go with a simple arithmetic age adjustment to OPS to get my corrected. I don’t see a need to correct for PA, that just favors leadoff hitters. If we have a small sample size issue for anyone, I just exclude them. I’ll take the average of PP’s appropriate age range for each level and add 25 OPS points for each year younger than that average and subtract 25 OPS points for each year over the average.

    That works a lot better in comparing a guy like Valle, who is a year younger and a level higher than a guy like Mendez. On PP’s system, Mendez comes out well above Valle in both raw and corrected score, yet Mendez has a very pedestrian OPS and Valle has a very impressive OPS. Valle has been just killing the ball. Of course, I don’t know the date of the stats PP used. Mendez has been slumping, while Valle has been on a tear, but even several day old stats show Valle’s OPS above Mendez’s.

    1. Allentown, sabermetricians have never said that strikeouts don’t matter, full stop. What they have said is that in terms of evaluating run production, the frequency at which a player makes outs is what’s important, not the kind of outs. And thus a guy with a .900 OPS who strikes out 150 times a season is just as valuable as one with the same OPS who strikes out on 50 times.

      But the problem with strikeouts is that the more you have, the more damage you have to do when you make contact to be a productive hitter. This gets harder as you climb the ladder to the majors and that’s why position prospects with high strikeout rates get dinged.

      1. Okay, I can see the validity in that, especially if a guy is striking out a third of the time, but don’t we have to consider OPS also when evaluating the minor leaguers. I’m looking at Valle, who has an .892 OPS as a 20 year old at CLW. He strikes out about 20% of the time, which really isn’t all that bad. Compare him to Geancarlo Mendez, who is age 21 at Lakewood. Mendez has an OPS of .759, but only strikes out a bit under 15% of the time and steals some basis, which Valle does not. Under PP’s system, Mendez comes out way ahead of Valle, while both Valle’s performance and his age/level say to me that he is the better offensive prospect. Can we really say that Valle’s high batting average has just been luck with balls falling in, while Mendez’s so-so batting average is merely bad luck?

        1. OPS is a flawed statistic.

          * It under-weighs on base percentage, which is a bit more valuable than slugging percentage. I don’t have the study handy for this, but it was produced a number of years ago.

          * OB% is a flawed stat in that it is only a raw measure of times on base, not how you reached base. Because of batted ball luck, players who don’t walk much but have high batting averages are going to be subject to a lot of fluctuation in their batting average, unless they rarely ever strike out. See the Polanco example mentioned up thread.

          * Slugging % is flawed, because it doesn’t measure raw power. A player with a .350 batting average can have a .450 SLG%. without context, a .450 SLG percentage is decent, but that’s a .100 ISO, which equals poor raw power.

          SecA removes singles from the equation and focuses only on extra base hits. I agree that the weighting on SB v CS isn’t perfect.

          But again, everyone has gone way above and beyond what the point of this exercise was. And people are saying “well just add X to the player so his score is higher and it matches up with what we think the scouting reports should say”….that’s not the intention.

          The age weighting is tough. And I admit that there should be a better way to do it, but the method I used, normalizing for age at the level the player is at, is very quick. I’ll look at this before the next update. But I can’t say I’ll change it for sure, as it might be something to consider this winter. You can argue that a player “holding his own” at a level he is too young for should be rewarded more than the current system does, but holding your own is only so valuable in the long term.

          1. That was a great post, James.

            I, for one, am now thinking about how much emphasis I place on OPS. In my defense, I also look at ISO quite a bit to gauge a guy’s power as well as looking at that yet unnamed stat which is OBP – BA to gauge a player’s onbaseability.

            I agree that net steals can be a bit flawed. Didn’t Utley have a year that he had around 25 steals without being caught? To me, that’s more effective than a player who had 40 steals but was caught 15 times. YMMV.

          2. I concede that OPS has the flaws which you say it does. On the other hand, I don’t feel that it is justified to just throw out BA, when evaluating a player, because as the AB start to approach a half season, the likelihood that the high BA is strictly due to luck becomes less and less likely.

            I think holding your own, and really being almost league average is more than barely holding your own, at a very young age for a given league is a big plus. First of all, the organization does put a kid in that position unless they think very highly of both their ability and their makeup. I can think of a lot of Phillies who were put in that position and barely held their own or worse, and then zoomed along from there. Valle at Lakewood as a youngster is one. Stutes and Worley had pretty poor, barely hold their own, years at Reading when they were double jumped. That’s the encouraging point I tell myself as I follow Altherr’s struggles at Lakewood and to a lesser degree Santana’s less than stellar output. Both are young. Both looked very good last season. The Phillies are getting them serious AB and challenging them. Same for Biddle. Starting your first full season in A ball as a normal-aged HS signee is a plus. Repeating short season is at least a yellow flag, even if the short-season repeater is putting up better numbers than the guy that skipped to full-season at the same age.

            1. I guess it depends on how much ‘credit’ is given for poor performance. From a purely statistical standpoint I think it is difficult to know. Also, this ‘toy’ is measuring current performance, with age as a weighted factor. I do know how well it applies as a predictive ranking.

              Knowing context can correct for some outliers: e.g. Biddle just working to get his command and learning about his pitches (he will be inconsistent); Valle developing as a catcher and hitter (who cares about plate discipline yet); Galvis finally growing up physically….

              However, a younger player performing well should get extra credit, as these players are more likely to become major leaguers. Players adjust at different rates and those adjustments may only appear in some specific statistics, if at all.

Comments are closed.