- #1 NOVA, #2 UVA, #8 AUB, #13 ARI, #19 WVU, and #23 NEV lose "home games" to unranked teams
- #10 KU, #15 TENN, #20 MICH, and #25 MIA lose road games to unranked teams
- #3 PUR, #17 OKLA, and #24 UK lose both games played for the week (and to PUR's credit, both of their losses came against Top-14 teams).
- #5 XAV gets two road wins, the first to some of the best officiating I've seen in quite some time and the second to the absolute worst late-game officiating I've seen in quite some time.
- Most importantly, the Selection Committee released their annual in-season mock tournament field with the Top 16 teams and their seedings on Feb 11, exactly four weeks before the real deal on Selection Sunday Mar 11. While I will refrain from issuing my full thoughts on the contents of this release, it should suffice enough to say: "What a way to end this week!"
Hypothesis
As the title suggests, the model will focus upon "experienced talent," a characteristic implied to correlate to stable (or chalky) tournaments. Up until now, PPB has used the phrase "experienced talent" in a conceptual context. The Experienced Talent Model will attempt to give a statistical and mathematical flavor to the concept.
Data & Methodology
The first task is to find a data source that measures talent. For this model, I will be using the high school senior class rankings, provided here, to quantify 'talent'. This data source -- RSCI Hoops* -- has been used before on the PPB blog discussing experienced talent in college basketball, and it can be found in this article. Each year, RSCI compiles the Top 100 player rankings of various high school basketball scouting services and produces a composite Top 100. The data set being used for this model is the "Final Rankings" for each year. Each rank is worth a specific value inversely related to its rank: The top-ranked player is worth 100 'talent' points, the second-ranked player is worth '99 talent points', and so forth until the 100th-ranked player becomes valued at '1 talent point'. When two players are tied at the same rank, they receive the same value for that rank, and the next player in the list will be two-talent values away from the tied players. For example, if two players are tied for 25th in the rankings, they both will receive 76 talent points, there will be no player ranked 26th, and the next player in the list will be ranked 27th overall and receive 74 talent points.
The next step is to transform this data set into a value scale that quantifies the experienced gained by the talent. For this model, a multiplicative relationship will be used to quantify experienced gained by the talent. In other words, the top-ranked player is worth 100 ET points (1x100) as a freshman, 200 ET points (2x100) as a sophomore, 300 ET points (3x100) as a junior, and 400 ET points (4x100) as a senior. Thus, each talented player's value (determined by their ranked) is multiplied by an experience factor (determined by the collegiate class eligibility) to produce a value that will represent 'Experienced Talent' (ET).
The final step involves organization of the data. For more practical uses, the data has been organized according to team affiliation of the 'experienced talent'. For example, if a specific team in a specific year has the 32nd-ranked player as a senior, the 54th-ranked player as a junior, the 88th-ranked player as a sophomore, and the 62nd-ranked player as a freshman, then the team's ET value for that given year is 482 points.
- 32nd-ranked player is worth 69 talent points, and as a senior, is worth 276 ET points (4x69).
- 54th-ranked player is worth 47 talent points, and as a junior, is worth 141 ET points (3x47).
- 88th-ranked player is worth 13 talent points, and as a sophomore, is worth 26 ET points (2x13).
- 62nd-ranked player is worth 39 talent points, and as a freshman, is worth 39 ET points (1x39).
- 276 + 141 + 26 + 39 = 482 ET Points.
Assumptions
Before we dive into the all-important results and analysis section of the article, it is necessary that the assumptions built into the model be clearly stated.
- Inter-year equality: This is the assumption most easily blown apart by reality. Each talent at a certain rank is given the exact same talent-value, even though two players at the same rank may be further apart on the absolute talent scale. This truth can be demonstrated simply by looking at the top-ranked player in each class. In the last six years (senior classes of 2012 to 2017), the players are Marvin Bagley III (DUKE), Josh Jackson (KU), Ben Simmons (LSU), Jahlil Okafor (DUKE), Andrew Wiggins (KU), and Shabazz Muhammad/Nerlens Noel (UCLA/UK). Without going into fully detailed player analysis for each of these players, it is clear that each of these players brought a different skill/ability set to the game, but the model grades each of these players at 100 talent points for the sake of simplicity.
- Linear growth rates: Essentially, this model assumes that players grow/develop at a linear rate, and for that matter, at the same rate. This assumption, which is not entirely true in reality, can happen in specific situations and can happen over the course of a player's career. Some players are "impact freshmen" and some players regress in their freshman season. Sometimes, a player has a sophomore explosion and levels out over the final two years, and other times, a player has a sophomore slump and catches up over the final two years. The amount of player transfers (switching team affiliation) that happen in the rankings is very telling of variable growth rates. I would assume that the model would be far more accurate if talent values were recalculated each year (according to on-court production), but for the sake of simplicity, we will accept linear growth/development rates for players.
- Scouting accuracy: This is the most obvious assumption, but it still must be stated. The model assumes the scouts are 100% accurate, that the top-ranked player is indeed the best player in the class. Likewise, it assumes that the 46th-ranked player is not as good as the 45 players listed ahead of him but still better than the 54 players ranked after him. For the most part, this assumption may be the closest to reality because the RSCI rankings is a composite ranking of 5-7 different scouting services. Where one scouting service may have an outlier ranking, the other services will bring that player's rank closer to its actual rank.
- Data scale: For those of my readers with a statistical background, you will know what I mean by data scale. The data being used is interval-scale, and for those without a statistics background, the best example of interval-scale data is temperature data. Interval-scale data indicates rank and distance from an arbitrary zero-value in unit intervals. The player with 88 talent points is not twice as good as the player with 44 talent points, just as 88-degrees Fahrenheit is not twice as hot as 44-degrees Fahrenheit. The interval scale only tells you that the 88th-ranked player is better than the 44th-ranked player, just as 88-degrees is warmer than 44-degrees. Even the experience-factor demonstrates interval-scale qualities. A junior is not three times as good as a freshman even though the experience-based multipliers (1x for a freshman and 3x for a junior) would suggest this ratio. As a model with interval-scale data, it does limit the statistical methods which can be applied, but it does allow us to measure a characteristic for which we typically would not have any data whatsoever.
- Eligibility Bias: This particular assumption of the model is insignificant when it pertains to the big picture (tournament quality), but it can matter when it comes to individual teams. The model counts experience based on eligibility (if a player used up a specific year of college eligibility), not on participation (if a player actually played on the team during tournament time). For example, if a player was injured in the second-half of the season (UK's Alex Poythress) or was suspended for team violations (DUKE's Rasheed Sulaimon), they are still counted towards that team's total because they used up the year of college eligibility even though they did not contribute to their team's post-season performance. To make this model more applicable to predicting individual team tournament performance (or to make it more reliable as a tournament tool), the eligibility bias must be corrected in the data.
Results and Analysis
Per usual, I'll start with a table and then go into my long-winded explanations as to what they mean.
First things first:
The first part of the table is the "Total" row. This is the total 'Experienced Talent' value for all teams in the given year. The thing that should jump off the page is the 2018 Total being 3,886 and 5,452 ET points lower than 2017's and 2016's totals. With less "experienced talent" in 2018, we assume 2018 to be crazier (more prone to upsets) than its 2017 or 2016 counterparts. For reference, 2017 had 10 upsets (4-4-2-0-0-0) and a Mad-o-Meter® rating of 11.13% and 2016 had 11 upsets (8-2-0-1-0-0) and a Mad-o-Meter® rating of 18.35%.
The second part of the table is the "Tourney" row. This is the total 'Experienced Talent' value for all teams selected to the NCAA tournament in the given year. You will notice two things about this line. First, the 2016 tournament total is 1108 ET points higher than its 2017 counterpart, which is unusual since 2016 was slightly wilder than 2017.
Second, the value for 2018 has been determined as a result of the release of the bracket. Strangely enough, the value is only 1,254 points away from the 2017 tournament total. Since 2017 fell from 2016 and 2017 was more stable than 2016, this would lead us to believe that 2018 should be chalkier than 2017. I don't believe this to be the case because this pattern is the outlier to all of the others in the table. In fact, the 2018 tournament total could actually have been much lower if teams like SDST (411 ET), UCLA (413 ET), SYR (355 ET), OKLA (217 ET), and TEX (802 ET) were among the last teams invited to the tournament (and some of these teams "questionably invited" in my opinion).
Second, the value for 2018 has been determined as a result of the release of the bracket. Strangely enough, the value is only 1,254 points away from the 2017 tournament total. Since 2017 fell from 2016 and 2017 was more stable than 2016, this would lead us to believe that 2018 should be chalkier than 2017. I don't believe this to be the case because this pattern is the outlier to all of the others in the table. In fact, the 2018 tournament total could actually have been much lower if teams like SDST (411 ET), UCLA (413 ET), SYR (355 ET), OKLA (217 ET), and TEX (802 ET) were among the last teams invited to the tournament (and some of these teams "questionably invited" in my opinion).
The third part of the table (the final three rows) show the average and the standard deviation of ET values for all tournament teams for the given year. I'm not sure how much individual predicative value these statistics have due to our range of ET values not having a maximum bound, so I combined them into the TQRatio (Team Quality Ratio), which is the average divided by the standard deviation. Since the average (base) tells roughly the same story as the tourney total, the
standard deviation (parity) and the TQRatio should help to provide a
fuller picture.
Interpreting the Results
With the results fully detailed, let's see if we can't uncover some insights. The most confusing result of this model is 2016's Total being almost 1566 ET points higher than 2017's total, which means 2016 should have been calmer than 2017. However, the tournament sanity/insanity ratings for 2016 and 2017 referenced above, 2017 was actually calmer than 2016. Why did this result happen? The easiest explanation can be found in the assumptions section. The combination of the first three assumptions --inter-year equality, linear growth rates -- may be to blame. Without going into every detail, I can see many instances in the data that disprove these assumptions. For example, UNC's Brice Johnson was awarded the KenPom MVP for most efficient player in the 2016 season. However, in the 2012 rankings, Brice Johnson is ranked 40th overall, making him worth 61 talent points and 244 ET points as a senior, which are respectable numbers. Unfortunately, the linear growth rate assumption rates 22 players higher than him as a college senior because they were ranked higher as a high school senior, meaning 22 players have their ET values over-inflated in 2016. To disprove the inter-year equality assumption, the 40th-ranked player in the 2013 class is Conner Frankamp, currently a shooting guard at WICH and providing the same 244 ET points to 2018 WICH that Brice Johnson added to 2016 UNC. It is also possible to point to the 100% scouting accuracy assumption to explain the discrepancy, but I believe you get the idea.
Another insight we can gain is the appropriate application of this model. How are we supposed to use it for tournament predictions? This model was designed to assess tournament quality -- Chalkiness or Chaos -- and it should not be used for anything else. Here is the reason. Two of the three largest team ET values (in the team-organize ET table in the Data and Methodology section) belong to 2016 UNC and 2017 UNC. The sheer size of these two values compared to the rest of the table suggests these two teams should have been playing in the National Title game, if not altogether winning it. Using this model to make this type of prediction would not have worked in either year. In 2016, UNC fell to NOVA despite having the largest team ET value for 2016. In 2017, DUKE would have been predicted to defeat UNC in the title game, yet UNC won it all and DUKE won one game. To see the error in the method for 2018, UNC has a higher ET value in this year compared to MIST, yet the two teams have played against each other already in which MIST dominated the game start to finish. I would not be surprised if this model could be tweaked in order to make predictions of this kind, but in its current state, I would only apply this model to predicting overall tournament quality - chalk or chaos.
The most intriguing part of the table is how confusing it is. The table doesn't really explain (or predict) anything since it seems to suggest 2016 should have been the most stable tournament of the three. The only statistic that properly correlates with tournament quality values (number of upsets and M-o-M rating) is the TQRatio, and it correlates stronger with the number of upsets than it does with the M-o-M rating. Since the size of the standard deviation can increase at the same rate as the average, combining these two statistical values into this ratio unveils which statistic is growing faster. Naturally, I would assume that lower values for the TQRatio should mean less parity (chalkier tournaments) and that higher values for TQRatio should mean more parity (crazier tournaments).
The most intriguing part of the table is how confusing it is. The table doesn't really explain (or predict) anything since it seems to suggest 2016 should have been the most stable tournament of the three. The only statistic that properly correlates with tournament quality values (number of upsets and M-o-M rating) is the TQRatio, and it correlates stronger with the number of upsets than it does with the M-o-M rating. Since the size of the standard deviation can increase at the same rate as the average, combining these two statistical values into this ratio unveils which statistic is growing faster. Naturally, I would assume that lower values for the TQRatio should mean less parity (chalkier tournaments) and that higher values for TQRatio should mean more parity (crazier tournaments).
Conclusion
I hope this article shed some more light onto the 2018 tournament because if our understanding of 2018 is better, then our predictions will be better. If the model itself or the results/analysis were overwhelming or confusing, then try to read the article as a template for model-building. It is important to know what you want to discover (hypothesis), how you intend to quantify your objective (data and methodology), the limitations of your model (assumptions), and the overall applicability of your model (results and analysis). By following the template in this article, it will help you gain a complete inside-out understanding of your subject matter as you conduct your own bracket research studies. As always, I appreciate you reading my work, and the next article -- The March Quality Curve Analysis -- will be out on Feb 26.
Data Sources:
*RSCI: https://sites.google.com/site/rscihoops/home
Since IRL took over pretty quickly last season (seems like it was on Wednesday of Bracket Week), I was unable to get a lot of the updates to models/articles when I promised them. This article has been updated to reflect the changes I would have made at the time they would have been made. Any changes to the text in the main article as a result of this update will be highlighted in red. I will discuss them below and one possible alteration to this model.
The first line of changes reflects deals with completed data sets. At the original writing, only the data sets from 2012-2017 were complete, but some of 2016's Totals required data from the 2011 data set (which is now complete). As a result, an estimator was used instead of the actual values. With the completion of the 2011 data set, the update now uses completely accurate data for all three ET-years being studied (2016-18). The second line of changes reflects changes to the 2018 totals. At the original writing, the 2018 tournament field had not been set, so the 2018 totals were incomplete, and an explanation was given as to why an estimator was not used (like it was for 2016's totals). Also, the original article did not create the TQRatio, so a definition in the update had to be included.
The third line of changes deal exclusively with the TQRatio, and I want to talk a little bit more about how this statistic (being the only statistic that correlates with tournament quality ratings) led me to hypothesize an alternate form of this model. I listed five teams that I thought probably didn't deserve to be in the NCAA tournament field, and I will admit that I may not be able to find five replacements for them (OKST, USC, and DAME are three that come to mind). Nonetheless, these teams made the field, they made the field later than other teams (10-12 seeds), and they possessed higher than average ET values (OKLA being the lone exception with an ET below the field average of 330.02). If these above-average ET-teams are being seeded on the 10-12 seed lines, then it stands to reason that below-average ET-teams are being seeded above them. Thus, what if the ET Model was expanded to create a seed-based variant? Much like the Seed Curve is implemented to readjust the Quality Curve in order to gain insights into specific areas of the field, what if a seed-adjusted ET Model could predict number of upsets, M-o-M ratings, or even which seed lines will over-perform/under-perform using the standard ST model as a baseline? It could be something that I look into for future seasons, but it is another theory/insight of mine and I thought I would share it with you.
As always, thanks for taking time out of your day to read my work. There should be one more update article before the new season of Project Perfect Bracket kicks off on Nov 1. I look forward seeing you then.
Post-Season Update
Since IRL took over pretty quickly last season (seems like it was on Wednesday of Bracket Week), I was unable to get a lot of the updates to models/articles when I promised them. This article has been updated to reflect the changes I would have made at the time they would have been made. Any changes to the text in the main article as a result of this update will be highlighted in red. I will discuss them below and one possible alteration to this model.
The first line of changes reflects deals with completed data sets. At the original writing, only the data sets from 2012-2017 were complete, but some of 2016's Totals required data from the 2011 data set (which is now complete). As a result, an estimator was used instead of the actual values. With the completion of the 2011 data set, the update now uses completely accurate data for all three ET-years being studied (2016-18). The second line of changes reflects changes to the 2018 totals. At the original writing, the 2018 tournament field had not been set, so the 2018 totals were incomplete, and an explanation was given as to why an estimator was not used (like it was for 2016's totals). Also, the original article did not create the TQRatio, so a definition in the update had to be included.
The third line of changes deal exclusively with the TQRatio, and I want to talk a little bit more about how this statistic (being the only statistic that correlates with tournament quality ratings) led me to hypothesize an alternate form of this model. I listed five teams that I thought probably didn't deserve to be in the NCAA tournament field, and I will admit that I may not be able to find five replacements for them (OKST, USC, and DAME are three that come to mind). Nonetheless, these teams made the field, they made the field later than other teams (10-12 seeds), and they possessed higher than average ET values (OKLA being the lone exception with an ET below the field average of 330.02). If these above-average ET-teams are being seeded on the 10-12 seed lines, then it stands to reason that below-average ET-teams are being seeded above them. Thus, what if the ET Model was expanded to create a seed-based variant? Much like the Seed Curve is implemented to readjust the Quality Curve in order to gain insights into specific areas of the field, what if a seed-adjusted ET Model could predict number of upsets, M-o-M ratings, or even which seed lines will over-perform/under-perform using the standard ST model as a baseline? It could be something that I look into for future seasons, but it is another theory/insight of mine and I thought I would share it with you.
As always, thanks for taking time out of your day to read my work. There should be one more update article before the new season of Project Perfect Bracket kicks off on Nov 1. I look forward seeing you then.
Thank you for continuing Pete Tiernan's work! Looking forward to reading this over the next month.
ReplyDeleteYou are very welcome! Pete was one of the biggest influences on my March Madness, even though I don't think I deserve to be mentioned in the same sentence as him. It's always nice to hear from people who thought the same about him. As always, thanks for reading.
Delete