The Ranking Project

First off…remember that all of this is copyrighted. What is here is intended only for commentary and education. Any other use by anyone other than me, without my express written permission, is simply a big no-no. Don’t do it.

The Ranking Project was founded on a simple premise: instead of griping about how unfair, arbitrary, and political the process of selecting teams for the NCAA tournament is, why not try to offer a solution?

Admittedly, a lot of my choices have been based on my own arbitrary point of view of how teams should be selected for a tournament. While allowing for the automatic bids of conference champions, I want the top teams in the country to compete against each other to see who is best. The trouble comes in trying to rank teams when they have few, if any, common opponents. The NCAA’s method is win-loss based, with its RPI focusing on the winning percentage of teams, their opponents, and their opponents’ opponents. It’s not an awful system, but I feel that simply looking at whether a team won or lost a game is too limiting. It doesn’t consider that a top team should beat a lower ranked team. It doesn’t reward a lower-ranked team for giving a much better team a good game of it.

Win Quality is my primary bias with this project. I want to see if teams are living up to (or down to) expectations. All things being equal, if a team loses well, or wins poorly, then that should be represented by their positioning relative to other teams. That being the case, we start with the basic WQ (win quality) formulae:

For game winners:

And this for losers:

OT = an overtime game
wq = win-quality value
P = the point difference in a game [always a positive number];
S = the “Sweet Spot”, a point value considered to be an expected point difference between teams that are comparable matched.)

The philosophy is this…both teams start with a point value of 1.000, and that is adjusted up or down based on a logarithm of the point ratio. A loga-what? Basically it’s a sliding scale that effects the game value less and less as the game becomes a greater blowout. The reason winners and losers get two formulas is my determination that there should be an ideal point margin for a game that is neither “too close” (i.e. a function of luck), or “too one-sided.” This will give the winner a value of 1.1 and the loser a value of 0.9. (For the purposes of these articles, I chose S=7, which factors as either three 2-pt field goals, or two 3-pt goals.)

I’d also add that for overtime games the winner formula stays in effect, but the losers value simply becomes 1.0….an almost-win because the teams did tie at the end of regulation. I think they merit this reward for playing a team even (another personal bias), and shouldn’t be penalized simply because ties aren’t allowed.

Using the WQ formula, each team in each game now has a value. I then averaged them up for every team, and sorted the list. This gave me my first rough ranking that was more-or-less equivalent to a straight win-loss list.

NCAA (win %)            WQ (ave)
 1    North Carolina      North Carolina
 2    Bowling Green       Duke
 3    Ohio St.            LSU
 4    LSU                 Bowling Green
 5    Chattanooga         Ohio St.
 6    Hartford            UConn
 7    Duke                Tennessee
 8    Oklahoma            Chattanooga
 9    UConn               Maryland
10    Tennessee           Hartford
11    Maryland            Rutgers
12    Sacred Heart        Sacred Heart
13    La. Tech            Baylor
14    Rutgers             La. Tech
15    Indiana St.         Stanford
16    BYU                 Indiana St.
17    Tulsa               Oklahoma
18    Liberty             DePaul
19    DePaul              Utah
20    Baylor              Purdue

Now it gets interesting.

The next step is to create an adjusted WQ (WQadj) that evaluates a team’s performance against an opponent versus expectations (we only consider DIV I vs DIV I games from now on–though we considered ALL games in computing the WQ–as there’s no mechanism in place to evaluated relative strength of schools in inter-divisional matchups).

First we figure out what the point spread for a game should be based on the difference of each of the two team’s average WQs.

From there we figure out the point spread,

and then calculate a normalizing factor that takes into account a fudge factor of the target score before we’d start worrying about deviations from what was expected (I chose a couple of 3-pt baskets, either way).

t = our ideal point target
WQA = is the total of WQ values for a team divided by the number of games
J = is an arbitrary adjustment factor that help to translate WQ values into points
V = the point “fudge” factor. I made it equal to the point “sweet spot” (S)
Z = the upper and lower limits of the fudged point spread
n = our normalized point difference (0 for being in the “fudge” zone, otherwise the number of points difference from that zone)

We then take the n value, divide it by another arbitrary ‘Y’ factor (divined from experimentation), and add it to the team’s WQave. This gives us the WQadj that we’ll use from here on out:

Now it’s just a simple matter of averaging out the WQadj for all of a team’s games, as well as for its opponents. This gives us two values, let’s call them TeamWQ and OppWQ. For our final tournament selection listing, all we need to do to weight each factor, total them, and then sort the result. But what weight? It really depends on how much you want to factor in the strength of a team’s opponents’ opponents.

From what has been pieced together by other analysts, the NCAA RPI places approximately 25% of the weight on a team’s win percentage, 50% on its opponents’ win percentage, and 25% on the opponents’ opponents win percentage. While this is great for figuring out strength of schedule, it seems to fall short on giving credit to a team for its own accomplishments. Even so, the strength of schedule can’t be under-emphasized, either. If you play in a tough or easy conference, then your cumulative WQadj should reflect that, as well.

Using as a basis the teams that made it to the NCAA tournament, I played with different weighting factors and decided that 40% TeamWQ and 60% OppWQ provided a good balance of performance against competition. It doesn’t mimic the NCAA selection committee’s choices or ranks, but does still match up pretty well.

Of course, once we have this list, there is still more work to do. The selection committee still plays games with seeding. Here’s how I think it should work: from the generated list (including the conference champions, as now) we split the list into tiers. Tier A has the top four teams; Tier B, the next four; Tier C, the next eight; Tier D, the next sixteen; and Tier E, the final thirty-two. Within each tier, we place each team into their brackets based on random selection. E.g. while a Tier A team will face a Tier E team in the first round, the Tier E team could be ranked as highly as 33 or as low as 64. I think this method preserves the feeling that higher-ranked teams should have some earned advantage in a tournament while also taking away the sort of coronation run that current seeding methods give to the top-ranked team. And if a team happens to fall into a bracket where their school is a host? Great. Let it happen. Why must everything always be so PC-neutral all the time? If the Fates say a school gets to play at home, who are we to argue?

Now, after this, we choose WNIT teams. We use the same method, but I’d propose one more restriction: no conference having 3 (or a fourth of their conference members, whichever is greater) or more teams in the NCAA tournament can send a team to the WNIT. Lower-ranked teams in power conferences that don’t make the dance? You’re out of luck.

So…what do the results look like? Compared to the NCAA tournament list, I got 30 out of 32, and 47 out of 64. If you factor in the 12 tourney champs that were ranked lower than 64, that means I filled out 59 teams out of the NCAA’s field of 64. Not too shabby. Still, I’d have been including several schools that didn’t make the NCAA this year [cough]Western Kentucky[cough]. Here’s a sampling based on Tiers:

Tier A (1-seeds)
NCAA                        WQ
    North Carolina              North Carolina
    Duke                        Duke
    LSU                         Tennessee
    Ohio State                  LSU

Tier B (2-seeds)
NCAA                        WQ
    Tennessee                   UConn
    Maryland                    Maryland
    UConn                       Ohio State
    Oklahoma                    Oklahoma

Tier C (3- and 4-seeds)
NCAA                        WQ
    Rutgers                     Baylor
    Baylor                      Rutgers
    Georgia                     Georgia
    Stanford                    Stanford
    Purdue                      DePaul
    Arizona State               NC State
    Michigan State              Arizona State
    DePaul                      Old Dominion

Tier D (5-, 6-, 7-, 8-seeds)
NCAA                        WQ
    UCLA                        Texas A&M
    Utah                        Western Kentucky (x)
    Kentucky                    Boston College
    NC State                    Virginia Tech
    Texas A&M                   Purdue
    Florida                     Michigan State
    Temple                      Utah
    Florida State               Vanderbilt
    George Washington           UCLA
    St. John's                  Chattanooga
    Virginia Tech               Bowling Green
    BYU                         Florida State
    Vanderbilt                  St. John's
    Boston College              Temple
    USC (x)                     Virginia (x)
    Minnesota (x)               La. Tech

Tier E (9-, 10-, 11-, 12-, 13-, 14-, 15-, and 16-seeds)
NCAA                        WQ
    Louisville                  Indiana State (x)
    Notre Dame                  Xavier (x)
    South Florida               George Washington
    Washington (x)              Kentucky
    Old Dominion                Kansas State (x)
    California (x)              Hartford
    Missouri (x)                South Florida
    Iowa                        BYU
    TCU (x)                     New Mexico
    New Mexico                  Texas Tech (x)
    Hartford                    James Madison (x)
    La. Tech                    Notre Dame
    Bowling Green               Tulsa
    Middle Tennessee            Florida
    Chattanooga                 Louisville
    Tulsa                       Miami (Fla) (x)
    Missouri State              Middle Tennessee
    Stephen F. Austin (x)       Stony Brook (x)
    Wis.-Milwaukee              Eastern Michigan (x)
    Liberty (x)                 Iowa
    Dartmouth (x)               SE Missouri State *(84)
    Northern Arizona            Sacred Heart *(89)
    Marist (x)                  Missouri State *(91)
    SE Missouri State           Northern Arizona *(100)
    Army                        Coppin State *(107)
    Sacred Heart                Wis.-Milwaulkee *(110)
    Coppin State                Oakland *(131)
    Pepperdine                  Army *(141)
    UC Riverside                Florida Atlantic *(156)
    Oakland                     Pepperdine *(173)
    Southern                    Southern *(180)
    Florida Atlantic            UC Riverside *(202)

'x' is a team not on the other list
'*' is a tourney champion that wouldn't have
    otherwise made the list (number in parentheses
    is WQ list rank)

As the ACC nabbed two more spots, and the Pac-10 lost three, there is some cause for further study (though the results might be accurate, nonetheless). I think any necessary tweak needs to come with the weighting. Again, it’s difficult to balance a team’s own accomplishments against the quality of their opponents. Do the power conferences gain too much of an advantage by factoring in the WQadj of the opponents? It also raises an ever-present question…is a marginal team in a power conference better than an above-average team in an average conference due solely to the competition they face for the half season spent in conference play?

At any rate…for now, I’m finished working on this project. It’s been an interesting exercise. I came in with some items I wanted to look at, but in the end I ended up with a modified version of the NCAA’s RPI…simply because when you try to look at things like conference vs non-conference play, time of year, etc., there are just too many variables and too many exceptions to allow for reasonably repeatable results.

I do have to grant that, based on this project, the NCAA selection committee mostly got it right. However, this project also gives hints as to where the committee opted for diplomacy over fairness. I’d like to think that my method is somewhat better (it’s certainly unaffected by outside social factors), and would allow some of the non-power conference teams to get a fairer look in the post-season.

Hope you enjoyed the trip.

Leave a Reply

%d bloggers like this: