Saturday, January 30, 2016

Put your seat belts on, this blog entry is going to be a long one. Hopefully there's some useful info in it. I just got done spending an hour and a half talking with Trevor Kronemann. Let me start off by saying I think this guy's heart is in the right place. I really do believe he's trying to do what's best, but it's complicated, really complicated.

Rating tennis players against one another is incredibly difficult. We have a "point" system in place (PPR - Points Per Round), and I think the SCTA is trying to move away from it. I think no matter what system you do have in place, people will be able to game it and exploit it in any way that they can for their own gain -- as they should! The first step of any rating system is to try to understand how it works. The point system is relatively simple - play tournaments, every tournament has a "level" associated with it, accumulate points based on how far you get in these tournaments, boom - you have a ranking. Is that the best way to do it? Is that the most accurate way to do it? Not so sure... People can game the system by traveling to locations where there are a bunch of weaker players, accumulate a bunch of points, and then be seeded and avoid the tougher players early in a bigger tournament, thereby gaining more points. It's self fulfilling in a way.

UTR (Universal Tennis Ranking - http://www.universaltennis.com/) is another rating system. I won't go into all the details about how this rating system works (mostly because I don't completely know myself!), but basically, it ranks you based on the level of player you faced, and not necessarily how far you got in a tournament in a draw. So you could travel to some remote area, and play a bunch of weak players and beat them, or you could stay closer to home, play a few stronger players, and get a better UTR ranking. I believe under the UTR ranking system, when you play a tough player, you can still be well served to get games off that player. In order to face the tougher players, you still have to play tournaments, and you may still need to get far, but getting knocked out in the first round by the #1 seed may not be so bad (especially if you were able to keep it close).

One of the goals of the UTR is to put more emphasis on how you play against people around the same level vs. becoming one of those crazy parents who drive all over the place to accumulate points (and crazy parents - you know who you are!). The more I talked with Trevor, the more it seemed like this was the direction the SCTA is moving in. Like anything in life, change is tough. Personally, I wish he'd just impose the system and be done with it. He has a more tactful approach and wants to bring people along gently -- again, I'd prefer he just rip the bandaid off :-).

Okay, so back to what happened in this tournament (Fullerton). He didn't actually Auto Seed, he manually entered the seeds. "Aha!" you say. Well, not quite so fast. What he did is he went into some weird auto-generated list of rankings, and based on that he made the seeds (and then he or director proceeded to manually enter them into the TDM). The ranking list that he used was not called "USTA ratings list" specifically, but he explained to me that that was yet another ranking list that was basically like NTRP (adult rating list) for juniors. This is yet another ranking list, and I did see it with my own eyes (he brought a laptop to show me). I can believe in his mind how he thought this was "Auto Seed". I'm not sure exactly how the transaction happened, but I can believe that he looked at this list and then told the director the seeds, and then the director entered the seeds manually. And I can believe in his mind he considered that "Auto Seed." The list was "auto"-matically generated, and he used that list, whether he hit the "Auto Seed" button in the TDM is an irrelevant detail.

I think there may be some weird political thing going on about what rating system to use - because UTR was not an invention of the USTA, people may be reluctant to use it. But it seems like as far as Trevor is concerned, UTR all the way (and no one has told him to NOT use it). The sooner you jump on the UTR bandwagon, the better.

One of the side effects of using UTR is that people may be less willing to play the lower level tournaments; why bother if no high rated UTR players are in it. The reality is that the point system is probably not going anywhere anytime soon. That's still going to be a factor for some things inside of the USTA (especially at the National level).  I could easily see them doing something like using the PPR ranking for entry into the bigger tournaments (with limited draw sizes) and UTR for seeding.

Another thing Trevor talked about was possibly having smaller draw sizes for the larger tournaments. And possibly even having qualifying draws for some of the spots in the main draw. Some tournament directors believe anyone that signs up for a tournament should be given a chance. I'm not sure how I feel about that. Today, my son play a fairly non-competitive match. He beat the kid 6-0, 6-0. So, basically, I had to wake up early, get my kid ready, drive an hour and a half up from San Diego, sit through a half hour match and be done for the day. As Trevor was talking me through that, I did think that a feed in draw would not be a bad idea. I think his ultimate goal is to have competitive matches at as many levels as he can. Obviously, nothing is perfect, but if he can tier the tournaments and have alternatives for people at the various levels, it may be able to move in that direction.

Trevor also talked about communicating how draws/seeds were going to be made up and posting that on the SCTA website. I think that would be helpful -- and again, I think he's trying, so that's good. And I'd even love to see directors posting on the front page of their tournament website, what method they are using to create the seeds. It's annoying that it isn't the same for every tournament and that someone doesn't just mandate the method, but I guess these things take time...

What would be really interesting is if they could hire a mathematician, and after each tournament, see how each of the different rating systems held up. If there is one rating system that is continually outperforming the others, then that should probably be the one they use (maybe they already did this with UTR being the winner??). This is a mathematicians dream job.

We talked about "All Factors Method" for rating - and I think we just have a fundamental disagreement here (it's okay for people to disagree). I don't like the concept of the all factors method because it introduces subjectivity into the process. Trevor likes the "All Factors Method" because it allows him to take things into account that no single rating system can. In my opinion he is probably trying to make too many people happy. I think that in the end, people want to know exactly where their seeding came from, more than they like the idea of thinking someone is manipulating the system. It's not worth it to get that extra 2% accuracy (and it's not even clear if it will).

I'm still a little bit bothered that it took so much effort to have him take a second look at everything, but he's overwhelmed and new to the job, so i'll cut him a little slack... this time :-) Like I said, I think he's trying to do the right thing - I really believe that. And I can tell you, I would not want that job.

The response from other parents to this situation has been mostly positive. But there are all types of people. I got this text message from one of the parents:

Phil, so I heard you were messing around with the doubles 12 seeds, thx, it messed up the 14 as well

Uh... Seriously? I tried to address something I saw wrong in the seeding of the 12s doubles and you're upset about something in a totally different age group -- and you're blaming me for that? What???

But I also had some great feedback as well.

Hi Phil,

When they changed the draw I didn't understand why. My son (WITHHELD) & his partner (WITHHELD) had a fairly easy match and now that the draw was re-done they face a much tougher opponent. After a friend sent me the link to the story you wrote I now understand why the draw was re-done and I stand by the decision 100%. I explained to the boys the seeding were completely off and the error was corrected.. It's about time someone stood up to this very questionable organization. Thank you very much for sending the wake-up call and turning the light bulb on.

Good luck to Zachary this weekend.

Good luck to everyone in the tournament, and everyone, do your reverse rain dance!

4 comments:

  1. Phil, just a point of clarification on the difference between USTA rankings and UTR ratings: one (USTA PPR) is an ordered list of players based on their point totals in relation to each other; the other (UTR) is simply a rating based on a player's head-to-head results regardless of the level of the tournament. USTA's PPR makes it much simpler to seed because you look at the list of selected players, put them in order of ranking (there is only one #1, one #2, etc.) then seed according to draw size. With UTR, you could have several selected players with the same rating; therefore, seeding becomes much trickier if you simply use UTR alone. I suspect what we'll all start to see is some combination of USTA rankings and UTR ratings to select and seed players.

    ReplyDelete
  2. These guys did a study with a bunch of different tournaments over the past year:
    http://www.tennisrecruiting.net/article.asp?id=2270

    It looks like UTR beats out the USTA PPR system every time. Tennis Recruiting's head-to-head system also does very well by comparison, but I don't know if they care that tournament directors use it for seeding.

    ReplyDelete
  3. UTR is the future. The Mid-West region has been using a combination of UTR & MW rankings for some time for selections & seedings. It works well.

    ReplyDelete
  4. EXAMPLE of a 64 draw in Mid-West used for all high level tourneys

    PLAYER SELECTION AND SEEDING
    This tournament will use a combination of USTA/Midwest Points Per Round and Universal Tennis Rating (UTR) for player selection and seeding. Players will be selected for the tournament in the following order:
    1 - 52 Players based on the most recent USTA/Midwest Standings List in the age division of the tournament
    2 - 12 Players based on UTR

    Players will be seeded for the tournament in the following order:
    1 - Top 8 seeds based on UTR
    2- Seeds 9-16 based on USTA/Midwest Standings List

    ReplyDelete