Tuesday, November 20, 2012

The Problem of Evaluating Coaches

Is Bob Bowman a good coach? Probably.
Yesterday, at the Golden Goggles, Bob Bowman was awarded the title of "Coach of The Year". Earlier this week, I wrote about the departure of Paulus Wildeboer from Denmark, almost universally acclaimed as that country's best ever national team coach.  Yet, if you were to anonymously poll American swim coaches about Bob Bowman's success, I think a significant amount of them would say that Bowman was nothing more than a jockey to Michael Phelps' horse for these past twelve years, while just weeks ago prominent members of the Danish swimming community were calling for Wildeboer's head. All of which begs the question: how do we know if a coach is any good?

It's a vexing question in a sport full of definitive answers. Races are swum and the fastest time wins. We know who the fastest male 200 freestyler in the world is (Yannick Agnel), especially with how little doubt he left in London and since. Coaches are much harder to evaluate. They are a layer removed from the times swum in the pool. The rules for their competition are far less clearly defined.

The most common shorthand is how fast a swimmer you coach. By this criteria, Bob Bowman is definitively the best coach of the last decade because Michael Phelps has been so dominant over that time period. Todd Schmitz has seized that baton on the women's side for the time being as Missy Franklin's coach. In Omaha this summer, wild rumors flew that Schmitz was demanding an assistant coaching position at whatever school Missy committed too. This turned out to be pure fallacy, but the rumors held weight only because Schmitz coaching mojo will undeniably take a hit when Franklin leaves his program, which is incredibly silly when you think about it. Will he somehow become far worse at coaching?

People who want to look a layer deeper will look at overall team success. This too, has some serious drawbacks. The most successful club teams in America all draw from gigantic club numbers. Sheer probability predicts that they will have successful swimmers in the higher ranks. Basic arithmetic tells you that the coach of a 70 swimmer club will never best a 1200 swimmer club in any meaningful club championship meet. In college, the coaches of the day are the duo at Cal: Teri McKeever and Dave Durden. In recent memory, Durden led the University of Maryland Swimming and Diving team (RIP) to middle of the ACC results- and yet now he seems to be building and NCAA dynasty. I bring up Maryland not as a knock on Durden, but more to point out that the list of coaches who could possibly build an NCAA dynasty at Cal is considerably longer than the list that could do so at Maryland. McKeever? I'm such an admirer of her results I can't begin to say anything objective about her.

The last metric, for the hardest core, is improvement, and yet this still comes up short. When I was a collegiate swimmer in Division 3, I admired a lot of coaches for their ability to get huge improvement from swimmers at that level. Those raw time improvements happen far less at higher levels. Does this mean that Division 3 coaches are better than Division 1? No, although I do think that the results of Division 3 coaches are dismissed far more than they should be. Slower swimmers are, on average, easier to improve.

Perhaps the biggest problem in evaluating coaches is the truth that coaches desperately have to avoid: they don't determine a majority of why a swimmer performs well or not. In fact, they may be only contributing a very small piece. The athlete's themselves, their growth, their maturity, their determination play the biggest part, and you can only augment these characteristics so much. I say they have to avoid thinking about this, because it's pretty demoralizing to show up to work everyday telling yourself "I'm not going to make much of a difference today!". As a coach, you have to believe irrationally that you can, through your own singular efforts, make someone better.

All that to say, that if you were to combine all the metrics above, assigning proper weight to each, you might begin to understand which coaches are better than others. It might be akin to lighting a match in a pitch black room but it's better than wandering around in the dark.


  1. The hardest part of all of this, is looking at the development of talented swimmers or managing and training talented swimmers. Depending on your parameters, someone of Bob Bowman's stature might not be seen as the best coach. Does the coach have the ability to adapt themselves to the developmental level?

    Could the best coach in the world not want to work with Senior level swimming, even though they may possess the ability to coach at the elite level?

    It is very possible the best is some where teaching kids alternate breathing and finishing practice with cannonball contests...

    1. I don't think you can be a great coach without ending a practice with cannonball contests!

  2. This post really resonated with me as I recently watched an 8 year old boy on my Y team go 14.1 in the 25 free breaking a long standing 8 & U pool record. As his coach I was subsequently congratulated for the job I had done. All I could do was chuckle at the absurdity of it. Sure, there are things that coaches can teach and improve but 99 of 100 8 year olds are not going 14.1 no matter what level of "genius" a coach imparts upon them. As long as this kid is swimming he is going to be fast no matter what.

    I have always thought that the job of a coach is to not inhibit a swimmers progress and I am ok with that. I think there is a set of best practices that all coaches should strive for; an understanding of mechanics, physiology, psychology and administrative aptitude. I am not saying these best practices are easily achieved but if they are present then the environment is right for a swimmer to succeed. From there it is a matter of who the swimmers are in your program.

    The great coaches that we all talk about are no less great because of the fact that their impact may not be as big as they once thought. It is just nearly impossible to definitively rank their greatness.

    Thanks for a great post

    1. Agreed, 100%. Our job is to not screw up talented kids and the best are okay with that.

  3. There's a reason the best will continue to flock to some truly great club programs. The coach is instrumental in creating the environment, the swimmers feed of it and thrive. Would Michael Phelps be Michael Phelps without Bob/NBAC? Who could know ..... The funny thing is that in the end it's the swimmer that makes the coach great.

    Either way, great post. Great coach is highly subjective. It's also totally based on the level of swimmers they are working with. If I'm coaching learn to swim and ask them to do MP sets I'm a terrible coach, but if I do the same sets with MP, I may very well be a great coach.

  4. I think program building is a huge part of what makes a head coach successful, especially at the college level. In my conference, there is a coach that always has a team of about 14 men and 14 women. They come in with very slow times, and improve a ton. Meaning, they go from being terrible swimmers to being average to pretty good swimmers. This team is perennially in the bottom third of our conference at the end of the year.

    Another team in our conference brings in about 15-20 freshman guys and girls each year to their team. These kids come in with fast(relative to our conference)times out of high school. Each year these swimmers go (at best) almost as fast as they did in high school. They also have a very high attrition rate due to grades or just quitting. They finish at or very near the top of the conference every year, and their coach often wins "Coach of the Year"

    These are two extreme examples, but who is the better coach? If you are looking to improve your times, probably number one. If you are looking to win, probably number two. Obviously you want a combination of the best attributes of both ways, and that is what the best coaches are able to do.

  5. This is the same problem that a lot of school districts are struggling with right now, except they have the added dimension of having to fight with the unions about it. It is very difficult to evaluate the effectiveness of teaching, or coaching. Is it the most improved, as mentioned above? Is is the (maybe only one swimmer) that reaches the highest level? How do you account for the fact that swimmers will flock to a program that already has a big name?

    The best coach for you may not be the best coach for me. It is subjective no matter how you look at it.

    I think the time is better spent making sure that the coaches that harm, or have the potential to harm, young swimmers are escorted quickly out of the profession.

  6. I think you hit it on the head, my friend. As coaches, we should stay away from this debate. The problem is that coaches love to hate on other coaches. It's an ego/insecurity thing. It's easy to play monday morning quarterback, but if you are coming up with reasons to not give credit to a coach you're probably taking credit away from yourself too.

  7. This comment has been removed by the author.

  8. Also consider consistency over time. If Schmitz could produce another Missy Franklin and then another, etc... over the next decade, you'd certainly give him more credit than he gets now for her success.

    And if a coach can deliver in all three of those areas with one group of swimmers after another, building a program, delivering championships and producing champions over many years with an evolving pool of talent, you'd be able to attribute the success to his skill as much as the talent and work of his swimmers.