Can Hiring Committees Learn Anything from Sports?

With the NBA lottery draft order being decided tonight, I think this is a good time to throw out an idea I’ve been mulling over for quite some time.  Tell me if I’m off base, right, or something in between.

In the months leading up to the 2004 NBA draft, the Orlando Magic, holding the first pick, had to choose between Emeka Okafor and Dwight Howard.  Okafor had proven himself to be a star in college.  Howard was the unproven prospect with limitless potential who was coming straight to the NBA from high school.  The Magic, of course, chose Howard.  And it was the wise choice.  Okafor is a decent role player.  Howard is a superstar.

In terms of entry-level hiring, the Okafor candidate would get hired over the Howard candidate at basically every law school.  The reason why, I think, is because GMs and scouts of the NBA, NFL, etc. think that being “talent evaluators” is the best way to improve their team.  Hiring committees, on the other hand, opt to be “talent predictors.”  If my hunch is true, it's a big mistake.

Let me explain the difference.  Talent evaluators look at the entire candidate—their skill-set, ambition, love for the craft, personality, etc., AND past performance to make an assessment into how one will perform in the future.  Talent predictors look almost solely at a candidate’s past performance as the predictor of future performance.  Talent evaluators can see the potential in guys like Cam Newton, LeBron James or Bryce Harper.  Talent predictors would never get behind any of those three, but would vouch for guys like Ron Dayne, Kellen Moore and Shane Battier.

The sports analogy does not perfectly fit with the legal academy.  But I use the analogy as a launching pad for a discussion concerning whether hiring committees are looking at their duties in the best way.

 

Comments

[Jeffrey Harrison]

Of course not but this is hardly a secret. I written a lot about it on classbias and moneylaw as have many many others. They hire largely on the basis of institutional authority but no hiring committee has assessed the validity of this measure. They just use it without any empirical evidence that it is the best predictor. In a round about way a sign that they are not doing the job is how few new hires move up to better schools yet virtually every new hire does get tenure where he or she was originally hired. Making the right choices in sports mean choosing someone for which there will be a market when free agency sets in. In faculty hiring we are all free agents all the time but there is little upward movement.

Here are some of the things I would ask new recruits to cut through the window dressing in order know the person.
1. What are you reading now that is not about law.
2. Name five of the books on your nightstand that you plan to read. (Do not have 5? Bye.)
3. Do you know anyone currently in prison or who has been convicted of any crime at all.
4. What book do you keep in your car for reading when waiting at the doctor's office.
5. Give me five names of people who will vouch for your work ethic and who are not at elite schools.
6. What have your summer jobs been?
7. What electives did you take as an undergraduate.
8. How was your education paid for.
9. Can you name 3 operas and their themes.

Many others but not enough time. Unless they educate themselves, (and many do) the graduates of elite schools tend to be one issue experts, traditional thinkers and, well, pretty boring. It is important to know them and all recruits beyond "how privileged are you and how are you at doing law school" which is what we typically look for these days.

[Peter Yu]

One major distinction between law school hiring and NBA drafts is the extra information teams have based on scouting reports, pre-draft camps, team workouts and mock drafts. We certainly don't have this luxury. Whether as a member of the hiring committee or the faculty at large, it is amazing how often we need to make decisions based on little information.

[İkinci El Makina]

İkinci El Makina

[Matt]

My understanding is that none of the traditional criteria used in hiring (school of JD, law review membership, clerkship, law firm, etc.) are good predictors for scholarship. The only good predictor for that, as far as I understand, is having produced scholarship. Now, for everyone who has produced scholarship, there was a time when she or he hadn't, so that's isn't perfect, but my understanding is that it's the only trait that's looked at that is a good predictor. Of course, producing scholarship isn't the only thing one looks for in hiring decisions, but my guess is that many of the other desired traits are even harder to predict for.

[Matt]

Jeff- I'd be interested to know what you think these questions predict. I'll admit that they seem like a hodgepodge that won't predict anything to me, but would be interested to hear your theory.

Also, I wonder if Cam Newton fits the model quite right that you want, Brando. Or maybe I don't understand it. Newton, after all, won the Heisman Trophy and a national championship and was an amazing physical specimen, having a stereotypical NFL body. The only question against him was whether he was enough of a passer for the NFL. But he seems like an "easy" pick, not one that you'd have to "predict" would do well. Or maybe I'm misunderstanding your comparisons.

[Brando Simeo Starkey]

Hey Matt,

The reason why I think Newton fits the model is because he: played only one year in college; completed only 185 passes; and he played in a gimmicky run-first offense. That is to say, Newton had, from a passing perspective, a very thin resume. But, the Panthers wisely selected him anyway because of his arm strength, speed, and other physical tools. His talent, not his past performance, predicted his greatness.

[Matt]

You're probably right, Brando- I only barely follow pro football, so don't know very well. If I'd been a scout, I think I would have thought of Newton as "younger and probably better Steve McNair". McNair did pass a lot more, but against much weaker competition.

[Kendall Isaac]

I agree past performance predicts future performance is the norm, it is even the mantra used in corporate recruiting. Using this approach is more safe. After all, who wants to be the one to take a risk on the untapped potential candidate only to find out the person won't reach that potential (Jamarcus Russell comes to mind).This being said, I think every faculty should make such a calculated risk periodically, just not every time every year.

Do you seriously ask those questions Jeff? I have interviewed hundreds of people and wouldn't fathom asking most of those, except maybe 1, 5, 6 and perhaps 7. 3 & 8 seem inappropriate and potentially invasive. If I were general counsel at your school, I would cringe at those two questions! But to each his own I suppose.

[anon]

Reading this discussion should send shudders down the spine of anyone facing the meat market.
The attitudes reflected by this discussion are just the sort of attitudes candidates fear that hiring committees harbor.
To see those attitudes expressed in writing, especially without the slightest hint of self-awareness, is sort of nauseating.

[Matt]

To see those attitudes expressed in writing, especially without the slightest hint of self-awareness, is sort of nauseating.

Which attitudes? That Cam Newton was somewhat risky because of the run-first nature of his offense? I'm still not sure I'm convinced by that, but it hardly seems nauseating. Or maybe you had something else in mind?

[anon2]

I would guess anon is referring to Jeff Harrison's comment.

[Howard Wasserman]

Two thoughts. First, recent studies have found that the quarterbacks who do succeed in the NFL are those who: 1) played a lot in college and 2) passed accurately in college. This came up a lot in the debate over whether to select Andrew Luck or Robert Griffith III this year. So Newton may be the outlier.

Second, what are the law school equivalents to the talent evaluators you describe? Personality, ambition, and commitment can come through in interviews, either at the meat market or back on campus. I (and probably most faculty members, I expect) try to focus on those things in talking with candidates and making decisions. Yes, we want scholars, but we also have to work closely with the people we hire, so what you describe is important.

But personality, ambition, and commitment can't come through on a CV or FAR form, so the candidate still needs something to get her into the room. So what are the equivalents of Dwight Howard's size and athleticism or Cam Newton's arm strength that can come across on paper? Doesn't this bring us back to pedigree: Degrees, performance in law school and who that performance was against (great grades at the University of Chicago beats great grades at Big Football State School of Law, just as great passing stats at Big Football State beats great passing stats at Harvard), clerkships, etc.? And doesn't that reify the criticism of law faculties reproducing themselves?

[Ed]

I think this is an interesting observation, but it needs to be refined somewhat. Given that law schools almost universally require a JD (unlike the NBA, which permits drafting youngsters without college training -- though post-Howard, the CBA restricts this somewhat), the analogy might be to the treatment of VAPs, fellowships, and non-law advanced degrees. I take it that emphasizing these, and the scholarship that is produced during them, would count as "talent predicting" in your book. Before those became so common, it could just as easily be argued that hiring was done in a "talent evaluator" mode more than anything else; it depends on whether one views a shiny pedigree, clerkships, and recommendations from professors and judges as salient "performance" that runs toward prediction (in terms of teaching and scholarship), or rather as illustrating a love for the law, drive, and ambition, and so forth. I think this puts me in agreement with Howard Wasserman's comment . . . at the least, you need to be specific about the kinds of inquiries that count in each of the two modes.

For candidates reading this thread with alarm, I think Jeff Harrison would be the first to say that his approach is highly unorthodox. It may be resisted because of elitism, as he would say, or because it is extremely idiosyncratic and a terrible means of evaluating talent, as others might, but in any event it is not par for the course. If, on the other hand, you are abreacting to the advent of the "talent prediction" mode, your concern is more legitimate.

[On the market in 2012 (maybe)]

I'm confused. Aren't hiring commmittees much more likely to select the "Dwight Howard" candidate these days -- it's just that a PhD or VAP or fellowship has been added as an important predictive variable?

Or in other words, assume the following two candidates:

Candidate A: Yale JD, COA clerkship, two years practicing at an elite DC firm, non-Bigelow/Climenko VAP, one publication in the top 100.

Candidate B: Michigan JD, district court clerkship, eight years practicing at a slightly-less-elite LA firm and a DA's office, no VAP, three top 100 publications (including one top 30).

Doesn't Candidate A get the nod most of the time (I could be wrong about this)? And isn't that because her predictors (law school, clerkship, less time in practice, VAP) outweigh Candidate B's demonstrated performance (publishing).

[anonprof05]

On the market: from my impression at at a mid-100 law school, I think both your hypothetical candidates would get an initial interview at the AALS, and whether either got a callback would depend on (1) how well they did, (2) how good the written work really was, and (3) their fields of interest -- they'd both be marginal constitutional law profs, but very attractive if they were commercial law people.

More generally, your question assumes that there is some strict way to sum up candidates' credentials and figure out, in advance, how well a candidate will do on the market. There isn't. There are, at least at my school, probably threshold requirements -- a degree from a top 25 or so law school (and if you're from #25, you had better have Coif, etc.), a competent article beyond a Note, some other indication of academic interests -- but after that, it will depend more on your scholarly abilities, as demonstrated in both your article and other ideas, and people skills. I have seen Yalies with all the credentials one could want flame out in on-campus visits, and candidates with much less impressive credentials (on paper) impress the entire faculty and receive job offers.

Oh, and some candidates may have been spooked by the bizarre set of questions posted by Jeff Harrison. They're just silly, and appear to have much less to do with finding good scholars and teachers than with identifying amusing lunch companions. No decent search committee will ask such questions. Ignore them.

[Anon]

As a faculty member, I'd be very cranky with a colleague pestering candidates with questions about opera, numbers of books on nightstands, or whether they carry a book around in case they have to be in a doctor's waiting room. Unless those were very friendly questions in casual conversation, as opposed to some sort of idiosyncratic and demanding questioning indicating that the questioner thinks there are right and wrong answers to those questions that would have any bearing whatsoever on whether an individual is worthy of a faculty appointment. I truly hope candidates on the market reading this thread don't believe that's a normal and typical occurrence during an interview process. It's not.

[On the market in 2012 (maybe)]

anonprof05 --

Thanks for the advice! I'd love to get your further thoughts over email -- I'm trying to figure out whether it makes sense to submit a FAR form in August or not.

onthemarketin2012 at gmail dot com if you're amenable. And no worries if not.

[Howard Wasserman]

I agree with everything anonprof said. Both candidates get meat market interviews. And once you're in the room, personal skills really take control. That goes back to my earlier point: We are always evaluating intangibles (or trying to) that come across in conversation in considering who to hire. The issue is always how to get in the door with a form CV that, to be honest, is poorly structured and does not allow for complete or fully accurate information.

[anon]

It's easy to reverse this argument in both the sports and law faculty hiring. For example, for every "Dwight Howard/Emeka Okafor" example where a more talented/less experienced player ended being a better choice than than a less talented/more experienced player, there is a counterexample. Are talent "evaluators" doing a better job taking Kwame Brown than talent "predictors" were taking Shane Battier 5 spots later? Either approach has its limitations (one is high risk/high reward, the other low risk/low reward, arguably), in both faculty hiring and sports contexts. At the very least, the NBA is concerned enough about its "talent evaluating" skills that it has insisted on an age limit (and is looking to raise that limit), while the NFL has an even more stringent age limit and MLB has a minor league system. Sports teams don't seem comfortable with a total "evaluator" model, but want some "prediction" based on actual performance under reasonably similulted game conditions (college or minor leagues).

Law schools seem to be behaving the same way - moving away from "evaluating" toward "predicting." I think law school hiring committees have typically behaved more like "evaluators" going after Dwight Howard (or Kwame Brown) than "predictors" going after Tim Duncan (or Danny Ferry). Until recent years, hiring committees focused on law school attended, grades, prestigious clerkships, references, and performance in job talks and interviews (these criteria obviously still play a large, perhaps disproportionate role, in hiring). But using these criteria strikes me as most like using combines and workouts to evaluate talent in a draft. It's not about actual performance in the job you're hiring them to do, but instead proxies for the kind of talent it takes to do that job. It would be like watching Dwight Howard (or Kwame Brown) run and jump and lift weights, measuring height, weight, body fat, and seeing workouts, but not evaluating actual games (except amateur, low competition level high school games). On the other hand, hiring committees are moving increasingly to relying on VAPs/Fellowships and publication records in hiring decisions (I don't know if these factors have overtaken the others in weight, but they seem to be getting there, at least). This seems more like watching Tim Duncan (or Danny Ferry) play for four years in a tough, NCAA conferences (or perhaps the better analogy is minor league baseball). Entry-level candidates are evaluated increasingly on performance in jobs very similar to the one they are trying to get (writing and teaching). The growing importance of the VAP and publication record in entry level hiring suggests law schools are behaving like the NBA when instituting an age limit - it's a hybrid approach where "evaluating" still has a large place, but institutions insist upon a certain level of production and experience to assist in "predicting."

[Paul Gowder]

Jeff, I have to ask. I take it from your other blogging that you're against the de facto dominance (to the extent it exists) of the upper-middle-class and above in academia. In light of that, how do questions like "[c]an you name 3 operas" help? Unless, that is, you plan on rejecting people who can in fact name them...

(I'm really glad I didn't get that question at the AALS last year. I can't stand most opera, so I'd have had to answer "1) The magic flute. Theme: Mozart was a freemason. 2) The ring cycle. Theme: Yeah Prussia! 3) All Gilbert and Sullivan. Theme: love is silly.")