AI as Talent Scout

This NYT article profiles Eightfold, an analytics startup, that is developing a tool to help recruiters identity nontraditional candidates who are as qualified and high performing as traditional candidates. The CEO argues that this type of analytical tool could be a win-win for both employers and job seekers. For employers, it could help surface qualified candidates who are willing to accept a lower wage because of their nontraditional background. For job seekers, it could help them gain the attention of firms that had previous overlooked them.

The way the tool works is by extracting information from a candidate’s resume and then scoring the candidate based on a proprietary database of variables that are correlated to certain outcomes. For example, the CEO points out that since 90% of software engineers at Intuit knows the programming language Java, their algorithm “is on safe ground inferring that an Intuit software engineers knows Java, even if he or she doesn’t list the skill on a resume”. Other examples of correlations the CEO gave include chess players tend to be skilled at coding, basketball players are good at sales, and natural language processing majors at University of Massachusetts tend to become high-achieving workers.

I see a few pros and cons with this type of product:

Cons:

1. Since the proprietary database is built by analyzing employers’ current employees and performance evaluations, it inherits any bias that exists in these past hires. By relying on this potentially biased database to score future hires, it could perpetuate biases. For example, if a company traditionally only hired men for a certain job, then the algorithm will blindly look for only men as it scans through resumes. Eightfold needs to address this problem in order to accomplish its mission of helping companies look beyond the traditional candidate.
2. The proprietary database could be incredibly hard to scale. Since non-traditional candidates, by definition, are in the minority at most organizations, Eightfold would need to sign up many customers in order to have valuable insights on attributes of non-traditional candidates. For example, their insight that natural language processing majors at University of Massachusetts tend to be high-achieving workers is an interesting one, but I question how big of a sample size the insight is based on given how specific this variable is.
3. The product only considers the candidate’s resume attributes (prior work and educational experiences). Given the limited scope of data, I wonder what the p-value and r-squared statistics are for the company’s predictions and what the company defines as good enough evidence to use as insights for their customers. I also worry about the potential of mistaking correlation for causation. For example, if the insight that basketball players tend to be better at sales comes from a customer that sells basketball shoes, then this particular insight may not apply to a customer that sells other products.

Pros:

Competing in the Age of AI

Online HBS Executive Education Program

4. I like the fact that this product is used alongside human recruiters. Given some of the potential drawbacks and limitations described above, it would be important to have human in the loop to sanity check any recommendations from the system. Recruiters could use the tool as a way to ensure they haven’t overlooked any candidates since they’re often sifting through thousands of resumes in a limited amount of time.
5. If the product works the way it is advertised, it could help alleviate student’s pressure to pad their resume. In today’s competitive job market, employers heavily rely on the brand recognition (schools, prior firm) to judge whether a candidate is qualified. For example, private equity firms prefer candidates with Ivy league and investment banking backgrounds as a way to make sure the candidates have been vetted. While this approach is totally logical, it has the negative consequence of incentivizing students to seek out opportunities just for the sake of the credentials. My personal experience working in the industry suggests that candidates with non-traditional backgrounds can do the job just as well as the ones with traditional backgrounds.
6. The product could also help educational institutions better market themselves. I wonder if the University of Massachusetts knows that their natural language processing majors tend to be high achievers. By arming itself with this type of data, it maybe able to better market this particular major to prospective students. The potential draw back of this idea is that sharing the inner details of the algorithm may invite people to try to game the system. For example, a software engineering job seeker could list chess under interests just because he/she knows that the algorithm would rank them higher.

Sources:

Scheiber, Noam. “A.I. as Talent Scout: Unorthodox Hires, and Maybe Lower Pay.” The New York Times, The New York Times, 7 Dec. 2018, www.nytimes.com/2018/12/06/business/economy/artificial-intelligence-hiring.html.