Jun 30, 2017 BACK
Volume VII, #13
At the end of college, I had a final interview for a job with a national consulting firm that had reserved guest rooms at a country club near campus. The rooms were tiny and – unfathomably – had no chairs. So, in the most awkward half-hour of my nascent professional life, both the interviewer and I sat uncomfortably on the king-size bed. The other notable thing about that interview – albeit dwarfed by the louche setting – was that we talked about little of substance other than the fact that a 40-something man and a 20-something man were sitting together on a hotel room bed (which was so weird, nothing else seemed remotely as substantial). It was only later, once I had begun working at another consulting firm, I learned that the primary purpose of these final interviews was to administer the “Pittsburgh Airport Test” to the candidate – determining whether or not you’d want to be stuck in the Pittsburgh Airport for 8 hours with this person.
Of course, this Pittsburgh Airport Test isn’t a test in any real sense any more than having drinks with Bunny and Digby is a test (or as my industrial-organization psychologist and psychometrician friends would say, it has no validity). And there are signs of employer discontent at this “country club” pedigree- and degree-based approach to hiring.
Last week, the New York Times breathlessly declared that you should “start solving puzzles” if want to work for Jaguar Land Rover. While Jaguar will still consider the pedigrees of “traditional” applicants, those who are able to assemble a virtual sports car and then complete several code-breaking puzzles will “fast-track their way into employment.” The Times went on to cite a range of other employers including Axa Group, Daimler and Britain’s intelligence agency GCHQ. But in a notable omission for America’s paper of record, what the Times didn’t say is that all of this is happening outside the U.S.
Employers have always utilized assessments in the hiring process, but historically only well down the hiring funnel i.e., after identifying a shortlist of candidates. Two things have changed in the past few years. First, nearly all jobs are posted online, producing hundreds of applicants for every position and creating an urgent need for an initial filter more effective than keyword-based Applicant Tracking Systems that match resumes against job descriptions. Second, online delivery makes it simple to deliver short assessments to all applicants at the very top of the funnel.
These micro-assessments can range from Jaguar’s code-breaking puzzles that test general cognitive and problem solving skills, to writing and communication skills, to job-specific situational judgment tests or technical quizzes. Constructed by qualified industrial-organization psychologists and psychometricians, online micro-assessments are already much more effective than keyword matching at reducing false positives and false negatives from the first sort, thereby allowing employers to focus on soft skills and cultural fit in the interview process.
Of course, one of the most common keywords in job descriptions is bachelor’s degree or B.A. And that’s the first thing out the window when employers begin applying micro-assessments at the top of the hiring funnel. You might have heard about the UK branch of the global accounting firm Ernst & Young removing degrees from job descriptions. Applicants for all positions at EY are now first presented with a series of micro-assessments in order to ascertain whether they’re a good match. The result? Last week, EY proudly announced it had been recognized as the UK’s 16th best employer for social mobility. By utilizing micro-assessments to shift away from pedigree- and degree-based hiring, EY saw a 75% increase in applications, a 10% increase in new hires from state schools, and a 7% increase in new hires who were first generation university graduates.
Powered by several micro-assessment vendors that are achieving scale, many other employers in Europe and Asia are beginning to see similar results. (University Ventures recently invested in one such vendor, Shortlist, which serves employers in India and Kenya.) I’m in contact with the leaders of many of these companies, and when I ask them about the U.S. market, they tell me they plan to steer clear. Why would they stay away from the world’s largest market for such tools, leaving the U.S. locked in an uncomfortable country club room while the rest of the world moves to a more spacious competency-based hiring model that increases diversity? The reason couldn’t be more ironic: decades-old Civil Rights Laws.
Since the Civil Rights Act of 1964, U.S. employers have been prohibited from engaging in significantly different rates of hiring or promotion based on race, sex, national origin, or other prohibited bases, by using criteria that are not demonstrably predictive of job performance. Employment policies need not be intentionally discriminatory; judges consider whether hiring practices have an “adverse impact.” Practices are deemed to have an adverse impact if the applicant-to-hire ratio for the disadvantaged group is lower than the ratio for the advantaged group and the difference is statistically significant. A 20% difference is the rule of thumb.
Sounds fair, right? The problem is that pedigree- and degree-based hiring has never been tested for adverse impact – primarily because degree requirements dissuade non-degree holders from applying in the first instance – but would probably fail given the fact that whites earn degrees at about twice the rate of minority groups. (And if adverse impact protected economically disadvantaged Americans – which it should – it would be an even bigger fail; while 50% of 24-year-olds with family incomes of $90k+ have earned bachelor’s degrees, the number for families with incomes < $35k is less than 6%.) Meanwhile, most of the adverse impact charges handled each year by the Equal Employment Opportunity Commission (EEOC) involve the use of assessments and background checks in the hiring process. According to the CEO of a leading micro-assessment provider, “the industrial-organization psychologist at one U.S. client described the last 10 years of her career as being spent in court defending adverse impact cases brought by the EEOC.” Another micro-assessment executive told me “what’s remarkable about the U.S. is how quickly lawyers and general counsel get involved.”
The only defense to an adverse impact claim is to demonstrate content validity, construct validity, and criterion validity i.e., that the test is predictive of job performance. While this is entirely appropriate, the process of demonstrating a priori validity for each and every assessment-position combination is so time- and resource-intensive, U.S. employers are either not interested in implementing micro-assessments, or only interested in micro-assessments for hiring employees outside the U.S. One new vendor recently shared with me that 70% of micro-assessment usage by its U.S. clients was for positions outside the U.S. International companies like Unilever that have begun utilizing micro-assessments equally across all markets are the exception. And to the extent American employers are utilizing micro-assessments, the clients are naïve hiring managers situated far from knowledgeable lawyers and corporate HR, which isn’t promising in terms of scalability.
Two weeks ago I had the opportunity to ask questions of two lawyers from the EEOC’s Office of Legal Counsel (OLC), Corbett Anderson and Aaron Konopasky. They had two responses to this conundrum. First, the EEOC does not distinguish micro-assessments at the top of the hiring funnel from the assessments further down the hiring funnel that the EEOC has always prosecuted, from which I infer they expect to continue to pursue adverse impact cases for assessments with as much vigor as ever. Second, counsel takes what I would characterize as an “efficient markets” perspective: namely, if micro-assessments are effective in improving hiring, employers will adopt them.
The problem with this view, of course, is that markets are inefficient if there’s a regulatory roadblock or “tax” to adoption, which is exactly what’s happening here (i.e., tax = cost of demonstrating validity). Other countries have no such requirements and are already improving human capital allocation and diversity. Meanwhile, the U.S. remains stuck in the country club.
To catch up with Europe and Asia, America needs an adverse impact “safe harbor” if employers can demonstrate that diversity has been enhanced by utilizing micro-assessments at the top of the hiring funnel to shift away from pedigree- and degree-based hiring – exactly what EY has shown in the UK. EEOC counsel claim the Commission doesn’t have the authority to create such a safe harbor, citing Title VII of the Civil Rights Act, which says the EEOC’s regulatory authority is limited to procedural matters. As a result, we either need EEOC leadership that is willing to claim such authority – unlikely to happen quickly due to the Commission’s fixed, staggered terms – or Congress needs to act.
America should be proud of its Civil Rights laws. The problem in this case is they don’t go far enough: they don’t break down the biases inherent in higher education credentials (i.e., degrees). If we don’t solve this problem, we’ll be at a competitive disadvantage in matching the right talent to the right opportunities, which will continue to have negative social and economic consequences. Moreover, when the question of how to stop American companies from offshoring jobs is at the top of everyone’s agenda, it’s crazy to give our employers a reason to think they can hire better/more accurately outside the U.S.
In other countries, micro-assessments are shifting hiring away from pedigrees and degrees. By getting clarity on adverse impact, America can move out of the country club and into less awkward more appropriate settings.
University Ventures (UV) is reimagining the future of higher education and creating new pathways from education to employment. UV portfolio companies are making higher education more affordable, pioneering entirely new approaches to learning, and helping employers think differently about how and where they discover talent. UV’s approach draws upon the values and traditions of higher education to play a sustainable role in transforming the path from education to a stronger economic future for students, universities, and employers. UV is led by principals with decades of experience as entrepreneurs, investors, authors, and leaders in higher education.