This Harvard Business Review article gives a great background on the reasons why companies are still largely homogenous in their hiring practices. It explains that the hiring process which “is actually composed of five different processes: assembling a pool of candidates, resume review, interviews, offers, and compensation packages,” remains inconsistent. For a true DEI initiative to work, capture company metrics before implementation of strategy. Allow the metrics to drive the strategy and change the company culture throughout the organization, but start by addressing and correcting the hiring process.
– Karen Guy-Smith
To Build a DEI Program That Works, You Need Metrics
by
October 12, 2022
HBR Staff/Image Source/Getty Images
Summary. The authors’ recent and ongoing research describe several situations where data has moved the needle in helping companies make progress on diversity, equity, and inclusion. At one law firm, DEI advocates wanted to change the firm’s system for allocating work…more
It’s easy to set aside two hours for a workshop and call it a day, but companies that are truly committed to making progress on their diversity, equity, and inclusion (DEI) are ready for a heavier lift. Accomplishing real change requires changing the systems that corrode inclusion, which means people have to build new habits.
Even though people like their old habits.
Here’s an example. When one law firm’s director of diversity and inclusion and partner in charge of diversity and inclusion proposed changing the firm’s system for allocating work opportunities, they met a brick wall of skepticism. Partners are under a lot of pressure to deliver while keeping billable hours charged to the client low, and many believe that the least risky path to success is to keep giving the high-profile work to people who they’ve worked with in the past. In other words, their old habits were working for them, and they saw a risk in trying to change them.
But these old habits weren’t working for everyone. Changing the way people access valued opportunities is one of the most central challenges faced by organizations committed to improving DEI. In industry after industry, our research shows that 81% to 88% of white men report fair access to career-enhancing assignments. For other groups, that percentage slips as low as 50%.
What’s the shortest path around a brick wall? Metrics.
Using Metrics to Get Buy-In
At a law firm, where the business model is based on billable hours, getting enough hours is a prerequisite for success. So, the director of diversity and inclusion and the partner in charge of diversity and inclusion crunched some numbers for early-career associates in two of their largest practice groups. They looked at billable hours for the same six-month period across two years, and found that white men averaged up to 225 more hours a year than all other groups, and up to 339 more hours than people of color.
This data turned one of their biggest skeptics into their biggest advocate: “He went from yellow light to double green after looking at the data.” With the help of the task force, they proposed a change: an option to channel assignments for attorneys in their first two years of practice through a new centralized system. They also created an incentive to do so: Partners could “write off” hours assigned through the new system (i.e., not charge them to the client). This minimized the risk of using the new system.
The next step was a pilot in one of the firm’s largest departments. Again, metrics played a central role. The director of diversity and inclusion and the partner in charge of diversity and inclusion created charts showing each junior attorney’s billable hours and how many partners they had worked with (another important metric for advancement). They then shared these with every supervising attorney. “I explain that what we want is a pie chart that’s nicely balanced and ask what would make it easier for them to give work to people who are getting fewer hours,” the partner in charge of diversity and inclusion said.
These conversations have proved transformative. One partner protested, “I use everybody, I’ve got a ton of work” but bought into the new system when the data showed otherwise. “The data is the most powerful tool,” said the partner in charge of diversity and inclusion.
Using Metrics to Pinpoint What Needs Fixing
Getting buy-in is just one way metrics prove essential. Metrics also can pinpoint where problems are arising. Take hiring. Hiring is actually composed of five different processes: assembling a pool of candidates, resume review, interviews, offers, and compensation packages. If your hiring is overly homogeneous, you need to know which one(s) of these processes to fix.
Two of our experiments, currently in progress, highlight this point. Both involve hiring, one in a mid-stage tech company and one in a tech unicorn. If you look at the data, it turns out the companies have two quite different first-order problems.
The tech unicorn relies heavily on “leads,” which include internal referrals as well as candidates vetted by recruiters. The data showed that men of Asian descent made up 20.3 percentage points more of the leads, as compared with their composition in the application pool. Women of Asian descent made up 7.2 percentage points more, while white men made up 2.4 more. White women, and non-Asian underrepresented minorities (URMs) of all genders made up 3.2 to 14.2 percentage points less of the referrals as compared to their respective representation in the application pool.
If this company were to change one element of the hiring process, it should be evening out the pool of leads. This will require reaching out proactively to sources of diverse talent, limiting referral hiring, or both.
The mid-stage tech company faced quite a different challenge. When we analyzed a subset of ratings candidates received during interviews, we found that white men got job offers with interview scores far lower than any other group, while URM women and white women needed the highest scores to get offers. White men were only rejected if they got really low scores, while URM women were rejected even when they had sharply higher scores than those of rejected white men.
If this company were to change one thing, it should be using metrics, rubrics, and objective selection criteria in order to control this pervasive “prove-it-again” bias. A recent study found that 30% to 50% of the gender promotion differential is caused because white men are judged on potential, whereas everyone else needs to prove themselves over and over.
This kind of information can save a company a lot of time, money, and trouble. Without precise metrics, a DEI effort can spend a lot of time and money trying to fix the wrong thing.
That’s just one message the two tech companies’ metrics held. At the most basic level, the data showed that different groups were struggling at the two companies. In the mid-stage company, white men experienced an invisible escalator that consistently compounded their advantage: They received offers at higher rates overall (as compared to their application rates), and they made it through both resume review and interviews at higher rates than one would expect given their representation in the initial pool. At the unicorn, Asian-Americans received an early boost at the referral stage that persisted through the process. However, there was a paucity of Asian-American women at the higher levels, which should no doubt be a focus of attention.
Metrics also showed that people from underrepresented groups faced different challenges at each company. At the unicorn, but not the mid-stage tech company, URMs received offers at a lower rate as compared with their application rate. At both companies, URM men faced challenges different from those faced by URM women. URM men applied to both companies at much lower rates than their population share. The same was not true of URM women … but they were less likely than URM men to get offers once they entered the pool. So, the solution for both companies is more effective outreach to URM men — but also controlling bias in hiring so that it does not derail URM women.
At both companies, white women made up a dramatically smaller share of the offers pool than one would expect given the number of applications. Bias has a compounding effect: Although white women were only slightly less likely to make it past each stage, by the offer stage their representation in the pool had fallen by 7-9 percentage points as compared to their application rate.
Both companies need to work on controlling bias in interviews: White men had the greatest success in interviews, probably as a result of in-group favoritism — dominant groups tend to favor others of their group. White women got dinged at both companies, probably reflecting “tightrope bias”: Women who were assertive may have been seen as “too much,” and women who weren’t as “too meek.” But the other group that fared relatively poorly in interviews differed in the two companies: URM men and women at the unicorn startup and — fascinatingly — Asian men at the mid-stage company. In that company, bias in interviews may be knocking out men of Asian descent, despite the fact that their average ratings were higher than white men, URM men, and Asian women.
Have we convinced you that it is worth investing in DEI metrics? Without metrics, you’re throwing money into the wind in the hopes it will blow back into your pocket. Savvy companies have stopped doing that. We continue to launch experiments in companies, with a generous grant from Walmart. In fact, in partnership with the Conference Board, we are opening up a new cohort of 30 companies that want to collect metrics, and use evidence-based bias interrupters to solve the problems they find. We are recruiting those companies now; stay tuned for more insights – and contact us if you are interested.