Bias in Seeing Bias

What a new study gets wrong

I’m sure that most women have had the experience of describing any type of gender discrimination to a man, only to be told that they’ve seen a bogeyman:

  • “Oh - he’s an asshole to everyone. It’s nothing personal.”

  • “We don’t just hire men. For some reason women don’t apply to work here.”

  • “Stop trying to take all the fun out of work. He was just playing with you.”

  • “Do you really think everyone is sexist?”

  • “We didn’t promote him because he’s a man, he just has more fire in his belly.”

Even though I’ve had these types of experiences more times than I can count—or perhaps in part because of it—I’m careful to call out bias if I don’t know all of the facts. Both because I know I’m likely to be dismissed, and because I want to be fair and objective, I’m actually less likely to describe something as biased than others might think.

For this reason, I was intrigued to read about a recent study in the European Journal of Social Psychology which purported to find two things about bias. First, it concluded that individuals with strong commitments to gender equality are more likely to trust rigorous studies showing bias against women. Second, it found that those same commitments can lead to biased reasoning, causing people to infer discrimination even when the evidence says otherwise.

In this week’s issue of Comes Now, let’s talk about this new study and its findings, and whether it truly supports the conclusion that we often see bias where it doesn’t in fact exist.

The Study

European researchers Hualin Xiao, Antoine Marie, and Brent Strickland designed a study to examine the influence of moral commitment to gender equality on perceptions of gender bias in hiring. To perform the study the researchers recruited over 3,5000 participants from the United States and the United Kingdom through online platforms. They presented the participants with summaries of scientific studies that either demonstrated or refuted the existence of gender discrimination in hiring practices.

Before talking about the specific studies the researchers conducted, it’s important to understand how they defined a “moral commitment to gender equality” (MCGE). They defined it as “the extent to which individuals deem equal treatment of men and women to be of unnegotiable importance and as central to their moral identity.” They elaborate:

Strong moral commitment to an issue is characterised by a collection of subjective and behavioural features such as perception of the issue or cause as a matter of objective right and wrong; prioritisation of the cause above all other values, and reluctance to trade it off against other goals; centrality of the cause to the person’s social identity or self-concept; rejection of messages, legislations and information sources suspected of impeding advancement to the cause; willingness to physically distance oneself from individuals with different views on the issue; and acceptance of violence to advance the issue. To illustrate, women and men who are strongly morally committed to gender equality might tend to see critics of feminism as political enemies who are just objectively wrong and need to be avoided, censored, cancelled and potentially slapped in the face.

p.1212

(Let’s just leave that last sentence there.)

Though the paper defined MCGE in the manner quoted above, the studies the researchers conducted measured MCGE by asking participants the extent to which they agreed with three propositions: (1) Achieving gender equity in society is an absolute moral imperative; (2) The conviction that we need to fight for gender equality is central to my identity; and (3) Furthering gender equality should be the government’s utmost priority.

How those committed to gender equity evaluated studies finding the existence of gender bias

In the first series of experiments, the researchers sought to investigate whether participants’ moral commitment to gender equality influenced their evaluations of a study showing evidence of gender discrimination. To do this, the researchers used a well-known study which found that male applicants were rated more favorably than equally-0qualified female applicants for a lab manager position. Participants in experiments were asked to evaluate the quality of the study, the accuracy of its findings, and the reliability of its methods.

The researchers found a clear relationship between participants’ moral commitment to gender equality and their evaluations of scientific evidence. Participants who expressed stronger moral convictions about gender equality—whether men or women—were more likely to positively evaluate studies that provided rigorous evidence of gender discrimination against women in STEM hiring processes. These individuals rated the studies as more accurate, reliable, and of higher quality compared to participants with weaker moral commitments.

Whether those committed to gender equity saw bias when none existed

In another experiment, the researchers randomly assigned participants to read manipulated summaries of studies showing either a hiring bias against women or a hiring bias in favor of women. Participants with high MCGE had high levels of trust in studies finding bias against women. However, MCGE did not affect the degree to which participants trusted studies finding bias in favor of women.

The researchers then expanded on these results in a third experiment with two-steps. In the first step, participants were asked to predict the outcome of a study before being shown the results. Specifically, they were presented with a study summary about gender discrimination in hiring, but unlike in previous experiments, the study was described as being about to be conducted rather than already completed. After making their predictions, participants were shown the actual results of the study, which either confirmed or contradicted their predictions.

Participants with stronger moral commitments to gender equality predicted that the study would show bias against women. When the results aligned with their predictions, these participants rated the study as more credible and reliable. However, a participant’s MCGE did not affect how participants rated studies showing bias in favor of women.

Finally, the researchers presented participants with fallacious scientific studies that drew incorrect conclusions from their data. In these experiments, researchers said that participants were shown a study summary that concluded there was discrimination against women, but the data within the study actually showed the opposite—women were more likely to be hired than men.

Here’s the “fallacious” study participants were given:

The authors explained that it was fallacious to conclude that this study reflected the existence of gender discrimination because “this only shows women are underrepresented in the labs compared to their base rate in the population (about 50%). It provides no evidence of hiring processes being biased against women because women may be less numerous in a profession because they are less interested in the job, for instance, and apply less.” p.1221 The researchers also believed the study to be fallacious because the percentage of women increased after hiring.

The researchers found that participants with higher levels of MCGE were more likely to accept the “fallacious” findings of the above study than those who did not. They also found those participants were more likely to accept fallacious findings about the above study than they were studies on control subjects not related to gender.

Why I Don’t Buy It

This study was reported as showing the “dark side” of valuing gender equity—if you value gender equity, you’re more likely to see discrimination where none exists. I don’t think this study supports that conclusion for a few reasons.

First, it’s important to look at the charts summarizing the results of each of the studies the researchers conducted. Every single one showed a continuum: the more someone valued gender equity based on the researchers’ criteria (see above), the more likely they were to find the existence of gender discrimination. For example, the chart summarizing the results of the last study, with the fallacious study, is below:

At best, this shows that the more extreme your views, the more likely you are to fall down rabbit holes. This would be true of people who believe strongly in gender equity as much as it would people who strongly believe in any other subject. We hardly need more studies to confirm that myside bias occurs with more frequency in zealots.

Second, it seems dangerous to characterize a person’s reasoning as “fallacious” because they draw the “wrong” inference from a one-page summary of a study. For example, perhaps the people who read the researchers’ summary assumed that gender equity would result in the hiring of more women, rather than the same percentage who applied to the position. Perhaps they assumed that the employer must have engaged in discrimination because it attracted so few women. While neither of those assumptions are supported by the text the researchers provided, they are not unreasonable. Certainly those inferences are no more unreasonable than the ones made by the researchers themselves, who are quoted in an article about their study as saying:

Also, we noticed that in everyday conversations, people often conflate the fact there are smaller numbers of women than men in some jobs with the idea of women being discriminated against during hiring processes just because they are women. This is a wrong interpretation: women may also apply less to some jobs because they are less interested in them – and the same for men. This belief is in part due to the fact that many people believe that men and women are a blank slate at birth, which we don’t necessarily believe to be true.

They were also quoted as saying that “women are typically less interested on average to go through hyper-competitive career paths than men. They are also more constrained to choose between having children or staying in academia because of a shorter fertility window and greater maternal duties.”

Hmmm. Can anyone think of other reasons women might not apply to STEM jobs or pursue “hyper-competitive” career paths? Why is it more reasonable—or less “fallacious”— to assume that hiring decisions occur in a vacuum than assume that they are made by people who’ve had decades of societal conditioning?

I recognize the irony of what I’m saying. I’m sure these researchers would say that, because I believe in gender equity, I’m seeing discriminatory motives at play where they don’t exist. But understanding that bias in the workplace doesn’t happen in isolation does not make my thinking less rigorous than someone who assumes that it does. Indeed, that’s the danger of their study: even though it doesn’t support the conclusion that people who believe in gender equity are less capable of spotting discrimination, this study and others like it can be used as yet another way to discount people—mostly women—who have those values.

In short, to me this study is a big nothing-burger that unfortunately may be used to lift far more than its weight.

But then again, I’m biased. . . .;)

Share Comes Now!

If you’re enjoying this newsletter, please share it with others and encourage them to subscribe. I draft it on Beehiiv (Comes Now (beehiiv.com)) and distribute it on Tuesday evening, but also post the issue as a LinkedIn newsletter on Wednesday mornings.

Have a topic you’d like me to address? Want to tell me where I got something right or wrong? Send me an email at [email protected].

Reply

or to participate.