Many news articles and popular debates have presented the idea that “much of the gender wage gap is explained by women’s choices”. This is then often used to enter into long scientific investigations into the nature of preference differences across the sexes, appealing to biological determinism and genetic neuroscience. And in the end, this view is presented as implying that antibias initiatives are misguided, diversity hiring initiatives are wrong, and that the gap is “natural”.

But the studies which are referenced do not make these implications. In fact, the data that we have collected about occupational stratification is perfectly consistent with the idea that most of the wage gap is due to outgroup bias. And the evidence from bias research has shown that it is certainly there and measurable. And the evidence of historical trends has shown us that antibias initiatives and diversity hiring have done much to improve the situation.

What is it about the idea that it is “natural” that has so much staying power? We have such a long history of cultural enforcements of group roles and other group oppressions that the biased oppression of groups and their subsequent causal position in the power structures is what we’ve always seen. Becoming aware of our own biases is hard and error-prone and very humiliating.

But I think a big part of the problem is that people don’t know there is a mathematical reason for the response. They think that arguing about biases is all informal and imprecise and science has no cause to be making assertions on bias. Many people don’t understand that saying “most of the gap, even that associated with occupational stratification, can be explained by bias”, that it is actively countering that occupational stratification data has anything to say against bias. Occupational stratification can be caused by bias.

There is a model that explains this. It’s a common very basic skeletal model in economics, it uses a few very general pieces, and it’s foundational to many kinds of economic models of structural advancement. But where it is explained in economics, you rarely see the association with bias wage gaps, and where bias wage gaps are discussed, the model is typically sketched as an amathematical aside that the economists know that the fundamental correlation below exists and never pointing to it’s exposition in the references. At least, that has been my experience reading the literatures, where they talk about the same effects, but in their own preferred languages and typically only indirectly referencing each other.

So this is my attempt to provide a single resource that shows this result. It’s one of those results that is central to the discussion of bias and our scientific models of it’s results. I will probably also try to write some later papers that come back to this model with other semantic interpretations, as you will see that the mathematical structure is really the model of all stochastic or epistemically-Boltzmannian statistical theories of advancement and there are many ways that bias can affect advancement processes.

It starts with People and CareerPaths. Each Person chooses a CareerPath when they start adulthood. This is a complex process with many influences, but a good starting model is to just have them choose randomly and uniformly and use this as a baseline to investigating individual forces of CareerPath choice.

CareerPaths have levels of advancement. This is a purely general notion of level — it’s not meant to only represent some formal notion of position. Any ordered transition chain of a Person’s changes at regular intervals throughout their career might fit. The level transitions are thought of as advancements and are the primary value of the model. Each Person begins at level 0.

The evolution of the model is iterative. At each round, each Person has some potential to undergo an advancement transition. Again, these are complex scenarios to model, with a lot of competing forces, and they may not even mean the same thing for different People, even at the same putative level. Does your boss give you a promotion? Did you demonstrate an important skill that will drive your future positions? Again, starting with a uniform random chance of advancement helps show the base dynamics on which to build these complex forces.

Each iteration: x% chance to advance. Each Person has a lifetime of R iterations before they Retire. Each iteration, N new People enter the workforce by again randomly choosing a CareerPath at level 0.

Mathematically, this selection process and extinction mechanism is easy to describe. Each generation can be seen as independent — indeed each Person is independent as described. You just have a lifetime R of chances for transitions before retirement. After each iteration S, that Person has a

Mathematicians should recognise this as the Binomial Distribution. At any stage, if we analyse the population statistics of the group all having participated in S iterations, that population should obey the Binomial Distribution. In particular, if you have a simple heuristic that retirement always occurs at iteration R, then the retirement population of a simulation should, within the parameters of noise, appear Binomial.

Indeed, running this simulation shows just this:

Here, I have taken the parameters of 20% chance advancement per iteration with a lifetime of 50 iterations.

Of course, the composition of a CareerPath at any time won’t look like the retirement graph. CareerPaths consist of all the people who entered them S iterations ago, summed from S=0 (new hires) to R-1. At Level L, then, the expected proportion is

A simulation shows the shape this gives:

Employee totals by level attained

Now divide the People into 2 groups: Ingroup and Outgroup. This is just an arbitrary distinction. If you look at the population statistics of either group, they will have approximately the same shape as above, with sampling noise. But now, with this distinction, we can give a model of Bias.

Define “Bias” as a statistical favoring of the ingroup over the outgroup in the advancement process that does not yet have an explanation in terms of CareerPath criteria. We don’t have to postulate bigotry here, even if it is often the most likely problem ontology, because anything that is selecting outside stated criteria is problematic. We can add to the existing model a description of Bias across CareerPaths that can be used to investigate it’s properties.

On creation of the CareerPaths for the model, assume that each has been assigned a probability b which measures “the proportion of the ingroup advancement rate that applies to the outgroup”. So now there are two advancement rates: x for the ingroup and bx for the outgroup. If b=0, there is no advancement by the outgroup. If b=1, there is no bias in that industry.

In this model, it’s easy to see the effect of bias in the advancement process. The simulations start to show more of the ingroup at higher levels of advancement and more of the outgroup at lower level. For instance, using a simple uniform random assignment of bias from 0.8 to 1.0 over all CareerPaths in a simulation shows:

By retirement, the ingroup will have statistically better advancement than the outgroup.

With this model, each CareerPath is still kind of it’s own submodel or sampling of model space. It’s unclear why one would need multiple careers except in the ideas of the “statistics of careers”. But they are important in the real world because they have a lot of interactions between themselves, and there is one interaction that is particularly important for these kinds of dynamical simulations on achievement: career change.

Modeling how people change careers is very complex. There are a lot of forces involved. But keeping with the philosophy described above, we can start by looking at it as a random process. One first attempt might be:

Every time a Person does not advance in an iteration, they have a c% chance of switching career at random (maintaining level).

This is not a very intelligent strategy, as nothing prevents one from picking a worse career path. You really want to model the kind of behavior we see where people change careers because they are trying to do better than they currently are.

In this model, there is no deterministic variable on performance except for bias. But in the real world, the bias b for any CareerPath is not something widely published and reviewable. And calculating that can be a difficult thing that requires running polls and sampling, because industries don’t often readily reveal their data. So simulation rules on career change based on actual bias are still pretty unrealistic.

But one strategy that is an interesting modification to investigate is:

c% of the time after a rejected advancement, sample a working Person at random.

If the sampled Person is in the same group as you (in/out) and a higher level, switch to their career (maintaining level).

If the sampled Person has a lower or equal level than you or is not in the same group, and you have sampled less than some limit (say “friend size” or “social awareness size”), you can sample again.

In other words: “look around for someone like you doing better than you and go there”.

Notice what this kind of strategy implies. This strategy says “I think I might not be advancing because of bias in my profession, so I am switching professions in hope of making better progress.” It is a response to bias, not a response to preferences. Nothing about the actual work in any of the CareerPaths is specified in this model.

How does this change the predictions of the model?

One way to look at this is to look at the proportion of group representatives in the different careers. We can calculate the ratio of in-group employees in a CareerPath to the out-group employees. In the base case without transfer, this is just the random distribution of employees to careers, so the distribution is something that looks Cauchy:

When you allow transfer, though, the distribution takes on a whole new character.

Some of the features here are just noise of the simulation and low number of CareerPaths, but some details are real and informative. First, the width has doubled with just a small transfer rate. Also, there is, in the polymodality, a separating bimodal signal with one peak above 1 (~1.06) and one below 1 (~0.98). In other words, we are starting to see the migration of groups into CareerPath clusters.

We can explore this migration more by cluster plotting the CareerPaths by in-group / out-group ratio to the bias factor. When there is no transfer, such a cluster plot just shows that these are independent variables.

But, when the transfer is calculated, the plot makes an astonishing change.

The transfer is actually working to relieve Bias without the People having any direct insight into the bias or even knowing whether it existed! Out-group members tend to migrate towards CareerPaths where their advancement rate is close to 100% of the in-group, leaving the high bias careers populated by the in-group.

One can even ask if the transfer would separate the CareerPaths if there were no bias. This is where it gets interesting because the answer is no.

Random fluctuations in the proportion due to transfer are not stable because the bias is what causes the advancement differences that stabilize it. In other words, the existence of in-group / out-group career separation in these models is a consequence of Bias.

This is the primary result. It is often stated in a slightly different way:

Vertical stratification causes horizontal stratification.

If industries are biased against the advancement of certain groups of individuals, they will start to avoid participating in those industries.

It is a common trope in the gender gap apologism literature that “much of the gap is explainable by the preferences of women towards certain careers that pay less”. It’s brushed off as a choice. The evidence typically presented is the existence of CareerPath group ratio bimodalism. What these models show is that this is not sufficient evidence to say that the gap is not completely explainable by real bias and it’s effects. In fact, CareerPath differences between groups is a prediction of the Bias model, not a counterpoint.

In fact, the model shows all the main characteristics of career distributions for a variety of in-group / out-group distinctions, including those based on gender, race, religion, and sexual orientation. Bias has a common structure before the complexities of choices and environments even start getting factored in. This shows the power law differences in participation at the top levels, it shows the common stratification shape, and the eventual group bimodalism across careers.

And as mentioned earlier, all of this has been measured. We’ve measured the existence of bias in the selection of resumes for interview, in the selection of candidates with equivalent education and experience for hire, and in the promotion process in industries with standardized review processes. These have been measured consistently and repeatedly over decades.

And now, with the rising white nationalism needing biological reductionism to justify itself, we see the resurgence of apologism in the writings of people like Jordan Peterson, in the Damore screed, in articles by popularizers of bigotry like Debra Soh, in talk by people anywhere along the spectrum from Sam Harris to Steven Pinker — people who clearly do not understand this model and thus fall to the fallacy of “explaining away the bias as choice” without any methodology to pull apart the very real effect above. And in the process, as Damore shows, this ignorance turns to calls for the removal of antibias training and an outright assault on out-group protections. Clearly, the research community needs to present this information in a much more comprehensive and inciteful manner than they have so far accomplished if we are going to protect these marginalized communities from open bigotry.

These models are not the most refined and causal discussed in the literature. Numerous other effects have been measured and discussed. In particular, refinements may be made by:

Associating economic values (salaries) to different levels in different careers.

Exploring greater variation in

— The sizes of different industries

— The biases across levels

— Natural levels of talent of the people towards different industries

— Different career lifetimes

Describing career choice forces better with supply / demand modeling and preferences

Looking at more refined or varied career change strategies

Describing unemployment and it’s forces

Evolving the careers with their own creation and extinction events

Note, though, that none of these refinements would change the forces and effects described in this report. That is a pretty critical point, too. These are robust features of the model.

Also important to note is that the actual dynamics can be used to represent other segregated advancement processes where bias exists. There is a simple application to learning processes, for instance, and models like this can show how group differences in measured IQs can come about through environmental selectors that are bias-originated (and how skillset can become correlated to group membership because of these biases). These effects will appear heritable when they are due to biases based on heritable traits, but instead of genetic origins they will be pointing to epigenetic influences of institutionalised social bigotries.

There are many additional characteristics of this model, it’s forces and assumptions, and the literature does have many papers that explore various aspects in quite some detail. Typically these analyses are not in reference directly to such a base model, though, although they often use a language that indicates reference to the same ideas. This is a shame, as I think it tends to obscure some very basic mathematical arguments why the “by choice” apologism is so deeply flawed.