In economics textbooks there is a simple story told about racism. It explains how the free market punishes prejudice of any kind and instead builds an environment where all people compete equally regardless of their background. The explanation is simple – any business that refused to serve certain customers would earn less profit than a more tolerant rival. If you refuse to hire the most competent job applicant due to their race, gender, sexuality etc then they will work for your competitors or set up their own business and undercut you.

Economists don’t mean anything bad when they teach this theory as fact, they probably just want to apply economic theory to real life to show its relevance to students. They probably think it has a positive message by depicting discrimination as irrational. However, what they don’t realise is the darker implications of the theory. You see, if the free market eliminates racism, then there is no need for the state to be involved. If the free market is left alone, then all racism will completely disappear, in fact we don’t need to take any action against discrimination because the market does it for us. If there are any differences in pay or employment based on race, then they must be due to skill and productivity, they can’t be due to bigotry. Libertarians used this very argument to propose repealing anti-discrimination Civil Rights legislation and you hear similar arguments to oppose laws mandating bakers must sell to gay couples or that women must receive the same pay as men.

Yet it should be obvious that this theory isn’t true. Anti-discrimination legislation was brought in for a reason, before that time racism and sexism were rampant. There were numerous areas in America that refused to serve black people and countless businesses that operated in a way that economic theory calls irrational. Private businesses refused to serve black customers even when there was no legal obstacles, such as with the Greensboro Sit-ins. How was this possible?

The essential point is that discrimination should be not studied in isolation, but rather on a broad societal level. Instead of viewing one single racist, we should examine the situation where racism is prevalent. If most of your customers are racist and don’t want to be served by non-white people, then it is economically rational to only employ white people. Your staff may also prefer to work with people from the same background and may even experience a productivity boost due to discriminatory hiring.

This can even be the case if the staff is not overtly racist, for example, while in America, I worked in a business that only hired Irish people. As a result, there was an immediate connection between the staff and a co-operative working environment. This degree of trust is a possible explanation for why ethnicities tend to cluster in certain occupations and businesses often hire family members. An employer wants an employee they can trust to work hard even when not supervised, but it’s difficult to know if someone is trustworthy before they are hired. Ethnic and family bonds provide this trust.

A major problem for hiring employers is a lack of information about potential employees. There is no way to know to properly asses the skills and suitability of a candidate until they actually start working. So instead they use proxies to estimate the candidate’s suitability. This can include the school they attended, the area they are from, whether they have a recommendation and if they seem like hard workers. But all of these proxies can be tainted by prejudice. Somewhere may be judged to be a “good” neighbourhood because it is inhabited by the dominant race and being inhabited by a despised minority can be enough to deem a neighbourhood “bad”. Likewise for schools, which in a segregated society act as ethnic markers especially if they are no visible signs. Or they simply use crude ethnic stereotypes as a proxy, especially in a society with low education and therefore lacking more reliable measures.

Business can discriminate without even realising it. Traditionally the main way of hiring staff was through personal connections and recommendation from existing staff (because official interview procedures are long and time consuming). Naturally these recommendations would come from friends and family, which in a segregated society would usually be of the same ethnic group. Interviews are based on non-tangible experiences, which makes them prone to biases. If someone subliminally thinks black people are untrustworthy, they then may project this prejudice onto a candidate while still believing they are judging them as an individual.

There is a principle known to employers called the 70-20-10 principle, which is a rough rule of thumb that 70% of skills are learned on the job, 20% from co-workers and 10% from education. As so much of the necessary skills are learned on the job, most new hires are near useless and must be trained in everything. Even jobs with the same title and job description can be very different in different companies. Each business has its own system, its own process as well as personal networks such as knowing who to contact in regard to each possible scenario. All this means that there is not a huge difference between job applicants regardless of race or gender, which means a discriminatory business is not suffering a heavy penalty.

Racism doesn’t just affect people at the hiring stage, it begins long before then. Racial minorities often have poorer quality of schools and lack resources that give wealthier children an advantage (such as private tutoring or funds for college). Lack of savings is both a result and cause of poverty, as it hampers investment in self-development and training. Higher crime rates can further deplete meagre savings and destroy investment. If they live in areas of high unemployment and with weaker social networks then they are less likely to get work experience. In fact, their social networks could be more of a hinderance than a help if their friends and family are in poverty and need support. Small differences can quickly widen to become an insurmountable gap.

Discriminated groups can only set up competing businesses if all groups are equally wealthy, which is never the case. If a minority group is disproportionately poor, then it probably lacks the capital to set up a business or to access credit. If they can’t get hired in the first place, then how do they obtain the skills and experience necessary to run the business? If they are stereotyped as dangerous and criminal, then customers will be less likely to go to their neighbourhood to do business. All start-ups suffer from high failure rates due to the large initial costs and need to build a customer base from scratch, while competing with established rivals. Adding prejudice to the list of challenges shows why competition isn’t a simple solution.

A society can unintentionally be discriminatory if the initial structure was racist. Imagine a situation where the majority group controls all the wealth, political power, runs all the businesses etc while the minority group has nothing (they could be recent immigrants or liberated slaves). Even if the majority group doesn’t intend to discriminate, decades later they could still maintain a lion’s share of the wealth. If society is segregated and they only hire people they personally know, then few if any of the minority group could even get in the door. If only the majority group is earning experience and building contacts, then they will be the only choice for promotion and higher positions. As the minority group slides into poverty, they are even less likely to be hired due to stereotypes of being dangerous and lazy. They get caught in a negative feedback loop and poverty cycle.

It is an unfortunate fact of life that some groups suffer racism and discrimination. Economists may have good intentions in arguing this is irrational, but economics must explain how the world is, not how it should be. There are situations where bigotry is profitable and there is a good reason why we needed anti-discrimination legislation in the first place.