Google has spent much of the past 72 hours insisting its commitment to diversity is “unequivocal” after the internal publication and subsequent leak of an anti-diversity polemic by a Google engineer. The unidentified software engineer argued, among other things, that biological differences between men and women account for the extreme gender imbalance at Google and other technology companies.

“We are unequivocal in our belief that diversity and inclusion are critical to our success as a company,” said Danielle Brown, Google’s vice-president of diversity, integrity and governance.

“Building an open, inclusive environment is core to who we are, and the right thing to do,” added Ari Balogh, the company’s vice-president of engineering, “‘Nuff said.”

Google might prefer the discussion to end there, but the reality is there is a lot more to say about the company’s commitment to diversity.

The public relations blitz may be a corporate necessity given the virulent backlash against the document by many of Google’s own employees. On Monday night, Bloomberg reported that the engineer said he had been fired; Google declined to comment on individual employee cases.

But public commitments to diversity from Google executives do not tally with the company’s workforce data.

Google’s workforce is, by its own accounting, 69% male and just 2% African American. Just 20% of technical jobs are held by women. Google may be unequivocal in its “belief” about diversity, but the figures make its shortcomings clear. The company tends to hire white and Asian men over women and other racial minorities.

Lack of diversity in Silicon Valley is an old story. Eighteen years ago, civil rights leader Jesse Jackson first launched a campaign to encourage the region’s tech companies to hire black and Latino workers. At the time, he was accused of “terrorism” by Scott McNealy, the co-founder of early Silicon Valley giant Sun Microsystems.

Tech leaders may have changed their tune in the intervening decades – all the top CEOs today loudly proclaim a commitment to “diversity and inclusion” – but in other ways not much has changed in almost two decades.

McNealy, now the chairman of a digital marketing startup, stands by his statements on Jackson, though he concedes that “terrorism” might have been an overstatement. “Probably the right word is blackmail,” he told the Guardian. “I just don’t have time for race baiters. Stop baiting me.”

The Google campus in Mountain View, California. The company is facing a backlash over a leaked anti-diversity document written by an employee. Photograph: Bloomberg/Bloomberg via Getty Images

Google is the subject of an investigation by the US Department of Labor, which has accused the technology corporation of systematically discriminating against women (the company denies the charge.) Much of Uber’s top tier of executives has left the company amid complaints of systematic sexual harassment and gender discrimination. And the tech industry has lately been shaken by allegations that high-profile venture capitalists have abused their position to prey on female startup entrepreneurs.

Meanwhile, the representation of black, Latino, and female employees at top Silicon Valley technology firms remains so disproportionately low that a government report published last year described the problem with the same word that Jackson uses: “segregation”. For all its forward looking technologies, Silicon Valley is in many ways mired in the ugliest practices of the American past.

A tale of two tech cities

Picture a technology hub where more than 17% of high-tech workers – from programmers to security analysts to software and web developers – are African American.

This isn’t some kind of utopian diversity thought experiment. It is the greater Washington DC metropolitan area, home to more than 200,000 high tech jobs, many of them with the federal government or government contractors.

“You’d be hard pressed to have someone out here who thinks that blacks doing computer work is weird,” said William Spriggs, a professor of economics at Howard University. And lest you think that the computing in DC is less advanced than that in Silicon Valley, he adds: “We don’t do Mickey Mouse stuff out here. This is the number one place if you want to do cyber security.”

The DC area is a kind of mirror image to Silicon Valley when it comes to hiring African Americans. Overall, blacks make up 14.4% of the workforce nationwide and 7.4% of high-tech employment. In the DC metro area, which includes parts of Virginia, Maryland, and West Virginia, blacks hold 17.3% of the jobs in 12 computing occupations, according to government employment data.

But cross over to the west coast, and in Silicon Valley African Americans hold just 2.7% of the jobs in the same categories. At premiere employers like Google and Facebook, black representation in technical jobs drops below 2%.

To Spriggs, there is simply no excuse for Silicon Valley’s failure to hire a more diverse workforce. “The thing that always irritates me is that they say, ‘We can’t find them,’” he said. “You run a freaking search engine!”

So how did Silicon Valley end up with fewer than 5,000 black people in highly technical jobs, while DC has more than 35,000?

One obvious difference between northern California and the mid-Atlantic region is the underlying demographics. The DC metro area is approximately 25% black, while Silicon Valley is about 6.5% black.

For all its forward looking technologies, Silicon Valley is mired in the ugliest practices of the past

But companies like Google, Facebook and Apple are known to recruit aggressively across the country – and throughout the world. And the fact that northern California’s workforce is heavily Latino (more than 20%) is not reflected in the area’s tech companies (about 6% Latino).

Spriggs argued that a significant difference is that in DC, the tech industry grew up around the federal government. Affirmative action provisions for federal contracting encouraged African Americans to start businesses in computing or data processing in the late 1970s and early 1980s. The first domain name registrar for the internet, for example, was the black-owned company, Network Solutions, which was founded in northern Virginia in 1979.

“Having black-owned companies helped get people in,” Spriggs said. “It’s partly entrepreneurship, partly because the federal government does not discriminate, partly because you have to have [security] clearance, which favors American citizens, and partly because the area is heavily black.”

Schools in the region focused on preparing their students for technology jobs with government contractors as well.

Washington DC is a kind of mirror image to Silicon Valley. African Americans make up 17.3% of the jobs in 12 computing occupations, versus 2.7% in the California tech hub. Photograph: Danita Delimont/Getty Images/Gallo Images

“The industry [has been] reaching out to and working with the historically black colleges and universities in the area,” said Ben Jealous, the former president of the NAACP who is now a partner at venture capital firm Kapor Capital. “Morgan State, Virginia State, the University of Baltimore Maryland – all of those schools are Stem schools that have focused on providing people to legacy tech companies.”

The relationships are advanced enough, Jealous said, that companies will inform universities what kind of skills they project needing five-years out, so that curricula can be adapted to ensure a trained workforce.

The result is a technology hub that looks like what Silicon Valley, a supposed wellspring of innovation, claims to want. Yet it is not just demographics that differ 3,000 miles away – the politics is different too.

‘Few are making progress’

Where the DC tech industry grew in a symbiotic relationship with government, many of the pioneers of Silicon Valley were techno-libertarians, ideologically opposed to government regulation and oversight.

The political climate in Silicon Valley in the 1990s was anti-government, anti-affirmative action, and anti-immigrant, said Butch Wing, who has worked with Jesse Jackson on the Rainbow Push Coalition’s Silicon Valley project, an effort to increase the participation of people of color in tech, since 1999.

California voters approved statewide ballot initiatives to ban affirmative action in college admissions and bar undocumented immigrants from going to public schools or access public services in 1996 and 1994, respectively.

Tech leaders like McNealy and Cypress Industries CEO TJ Rodgers were openly hostile to the idea of consciously diverse hiring. Rodgers wrote an op-ed in the San Jose Mercury News in 1999 declaring that “the only sharecropper I know is my dad” and calling Jackson “a hustler who exploits white shame for his own financial and political ends”.

It’s a far cry from the Silicon Valley of 2016, where Facebook erected a banner reading “Black Lives Matter” in the center of its Menlo Park headquarters and all the major employers have hired diversity and inclusion executives.

And yet, Wing suspects that the same ideology lingers.

There’s a love fest with diversity and inclusion, but few companies are making any actionable, measureable progress Butch Wing, diversity activist

Facebook CEO Mark Zuckerberg, who last year chastised his employees for crossing out “Black Lives Matter” on the walls and replacing it with “All Lives Matter”, is also friends with and was reportedly mentored by Peter Thiel, the Pay-Pal co-founder who co-authored an anti-multiculturalism screed, The Diversity Myth, in 1995.

“I’ve not seen a single editorial against diversity since 2014,” said Wing. “Communications-wise, companies today will all say diversity is in their DNA. There’s a kind of love fest with diversity and inclusion, but few companies are making any actionable, measureable progress on hiring underrepresented minorities and overcoming the ‘2% dilemma’.”

At the top 75 companies in Silicon Valley, only 3% of employees are black, according to the Equal Employment Opportunity Commission. Premier employers Facebook and Google have yet to crack 2% in technical jobs.

Facing renewed pressure from Jackson and other activists in 2014, the top firms began releasing data about their workforces and publicly committing to diversity programs.

But as the companies continue to grow, their hiring of underrepresented minorities has remained too small to make a real impact. The numbers of employees they would need to hire to achieve anything close to proportional representation keeps getting larger.

The pipeline ‘problem’

The Facebook main campus in Menlo Park, California. Photograph: Robyn Beck/AFP

Any discussion about diverse hiring in Silicon Valley inevitably becomes a discussion about “the pipeline”. The pipeline, the story goes, is the steady stream of able and willing workers that are pumped out of colleges and universities each year, computer science degrees in hand, ready to populate the tech company campuses that dot the suburbs of the Bay Area.

It’s not the fault of tech companies that the pipeline is overwhelmingly filled with white and Asian people, Silicon Valley’s defenders claim. It’s the fault of the education system. “Minorities are the minority by far in computing programs,” conceded Dr Juan Gilbert, chair of the computer science and engineering department at the University of Florida.

But there’s a problem with that argument: black students are earning computer science degrees at higher rates than they are being hired by Silicon Valley companies. In 2014, they received 9.7% of the bachelor degrees awarded in computer science, according to the National Science Foundation.

“If the pipeline doesn’t lead anywhere, then all that work is for nought,” said Catherine Bracy, co-founder of the TechEquity Collaborative. “The people who come out of those programs need to be able to find jobs in the industry.”

Silicon Valley companies don’t want students with computer science degrees from just anywhere, said Leslie Miley, the director of engineering at Slack. The founders and hiring managers of Silicon Valley companies want students with degrees from the same schools they went to.

African American students hear Silicon Valley and they think, ‘I’m not going to see people like me' Juan Gilbert, University of Florida

“How difficult do you think it would be to go to an engineering meeting and tell all these people who went to Cal, Stanford, and MIT that the person coming from the University of Texas El Paso or a community college can do their job as well as they can?” Miley asked. “You will not be able to convince them of that. They don’t want to believe that they’re not special.”

The preference for an elite resume severely restricts the so-called pipeline – and results in a much less diverse group of candidates for Silicon Valley jobs.

In 2014, Wired analyzed LinkedIn profiles to come up with a list of the top five feeder universities for Microsoft, IBM, Google, Apple, Yahoo, Facebook, and Twitter. The thirteen US universities included the elite private schools Stanford, Carnegie Mellon, and MIT, as well as public schools such as UC Berkeley and the University of Washington.

One thing the schools had in common were student bodies with significantly fewer African-American students than the national average for four-year universities of 14%. Stanford had the highest rate of black students, at 7.8%.

Basic network factors also likely contribute to tech’s failure to find the underrepresented minorities in the pipeline.

“Tech is heavily referral-based, not just for jobs, but for funding,” said Y-Vonne Hutchinson, a diversity consultant and one of the founding members of Project Include. “For white people, 90% of their networks are white. You have this exponential deepening impact of homogeneity.”

Gilbert, whose program has more black computer science PhD students than anywhere else in the country, said that he’s also seen black undergraduates self-select out of Silicon Valley, in favor of working for companies on the east coast.

“Among the African American students that I engage with, I don’t hear many of them aspiring to work at Google.” he said. “They hear Silicon Valley and they think, ‘I’m not going to see people like me.’”