Yet another dubious report claiming to show widespread sexism in Silicon Valley has been published, and the tech press is characteristically lapping it up.

The survey, given the headline-friendly title “The Elephant in the Valley,” is scarcely academic, consisting of a collection of stories published on a purpose-built website. It nonetheless contains a number of eye-grabbing statistics: 90 percent of respondents claimed they had witnessed sexist behaviour at work, 87 percent reported demeaning comments from male colleagues, and 65 percent reported unwanted sexual advances.

Upon closer examination, however, the survey — and, by extension, its findings — is riddled with basic flaws.

The authors, who say they were inspired by Ellen Pao’s failed gender discrimination lawsuit against Kleiner Perkins, do not reveal how they selected the 200+ women who make up the survey’s respondents. Were they randomly selected, or were they friends of the authors? Gamers will no doubt remember the notorious Game Developers Conference (GDC) study on sexism in video games, which relied on the author’s Facebook friends and Twitter followers for responses.

It’s clear that the authors aren’t particularly interested in a robust methodology. The website encourages others to share their stories anonymously. Not only does this encourage a self-selecting sample, but the addition of anonymity also creates a tempting window of opportunity for hoaxers and trolls. With a selection of stories in each category of sexism picked out for maximum media impact, it’s hard to escape the impression that the authors are more interested in making headlines and appearing on podcasts than conducting a robust study.

All the authors tell us about the survey’s respondents is that they all have over 10 years’ experience in the industry, that 91 percent of them work in San Francisco’s Bay Area, and that 77 per cent of them are over 40. 25 per cent are C-suite executives, 11 per cent are founders, and a further 11 per cent are in venture capital.

In other words, the survey is hardly representative. Silicon Valley is a hotbed of progressivism, and the panic over sexism in tech primarily flows from activists and media types who are based there. Tech workers in less paranoid cities might be less inclined to see sexism behind the smile of every co-worker. The fact that 77 per cent of respondents are over 40 is also cause for concern: it means the survey tells us little about the experiences of younger female tech workers. Furthermore, it raises the question of whether these allegedly sexist incidents occurred recently or decades ago.

Another crippling failure of the survey is a weakness that plagues almost every hyped-up study on sexism or misogny: a failure to define terms. 90 per cent of respondents said they witnessed “sexist behaviour” – but we are not told how the authors defined “sexist behaviour,” or whether it was defined at all. 87 percent reported “demeaning” comments, but we are again not told what definition of “demeaning” was used.

The defining weakness of the survey, however, lies in its subjectivity. Respondents reported that they felt “excluded,” that they “weren’t given opportunities” that were granted to male counterparts, that they were told they were “too aggressive,” and that male colleagues made less eye contact with them than with other male workers. Another common complaint from respondents was that they were made to do “lower-level tasks” like ordering food and note-taking.

The problem with all of these, of course, is that they are precisely the sort of insecurities that employees of any gender might have in a hyper-competitive world like big tech. Whether women are more likely to report them (as they are in the cases of mental health, online harassment, and a range of other complaints) is an interesting question, but collecting accounts of bad experiences that could happen to either gender is no evidence of sexism.

Follow Allum Bokhari @LibertarianBlue on Twitter, and download Milo Alert! for Android to be kept up to date on his latest articles.