Lois, a medical researcher, endured more than five years of sexual harassment during her postgraduate study at a leading UK university. It started when she worked on a project between her MSc and PhD. The professor overseeing the research bombarded her and the other women in the group of junior researchers with crude and humiliating sexual comments.

“He said I looked so sexy in overalls that he had to resist the urge to rip them off me. One day it was raining and he came in wet and he announced that, him being our boss, he should make us dry him with our naked bodies,” she recalls. Sometimes she would scream to block out the lewd comments; on other occasions she and her female colleagues would “leave the room and lock ourselves in a bathroom before going back to work”. She adds: “Sometimes he would make a comment about someone’s arse, and you’d respond by not paying any attention; but sometimes when it was particularly bad you’d have to walk away.”

The harassment escalated when she began her PhD, even though she was no longer working in his lab. Now that he saw her only twice a week, rather than every day, he made a point of touching her every time he saw her and commenting on her appearance.

“Once I was talking to two postdocs in the corridor and he stopped and said to me ‘Turn around’, and I did and he looked me up and down and said ‘Mmm…’, perving on me. And everyone in his group – postdocs, MSc students, PhD students – just laughed.”

In a bid to hold her harasser to account, Lois wrote to the university’s provost and the director of her institute, and met with human resources and the women’s officer. The result? “The provost never replied. The old director passed it to the new director. The new director basically said this is the culture here and it’s difficult to change, and then asked when I was finishing [my PhD]. HR gave me two pages of the university’s policy on harassment and where to get counselling. The women’s officer took a month to reply and said that while institutes sign up for zero tolerance to sexual harassment, it’s difficult for them to implement it.”

Q&A Tell us: have you been affected by sexual misconduct in science? Show Have you had experience related to this story that you would like to discuss with our journalists?

You can share your experiences by filling in this encrypted form – anonymously if you wish. Your responses will only be seen by the Guardian and we’ll feature some of your responses in our reporting. You can read terms of service here.

Lois decided against making a formal complaint of sexual harassment, partly because she discovered one had previously been made against the same professor. She says: “I just knew it wouldn’t be worth it, and most likely nothing would happen. Very few people would back me up.”

But the main reason for her reluctance to make a formal complaint was being a junior woman in a small, male-dominated field. “I’m applying for jobs and really hoping they don’t know what happened at my institution. If I’m known as a ‘troublemaker’ they won’t want me. In a lab group there’s a very hierarchical system and they can prevent you from doing your own work. They can be very buddy-buddy – it’s not like a big office, where a manager may have more of an unbiased stance.”

Lois’s experiences are typical of more than 30 victims from the UK and the US who have told the Observer how the hierarchy, working environment and male-dominated culture of science, not only in their university but in their specialism/discipline, makes tackling sexual misconduct more complex and challenging than in other academic fields and industries. Not only can harassment have a devastating impact on the victim’s career and mental health, it often results in them leaving science altogether, depriving society of their contribution to solving important problems, from health to the environment and engineering.

A protest against sexual harassment at the University of Rochester, New York, in September 2017. Photograph: Rachel Jerome Ferraro/NYT

The extent of sexual harassment in UK university science departments is difficult to gauge. In 2014 the National Union of Students found that more than four in 10 students (across all subjects) reported suffering unwelcome advances and assault, including sexualised comments, inappropriate touching and rape. Although there has been no specific research into the incidence of sexual misconduct in science departments, many of the 200-plus victims who responded to recent Guardian appeals regarding sexual harassment by university staff were science students or junior academics, almost all of whom were women. We heard multiple detailed accounts of campaigns of sexual harassment by numerous professors at various institutions – and of toothless, negligent and often nonexistent disciplinary procedures.

In the US, a report published last month by the National Academies of Sciences, Engineering, and Medicine (the collective scientific national academy of the US) included a survey by the University of Texas System, which found that about 20% of women at undergraduate or postgraduate level in the sciences, 27% in engineering and 47% in medicine say they have been sexually harassed by members of staff or faculty.

Professor Carole Mundell, head of astrophysics at the University of Bath, is one of the senior academic scientists in the UK campaigning for universities to improve their handling of sexual misconduct. She says young female academics and students are particularly vulnerable due to the structure and hierarchy of lab working. “You [may] have privileged access to facilities or equipment, or a particular dataset, and that access comes through that supervisor [who is harassing you]. I’ve heard of cases where students have put complaints in and then been told they need to change projects. It’s the student that takes the hit for speaking up.

“Victims are often the least powerful people,” Mundell says, “and they’re often the ones carrying the heaviest load of doing the reporting, breaking the behaviour, educating everybody around them.”

As in many scientific fields, women in astrophysics face a challenge because there are relatively few of them: a survey by the International Astronomical Union in 2016 found only 17% of its members were female. “It is really incumbent on senior people to speak up,” says Mundell. “Academia is about networking. Students rely on their supervisors for letters of reference but also on networks for collaboration and career progression. So if there is a culture of misconduct, that also puts the fair use of those networks in jeopardy. It’s the victims who are having to adjust, not the perpetrators.”

‘It’s the victims who are having to adjust, not the perpetrators,’ says campaigner Carole Mundell, a professor at the University of Bath. Photograph: Adrian Sherratt/The Guardian

Dr Kate Clancy, of the anthropology department at University of Illinois, has carried out research into sexual harassment in scientific field trips. “There’s actual sabotage of people’s work, there is alienation and exclusion,” she reports. “We heard from a number of women who were denied access to fossils or other types of materials that they needed to conduct their work, and others who if they did not have sex with the field site director were not allowed to come back to the site.” In other cases, she says, women were only allowed to clean archaeological finds, rather than work on the excavations.

“Often, people are startled when you point out that it is actually the structure of research in academia that makes it so problematic,” says Dr Karen Kelsky, an academic coach, former professor and author of the book The Professor Is In, pointing to the long hours, isolation and late-night research many scientists have to undertake. Another problem is science’s strict hierarchy – the pecking order is intractable, with the principal investigator, usually a professor, at the top. And the power they wield is immense. “There is a complete dependency, in a way that there isn’t in the corporate world, on the people who are above you,” she says. “[Academics and committees] have to pass your comprehensive exams; they have to pass your dissertation proposal; they have to pass your defence of your dissertation; they have to write you letters for your first job, they have to write you letters for your funding – at all of those stages you are vulnerable. If they say no, you have no recourse; there is nobody else you can substitute for them to write that letter for you.”

Kelsky said another problem is the “mythos” around intellectual work. “Academia absolutely has a belief in its own exceptionalism. Academics believe that they are sort of immune to the strictures of the normal business world,” she said. “It is analogous to the tortured artist, where you have to be given free reign to be as outrageous as you want in order for you to have the freedom to do your great work.”

In the US a number of high-profile scientists from prestigious institutions have been investigated following allegations of sexual misconduct in recent years – some are calling it science’s “Weinstein moment”. In 2015, Professor Geoffrey Marcy of the University of California, Berkeley was forced to resign after an investigation into sexual harassment. Allegations included kissing and groping them. In late 2016, palaeoanthropologist Brian Richmond resigned from the American Museum of Natural History after facing allegations including sexual assault. And last year, geologist David Marchant was sacked by Boston University after it concluded he had sexually harassed a student on a field trip to Antarctica two decades ago. Just last month the celebrated evolutionary biologist Francisco J Ayala resigned from the University of California after a month-long investigation into sexual harassment allegations.

For the past two years, the department of brain and cognitive sciences at the University of Rochester in New York state has been embroiled in a battle over the conduct of Professor Florian Jaeger, who is accused of sexually harassing students and entering into inappropriate sexual relationships with them. The allegations have led to a discrimination complaint to the Equal Employment Opportunity Commission and a federal lawsuit, which is ongoing, the resignation of the university president and one of the department’s founding members, and the departure of six other members of the faculty.

In 2016, an internal investigation cleared Jaeger of violating the university’s policy on discrimination and harassment. But in January 2018 a subsequent investigation, commissioned by the university into the allegations and into its own handling of them, concluded that Jaeger had multiple sexual relationships with students and engaged in “inappropriate, unprofessional and offensive” behaviour with colleagues and graduate students between 2007 and 2013. However it didn’t find him in breach of university policies regarding staff-student relationships, a policy they overhauled and tightened up in 2014. Jaeger apologised if his “former behaviour made some students uncomfortable and may have discouraged them from working with me”.

Geoffrey Marcy, University of California astronomy professor, resigned after being found guilty of harassing students. Photograph: Alamy

The nine complainants (who were not spoken to for the investigation) described its findings as “ridiculous”. The complaints against Jaeger ranged from him approaching female students for sex in bars to making others feel so uncomfortable that they abandoned their research to avoid contact with him. One of the complainants, Professor Celeste Kidd, says: “There was a daily barrage of unwanted commentary about his sexual desires for other people, including students. The clear goal of what he did to me and to others was to overwhelm us with constant criticisms that were designed to humiliate and intimidate us. The comments weren’t subtle. For example, he complained about the taste of my classmate’s vagina.”

Kidd sought to protect herself by working in locked rooms and changing the focus of her research. She says: “Science presents unique challenges for providing protections to victims, because it needs to go beyond just the particular department they’re in. Your place of work is not just your institution, it’s your field.”While Jaeger is set to resume teaching in the autumn, most of the complainants have now left the department, citing the university’s response to their allegations and the embittered divisions among the faculty as the reasons for their departure. Among those who left was the department chair, Professor Richard Aslin, who helped steer the department to become the fourth-ranked in its field in the US. He says he is still shocked by the attitude of some former colleagues who blame him and his fellow complainants, rather than Jaeger or the university’s administration, for the crisis in the department. “There are faculty members I’ve known for 20 years who have not spoken to me for nine months,” he says.

“Even when [victims] do speak up,” says Kelskey, “the colleagues of the powerful band together to protect the powerful. And that includes women, unfortunately. [Academics] are not respected by society as a whole, especially in the US, and they aren’t paid all that well, given their level of education – so prestige is all they have, and the power that they have over their underlings. And so they are going to defend that to their dying breath.”

Some leading UK science institutions have introduced reforms in an attempt to improve the handling of sexual misconduct. In 2015, Imperial College London commissioned a review of its institutional culture and gender equality, following an investigation into the sexist treatment of the college’s women’s rugby team. The review, by Dr Alison Phipps of the Centre for Gender Studies at the University of Sussex, published in December 2016, found many examples of bullying and discrimination towards staff and students, with women in particular reporting being silenced in various ways. Relationships between principal investigators, who were usually male, and junior researchers, who were more often women, were reported to be especially difficult.

Prof Stephen Curry, assistant provost for equality, diversity and inclusion at Imperial, says a women’s refuge has carried out an evaluation of all their policies and procedures on sexual harassment, which is expected to be delivered to the university within a few weeks. At the moment, he says, “people don’t know who to turn to. We need to more clearly signpost what support is available and build credibility so people have confidence in coming forward. One of the problems is we don’t really have a [sense of the] scale of the situation, but there’s little doubt it’s worse than we suspect,” he says.

“Woman after woman says ‘I am afraid to be alone in a room with a man’​. How are you going to have a productive career?

In a bid to address this, he says Imperial is likely to adopt the anonymous reporting system pioneered by Cambridge University, which led to a rise in the number of recorded incidents from a handful to more than 150 in a single year.

Other leading university science departments are accused of failing to deal with the problem. Last year, University College London established a Preventing Sexual Misconduct Strategy Group to review its policies and procedures on how it handles complaints. However, victims and other members of the university say the reforms have yet to make an impact. In late June a letter signed by 465 UK and international academics was delivered to the provost and president of UCL, Michael Arthur, calling on management to reform their procedures to address sexual misconduct so “that survivors are protected and harassers are sanctioned to the greatest possible extent”. The letter states: “We fully support all those who come forward and in particular those who have reported at UCL. We recognise the profound personal and professional cost of being harassed, and the acute burden that is imposed on survivors by current reporting and disciplinary procedures. Reform of institutions and the community is urgently required.”

Professor Arthur says he is now personally overseeing the new strategy to address sexual harassment at UCL. This will include a campaign to combat bullying, harassment and sexual misconduct in the autumn, and an anonymous reporting system for sexual misconduct. “We’ve got a significant problem that we need to address,” he says. “My sense is we’ll probably find some hotspots [due to anonymous reporting] and if we do we’ll tackle that head on ... through the head of department and investigate appropriately.”

Arthur expects a new process for handling sexual misconduct complaints will be brought in as current statutes and procedures are not suited to tackling the issue, and can give more support to the accused than the accuser. “We’ll put our hands up to say we’ve had some difficult cases and we’ve had to learn from those cases about what we need to do differently,” he adds. “[The process] needs to be faster, to be more supportive of the complainants, [and] less adversarial, so that the complainants can give evidence in a way that’s non-threatening.”

The exodus of women in science as they climb the academic rungs is well known – and dubbed the “leaky pipeline”. But while the trend had prompted much hand-wringing, and even research, experiences of sexual misconduct are rarely raised as a contributing factor.

A 2008 report from the Royal Society of Chemistry into women’s experiences of graduate study revealed that while women had worse experiences of studying for a PhD than men – including significant supervision issues, isolation and exclusion in the research group, problems with a macho culture and being advised that they would face significant challenges in embarking on an academic career because of their gender – there was no mention of sexual misconduct.

Dr Jessica Cantlon (left) and Dr Celeste Kidd, two of nine women who accused Dr Florian Jaeger of sexual harassment. Photograph: Rachel Jerome Ferraro/NYT

Dr Sean McWhinnie is, an independent consultant who coordinated the research, and is also involved with the Athena SWAN charter for women in academia. He says the absence of any mention of sexual misconduct in the report, which involved telephone interviews and focus groups might have been because participants were not prepared to raise issues around sexual harassment in a group environment. “I am not going to deny that it happens, but I am afraid, to be honest, I don’t think we necessarily sought it and it certainly wasn’t reported to me that it was raised as an issue.” But, he adds, “nobody has ever come to me in the work I have done over the years and talked about sexual harassment.”

When we asked Chris McEntee, executive director of the American Geophysical Union, if the AGU believed sexual misconduct helped fuel the leaky pipeline, she was unequivocal. “Yes, we do,” she said. While not the only factor, she said that the loss of women from science was a huge issue, particularly given the need for research to tackle issues facing the planet. “A mind is a terrible thing to waste,” she said.

The report by the US National Academies of Sciences, Engineering, and Medicine also acknowledged the link: “What’s especially discouraging about this situation is that at the same time that so much energy and money is being invested in efforts to attract and retain women in science, engineering and medical fields, it appears women are often bullied or harassed out of career pathways in these fields,” the authors write.

If university procedures are slow-moving and unconvincing, perhaps the answer lies in hitting them where it hurts: their pockets.

Clancy is among those pushing for stronger sanctions. In 2016 she teamed up with Democrat representative Jackie Speier of San Francisco to pass a bill that would allow funding bodies to take into account whether applicants have a record of sexual misconduct.

“The bill didn’t really go anywhere. But then very recently the National Science Foundation, one of our federal funding agencies, decided to just do it on its own,” said Clancy. The upshot, she says, is that the NSF now requires institutions to report sexual harassment findings.

In late 2017 the American Geophysical Union announced that among its policy changes, sexual harassment will now be considered a form of scientific misconduct. McEntee said the decision came following a meeting with members after a spate of high profile cases of sexual harassment in the sciences in 2015. “Research has shown us there is a profound destructive effect that harassment, discrimination and bullying can have, not only on the people directly involved, but on their ability to do research, on their institutions, the students and the faculty and colleagues surrounding that misconduct,” McEntee said, noting that just as falsifying data harms high-quality research, so too does such behaviour. “That is why our view is that we are on firm ground to call that scientific misconduct,” she said.

Institutions in the UK are beginning to follow suit. Among them, in May, the Wellcome Trust announced that universities must now report findings of misconduct investigations, and that the trust will consider withholding funding from academics who have been found guilty of bullying or harassment.

In the wake of the Harvey Weinstein scandal, Dr Karen Kelsky decided it was time to gauge the extent of the problem across academia. She set up a spreadsheet, allowing individuals to share their experiences.

The results are staggering: more than 2,400 entries, and counting, of detailed testimony – some recounting violence, others manipulative behaviour. The allegations span universities and all fields of research, science included, and include sexual harassment by faculty members.

“Conference – male PI [principal investigator] put his hand down my pants,” writes one plant science researcher. A biologist recalls having to give a massage to a professor in front of others, while an anthropology student reports being sent emails by her professor asking her to perform sex acts. An archaeologist talks of how a field director pressed his body against hers at a social event and carved her name on an ancient artefact. The list goes on. And on.

It is not just about the immediate impact, Kelsky stresses, pointing to a section of the spreadsheet that looks at the repercussions for victims. “Look at my spreadsheet, look at that column,” she says. “Woman after woman says ‘I am afraid to be alone in a room with a man’… How are you going to have a whole, productive career when you are terrified every time you end up alone in a room with a man? You are so debilitated.”

The ramifications – mental, physical and professional – are striking, ranging from anxiety, depression and post-traumatic stress, to individuals abandoning their dream career and even turning to drink and drugs.

“The biggest surprise, the biggest insight that I gained from doing this spreadsheet is that it is really a false dichotomy to claim that there are serious incidents and less serious incidents, and you can say ‘Well stalking and rape, those are bad, but continued daily comments about the size of your advisee’s breasts, that those are not bad’,” says Kelsky.

What the spreadsheet shows, she says, is that society suffers when women are not free to do their best work. “Let’s not fetishise the “poor man”, the “great man” whose work might be curtailed. Let’s not focus on him, I’m not interested in him – he has been the focus for 2,000 years,” she said. “What I am interested in are the 40 women or the 100 women or the 1,000 women – and we don’t know – that that one man may have victimised and what did they not do because of him.”

Some names and identifying details in this article have been changed to protect the privacy of individuals.