When prisoners at a notorious Alabama women’s prison came forward to complain of sexual abuse and harassment, state investigators time and again classified the complaints as unfounded or unsubstantiated and often recommended that the matters be closed without further action, according to investigative reports obtained by the Associated Press.

In only a small fraction of cases in the past three years did corrections officials consider the allegations fully credible, according to roughly four dozen reports released to the AP under the state’s open records law.

In a scathing report this year, the Justice Department chronicled an “unabated” culture of sexual abuse and harassment at the Julia Tutwiler prison for women, where prisoners “universally fear for their safety” and guards are not properly disciplined for bad behaviour.

Yet state investigators who looked into the allegations of misconduct, usually involving corrections officers, repeatedly categorised the complaints as unfounded or said there was insufficient evidence to move forward.

Their conclusions are difficult to square with a federal inquiry that painted the prison as a cauldron of sexual violence, and they raise questions about whether more should have been done in at least some of the cases to get to the bottom of prisoner allegations. The thoroughness of the state’s investigations is among the concerns addressed in the Justice Department report, which said investigators “often fail to use basic investigative tools and techniques” and close cases prematurely with limited evidence.

The Alabama corrections commissioner, Kim Thomas, said his department thoroughly investigates abuse allegations, noting investigators work for the state rather than the prison and are therefore independent. He said the state strongly encourages prisoners to report abuse and that allegations, even if unsubstantiated, are forwarded as a matter of routine to the local district attorney for review and potential prosecution.

The state investigative reports represent only a “snapshot in time” of the records the Justice Department had access to in making its findings, he said, calling the prison much improved since then.

“Nothing like that is ever perfect. It’s never going to be perfect, but it’s not going to be because we have our heads in the sand,” Thomas said.

Problems at the prison in Wetumpka, Alabama, a maximum-security facility that houses death-row prisoners, are well documented. The Justice Department in 1995 warned of unconstitutional medical and mental health conditions and in 2007 ranked Tutwiler as the women’s prison with the highest rate of sexual assaults in the nation. Even before this year’s report, Thomas issued 58 directives for improvement.

Justice Department investigators visited in April 2013, interviewing dozens of prisoners and staff and reviewing disciplinary reports, logs and other documents. Their January report was unsparing.

It said staff raped, fondled and harassed prisoners with impunity over the previous two decades, invaded their privacy in showers and bathrooms, made sexual advances and created a “toxic, sexualised environment”. The report said multiple women, with no opportunity to collaborate, told a “markedly similar story”.

“They don’t come into states and go this wide and this deep unless there is pretty convincing evidence of a pattern and practice of abuse,” American University law professor and prison violence expert Brenda Smith said of the federal scrutiny.

The AP reviewed 49 investigative reports between 2011 and July 2014 regarding a broad variety of misconduct, including graphic sexual comments and inappropriate touching. Most of the allegations concerned corrections officers or other prison staff, though a few were made against other prisoners. One woman alleged she was abused at a hospital where she had been taken but later retracted the allegation.

Of the complaints, at least 42 are categorised as either “unfounded” or “unsubstantiated” or investigators made clear in summarising the complaint that they did not consider the allegations fully credible. Most of the ones deemed substantiated involved inmate-on-inmate allegations.

The reports, many of them multiple pages, offer varied reasons for cases being closed. Sometimes prisoners who reported abuse later recanted allegations, scored erratically on lie detector tests or were described as mentally unstable. Other cases lacked corroborating witnesses. In some instances when someone anonymously would accuse a guard of having sex with a prisoner, both would deny it and the allegation was dismissed.

Corrections officials investigated a complaint from a prisoner who said a guard made crude sexual comments. But the officer denied it and the matter was closed. In another case, a prisoner said an officer stared at her in the shower as she dried off, and at others in the bathroom. The case was closed after the officer denied wrongdoing and other witnesses did not corroborate it.

It is impossible to say whether more allegations should have been deemed credible. But the Justice Department report strongly criticised the state’s investigative process, saying officials were quick to close out inquiries because of uncooperative accusers, too often failed to seek out witnesses or check verifiable facts and relied too heavily on prisoner polygraphs.

The state says it regularly refers cases for prosecution, though juror doubts about prisoner credibility can make convictions difficult. CJ Robinson, an Elmore County prosecutor, recalled one case this year that ended with a guard’s acquittal after jurors apparently did not believe the prisoner.

“When you start talking about an inmate, it becomes increasingly difficult. I’m not saying every single case, but in most cases, the inmate is either hesitant to cooperate, or they ask for something in exchange for telling the investigator what happened,” he said.

Thomas said the department has made multiple improvements in the investigative process, including training sessions, streamlining investigations so the accuser does not have to tell her story repeatedly and encouraging unannounced visits from professional coordinators.

“I want each allegation investigated thoroughly and for us to take some appropriate action [on] those,” he said.