An inter-state police force that serves a polygamous fundamentalist Mormon community won’t be required to disband—despite alleged discrimination against nonbelievers, surveillance of dissidents, and facilitation of child abuse.

Instead, officers in the Colorado City Marshal’s Office will be required to attend annual training sessions to ensure they comply with federal laws and don’t discriminate against the community’s non-religious minority. The Marshal’s Office has been serving the adjoining towns of Hildale, Utah and Colorado City, Arizona since 1985.

Together, the two towns make up Short Creek, a 7,500-person community that’s home to the Fundamentalist Church of Jesus Christ of Latter Day Saints, a polygamous offshoot of the Church of Jesus Christ of Latter-Day Saints that splintered from the Mormon church when the latter renounced plural marriage at the beginning of the 20th century.

The U.S. Department of Justice called for the disbanding of the CCMO as part of a lengthy anti-discrimination lawsuit against the twin towns that comprise Short Creek that began in 2012. According to the complaint, the community’s powerful FLDS leaders denied non-members access to housing, police protection, and public services including water and electricity.

In March 2016, a jury determined the community’s leadership had discriminated against non-FLDS members and awarded $2.2 million in damages to six Short Creek residents. The Justice Department then demanded the federal government disband Short Creek’s police force, which it claimed was enforcing the church’s discrimination against non-members.

Prosecutors claimed the Marshal’s Office was using its “state-granted law enforcement authority” to “carry out the will and dictates” of Warren Jeffs, the infamous FLDS leader who is currently serving a life sentence for sexually abusing two young girls whom he called his “spiritual wives.”

In October 2005, marshal Fred Barlow wrote a letter to Jeffs, a fugitive at the time. “I want to fill the position that you would have me fill and do the job the way you would like it done,” Barlow wrote. “We will continue to do that directive unless you would like us to do something different.”

Jeffs continued to lead the church from behind bars, and according to the suit, the practice of older men marrying underage women did not end with his arrest. Instead, it was allegedly ignored—and even practiced—by the Marshal’s Office, which allegedly continued to carry out Jeffs’ orders after his arrest.

In 2007, Barlow was decertified as a marshal for failing to assist with an Arizona Attorney General investigation. An administrative law judge who reviewed Barlow’s decertification determined he “displayed bias in favor of Warren Jeffs and the FLDS, in derogation of his oath to neutrally enforce the law,” according to the civil complaint.

Helaman Barlow, who served as a marshal for 20 years, testified in February 2016 that he was instructed to be a protector of the church, not a police officer. When Barlow began his tenure at the CCMO in 1994, he asked then-FLDS leader Rulon Jeffs if he had any advice for a new cop. Jeffs allegedly told him, “Number one, you aren’t a cop. You are a peace officer. Number two, your calling is between the church and harm,” Barlow testified, as reported by Courthouse News.

Barlow served as chief marshal between 2012 and 2014 and said that under his leadership, the Marshal’s Office didn’t investigate claims of FLDS members—including Colorado City’s former mayor Joseph Allred—marrying underage girls.

“If it was a church marriage, I as a church member saw it as a valid marriage,” Barlow testified. When the local sheriff’s office began to investigate allegations of child marriage in the FLDS, Barlow said it “felt like he was there to attack our marriages, our beliefs.” Barlow left the church in 2014.

Local authorities also claimed the Marshal’s Office actively interfered in their investigations, including child custody disputes. “When they know that we are coming, those children are moved, making it harder to find them,” Washington County Sheriff Cory Pulsipher testified in October 2016. James Schoppman, special counsel for the Mohave County Sheriff’s Office, reportedly testified to a “history of abuse” by the Marshal’s Office. Both sheriffs offered to fill the void in the event that the Marshal’s Office were disbanded.

But on Tuesday, a federal judge ruled it would be “unreasonably expensive” for the Marshal’s Office to disband.

“Disbandment would approximately double the cost of law enforcement for the Short Creek community or necessitate a reduction in police protection,” U.S. District Judge H. Russel Holland declared. The Marshal’s Office won’t, however, be allowed to return to business as usual either. “The CCMO’s Chief Marshal and officers are plainly in need of training and supervision,” Holland added.

The Marshal’s Office now has approximately three months to hire both an outside consultant and an independent mentor for the chief marshal.

“The changes the judge is requiring from the police department are changes the department had begun to make anyway, in terms of hiring a consultant to help them change their policies,” Jeff Matura, lead attorney for Colorado City, told The Daily Beast.

“From the beginning of this case, the federal government sought to disband the local department,” Matura continued. “The judge’s decision to deny that request is not only a victory for the local residents, but really a victory for all small, regional municipalities throughout the country who are fighting against the overreach of the federal government into their daily affairs.”

Lawyers representing Colorado City and Hildale—including Matura—unsuccessfully tried to get a judge to bar evidence of polygamy, underage marriage, and church teachings from the trial, the Associated Press reported last year.

Matura also denied any allegations of ongoing child abuse and underage marriage. “We hear that allegation all the time. We had an eight-week trial last year; there was no evidence of that,” Matura told The Daily Beast.

Regarding Warren Jeffs’ continued power over the Marshal’s Office, Matura said, “The CCMO has undergone a great transition over the years and is now moving forward with the Court’s order to ensure that all residents, regardless of their religion, have confidence in the police services they will receive.”