Five pregnant schoolgirls in Tanzania were arrested so law enforcement could interrogate them about the men who impregnated them, officials said.

The schoolgirls, whose names and ages were not disclosed, were arrested last week. They were later released on bail, Tandahimba District Commissioner Sebastian Waryuba said late Monday.

He said people are looking to question two other girls in the district, which is on the Mozambique border.

The East African nation has among the toughest laws against those who impregnate school-aged girls, sentencing them to at least 30 years prison. Officials said the penalties are intended to discourage early marriages and teenage pregnancies that lead to school dropouts.

But critics say that in its zeal to curb teenage pregnancies, the country is also hurting and shaming the vulnerable teenagers. Human rights groups have criticized authorities in the country for implementing policies that don’t allow schoolgirls to return to class after they’ve given birth.

In 2002, a law was passed that allowed schools to expel girls for “offences against morality” and “wedlock,” Sky News reported.

Human Rights Watch said many girls regularly experience sexual harassment and exploitation by teachers in school and that schools lack adequate protection and confidential reporting mechanisms.

Last year, Tanzania’s president faced widespread criticism after sending out a threatening warning to school-aged girls.

“After getting pregnant, you are done,” he said, according to Sky News.