WASHINGTON—Law-enforcement agencies are increasingly finding ways to unmask users of a popular Web service designed to hide identities and allow individuals to exist online anonymously.

To keep their identities secret, users and administrators of a recently shuttered child-pornography website used a network called Tor that obscures the source of Web traffic, authorities said in March. Agents from Homeland Security Investigations tracked many of them down anyway, largely because of mistakes that even some of the most sophisticated users eventually make.

Tor and other programs designed to hide users' identity online have grown in popularity as people try to protect their privacy in an age of digital surveillance. When paired with bitcoin or other virtual currencies that don't use the banking system, Tor can help hide the identities of people behind financial transactions. Such programs also have become a tool for those seeking to evade the law, including child-pornography traders, hackers and other criminals, creating challenges for law enforcement.

But officials are becoming more confident that Tor's shield of anonymity isn't impenetrable.

"There's not a magic way to trace people [through Tor], so we typically capitalize on human error, looking for whatever clues people leave in their wake," said James Kilpatrick, one of the HSI agents who is part of Operation Round Table, a continuing investigation into a Tor-based child-pornography site that has so far resulted in 25 arrests and the identification of more than 250 victims, all children.