CHARLESTON, S.C.—A first-of-its-kind fraud prosecution of a small technology company and its owner has shed light on how the architecture of the internet allows spammers, hackers and other bad actors to flourish online while cloaking their true identities.

Amir Golestan and his company, Micfo—a web-services provider based above a cafe in a building in this city’s historic district—face 20 counts of wire fraud in a case brought in U.S. District Court in South Carolina. Both Mr. Golestan and the corporation have pleaded not...