KUNMING, China—The email attachment would tempt anyone following the diplomatic standoff between China and other countries in the South China Sea. The Microsoft Word document contained text and photos depicting Thai naval personnel capturing Vietnamese fishermen and forcing them to kneel at gunpoint.

But the attachment was a decoy: Anyone who opened it inadvertently downloaded software that searched their computers for sensitive information and sent it to an obscure corner of the Internet. Manning that corner, according to...