Few government agencies routinely collect more information on every law-abiding American than the Internal Revenue Service. And by targeting the IRS, a group of seemingly sophisticated hackers has now collected their own chunk of that detailed data.

On Tuesday the IRS admitted that it had been the target of a breach that compromised 100,000 taxpayers' files between February and the middle of this month. And though that may seem like a relatively small set of victims compared with recent breaches like the one affecting Target or the health insurer Anthem, the IRS says the attackers gained the full tax return transcript of the affected taxpayers, which could included a detailed dossier of their personal information including income and social security numbers. And that may have been enough to pull off a tax-refund-stealing scheme first reported in March.

Just as disturbing than the information stolen, in fact, may be the intel the attackers appear to have possessed about their targets before the breach. According to the IRS statement, the attackers used a collection of detailed personal information from another "non-IRS" source to crack the authentication process on a "get transcript" feature on the IRS website.

"The IRS announced today that criminals used taxpayer-specific data acquired from non-IRS sources to gain unauthorized access to information on approximately 100,000 tax accounts through IRS’ 'Get Transcript' application," reads the IRS's statement. "These third parties gained sufficient information from an outside source before trying to access the IRS site, which allowed them to clear a multi-step authentication process."

According to the IRS, that means the hackers had prior access to their targets' "Social Security information, date of birth, tax filing status and street address" even before they began their attack, as well as the answers to "several personal identity verification questions that typically are only known by the taxpayer."

With the information from the victims' tax transcripts, the hackers may attempt to redirect tax refunds to their own bank accounts. Blogger Brian Krebs reported in March on a scam in which hackers stole one of his readers' tax refund by obtaining his tax transcript and using its data to file a fraudulent tax return.

Aside from the potential to pull off a possible tax-return theft scheme, it's not clear yet whether any credit card numbers or any other directly exploitable financial information was included in the breach. The government agency says its criminal investigation division is looking into the attack. And it will alert by letter the 200,000 taxpayers who were targeted in the breach—the attackers only successfully cracked the "get transcript" authentication in half of those cases—and will pay for credit monitoring for the 100,000 known victims of the data spill.

Given the detailed dossier of information the attackers possessed before they touched the IRS site, anyone who receives one of those letters faces an uncomfortable realization: Not only did hackers access your full tax transcript, but they'd likely obtained much of your personal data months earlier. Better, perhaps, to have the rude awakening of a breach disclosure letter from the IRS than to let those intruders exploit your information in secret.