A tax scam aimed at stealing consumer refunds has ballooned, with the IRS reporting that the number of potential victims has soared from a few hundred to "several thousand in just days."

Uncovered in the first week of filing season, the con appears to have started in tax preparers' offices, where infected computers provided access to complete return data on thousands of consumers. The crooks are using the purloined data -- including Social Security numbers, dependents data and even bank account information -- to file refund claims on behalf of the victimized taxpayers.

The IRS does not know how many taxpayers will ultimately be affected. But in a sign of how seriously it's taking the scam, the agency on Tuesday issued a second notice to tax preparers urging them to notify the agency, the FBI, local police and their clients as soon as they discover a data breach.

Get Breaking News Delivered to Your Inbox

"Speed is critical," the agency said in its advisory. "If reported quickly, the IRS can take steps to block fraudulent returns in a preparer's clients' names."

Because the refunds are going to the victimized taxpayers -- sometimes by direct deposit into their bank accounts -- the crooks then contact the victims by phone.

Posing as IRS agents or collection agents working for the IRS, the criminals demand that the refund money be returned. In one version of the scam, a live agent calls the taxpayer posing as a debt collector, who tells the taxpayers that the fraudulent refund money should be sent to the debt collection firm.

Alternatively, the taxpayer may receive a recorded call charging the taxpayer with fraud, threatening the victim with arrest and a "blacklisting" of their Social Security number. The recorded voice then tells the taxpayer to call a number for information on returning the refund.

Ironically, while the IRS is not calling anyone, the agency does want victimized taxpayers to return the improperly-issued refunds. The ideal, if the refund was direct-deposited to your bank, is to have your bank's automated clearinghouse department send the funds back to the IRS -- after you call the agency on its toll-free number (800-829-1040 for individuals or 800-829-4933 for businesses) to let them know why the money is coming back.

If you got a direct-deposited refund that you did not expect, you should also talk to your bank and potentially close that account, since your account information has been accessed by crooks.

If you received a paper check for a tax refund that you did not expect, the best answer is to write "VOID" on the front of the check and return it promptly, possibly with your paper filed refund and IRS Form 14039 -- the identity theft affidavit -- to the same regional service center that the check came from (which is likely the same center where you'd need to send a paper return). If you send the check without your return, be sure to include a note, saying that you were a victim of identity fraud.

Finally, if you cashed an improperly issued refund, the IRS suggests that you send a personal check to your local IRS service center with a notation that it is repayment for an erroneous refund. Be sure to include an explanation that you were a victim of this tax preparer refund scam.

The IRS was unable to answer a number of questions on Tuesday, including how victim refunds were deposited in bank accounts within hours of the launch of filing season. The agency did not start accepting tax returns until January 29. It normally takes at least 10 days for refunds to be issued by direct deposit -- the fastest means. But in this case, the IRS was already warning about refunds erroneously deposited in consumer bank accounts just three days into the filing season.

That has led some to wonder whether the crooks may have been aided by dishonest IRS insiders.

IRS denied the implication. "There is absolutely no indication this information originated from ‎IRS data or systems," the agency said in a prepared statement. "Any speculation to that effect is simply wrong."

How did returns get processed so rapidly? According to the agency: "We test our systems prior to the official opening of filing season and accept some returns for processing; however, refunds are not issued until after the start of filing season."

One tax preparer, who asked not to be named for fear of retribution, noted that several new clients were impacted by the scam, some of whom had erroneous refunds deposited in their accounts on January 30th -- just hours after the IRS started accepting returns.

"That seems impossible unless this is an inside job," he said.

Tax authorities have been battling identity theft for years and even have a consumer publication on the topic -- the Taxpayer Guide to Identity Theft -- that provides a step-by-step guide for affected consumers. Among the unpleasant surprises for victims is that their tax refunds are likely to be delayed for months.

IRS spokesman Terry Lemons previously told CBS News that it was likely to take four months to get affected taxpayers their refunds.