After superstorm Sandy hit the east coast nearly two years ago, the federal government quickly sent out $1.4bn in emergency disaster aid to the hurricane’s victims.

Now, thousands of people might have to pay back their share.

The Federal Emergency Management Agency is scrutinizing about 4,500 households that it suspects received improper payments after the storm, according to program officials and data obtained by the Associated Press through a public records request. As of early September, Fema had asked around 850 of those aid recipients to return a collective $5.8m. The other cases were still under review.

Fema’s campaign to recover overpayments, called “recoupment” in agency lingo, typically involves instances where the agency believes a household got more money than allowed under program rules, but not necessarily because of an intentional attempt to cheat the system. Fraud cases are handled separately.

Many people asked to return money were deemed ineligible because their damaged properties were vacation houses or rental properties, not their primary residences. Others had double dipped into the aid pool, with more than one household member getting payments. Some received Fema money for things later covered by insurance.

As of 30 July, the average demanded refund was $6,987, a sum that could be difficult for many, given the modest annual incomes of most aid applicants. Roughly half of the households under scrutiny reported an annual gross income of $30,000 or less.

Hugh L Carey Tunnel sits flooded after a tidal surge caused by hurricane Sandy, on October 30, 2012. Photograph: John Moore/Getty

The larger pool of cases still under review as of that date involved $53m in aid payments – or about 3.7% of the total given out by Fema through its individuals and households program – though any potential refunds would likely involve only a portion of that money.

“For most people, the money is long gone and long ago spent on storm recovery,” said Ann Dibble, director of the New York Legal Assistance Group’s storm response unit, which has been helping about a dozen families fight a Fema clawback.

Twenty-five cases under review involved households reporting a gross annual income of $500,000 or more.

The list of people asked to return cash includes Gary Silberman of Lindenhurst, New York, who got a letter last November demanding just under $17,000. The agency said he was ineligible partly because he and his elderly father, Albert, had both applied for disaster funds even though they were living together. That’s a violation of Fema double-dipping rules.

The Silbermans also were barred from getting some types of aid because they had failed to buy flood insurance after getting $25,000 in Fema aid for flood damage during Hurricane Irene a year earlier. Federal rules require homeowners who accept disaster aid to buy insurance to protect them from future storms.

Silberman says he should still qualify for the money because he was a rent-paying tenant in his father’s house, not a dependent, but Fema has so far rejected his appeals.

People gather around the remains of burned homes after superstorm Sandy on October 31, 2012 in Queens, New York. Photograph: Mario Tama/Getty

“I lost my home. I lost everything. I don’t have $17,000 to give back,” Silberman said.

Sandy was among the costliest hurricanes in US.history. More than 280 people died in the US and the Caribbean. When the storm struck the New York and New Jersey coastlines, the surging ocean poured into densely populated seaside neighborhoods, flooding hospitals and subway tunnels and turning entire communities into soggy, moldy wrecks.

About 179,000 households in New York and New Jersey received Fema payments for temporary housing or damaged property. Most of the recoupment actions have also been aimed at residents of those states, but the agency is also reviewing payments to households in Connecticut, Maryland and Rhode Island.

Fema mobilized for Sandy hoping to avoid problems that plagued the aid distribution process following Hurricane Katrina’s strike on the Gulf Coast in 2005. That destructive storm forced the overwhelmed agency to relax internal controls to speed up relief efforts, which led to huge numbers of people getting money they shouldn’t have received.

People walk down a flooded street in Atlantic City, New Jersey. Photograph: Mario Tama/Getty Images

Fema’s attempts to recover hundreds of millions of dollars, often from people who couldn’t afford to pay, led to a court fight and a procedural overhaul. By 2011, the agency had mailed out letters to at least 90,000 households asking for aid refunds. Congress authorized the agency to waive much of that debt.

The agency says it has since gotten better at making sure aid only goes to the right people, and in proper amounts.

“They have a lot more controls in place,” said John Kelly, the Department of Homeland Security’s assistant inspector general for emergency management oversight.

In most cases, aid recipients won’t know they’re under scrutiny before a notice arrives in the mail demanding repayment. Fema says it has reviewed thousands of cases and decided the payments were warranted.

Fema isn’t the only government agency trying to recover Sandy-related funding.

Last spring, the US Department of Housing and Urban Development urged officials in New Jersey and New York to step up their scrutiny of hundreds of millions of dollars in other federal rebuilding grants.

New Jersey’s Department of Consumer Affairs says it has asked 51 homeowners to return a combined $940,000 – a number that will probably will rise as the state looks at more cases.

Manhattan in flood water after Hurricane Sandy. Photograph: Christos Pathiakis/Getty Images

Ted Friedli got a letter from the state in July giving him five days to return a $26,101 grant he’d received to repair his flooded house in Long Branch, New Jersey.

The letter explained that auditors had pegged the cost of fixing the house and raising it up on a higher foundation at less than the family had already received from their insurance policies and a low-interest federal disaster loan. That meant that the state grant was pure surplus and had to be returned.

“I was stunned,” Friedli said of his reaction when he got the notice, which he said came with no warning and no mention of any possibility of appeal.

New York officials were unable to say how much money they had recouped so far from grant recipients who had been overpaid or belatedly declared ineligible.

During one audit earlier this year, HUD officials warned that New York wasn’t doing enough to ensure that rebuilding grants weren’t “unduly enriching homeowners beyond their actual expenses.”

It cited several examples in which checks had been sent improperly. One homeowner returned a check for $119,138 after realizing that it included payment for damage covered by insurance. A month later, she got another letter saying she had been approved for an even larger overpayment of $121,731.

Office of Storm Recovery spokeswoman Barbara Brancaccio said New York state has since improved its record-keeping procedures.

Federal prosecutors have, so far, brought far fewer fraud cases against aid recipients than they did after Katrina, though it is unclear whether that is because less theft has occurred or because investigators haven’t been as aggressive.

The aftermath of Hurricane Katrina in New Orleans in 2005. Photograph: Larry W Smith/EPA

By Katrina’s second anniversary, more than 768 people had been charged with hurricane fraud-related crimes in 41 federal jurisdictions. By comparison, only seven people so far have faced federal fraud charges related to disaster aid payments in the states where Fema distributed aid after Sandy, according to an AP survey of prosecutors’ offices.

Kelly, the assistant inspector general, said the modest tally of Sandy-related fraud cases likely reflects a shift in law-enforcement priorities. The Justice Department enforced a “zero tolerance” policy for Katrina fraud, he said. But he believes it has since given prosecutors more freedom to turn down cases involving relatively small sums of money.

Justice Department spokesman Peter Carr declined to comment on the number of Sandy-related prosecutions, but he noted that a federal task force on disaster fraud has received more than 30,000 complaints from the public about possible Katrina-related fraud compared to 1,758 complaints related to Sandy.