George Kasimos felt bad about cursing at the governor.

"For about a minute," he said. "Then I figured, 'He's the governor. He should know better.' "

At issue was the number 1,700.

That was how many people Gov. Chris Christie said remain displaced by Hurricane Sandy when he spoke last week in Seaside Heights on the fourth anniversary of the storm.

Christie's exact quote during the news conference at Jimbo's Bar and Grill on the boardwalk went like this:

"It truly is something to highlight that, in four years, we've helped to restore the households of nearly everyone affected by the storm that impacted nearly 365,000 homes and we're down to 1,700 left of people who are not back in their homes."

Kasimos' exact quote went like this:

"That's a bull(----) number and you know it," Kasimos shouted.

"When he said that, I went through the roof," Kasimos said this week. "I mean, I personally know a few hundred people still out, just in my organization alone."

That group is Stop FEMA Now, started by Kasimos and others in January 2013 in response to new federal flood maps that forced homeowners to elevate their houses and raised their insurance premiums.

"In Toms River alone, we still have about 600 to 700 homes substantially damaged," he said. "That's just Toms River! What about the rest of the state?"

Good question. From Moonachie to Newark's Ironbound to Union Beach, to Ortley Beach to the lagoon areas of the Metedeconk and the Toms rivers, there are still pockets or swaths of vacant homes. Some are under construction and some are just the way Sandy left them, except for the plywood boards nailed over the windows or doors.

MORE: Recent Mark Di Ionno columns

At the Friday event, Christie hadn't even finished saying "1,700" when Amanda Devecka-Rinear yelled out, "That's not true, governor!"

Devecka-Rinear is the head of the New Jersey Organizing Project (NJOP), formed on the second anniversary of Sandy to give a unified voice to storm victims.

Devecka-Rinear not only objects to the number 1,700, but also to the nearly "365,000" homes the governor obliquely claimed the state helped make whole.

That number has often been kicked around, but when the state's umbrella agency for the recovery, the Department of Community Affairs (DCA), applied for federal Sandy aid, it said "40,000 owner-occupied homes in the state (were) severely damaged or destroyed."

That does not include second homes, which are ineligible for government recovery aid. A ride through Ortley Beach and other parts of the barrier island north of Seaside Heights proves many vacation homes are still boarded up or under construction.

According to the DCA website, a total of 7,995 homeowners entered the state's Reconstruction, Rehabilitation, Elevation and Mitigation (RREM) program and Low-to-Moderate Income Homeowners Rebuilding (LMI) program.

The state website suggests 4,115 of those homes are still being worked on.

"We just don't know the real numbers," Devecka-Rinear said. "We know, at one point, the state said there were 15,000 people who applied for RREM grants, which means 7,000 dropped out. What happened to them?"

Another good question.

That's why NJOP is developing a statewide survey of storm damage called the "Sandy Truth Project."

While the research may not be all-encompassing, Devecka-Rinear said she hopes it will "record the stories of what the recovery actually looks like.

"This is more than just about the numbers," she said. "It's about the impact of the storm on people's financial and mental health, and all the other elements of adverse impact."

That includes the frustration of dealing with insurance claims and government recovery programs, or quantifying the problems people had with insurance companies and contractors.

Part of the point is to understand what help victims of Sandy still need. For instance, NJOP pushed for legislation which would have staved off foreclosure for people still out of their homes, but that bill was vetoed by Christie earlier this year.

"We're still pushing that bill," Devecka-Rinear said. "That's a good example though of what you learn when you hear people's stories."

The idea for the survey came from a summer-long tour of Sandy-impacted areas by NJOP and other nonprofit groups looking to help storm victims.

Part of the tour was a portable "Sandy wall" that displayed photos, messages and other remembrances of the storm and the recovery.

"We had so many people come out and tell us their stories, we wanted to do something more comprehensive," Devecka-Rinear said.

The group is training volunteers to conduct surveys in several impacted communities, with help from Rutgers and Stockton State universities.

Sign-ups for the survey are available online at newjerseyop.org. Devecka-Rinear knows many of the people most impacted might not be internet savvy, so they can also call NJOP at (609) 312-3899 to participate.

"We certainly want to reach the older people and people who might not have access to the internet," she said. "That's the only way to a get a complete picture."

Mark Di Ionno may be reached at mdiionno@starledger.com. Follow The Star-Ledger on Twitter @StarLedger and find us on Facebook.