David Cameron was heckled by angry residents of the flooded Kent village of Yalding as he pledged to make flood protection a higher priority of the government.

The prime minister was confronted during a walkabout of the village, where about 100 homes were evacuated on Christmas Eve and Christmas Day when the converging rivers of the Beult, Tiese and Medway burst their banks and sent a torrent of water down the high street.

Erica Olivares, 49, told Cameron she was "absolutely disgusted" by the response of government agencies. "We were literally abandoned," she told the prime minister in the street outside her devastated 18th-century cottage, where the water rose to waist height on Christmas Eve. "We had no rescuers, no nothing for the whole day."

Cameron asked what she needed and urged her to "get on to the council", but she replied: "They all decided to go on holiday."

Cameron visited her cottage and said he would make calls on her behalf to try to sort out her electricity supply, and to the council to provide waste collection for her wrecked furniture.

"We had no practical help whatsoever," Olivares told the Guardian. "The Environment Agency said it was up to the council and when I did get through to the council they said if you need sandbags, get your own. On Christmas Day we saw absolutely no one."

She complained that there had been only a couple of hours notice of the flood, which poured down the village's main road like a rapid.

Later, as the prime minister talked to Environment Agency officials about the possibility of building flood defences around the village, he was heckled by Sean Matthews, 54, a tube driver who said he was angry at how the villagers had been left to fend for themselves.

"The people he's talking to, the Environment Agency and so on, weren't here," he said afterwards. "I swam this road on Christmas Day pulling people out on my own. There was no one here on Christmas Day or Boxing Day. A lot of families have lost a lot."

Cameron spoke to a group of firefighters from Kent fire and rescue who posed with him beside an inflatable boat that appeared to have been ferried in 10 minutes before Cameron arrived in his prime ministerial Range Rover. It departed soon after he left.

Yalding was badly flooded in 2000 and no new flood defences have been built since then. Three options have been proposed including an earth berm around the village, upstream storage and house-by-house defences. The earliest that any work will start on the defences will be 2017, Environment Agency officials said.

Many villagers said they felt let down by the failure of authorities to make good on assurances that the floods of 2000 would not be repeated. Geraldine Brown, chair of the local parish council, told Cameron the solution to repeated flooding would cost £20m. Cameron promised "a complete review following these floods".

"There is money there, it is a question of working out which things it could fund," he told her, adding he felt the floods were "completely awful". "You only have to watch the news over the last few years to know these events are happening more often now," he said. "It needs to be a bigger priority for the government and it is."

Speaking outside the flooded village pub he added: "There are a lot of flood defences being built and something like 80,000 houses were protected this time, but we have got to do more. We need to work with the Environment Agency to see what more we can do from this flood and other floods, but right now the priority is to help people recover from the terrible shock of this happening to them on Christmas Eve and Christmas Day."

He conceded that warnings from the Environment Agency were not always accurate, but he said: "Sometimes these are very, very tragic events and it is impossible to protect everybody, but we have got to do more and we have got to do better."