Brattleboro, Vermont (CNN) -- Floodwaters brought by Tropical Storm Irene began to recede Monday in parts of Vermont, but the governor warned that further flooding and loss of life are likely ahead for the small, rural state.

"It's just devastating," Gov. Peter Shumlin said Monday. "Whole communities under water, businesses, homes, obviously roads and bridges, rail transportation infrastructure. We've lost farmers' crops," he said. "We're tough folks up here but Irene ... really hit us hard."

Hundreds of people remained trapped Monday in communities cut off by raging floodwaters that washed out or otherwise damaged 263 roads and bridges, Shumlin said. Exactly how many were stranded remained unclear, he said.

"It's hard for us to know, frankly, because it's hard for us to get into the communities we need to get to," he said.

Highlighting the transportation problems, the Vermont National Guard had to travel through neighboring Massachusetts to get rescue crews to the small, cut-off town of Wilmington, the governor said.

Three deaths had been confirmed as a result of the storm.

A woman who was standing near the Deerfield River in Wilmington died after she was swept away by floodwaters. Her body was recovered, according to authorities.

Also, a male was killed in Mendon after being swept away by floodwaters, according to Vermont Emergency Management. Another male was found dead in Lake Rescue in Ludlow earlier Monday, authorities said.

Shumlin said Monday afternoon that authorities were searching for another missing person.

Images of the flooding showed normally tranquil streams pouring through city streets and thrashing against buildings and bridges, including some of the state's iconic covered bridges. Four to six of the covered bridges were destroyed in the flooding, officials said.

Even the state emergency management headquarters in Waterbury flooded Sunday night, forcing officials to evacuate to Burlington, about 20 miles away.

"The storm essentially shut southern Vermont down," said state police Capt. Ray Keefe. "It's terrible."

President Barack Obama signed a disaster declaration for the state on Monday, allowing federal aid to the state.

Federal Emergency Management Agency Administrator Craig Fugate planned to tour flooded communities in Vermont on Tuesday, according to the Department of Homeland Security. No specifics of Fugate's trip were given.

The governor described conditions as "probably the toughest flooding that we've seen in the state of Vermont in our history."

"You've got to remember that Vermont is a lot of beautiful mountains with valleys and small brooks that run into bigger rivers," Shumlin said on CNN's "The Situation Room with Wolf Blitzer."

"Our small brooks have crested (but) our larger rivers have not -- so we know there's more trouble ahead."

Ashley Grote lives in South Royalton, Vermont, along the White River. Her home is on a hill, so the only damages she suffered were a washed-out driveway and a "small river running through our backyard."

But many others in the area were not so fortunate, Grote said Monday after taking a walking tour of the area. She saw washed-out roads, farm fields covered in silt and other damage. She said she knows some houses were swept away, as well.

"The massive amounts of damage caused by water is so unbelievable, and until you see it you would not believe it," she said. "There are so many farmers whose fields are beyond repair, and are losing food for their animals or hay that they sell for an income."

Many areas of the state remained under flood warnings Monday, with the National Weather Service reporting continued record flooding in some locations.

For instance, Otter Creek in Rutland, Vermont, went from a depth of less than 4 feet Sunday morning to more than 17 feet at 1:45 a.m. Monday -- nearly 4 feet higher than the record set in 1938, according to the National Weather Service. While it was falling Monday morning, it was still 8 feet above flood stage.

Shumlin said additional flooding problems were likely as water poured out of smaller streams into larger rivers.

CNN iReporter Andrew Cliver had been through a hurricane before, but he said Vermont's mountainous terrain amplified the effects of Irene's rains.

"Because it's so mountainous here, the water all drains into these little creeks and rivers with a rushing force that's powerful enough to move buildings that have been standing for 100 years," he said.

Unlike many states, Vermont did not order or suggest evacuations. Many of the state's towns are in lowlands and there are few large areas of dry land, so large-scale evacuations are impractical, Shumlin said.

Some towns did partially evacuate overnight, including the state capital of Montpelier, but the situation in the town of 8,000 located 50 miles from Canada was not as bad as feared, officials said. Floodwaters appeared to peak on the Winooski River early Monday.

"There does not appear to be much additional flooding from what has been previously reported," said City Manager William J. Fraser. "Overall we do not expect conditions to get worse."

In Brattleboro, a city of 12,000 people on the New Hampshire border, Whetstone Brook flowed out of its banks and undermined a three-story building, threatening to bring it down.

"We've seen nothing like this," said Barbara Sondag, town manager for Brattleboro.

Lifelong resident Jesse Stone watched the White River rip away at the footings of the historic Quechee covered bridge as it washed through the heart of the town.

"It is just about impossible to imagine this bridge being taken out," Stone said in an iReport. "It's usually (far) above the water level."

CNN's Greg Botelho, Phil Gast, Divina Mims, Ed Payne and Justine Redman contributed to this report.