As the death toll from a series of mudslides rose to 141, thousands of Bangladeshi residents were allowed to return to their destroyed homes Thursday to survey the damage.

Cash and materials were offered by the government to rebuild their lives following the devastating landslides early Tuesday. In the hard-hit Bandarban district, government official Dilip Kumar Banik said the road network had been largely restored and some 2,000 people were able to leave 13 shelters.

One such evacuee was Azizul Haque, who said he hoped to return to his home district of Cox's Bazar to be with his family for the upcoming Eid holiday. Those plans were destroyed when the land gave way.

"I have lost my wife and my 10-year-old daughter. I have lost everything," Haque said, sobbing. "My life was very difficult, now it has become even more difficult. I do not know why Allah is punishing me this way."

(MORE: Tornado Outbreak Hits the High Plains )

Banik said poor day laborers and migrants had built homes in vulnerable areas despite warnings from officials.

After torrential rains began Monday, officials used loudspeakers to ask people to leave their homes for safety, and some were evacuated.

"But that fateful night, the hills started collapsing. They are poor and they do not want to leave their homes, their belongings so easily. They have few options," he said.

Shah Kamal, secretary of the Ministry of Disaster Management, said Thursday that officials from various agencies were assessing what can be done to prevent such disasters in the future.

Rescuers on Thursday recovered the body of a woman from a mass of mud that collapsed onto a village in southeastern Bangladesh and were searching for several others who were still missing, officials said.

Officials reported 104 dead and at least 5,000 homes destroyed or damaged in Rangamati, where mostly tribal villagers live in small communities near a lake surrounded by hills.

Another 28 were killed in the coastal Chittagong district, six died in Bandarban, two in Cox's Bazar and one in Khagrachhari.

The delta nation is frequently hit by strong storms, flooding and landslides. Experts said this week's tragedy was also the result of uncontrolled denuding and soil harvesting on hills above unplanned settlements.

Many people in hilly regions ignore authorities' calls to avoid constructing homes on slopes.