LOS ANGELES, California (CNN) -- Pedro Pablo slowly folds up his American flag blanket and stuffs it in his duffel bag. With it goes his American dream.

Pedro Pablo, an illegal immigrant from Guatemala, headed home recently due to the bad U.S. economy.

"I left my family and lost four years with them. I will ask them to forgive me," he said.

Pablo is an illegal immigrant from Guatemala who came to the United States to support his wife and five sons back home. When he arrived, construction jobs were plentiful. Over the last year, he says, he's worked three days.

He recently boarded a bus with a one-way ticket home, paid for by the Guatemalan consulate in Los Angeles. "I thought I could get ahead here. I regret coming." Watch day laborers head home »

Across the United States, tens of thousands of immigrants -- those here legally and illegally -- are facing a similar dilemma: Do they continue to search for jobs in a struggling U.S. economy or return home to an even bleaker economic situation?

"Things are very dire, and I think it's impacting those at the very bottom even more so," said Abel Valenzuela, a professor at the University of California-Los Angeles who has spent years studying day laborers.

"Day laborers are being really, really impacted."

America's economic boom during the 1990s and 2000s created a high demand of day workers needed for anything from building homes to picking fruit and from working at slaughterhouses to working as nannies. Many of those jobs have since evaporated, resulting in more and more people -- immigrants and native-born Americans -- flooding day labor job sites and standing on street corners in search of any type of work they can get.

"All of them are competing for the few jobs being dispatched," Valenzuela said.

Immigration experts say it's not yet clear how large an immigration exodus of Latin Americans is under way. But they say anecdotal evidence suggests day laborers, like Pablo, have begun packing -- a result of the economy and tougher immigration enforcement.

For some immigrants, the experts say, the reasons for toughing out the U.S. economic recession outweigh the reasons for leaving, including:

• One or two days of work per month at $8 an hour is often better than what they can make back home;

• Tougher border enforcement along the U.S.-Mexico border has made it harder for them to return once they leave;

• Smuggling costs to get into the United States from Mexico have skyrocketed from about $1,500 three years ago to about $6,000 today.

"I'm not convinced it's a tidal wave of exodus," Valenzuela said. "There really is a fear mentality [of leaving], and as a result many immigrants are buckling down -- that is they're hiding or living in the shadows of our law. So they think more than twice about whether or not they want to go back to their country of origin, because they know very well that it's going to be extremely difficult and very expensive to come back if they want to pursue their dream."

Steven Camarota with the Center for Immigration Studies, a Washington-based think tank that seeks a "pro-immigrant, low-immigration vision," said Census data indicate that more than 1 million illegal immigrants left last year, a departure that began even before the nation's economy took a turn for the worse toward the end of the year.

He said better border enforcement and workplace raids on illegal immigrants "let people know that the immigration law was back in business."

With illegal immigrants returning home, he said, "It's certainly good for two groups: taxpayers and less educated natives."

The lack of work in the United States has had a trickle down effect in the immigrants' countries of origin. The money sent back home by Mexican immigrants in 2008 fell for the first time since record-keeping began 13 years ago. Watch lack of remittances has "real large economic implications" »

The remittances dipped 3.6 percent, from $26 billion in 2007 to $25 billion, according to Mexico's central bank. Remittances are Mexico's second-largest source of foreign income, behind only oil. Other Latin American countries also have seen money sent from immigrants in the United States slow.

Erik Camayd-Freixas, a professor at Florida International University who has served as an immigration court interpreter for two decades, recently traveled to Guatemala, where he saw the effects of less money being sent home by immigrants.

"Everybody was talking about it," he said. "The local economies are severely impacted and the unemployment is rampant."

He said scores of people deported from the United States are trying to find any type of job in Guatemala. "They've been there six months and they have no work," Camayd-Freixas said.

That's why he said he believes most immigrants already in the United States are willing to stick it out, despite the hard-scrabble times in America.

"The truth of the matter is, despite our 7.6 percent unemployment, most Americans are not going to do those jobs that migrant workers do," Camayd-Freixas said. "They're certainly not going to pick tomatoes, grapefruits and oranges."

Camarota disagrees. He said Americans most likely to compete for day labor jobs -- those with a high school degree or less -- are currently unemployed at an astounding rate of about 15 percent. "It's very hard to argue that we're desperately short of unskilled workers," he said.

Jeronimo Salguero is the director of a day labor site in Los Angeles. He says employment for day laborers has dipped 75 percent over the last year. He said on any given day three years ago, his center found work for nearly 50 percent of the people who gathered there. Now, that figure is about 5 percent of the 250 men who huddle daily searching for work.

A study issued in January by the Pew Hispanic Center found that nearly three of four Latinos surveyed said their personal finances were in fair or poor shape, compared to 61 percent of the general U.S. population. Among Hispanic immigrants who sent remittances home over the last two years, about 70 percent said they sent less in 2008 than in 2007.

Salguero said to help offset the hard times, immigrants are crowding about 20 men into apartments. He's now working with the Guatemalan consulate. Once a week, he said, the consulate purchases a bus ticket for an immigrant to go home.

"They are completely desperate," he said. "Each day, I have workers coming into the office and say, 'Geronimo, help me. I want to go back to my country.' "

Pablo was one of those men. He had lived in a one-bedroom apartment with seven other men. His "bedroom" was a corner of the living room where he kept his blanket, duffel bag and picture of his family.

"I can't make it here," he said. "If I have to suffer, it's better to suffer in Guatemala with my family."

CNN's Traci Tamura and Gregg Canes contributed to this report.

All About Immigration Policy • Border Control and Customs • U.S. Bureau of Immigration and Customs Enforcement