New York (CNN) -- Protesters rallied in downtown New York City Thursday to voice their anger over what they perceive as the roles Wall Street and big banks played in America's economic crisis.

Marching from City Hall to Wall Street, the protesters chanted "good jobs for all," and held signs with messages including "Hold banks accountable," "Make Wall Street Pay," and "Reclaim America."

The AFL-CIO organized the rally, and union President Richard Trumka addressed the crowd, saying, "How long will we allow the spirit of greed to continue to drive us into economic holes?"

The National People's Action, a group that was involved in organizing the protest, said in a news release that demonstrators represented unemployed workers, foreclosure victims and community activists.

Protester Gerard Pettine said he just wants Wall Street to be held accountable for its involvement in the economic collapse.

"They need to have some integrity and some honor and do the right thing," Pettine told CNN.

Protester Elizabeth Soto, who came with her brother to the rally, was concerned about the lack of job creation.

"We are here to say Wall Street's got to help Main Street," she said. "We supported the stimulus bill, which bailed out Wall Street, but we expected the jobs to come back to Main Street."

The New York Police Department estimated that 6,000 people participated in the rally.

Earlier Thursday, as part of a larger "day of action" against Wall Street, protesters descended upon the headquarters of two New York City banks -- JP Morgan Chase and Wells Fargo -- leaving letters for the banks' chief executive officers.

The letters, according to the news release from the National People's Action, said that the banks have an opportunity "to step up to the plate and be leaders in rebuilding the American economy."

JP Morgan spokesperson Mike Fusco declined to comment on the letter, but said that there were a "couple dozen" protesters who were in the building for less than a half hour.

Fran Durst of Wells Fargo said that about 100 protesters flooded their building's lobby and tried to deliver their letter to the bank's CEO. Durst said the event didn't cause a major disruption to the bank.

Other protests were scheduled in San Francisco, California; Kansas City, Missouri; and Charlotte, North Carolina, according to the National People's Action news release.

CNN's Mary Snow, Jennifer Rizzo, Cassie Spodak, and Kristen Hamill contributed to this report.