Thousands of protesters descended on Washington DC on Sunday demanding Barack Obama shut down the Keystone XL pipeline project to show he is serious about taking action on climate change.

Organisers said a crowd of 35,000, carrying placards in the shape of bright red stop signs, gathered at the Washington monument on a bright but bitterly cold day for the march on the White House.

The event, billed as the largest climate protest in US history, was intended as a show of force before Obama renders his decision on the pipeline project in the next few months.

Protesters were bussed in from 30 states and Canadian provinces. Marchers held aloft banners proclaiming "Don't be fossil fools" and "It's time to cut carbon" during the rally.

The DC event was accompanied by smaller one in cities including Austin, during which protesters chanted that the president had "sold out Texas" over the pipeline.

A protest in Austin, Texas, against the proposed Keystone XL pipeline. Photograph: Julie S Dermansky for the Guardian

Opponents of the project – environmental activists, indigenous Canadian groups, and landowners alone the pipeline route – have cast the Keystone XL as the defining test of Obama's second term.

They said Obama's entire presidential legacy rests on the Keystone XL, which they framed as a critical test of Obama's environmental credentials and his sweeping promises to use the next four years to protect future generations from climate change.

"Obama holds in his hand a pen and the power to deliver on his promise of hope for our children," said Michael Brune, the director of the Sierra Club. "Today we are asking him to use that pen to reject the Keystone XL tar sands pipeline, and ensure that this dirty, dangerous, export pipeline will never be built."

Others said Obama owed it to his supporters to reject a project, which would open up a vast store of carbon. "I think it's really important for Obama to realise that his base, the people who supported him, do not want this," said Judy Dufficy, a former teacher from Chicago. She said she had volunteered for Obama in Iowa, where he started his run for the White House in 2008.

The president is under even stronger pressure from the oil industry, business groups and the Canadian government to approve the project, which will open new outlets for the vast crude reserves of Alberta's tar sands.

Protesters said Obama's entire presidential legacy rests on the Keystone XL project. Photograph: James Fassinger for the Guardian

The Canadian government has deputed a well-connected Conservative MP, Rob Merrifield, to build support for the pipeline in Congress. The MP told the Globe and Mail he now spent about half of his working life in Washington.

But some pipeline opponents believe they now have a fighting chance in the David v Goliath scenario. They also consider they have already won a victory of sorts, just by hanging on.

Until late 2011, Obama had been expected to approve the pipeline, with barely any controversy outside a relatively small circle of activists.

Most of the big Washington-based environmental groups seemed resigned to the project.

But a relatively unknown activist group, 350.org, adopted the Keystone XL as their cause. Organisers called a series of protests at the White House and in 2011 the arrest of more than 1,000 activists, including high-profile climate scientists and celebrities, brought new scrutiny to the project.

Obama rejected the project early last year because of objections to the proposed route from the state of Nebraska.

Other events began to work in the protesters' favour. After a year of extreme weather events, crowned by Sandy, climate change returned to the public agenda.

Obama adopted climate change as one of the three priorities of his second term.

By Sunday afternoon, the protesters had been endorsed by Robert Redford, who has called the Keystone XL a "national disgrace", and Democratic members of Congress.

Meanwhile, Canadian newspapers were reporting the fate of the project now hung in the balance.

Sheldon Whitehouse, Democratic senator from Rhode Island, told the crowd in DC to keep fighting. "You know the polluters don't want you here. They've got their lobbyists. They've got their Super Pacs. They've got the whole town in their pockets," he said. "And then you came and showed them."