If Barack Obama was given a do-over for the first term of his presidency, his first order of business would be to close Guantánamo Bay, he said on Wednesday.

Obama’s comments followed an economic speech he delivered in Cleveland, Ohio. An audience member asked the president what advice he would give to himself looking back on his early days in the White House.

“I think I would have closed Guantánamo on the first day,” Obama said, noting that he didn’t anticipate how the politics around the controversial detention camp would change.

“We had a bipartisan agreement that it should be closed … and I thought we had enough consensus there that we could do it in a more deliberate fashion,” Obama said. “But the politics of it got tough and people got scared by the rhetoric around it, and once that set in then the path of least resistance was just leave it open even though it’s not who we are as a country.”

Obama has repeatedly stated his intention to shut down the prison, which is located at a US naval base in southern Cuba and has drawn international criticism for its treatment and force-feeding of prisoners. The president revitalized the pledge in his State of the Union address this year, but skeptics pointed out that Obama made the same call the year before only to be thwarted by Congress.

Closing Guantánamo would require cooperation from Congress, where Republicans have shown little indication that they would be willing to lift current restrictions on the transfer of detainees out of the prison.

In his remarks on Wednesday, Obama made a reference to Arizona senator John McCain’s support for shutting down the facility. But even McCain backed GOP legislation imposing harsher restrictions on the transfer of prisoners and a ban on moving any of them to the United States for detention or trial, signaling the obstacles Obama faces if he tries to make good on his promise.

The president has been able to reduce the population at Guantánamo to 122 prisoners, but civil liberties advocates have criticized his commitment to the camp’s closure. The Obama administration has displayed a renewed aggression on the issue since the fall and transferred more than two dozen detainees to other countries in less than three months.