Thousands of protesters have called for Brazil’s president, Michel Temer,to leave office as he lost yet another key adviser amid a deepening political crisis.

Scuffles between police and protesters broke out as the demonstrations neared Congress.

Temer is struggling to retain power after the release of a recording that appears to capture him approving hush money for a convicted former lawmaker. The president is resisting calls to resign and has said he will fight the accusations. Brazil’s top court is investigating him for alleged obstruction of justice and involvement in passive corruption.

Even before the scandal broke, Temer’s popularity was low, in part because of opposition to economic reforms he is trying to push through Congress. Several of his allies and aides have also been caught up in a wide-ranging corruption investigation that has jailed business leaders and senior politicians.

Only one of the five special aides to the president now remains following the resignation Wednesday of former lawmaker Sandro Mabel, one of three who had been linked to corruption allegations. Mabel published a letter saying he needs to spend more time with his family.

In Brasilia, the capital, thousands of protesters marched toward Congress in an effort to stop proposed reforms to the country’s retirement system that unions and others say will force Brazilians to work too many hours and will unfairly reduce pensions. Temer says those and other reforms are necessary to drag the economy out of a deep recession.

But protesters also took aim at the president himself, shouting “Out with Temer!” and carrying signs calling for immediate direct presidential elections.

If Temer resigns, the Constitution says that Congress would vote to elect the next president, who would hold power through the end of 2018. But many Brazilians, disgusted with the political class, want to vote themselves.

While the demonstration was initially peaceful, police and protesters began clashing as the protest neared Congress. Police in riot gear, some on foot holding shields and others on horseback, lined up near the chamber. In television images, police could be seen spraying tear gas or pepper spray at protesters and detaining others who tried to break through a cordon.

In Rio de Janeiro, demonstrators gathered in front of the state assembly to protest proposed austerity measures.

“The state today is in a sea of debt because of corruption,” said Julio Cesar Azevedo, a leader of a union that represents prison guards. “It’s not right that public servants are paying this bill.”

The state of Rio is in serious financial trouble and many public servants have received their salaries late or not at all.