Four white extremists have been arrested in South Africa over suspected acts of terrorism, police say.

The raids were made on Sunday, the opening day of the African National Congress conference in Bloemfontein, Free State province. Police have denied reports of a rightwing plot to bomb the event.

Brigadier Billy Jones, a police spokesman, said four people had been detained: one in Bloemfontein, one elsewhere in Free State, one in Gauteng, the province containing Johannesburg, and one in Limpopo province.

"The suspects are aged between 40 and 50," he said. "Their premises were searched and evidence supporting the investigation was seized.

"At this stage we are not linking any of the arrests with the conference at all. The acts of these suspects were countrywide."

The four would be charged and brought before a court imminently while police investigate further, Jones added. "We are expecting to make more arrests, but we cannot reveal further details at the moment."

The Federal Freedom party (FFP), a fringe group which claims it is pushing for self-determination for the country's Afrikaner minority, said at least two of the people arrested were believed to be members of the party.

Francois Cloete, its national secretary, told Reuters: "We were not involved and do not associate ourselves with their actions."

Cloete described the FFP, which was established two months ago, as a "party that promotes self-determination of the Afrikaner/Boer people in a confederal political model".

More than 4,500 people, including President Jacob Zuma and dozens of ministers and top business people, are attending the five-day conference in Bloemfontein, in the municipality of Manguang, amid a heavy police presence.

Meetings are being held on the campus of Free State University, which became infamous four years ago for a video showing white students humiliating black employees and urinating on their food.

The few vehicles allowed onto the campus are searched by police and sniffer dogs, while all individual bags are inspected. One newspaper editor described the security arrangements as far more extreme than the last conference in 2007, indicating "paranoia".

On Monday, two ANC spokespeople did not respond to the Guardian's requests for comment. A third denied any knowledge of the arrests, saying the party was focused on internal elections.

Most white South Africans accepted the ANC's victory in the 1994 election that ended decades of racial apartheid and brought Nelson Mandela to power. But a dwindling handful continue to oppose the historic settlement.

In July, a former university lecturer was found guilty of orchestrating a plot in 2002 to assassinate Mandela, overthrow the ANC and drive South Africa's black majority out of the country.