(CNN) Nigeria will evacuate hundreds of its citizens from South Africa following a string of xenophobic attacks, a government official told CNN on Monday.

Flights carrying the returnees will start leaving Johannesburg by Wednesday, said Abike Dabiri-Erewa, chairwoman of the Nigerians in the Diaspora Commission.

"We have 640 Nigerians that want to come back from South Africa, and that will require two planes. There may be more people who want to leave, but we will know when we get to South Africa on Wednesday," Dabiri-Erewa told CNN.

Nigeria's President Muhammadu Buhari is deeply concerned about the "intermittent violence" against its citizens in South Africa and called on the country's government to take steps to end it, his media aide Femi Adesina said.

"President Buhari is worried that the recurring issue of xenophobia could negatively affect the image and standing of South Africa as one of the leading countries on the continent, if nothing is done to stop it," Adesina said in a statement.