Yemen's out-going president, Ali Abdullah Saleh, has formally handed over power to his vice-president after 33 years in power.

A ceremony was held in the capital, Sanaa, Al Jazeera reports, during which Saleh congratulated his successor Abd-Rabbu Mansour Hadi and said he hoped for a peaceful transition of power.

"All of Yemen's proud citizens are behind him," Saleh said handing Hadi a Yemeni flag.

Hadi became acting president in November in an agreement brokered by the Gulf Cooperation Council, which included a clause giving Saleh immunity from prosecution in return for stepping down. Hadi was chosen to replace Saleh last week after elections in which he was the sole candidate.

More from GlobalPost: Yemen votes out President Saleh (PHOTOS)

According to Reuters, Hadi vowed to see Yemen through its transition to democracy during his inauguration speech, and told parliament: "In two years I hope to be standing where Ali Abdullah Saleh is now, with a new president standing where I am."

However, he warned that Yemen is at a "complex and difficult phase."

As his inauguration was taking place, a car bomb killed more than 20 members of the security forces in Hadramout province, near the presidential palace in the provincial seat, Mukalla.

Associated Press also reported that thousands of Yemenis protested in the capital Sanaa, demanding the prosecution of former president Ali Abdullah Saleh.

Al Jazzera's reporter in Sanaa said that the atmosphere in the city was 'tense' and that the Prime Minister Mohammed Basindawa was boycotting the inauguration ceremony.

More from GlobalPost: Yemen's President is out but real change remains elusive

Other reports quoted aides to Saleh as saying he and members of his family were planning to leave the country and go into exile in Ethiopia, amid fears of fresh violence.

A separate Associated Press article said senior sources had confirmed that arrangements had been made for the departure to take place by Thursday.