Story highlights At least five people have fled the cafe, Seven Network video shows

Police negotiators are in contact with a hostage taker, the official says

Australia's PM says it's unclear whether the violence is politically motivated

Local media carry images of people holding a black flag with Arabic writing

Five people have fled from a cafe in the heart of Sydney where a dramatic hostage taking is unfolding Monday.

But an unknown number of people remain in the cafe, held by at least one armed hostage taker.

Hundreds of police officers have shut down a usually bustling area in Australia's most populous city amid the crisis.

Chilling images from local media showed people, believed to be hostages, with their hands pressed against the glass of the Lindt Chocolate Cafe in Sydney's central business district.

Footage showed them holding up a black flag with Arabic writing on it that reads: "There is no God but God and Mohammed is the prophet of God."

Three people who were seen running out of the cafe "are now out of the building and they are now with police," said New South Wales Police Deputy Commissioner Catherine Burn, declining to give details about whether they escaped or were let go.

Video from CNN affiliate Seven Network later showed two more people running out of the cafe.

Authorities said many of the details about the situation remained unclear, including the number of people inside the cafe and the motivations of the hostage taker.

"We don't know whether this is politically motivated, although obviously there are some indications that it could be," Australian Prime Minister Tony Abbott told reporters.

Photos: Photos: Sydney cafe siege Photos: Photos: Sydney cafe siege Sydney cafe siege – A message of condolence is written on the pavement at Martin Place, Sydney on Wednesday, December 17. #illridewithyou became a trending hashtag on Twitter during the siege, expressing solidarity with Australian Muslims in the wake of deadly siege at the Lindt Cafe in Sydney. Hide Caption 1 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – A woman walks past flowers placed by people as a mark of respect for the victims of Martin Place siege on Tuesday, December 16. Hide Caption 2 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – Members of the Muslim community show their respect at Martin Place on December 16. Hide Caption 3 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – Prime Minister Tony Abbott arrives with his wife Margaret to pay their respects at Martin Place on December 16. Hide Caption 4 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – Australian flags fly at half mast on Sydney Harbour Bridge on December 16 following the fatal siege. Hide Caption 5 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – An injured hostage is carried to an ambulance from a cafe in Sydney on December 16. Gunfire erupted early Tuesday as police stormed the Lindt Chocolate Cafe, where a gunman had been holding hostages since Monday morning. Hide Caption 6 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – Hostages run toward police officers on December 16. Hide Caption 7 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – A police officer and a paramedic escort a hostage away from the scene of the siege on December 16. Hide Caption 8 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – An injured hostage is wheeled to an ambulance after shots were fired during the raid on the cafe. Hide Caption 9 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – Two armed police officers stand ready to enter the cafe on December 16. Hide Caption 10 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – A hostage stands by the front entrance of the cafe as she turns off the lights on Monday, December 15. Hide Caption 11 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – A hostage runs toward police on December 15. She was one of five people seen fleeing the cafe. Hide Caption 12 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – Police gather near the cafe on December 15. Hide Caption 13 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – Hostages behind the cafe's window hold up a black flag with Arabic script on December 15. Hide Caption 14 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – A police officer listens to a radio on December 15. Hide Caption 15 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – Two hostages make it to safety outside the cafe on December 15. Hide Caption 16 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – A police sniper walks to his vehicle on December 15. Hide Caption 17 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – A journalist reports on the situation in Sydney. Hide Caption 18 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – Armed police are seen outside the cafe on December 15. Hide Caption 19 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – People rush past armed police near the cafe on December 15. Hide Caption 20 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – Police close a street near the scene on December 15. Hide Caption 21 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – Police officers talk at the scene on December 15. Hide Caption 22 of 23 Photos: Photos: Sydney cafe siege Sydney cafe siege – Armed police patrol near the Sydney Opera House. Major landmarks in Sydney were evacuated as police responded to the hostage situation. Hide Caption 23 of 23

JUST WATCHED Police: 'We are being tested today' Replay More Videos ... MUST WATCH Police: 'We are being tested today' 03:36

JUST WATCHED Abbott: Incident is disturbing, terrifying Replay More Videos ... MUST WATCH Abbott: Incident is disturbing, terrifying 03:44

The Islamic writing on the banner stoked fears that the crisis in Sydney could be linked to extremists. Australia, which is part of the international coalition fighting the terrorist group ISIS in the Middle East, said in September that it had foiled a plot by Islamic militants to carry out a public execution.

New South Wales Police Commissioner Andrew Scipione said it wasn't yet clear whether the situation at the cafe was a terrorist event but that police are on "a footing that would be consistent with a terrorist alert."

Burn said police negotiators are now in contact with a hostage taker, who appears to have taken staff and customers captive as people were heading to work on Monday morning in the Martin Place area.

Chris Kenny, an editor at The Australian newspaper, said he was at the cafe Monday morning. He left shortly before the siege started.

"As police quickly swarmed and cleared the area, I turned to see a man against the window, facing out with his hands raised," he wrote in his account of the incident. "At first I was relieved thinking this was the gunman responding to police -- but soon the awful realisation the customers were being forced against the windows."

Seven Network reported that at least 13 people are being held at the cafe, but Scipione declined to say how many were in there.

Police barricaded off streets and evacuated buildings near the cafe, bringing an eerie quiet to a district typically buzzing with pedestrians and vehicles.

MAP: Sydney CBD

The Martin Place train station was shut down, according to police. They urged people to stay away from the area, but some local office workers gathered at the scene to try to find out what was going on.

The buildings evacuated included the U.S. Consulate General, said Alicia Edwards, a spokeswoman. All personnel have been accounted for, although it's not known whether there are any U.S. citizens among the hostages. U.S. President Barack Obama has been briefed on the situation.