Thousands of Syrians were stuck in and around Aleppo on Saturday as a deal to evacuate fighters and civilians from four towns as part of a coordinated population swap stalled, according to a monitor and activists.

The agreement involves Zabadani and Madaya, two government-besieged towns near the capital Damascus, and Foua and Kefraya in northwestern Idlib province, which have been encircled by Syrian rebels since March 2014.

Rebels and civilians who had left Madaya on Friday sat outside rows of coaches in government-held Aleppo city, waiting to move on to their final destination of rebel-held Idlib, pictures sent by a pro-opposition activist showed.

Meanwhile, residents and pro-government fighters who had left Foua and Kefraya waited in rebel territory on Aleppo's outskirts to cross into the city, the British-based Syrian Observatory for Human Rights (SOHR) said.

READ MORE: 'Two years under siege, and it ends with displacement'

"The problem is described as logistical, but in reality they are trying to balance up the numbers," said Al Jazeera's Andrew Simmons, reporting from Beirut, Lebanon's capital.

"There are last-minute deals; some people are refusing to leave the towns, others are getting on the buses, some have been registered and others haven't," he added, describing the operation as "colossal".

"It's remarkably complex; civilians, fighters and their families, injured and sick, are involved in these evacuations - and the numbers have to be tallied up between the regime and the opposition."

The SOHR said the delay was because rebel fighters from Zabadani had not yet been granted safe passage out.

"It's a pretty grim situation for those involved," Simmons said. "Those who've been suffering all this desperation, all the sieges, they are going to a place they are not even sure of and having to leave families behind."

INTERACTIVE: Idlib - Who controls what

The SOHR said at least 2,100 residents, including rebel fighters, are due to be evacuated from Zabadani and Madaya and transported to Idlib.

The transfer is part of a deal brokered by Iran and Qatar that will see more than 10,000 people evacuated and hundreds of prisoners exchanged.

According to the United Nations, at least 4.7 million people are living in hard-to-reach and besieged areas in Syria and are exposed to grave security threats.

As the Syrian conflict enters its seventh year, more than 465,000 people have been killed in the fighting, more than a million injured and more than 12 million Syrians - half the country's pre-war population - have been displaced from their homes.