One day in February 2014, a dusty and dented pick-up truck approached an Isis checkpoint outside the Syrian border town of Tell Abyad, carrying two men dressed in the simple djellaba robes and loose keffiyehs worn by local farmers. The fighter on duty checked their identity cards and cast an eye over the fertiliser bags and scraps of wood piled in the back of the vehicle. The driver and his passenger said they were in the area to visit relatives, and the fighter waved them through.

The two men drove across the Turkish border, having cleared the last – and potentially most lethal – obstacle on a long clandestine journey. Hidden under the sacks of fertiliser in the back of the truck was a batch of documents salvaged from the battlefields of Syria’s bloody conflict, and smuggled across the country at enormous risk. Amid the thousands of pages of military orders and government reports that had just come across the border was vital evidence of war crimes, which could one day form the core of an international prosecution of Bashar al-Assad and his regime.

The driver of the pick-up, a stocky man in his 40s named Adel, was the chief investigator for the Commission for International Justice and Accountability (CIJA). An independent organisation set up by experts on humanitarian law, with funding from western governments, its aim is to collect evidence of atrocities committed by the Syrian regime and opposition, in preparation for the day when they can be judged by a tribunal. Adel and his team of 50 investigators had made many such trips in the three years since the CIJA was established, but these smuggling runs through Tell Abyad in the first months of 2014 would prove to be the most fruitful. They were carrying the greatest find of the investigation so far: a complete set of documents from the provinces of Raqqa and Deir ez-Zor, which provided a clear picture of the regime’s machinery of repression, and showed how tightly it is controlled by Assad and his inner circle.

Adel had visited Raqqa and Deir ez-Zor in December 2013, with introductions from mutual acquaintances to a handful of the commanders of Islamist militias in the region. These militias had scored a string of victories over the Syrian army earlier that year, seizing government buildings in the process. Adel was interested in what was inside these buildings – the paperwork left behind in filing cabinets and underground archives. In Raqqa, leaders of the local Salafist militia offered to help collect what Adel was looking for; over the next few days, they came to him with plastic bags and cardboard boxes full of papers from the abandoned secret police headquarters in the town of Taqba and from Raqqa city itself.

The regime's security branches kept copies of orders coming down from Damascus and the reports going up the chain

In Deir ez-Zor, the situation was more complicated. The dominant military force there was the Nusra Front, an al-Qaida offshoot opposed to any venture associated with the west. But one of the group’s local commanders – a man of “grace and education”, according to Adel – agreed to covertly provide assistance. His fighters allowed Adel’s investigators to comb through the military intelligence building and sweep up the files and loose papers scattered around its deserted shell.

By January 2014, Adel’s archive had rapidly grown to fill a dozen boxes – about 150kg of paper – which were stacked against the walls in a house he had rented in Raqqa. He had collected documents from many abandoned government facilities in his earlier sorties into Syria, but had never seen anything like this. “I opened the first box of documents and I saw right away how important they were,” he said. “They were from the security service, not just the military, and they provided a blueprint of how things happened in the regime’s security apparatus. What was particularly important in the documents from Raqqa and Deir ez-Zor was that their security branches kept copies of orders coming down from Damascus and the reports going up the chain. They provide the vital linkage evidence of crimes occurring on the ground.”

* * *

As investigators, Adel and his team perform what has to be one of the world’s most dangerous jobs, repeatedly diving into the murderous chaos of Syria to follow a paper trail of war crimes. While attempting to smuggle a batch of documents across the northern Syrian border, one of Adel’s colleagues was ambushed and killed. Another was shot four times when his car came under fire from a government position. Several have been detained and tortured. Adel has come close to death several times. In Daraa last year, a regime helicopter dropped a barrel bomb on a stone house next to a home he was visiting, sending out a lethal spray of stone and shrapnel. “In my mind, it is as if I can still hear the explosion,” he said.

Adel trained as a lawyer in Daraa province, in south-western Syria, under a judicial system built on the foundations of French law – almost all of which was negated by an emergency law imposed in 1963, when a military coup brought the Ba’athist party to power. Assad made a show of lifting emergency rule in March 2011, in an apparent concession to the Arab spring protests that had spread to Syria. But even as the repeal announcement was being made, government forces were opening fire on crowds of unarmed demonstrators, while thousands of dissidents were being rounded up and imprisoned, never to be seen again. The cruelty with which the civil unrest was crushed ignited the civil war. Adel was living and working outside Syria when the conflict started. “I knew right away that I had to be involved in some way,” he told me. “I could not sit outside and do nothing.”

He attended UK-funded human rights seminars in Turkey and joined the CIJA in 2012, soon after it was set up by veterans of the international criminal court (ICC) and war crimes tribunals for the former Yugoslavia and Rwanda. Initial funding came from Britain, the United States and the European Union. Adel’s eyes move constantly as he speaks, and the skin around them has turned as dark as the black stubble on his jowls. His family has suffered unbearable losses. His sister’s family fled to Jordan in 2011, but after a few months, they sent back his 16 year-old nephew, Hamid, to pick up some possessions left at the family home. He was detained by the Syrian security services as he was crossing the border, and then transferred to a detention centre in Damascus, where he is thought to have been killed within a few weeks. At about the same time, another nephew defected from the army to join the FSA and died in battle, at the age of 21. One of Adel’s uncles was tortured to death in prison.

I met Adel earlier this month in a hotel room in a Gulf city with a translator and another CIJA colleague, while he was on a break between Syrian collection trips. He looked like a local businessman on his day off, wearing a short-sleeved checked shirt and beige trousers. But it was clear he felt ill at ease. He gripped the arms of his chair and refused all offers of tea, or even water.

Evidence against the Assad regime is smuggled out of Syria by truck. Photograph: Commission for International Justice and Accountability

Going in and out of Syria has inflicted a psychological and emotional whiplash, moving from the horrors that have consumed his family and his country to the prosperous normality of the Gulf. His immediate family is now safe outside Syria but goes through bouts of worry whenever he travels back across the border. Adel is a pseudonym. His real name and other details of his life cannot be published, in order to safeguard his security.

The risks keep getting steeper. Three days after Adel left Raqqa in January last year with his load of incriminating documents, Isis swept into town and began executing commanders and fighters from the Nusra Front and Ahrar al-Sham. Adel retreated to Deir ez-Zor and spent the next month looking for ways to sneak his stash of documents through Isis checkpoints. He split the load into smaller consignments so they could not all be destroyed or captured at once. All of them got through.

Adel and his team cannot go back to Deir ez-Zor. He has heard from his contacts that Isis commanders are aware there were investigators from a western organisation operating under their noses, and have vowed to kill them. Adel’s team in the city has now been withdrawn. The papers they have gathered since January 2014 – with more vital evidence of regime atrocities – have been buried underground, to be recovered at a time when transporting them across the border will not be suicidal.

* * *

More than 220,000 people are thought have been killed in the Syrian civil war so far, more than double the death toll of the war in Bosnia two decades ago. It is hard to see any end to the country’s agony: the Assad regime has stepped up the use of barrel bombs dropped on residential areas; Isis continues to execute civilians and post grisly videos of its massacres online. Unlike the Bosnian, Rwandan and Cambodian genocides, however, there is no tribunal to judge war crimes and crimes against humanity in Syria, nor is there likely to be in the foreseeable future. The international appetite for such courts has waned in recent years, along with faith in the very notion of universal justice for mass atrocities. Russia, the Assad regime’s most powerful ally, has blocked any attempt to create a court.

However, as part of its support to opposition and civil-society groups in 2011, the UK Foreign Office earmarked a few hundred thousand pounds to fund training for Syrian human rights lawyers and activists to research and record war crimes. William Hague, the foreign secretary at the time, was very aware of the failure of the Tory government of the 1990s to stop the slaughter in Bosnia and its subsequent obstruction of the international tribunal, which it believed would disrupt peacekeeping efforts.

To carry out the training, the Foreign Office turned to Bill Wiley, a former Canadian infantry officer who completed his doctorate on the laws of war while in uniform. He went on to work as a legal expert and investigator at the Yugoslav and Rwanda tribunals as well as the ICC. “I never saw a dead guy or had anyone shoot at me until after I left the army,” he said.

When the British hired him, Wiley was running his own company, Tsamota Ltd, which managed legal-reform projects around the world on behalf of western donors. He agreed to take on the job, but on his own terms. These were the product of firsthand experience of earlier war crime investigations and their shortcomings.

“I said I would do it on the understanding that we wouldn’t do human rights training, because it’s a waste of time and money. Everybody’s doing it and it’s bullshit,” Wiley recalled telling the Foreign Office. By human rights training, he meant broad-brush briefings on a wide array of desirable rights – social and economic as well as political and humanitarian. Wiley had in mind something much more focused.

He told the British diplomats: “What we’ll do is take these activists – young, enthusiastic, self-appointed, social-media-savvy, with considerable personal courage – and we’ll sensitise them to the sort of evidence that is required to inform an international criminal case, so they’re not running around collecting stuff that’s of no use for a criminal process.”

Wiley and Tsamota carried out three training sessions for Syrian volunteers in Istanbul at the end of 2011 and the beginning of 2012. Adel was one of the early trainees. Starting in October 2012 they held follow-up practical sessions in the Lebanese cities of Beirut and Tripoli, and along the Lebanese-Syria border. The trainees observed CIJA personnel interviewing defectors. The nature of the training, and the risk to the trainees, required a certain amount of discretion. CIJA block-booked rooms at small hotels in Istanbul that did not require guests to show their passports. In Lebanon, the CIJA’s local fixer ensured they did not stay in any establishment which might leak details to the Syrian regime’s allies in Hezbollah.

A child walks past damaged buildings in Deir ez-Zor in 2013. Photograph: Khalil Ashawi/Reuters

This British-funded programme, however, raised an unavoidable question. If these 60 brave Syrians were to be sent back into the country to collect court-usable evidence, to whom should they present it and to what purpose? With no court or prosecutor’s office ready to handle it, was it worth risking their lives?

In an effort to answer that question, Wiley and a group of former colleagues from the Hague tribunal put together a joint proposal in 2012 to set up an investigating body that would collate and analyse the documentation, and build prosecution cases. But the suggestion initially bewildered the officials they approached. “We were naive,” Wiley recalled. “This had never been done privately before. People who work for states tend to be kind of statist.”

Wiley, a lanky blond man of 51, has a manner that is both laconic and blunt. He goes to the CIJA offices most days in T-shirt, cargo trousers and lightweight walking shoes, and generally stays late. He takes a break for a couple of weeks each year in a cabin in the wilds of Newfoundland, where he fishes for trout, and hunts rabbits, partridges and moose. In the relatively small world of war crimes investigators, he is widely known both for his work in the field and his academic papers on evidence-gathering methodology. Running the CIJA, however, has taken him further into the world of diplomats and government bureaucracies that he would otherwise have liked, and his description of his dealings with them is tinged with impatience.

The question of whether war crimes investigations would have any credibility if they were run by profit-driven companies was a big obstacle. In an attempt to sidestep it, Wiley set up a non-profit organisation, initially called the Syrian Commission for Justice and Accountability, before it was renamed as the CIJA in 2013. It was registered in the Hague, which has emerged as the world’s primary centre for international criminal humanitarian law, hosting the ICC, the Yugoslav and Rwandan tribunals and the Special Tribunal for Lebanon.

We were naive. This had never been done privately before. People who work for states tend to be kind of statist Bill Wiley

“We set to work raising money. It was a nightmare,” Wiley said. “We had never done it before. Frankly we didn’t know what we were doing, in addition to the idea that we were selling something that most people believe only states can, and indeed should, do.”

However, if any kind of hope was to be kept alive for an eventual accounting for Syria’s many war crimes, there were no real alternatives. Moreover, Wiley and his team had worked in the engine rooms of almost all the world’s contemporary war crimes courts. They were not widely known names, but the whole idea behind the project was to be quiet and effective. It would not be the sort of operation that needed a brass plaque. In fact, the CIJA headquarters, in a nondescript suite of offices in a western capital that cannot be named for security reasons, does not have a plaque at all. The pale stone lobby is featureless and anonymous.

After uncertain beginnings, the organisation is now on a solid financial footing. Following an initial £1m to fund the training sessions for investigators, the UK contributed another £800,000, the EU provided €2m and the US gave $1m. Additional funding came from Switzerland, Germany, Norway, Canada and Denmark.

As a consequence, Wiley has been able to afford to recruit analysts to process Adel’s haul of documents. In the CIJA offices, the batches of smuggled papers are unpacked, and each page is scanned, labelled with a bar code, and filed. There are now half a million pages of original documentation in the archives, about 2.3 tonnes of paper, stacked on shelves in numbered boxes. About 80% of it has so far been scanned, and the digital versions of the documents are reviewed by a senior analyst to determine which should be translated into English for broader scrutiny.

Meanwhile, in a city in south-eastern Europe, a team of Syrian refugees has the soul-crippling job of watching the many videos that have been smuggled out from the conflict. More than 470,000 videos have been downloaded on the CIJA server, thousands of hours of gruesome viewing, from the aftermath of regime barrel bomb attacks on schools to Isis’s elaborately staged mass beheadings. Each video is analysed and key data – names of places, victims or perpetrators – extracted. The staff have regular counselling sessions to handle the daily trauma of their jobs.

The video evidence will be most relevant to a parallel investigation of opposition war crimes, while the regime investigation is driven principally by the document archive. Witness interviews only began last May, once the documents had taught the investigators whom to look for and what questions to ask. So far 380 interviews have been conducted, including 49 with defectors up to the level of lieutenant colonel.

For Wiley, the CIJA’s tonnes of documents represent the concrete from which his cases will be built. He pointed out that the first witness called at the Nuremberg trials was the Third Reich’s state archivist, and he showed me a graphic to illustrate his approach. It was a layered pyramid, with the words “crime base” on the bottom level, “perpetrating organisations” above that, and “leadership” on top.

“At the beginning, we don’t do much with the crime base,” he said. “In a situation like Syria, the criminality is so widespread and deep that you’re not worried about specific incidents at the beginning. Any state institution or security intelligence agency is going to be involved in criminality in some way. What we focus on is understanding the perpetrating organisations. So this is 80% of the work.”

At the Hague tribunal for the former Yugoslavia, where Wiley worked between 2000 and 2003, he reckoned 80% of the investigative effort was on the crime base. This was in part because the first investigators were seconded policemen and that was how they did their job back home. They drew a chalk line around the body and started interviewing eye-witnesses. Consequently the trials of the former Serbian president, Slobodan Milosevic, and his top lieutenants, were clogged with heartbreaking testimony from the scenes of crimes, with very little documentary evidence directly incriminating the leadership. Milosevic’s trial had entered its fifth year when, in March 2006, he died in cell of a heart attack. His secret police chief and his immediate underling were later acquitted for lack of direct evidence linking them to atrocities carried out by paramilitaries under their command.

Wiley’s strategy is to invert the pyramid, standing the old methodology on its head, starting with the paperwork produced by the state instruments of repression and finishing with a few well-documented illustrations of the abuses they have committed. Syria represents a near-ideal case study for a paper-driven approach.

“It is highly bureaucratised,” he said. “It generates an awful lot of paper, because it is a culture in which decision-making by subordinates is implicitly discouraged, so people are forever reporting upwards, trying to get others to take responsibility for decision making, and covering their ass … that produces paper.”

* * *

On his first trip to Syria after his training, Adel went to the frontlines near the town of Khanasser, in Aleppo province, where rebel forces had come under attack from a Syrian regime air defence unit, which was targeting civilian vehicles with anti-aircraft guns. On a winter evening in December 2012, Adel met with officers from one faction of the rebel Free Syrian Army (FSA), which had made significant territorial gains in the area, capturing government facilities and military bases. Adel asked about the fate of the office records, pointing out they would be essential in writing the real history of the Syrian regime. The FSA officers were astonished.

Bill Wiley trains Syrian volunteers to be war crimes investigators. Photograph: Sander de Wilde

“They told me if they had known how important the papers were, they would never have burned them,” Adel said. A few days later they shamefacedly brought him a few plastic bags full of documents they had salvaged. The rain was pouring, and much of the paperwork was sodden, but Adel was able to rescue a few scraps, including a potentially significant series of exchanges between Ba’ath party political commissars in the military. He stuffed them in his computer bag, and was driven out of the area in an black FSA Mercedes, using the cover of thick fog to creep past the 23mm guns of the trigger-happy air defence unit.

A few months later, in his home province of Daraa, Adel obtained a stash of regime security service documents in the area of al-Lajah, where the battle lines between rebels and the Syrian army were constantly shifting. He decided it was too dangerous to move around with the papers, and set out to find somewhere to hide them. “We found a shepherd’s shack, a very simple place, which he thought the government would never search,” Adel said. “The shepherd agreed to keep the documents, but he did not tell his wife. She knew nothing about it. He went off to tend the sheep and while he was gone, his wife used the paper as fuel to do the cooking. When I came back, all there was left were charred remains in the oven.”

When Adel and his investigators come across a cache of abandoned documents, they must make an assessment of their overall significance, but they are not supposed to winnow out individual pages in search of an elusive “smoking gun” memo. This is partly to prevent the exclusion of potentially exculpatory evidence, and therefore to pre‑empt possible procedural objections from the defence in any future trial. But ostensibly mundane documents often prove to be the most damning. When Wiley was raising funds from donor governments in the early days of the commission, he warned them not to expect sufficient evidence to indict Bashar al-Assad and the top leadership. Wiley’s initial assumption was that CIJA would end up, at best, building cases against regional commanders. But the regime’s mania for documenting orders as they travel down the chain of command – and after-action reports that flow back upwards – has meant that the paper trail led unexpectedly to the very centre of power in Damascus.

The regime inadvertently made following that trail even easier: senior officials typically sign each document as the papers cross their desks; in the ministries, these autographs are accompanied by stamps identifying the signatory. Some of the signatures are even colour-coded. (The ministry of interior always uses green ink.) The intelligence agencies are more slippery – they do not use signature stamps – but their efforts at anonymity were undermined by one police note-taker, whose minutes from a 2011 security agencies meeting include a tally of every person present, with their names and job titles listed alongside their signatures. That document has since helped the analysis cell decipher the names scrawled on scores of others.

From this forensic analysis, the CIJA has been able to construct a detailed picture of the Assad regime’s wartime structure, and the direct involvement of senior officials in the commission of atrocities. At the apex is Assad’s war cabinet, the Central Crisis Management Cell (CCMC), which includes the heads of the armed forces, the ministers of defence and the interior, along with the chiefs of the dominant intelligence agencies. The CCMC convenes every day, and the minutes of each meeting, along with recommendations from the security chiefs, are taken immediately by courier to Assad. According to recovered documents and insider witness testimony, the courier waits at the palace for Assad to review the minutes and then drives the annotated versions back to the CCMC, which sends the president’s approved directives down the chain of command.

Underneath the CCMC is the National Security Bureau (NSB), the key apparatus of repression in wartime Syria. The NSB is where the heads of the four main intelligence agencies coordinate their decisions. It directly controls the security committees in each governorate – each one chaired by the provincial head of the Ba’ath Party – which carry out the orders of the CCMC. This structure, according to Wiley, demonstrates the party’s transformation from a largely ineffectual peacetime organisation into a ruthless enforcer of wartime loyalty. “You could almost say the Ba’ath party comes into its own as an element of the state once the war starts,” he said. “It’s really not important prior to 2011. People join for opportunistic reasons, and it has no meaningful ideology at its centre. But it is utterly central to the regime organising itself to hang on to power.”

The CIJA is investigating the Syrian government’s wartime conduct – including its use of barrel bombs and chemical weapons – as well as the atrocities committed by opposition extremists. But it has already compiled three separate cases against the regime – including 25 named individuals – for earlier crimes against humanity, based on documentation and witness testimony. All three cases focus on the violent suppression of protests at the start of the conflict, when tens of thousands of people were detained, many of whom were tortured and killed. The first is against the CCMC, and specifically against President Assad, the interior minister Mohammad al-Shaar, and Mohammed Saeed Bekheitan, an assistant secretary in the Ba’ath Party who headed the CCMC for its first six months of operation, until September 2011.

The second is against the NSB, which includes the heads of the intelligence and security agencies. The third case pinpoints the Deir ez-Zor security committee, which also controlled the intelligence agencies in neighbouring Raqqa province. Adel’s smuggled document cache has provided a complete case study of how precise orders to crush the popular uprising flowed from Damascus to the governorates.

A memo dated 5 August 2011, from the NSB to the Deir ez-Zor security committee orders the formation of “investigation committees” to carry out the arrests of protest organisers as well as “those communicating with people living abroad, and those who tarnish the image of Syria in the foreign media and international organisations”. Documents show the orders filtering down to local branches of the political security agency, along with wanted lists of dissidents to be arrested. Reports flow back confirming the arrests of named suspects and the reason for their detention, which in some cases amount to no more than “discussing the events in a negative manner”.

Those arrested were kept in massively overcrowded holding pens, in which prisoners were squeezed together for days or weeks, until many died. Common torture methods have been documented across several governorates, suggesting a centrally guided policy. They include the dolab, in which an inmate is stuffed inside a tyre and beaten, and the shabeh, in which he is suspended by his wrists. Also of note is the bisat al-reeh (flying carpet), in which a prisoner is strapped to a wooden cross with a hinge in the middle, and the two halves of the cross then forced together. The documents show that “high-value” prisoners are filtered out of the system and moved up the chain to Damascus. It is this forensic level of documentation which the CIJA believes sets its work apart from other human rights enquiries, like the UN Independent International Commission of Inquiry on Syria, which has also accused the regime of war crimes. Wiley’s team say they have written and documented their cases in such a way that they are ready to go to court.

There are thousands of names of victims, but once the mechanism of repression has been established through the documents, Wiley points out, “you don’t need a lot. To put Assad in jail for murder, you just need one victim.”

* * *

The CIJA has painstakingly corrected many of the mistakes of past attempts to prosecute war criminals, and built robust court-ready cases against the Syrian leadership, but for now there is no court to hear them.

There is interest in prosecuting war crimes from various western judicial systems, but principally with a focus on jihadist opposition groups. Video footage, along with other open-source material documenting atrocities, offers a ready-made package for European prosecutors to bring cases against homegrown jihadists who have travelled to fight in Syria. But that is very far from a comprehensive accounting for a brutal war, and it provides no arena for the considerable evidence that Assad and his regime directly and deliberately orchestrated the mass killing of civilians. Moreover, there is no sign that Moscow will reconsider its veto on referring the Syrian war to the ICC, despite the fact that opposition extremists would also be on trial. Even if the Russians relented, the ICC has only shown a capacity to manage a handful of cases at once.

For the CIJA’s supporters, however, the absence of a court is no reason to stop collecting the evidence. If such work was put off until a tribunal was established, they argue, much of the evidence would have evaporated. “This organisation includes some of the more effective people who have worked in the other tribunals,” said Stephen Rapp, the US ambassador-at-large for war crimes issues. “Wiley has more people in his investigative division than the ICC has covering the whole world, and these are folks, both the internationals and the Syrians, who know what they are doing. They are doing this according to standards followed by criminal justice agencies whose procedures have stood the test of time.”

Rapp argued that the collection of evidence, and the compiling of cases, will in itself build pressure for the eventual creation of a war crimes tribunal for Syria. “I know some of the investigators are frustrated and they want this to matter,” he said. “But it will never be for naught. We had documentation centres in Cambodia working for 10 years before there was a court. Documentation on the holocaust is continually being mined by scholars and families. There will come a point at which the people of Syria will seek to memorialise these crimes, and having that kind of depth of information is of immense value, no matter what.”

For now, Adel is biding his time in the Gulf, and planning his next trip into Syria some time next month. The recent military losses suffered by the regime are likely to bring more documentation to the surface as the Syrian army and security services retreat from former strongholds. Furthermore, there are signs that the regime is beginning to consume itself under pressure. On 24 April, the head of the Political Security Directorate, Rustum Ghazaleh, died in hospital after a clash with the head of military intelligence, Lieutenant General Rafiq Shehadeh. According to some accounts, Shehadeh beat Ghazaleh unconscious with his own fists. (Other reports say Shehadeh’s bodyguards administered the beating.) In either case, such violent infighting at the centre of the regime suggests that new high-level defections are likely. Rather abruptly, a scenario in which Assad falls – opening the way to a possible peace deal, and ultimately some form of judicial reckoning – no longer looks quite so far-fetched.

Our interview drew to a close as evening was falling. Adel was eager to get back to work: he had a series of trips to make around the region, meeting with members of his investigative teams as they emerge from Syria, and interviewing regime defectors. He also hoped to spend at least a little time with his family. Before he left the hotel, he insisted that he was not demoralised by the fact there was not yet a court to hear the evidence he is risking his life to collect. “I believe in this work,” he said. “So many people are being killed, and this is the one weapon we have.”

• Follow the Long Read on Twitter: @gdnlongread