On 24 January 2015 a private jet touched down in Saudi Arabia’s capital, Riyadh. On board were a handful of Foreign Office officials, security personnel and the then prime minister, David Cameron, who was visiting the kingdom to pay his condolences following the death of King Abdullah bin Abdulaziz.

The decision to charter the jet – at a cost to the taxpayer of £101,792 – raised eyebrows among Whitehall mandarins. But when it comes to Saudi Arabia, normal UK rules don’t seem to apply. For decades the two kingdoms have quietly enjoyed a symbiotic relationship centred on the exchange of oil for weapons. Analysis of HM Revenue and Customs figures by Greenpeace EnergyDesk shows that in 2015 83% of UK arms exports – almost £900m – went to Saudi Arabia. Over the same period, the UK imported £900m of oil from the kingdom.

Now this relationship has come under scrutiny as a result of a judicial review brought by the Campaign Against Arms Trade (CAAT), which has sent alarm bells ringing in Whitehall. The case follows concerns that a coalition of Saudi-led forces may have been using UK-manufactured weapons in violation of international humanitarian law during their ongoing bombardment of Yemen, targeting Iranian-backed Houthi forces loyal to the country’s former president.

The legal challenge comes at a crucial time for the UK’s defence industry, which makes about 20% of arms exported globally. In recent years Ministry of Defence cutbacks have led to the sector looking abroad for new sales, and the government, with one eye on the post-Brexit landscape, is keen on the strategy. Last month Theresa May heralded a £100m deal involving the UK defence giant BAE and the Turkish military, and many defence experts see this as a sign of things to come.

But the policy – as the Saudi case makes clear – is controversial. Many of the UK’s biggest customers have questionable human rights records and there are concerns exported weapons are used for repression or against non-military targets.

Thousands have died in the Yemen campaign, with the Saudis accused of targeting civilians. Four-fifths of the population is in need of aid, and famine is gripping the country. But despite this, and protests from human rights groups and the United Nations, the UK has continued to arm the Saudi regime, licensing about £3.3bn of weapons to the kingdom since the bombing of Yemen began in March 2015.

UK arms sales to the Saudis are the very large tip of a huge iceberg. It is now more than six years since the start of the uprisings that became known as the Arab spring. Far from leading to a flourishing of democracy across authoritarian countries in the Middle East and north Africa, the political turmoil caused many to descend into chaos. In the immediate aftermath of the uprisings, the UK parliament’s committee on arms export controls concluded that “both the present government and its predecessor misjudged the risk that arms approved for export to certain authoritarian countries in north Africa and the Middle East might be used for internal repression”.

The misjudgment may have proved embarrassing politically, but it was a money-spinner for UK arms manufacturers. In the years leading up to the Arab spring, the UK sold those countries £41.3m of small arms, £7m of ammunition and £34.3m of armoured vehicles. In the five years following the events of 2011, these figures, compiled by CAAT, had risen sharply to an annual average of £58.9m, £14m and £59.6m respectively. In some cases sales skyrocketed. UK exports of small arms, ammunition and armoured vehicles to the United Arab Emirates jumped from £18.3m to £93.2m, to Qatar from £2.3m to £33.4m, and to Egypt from £2.5m to £34.7m.

“The 2011 uprisings should have caused countries like the UK to re-evaluate how they do business with the Middle East and north Africa, but they did no such thing,” said Andrew Smith of CAAT. “The arms sales have increased, even where the repression is getting worse.”

British soldiers display the use of their weapons at an arms exhibition in Abu Dhabi. Photograph: Rabih Moghrabi/EPA

Ostensibly, this flies in the face of the government’s perceived commitment to arms control. On Christmas Eve 2014, the Foreign Office minister Tobias Ellwood welcomed the entry into force of the Arms Trade Treaty. “This treaty sets common international standards for arms export controls, and puts international law and human rights at the heart of the global arms trade,” Ellwood said, noting Britain had led efforts to secure the agreement. “The ATT has the real potential to reduce human suffering, and increase security across the globe,” he added.

But internal Whitehall emails, released as part of the latest court case, reveal the alarm in the Foreign Office, MoD and the Department for Business as mandarins realised the extent to which the government’s commitment to the treaty and other legal obligations would be subject to forensic analysis by the judicial review. They confirm that the government came within hours of suspending arms sales to Saudi Arabia over concerns that there was a risk they breached international humanitarian law.

A memo, marked “sensitive” and written by an MoD civil servant, revealed that the then business secretary, Sajid Javid, wanted the exports stopped. Dated 12 February 2016, it noted he “has indicated that he will suspend exports if he does not have cross-government agreement [by the end of the day]”. The day before, Edward Bell, head of the government’s export control unit, sent an email saying his gut instinct was to suspend, “given acknowledged gaps in knowledge about Saudi operations”.

Surprisingly, Javid dropped his opposition to the exports continuing, having secured the backing of both the foreign secretary and the defence secretary for his decision.

A government spokeswoman would not say what changed his mind: “Given the current legal proceedings we will not be commenting further at this stage.” She added: “We operate one of the most robust export control regimes in the world and keep our defence exports to Saudi Arabia under careful and continual review,” she added.

But the court case raises questions about the efficacy of this scrutiny. Internal Whitehall memos show the business department had concerns that a large number of the airstrikes carried out by the Saudi-led coalition in Yemen were not being monitored, so there was no way of establishing if they were targeting civilians.

One memo states: “We are also concerned that FCO/MoD appear only to have insight into Saudi processes in respect of pre-planned strikes and have very little insight into so-called ‘dynamic’ strikes – where the pilot in the cockpit decides when to dispatch munitions, which account for a significant proportion of all strikes.”

In an attempt to establish the true picture of what was happening in Yemen, the MoD has been recording reports of the Saudi-led coalition airstrikes, summaries of which were revealed in the court case and which suggest alarming gaps in the monitoring process.

One report, dated 13 December 2016, noted that the MoD was tracking “244 incidents of potential concern of which around half are assessed as ‘likely coalition’ strikes. The MoD has been unable to identify a military target for around three-quarters of these ‘likely coalition’ strikes.”

Roy Isbister of the conflict prevention group Saferworld said it was “highly likely” that UK-built planes “have been used to drop large amounts of ordnance” in Yemen. Human Rights Watch has produced evidence that at least one UK-made Paveway IV bomb was used on a civilian target in May 2015.

F-15 jets at a Saudi airbase. Whitehall is concerned it does not have data on how the pilots decide when to use their weapons. Photograph: Fayez Nureldine/AFP/Getty Images

In response to mounting international outrage, the Saudi-led coalition set up a joint incidents assessment team to investigate civilian fatalities. “It has investigated 14 incidents in almost two years of conflict,” Isbister said. “The MoD is currently aware of 252 allegations, so that’s 5.5% of the total. At the current rate of investigation it will take 33 years to investigate everything – and that’s assuming there are no more allegations added to the list.”

The situation in Yemen became so grave that last December the MoD suggested running workshops “to provide targeting guidance” to the Saudi-led forces. “As well as helping to improve coalition processes, running such a workshop would provide a valuable opportunity to develop a better understanding of how the air campaign is being conducted,” a memo states. “This would assist with the ongoing assessment of IHL [international humanitarian law] compliance and may provide additional evidence to support defence of the judicial review.”

The high court ruling, expected in a couple of months, has potential implications for exports to Saudi Arabia and other members of the coalition, which include Jordan, Egypt, Kuwait, the UAE, Bahrain, Morocco and Qatar. Anything that threatens arms exports to these countries – several considered “priority markets” by UK Trade and Investment’s Defence and Security Organisation, the government body that promotes the defence industry – would be a setback for the UK’s commercial and geopolitical ambitions. Since 2011 the UK has sold weapons to many countries, including £40m of arms to Thailand, £490m to South Korea, £151m to Malaysia and £299m to Indonesia. But it is clear where it sees its future.

On her recent visit to the Gulf, May talked about how improved trade and security ties could help “embed international norms”. The prime minister said: “We in the UK are determined to continue to be your partner of choice. I am determined that the UK will be at the forefront of a wider western effort to step up our defence and security partnership.”

But other major arms-producing countries are threatening to undermine this ambition. In 2015, the latest year for which figures are available, UK defence exports fell nearly 10% to £7.7bn, despite the market for global arms exports rising significantly, up 17% to £63bn.

Within days of assuming office, President Trump approved $300m of precision-guided missile technology for Saudi Arabia and a $3bn F-16 deal for Bahrain – deals blocked by the Obama administration over concerns that the Gulf kingdoms might use the weapons in violation of human rights.

“Russia is incredibly keen to sell overseas on the arms side,” said Paul Rogers, professor of peace studies at Bradford University and international security consultant to the thinktank Oxford Research Group.

“What we’re not sure about yet is whether China will be big in this field. It has a very big domestic industry and it is producing some very robust hardware, both air and naval. What you might find overall is that the competition gets fiercer, and I think, in that sense, ethical dimensions, to put it politely, will diminish somewhat.”

This may be welcome news to an industry that has found the going tough in the past five years.

“From 2010 to 2015, austerity ruled and there were significant reductions overall in UK expenditure,” said Paul Everitt, chief executive of ADS, the trade body that represents the interests of the UK defence sector.

Everitt outlines three key reasons why the UK should promote defence exports. First, they help secure international relationships and promote UK influence, which builds national security.

“Foreign governments do not buy from an individual manufacturer – they buy from the UK, and the relationship they have is with the UK government,” Everitt explained.

Second, exports produce economies of scale in the manufacture and development of military equipment, which benefit the UK taxpayer.

Third, they are a success story. “We employ 150,000 highly skilled people across the country, thousands more are engaged through the extended supply chain, and we invest huge sums in research and development,” Everitt said.

It seems likely that the government will now use the shifting tectonic plates to build on this success. In a submission to parliament, Amnesty International suggests that the UK leaving the EU “will lead to a flurry of new trade and investment agreements that will chart a path for decades to come”. The campaign group said it hoped that “transactions and projects that may contribute to human rights violations” will not be promoted.

Rogers said: “Leaving Europe may loosen the regulatory framework, but at the same time there will be the push to get arms sales. There will be a greater look at other markets – India is a good example, maybe some of the Latin American states. It’s going to become more desperate.”