In the months after the surrender of Japan on 14 August 1945, the British people were ready to believe that war was behind them. The newspapers were full of stories about possible home rule for India, and dockers going on strike in London, Liverpool and Hull. It is questionable how many readers of the Manchester Guardian on 6 December 1945 saw, let alone read, a short item that was tucked away at the foot of page six, nestled between a reader’s letter about the Nuremberg war crimes trials and a leading article about the foundation of the United Nations.

Under the headline “British in Indo-China” appeared a copy of a letter that had also been sent to Ernest Bevin, the foreign secretary. “It appears that we are collaborating with Japanese and French forces against the nationalist forces of Viêt Minh,” the letter read. “For what purpose is this collaboration? Why are we not disarming the Japanese? We desire the definition of government policy regarding the presence of British troops in Indo-China.” The letter was signed by the “British other ranks” of the signal section of an infantry brigade based in Saigon.

It was highly unusual – notwithstanding the egalitarian spirit of those postwar days – to see a group of low-ranking British troops so publicly demanding that the foreign secretary explain his government’s policies. But what was truly extraordinary was the disclosure that British troops were fighting in the former French colony against the local population, and that they were doing so alongside their former enemies: the Japanese army and the Vichy French.

Few members of the public were aware that the British government had been so anxious to see the French recover control of their prewar colonial possession that the entire 20th Infantry Division of the British Indian Army had been airlifted into the country the previous August, with orders to suppress the Vietnamese people’s attempts to form their own government. There were almost 26,000 men with 2,500 vehicles, including armoured cars. Three British artillery regiments had also been dispatched, the RAF had flown in with 14 Spitfires and 34 Mosquito fighter-bombers, and there was a 140-strong contingent from the Royal Navy.

On landing, the British had rearmed the Vichy troops with new .303 British rifles. Shortly afterwards, surrendered Japanese troops had also been rearmed and compelled to fight the Vietnamese – some under the command of British officers.

The British were operating in accordance with an order that they should show a ruthless disregard for civilians, who, consequently, were killed and maimed in large numbers. “There is no front in these operations,” the order said. “We may find it difficult to distinguish friend from foe. Always use the maximum force available to ensure wiping out any hostilities we may meet. If one uses too much force, no harm is done. If one uses too small a force, and it has to be extricated, we will suffer casualties and encourage the enemy.”

Many of the troops who were expected to act on such orders were appalled. One of the signatories to the letter to Bevin was Dick Hartmann, a 31-year-old soldier from Manchester. Hartmann later recalled: “We saw homes being burned and hundreds of the local population being kept in compounds. We saw many ambulances, open at the back, carrying mainly – actually, totally – women and children, who were in bandages. I remember it very vividly. All the women and children who lived there would stand outside their homes, all dressed in black, and just grimly stare at us, really with … hatred.”

Back in the UK, parliament and the public knew next to nothing about this war, the manner in which it was being waged, or Britain’s role in it. And it appears that the cabinet and the War Office wished their state of ignorance be preserved.

At the Allies’ south-east Asia headquarters in Ceylon, however, and at the War Office in London, British commanders and senior defence officials were enraged by the letter. Hartmann and his comrades were warned that a brigadier was coming to see them.

“He just came in one morning and gave us a haranguing about the evils of our ways. He said a few years before we would have been shot, but unfortunately he couldn’t do that now.” Hartmann was worried. But some of his comrades had many years of jungle combat behind them and were unimpressed by the brigadier and his bluster. They told him, bluntly, that they believed Britain’s cause in the country to be unjust, and that he should make himself scarce. The brigadier turned on his heel, and did just that.

But there were no more letters from Saigon, there was little press attention, and almost no comments were made in the Commons. Despite the size of its military commitment to Indochina, this was to be a British military operation that would be kept out of sight, and largely out of mind. And it would not be the last such campaign.

Almost 70 years later, in September 2014, David Cameron, the British prime minister, gave a statement in which he prepared the country for the resumption of military action in Iraq, this time against Islamic State forces. “We are a peaceful people,” Cameron said, standing in front of two union jack flags. “We do not seek out confrontation, but we need to understand we cannot ignore this threat to our security … we cannot just walk on by if we are to keep this country safe. We have to confront this menace.”

Nobody doubted that the prime minister was under pressure to act after Islamic State had filmed the brutal murder of a British aid worker and threatened the slaughter of a second. Moreover, nobody disputed his assertion that the British are “a peaceful people” who do not seek confrontation.

In fact, between 1918 and 1939, British forces were fighting in Iraq, Sudan, Ireland, Palestine and Aden. In the years after the second world war, British servicemen were fighting in Eritrea, Palestine, French Indochina, Dutch East Indies, Malaya, Egypt, China and Oman. Between 1949 and 1970, the British initiated 34 foreign military interventions. Later came the Falklands, Iraq – four times – Bosnia, Kosovo, Sierra Leone, Afghanistan, Libya and, of course, Operation Banner, the British army’s 38-year deployment to Northern Ireland.

For more than a hundred years, not a single year has passed when Britain’s armed forces have not been engaged in military operations somewhere in the world. The British are unique in this respect: the same could not be said of the Americans, the Russians, the French or any other nation.

Only the British are perpetually at war.

One reason that this is rarely acknowledged could be that in the years following the second world war, and before the period of national self-doubt that was provoked in 1956 by the Suez crisis, Britain engaged in so many end-of-empire scraps that military activity came to be regarded by the British public as the norm, and therefore unremarkable. Another is that since 1945, British forces have engaged in a series of small wars that were under-reported and now all but forgotten, or which were obscured, even as they were being fought, by more dramatic events elsewhere.

A great deal is known about some conflicts, such as the 1982 Falklands war and the 2003 invasion of Iraq, and Britain’s role in the two world wars has become in many ways central to the national narrative. But other conflicts are remembered only dimly or have always remained largely hidden.

One strategically vital war, waged by Britain for more than a decade, was fought for most of that time in complete secrecy. In January 1972, readers of the Observer opened their newspaper to see a report headlined “UK fighting secret Gulf war?” On the same day, the Sunday Times ran a very similar article, asking: “Is Dhofar Britain’s hush-hush war?” British troops, the newspapers revealed, were engaged in the war that the sultan of Oman was fighting against guerrillas in the mountains of Dhofar in the south of the country.

Four years earlier, the devaluation crisis had forced Harold Wilson’s government to pledge that British forces would be withdrawn from all points east of Suez by December 1971 – the only exemption being a small force that was to remain in Hong Kong. Now the Observer article was demanding to know: “Has Britain really withdrawn all her forces from the Persian Gulf and the Arabian Peninsula? Or is the British government, like the Americans in Laos, waging a secret war without the full knowledge of parliament and public?” The Observer located one of the insurgency’s leaders, who told its reporter that the war had begun with an “explosion” in the country on 9 June 1965, triggered by what he described as poor local governance and “the oppression of the British”. By the time the Observer and Sunday Times were publishing their first, tentative reports, Britain had been at war in Oman for six-and-a-half years.

Facebook Twitter Pinterest Captain Charles Kendall, the British consul to the sultan of Oman, in 1969. Photograph: Tony Prime/ANL/REX/Shutterstock

Situated on the south-west corner of the Arabian peninsula, the Sultanate of Oman is bordered by the United Arab Emirates to the north, and by Saudi Arabia and Yemen to the west and south-west. The country also sits alongside the Strait of Hormuz, the 33-mile wide waterway through which oil from the Persian Gulf makes its way to market. In the 1960s, more than 60% of the western world’s crude oil came from the Gulf, a giant tanker passing through the Hormuz bottleneck every 10 minutes. As the oil flowed, local economies flourished and became important markets for exported British goods: London became even more anxious to protect its interests in the region and the local rulers who supported them.

Throughout the 19th and 20th centuries, Britain maintained control of successive sultans of Oman to prevent any other colonial power gaining a foothold in the region. It achieved this through a simple means: money. In the mid-1960s, the country’s tyrannical ruler, Sultan Said bin Taimur received more than half his income directly from London. Only from 1967, when Omani oil was pumped from the ground for the first time, did the country begin to generate most of its own income.

Even then, Britain exercised enormous control over the sultan. His defence secretary and chief of intelligence were British army officers, his chief adviser was a former British diplomat, and all but one of his government ministers were British. The British commander of the Sultan of Oman’s armed forces met daily with the British defence attache, and weekly with the British ambassador. The sultan had no formal relationship with any government other than that of the UK.

Officially, the Sultanate of Muscat and Oman was an independent state. In truth, it was a de facto British colony

The official British position was that the Sultanate of Muscat and Oman was a fully sovereign and independent state. In truth, it was a de facto British colony. As such, successive British governments were responsible for the woeful political, social and economic conditions that the sultan’s subjects endured, and which both created and fuelled the popular revolt.

In the mid-1960s, Oman had one hospital. Its infant mortality rate was 75% and life expectancy was around 55 years. There were just three primary schools – which the sultan frequently threatened to close – and no secondary schools. The result of this was that just 5% of the population could read and write. There were no telephones or any other infrastructure, other than a series of ancient water channels. The sultan banned any object that he considered decadent, which meant that Omanis were prevented from possessing radios, from riding bicycles, from playing football, from wearing sunglasses, shoes or trousers, and from using electric pumps in their wells.

Those who offended against the sultan’s laws could expect savage punishment. There were public executions. Conditions in his prisons – where Pakistani guards received their orders from British warders – were said to be horrendous, with large numbers of inmates shackled together in darkened cells, without proper food or medical attention.

The people of Oman despised and feared both their sultan and the British who kept him in place and colluded with his policy of non-development. Unsurprisingly, the sultan often had to call upon the British to provide the military force required to protect him from his own people.

During the 1950s there were a number of uprisings in the north of the country, which were put down by British forces. Both the SAS and the RAF were critical to the success of these counter-insurgency operations. Between July and December 1958, for example, the RAF flew 1,635 sorties, dropping 1,094 tons of bombs and firing 900 rockets at the insurgents, their mountain-top villages and irrigation works. This was more than twice the weight of bombs that the Luftwaffe dropped on Coventry in November 1940.

Facebook Twitter Pinterest The SAS on patrol in Oman in 1961. Photograph: Topfoto

In 1966, a new rebellion broke out in the south of the country, among the people of Dhofar province. The following year, after surviving an assassination attempt, the sultan and his Dhofari wife retired to his palace on the coast at Salalah. He was so rarely sighted that many of his subjects became convinced that he must have died, and that the British were concealing that from them.

For the new Labour government, the close relationship with the client sultanate presented an ideological problem. The Labour party had been elected in 1964 on a manifesto that included a pledge to wage a new “war on want” in the developing world, and to fight for “freedom and racial equality” at the United Nations general assembly. It would cause the most excruciating humiliation were it to become known more widely, at home and abroad, that Oman was the last country on earth where slavery remained legal. The sultan owned around 500 slaves. An estimated 150 of them were women, whom he kept at his palace at Salalah; a number of his male slaves were said to have been physically deformed by the cruelties they had suffered.

After the rebellions of the 1950s, the Sultan of Oman’s armed forces were reorganised, with British advice, training, equipment and funds. More Omanis were recruited into the ranks, but all of the officers were British. Some were “seconded officers” while others were so-called contract officers, or mercenaries – men who had previously served in Oman with the British Army and who had chosen to return to earn handsome rewards.

Initially, the rebels they faced in Dhofar were Arab nationalists. However, to the west of Dhofar lay Aden, from which the British were forced to withdraw at the end of 1967, in the face of increasingly violent rebellions. British rule had been replaced by a Marxist state, the People’s Democratic Republic of Yemen, which received aid from both China and Russia. By early 1968, a Dhofari nationalist insurgency was developing into a Chinese-backed revolutionary movement with pan-Arabian ambitions. To the British officers, however, the foe was always simply the adoo – Arabic for enemy. By the end of 1969, the adoo had captured the coastal town of Raysut, and by early the following year they controlled most of the high plains and were within mortaring distance of the RAF base at Salalah.

Any enemy corpses we recovered were propped up in the souk as a salutary lesson to would-be freedom fighters Anonymous British officer

The new oil fields on the desert between Dhofar and the capital, Muscat, were beginning to look vulnerable. Some in London were developing a fearful Middle Eastern domino theory, in which they envisaged the Strait of Hormuz falling under communist control.

The British response was merciless. “We burnt down rebel villages and shot their goats and cows,” one officer wrote. “Any enemy corpses we recovered were propped up in the Salalah souk as a salutary lesson to any would-be freedom fighters.” Another officer explained that unlike in Northern Ireland, where soldiers were anxious to avoid killing or wounding non-combatants, he believed that in Dhofar there were no innocents, only adoo: “The only people in this area – there are no civilians – are all enemy. Therefore you can get on with doing the job, mortaring the area and returning small arms fire without worrying about hurting innocent people.”

In their determination to put down a popular rebellion against the cruelty and neglect of a despot who was propped up and financed by Britain, British-led forces poisoned wells, torched villages, destroyed crops and shot livestock. During the interrogation of rebels they developed their torture techniques, experimenting with noise. Areas populated by civilians were turned into free-fire zones. Little wonder that Britain wanted to fight this war in total secrecy.

There was no need to resort to the Official Secrets Acts or the D-notice system in order to conceal the Dhofar war, and the ruthless manner in which it was being fought, from the outside world. Two simple expedients were employed: no journalists were permitted into the country, and nobody in government mentioned the war. When Wilson published his account of the Labour government of 1964-70, for example, he mentioned the war that the US was fighting in Vietnam almost 250 times. His own government’s war in Oman was not mentioned once.

While the Wilson government had every reason to be sensitive about the military support it was providing to a slave-owning despot, whose rule might charitably be described as medieval, there were additional reasons for the all-embracing secrecy. This was an era in which the developing world and the United Nations had rejected colonialism, and Arab nationalism had been growing in strength for decades. It was vital, therefore, for the credibility of the UK in the Middle East, that its hand in Oman should remain largely hidden.

Facebook Twitter Pinterest Omani rebels, called the adoo by the British, in 1971. Photograph: Keystone-France/Gamma-Keystone via Getty Images

John Akehurst, the commander of the Sultan’s Armed Forces from 1972, suggests a further reason for the British government not wishing to draw attention to its war in Dhofar: “They were perhaps nervous that we were going to lose it.”

Certainly, by the summer of 1970, Britain’s secret war was going so badly that desperate measures were called for.

On 26 July, the Foreign Office in London announced that Sultan Said bin Taimur had been deposed by his 29-year-old son, Qaboos bin Said, in a palace coup. In fact, the coup was a very British affair. It had been planned in London by MI6 and by civil servants at the Ministry of Defence and the Foreign Office, and given the go-ahead after the election that brought Edward Heath into Downing Street.

The new sultan immediately abolished slavery, improved the country’s irrigation infrastructure and began to spend his oil revenues on his armed forces. Troops from the SAS arrived, first as the sultan’s bodyguards, and then in squadron strength to fight the adoo. Eventually, the tide turned, journalists were permitted into the country, and by the summer of 1976 the war was won.

Strategically, the Dhofar war was one of the most important conflicts of the 20th century, as the victors could expect to control the Strait of Hormuz and the flow of oil. Thousands died, the British won and the west’s lights stayed on. Today, the war is still studied at the Joint Services Command and Staff College in Britain. But because of the way in which information about the long campaign was so successfully suppressed at the time that it was being waged, it has been all but blanked out of the nation’s memory. Like the British wars in Eritrea, Indochina, the Dutch East Indies and Borneo, it is remembered in Britain only by those men who fought it, and their families.

Some aspects of Britain’s role in the coup and the war remain among the deep secrets of the British state. Wilson’s correspondence on Oman, for example, and that of his successor Heath, are to remain closed to historians and the public until 2021. In 2005, a Foreign Office memo was briefly made public that describes the way in which the old sultan’s own defence secretary, Colonel Hugh Oldman, had taken the lead role in planning the coup that deposed Oman’s ruler, in order to safeguard British access to the country’s oil and military bases. The document was then hurriedly withdrawn – its release, the Foreign Office said, had been an unfortunate error.

Facebook Twitter Pinterest Qaboos bin Said took over from his father in 1970 in a palace coup orchestrated by Britain. Photograph: Keystone/Getty Images

Judging from the last decade and a half, there is little sign that the British state is about to lose its appetite for war. The first conflict of the new century in which the UK became involved was the post-9/11 assault against the Taliban regime in Afghanistan.

This war enjoyed early success, but stuttered and soured after the UK’s mission expanded to Helmand in the south of the country. The war dragged on, costing an estimated 95,000 lives over 13 years, including those of 453 British servicemen and women, and brought little discernible benefit to the people of Afghanistan. The 21st century’s second war – the 2003 invasion of Iraq – was possibly the UK’s greatest foreign policy disaster since Suez. Casualty estimates vary widely, from 150,000 dead to more than a million. What cannot be disputed is that 179 of the dead were British. More than a decade later, Iraq remains in chaos.

The post-9/11 conflicts in Afghanistan and Iraq were fought in the full glare of the media and came to haunt the politicians who had initiated them. Despite this, Britain continued to invest in war – politically, technically and financially – as a means of projecting power and securing influence among key allies, and also, it seemed at times, in an attempt to impose order and a degree of familiarity upon a chaotic and unpredictable world.

But could this be done in secret? Surely, in the age of global media, 24-hour rolling news, social media, and the troops’ own ability to record and instantly share images of conflict, it would be impossible for a British government to go to war and conceal its actions, in the way that Britain’s war in Dhofar was hidden from the public for six-and-a-half years? Tony Jeapes, who commanded the first SAS squadron that was covertly deployed to Oman, considered this question, and concluded that while such secrecy was “an ideal state of affairs”, it would probably be impossible to repeat.

In the years since the Dhofar war, the UK’s special forces have been gradually expanded, and since 1996, all its members have been obliged to sign a confidentiality agreement. This has reinforced the discretion with which members of elite units within the military traditionally perform their duties, and it has rarely been broken.

Meanwhile, the evolution of successive generations of unmanned aerial vehicles, or drones, has presented military planners with greater opportunities to mount operations that could remain unknown, other than to those who are ordering, planning and executing them, and to those on the receiving end.

The reliance of modern societies on the internet and the increasing frequency with which states probe and attack each other’s cyber defences have led some analysts to talk of a hybrid warfare, much of which is shrouded in deniability. The result is that the line between war and peace is increasingly blurred.

Facebook Twitter Pinterest An RAF Reaper. The development of drones has presented military planners with greater opportunities to mount operations that could remain unknown Photograph: Cpl Steve Bain ABIPP/PA

In the years after 9/11, hints began to emerge, in the footnotes of the budget statements of the Ministry of Defence, and from scraps of evidence salvaged from the coastal villages of Somalia, the mountains of Yemen and the cities of Libya, that the British were once again waging war in secret. It appeared that a lethal trinity of special forces, drones and local proxies was being brought to bear in a way that would spare the British public the disagreeable details of the nature of modern war, and relieve parliament of the need to debate the wisdom of waging it.

In July 2007, less than a week after succeeding Tony Blair as prime minister, Gordon Brown had announced a series of sweeping constitutional changes that he said would make the British government “a better servant of the people”. One measure – clearly a response to the deeply unpopular war in Iraq and the calamitous and costly expedition into Helmand – was to give members of parliament the final say on declarations of war.

Six years later, in August 2013, parliament exercised its new right when MPs rejected a government motion that would have authorised military intervention in Syria’s bloody civil war.

Ministers of the coalition government were appalled by the vote – it was said to be the first against a British prime minister’s foreign policy since 1782 – and argued that it not only blocked the deployment of British troops, it also prevented the UK from providing any military assistance whatsoever.

“It is clear to me,” Prime Minister David Cameron told the Commons, “that the British parliament, reflecting the views of the British people, does not want to see British military action. I get that and the government will act accordingly.”

But those words – “act accordingly” – were not quite what they seemed.

In July 2015, the defence secretary, Michael Fallon, gave MPs an update on the renewed military operations in Iraq – the campaign that Cameron had announced while standing before two union jack flags and declaring the British to be “a peaceful people”. The RAF, he said, had carried out 300 air strikes in Iraq, there were 900 UK personnel engaged, and the operation had cost £45m in the previous 12 months. He reassured his audience that “our position remains that we would return to the House for approval before conducting air strikes in Syria”.

Before making this statement, Fallon was said to have been unsettled by talk in Washington political circles that the UK’s refusal to act in Syria could be seen only as a sign of British decrepitude. His statement was deeply misleading: for at least 18 months, RAF pilots who were said to have been “embedded” with the US and Canadian military had been carrying out airstrikes against targets in Syria. Others had been flying combat missions with the French military over Mali. They were said to be under the command of these foreign forces, but they were clearly a British contribution to a war that MPs had decided the country should avoid.

Two weeks later the truth was out, and Fallon was back on his feet in the Commons, explaining himself to angry MPs.

“Embedded” service personnel were nothing new, he declared; they comply with UK law, but “have to comply with the rules of engagement of the host nation”. He had not publicised what had been happening because these pilots had been assisting with other countries’ operations. Moreover, he made clear that the failure to publicise what was happening should be regarded as “standard practice”.

British pilots took part in anti-Isis bombing campaign in Syria Read more

In December 2015, MPs voted that overt military action against Islamic State forces should finally proceed. The government was given parliamentary approval for military operations that had already been covertly under way for two years.

In the Gulf, meanwhile, it was disclosed that British military personnel were sitting in the control rooms from which the Saudi Arabian air force was guiding its bombers on to targets across Yemen. The British were helping their Saudi counterparts key in the codes that would help them select and attack their targets. The Saudis were not only flying British-built aircraft and dropping British-made bombs, they were dropping vast numbers of them. Over a three-month period in 2015, the value of exports of British-made bombs and missiles had increased by 11,000%, from £9m to £1bn.

This bombing campaign has been heavily criticised by rights groups for causing thousands of civilian deaths. In parliament, the British government has had little to say about this, other than to insist that it “obeys the norms of humanitarian law”.

Once again, the government appeared to be quietly pulling the country into a Middle Eastern conflict without any parliamentary oversight or approval. And covert, undeclared and unreported warfare could be seen to be not merely a possibility, but the reality of many of the UK’s military operations.

This piece is an edited extract from Ian Cobain’s study of official secrecy in the UK, The History Thieves (Portobello, £20). To order a copy for £16, go to bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min. p&p of £1.99.