Today the Foreign Office has made public the first batch of thousands of "lost" colonial-era files believed to have been destroyed.

The documents were secretly sent back to the UK when former colonies became independent. They shed new light on how British officials ran countries such as Kenya, Cyprus and present-day Malaysia.

Thousands of documents detailing some of the most shameful acts and crimes committed during the final years of the British empire were systematically destroyed to prevent them falling into the hands of post-independence governments. Those papers that survived were flown back to Britain and hidden for 50 years in a secret Foreign Office archive in breach of legal obligations for them to be transferred into the public domain.

My colleagues Ian Cobain, Owen Bowcott and Richard Norton-Taylor were among the journalists who gained access to the files. Here they explain exactly how the documents came to be published and run through some of the most interesting material they found in them, including monthly intelligence reports on the "elimination" of the colonial authority's enemies in 1950s Malaya (now part of Malaysia), records showing ministers in London were aware of the torture and murder of Mau Mau insurgents in Kenya, including a case of a man said to have been "roasted alive", and papers detailing the lengths to which the UK went to forcibly remove islanders from Diego Garcia in the Indian Ocean.

Here's what they have published from the files today:

• The name of Barack Obama, the father of the American president, is on the top of a list of names revealed in a hitherto secret British colonial file of Kenyans studying in the US.

• In June 1957, Eric Griffiths-Jones, the attorney general of the British administration in Kenya, wrote to the governor, Sir Evelyn Baring, detailing the way the regime of abuse at the colony's detention camps was being subtly altered.

• The "elimination of ranking terrorists" was a repeated theme in secret monthly reports on casualty figures circulated by the director of intelligence in British-controlled Malaya during the 1950s.

• The extent to which successive British governments set out to hoodwink parliament and the public over the decision to give the US a military base in Diego Garcia and force out the islanders is now laid bare.

• Britain planned to test a very virulent type of poison gas in what is now Botswana, the colonial archives reveal.

And Caroline Elkins of Harvard University writes here that Foreign Office transparency is a carefully-cultivated myth.

I'll be looking through what other newspapers have discovered this morning and updating this blog with what I find.

Meanwhile if you want to ask Ian Cobain, Owen Bowcott and Richard Norton-Taylor more about what they found and what kind of records are there, please add a comment below.

How you can see the documents yourself

As yet the documents exist only in physical form at the National Archives in Kew, south-west London – but they are open to the public.

To view them, go to the National Archives, Bessant Drive, Richmond, TW9 4DU. The nearest train station is Kew Gardens or Kew Bridge.

You will need to get a readers' ticket – to get this bring two forms of ID (one with proof of name and signature and one with proof of address – acceptable forms of ID are listed here). The process may take up to 30 minutes. Once you have a readers' ticket it is valid for three years.

Then go and look in the catalogue to search for the records you want to look at. The records are not easy to search so National Archives staff have produced a guide – which you can view online here – to help you do so.

You can then order the documents you want and look at them in the reading room.

There are free public talks at 11am and 2pm in the National Archives' reading rooms today on how to access the records.

Here are the opening times of the National Archives:

Monday: closed

Tuesday: 9am-7pm

Wednesday: 9am-5pm

Thursday: 9am-7pm

Friday: 9am-5pm

Saturday: 9am-5pm

Sunday: closed

If you get the chance to go there and you find some records you think we should be writing about, please contact me at paul.owen@guardian.co.uk and let me know what you have found.

Owen Bowcott explains how a lot of what happened to the files was due to a blend of intentional destruction and bureaucratic incompetence. Owen writes:

The destruction of British colonial records in Kuala Lumpur was more haphazard than in Kenya. A file from the transferred Hanslope Park archive entitled "Disposal of Surplus Secret Records" (see photo at the top of this blog) contains lists of files to keep and others no longer needed. The contents of "List A" are marked: "Suggestion destroy all." "List B" files were to be transferred to the Colonial Office in London - though judging from named files not all reached the National Archives in Kew. The third category, "List C", includes some deemed of historical interest but others marked "destroy". A "Law and Order" sequence from 1948, which might have covered the alleged Batang Kali massacre of 24 villagers by British troops, were not kept. They include files marked "situation reports", "intelligence", "internal security committee", "supplies", and "emergency meetings LDC". Explaining the rationale for the selection, a relatively junior official in Kuala Lumpur, WJ Watts, wrote: "The strong room here is very overcrowded and the secretary of external defence has asked me to see if some of the older records cannot be cleared out … Presumably [some] are duplicated [in other archives]. If this is the case would you agree that they might be destroyed? ... List C deals with several bundles of old King's House files. I have been through them and it would seem that some contain items of historical interest in the event of anyone writing a history of the emergency or biography of former commissioners. "The others should be dealt with in detail but I have not time to do this. Would you agree to their disposal as suggested against individual files in the list. Do you think his excellency's approval is necessary?" Thus many key records may have been lost without even reference to senior officials.

Sam Marsden of the Press Association has also been through the files. He reveals:

• Britain had already made plans to deport Greek Cypriot leader Achbishop Makarios to the Seychelles when it launched talks with him about ending a violent rebellion in the 1950s. Whitehall ruled out holding Makarios in the former house of the chief justice of the Seychelles, called La Bastille, promoting Seychelles governor Sir WIlliam Addis to compose this rhyme:

Well in Sans Souci so let it be

Thus denying opposition opportunity

Of stating Archbishop ne peut pas rester tranquille [cannot sit still]

In a house with the name of La Bastille.

• Germans who claimed to be catching butterflies in pre-second world war Kenya caught the attention of British officials who feared the Nazis were plotting to invade east Africa. Two German men found camping in Kenya's Rabai Forest in August 1938 - one a suspected Nazi agent and the other a Palestinian of German descent - told British officials they were engaged in butterfly-catching. They claimed they had been recommended to go there by a well-known naturalist, but it turned out that he did not know them.

• On 5 January 1955 a British district commissioner seized a total of 30 sheep from four members of the Kikuyu tribe who worked on the farm of a Mr SJO Armstrong in the Naivasha district of Rift Valley Province. A short file on the case reveals that these were all the animals they owned. The action was prompted by suspicions that they had "harboured and fed" a 40-strong Mau Mau gang for a fortnight. An official wrote: "Owing to the fact that the Kikuyu labour was totally unco-operative and showed no signs of assisting security forces, native stock was seized."

Professor Richard Drayton, professor of imperial history at King's College, London, has emailed to say that he feels the release of the documents has been discredited by the Foreign Office's claim that the secret archive contained no sensitive documents from British Guiana. Drayton writes:

British Guiana saw one democratically-elected government overthrown by British troops in 1953, a heavy British security presence in the late 1950s, and then from 1962-64 a very well-documented joint British and American covert operation, which involved terror bombings, detention of politicians, and black propaganda. This United States involvement was unprecedented in any British colony. The decolonisation of British Guiana was part of the cold war: the security files kept at Government House would have been recognised to have potential future operational value and would have been repatriated, as were much less sensitive and useful documents from other colonies at the same time. Independent Guyana remained an area of interest for Britain and the United States: the British government would not have destroyed resources for its foreign and security policy, in particular the kind of thick data on local people which the Government House files would have held. When the Kenyan historians requested documents in the past, they were told repeatedly by the FCO that they had been destroyed, only for the FCO, under judicial pressure, to yield them. It is to be hoped that the FCO will at some point "discover" its British Guiana archive.

US officials complained that Kenyan students were becoming "anti-white" in the year Barack Obama's father enrolled at university, the files reveal, according to a BBC report. US officials told a British diplomat that Kenyan students had a poor reputation, for falling into "bad hands" and "becoming both anti-American and anti-white". But a US official said the best thing to do would be to "hope to achieve is to exert some influence over them while they are here".

A reader emails to ask if there is anything in the files on Zimbabwe.

The files released today cover Aden, Anguilla, Bahamas, Basutoland, Bechuanaland, British Indian Ocean Territories, Brunei, Cyprus, Kenya, Malaya, Sarawak and the Seychelles.

Further files will be released between now and November 2013. This webpage gives the order in which they are expected to be released.

The files relating to Southern Rhodesia, now known as Zimbabwe, will be released in September 2013.

Here's the full list:



Aden - 18 April 2012

Anguilla - 18 April 2012

Bahamas (now being released in two tranches) - 18 April 2012 - September 2012

Basutoland - 18 April 2012 - July 2012

Bechuanaland - 18 April 2012

British Indian Ocean Territories - 18 April 2012

Brunei - 18 April 2012

Cameroons - July 2012

Ceylon - July 2012 - April 2013

Cyprus - 18 April 2012 - September 2013

Fiji - July 2012

Gambia - July 2012

Gilbert & Ellice Islands - July 2012

Gold Coast - July 2012

Jamaica - September 2012

Kenya - 18 April 2012 - September 2012

Malaya - 18 April 2012

Malta - September 2012 - September 2013

Mauritius - April 2013

New Hebrides - April 2013

Nigeria - April 2013

Northern Rhodesia - April 2013

Nyasaland - April 2013

Palestine - April 2013

Sarawak/Brunei - 18 April 2012

Seychelles - 18 April 2012

Sierra Leone - April 2013

Singapore - April 2013 - September 2013

Solomons (BSIP) - September 2013

Southern Rhodesia - September 2013

Swaziland - September 2013

Tanganyika - September 2013 - November 2013

Trinidad - November 2013

Turks & Caicos - November 2013

Uganda - November 2013

West Indies - November 2013

Western Pacific - November 2013

Zanzibar - November 2013

Reader Celia Kerr writes to ask if there is much material in the archive relating to Aden, "one of those 'small wars' [1963-67] that prefigured much that has happened since".

As my colleagues Ian Cobain, Owen Bowcott and Richard Norton-Taylor have written, most of the sensitive documents kept by colonial authorities in Aden (now part of Yemen), where the army's Intelligence Corps operated a secret torture centre for several years in the late 1960s, appear to have been destroyed.

Dr Laleh Khalili, senior lecturer in Politics of the Middle East at the School of Oriental and African Studies in London, writes to say she spent the morning at the National Archives going through the files. She reports:

The Aden material was ludicrous. Given the extent to torture and ill-treatment, detention and protest in that colony in its latter years, it was laughable to see the released files having to do with fisheries. The Cyprus files only go up until 1938, as if keeping the more incendiary material from the period of Cypriot emergency for a future date when the hubbub around the files has died down. I glanced at the Kenya material but trust that Professors Anderson and Elkins would be giving those a good once-over, so I then moved on Malaya. The files mostly dated from 1955-1957 (rather than the period of 1948-1951 when half a million civilians were resettled in concentration camps quaintly called New Villages) and what was there was either oblique, opaque, or useless. Most strikingly, the keywords "detention", "resettlement", "New Villages", "reservations" (another quaint word for concentration camps, this time used in Kenya), or "detainee", don't even appear in any of the aforementioned files. It seems to me that there are some interesting things having to do with the Chagos Islands, Anguilla, and some parts of Africa (not those with major counterinsurgency operations, however), but those are not my areas of interest. It seems to me that the whole release is a whitewash intended to mollify critics and save the government from embarrassment.

On Aden (see 3.33pm), Ian Cobain writes:

There is already a great deal of material concerning Aden available for public view at the National Archives at Kew, including a mass of documentation generated by the visit in late 1966 by Roderic Bowen QC, a judge and former MP who had been asked to investigate following complaints that insurgents were being routinely tortured during interrogation. Those papers include some very candid admissions by British colonial officials about the utility of such interrogations. But while the Hanslope Park papers that were opened to the public at Kew today include some relative trivial documents from Aden - such as personnel reports and papers about transport matters - those historians who have been overseeing the transfer are struck by how little weighty material has survived. Some assume that the more important documents were destroyed. Given the inability of the British government to influence the nature of the post-independence government in Aden, and the manner in which the British withdrew in 1967 - steadily retreating until they remained behind a perimeter that embraced little more than the colony's airport - it is quite possible that a decision was taken to destroy a great deal of documentation, rather than take the risk of it falling into insurgents' hands.

Owen Bowcott adds:

A small amount of relatively pedestrian, bureaucratic material appears to have been passed over to the the National Archives on Aden. It was fewer than 40 files, most dating from 1962-67. They concerned subjects such as oil exploration, issuing of British passports and fisheries. Most of the Aden material is assumed to have been destroyed.

Dr Francis Gooding, a researcher on the Colonial Film: Moving Images of the British Empire website, writes: