At around 10am one morning earlier this week, a dazed and haggard man in surprisingly clean blue convict's fatigues walked out of Insein jail on the outksirts of the Burmese city of Rangoon. Tang Naing Oo had been in prison – held for the most part in a cell measuring 30ft x 50ft that he shared with 110 other inmates – for 14 months. He had originally been sentenced to three years in jail, back in September 2010, for distributing pictures of Aung San Suu Kyi, the famous Burmese pro-democracy campaigner and Nobel prize laureate, on a Rangoon pavement. Now, he was walking past the noodle sellers, the watermelon hawker, a crowd of waiting passengers at the ramshackle bus stop, to a form of freedom.

Tang Naing Oo had learned he was to be released only a few hours earlier. When he woke in the fetid cell at 5am, he saw "hope" on the faces of his fellow inmates, he says. His release came the day before the national celebrations commemorating the independence of Burma from Britain 64 years ago, and some kind of amnesty had long been expected from the government. However, the hopes of most inmates in Insein, and the vast network of other prisons and interrogation camps around the country, were disappointed. Of the between 600 and 2,000 political prisoners estimated to be in detention, only a couple of dozen were released. None were senior figures.

"If the government are serious they will release all the other detainees," Tang Naing Oo says, slumped against the grubby wall of a nearby shop-cum-home-cum-cafe. "This is just for getting more interest from the international community. It is not real change."

The international community arrived in Burma today (thurs)today in the shape of William Hague, the British foreign secretary. He is the first UK official of such seniority to come to the country since the army took over in 1962. In December Hillary Clinton, the first American secretary of state to visit for a similar period, flew in and Hague is following her exact itinerary. He arrived in Naypyidaw, the new capital hacked at huge expense out of swamps and scrub in the centre of the country, where he met Thein Sein, the retired general who was named president and head of the new, nominally civilian government last year by the dictator Than Shwe, following the latter's supposed retirement from public life. Hague then flew to Rangoon, the bustling city on the Irrawaddy delta, where he met representatives of civil society and ethnic minorities before having a private dinner with Aung San Suu Kyi herself. Tomorrow, as per Clinton's itinerary, there will be further meetings, some photo calls, a walk around a pagoda, and then it will be home in time for the weekend.

Prisoners are released from Insein prison in May 2011. Photograph: Soe Zeya Tun

The visit, British officials in Rangoon say, has been prompted by the reforms recently made by Burmese rulers and the desire of Her Majesty's government to encourage further progress on the path to democracy, stability and prosperity. Even 18 months ago such an ambition would have been laughable. Successive military regimes have won deserved reputations for brutality, corruption and human rights abuses. Burma, renamed Myanmar by one of those regimes, has been repeatedly shaken by uprisings, most recently in 2007. One of the world's longest running civil wars has pitted ethnic groups against the national army, creating a vast refugee crisis and reports of forced labour as well as systematic rape and torture. The nation is, despite considerable resources and a prize strategic position on the Indian ocean seaboard, currently one of the poorest in the world. A clique clustered around the top generals and their relatives live in great luxury, while only one in 10 villages has electricity. The government response to the catastrophic Cyclone Nargis in 2008 was a toxic mixture of cynical disregard for human suffering, secrecy and incompetence. Aung Sang Suu Kyi, who assumed the leadership of a popular pro-democracy revolt in 1988, has spent most of the subsequent 23 years in prison or under house arrest.

Things started to change in March 2010 with the appointment of the civilian government. Few analysts can say exactly why the notorious Than Shwe decided on this move. Some argue that the motivation was purely economic, as only improved relations with the west will allow Burma to join the ranks of the Asian tiger economies. Others point to a resentment at China's growing role in the country. Nay Zin Latt, the political adviser to the president, says that the decision was simply the result of a realisation that "for capitalism and the free market to flourish, democracy was necessary".

"We need western investment, technical knowledge, the art of management. If the country doesn't grow economically then there will be big problems, big unrest. The people with Mercedes cars won't be able to drive them around the streets!" Latt explained.

Aung San Suu Kyi, or "The Lady" as she is known locally, was released from house arrest in November 2010 and elections that she and her party, the National League for Democracy, boycotted, were held the same month. These were deeply flawed but many were surprised that they were held at all. The release of Aung San Suu Kyi, who won the Nobel peace prize in 1991 after the regime cancelled elections that her party had won, was seen as extraordinary.

Since then there have been other reforms. Many, such as new labour laws or legislation allowing protests, have had little practical effect on the ground. Others have had more impact. A handful of foreign journalists have been allowed in, surveillance of democratic activists is marginally lighter and work on a very unpopular Chinese-funded dam project, which would have generated huge amounts of cash for the regime while displacing tens of thousands of locals, has been suspended.

Local journalists have tracked the reforms through the degree of censorship to which they are subjected. "Before, printing any image of Aung San Suu Kyi was unthinkable. Then we could use pictures of her on the inside pages no bigger than 5 x 7in. Then suddenly we could put them on the front page," said Thi Ha Saw, editor of the Myanma Dana magazine.

It is not just the press. As all visiting reporters have remarked, there are posters of the Lady now on sale on street corners and her picture on mobile phones, walls and cars. The latest development is that Aung San Suu Kyi herself will lead her party in contesting byelections in the late spring. This is a risky and controversial decision that risks fracturing the fragile unity of the democratic campaigners in Burma. It will almost certainly result in the NLD entering parliament in some numbers – even if they will still be heavily outnumbered by soldiers in the assembly.

The government is considerably more enthusiastic about the prospect of the Lady in parliament than many of her supporters. "We need an opposition here. We need a strong NLD. The reforms will continue. Sometimes they will go slowly. Sometimes quickly. But they will continue. This is democratisation and that is the mission of the government," insists Latt, the presidential adviser.

This then is the process Hague has come to reinforce. If his visit is largely welcomed by pro-democracy campaigners – and Aung San Suu Kyi was sounded out first, as she was before Clinton's trip – others are more wary. There are many in the country who are concerned that the very adulation lavished on Aung San Suu Kyi, particularly in the west, could be something of a trap. These people know to what extent the Lady incarnates Burma's struggle for democracy for the international community, especially with a biopic about her on release, and they see her cult status as a potential danger. The worry is that the sight of their leader taking a seat in the Burmese parliament will be taken to mean that the problems in the country have been solved.

Aung San Suu Kyi with Hillary Clinton, December 2011. Photograph: Paula Bronstein/Getty Images

Normal relations will be established – not least because Burma is an important element in the ongoing effort to roll back Chinese influence in the region. Trade will follow the flag. The current American and European Union sanctions on Burma will be lifted and businesses will begin to move in to exploit an untapped market and a country with fantastic natural resources. As long as Aung San Suu Kyi remains in parliament, the argument goes, the authorities will have the figleaf they need. Shyan Saran, a former Indian foreign secretary and ambassador to Burma, puts it bluntly: "Aung San Suu Kyi is the regime's passport to legimitacy in the international community."

Yuza Maw Htoon, a Rangoon-based activist, politician and head of an NGO who stood as an independent in the 2010 elections, is more delicate. "The international community gives recognition only to the Lady and that makes the government happy," she says. "There is a need for other interlocutors too, both for the authorities here and for our friends overseas."

Even within the senior ranks of the NLD, there are those who fear that the focus on their leader could backfire. U Win Tin, 82, is one of the founders of the party. He remembers the dark days of the Japanese occupation of Burma during the second world war, as well as the freedom struggle against the British. He is deeply sceptical of the government's desire for "democratisation" and concerned that the west might let itself be deceived.

Speaking in the crowded, untidy offices of the party, he said he could "not accept this so-called change" on the part of the government. "I cannot trust it. There are still two motorbikes from military intelligence outside my home, like there have been for decades. There are still many friends in prison," he says. "If the west put the whole focus on [Aung San Suu Kyi] that could be very misleading. We trust in her and her intuition but this is all happening very quickly."

Another common fear, voiced by U Win Tin, is that strategic considerations will blind the international community to the problems that continue in Burma. "I have lived through periods when there was a real struggle in the region between the west and its enemies. There was the cold war, when Laos, Vietnam and Cambodia were under communist rule and anyone who was the communist's enemy was the west's friend. Now I am concerned that Burma will become a pawn in the effort against the Chinese," he says.

Aung San Suu Kyi appeared this week at the NLD headquarters for the celebration of the country's independence. Looking tired and drawn, she called for further efforts on the road to freedom. She smiled very little, though stood patiently to be photographed with group after group of party workers and spoke briefly with the half dozen low-level foreign diplomats who were present. Songs were sung about the political prisoners still in jail but the Lady did not mention them directly in her short speech. This disappointed some. Others spoke of "necessary compromises".

U Win Tin had a message for the British foreign secretary. "Hague should keep in mind that, yes, we have found a light in the tunnel here in Burma," he says. "But we are still in the tunnel. Maybe we can reach the light, maybe we can make it brighter, maybe we can even leave the tunnel. But we don't know yet. And meanwhile, we are still in the dark."