It is in the town of Gyongyospata, in north-eastern Hungary, that Europe's gypsy problem appears in all its spiny intractability.

An hour-and-a-half north-east of Budapest, this is a country town of some charm, with an imposing white church and the sort of plain, red-roofed stucco houses you find all over the country, dotted across the tidy, well-trimmed, verdant landscape.

Then at the edge of the town, on either side of a stream that often floods, is the gypsy ghetto.

I am inside a small, decrepit home where the ceiling is collapsing into the kitchen. There are two bedrooms in the house, each filled to bursting with two double-beds: 14 family members live here, I am told. We sit on one of the beds while a four-year-old girl does her party piece, a hip-shaking disco dance.

Then, in the tiny kitchen, where an outsize pan of rice is simmering on the stove for lunch, I inspect the hole in the roof while the mistress of the house tells me about her family. She has 24 grandchildren, she says with a proud smile; most seem to be with us here in the kitchen. How about one of them fixing that hole, I suggest. The proposal makes no impact: mending holes is the government's business. The existence of such a gaping hole is an evident grievance.

No one in the family has a job or the hope of one. The birth rate in the community is double that of non-gypsy Hungary. Few of the children make any headway at school. And, in ways that have been repeated numerous times down the centuries across Europe, the non-gypsies are beginning to lose their patience.

Right across the continent, from Bulgaria to Britain, Hungary to France, Europe is wrestling with a new outbreak of an ancient dilemma: the gypsies, the gajé (that's the majority of us, the non-gypsies) and how the two communities are to get along.

Since the arrival of the gypsies from India via the Middle East in the 15th century, relations have rarely been smooth. Hitler's attempt to exterminate the community resonated with the deportations they suffered centuries previously in Spain and Henry VIII's England. President Sarkozy's grandstanding expulsion of thousands of Romanies over the past two years was only the latest version of an ancient, atavistic urge. Such populist barbarism is rightly condemned. But even in countries where the gypsies' right to remain is not under threat, tensions continue to simmer. In Britain the number has soared in recent years, because of waves of immigration from Romania and Bulgaria (as distinct from the well-established Irish "traveller" communities); their skilful exploitation of our benefits system has brought down the wrath of the tabloids. But in Bulgaria and Romania, too, countries where they have lived in large numbers for centuries, the discrimination they suffer is still acute.

Even in Spain, the one European country which, since the death of Franco, has consistently tried to integrate them, and which has successes to boast of, the dropout rate from high school is 80 per cent. And in many countries the image persists of a community of lawless, feckless petty criminals, condemned to eke out a miserable living on the fringes of our civilisation.

Hungary is one of the countries where one might expect things to go better. After all, they have been in this small central European country for so long – about 500 years – that the words "Hungarian" and "gypsy" fit together snugly. As in Spain, the artistic imagination of the Romanies, especially in music, has become entwined with the nation's cultural identity: Franz Liszt argued that their music was at the heart of what the world considered Hungarian music.

Today that long relationship has yielded striking gains. Romanies are more prominent at a higher level here than anywhere else: there are four Romany MPs in the Hungarian parliament, and the only Romany MEP in Strasbourg is a Hungarian woman. Most of the government officials I met on my visit to Hungary were Romanies.

Yet it is in Hungary of all places – where there are no aggravating issues of immigration or language – that the gypsies, and the way society deals with them, pose the gravest potential threats to the nation's future.

One day in March, hundreds of uniformed vigilantes suddenly appeared in Gyongyospata. They stayed for three weeks. Wearing black paramilitary uniforms, they streamed into the gypsy ghetto and began ostentatiously to patrol in place of the police who, at that point, rarely set foot here.

They belonged to an organisation called Szebb Jovoert Polgaror Egyesulet, which means Civil Guard Association for a Better Future. They patrolled the town "day and night", said the European Roma Rights Centre (ERRC). "They reportedly prevent the Romani residents from sleeping by shouting during the night, [they] threaten them with weapons and dogs and follow them every time they leave their houses, unimpeded by local police. The desperate Roma residents are afraid to go to school, to work or even to buy food." The standoff ended only when the Hungarian Red Cross came and evacuated the gypsies with a convoy of buses.

The vigilantes were supported by a far-right political party called Jobbik (pronounced "Yobbik") which, its critics maintain, is anti-Semitic and anti-Romany. Jobbik stunned Hungary's political establishment last year when it won 17 per cent of the vote in the general election.

The rise of the far right, the familiar bogey of Eastern Europe since the fall of Communism, has got Hungary's liberals worried. The nation's gypsies, who may number about 800,000 – around 8 per cent of the nation's population – are their frequent victims.

There have been murderous attacks: Sinan Gokcen, media and communications officer of the ERRC, tells me that nine gypsies have been killed between 2008 and 2010. A favoured technique of the assailants is to target a house on the edge of a gypsy village, throw in a Molotov cocktail, wait for the inhabitants to flee the flames then shoot them as they emerge.

These accounts of vigilante patrols and shootings are shocking. But what is missing is any explanation, beyond blind prejudice, of why they should be happening now. In Gyongyospata I was told that housing is the problem: the gypsies' miserable homes – though some are better and more solid than others, and one family is cutting the lawn and painting the rooms as we visit – are all clustered together on the edge of town; when the Red Cross proposed rehousing some families closer to the town centre, that was enough to inflame local sentiment.

But it's not as if the Romanies are new here: they have lived alongside non-Romany Hungarians for hundreds of years. The gypsy population is well rooted. One Hungarian newspaper's account of Gyongyospata's troubles reported the gypsies singing the Hungarian national anthem in the vigilantes' faces. They had every right to do so.

It's Kalman Kali-Horvath, the official from the Ministry of Justice who accompanies us to Gyongyospata, who helps me to understand the situation better. Mild and somewhat diffident in manner, he is himself a Romany, a "Romungro", belonging to the Hungarian-speaking Romany community who account for around 70 per cent of the gypsies in the country, including most of the artists and intellectuals: the gypsies who have been in Hungary so long that they have become part of the furniture, in contrast to the Vlach gypsies (20 per cent of the total), who fled slavery in Romania in the 19th century and speak a Romany language called Lovari; and the Beas (10 per cent), who speak an ancient dialect of Romanian and arrived 200 years ago.

Like much else, Kali-Horvath explains, the problems in Gyongyospata owe their origins to Communism, and what happened after it collapsed. "In the Communist time," he says, "everyone had to work. If you didn't, you went to jail."

The Communist regime refused to regard the gypsies as a minority, but as a social problem: "Despite certain ethnographical characterisics," decreed the Hungarian Socialist Workers' Party in 1961, the Romanies "do not constitute a nationality". Those who pander to that view, they said, only "preserve the segregation of the gypsies and slow down their integration into society". The gypsies were merely undisciplined proletarians who needed to be forced into the same mould as the rest.

They were accordingly given poor-quality worker housing with very cheap mortgages, and obliged to toil, like everybody else. "Most Romanies worked in the cities," says Kali-Horvath, "in factories or on building sites. In the country they worked on small farms or in villages picking fruit or digging the fields. The regime didn't want them to get more education because they needed cheap unskilled labour." The gypsies vanished into the socialist system.

But when that system abruptly collapsed in 1989, this problem, like many others to which Communism had promised definitive answers, turned out merely to have been frozen – and upon thawing it was in many ways worse than before. Now the gypsies were both an ethnic minority and a social problem. As the Communists' uneconomic factories and plants closed down helter-skelter, it was the unskilled workers at the bottom – the Romanies in particular – who were left high and dry. Romany unemployment shot up from 15 per cent to 85 per cent in two years.

In the absence of work there was now welfare. "Milking the system became a survival strategy," says Kali-Horvath. "You got a little money from different entitlements: unemployment, maternity, child allowance and many others; there were many opportunities to increase benefits. You didn't get rich, but it was a nice lifestyle, not just for gypsies but for the others in deep poverty – the gypsies are only one-third of the families in deep poverty.

"After 20 years, everybody knows about this. Governments chose to do nothing about it because if they did they knew the human-rights lobby would attack them. Families got more and more payments, which cost the state more and more, and that's the point we are at now."

Then there was the question of crime. The way organisations such as ERRC write about Romanies and crime, the gypsies are always and only the victims; guilty of nothing except being what they are. As defenders of the community, that of course is their remit. But the everyday experience of ordinary Hungarians is different.

"There are huge hatreds in both communities," says Akos Balogh, a journalist in Budapest. "People have everyday experiences of this conflict. Many children have conflict with their Romany schoolmates. When I was younger I was mugged by a Romany on my way to school. So was my brother, and he was so terrorised he was sent to school in another town. I had Romany friends at school, but there were tough cases, too. Every year we had different Romany classmates because they often fell behind and had to repeat the year. One guy three years older than me was brought to school by his father in a red sports car; he threatened to rape one of the girls in his class, and he had scary elder brothers. Another had a couple of fingers missing – his brother had chopped them off with an axe."

While Balogh has had positive dealings with Romanies – "A Romany sold me my flat, he was very correct about it" – he understands why the hostility has accumulated over the past two decades. "In many small communities, people think, 'I can't protect myself, the police won't protect me...' If someone steals €10, the victim knows nothing will be done about it." One of the crimes which most infuriates the majority community is the theft of fruit and vegetables from their gardens. Balogh, who visited Gyongyospata recently, says the trees used to be full of fruit but nobody bothers to take care of them any more because whenever the fruit approached ripeness, the trees would be pillaged.

In the eastern Hungarian town of Tiszavasvari, the BBC correspondent Nick Thorpe visited the home of a 76-year-old widow, Margit Papp, who told him that almost every crop she planted in her garden was stolen by Romanies. Once, she said, they even stole the metal poles that held up the vines.

The police, according to Kali-Horvath, adopted an attitude of disinterest towards Romany crime. "It was called 'crime of survival', and somehow it became acceptable. But that was only true at the political level; ordinary people became more and more angry about it, and about the police's failure to take action. They said, if the Romanies can do that and get away with it, why is it a problem when non-Romanies do it?"

According to Thorpe, who has lived in Hungary for many years, the situation has deteriorated dramatically since the economic crisis of 2008. "In the old days, a Romany family of five would live off welfare, which arrives on the fifth of the month. By the 14th the money would run out, so they would run up credit at the local shop, and the men of the family would get a few days' casual work, building walls, fixing roofs, in the neighbourhood. But with the financial crisis, people stopped spending money and the work dried up. The Romanies have survived for the past 20 years but the economic crisis has driven them to the wall."

Kali-Horvath turns out to have an amazing life story. His family circumstances made Gyongyospata's shabby cottages look enticing by comparison: both his Romany parents were alcoholics, and when he was two they turned him over to an orphanage. "I went back home twice later," he says. "The orphanage was bad enough, but going home was much worse." He had several brothers and sisters, also brought up in orphanages, with whom he is only loosely in touch.

The orphanage gave him a tough upbringing. "There were hundreds of children in the orphanage, half of them Romany. The teachers were not well educated, there was a lot of violence, nobody cared what happened, they could do what they liked. But at the age of nine I decided I wanted to be educated." Some teachers tried to talk him out of it – there's no point they said, you won't go anywhere, why make things hard for yourself? But he was determined.

What changed his attitude was a half-Cuban boy in the orphanage, twice his age, the star of the institution and the pet of the staff, who won a national competition on state TV for reciting poetry. Kali-Horvath was so inspired by his example that, with the encouragement of one of his teachers, he turned himself into that half-Cuban boy's successor: he too made it on to TV, and won the same prize. He subsequently became a TV news reader and a successful painter, then an official in the Ministry of Justice.

His success is remarkable given all his handicaps, but then, none of the middle-class Romanies I met came from privileged backgrounds. All had battled their way to success. The willingness of Hungary to let them shine is one of the bright spots in this story.

While In most of Western Europe the Romany issue is marginal, in Hungary, thanks to the size of the community, the disastrous aftermath of Communism and the failed policies of the past 20 years, it has become central.

That's why Zoltan Balog, a Protestant pastor for 10 years before becoming the prime minister Viktor Orban's closest adviser, was at his own request given a new department of state to try to solve it.

A gaunt, grey-bearded 53-year-old who is widely seen as the conscience of Hungary's conservative government, Balog was behind that government's decision, during its six-month EU presidency (which finished in June) to make Romany integration Europe-wide a top priority. But at home the gypsy question is even more urgent: he gives his government three years to tackle the problem effectively or face disaster.

For Balog, the eruption of uniformed vigilantes in Gyongyospata's Romany ghetto was a symptom of Hungary's national crisis; but not because it portended the rise of the neo-Nazis. The problem of which it is a symptom, he says, is more fundamental than that.

"The real difference between our Romany problem and that of western Europe is the degree of risk," he explains. "In Italy and Spain we are talking about the integration of a marginal group, so it is a question of human rights. But in Hungary it is a question of national strategy that concerns the whole country.

"Romanies have twice the birth rate of other Hungarians. The majority Hungarian population is ageing while around half of the Romany population is under 20. In towns and cities in the deprived north-east, in 10 years' time every second child born will be Romany. But Romany unemployment is 85 per cent, and one-third of Romany children do not finish primary school. So this is not one problem among others but the major problem."

Like Hungary's other minorities, Romanian, German and 10 other groups, the Romanies have a certain say in running their own affairs, through what is called "National Roma Self-Government". Balog's answer to the national crisis is for the national and Romany authorities to create 100,000 new jobs together, starting with public-works employment in the communities in which they live; and at the same time to boost Romany educational standards massively, giving 20,000 vocational training and preparing 5,000 of the brightest for university. The scheme should begin showing results in three years. In the longer term the hope is to turn the Romanies from being a fatal drag on the economy into an asset: if their rate of employment rose to the regional average, that could trigger a growth of between 4 and 6 percentage points for Hungary's gross domestic product.

"It will be controversial," admits Balog. He is already under assault from the Romanies' defenders for his government's allegedly authoritarian approach, yet once the programme gets under way he expects also to be attacked by non-gypsies, including representatives of the very poor ones, for "once again" favouring this community.

"But we have to send a clear political message to the majority of Hungarians," he says, "to make them understand why this issue is so important. Our approach is not from the standpoint of the minority but from that of the majority. If we do not succeed in making these changes, our social structure, economy and labour market all face collapse. The Romany issue is a problem of survival."

Thirty kilometres from Gyongyospata there is another village called Tarnabod. Abandoned after Communism, derelict and full of desperately poor Romanies (and non-Romanies) with nowhere to go, it was taken in hand by young social workers from the Hungarian Maltese Charity Service. Today, in a converted cow shed in the village, a dozen formerly redundant locals work at stripping down old computers, work stations and other bits of obsolete technology for recycling. They earn the national minimum wage. Some are Romanies, some not: there is no easy way to tell them apart. Elsewhere in the village, the outsiders have set up a kindergarten, a canteen for children and parents, a culture centre with a screening room, a post-school tuition centre; everywhere the floors are scrubbed, the walls gleaming and the shelves lined with books for borrowing. A portrait of the first and so far only Romany saint, Boldog Ceferino Gimenez Malla, has pride of place in the library. At an outdoor lunch in the middle of the village, Kali-Horvath is amazed to spot Jozsef Choli Daroczi, Hungary's most famous living Romany writer.

After the poverty and hopelessness of Gyongyospata, it's a charming scene, though heavily reliant on the input and hard work of the outsiders. As one of the project's directors, Szilard Lantos, admits, "Our work will be successful when we are able to move on."