Oslo is not a city whose streets hum with urban tension and social decay. To the casual observer, the Norwegian capital is a study in frictionless living: clean, well-ordered, civic-minded, affluent yet essentially egalitarian in spirit. There are more paintings by Edvard Munch here than there are graffiti, and Saturday night in town can seem about as frenetic as a bank holiday in Sunningdale. The locals speak with metropolitan pride about the edginess of the "east side", where most of the city's non-European immigrants live, but from a British perspective even that neighbourhood seems like a model of residential tranquillity.

Yet these placid streets have produced countless psychopaths, serial killers, political assassins and degenerates of every conceivable stripe. Or at least they have in the work of Norway's many bestselling crime writers, such as Jo Nesbø, Anne Holt, Thomas Enger, KO Dahl, Gunnar Staalesen (who mostly focuses on Bergen) and Karin Fossum. Along with one of the world's lowest rates of real-life crime, Norway boasts one of the highest rates of fictional crime.

If this disjunction between an apparently settled state and a violently restless literary imagination is a well-established Scandinavian phenomenon, it is most pronounced in Norway, the most benign of the Nordic nations in practice and the most malevolent in prose. It's as if a generation of Norwegian crime writers took the advice of the nation's two giants of literature a little too literally. "Wake the people up and make them think big," said the dramatist Henrik Ibsen, and the Nobel laureate Knut Hamsun declared that writers should describe the "whisper of blood, and the pleading of bone marrow". The result has been as vivid and incongruous as spilt guts on virgin snow.

Norwegian crime writers have got used to defending themselves against the charge of pure fantasy. They usually respond by pointing out that, contrary to the global image, Norway does suffer from crime and social dysfunction, and there are dark forces abroad behind the facade of the social democratic idyll. But taking into account the vast oil and gas reserves that make Norway (tiny tax havens aside) Europe's wealthiest per capita nation, a princely welfare system, and murders running at a world-historic low of 0.6 per 100,000 people, no one took them very seriously. Not until 22 July this year. At around 3.30pm on that long summer's afternoon, a 32-year-old man named Anders Behring Breivik set off a car bomb outside government buildings in central Oslo which killed eight people and injured many more.

Visiting his widowed mother at her flat in the city, the novelist KO Dahl heard the explosion. Dahl is the author of a series of crime novels featuring two Oslo detectives, Gunnarstranda and Frølich. He approaches the genre from a socio-psychological angle, examining social conditions and character motivations in robustly gripping narratives.

Like many Norwegians, he turned on the television, followed the reports and speculated that the bomb was the work of Islamic extremists. Then a message flashed on screen that there had been a shooting on the tiny island of Utøya in the Tyrifjorden lake about 25 miles north-west of Oslo. The island is owned by the Arbeidernes Ungdomsfylking (AUF), or Workers' Youth League, which is the youth wing of the Norwegian Labour party. The AUF was holding a summer camp for around 500 members of the organisation, most of whom were teenagers.

Kjell Ola Dahl: ‘My niece was there. I was this comfortable writer and suddenly I was in hell. I did a lot of reflection on that’.

The Labour party has been the ruling party in Norway for most of the postwar years, and members of the AUF often move through the party machine to become the leaders of the future. Jens Stoltenberg, the current prime minister, for example, started out in the AUF. Among their number was Dahl's 15-year-old niece. She wasn't particularly political, but her friends were, and she went along to Utøya partly to be with them.

As soon as Dahl saw the newsflash, he started scanning the internet in silent panic. He knew his niece was on the island, and he did not want to alert his mother – his niece's grandmother – to the unfolding terror.

"This is a very special niece," Dahl says. "My sister died of cancer 10 years ago and it's her youngest daughter. So she's been some kind of mascot."

We're sitting in Dahl's beautifully appointed farmhouse on the banks of Lake Mjøsa, a dramatically serene setting 90 minutes' drive from Oslo. It's hard to imagine a more peaceful place. But that same observation was made about Utøya. On the internet, Dahl recalls, he came across a tweet that said the assailant on the island was dressed as a policeman. "And I said, OK, it's a Nazi, there's no doubt in my mind. It was a confirmation that the Nazis are still there and still active."

Ever since the Swedish couple Maj Sjöwall and Per Wahlöö began their influential series of crime novels in the 1950s, Scandinavian crime fiction – through Henning Mankell, Stieg Larsson and their Norwegian counterparts – has been written largely from a leftwing perspective, as a critique of the failings and hidden hypocrisies of social democracy. Within this viewpoint, neo-Nazism has performed the role of all-purpose bogeyman, as a kind of sinister spectre of capitalism haunting both the margins and the impenetrable heart of society.

Dahl is among the many Norwegian writers who have explored the neo-Nazi threat, in his case in a book entitled The Man in the Window. Even the hugely popular Jo Nesbø, whose recent biggest sellers have been apolitical tales of freakish serial killers, employed the neo-Nazi trope in one of his earlier novels, The Redbreast, in which a racist thug makes a speech berating Europe for abandoning national socialism and allowing mass immigration. "They let the enemy build mosques in our midst, let them rob our old folk and mingle blood with our women," says the white supremacist of the Norwegian political class.

Breivik's almost identical gripes were detailed at rambling length in the manifesto he published on the internet to accompany his killing spree, in which he described his hatred of Norway's ruling "cultural Marxists" and, in particular, foreign immigration. Notwithstanding his intuitions, Dahl knew none of this at the time, while the killer was loose on the island.

As Dahl grew increasingly anxious at his mother's flat, his mother saw a news report about Utøya and she immediately phoned her granddaughter. Amazingly, the girl answered. She was hiding close to the shore in the lake, terrified. She said that a killer was shooting people on sight. Her grandmother told her that he was not a real policeman and handed the phone to Dahl to give his niece advice.

"All I heard," recalls Dahl, "was 'He's coming'. Then the phone went dead."

Jo Nesbø: ‘I will not address the massacre itself. But it has so influenced our way of thinking and our society’. Photograph: Steve Black/Rex Features

Two and half hours later, after an excruciating silence, Dahl discovered that his niece had survived. She had dropped her phone in the water and pretended to be dead, a tactic that required steely nerves because Breivik shot several apparent corpses just to make sure. A girl who was hiding next to Dahl's niece was unable to maintain the act. Overcome by fear as Breivik approached, she began screaming uncontrollably. Without hesitating, the gunman shot her dead. After 90 minutes of remorseless slaughter, 69 people, some as young as 14, had been killed and a further 66 were injured.

In the wake of the killings, Nesbø wrote: "There is no road back to the way it was before." He was speaking of the disappearance of his country's extraordinary sense of security and innocence, in which the prime minister could walk in the street and chat with the public on first-name terms. Yet perhaps there was also a more specific meaning that referred to crime fiction. Where could it go after such a spectacular crime? For decades it had traded on the idea of a lurking menace, some faceless demon eating away at the Scandinavian dream. What to say now that it had revealed itself?

"I don't know how my writing will change," Nesbø later said. "It will change… I will not address the massacre itself. But it's so influenced our way of thinking and our society. So it will be there in my novels somewhere, I'm sure." Dahl, too, senses that his writing will be affected by the events of 22 July but also doesn't know how. I wonder if the intimate terror he suffered as his niece's phone was cut off will make itself known in the experiences of his characters. While he accepts the possibility, he says that what he's noticed has been the impact on how he interacts with people in real life. Previously he had been inclined to detach himself from the plight of victims of tragedy, not out of callousness but a sort of incuriosity.

"Then this happens and my niece was there," he explains. "Suddenly I was one of those people, though my niece survived. I did a lot of reflection on that. I was this comfortable writer and suddenly for two hours I was in hell. Every time I meet some emotional stress in me or my family or other people, I'm humble. It's important to try to respect what's going on."

What's going on in Norway is an extended period of national self-reflection. It's worth comparing this response with the reaction in Britain to the terror spree of David Copeland, the neo-Nazi "London Nail Bomber" whose attack on the capital's gay and black communities left three dead and 129 injured in April 1999.

There wasn't much soul-searching when Copeland was arrested and tried. Few thought that he was the product of the system's failings. By contrast, in Norway there is a great need to "understand" what caused Breivik's actions. This is partly due to the overdeterministic Scandinavian analysis of errant behaviour, which invariably ascribes criminality to society's faults. But it's also because there are just 5 million Norwegians and each of them felt directly affected by the carnage at Utøya.

This is a point that the literary novelist Jan Kjaerstad makes when I meet him in Oslo. Kjaerstad is the author of a celebrated postmodern trilogy that focuses on the sprawling life of a television presenter who is charged with murdering his wife. By turns picaresque and ironic, it's a revealing portrait of Norwegian preoccupations and insecurities.

"Norway is a small country," he says. "We have this history of no war from 1814 to now, with the exception of the Nazi occupation. Everyone knows someone who had some connection to these killings."

Just how small Norway is became apparent to me when Kjaerstad and I went to lunch at Bølgen & Moi, a cool brasserie in a renovated power station. As we sat down, he told me that the adjacent table was referred to as the "Crown Prince table".

"Why's that?" I asked unthinkingly.

"Because the Crown Prince sometimes eats there," came the obvious reply.

Jan Kjaerstad: ‘If someone goes into a bubble, you can’t reach them when they pass a certain level of rationality’. Photograph: Sutton-Hibbert/Rex Features

A while later a tall, broad-shouldered man with a beard walked in and sat down at the table, but not before exchanging pleasantries with Kjaerstad.

"See," said the author, turning to me and introducing the man, "I told you. This is the Crown Prince."

We shook hands and chatted for a while and then returned to our separate tables, as informally as any three blokes in a bar. Kjaerstad, who's a charmingly understated character, is also on first-name terms with the prime minister and the psychiatrist who is analysing Breivik to decide whether or not he is sane. While not everyone in Norway is as connected as Kjaerstad, it can seem like everyone knows someone who is.

Kjaerstad thinks it will take a long period of creative gestation before a good novel is produced dealing with 22 July. In the meantime, he's doubtful that the search for societal answers will prove all that fruitful. "People are going too fast and coming to very easy conclusions. It is of course an illusion that you can fix something like this. If someone goes into a bubble, you can't reach them. When they pass a certain level of rationality, they are beyond reach."

While acknowledging the murderous extremity of Breivik's bubble, Kjaerstad says he has witnessed similarly ideologically blinkered thinking before. When he was a young man in the 1970s, many of his contemporaries were drawn to a particularly uncompromising vision of Marxism-Leninism. "Only the Albanians were further to the left," he says ruefully. "We've had so few dramas that for these urban revolutionaries it was a kind of role play," says Kjaerstad. "If you see their writings, you can't believe it. They were close to making lists of people who they were going to kill when the Soviets took over. They were training in the woods. Everyone had noms de guerre. Maybe they were living in a bubble too."

Kjaerstad says that a number of these would-be revolutionaries are now high up in the cultural sector, as newspaper editors and TV presenters, which sounds uncomfortably reminiscent of Breivik's claims about "cultural Marxists". Kjaerstad dismisses Breivik's paranoid ravings – the media figures he's speaking about long ago abandoned their youthful posturings. But if there is a legacy from those militant times it may be the faint whiff of socialist realism in the prescriptive tendencies displayed by some Norwegian writers.

"When I started writing I had to fight a lot with political writers who were writing in red ink and everything was on the surface," says Kjaerstad. "The moral is like this and you must live like this!"

That lecturing tone can still be discerned, beneath all the blood and procedure, in some crime novels. For instance, in Anne Holt's Fear Not, published in 2009 in Norway, far right religious maniacs are on the loose in Oslo killing gays and immigrants. Like several other Norwegian novels, the book can now be read as a prescient warning about the hate-filled Breiviks skulking in cyberspace. But it's also a work whose political agenda is as insistent as a street corner pamphleteer.

I told Kjaerstad that I was going to meet Holt. "She's a very nice person," he said, and added with a smile: "She's on the border of being moralistic."

In a small cafe in the northern suburbs of Oslo I join Holt for a late afternoon coffee. A former journalist and lawyer, she was justice minister in the Norwegian government for a few months in 1996-7. By then she was already writing crime novels. No stranger to major life rethinks, she also left her husband to start a new life with her lesbian lover, to whom she is now legally married. Both the professional and romantic changes seem to have gone well. She and her wife have a daughter and live a life of some luxury in the hills outside Oslo. And as the legend on the jacket of the British edition of Fear Not testifies: "Over 5 million books sold worldwide". If that doesn't clarify the success, there's also a sticker that trills: "Step aside Stieg Larsson, Holt is the queen of Scandinavian crime."

An energetic and plain-speaking woman who gives the impression of being untroubled by self-doubt, Holt has little time for the idea that Breivik was just a deranged loner who came out of nowhere. "As we have a growing group of people living their lives on the internet who are very strongly opposed to immigrants in Norway, I think it's wrong to say it was totally unexpected. He was not even the worst person on these sites. Doing research for Fear Not, I entered all these hatred sites so I was very aware that they hate Muslims, they hate gay people and women, and what I was trying to do in the book was explore the connection between written and spoken hatred and actual physical violence."

She traces a decline in the tone of public debate to the rise in popularity of the rightwing Progress party, which at the beginning of the year was nearing 30% support in polls. Although Utøya was an avowed attack on the Labour party, it is the Progress party that has been most politically damaged – at least in the short term. Breivik was a member of the Progress party for a number of years before leaving around 2006, and its anti-immigration stance, on which much of its popularity was based, became an embarrassment following the massacre. In the local elections that followed, the party's share of the vote dropped to just 11.4%.

But if Holt is right and the warning signs of Utøya were there in internet chatrooms, was it and is it the duty of crime fiction to address them?

"Crime fiction is probably the No 1 genre when it comes to reflecting society," says Holt. "Before this happened I would have sworn that it was physically impossible, first to let off a bomb and then to go out there and shoot all these kids. Now we know that someone from Norway can do it. I didn't dare have someone from Norway [committing the killings] in Fear Not. I had a discussion with my editor and I said: 'What if I made this [homicidal bigoted religious sect] a Norwegian organisation?' And she said: 'No, nobody will believe that. You have to make it American because they do have organisations like that.' Now in retrospect I can say I regret the fact that I didn't."

In the event, Utøya and its aftermath have stirred two antithetical emotions in Holt. In the first instance, because the attack was aimed at the Labour party's support for multiculturalism, it exposed the racism that she believes is endemic in Norwegian society – something, she says, that makes her so ashamed that she wants to leave the country. But she also revelled in the public response to the killings, in which native and non-native Norwegians stood together in grief and solidarity.

"What was a success was the way the nation dealt with it afterwards. For the first time since I was kid I felt proud of being a Norwegian, because we all know how they reacted in the States," she says, drawing a rather stretched comparison between 22/7 and 9/11.

Unlike Kjaerstad, who was sceptical about the notion of explaining Breivik, and Dahl, who told me that he thought Norwegians had tired of the media fixation on Breivik, Anne Holt is hopeful that the whole investigative process and the blanket media coverage will be socially beneficial.

"I think this debate will go on and on," she says enthusiastically. "I heard yesterday that these two psychiatrists who are examining him are going to scan his brain. I think some of the most interesting effects of this tragic incident is that there are now discussions of these aspects of society. It's not only worthwhile but extremely necessary. We have to try to understand it, try to explore every aspect about this guy. I think along the way we'll learn a lot about ourselves and Norwegian society, its failures and successes."

Holt believes that this greater understanding of the Breivik drama, from its inception to the judicial conclusions, will eventually feed into and expand the parameters of Norwegian crime fiction.

"We don't write whodunnit books, but why did it happen. In due time I'm sure that we will benefit from the fact that we have to look into what created such a monster. If a person has had all the opportunities he has had – he's not even from the east side of the city, he's from the west, and he was given all the opportunities in the world and at the same time his discontent made him able to do this terrible thing – then we have a whole new field to explore in terms of what kind of criminals this society can create. Now the sky is the limit. We have seen what this society can produce."

Holt is not alone is speaking as though Breivik has vindicated every reservation she had ever held about Norwegian society. This attitude of self-suspicion runs deep in Norwegian cultural life and it captures a contradiction that is evident in crime fiction, and which may provide the creative tension that accounts for its success. Read as a distinct genre, the novels seek to celebrate something uniquely Norwegian, the nation's traditions and separateness, while often locating the cause of the crime in isolationist, nationalist or inward-looking thinking. Norway is not immune to the growing pains of globalisation, with its ethnic tensions and multicultural conflicts, but if they were as pronounced in actuality as they are in the fiction, then Breivik would loom far more real and tangible. Instead, despite the horror that he wreaked, the paradox remains that he seems like a blank-faced killer out of a Norwegian novel.

As she gets up to leave, Holt's parting words are: "We will be waiting many years before we get the ultimate 22/7 novel. And," she adds, perhaps to head off any hint of immodesty, "it's not going to be a crime fiction novel. It will be something else."

With that she disappears back into the quiet, safe streets of Oslo.