The Man Booker Prize is always guaranteed to generate controversy. This is nothing new and a healthy state for any literary prize to be in. But this year’s shortlist has caused a different kind of commotion with its inclusion of two crime novels, Graeme Macrae Burnet’s His Bloody Project and Ottessa Moshfegh’s Eileen.

Why does the presence of crime fiction cause so much consternation among the cognoscenti? After all, last year’s winner, Marlon James’s fiercely brilliant A Brief History of Seven Killings, was a stone-cold crime novel, a fact not acknowledged by either the judges or the majority of reviewers. This year we’ve seen several inventive attempts at describing Burnet’s book as anything but crime fiction (even the author, rather tautologically, stated it was “a novel about crime rather than a crime novel” – which seems to me like a rather accurate description of a crime novel), while Ottessa Moshfegh, when asked why she chose to structure Eileen as a crime novel, told the Guardian: “Because there are all these morons making millions of dollars, so why not me?”

Why does crime fiction produce such fury and prejudice? And why, despite this, are crime novels finally starting to get the recognition they deserve? I don’t know the answer to the first question – but I suspect a large part of it is a hangover from the Modernist disdain for plot. Such elitist arguments fail on the basis of logic – in the right hands, plot is character, is theme, is idea.

The answer to the second question is much simpler. Readers are finally discovering just how good a crime novel can be – how it can provide all the satisfactions of the literary novel while at the same time gripping you so much you miss your stop on the Tube.

There’s no reason why a book can’t be both a great crime novel and a great literary novel. No one had a problem with this until the early 20th Century. Crime is and always has been at the centre of human storytelling. The Bible has the third person ever alive murdering his own brother. In fact, quite a few of our most-beloved classics are books which today would be classified as crime novels – Oedipus Rex; Macbeth; The Moonstone; Crime and Punishment; Les Miserables; Therese Raquin; Despair. The canon abounds with crime novels in disguise.

Where recent literary fiction has been predominantly concerned with the individual, crime fiction is focused on society and, more specifically, the cracks, fissures and fault-lines of cities, countries and families. The crime novel gives lie to the myth of progress and civilisation by occupying the dark labyrinths within our collective systems and personalities. It provides a key to the deepest locked rooms of human psychology.

Crime fiction reveals a world far removed from the middle-class bubble many of us live in. Crime fiction is always about the outsider. It reflects society in real time by tackling contemporary issues whether it be terrorism, trafficking, or the rancour of a marriage gone sour. But it also poses deeper, older questions about justice, evil and death. It asks moral questions of us both as readers and as a society. Do we value the victim or celebrate the killer? What drives normally sane people to murder? How does society treat its outsiders? Can crime ever be erased?

Crime fiction is uniquely placed to answer such moral quandaries. It is also incredibly compelling to read. At the end of a crime novel the mystery may be neatly solved but only in order to open up the deeper mystery of human darkness. A good crime novel makes you look at the world in a new way and that, surely, is the point of literature.

Three great crime novels to get you started

The Mask of Dimitrios by Eric Ambler (1939)

A bored English crime writer living in Istanbul becomes obsessed with the death of an infamous brigand and crosses pre-war Europe in search of the story behind the man. Both a deft meta-fictional exploration of the gap between reality and its representations and a philosophical inquiry into the nature of identity, The Mask of Dimitrios is also a stunning plunge into the scorched-earth history of Europe.

The Power of the Dog by Don Winslow (2005)

James Ellroy called it the War & Peace of dope novels and, despite the hyperbole, he’s not wrong. Over a stunningly detailed canvas worthy of Bosch, Winslow charts the futility and depredations of the 30-year War on Drugs. Taking on organised crime, political corruption, serial murder and liberation theology, The Power of the Dog is the ultimate report on Mexico’s slow-burning dissolution and a stark warning about the dangers of organised crime co-opting politics. One of the most trenchant and disturbing missives of our time. The fantastic, and even darker, sequel, The Cartel, came out earlier this year.

The Killer is Dying by James Sallis (2011)

Sallis writes poetic, existential crime fiction soaked in melancholy but he’s also translated French surrealist poets, written science fiction, poetry, a book on Jazz guitar and several biographies. The Killer is Dying is his masterpiece – a moving, elegiac meditation on suffering and death that snakes through the streets of Phoenix following a terminally ill hitman, a disenchanted cop and a lost youth. Sallis slows the action down almost to a stop so as to extract the greatest emotion out of his steely and often startling prose.

Stav Sherez is crime novelist published by Faber & Faber. His latest novel, The Intrusions, is published in February. Follow him on Twitter via @stavsherez