"My uncle Frank was a burglar and our family never saw any harm in that," runs the opening paragraph of Burglar to the Nobility, the autobiography of John "Ruby" Sparks. "But my mother did object to the way that he smooched around in baggy trousers, his jersey a different colour under the arms where sweat made it look as if a custard tart had melted there."

It is 50 years since the idiosyncratic confessions of one of the last century's most colourful villains was published, but the appetite for criminal memoirs remains as strong as ever. This month, some of the country's best-known former inmates will discuss how one of the simplest escape routes from prison and a life of crime is writing, at the Theakstons Old Peculier Crime Writing Festival in Harrogate. The law-abiding reader remains prepared to forgive almost anything in exchange for a glimpse of the wild side of life.

Few criminals today would manage to slip a melted custard tart and their uncle's sweaty armpits into their opening paragraph but Sparks was writing for a public that was still uncertain about its attitude to criminals chuckling over their misdeeds. "Ruby" Sparks had a number of claims to fame, including the nickname he acquired as a boy when he burgled a Mayfair mansion and stole £45,000- worth of a maharajah's rubies which he then gave away, mistaking them for cheap imitation jewels. In the 1920s he introduced motorised smash-and-grab to the streets of London with the assistance of getaway driver Lillian Goldstein, a middle-class young woman from Wembley; and he was a leader of the Dartmoor Mutiny, during which the inmates briefly took over the prison in 1932.

His book is a classic of the period, ghosted in larky, Runyonesque prose by Norman Price, with every robbery a "tickle" worth loads of "crinkle". Sparks ponders on why criminals spend the money they steal so swiftly: "It probably sounds a bit milky if I was to say now the reason which makes thieves and villains get through their ill-gotten wages so sharpish is they must inside themselves feel somehow guilty about it, but there's got to be some explanation why we all raced the gelt like we did." He concludes by explaining that he is now content to run a newsagent's in Chalk Farm and just wishes he had gone in for a "straight business" 40 years earlier.

Sadly, Goldstein never wrote her own memoirs, although Sparks credits her with some of the most inventive of his criminal techniques: she encouraged him to take bulldog paperclips with him on his smash-and-grabs to hold the cuts on his hands and arms together until she could stitch them up. She is quoted in the book as she bids a final farewell to Sparks, explaining her unwillingness to take part in a robbery with two young criminals which would involve throwing ammonia in someone's face: "I've had this Bandit Queen lark, crime is for kids – like those two greasy-haired spiv wonders – not for grown-ups."

Sparks was following in the footsteps of Eddie Guerin, whose Crime: The Autobiography of a Crook had been published in 1928. The London-born Guerin made his name as "king of the underworld" in Chicago and Paris, where he robbed the American Express office in 1901. His reputation was partly based on the belief that he had escaped from Devil's Island although it had, in fact, been another French penal colony. He clearly relished his early career – "what a red-hot game it was" – but acknowledged that "probably there will be readers who turn up their noses in disgust over a criminal setting down in cold print the unsavoury experiences of the past."

The book that set the postwar standard for the genre was Billy Hill's autobiography Boss of Britain's Underworld, ghosted by Duncan Webb, the enigmatic crime correspondent of the People. The launch party, in 1955, was held at Gennaro's – now the Groucho Club – and the Sunday Times reported it as an event that "made even Soho gasp… The guests included Sir Bernard and Lady Docker [celebrities before celebrities existed], former CID officers and many of London's scar-faced underworld."

From left: Soho Ted, Bugsy, Groin Frankie, Billy Hill, Ruby Sparks, Razor Frankie, College Harry, Frany the Spaniel, Cherry Bill, Johnny Ricco, a female journalist and Russian Ted enjoy the launch party for Billy Hill’s autobiography. Photograph: Bert Hardy/Getty Images

Guests were presented with a souvenir document bearing a red seal with Hill's autograph and fingerprints attached, and the proceedings opened with a blast from a police whistle. Hill's pals dressed up in masks and toy police helmets. Facetious congratulatory telegrams were read out. "Sorry I can't be here – I'm in a spot, Jack," said one – a crack at the expense of Hill's great underworld rival, Jack "Spot" Comer. "Will you send us back our mail – we miss it. Postmaster general," read another, a reference to Hill's role in the 1952 £287,000 robbery of a post office van for which no one was ever charged. Lady Docker and Hill posed for photographs together, while champagne and saddles of mutton were served.

The event was also covered by the Picture Post, with snaps by Bert Hardy, one of the great photographers of the era, and included shots of a chap known as "Striper". Not all of the media was impressed. The Daily Sketch's Simon Ward told his readers that there had been "nothing like it since the days of Al Capone in Chicago" and suggested that Hill had thrown "an astonishing party to cock a snook at the police".

Crime fiction writer Dreda Say Mitchell, author of Geezer Girls and Running Hot, and chair of this year's Harrogate festival, studied the criminal memoir long before she started her career. "I was a big reader of them even before I began writing as so many are based in the East End, where I grew up," she says. "Lennie 'The Guv' McLean's scrapyard was round the corner from our estate and The Blind Beggar (where Ronnie Kray shot George Cornell dead) was half a mile away. When it comes to writing crime though, [criminal memoirs] are not that helpful because they're nearly always co-written with professional journalists who know what publishers and readers want to hear – that the hero is a cross between Jesse James and Robin Hood – so the material is edited accordingly."

How such memoirs are regarded inside prison is another matter. "I had never read a criminal memoir before I went to prison," says Erwin James, who wrote a column about prison life for the Guardian during the final years of the 20 he served. "Books about crime and prison are among the most popular with prisoners. I think that's because, as a convicted criminal, in my case convicted of murder, you feel the weight of society's disgust and disapproval heavily and reading about how others in similar situations dealt with it can bring a lot of comfort."

Erwin James, writer and ex-prisoner. Photograph: David Levene

James, whose latest book, Life Before A Life Inside, is published next year, rates Papillon, by Henri Charrière, as the best of the genre. Others he values are Guerin's book, and Respect by Freddie Foreman, Autobiography of a Thief by Bruce Reynolds and Autobiography of a Murderer by Hugh Collins. He recalls reading Knightsbridge: Robbery of the Century, by Valerio Viccei, about the safety deposit heist which netted an estimated £40m in 1987. Viccei – "the Italian stallion" as the tabloids had him – was killed in 2000 in a shoot-out with police outside Rome when on day release from prison. No newsagent's in Chalk Farm for him.

"I was never usually drawn to books that affected to make criminal lifestyles appear glamorous," says James. "Meeting the often charismatic people you read about in the popular press – who operated as highly organised professional robbers, sometimes stealing tens of millions in cash, or gold – while they can indeed be attractive and compelling individuals and stand high in the prisoner hierarchy, the only real difference is that the photos they display on their cell walls are of luxury cars, big houses and expensive holiday locations. As the years pass in prison you see them getting old, missing families and harbouring huge regrets just like everyone else."

Former armed robber Noel "Razor" Smith, author of the much-admired A Few Kind Words and a Loaded Gun (2004) and A Rusty Gun (2010) was also initially wary of the criminal autobiography. "The reason I didn't want to write my own memoirs in the first place was because I had read so many of them and they were all the same. Reading their books you might be forgiven for believing that these geezers never lost a fight, always nicked over a million quid and never had a moment of fear or self-doubt. In the end I wanted to do something unique in the true-crime autobiography game: tell the actual truth! Sometimes I got my head kicked in and sometimes I went on a robbery and got nothing; that's what life as a criminal is really like."

Ex-con turned writer Noel “Razor” Smith in prison. Photograph: Sean Smith for the Guardian

In A Few Kind Words, Smith ruefully recalls his botched jobs, including one where he tried to hold up a newsagent's with a Luger pistol only to be told by its Ugandan-Asian proprietor, with commendable sang-froid, "Your gun is unloaded – you are minus the magazine. And you swear far too much for such a young man." An abashed Smith bought a Mars Bar instead. He reflects in A Rusty Gun: "When you're young and strong and you can afford to throw away a decade or two in some pisshole prison and still have plenty of life left to live, it's all a big laugh. Then you wake up one morning and see a strange face staring back at you from the shaving mirror. Some old geezer with bitter, weary eyes where there used to be a devil-may-care twinkle."

Cass Pennant was jailed for four years as a leader of the notorious West Ham football hooligans, the Inter City Firm (ICF). He emerged from prison not only to write a successful memoir, Cass, which was made into a film in 2008, but also to start a publishing house, Pennant Books, which has since brought out other tales of life on the wrong side of the law. He attributes the success of his book to the fact that it was truthful. "One thing I wanted to be sure of was that, if I had my own story published, it would be an honest one." McVicar by Himself, the autobiography of former robber John McVicar, is another volume that led to an eponymous film, starring Roger Daltrey.

Among the most original memoirs is Gentleman Thief (1995) by Peter Scott, who made and lost a fortune burgling the homes of the wealthy and notoriously stole Sophia Loren's diamonds. "Readers may relish the idea of a 'master criminal'; alas, such people don't exist," wrote Scott, now 80 and living on a rough estate in King's Cross, London. "Raffles was the stuff of fiction. Thieves in the main get caught. Persistent ones get caught more frequently, few escape the narrow aisles of pain." Freddie Foreman, author of Respect (1997), concludes by telling his fellow cons to "read everything and try to educate yourself towards a better life. The old ways have gone. Computers. Now that's the best advice I can give. There must be a clue there to have a good touch."

Reggie Kray with Barbara Windsor. Photograph: popperfoto

While the best book on the Kray twins remains a biography (John Pearson's Profession of Violence), the brothers' criminal careers helped to spawn more than a score of mainly unremarkable reminiscences by various henchmen. One familiar strand in such memoirs is to sound off about the sex offenders with whom the author has to share prison space. Thus Ronnie Knight (club owner, wide boy, former husband of Barbara Windsor) in Blood and Revenge: "What they need is branding on the forehead with a red-hot poker then decent rascals would know to blank them." So far, this is not among the government's beefed-up criminal justice proposals. Most memoirs are by former gangsters and robbers with only the occasional drugs smuggler (Howard Marks and his best seller Mr Nice) or informer (Maurice O'Mahoney and King Squealer) making an appearance.

The most successful are those that include self-reflection. Jimmy Boyle, former Glasgow hardman-turned-sculptor, wrote his autobiography, A Sense of Freedom, in 1977 in Barlinnie Prison. Boyle noted: "In writing the book in a manner that expresses all the hatred and rage that I felt at the time... I have been told that I lose the sympathy of the reader and that this isn't wise for someone who is still owned by the state and dependent on the authorities for a parole date... The book is a genuine attempt to warn young people that there is nothing glamorous about getting involved in crime and violence."

Two laws passed in the last decade could affect the business of true crime writing. The Criminal Justice Act of 2003 effectively brought an end to the concept of "double jeopardy". Previously, once a person had been acquitted of a crime they could not be retried for the same offence. This meant that a murderer could, if found not guilty, boast of it in a memoir without fear of the consequences. No more. Now an admission of a gangland hit could be all that the Director of Public Prosecutions requires to reopen a case.

No law prevents criminals from publishing their memoirs but there are restrictions on profiting from them, as enacted by the Coroners and Justice Act 2009. This enables courts to recover any assets that a criminal has acquired as a result of writing about their crimes. That money then goes into something called the Consolidated Fund. Those still serving a sentence can be prevented from publishing while in custody, as happened to serial killer Dennis Nilsen, whose autobiographical manuscript was confiscated. During the debate on the bill Baroness Rendell of Babergh (aka Ruth Rendell) pointed out that Jean Genet's The Thief's Journal might have fallen foul of such a law. In fact, great train robber Bruce Reynolds called his own memoir Autobiography of a Thief as a homage to Genet, whom he admired.

But neither of these laws, which have been only half-heartedly enforced since their enactment, seem likely to dent the public fascination with first-hand accounts of the criminal life. What George Orwell described in his essay, Decline of the English Murder, as the nation's "all-prevailing hypocrisy" should ensure that true crime will always find shelf space beside its fictional brothers and sisters. "When God erects a house of prayer, the devil always builds a chapel there," wrote Daniel Defoe, an ex-con who did a bit of writing himself. "And 'twill be found, upon examination, the latter has the larger congregation."

The Theakstons Old Peculier Crime Writing Festival runs from 21-24 July (harrogate-festival.org.uk/crime)