In a banqueting hall lined with mahogany panels and lit with chandeliers, in the august surroundings of the Union League, an imposing civil war-era building that occupies an entire block of Philadelphia, about 100 men and women are sitting down to a very good lunch. The starter of squash soup is followed by chicken breast smothered in parsley sauce with creamed potatoes, and a rich cheesecake. But as I'm tucking into the dessert, a woman at my table leans over to me and says: "I'd go easy on that, if I were you. It may not agree with the presentation."

The comment is puzzling. But when the plates are cleared away, coffee served, and the real business of the lunch gets under way, it quickly becomes clear what she means.

The presentation begins with a giant photograph projected on to a screen a few feet away from my table. It shows a woman lying on her back, wearing a blue sweatshirt, her head turned slightly to one side. Her face has become a strange shade of purple. Blood has congealed around her nose and mouth and a pool of it has collected behind her tussled ginger hair.

The blow-up of the dead woman, and the cheesecake now sitting heavily in my stomach, make decidedly awkward companions.

This is the monthly lunch of one of the world's most exclusive clubs that, as its motto – Cuisine & Crime Solving – suggests, is devoted to the twin obsessions of food and murder. And unlike me, the members are well accustomed to the peculiar combination of fine food and grizzly gore.

The assembled gourmands include some of the best detective brains in America and from across the globe – public prosecutors, FBI profilers, murder detectives, forensic scientists and artists, psychologists and anthropologists, security consultants and coroners. At my table, there is a woman who specialises in forensic anthropology (the analysis of human bones) at the Mütter museum, Philadelphia's famous collection of medical oddities (she was the one who warned me about the cheesecake), a leading forensic toxicologist and an authority on ritualistic murders and mutilations.

They are members of the Vidocq Society, a network of detectives that not only acts as a social meeting place, bringing members and their guests together on the third Thursday of every month, but also as a sort of giant game of Cluedo, pitting their enormous collective wealth of forensic and sleuthing expertise against some of the 100,000 murders that remain unsolved in America today. Between them, they come from 17 states across America and 11 countries. It's like seating Sherlock Holmes, Hercule Poirot, Miss Marple, Inspector Morse, Jessica Fletcher, Kurt Wallander and Kojak in front of a sumptuous meal, presenting them with a cold case that needs an injection of new thinking, and watching the sparks fly.

The society was founded in 1990 by three acquaintances in crime solving – William Fleisher, Richard Walter and Frank Bender. Fleisher is a large, avuncular, bearded polygraph expert, who was an FBI special agent and top customs investigator. Walter is one of the world's most respected forensic psychologists specialising in delving into the dark mind of the serial killer. Rake-thin and spiky, he has worked across America, but because of his demeanour, people call him "the Englishman". Bender is an artist, who stumbled into forensic work. He uses pencil and clay to reconstruct the facial features of victims and suspects as an aid to identification.

The three got to know each other working on live cases and, finding that their diverse skills were highly complementary, came up with the idea of a club that would allow information to be shared and connections made across disciplines, in a way that red tape-bound police bureaucracies often don't do. It was Fleisher's idea to name the society after his hero, Eugène François Vidocq, the father of modern detective work. After many years spent in prison, Vidocq turned himself around, and in 1811 set up the first French state investigative agency, Brigade de la Sûreté, going on to advise Sir Robert Peel on the formation of Scotland Yard. Later, Vidocq created his own private detective agency, Le Bureau de Renseignements, where he pioneered many new forensic techniques, including the use of indelible ink, undercover work and plaster casts of footprints to trace culprits. "He was the first really great detective," Fleisher says. "It seemed a shame to me that someone who had contributed so much to my field had largely been forgotten."

Fleisher is disarmingly honest about the initial purpose of the society. "To be frank, we started out just to get together and have a good lunch. And why not?" But then one of the first members to sign up was asked to speak at a conference in Texas organised by parents of murdered children. He came back ashen-faced and determined to change the society's mission.

"He said we must dedicate ourselves to solving cases for people who are hurting, not just have a good time," Fleisher recalls. "And that's what we did. Our mission today is to act as a voice and a catalyst for all the people touched by a murder, particularly the victims' relatives."

The society grew rapidly until it had filled its quota of 82 members – a symbolic number as Vidocq died in 1857 aged 82. The membership remains at 82, with members holding their positions for life. New members have to be nominated by existing ones, though the overall ranks of lunch-goers has swollen as a result of guests and associates tagging along. They quickly established a format for meetings – first would come the fine cuisine, then the hard work. Each lunch would be addressed by an interested party – relatives, police officers or prosecutors – who would set out the basic details of an unsolved case that was at least two years old. They would then open the floor to questions and comments from Vidocq Society members.

At today's lunch, the visiting presenters are detectives from North Carolina who have come to talk about the case of the dead woman whose body is being beamed gruesomely on the screen. Under society rules, I can describe the broad details of their presentation, but must not divulge names because that might impair the reopening of the case.

The victim, let's call her Sally, was working a farm with her husband, say Tony, in North Carolina. She went missing in December 2005. On the day she disappeared, Tony had been seen in the morning driving out to a nearby field in a tractor with Sally following in her pick-up truck. Neighbouring farmers heard two shots around that time being fired from the direction of the field. Later, Tony returned alone from the field saying he didn't know where his wife had gone and asking his farm workers to wash down the tractor.

Her body was found 10 days later in the toolbox of her pick-up truck, which had been abandoned across state lines. She had died from two gunshot wounds to the chest. The body seemed to still be fresh – there was no smell, flies or rigor mortis. That puzzled detectives, because she had been missing for more than a week and yet there were no signs that she had been held captive or restrained.

Detectives investigating the case found that Tony had been married before and they interviewed the previous wife. She said she had left him suddenly after she became alarmed by his actions. He had taken out a life insurance policy against her name for $400,000 (£250,000). In the space of a week, she had twice been rendered unconscious by someone creeping up on her from behind. On one of those occasions, Tony had then locked her in a metal box for half an hour, only releasing her after she begged him to do so.

Eventually, police amassed enough circumstantial evidence to charge Tony with the murder of Sally. He was put on trial, but the hearing ended with a hung jury. Eleven jurors found him guilty, but one opted to acquit. The judge had ruled that the story of the first wife was inadmissible in court, which was a blow to the prosecution case. The other problem was the time of death. The defence lawyer focused on that issue, telling the jury that the relatively fresh condition of the body suggested she could not have been killed on the day she disappeared, which would negate much of the evidence against Tony.

As the presentation gets under way, an extraordinary air of silent concentration settles over the banqueting hall. The clink of cutlery against china fades and is replaced by pens scratching on notebooks. When it is revealed that Tony's trial ended in a hung jury, there is a collective groan across the room.

Then the questions start to fly. Was Tony likely to have profited from Sally's death, as he apparently tried to do with his first wife? Yes, the detective says, he was likely to have gained close to $6m (£3.7m) in insurance payouts. Had all relevant phone records been traced? Yes, but that had led nowhere. Was a metal detector used to search the field where Tony and Sally had gone, in the hunt for spent bullets? Yes, again with no result. Had Sally's body been frozen? No, there was no sign of refrigeration.

What's happened to Sally's body? "Cremated," says the detective.

"Damn! No bones!" exclaims Anna Dhody, the woman sitting next to me. "I would have wanted to have exhumed the body," she says. "The body was covered in trace evidence. I'm hoping that her clothes have been preserved and in the absence of the body, that might yield some clues."

Such probing questions and attention to detail are typical of the Vidocq Society's approach to the cold cases that are presented to it. Its track record in breathing life back into cases, some of which have been inactive for years, is impressive. The society reckons it has helped solve around 300 murders, and provided a leg-up in about 90% of the cases it's heard.

Take the case put before a recent lunch. It is still active and details must be kept sketchy, but it involved the murder of a woman that happened 20 years ago. The police at the time had concluded it was the work of a random stranger and eventually admitted defeat in trying to find the culprit. The case lay dormant until a few months ago when relatives contacted the society, and it in turn enlisted the help of Lee Goff, a world-renowned forensic entomologist. He noticed that the woman's body had been covered in maggots when it arrived at the morgue and, based on the hatching cycle of the blowfly, put back the time of death by 18 hours. The shift in time frame was crucial: it destroyed the alibi of the woman's husband, the case was reopened and he is now awaiting trial.

Several of the society's most famous cold cases are profiled in a new book, The Murder Room, by Michael Capuzzo. They include the story of Scott Dunn, a young man who went missing in Texas in 1991. The local police thought they had a missing person on their hands and the case went nowhere. But it was brought to the attention of the Vidocq Society and its founding member Richard Walter. He psychologically profiled Dunn's then girlfriend, Leisha Hamilton, and concluded she was a murderous psychopath. Under Walter's incessant encouragement, fresh forensic work was done on Dunn's apartment that showed copious blood stains on the walls. The results were put in front of the London-based forensic pathologist, Richard Shepherd, who concluded that Dunn must have died from being struck at least four times. Despite no body or weapon ever being found, Hamilton was convicted of murder and is in jail today. "I'll do whatever I have to do to get justice," Walter says. "People had laughed at me and said I'd never get a conviction, but because I'm stubborn, I just wouldn't give up."

Fleisher's favourite case is the society's longest-standing, and as yet unsolved, challenge. It is the case of the Boy in the Box, or America's Unknown Child, a notorious Philadelphia murder of an unidentified boy, aged four or five, who was found in a cardboard box in 1957. Fleisher remembers learning of the murder when he was still a teenager. "It was my first exposure to the shocking reality of the world, seeing a poster with that boy's photo on it. I remember thinking here's a dead child who was murdered, found not far from where I lived. That always stuck in my mind."

Fleisher and other Vidocq Society members continue to work on the case, keeping it alive after more than half a century. New leads are currently being followed up.

At the lunch, the question-and-answer session has ended and Vidocq members are queueing up to talk to the North Carolina detectives to give them one-to-one advice on how to revive the hunt for Sally's murderer. An expert in cellphone technology tells them of a federal database created in the wake of 9/11 that allows the tracking of most calls going back 15 years; the former chief prosecutor for Philadelphia tells them to bring Tony in for further questioning and videotape the conversation; a former FBI agent encourages them to investigate what Tony did with the life insurance money, and a medical examiner tells them that the fresh state of Sally's body should not distract them – it could simply be explained by the fact that the corpse was placed in the pick-up truck's metal toolbox, which in winter acted as a refrigerator.

At the end, the detectives look a little dazed by the explosive input of specialist knowledge that has been fired at them. "This has given us a whole new insight and new things to think about," one of them says.

As the Vidocq Society's members file out of the hall, returning to their day jobs, Sally's bloodied face stares down on them from the projector screen. She came to a sorry end, but at least for the first time in months, there is hope in the air that justice might eventually be done.