Fossil Ida only came to light with the help of a private dealer, through a bargain struck in a Hamburg vodka bar. Photograph: Atlantic Productions Ltd

Rumour had it there was something special in the hands of a private fossil collector who lived in the countryside on the outskirts of Frankfurt. Jonathan Blair, a photographer working for National Geographic magazine, got in a car with his contact and drove out to take a look. Once inside, they were ushered into the living room. There on the wall was a beautifully preserved, nearly complete fossil of an ancient lemur-like primate. It was almost certainly the same fossil that was unveiled to the world this week as Ida, a supposed "missing link" in the evolutionary tree.

It is 10 years since Blair saw the fossil and was told not to take pictures of it. The remarkable specimen came to light only after the collector enlisted the help of a private dealer, Thomas Perner. He brokered a $1m (£629,000) deal for the 47m-year-old primate with a paleontologist from Oslo's Natural History Museum in a vodka bar in Hamburg. The identity of the collector remains a well-guarded secret.

"I tried to photograph it, but the guy, a dentist I think, said no. It would have been a real headliner. We were trying to get the scoop, but we didn't manage it. It was all off the record," Blair told the Guardian from his office in New York.

Scientists have yet to reach a consensus on the importance of Ida to our evolutionary history, but she has already shed light on the murky world of fossil dealing. This is an international business, where middlemen, who often work with unnamed buyers and sellers, negotiate staggering sums of money for fossils that are sometimes of uncertain provenance and legality. While academics spend years unearthing and characterising fossils to further our knowledge of life's history, there are private fossil hunters driving around with picks and shovels, intent on grabbing what they can to sell to the highest bidder.

In most countries, if you find a fossil on your own land it is yours to keep, but any others belong to the state. The law works, but only up to a point. Private fossil hunters have been known to offer landowners derisory sums for fossils, before selling them on at vast profit. Prime specimens, like Ida, can be kept secret and go unstudied for decades. Often, fossils split in two during excavation and each part is sold to a different collector. And when it comes to policing government land, there is simply too much of it to keep an eye on.

"There are people who get caught collecting on federal land, but elsewhere it is worse. In China, a lot of specimens are collected and basically smuggled out of the country to be sold," said John "Jack" Horner, a world-renowned paleontologist at the Museum of the Rockies in Montana. It's not just private hunters who fall foul of the law. Earlier this year, a leading palaeontologist, Nate Murphy, pleaded guilty to stealing a raptor fossil worth up to $400,000 from federal land in the US.

In academia, there are mixed feelings about private collectors. Some argue that they stifle scientific research, because journals won't publish details of fossils held in private collections. But others concede that many collectors are legitimate and passionate about their fossils. Without them, some specimens would undoubtedly still be lodged in the ground.

"It is certainly a problem and some very shady deals go on. Commercial collectors sell fossils to anyone, and if private collectors buy them, then no one else knows about them," says Horner. "There are probably some fantastic specimens in private hands, and the owners themselves won't even know it. I would be willing to bet there are archaeoptryx in private collections somewhere out there." Archaeopteryx is one of the most striking fossils on record. The creature lived about 150m years ago and is regarded as a true ­missing link between dinosaurs and modern birds.

Among fossil collectors, the specimens from the Messel pit near Darmstadt where Ida was unearthed are particularly desirable. Many are extraordinarily well-preserved thanks to the unique conditions of the oil shale in the pit. Blair speaks of the specimens as objects of the rarest beauty. "The Messel stuff is gorgeous. In some cases, you can almost feel the animal crawling up a tree or running along the ground," he says. The pit has proved an ancient treasure trove of exquisite pygmy horses, anteaters, tapirs, birds and bats. Some have price tags of more than $200,000. To enthusiasts, this is not like having a private art collection. The fossils are more prestigious than that.

Of all the ancient remains that have made it on to the open market, one had more impact on prices and fossil fakery than any other. In the 1990s, fossil dealers were pitted against academics over the stunning remains of a Tyrannosaurus rex that was dug out of a cliff in South Dakota. Named Sue after its discoverer, Susan Hendrickson, the 65m-year-old fossil sold at auction for more than $8m. "As soon as that happened, a whole bunch of people thought well, if I have a dinosaur, maybe it'll be worth millions as well," says Horner.

The high prices that fossils can fetch have encouraged a booming industry in fakes, many of which have originated in China. One, known as Archaeoraptor, was covered in National Geographic in 1999 as a missing link between birds and dinosaurs. It was later confirmed to be a fake, cobbled together by gluing a dinosaur's tail on to a bird. It was duly renamed Piltdown Turkey, a reference to the most famous paleontological hoax in human history.

When Blair saw the recent unveiling of Ida, he hoped the phone would ring and he would finally be on his way to photograph the fossil. "I don't know why someone would want it in the house. It's worth money, and someone is going to come and take it. The last time I saw it, it was hanging on that wall. That was long before they decided to call it Ida," he says.