It remains one of the greatest human fossil discoveries of all time. The bones of a race of tiny primitive people, who used stone tools to hunt pony-sized elephants and battle huge Komodo dragons, were discovered on the Indonesian island of Flores in 2004.

The team of Australian researchers had been working in a vast limestone cavern, called Liang Bua, in one of the island's remotest areas, when one scientist ran his trowel against a piece of bone. Carefully the group began scraping away the brown clay in which pieces of a tiny skull, and a little lower jaw, were embedded.

This was not any old skull, they quickly realised. Although small, it had special characteristics. In particular, it had adult teeth. "This was no child, but a tiny adult; in fact, one of the smallest adult hominids ever found in the fossil record," says Mike Morwood, of Australia's University of Wollongong and a leader of the original Flores expedition team.

The pieces of bone were carefully wrapped in newspaper, packed in cardboard boxes and then cradled on the laps of scientists on their journey, by ferry and plane, back to Jakarta. Then the pieces of skull, as well as bones from other skeletons found in Liang Bua, were put together.

The end result caused consternation. These remains came from a species that turned out to be only three feet tall and had the brain the size of an orange. Yet it used quite sophisticated stone tools. And that was a real puzzle. How on earth could such individuals have made complex implements and survived for aeons on this remote part of the Malay archipelago?

Some simply dismissed the bones as the remains of deformed modern humans with diseases that had caused them to shrink: to them, they were just pathological oddities, it was alleged. Most researchers disagreed, however. The hobbits were the descendants of a race of far larger, ancient humans who had thrived around a million years ago. These people, known as Homo erectus, had become stranded on the island and then had shrunk in an evolutionary response to the island's limited resources.

That is odd enough. However, new evidence suggests the little folk of Flores may be even stranger in origin. According to a growing number of scientists, Homo floresiensis is probably a direct descendant of some of the first apemen to evolve on the African savannah three million years ago. These primitive hominids somehow travelled half a world from their probable birthplace in the Rift Valley to make their homes among the orangutans, giant turtles and rare birds of Indonesia before eventually reaching Flores.

It sounds improbable but the basic physical similarity between the two species is striking. Consider Lucy, the 3.2 million-year-old member of Australopithecus afarensis. She had a very small brain, primitive wrists, feet and teeth and was only one metre tall, but was still declared "the grandmother of humanity" after her discovery in Ethiopia in 1974. Crucially, analysis of Lucy's skeleton shows it has great similarities with the bones of H. floresiensis, although her species died out millions of years ago while the hobbits hung on in Flores until about 17,000 years ago. This latter figure is staggeringly close in terms of recent human evolution and indicates that long after the Neanderthals, our closest evolutionary relatives, had disappeared from the face of the Earth around 35,000 years ago, these tiny, distant relatives of Homo sapiens were still living on remote Flores.

The crucial point about this interpretation is that it explains why the Flores people had such minuscule proportions. They didn't shrink but were small from the start – because they came from a very ancient lineage of little apemen. They acquired no diseased deformities, nor did they evolve a smaller stature over time. They were, in essence, an anthropological relic and Flores was an evolutionary time capsule. In research that provides further support for this idea, scientists have recently dated some stone tools on Flores as being around 1.1 million years old, far older than had been previously supposed.

The possibility that a very primitive member of the genus Homo left Africa, roughly two million years ago, and that a descendant population persisted until only several thousand years ago, is one of the more provocative hypotheses to have emerged in anthropology during the past few years," David Strait of the University of Albany told Scientific American recently. This view is backed by Professor Chris Stringer of the Natural History Museum, London. "We are still grappling with what this discovery has done for our thinking and our conventional scenarios."

In addition, Mike Morwood says he has now uncovered stone tools on nearby Sulawesi. These could be almost two million years old, he believes, which suggests the whole region was populated by very ancient humans for a startlingly long part of human prehistory. "This is going to put the cat among the pigeons," Morwood says.

However, it is the hobbits' similarity to ancient African apemen that provides the most compelling evidence for their ancient origins. In the Journal of Human Evolution, a team led by Debbie Argue of the Australian National University, recently reported that analysis of H. floresiensis shows they most closely resemble apelike human ancestors that first appeared around 2.3 million years ago in Africa. In other words, their stock may be not quite as old as Lucy's but probably comes from a hominid, known as Homo habilis, that appeared on the evolutionary scene not long after Lucy's species disappeared. Homo habilis's features now seem to match, most closely, those of H. floresiensis.

Consider those hobbit feet, for example. The skeleton unearthed on Flores had a foot that was 20cm in length. This produces a ratio of 70 per cent when compared with the length of the hobbit's thigh bone. By contrast, men and women today have foot-to-thigh bone ratios of 55 per cent. The little folk of Flores had singularly short legs and long, flapper feet, very similar to those of African apemen, even though limbs like these would have made their long march from Africa to Flores a painful business.

Similarly, the hands of H. floresiensis were more like apes than those of evolved humans, their wrists possessing trapezoid bones that would have made the delicate art of stone tool-making very difficult. Their teeth show primitive traits while their brains were little bigger than those of chimpanzees, though CT scans of skull interiors suggest they may have had cognitive skills not possessed by apes.

Nevertheless, this little apeman, with poor physique, a chimp-sized brain and only a limited ability to make tools, now appears to have left Africa, travelled thousands of miles and somehow colonised part, if not all, of south-east Asia two million years ago.

Scientists had previously assumed only a far more advanced human ancestor, such as Homo erectus, was capable of undertaking that task and only managed to do so about a million years ago when our predecessors had evolved powerful physiques, a good gait and the beginnings of intellect. Without these, we would have got nowhere, it was implied.

Then along came little H. floresiensis which, quite simply, has "no business being there," says Morwood. And you can see what he means. Apart from the sheer improbability of a jumped-up ape travelling from Africa to Indonesia, there is the particular puzzle of how it got to Flores.

Primitive hominids were almost certainly incapable of sailing. So how did it arrive on the island in the first place? It is a puzzle, although Stringer believes the region's intense tectonic activity is significant. "After the Indian Ocean tsunami in 2004, people were found far out at sea clinging to rafts of vegetation. Things like that could have happened regularly in the past and people could have been swept out to sea and washed ashore on Flores. Alternatively, there could have been short-lived connections between now separate islands."

Thus, ancient African apemen travelled half the world, made homes across Indonesia and, in one case, were washed out to sea to end up colonising a remote island that was already populated with pygmy elephants, called stegadons, and giant Komodo dragons, which are still found on the island. It is a truly fantastic tale, worthy of Rider Haggard, and it has turned the study of human evolution on its head.

And then there is the report that dates the stone tools found on Flores as being 1.1 million years old. "That is utterly remarkable on its own," adds Morwood. "Until we found these dates, the longest period of island isolation that we knew about occurred on Tasmania where the aboriginal people were cut off from mainland Australia 11,000 years ago. We thought that was an amazing length of time. But now we have found an island where early humans were cut off from the rest of evolution for more than a million years." In addition, there are those completed digs carried out by Morwood which suggest that some type of human being was making stone implements up to two million years ago.

A crucial aspect to this remarkable story is the region's geography, Morwood believes. The ocean currents and the remoteness of Flores make the island difficult to get to, so once a species does get there, it will remain well protected on it, he argues. "Flores seems to protect species that are long past their use-by dates. There were those pygmy elephants, and the Komodo dragon, for example. And now we have Homo floresiensis. It may be that only a few animals get there but when they do arrive they tend to survive for a long time, which has been science's good fortune."

That is putting it mildly. Had not the original Australian team, led by Morwood, uncovered those hobbit remains in 2004, the story of humanity's African exodus would have been considered a fairly simple affair.

According to this version of events, Homo erectus evolved from apemen predecessors, such as Australopithecus africanus, in Africa and then headed off around the Old World more than a million years ago, armed with a great physique and a modest intellect. These allowed it to settle across Africa, Asia and Europe. This diaspora was then followed by a second wave of humans – our own species, Homo sapiens – which emerged from Africa 100,000 years ago and took over the planet, replacing all pockets of its predecessors it encountered.

Now a far more complex picture is emerging. Ancient apemen, who might have been thought to lack the nous for global conquest, appear to have done the trick almost a million years earlier. One of the major tenets of human evolution, the story of our world conquest, is now urgently in need of revision.

As to the fate of H. floresiensis, that is unclear. The species disappears abruptly from the archaeological record 17,000 years ago. But why? They had apparently survived quite happily on the island for more than a million years. So what did for them in the end?

There are two competing answers. The first suggests that the species, after all the good fortune that had helped it endure the vicissitudes of life in the Malay Archipelago, ran out of luck. "There is a thick layer of ash in the Liang Bua cave above the most recent hobbit remains," says Stringer. "We now know this was caused by a major volcanic eruption which occurred about 17,000 years ago. So it may be that they were just unlucky with the local geology." According to this vision, the little folk of Flores were wiped out by choking plumes of volcanic ash or died of starvation on an island denuded of vegetation.

It would have been a pretty terrible way to go. Yet neither Stringer nor Morwood is convinced that was what happened, despite the tight link between dates of eruptions on the island and the disappearance of the species from the fossil record. Instead, they suspect a very different agent: the bloody hand of modern humans. "Look at our track record," says Morwood. When Homo sapiens entered Europe 40,000 years ago, on its route out of Africa, they would have encountered the continent's original inhabitants, the Neanderthals. Within a few millenniums, the Neanderthals had been rendered extinct.

Stringer agrees. Homo sapiens left Africa about 100,000 years ago and by the time hobbits became extinct on Flores, modern humans were all over south-east Asia. "I cannot see Homo floresiensis keeping modern humans off the island. There must have been encounters between them and us. It is wonderful to speculate what might have happened when they met up, but I suspect that those moderns used up the resources that the hobbit needed to survive."

Robin McKie is the science editor of the Observer