Fragmentary fossils found in southwestern Texas 3 decades ago belong to a strange group of extinct animals known as “bear dogs,” according to a new study. Though only about the size of a Chihuahua when they first appeared, some creatures in this group of carnivorous mammals evolved to become top predators in their ecosystems tens of millions of years ago. The study also suggests that bear dogs could have originated in this part of North America, which may have been a hot spot of evolution for the group.

Bear dogs, scientifically known as amphicyonids, get their common name from their general resemblance to modern-day bears and dogs, especially in their body shape and posture, but they are, in fact, only distantly related to these lineages. Neither dogs nor bears had evolved when amphicyonids first appeared about 40 million years ago, says Susumu Tomiya, a vertebrate paleontologist at The Field Museum of Natural History in Chicago, Illinois. The first known species in the group weighed just a few kilograms, but over millions of years the lineage expanded to include fox-sized, coyote-sized, and even bear-sized creatures, all of them meat eaters. It’s not clear where or when amphicyonids evolved, but they apparently lived in North America, Asia, and Europe, Tomiya says.

While walking through The Field Museum’s collections one day, Tomiya spotted a fossil of a small carnivore that he thought might be an unrecognized amphicyonid. So he and vertebrate paleontologist Jack Tseng of the State University of New York at Buffalo took a closer look at the specimen, plus a similar one that had been unearthed in the same area of southwestern Texas, about 300 kilometers southeast of El Paso. Those fossils, first described in 1986, are about 37 million or 38 million years old. One of the creatures, known from only an 8-centimeter-long skull with a few teeth missing, probably weighed about 2.3 kilograms (5 pounds) and was about the size of a Chihuahua. The other animal, based on the size of its skull, was slightly larger and probably the size of an average house cat, Tomiya says.

Earlier analyses of these fossils only considered their external features, and the remains were so fragmentary that paleontologists could only say the creatures were carnivores of some sort. But Tomiya and Tseng’s new study used x-rays to generate detailed 3D scans of the fossils’ internal features. Those scans revealed a distinct pattern of blood vessel channels in the base of the skull that identified the creatures as amphicyonids, making them among the oldest known members of the lineage, the researchers report online today in Royal Society Open Science . Both creatures have been renamed to reflect their revamped spot on the mammalian family tree, Tomiya says. The researchers placed the smaller amphicyonid in a new genus, Gustafsonia, (honoring a paleontologist who studied fossils from this area). They put the cat-sized bear dog into the new genus Angelarctocyon, which, translated from the Greek, means “messenger bear dog.”

The new research is “an elegant study” that uses tools not available to previous generations of paleontologists, says Zhe-Xi Luo, a vertebrate paleontologist at the University of Chicago. The Texas creatures “are now recognized as previously lost cousins” to the amphicyonids, he notes, adding immensely to the diversity of bear dogs known from southern North America at the time.

Fossils of another early amphicyonid have been unearthed from slightly older rocks in Europe. But the new fossils make clear that southern North America was a hot spot of evolution for these creatures, with half a dozen species early in their history, says Xiaoming Wang, a vertebrate paleontologist at the Natural History Museum of Los Angeles County in California.

It’s not clear why the amphicyonid lineage eventually died out a few million years ago, Luo says. But maybe it had to do with competition from the ancestors and close cousins of today’s cats and dogs, he suggests. Those creatures, like their modern-day kin, walked on their toes and were more well adapted to run and chase prey. But amphicyonids were, for the most part, flat-footed predators like most modern bears. During the last days of the bear dog reign on Earth, the planet’s climate was becoming cooler and drier and many ecosystems were becoming less forested and more open—not a good trend for relatively slow, relatively specialized meat eaters like the amphicyonids, Luo says.