Rare, 7-million-year-old fossils of two extinct pygmy sperm whales are helping researchers learn about the evolution of the ocean's largest toothed whale, a new study finds.

An analysis of the fossilized skulls indicates that even though they were pygmies, the newly discovered species actually had larger spermaceti, an organ that sits on top of the head and is involved in sound production and echolocation (finding an object's location via sound), than their modern-day relatives.

It's unclear why the sperm whales' spermaceti organ shrank over time — twice in the evolutionary record, according to an analysis of several fossils — but perhaps at one time, larger spermaceti were used to attract mates, said the study's lead researcher, Jorge Velez-Juarbe, a curator of marine mammals at the Natural History Museum of Los Angeles County. [Whale Album: Giants of the Deep]

"We really need to test this hypothesis," Velez-Juarbe told Live Science. "We need to find more complete fossils."

Another idea is that the enlarged spermaceti organs helped these prehistoric whales find prey, "because they used echolocation to hunt," said Nicholas Pyenson, a curator of fossil marine mammals at the National Museum of Natural History in Washington, D.C., who wasn't involved with the study.

Caribbean fossils

The researchers found the pygmy sperm whale fossils buried in a sea cliff along the Caribbean coast of Panama in 2012 and 2013. After determining that the two unique individuals were a new species, the researchers named them Nanokogia isthmia; "Nano" from the Latin "nanus," which means dwarf, and "kogia" for the whale's genus, "Kogiid." The species name comes from the Isthmus of Panama, the strip of land where the fossils were found.

The findings have electrified the world of marine paleontology: many whales, dolphins and porpoises have comprehensive fossil records that help scientists study their evolution. But only a few fossils of sperm whales and their elusive living relatives, the pygmy and dwarf sperm whales, have been recorded, the researchers said.

An evolutionary tree shows the relationships between extinct and living sperm whales, and when the spermaceti, an organ used in sound production and echolocation, shrank over time. (Image credit: Natural History Museum of Los Angeles County)

"It's exciting to know more about the evolution of this enigmatic group," Pyenson said. "An entire description of a new fossil pygmy sperm whale — that's really valuable."

N. isthmia was small, measuring about 6.6 feet (2 meters) long. In fact, it's smaller than modern-day pygmy and dwarf sperm whales, which measure between 8 feet and 12 feet (2.4 m and 3.7 m) long, Velez-Juarbe said. The behemoth of the family, the sperm whale (Physeter macrocephalus), made famous by Herman Melville's "Moby Dick" (1851), can reach about 52 feet (15.8 m) in length. Female sperm whales are typically even larger, measuring about 36 feet (11 m) long.

After analyzing the two N. isthmia skulls, the researchers found that they likely sported large spermaceti organs. The organs are made of fatty tissue, muscles and air sacs, and don't last in the fossil record. But the bone that attached them to the skull remains, allowing scientists to measure it and use it as a proxy for spermaceti size.

Other features on the ancient skulls could provide clues about Kogiid evolution. For instance, "The two living species of pygmy sperm whales have among the most bizarre skulls of any mammal," but it's unclear "how their unusual skull shape evolved," said Jonathan Geisler, an associate professor of anatomy at the New York Institute of Technology, who was not involved in the study. [The 12 Weirdest Animal Discoveries]

Modern pygmy sperm whales lack nasal bones, and the right side of the skull is different from the left side, meaning it's asymmetric.

"The new fossil species these authors describe show a transitional state between other fossil Kogiids and the two living species," Geisler told Live Science in an email. "Its skull is like those of the living species except that it has a longer snout that is not downturned."

However, since modern Kogiids are similar to one another, it's likely that there are more fossil species yet to be discovered, he added.

"New insights into the evolution of this group will primarily come from the fossil record, not additional study of the living species," Geisler said.

The findings were published online today (April 29) in the journal PLOS ONE.

Follow Laura Geggel on Twitter @LauraGeggel. Follow Live Science @livescience, Facebook & Google+. Original article on Live Science.