As the summer officially begins, with the Summer Solstice occurring in the Northern Hemisphere on Thursday, those who enjoy Western astrology will be checking out their Summer Solstice horoscopes to try to use the stars to figure out what the season might have in store.

While some horoscopes sites may promise predictions based on the “movement” of the stars, it’s important to remember that it’s the Earth that’s moving, not the stars. The reason why stars look like they’re moving, both throughout the night and over the course of the year, is because the Earth rotates on its axis and orbits around the Sun. But, before most humans knew that, they spent a lot of time thinking about what was happening up there in the sky.

So, though astrology — looking for answers, signs and predictions in the movements of the celestial bodies —isn’t itself a science, there’s a long history of humans looking up at the stars to plan their lives. Farmers used the skies as a calendar as long ago as Ancient Egyptians, when the rising of Sirius, the Dog Star, around mid-July, was seen as a marker of the imminent annual flooding of the Nile. Travelers used the skies as a compass, following the stars to know where to go. And many people used the skies as a source of mystical direction, too.

But who first looked up at the sky to make sense of what was happening down on the ground and why their fellow humans were behaving in certain ways? Exactly who came up with this way of thinking and when is unclear, but historians and astronomers do know a bit about how it got so popular today.

Where did zodiac signs come from?

The stars are just one of the many things in the natural world that human beings have turned to for answers over the years.

“We don’t really know who first came up with the idea for looking at things in nature and divining influences on humans,” says astronomer Sten Odenwald, the director of Citizen Science at the NASA Space Science Education Consortium. “There’s some indication that cave art shows this idea that animals and things can be imbued with some kind of spirit form that then has an influence on you, and if you appease that spirit form, then you will have a successful hunt. That was taken over by the idea of divination, where you can actually look at things in nature and study them carefully, such as tea-leaf reading.”

Some form of astrology shows up in various belief systems in ancient cultures.

In Ancient China, noblemen looked at eclipses or sunspots as portents of good or bad times for their emperor, though it’s thought that those signs had less application to the lives of other individuals. (Odenwald points out that in societies where people in the lower classes had less control over their lives, divination could seem pointless.) The Sumarians and Babylonians, by around the middle of the second millennium BC, appeared to have had many divination practices — they looked at spots on the liver and the entrails of animals, for example — and their idea that watching planets and stars was a way to keep track of where gods were in the sky can be traced to The Venus tablet of Ammisaduqa. This tablet, which is dated to the first millennium BC and tracks the motion of Venus, is one of the earliest pieces of what’s been called Babylonian planetary omens. The ancient Egyptians contributed the idea that patterns of stars made up constellations, through which the sun appears to “move” at a specific times during the year.

It’s thought that all of these ideas came together when Alexander the Great conquered Egypt around 330 BC.

“There must have been a lot of exchange that got the Greeks on-board with the idea of divination using planets,” says Odenwald, and because they were deep into mathematics and logic, they worked out a lot of the rules for how this could work.”

Here’s how NASA has described how that logic led to the creation of the familiar zodiac signs known today:

Imagine a straight line drawn from Earth through the Sun and out into space way beyond our solar system where the stars are. Then, picture Earth following its orbit around the Sun. This imaginary line would rotate, pointing to different stars throughout one complete trip around the Sun — or, one year. All the stars that lie close to the imaginary flat disk swept out by this imaginary line are said to be in the zodiac. The constellations in the zodiac are simply the constellations that this imaginary straight line points to in its year-long journey.

What are the 12 signs of the zodiac?

It was during this Ancient Greek period that the 12 star signs of the zodiac with which many people are likely familiar today — Aries (roughly March 21-April 19), Taurus (April 20-May 20), Gemini (May 21-June 20), Cancer (June 21-July 22), Leo (July 23-Aug. 22), Virgo (Aug. 23-Sept. 22), Libra (Sept. 23-Oct. 22), Scorpio (Oct. 23-Nov. 21), Sagittarius (Nov. 22-Dec. 21), Capricorn (Dec. 22-Jan. 19), Aquarius (Jan. 20 to Feb. 18) and Pisces (Feb. 19 to March 20) — were set down. These Western, or tropical, zodiac signs were named after constellations and matched with dates based on the apparent relationship between their placement in the sky and the sun.

The Babylonians had already divided the zodiac into 12 equal signs by 1500 BC — boasting similar constellation names to the ones familiar today, such as The Great Twins, The Lion, The Scales — and these were later incorporated into Greek divination. The astronomer Ptolemy, author of the Tetrabiblos, which became a core book in the history of Western astrology, helped popularize these 12 signs.

“This whole idea that there were 12 signs along the zodiac that were 30° wide, and [that] the sun moved through these signs regularly during the year, that was codified by Ptolemy,” says Odenwald. Even the word “zodiac” comes from the Greek, from a term for “sculpted animal figure,” according to the Oxford English Dictionary, and the order in which the signs are usually listed comes from that period too.

“Back at the time of the Greeks,” Odenwald explains, “the first day of spring started when the sun appeared in the constellation Aries and then everything was marked from that time forward around the circuit of the year.”

However, the Earth has moved on its axis since then, a process known as precession, so now the dates that are used to mark the signs don’t really correspond to the background constellations that give them their signs names. In fact, the chronology has really shifted one sign to the West. That means zodiac sign dates, based on the mathematical division of the year, basically correspond today to the presence of the sun in the constellations of the signs that come before them. (The set nature of the signs is also why the Minnesota Planetarium Society’s 2011 argument that there should be a 13th zodiac sign now, Ophiuchus, didn’t actually result in a big astrology change.)

“Before, astrologers looked at where the sun was relative to background constellations in general, and that generally matched up almost exactly with the signs of zodiac defined by Ptolemy,” says Odenwald. “Now astrologers do their calculations and forecasting based on where the planets and the sun are relative to the 12 signs —which are fixed — and not based on where they are relative to the constellations. Astrologers say if the sun is in the sign of Sagittarius on the day you were born, then you’re a Sagittarius.”

Read More: Mercury Is Entering Retrograde Again. This Is Why So Many People Care

What’s the difference between astrology and astronomy?

For centuries, astrology (looking for signs based on the movement of the celestial bodies) was considered basically the same thing as astronomy (the scientific study of those objects). For example, revolutionary 17th-century astronomer Johannes Kepler, who studied the motion of the planets, was at the time considered an astrologer. That changed around the beginning of the Enlightenment in the late 17th century.

Once Sir Isaac Newton basically turned the sky into a calculator, mathematizing the motion of the planets and realizing that gravity controlled everything, Odenwald says, “that started a whole new scientific approach to looking at the sky and the motion of planets and the earth.”

That’s the point at which astronomy came to be known as a science and astrology was acknowledged as not a science. But its popularity relies on factors that numbers can’t compute, and the appeal of looking to the stars for answers has not waned — in fact, in recent years, it seems to have expanded. After all, a 2014 National Science Foundation poll found more than half of millennials think astrology is a science.

And Odenwald argues that, even if astrology’s answers aren’t based on scientific study, the reason people keep turning to the sky does come down to something very real — a psychological phenomenon he calls the human tendency for “self-selection,” the search for interpretations that match what we already hope to be true.

“People magnify the positives, they forget the negatives,” he says, “and that’s just how we’re designed.”

Write to Olivia B. Waxman at olivia.waxman@time.com.