Is beauty really in the eye of the beholder? A new study says yes, discovering that 50% of people’s preferences for faces is unique to them, and who we find attractive is most strongly influenced by our life experiences.

“If you were to rate faces [for attractiveness] and I were to rate the same faces, we would agree about 50% of the time,” says study author Jeremy Wilmer, an assistant professor of psychology at Wellesley College whose new research was published Thursday in the journal Current Biology.

The two-part study first had around 35,000 people rate faces for attractiveness on the website Testmybrain.org. Prior research has suggested that people generally agree that symmetrical faces are better-looking. That, and other factors, may account for the 50% consensus on beauty.

As for the other 50% of faces? That’s where the science really gets interesting.

In the second part of the study, the researchers studied the facial preferences of 547 pairs of identical twins and 214 pairs of fraternal twins. The hope was that by studying people who are genetically the same (or at least very similar) and who grew up and in the same environment, they would gain some insight into the nature or nurture effects of where the “eye of the beholder” variation in attractiveness perception comes from.

If something is really influenced by genes you would expect identical twins to be more similar to each other than the fraternal twins, Wilmer says. On the other hand, if family environment is highly influential, the researchers would expect fraternal twins to be quite similar to each other in preferences. But that’s not what they found.

“In our case, we found that even though identical twins share all of their genes and their family environment they were really, really different from each other in their facial aesthetic preferences,” says Wilmer.

It’s likely personal experiences inform who we find attractive, and because they are so personal, we can greatly differ from someone who—on paper—is quite similar to us, the research suggests.

So what types of experiences influence who we find attractive? The researchers didn’t study this in their paper, but study author Laura Germine of Massachusetts General Hospital says that based on previous literature on this subject, they have some theories.

“[Prior research] has found things like, if you take a face and you pair it with positive information, that face then looks more attractive, and faces that are similar to it also look more attractive and vice versa,” she says. “So you can imagine as you go through life and you form relationships and have friends and people you have a more positive relationship with, you may come to find their face characteristics more attractive, and then other people who look similar to them are then more attractive to you.”

“Exposure to certain faces makes them seem more attractive,” she adds. That means a face that is very different from a face that you have never seen before tends to be judged as less attractive. It also means the kinds of faces you are exposed to in your work environment, in your relationships or even the face of your spouse could shift the kind of faces that you find attractive, she says.

Basically, who you find attractive may be less about where you grew up and where you went to school and more influenced by experiences that are very unique to you. More research is still needed, but the study suggests beauty really may be in the eye of the beholder.

Get our Health Newsletter. Sign up to receive the latest health and science news, plus answers to wellness questions and expert tips. Please enter a valid email address. Sign Up Now Check the box if you do not wish to receive promotional offers via email from TIME. You can unsubscribe at any time. By signing up you are agreeing to our Terms of Use and Privacy Policy . This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply. Thank you! For your security, we've sent a confirmation email to the address you entered. Click the link to confirm your subscription and begin receiving our newsletters. If you don't get the confirmation within 10 minutes, please check your spam folder.

Contact us at letters@time.com.