Part of that is because our faces are asymmetrical. The left and right side of your face may not seem that different, but as photographer Julian Wolkenstein illustrates with his portraits, which duplicate each side of a face to create strikingly different versions of the same person, that's not the case. When what we see in the mirror is flipped, it looks alarming because we’re seeing rearranged halves of what are two very different faces. Your features don’t line up, curve, or tilt the way you’re used to viewing them. (An episode of the Radiolab podcast, about symmetry, demonstrated this when it flipped a popular photo of Abraham Lincoln. The asymmetry can be surprising even when looking at images of faces we’re very familiar with, not just our own.)

“We see ourselves in the mirror all the time—you brush your teeth, you shave, you put on makeup,” says Pamela Rutledge, director of the Media Psychology Center. “Looking at yourself in the mirror becomes a firm impression. You have that familiarity. Familiarity breeds liking. You’ve established a preference for that look of your face.”

That’s not just an anecdotal observation, that’s science. According to the mere-exposure hypothesis, people prefer what they see and encounter most often. In terms of self-perception, this means that people prefer their mirror images to their true images, which are what other people see. Experiments conducted at the University of Wisconsin, Madison in 1977 support this idea: When presented with photos of their true image and their mirror image, participants preferred their mirror image while friends and romantic partners preferred their true image. When asked to explain their preference, participants pointed out camera angles, lighting, head tilt and other differences that didn’t actually exist because the photos were made from the same negative. (According to the founders of True Mirror, which reflects back one’s true image by angling standard mirrors at right angles, only 10 percent of people prefer their real image to their mirror image.)

“The interesting thing is that people don’t really know what they look like,” says Nicholas Epley, a professor of behavioral science at the University of Chicago Booth School of Business and the author of Mindwise: How We Understand What Others Think, Believe, Feel, and Want. “The image you have of yourself in your mind is not quite the same as what actually exists.”

The image in our minds, according to Epley’s research, is way prettier. In a study published in a 2008 issue of the Personality and Social Psychology Bulletin, researchers made participants’ faces more or less attractive in 10 percent increments by morphing their features to resemble composites of conventionally beautiful people (or, for the unattractive versions, people with craniofacial syndrome). When asked to identify their face out of a line-up, participants selected the attractive versions of their faces more quickly, and they were most likely to identify the faces made 20 percent more attractive as their own. When asked to pick the experimenters’ faces out of the line-up, the participants showed no preference for more attractive versions of relative strangers.