I’ll pause now to address the most common response when this issue comes up: “Just don’t wear makeup!”

It’s true that some women never wear makeup for various reasons. Some look better without it than others do. Some object on principle, or prefer to maintain a vaguely earthy-crunchy vibe. Others simply don’t have the time, can’t afford it, or have jobs that don’t involve interacting with others.

But for many of us, showing up at the office or a bar without at least a swipe of blush and some mascara results in a day spent being asked if we have the flu. Amy Schumer nailed this phenomenon in her perfectly titled sketch, “Girl, You Don’t Need Makeup.” Its takeaway: The “just free yourself from makeup!” crowd, particularly its male contingent, has no idea how makeup-wearers look after they wipe it all off.

Most women wear at least some makeup, some of the time. The polls around cosmetic use are notoriously bad—they’re often sponsored by beauty companies—but they’ve reported that between 50 and 80 percent of women use it at least occasionally. (According to another survey, though, two-thirds of women wear fewer than three products daily.) When University of New Hampshire student Ann Marie Britton surveyed 137 of her classmates for a thesis in 2012, at least half of respondents said they were “likely” or “very likely” to wear makeup to class, work, a job interview, to socialize, or on a date. “Mascara was used in almost all situations,” she found.

But more importantly, women on TV wear it. Many of our moms wore it, as did our elementary-school teachers. Magazines bombard girls with tips on “looking flawless.” That’s just how women look, in the collective mind’s eye: With unnaturally shiny lips and dark eyes.

For men, the closest analogy to being stuck without makeup, for women who usually wear it, is being forced to wear a stained shirt to a meeting. It’s probably fine to run errands in a shirt with dribble of barbecue sauce down the front. (There’s even a country song about it!) But if a man were to arrive at work for an important meeting, having somehow forgotten that his shirt was stained, and finding himself without an emergency clean shirt to don, he’d probably feel deeply uncomfortable. I feel roughly the same way about my five most essential tubes of face-goo.

Makeup, in short, is a norm, and nothing ruins a first impression like a norm violation. Some women contend they only wear makeup to “boost their confidence,” but the reason they feel less confident when they don’t wear it is that there’s an expectation they will.

Makeup works by enhancing facial contrast—the color difference between your lips and nose, for example. Facial contrast is closely associated with femininity, and femininity with female beauty, in Western cultures. In a study I reported on last year, both male and female participants thought “regular” women looked best when they applied a moderate amount of makeup. Another study found that subtle makeup made women seem more competent, likable, and attractive.