When Westerners—including second-generation immigrants like myself—talk about cultural appropriation, it’s almost always as a clean distinction between races. Katy Perry wearing a “kimono” to sing a song about unconditional love ? Brings up orientalist geisha fantasies . Justin Bieber and Miley Cyrus wearing dreads and mainstreaming twerking ? “Don’t cash crop on my cornrows.” Kylie Jenner being included in a roundup of Latina celebrities ? For that matter, the Kardashian-Jenner family’s seeming quest to be fully assimilated into cultural blackness? Where do you start?

Though it’s not only American public figures being called out for cultural insensitivity, American-centric conversations around appropriation are almost always uniformly negative, and almost always centered on whiteness. These kinds of critiques are valuable, but tend to miss another part of the discussion: the larger, less binary relationships between histories, peoples, and cultures around the world, and how the concept of appropriation changes within them.

A few months ago, I visited Japan during the Shichi-Go-San festival , in which families and in particular specific young children (three- and seven-year-old girls, five-year-old boys) visit shrines and temples while wearing traditional Japanese clothing. While most Japanese people I encountered during my week in the country wore street and business, there were a decent amount of people wearing traditional clothing like kimonos and tabi/geta at various shrines and temples. At Kyoto’s Ginkakuji, I noticed a white couple posing with an East Asian woman dressed in a kimono—but after taking the photo, she joined her companions (also dressed in traditional Japanese clothing) and they all started speaking Chinese to each other. My parents, themselves Chinese immigrants, wouldn’t look at them as we walked past; my mother muttered “Disgusting” in English under her breath.

Katy Perry, via Getty

For those unaware of East Asian politics, the relationships between China, Japan, South Korea, North Korea, and Taiwan are, in a word, complicated , with national and ethnic conflicts (and cultural exchanges) going back centuries between the various nation-states and communities contained within “East Asia.” If you try to speak to most locals about actual East Asian unity, you’ll find yourself in a landmine of national, ethnic, and religious group conflicts; to my parents, a Chinese person not only wearing a traditional Japanese garment but actively passing (and accepting that passing) as Japanese was a personal affront. When I try to explain this situation to non-East Asian peers, their first reaction is, “What?”

The subtlety of intraracial passing (intraracial, or “within race,” distinctions in general) is usually lost on white Westerners, both through ignorance and on purpose. (Though there are exceptions, particularly when other factors like religion create divides in an otherwise homogeneous population.) In America, I grew up hearing that all Asian people “look the same,” particularly East Asian people with pale (though characterized as yellow) skin and slanted eyes. In grade school, I was regularly addressed by another East Asian or East Asian-American girl’s name in class. In adulthood, I live in Los Angeles and its multitude of ethnic enclaves; when I walk around Koreatown or Little Tokyo, it’s not uncommon for strangers to fling broken Chinese/Japanese/Korean at me, or even have older immigrants ask me questions in their native languages.

While it’s one thing to be mistaken for another “kind” of Asian by obtuse strangers or confused elders, it is something else to be told by your family that you don’t look or act like them. My dad’s grandfather was adopted in Manchuria, a region in Northeast China that has (compared to the rest of eastern China) significant Mongolian and Korean minority populations. From various members of my extended family, I’ve been told that I look Japanese or Korean and act accordingly (especially when I was an anime-worshipping teenager)—never meant as a compliment. Other examples of historically and politically fraught intraracial conflicts: Pakistan and India, Catalonia and Spain, Taiwan and China. For conflicts like those, distinction matters and can have deadly consequences.

Still, it’s not an accident that in almost every instance of celebrity call-out in regards to cultural appropriation, the appropriator is white. For people of color (POC), it’s easy to understand white-on-other appropriation; it’s modern colonization, a matter of punching down at more historically and actually marginalized groups. This is why, generally speaking, it’s impossible for POC to appropriate whiteness. As ethnographer and writer Christina Xu , who collects essays and media about creative young people in China for her project Multi Entry , explained in an interview with Complex, “Is it appropriation or respectability drag [when a POC adopts an aspect of another culture]? Looking at the big picture, it's usually the latter. Most of the world has been traumatized in some form or another by European & American imperialism.” That this oftentimes comes off to people, white or otherwise, as cultural “policing”—even as we live in a world characterized by global cultural exchange—is a point of frustration. But the question isn’t necessarily if someone can wear or say or do a certain thing, but whether that choice disenfranchises other populations who do and live those things as part of their lifestyle, and are penalized by a (white) majority for doing so.

Eddie Huang, via Getty

And what happens when appropriation is coming from another person of color (like in the case of Chinese-Taiwanese-American chef Eddie Huang and his open appropriation of blackness ), or even from within your racial group? Or, when what Westerners consider appropriation is not only accepted, but encouraged? All around the Japanese temples I visited, businesses offered kimono rentals specifically to benefit visiting tourists, Western or otherwise; I’ve seen the same thing happen at Chinese landmarks like the Forbidden City with Chinese imperial court clothes and costumes. This practice is not so dissimilar from what Boston’s Museum of Fine Arts wanted to do with its “Kimono Wednesdays.” Yet on a gut level, I feel okay about the former and uncertain about the latter.

As Xu explains it, the conversation about cultural appropriation is largely defined by the nature of the histories between the approrpiator and the appropriatee. In the case of the U.S., “[It] is a place where LOTS of different immigrant groups coexist, with a legacy of European colonial genocide and slavery and occasionally xenophobic oppression.”

When this conversation is carried over to someplace like East Asia, “Where do you even draw the bounds when there's so much inter-ethnic blending and migration and HISTORY that's not as one-sided as the story in the U.S.? Is Chinese people making/repping ramen ‘appropriative’ of Japanese culture, even though ramen is a descendant of Chinese noodle soups? Personally, I don't really want to live in world where we're trading thinkpieces about that,” Xu says. She stresses that fair cultural exchange is entirely possible, but that it’s the power dynamics behind that exchange that dictate appropriation versus appreciation.

Is someone playing at an identity in a way that makes it inaccessible for the people who actually have it?

And when it comes to Westerners and POC tourists dressing up in garb traditional to the place they’re visiting? Xu cites academic Lisa Nakamura’s theory of “identity tourism”: “Is someone playing at an identity in a way that makes it inaccessible for the people who actually have it? All that is to say: yes, of course there's intraracial appropriation. But it means something really different.”

Naturally, the conversation about cultural appropriation becomes more complicated when you can visually pass into a population, but otherwise cannot. It leads to a flattening of diversity: this is how, for example, mainland Chinese actresses can be cast as Japanese in a Hollywood film , or how any East Asian (or black, or Latino, or otherwise POC) person is subject to a cornucopia of slurs and stereotypes, from the specifically incorrect (like making a Communism joke to a Japanese person) to the broad generalizations (yelling “dog eater” at anyone who appears East Asian). Or, in the case of the Chinese woman wearing a kimono, you can tacitly pass into a population. Xu herself experienced this during a research trip to China; her Chinese peers refused to believe that she was foreign, let alone from America. Her solution was, strangely enough, to adopt a Taiwanese accent, as a way of signaling her outsider status in a way that matched her East Asian face.

The opposite, being treated as extra-foreign while traveling in countries where you can’t pass into the dominant demographic, is also something that happens more often than not to “minority” populations abroad than to white/white-passing Westerners, even those who travel in Europe. I’ve heard “Ni hao!” and “Konnichiwa!” in Paris, London, and Cancún, and felt a spotlight forced onto me; I can’t imagine that the average vacationing white American girl is ever addressed in French or German or Italian, or would feel anything other than thrill at “passing” for those populations.