White people in America—especially well-meaning white people—have a long history of calling the police when they suspect that black people are up to no good. Sometimes the responding officers merely harass or arrest the black folks in question—like the two men arrested while waiting for a business meeting at a Starbucks this year in Philadelphia or the napping Yale student who was confronted by police last month after falling asleep in her dormitory. But on other occasions when phones are deployed before facts—think John Crawford or Stephon Clark—African Americans end up dead.

This violent history weighs on me every time I take my sons out of our house—to a park, playground, swimming class, or doctor’s office. They’re white. I’m black. And in America, few things appear more suspicious than a dark man living with, laughing with, and loving white children.

My sons are beautiful, sweet, and perfect—I’d change absolutely nothing about them. But I never imagined they’d turn out white.

My sons are beautiful, sweet and perfect—I’d change absolutely nothing about them. But I never imagined they’d turn out white.

When my husband and I began to search for an egg donor to help us start a family a few years back, we immediately knew what we weren’t looking for—the blond-haired, blue-eyed donors so conspicuously desired by certain types of gay men eager for children who fit some sort of anachronistic “all-American” ideal. I’m mixed black and Jewish, my husband is an olive-toned Argentinian. We wanted a biological mother—or “bio-mom”—whose complexion and ethnicity would place her somewhere between us both. The donor we eventually chose seemed perfect—mixed Latina and Celtic, with skin the color of dulce de leche, piercing green eyes, and an endearing smile. She was healthy and smart and, unlike myself, remarkably athletic and slim!

While there were no guarantees her eggs would actually “work,” we figured any babies that resulted from this union would be lighter than me, darker than my husband—and certainly not Caucasian-appearing.

The donor’s eggs did work, so well, in fact, that we’re now parents to a pair of nearly 19-month old twin boys who are the lights and delights of everyone they encounter. They’re charming and chubby and affectionate and adorable and make me wish we were 10 years younger so we could give them a sister or brother.

They are also far whiter than we ever imagined. Aaron, born first, has a slightly ecru-colored complexion and beautiful auburn-colored hair that flows into loose curls reminiscent of a Greco-Roman statuary. Upon closer inspection, he’s clearly of ambiguous(ish) ethnicity—and can certainly look “of color” in the shade. But he’s very, very fair. Luca, meanwhile, wound up with milk-colored skin and piercing blue eyes—far lighter than my other half; he is, in a word, white.

I’ve spent my whole life at the intersections of ethnicity and identity and sexuality. Raised by my Ashkenazi Jewish mother without my black Baptist father—and with, perhaps, the most “Jewish”-sounding name imaginable—I’m far too accustomed to people prying into my racial background and family structure.

Well into adulthood folks would openly wonder “how I knew” the woman, my mom, sitting or standing or chatting next to me. And while I understood that my own family—what, with its two dads—would also invite intrusion and confusion, I hoped (if not prayed) that folks would never, ever question my inviolable status as their father.

So far, most haven’t—not really—but I know it’s merely a matter of time. In Manhattan, where we live, there’s nothing unusual about dark-skinned women toting white kids around town; they’re usually the nanny. Indeed, generations of white American children have been raised by black and brown women whose servitude—often forced and unpaid—kept them from their own families and children as they toiled away in the “big house.”

But no parallel examples exist for the opposite setup: Men like me, dark-skinned with light children. Which is why I’m so often afraid.

On the other extreme, there’s been a mini “boom” in white female celebs—think Sandra Bullock, Angelina Jolie, and Madonna—adopting African and African American children. Prominently featured in endless paparazzi shots, the resulting families have helped accustom many to the optics of the white-mom/black-child trope.

But few parallel examples exist for the opposite setup: Men like myself, dark-skinned with light children. Which is why I’m so often afraid.

America was built on the fear, loathing and labor of black men; we are the literal bogey men—our black lives often truly do not matter. At the root of this legacy is black male access to white privilege, property, and people—particularly white women and children. America’s ghastly love of lynching was steeped in fears of miscegenation while the pernicious “one drop” rule —which declared that any amount of African American blood rendered folks black—ensured that American families would maintain apartheid-like levels of segregation no matter their actual skin color.

While black women were “permitted” to raise white charges, social, cultural and institutional constructs did everything possible to keep black men from having any legitimate claim to white children. There was, literally, no real reason for intimate relationships between the two.

But where does that leave families like my own? I’m not entirely sure. For one thing, we are certainly not alone: Since 1970, the percentage of mixed-race children has spiked from 1% to 10% today, according to the Pew Research Center. And yet there are clear indications that the US isn’t moving forward in the fight against racism, but backwards.

While black women were “permitted” to raise white charges, social, cultural and institutional constructs did everything possible to keep Black men from having any legitimate claim to white children.

In my case, my boys are still too young for us to attract much notice—though I see people eyeing us in confusion almost every time we’re in public. I worry when they’re older and louder and—like most boys—fussier and disobedient. I worry…say…about the day on the subway when one—though likely both—refuses to sit in their seatsproperly or hold on to a safety rail. I fear the resulting discipline—direct, stern, and loving—might catch the eye of some well-meaning white person who could challenge my parentage, question my legitimacy and—entirely baffled—call the police. They cops might ask me to “prove” my parentage, like the white woman with a biracial son who was asked to confirm she was his mother as she tried to board a Southwest Airlines flight last month. Or worse.

I wish I lived in a world where this were mere hyperbole; I wish such fears were far-fetched and unfounded, But unlike my dreams of becoming a father, these wishes will probably never come true.

In the meantime, much like my mother before me, I go about the quotidian duties of being a parent—too sleep-deprived and diaper-laden, too absorbed in my sons’ sheer deliciousness—to allow myself to fully live in fear.

There have been moments—mostly benign, but occasionally cringe-worthy when our sense of normalcy has been disrupted. Last summer, when the boys were just infants, I was sitting in an ice cream shop slurping a scoop which Luca was eyeing greedily. The woman next to me—white and middle-aged and completely unremarkable—couldn’t take her eyes off of us. She seemed sad for my boy—all covetous and gelato-deprived. But he was just too young for a taste. Still, she clearly needed to get a word in, but was obviously too confounded by our relationship to know exactly what to say.

So she chose her words carefully, slowly and diplomatic. “Perhaps this nice man you’re with will buy you some ice cream,” she said looking Luca’s way and clearly baffled by my presence. “Of course I will,” I retorted, “but not until he’s older and I think he’s ready, because I’m his father.”

This article is part of Quartz Ideas, our home for bold arguments and big thinkers.