Last fall, San Francisco passed a bill designed to give the city’s dwindling black population a better shot in the lottery for subsidized housing. The legislation reserved 40 percent of all new subsidized housing for people already living within a half-mile of the project or within its municipal district.

On Wednesday, the San Francisco Chronicle reported that the U.S. Department of Housing and Urban Development had rejected that plan for a senior housing development in the Western Addition neighborhood, writing to the city that it could “limit equal access to housing and perpetuate segregation,” in violation of the Fair Housing Act.

It’s a warning that could resonate beyond projects that, like that San Francisco senior center, have been financed directly by HUD. The department has long supported the use of so-called local preferences in public housing, based on “local housing needs and priorities.” But the letter seems to indicate that Washington may be changing its position as part of an aggressive push toward integration in housing. While Fair Housing Act complaints are decided in the courts, HUD has an enforcement arm that investigates complaints and files charges.

Giving priority to locals can protect poor residents in gentrifying neighborhoods, serve the voters who approve the projects, and provide an incentive for politicians to accept new development. But it also perpetuates segregation, by ensuring that the racial composition of a neighborhood or town is maintained in new buildings.

Plenty of cities use addresses (among other characteristics) to determine who gets priority on lists and in lotteries for affordable housing. In April, for example, officials in Glendale, California, near Los Angeles, argued that a lottery for affordable artist housing should favor artists currently living in Glendale more than those who merely worked there.

In New York City, where affordable housing lotteries get about 850 applicants for every open apartment, fully half of all units in subsidized buildings are reserved for applicants from the neighborhood. Supporters (in both New York and San Francisco) argue that community preference quotas allow low-income tenants threatened by rising rents to remain in (and reap the benefits of) gentrifying neighborhoods. Local preference also engenders neighborhood stability, by allowing parents to keep their kids in the same schools and residents to maintain social networks. It’s not just housing officials who feel that way. An October survey of the South Bronx found that 94 percent of residents felt that at least half of new housing should be set aside for local residents.

If asked, though, some of those South Bronx applicants might want better odds of living in Inwood, Harlem, or the Upper West Side. Opponents of community preference say it essentially exacerbates patterns of segregation by maintaining the racial makeup of New York’s highly segregated neighborhoods. If that’s what HUD thinks about San Francisco, it’s hard to imagine the department could think otherwise of New York, whose preference policy it reviewed in February 2015.

New York City is currently being sued by a trio of black New Yorkers who applied via lottery for affordable apartments in three majority-white Manhattan neighborhoods. The complaint, which is in court now, alleges that as a result of community preference, “entrenched segregation is actively perpetuated, and access to the neighborhoods in this city with high quality schools, health care access, and employment opportunities; well-maintained parks and other amenities; and relatively low crime rates is effectively prioritized for white residents who already live there and limited for African-American and Latinos.” The suit could benefit from a 2015 Supreme Court ruling that expanded the reach of the Fair Housing Act.

It wouldn’t be the first time a local preference ordinance was found to violate the Fair Housing Act. In 1980, the U.S. Department of Justice successfully demonstrated that Chikasaw, Alabama, was using a public housing residency requirement to ensure an all-white population despite the large black population in surrounding Mobile County.

The history of the local preference runs in broader currents of housing policy racism. In the 1990s, officials in Harrison, a town in Westchester County, New York, abandoned an affordable housing project when they learned they might not be able to reserve units for local residents. (The town was 1.5 percent black at the time, while the county was 25 percent minority.)

New York City’s neighborhood preference program for public housing was used to keep white projects white, even as the waiting list grew to be dominated by minority applicants, a 1983 HUD review found. “It was all part of an operation to get people to accept the housing,” former mayor Ed Koch said of the local preference program. “Nobody wants low-income housing next to them, or very few do.”

Koch was right about one thing: Community preference continues to be used as a carrot to convince neighborhoods to accept new development. (This is especially true since so much new affordable housing in New York comes with market-rate development—which, in this city of tenants, is much more openly loathed than affordable housing.) This week, reserving 50 percent of a project for affordable housing (an imperfect designation, I know), and 50 percent of those units for local residents, wasn’t enough to ward off the anti-gentrification NIMBY brigade during a confrontation over rezoning in Manhattan’s Inwood neighborhood.

Without the assurance of local benefit those buildings become a very hard sell. The dispute illustrates an unpleasant reality of the affordability equation: Requiring affordable housing be better-located and more diverse might mean getting less of it.