The Trump administration wants to change a rule aimed at forcing cities to actively combat housing segregation -- a move which worries fair housing advocates.

The Affirmatively Furthering Fair Housing rule, put in place by the Obama administration, was designed to help cities and counties identify the causes of housing segregation and discrimination and create a plan to improve to address those problems.

Fighting discrimination in housing is seen as one of HUD's biggest responsibilities under the Fair Housing Act since where someone lives often determines access to jobs, education, and resources.

Under the AFFH rule, public housing agencies in dozens of cities have conducted studies and gathered public input on how to improve housing equality in their community. New Orleans' plan, for example, proposed expanding affordable housing options in areas with lots of economic opportunities and investing in public transit, schools, and parks in underserved communities.

In Philadelphia, officials identified challenges to housing equality that included gaps in outreach to non-English speakers and outlined plans for improving access and awareness to affordable housing.

Debby Goldberg, vice president of housing and special projects at the National Fair Housing Alliance, said the country is desperately in need of affordable housing and that her organization completely disagrees with HUD's assertion that the rule needs to be changed but thinks a lot of progress is being made.

"The rule is very clear that there are two kinds of strategies that we need, given our history of segregation in this country, which the government played a very significant role in creating. We have to do two things, one is we have to expand the housing choices available to people so that where you live isn't limited by the color of your skin, your national origin, whether you have kids or whether you have a disability. And we have to look at places that have been segregated and disinvested for a long time and figure out ways to make sure that people who live in those neighborhoods have access to all the community resources and amenities that people elsewhere have," she told ABC News.

However, the Department of Housing and Urban Development announced this week that it plans to change the rule to reduce some of the requirements on local governments.

Goldberg said housing advocates are worried HUD is trying to eliminate the AFFH rule and go back to the system that was in place before 2015, which both HUD and a federal government watchdog found were insufficient.

HUD didn't release specific details on how it will change the rule but said that, among other things, it wants to increase housing options in low-income communities.

HUD spokesman Brian Sullivan said they have heard from some officials that the Obama-era rule was confusing and that some cities and counties were worried they could be pushed to put low-income housing in wealthier areas instead of investing in distressed neighborhoods.

"We certainly don't want a regulation to discourage investment in places that need it most. We were getting a lot of comments about mostly not understanding, finding it difficult to use, a lot of functionality complaints. If nothing else I think one of the reasons for all of this is to make it clear to people and to reduce the burdens when those burdens become excessive," Sullivan told ABC News.

Secretary Ben Carson said the rule "dictated unworkable requirements" and actually blocked development of new affordable housing.

“It’s ironic that the current AFFH rule, which was designed to expand affordable housing choices, is actually suffocating investment in some of our most distressed neighborhoods that need our investment the most,” Secretary Ben Carson said in a statement. “We do not have to abandon communities in need. Instead, we believe we can craft a new, fairer rule that creates choices for quality housing across all communities.”

Goldberg said HUD's argument is a complete mischaracterization of the rule and that the majority of jurisdictions were able to navigate the rule with minimal problems.

"I don't know where HUD comes up with the concept or the notion that this prevents investment in certain neighborhoods and only promotes housing mobility or new housing only in high-income neighborhoods," Goldberg told ABC News.

HUD didn't release specific details of how it will change that rule this week but announced that it is beginning the process of changing it to focus on expanding development and reducing regulations on local governments. HUD previously delayed deadlines for cities to submit plans on how they would combat housing discrimination required by the rule, prompting a lawsuit from several advocacy groups.

Secretary Ben Carson told the Wall Street Journal the rule could tie HUD grants to more relaxed zoning regulations as an incentive for cities to allow more development.

At the time, Carson compared the rule to busing policies and a socialist experiment in an op-ed during the presidential campaign, arguing that the rule would force cities to build housing targeted at minority or low-income populations in wealthier areas.

In cities like New York, average rent has increased and income has stagnated, leading to fewer housing options for people who are priced out of increasingly expensive neighborhoods. The New York City Department of Housing Preservation and Development says on its website there are more than 970,000 low-income households for less than 450,000 available low-income homes for rent.

Carson's op-ed referenced New York as an example of issues created by the AFFH rule that cities were concerned it would block investment in low-income neighborhoods, but the department told ABC News that was not the case at all. A spokeswoman said Mayor Bill de Blasio's affordable housing plan that Carson mentioned has actually exceeded goals and on track to fund 300,000 affordable homes, and that despite concern the rule would prevent construction it actually provided more guidance for the city's efforts.

"HUD’s rule was intended to confront segregated living patterns and housing discrimination that continue to persist across our country fifty years after the passage of the Fair Housing Act, disparities that will not be wiped away simply by lessening land use regulations and increasing supply. While New York City will continue to forge ahead with its process and invest significant resources in pursuing fair housing, we need strong leadership at the federal level to push all localities to confront the legacy of structural segregation," the department said in a statement to ABC News.