On Thursday Dr. Ben Carson, President-elect Donald Trump’s pick to be the next secretary of the U.S. Department of Housing and Urban Development, will have his Senate confirmation hearing. His primary qualifications, if they count, for the post include growing up with modest means (but not in public housing) in Detroit and the fact that he served primarily urban populations while chief of pediatric neurosurgery at Johns Hopkins Hospital in downtown Baltimore. Carson has never served in any post at any level of government, unlike Julian Castro, the former mayor of San Antonio and current HUD secretary. Nor does Carson have a notable track record of policymaking in the complex world of affordable housing, like Castro’s HUD predecessor and the former commissioner of New York City’s Department of Housing Preservation and Development, Shaun Donovan.

Carson’s lack of expertise in housing issues is a mockery of HUD and its far-reaching urban policy efforts—but for HUD, it’s also nothing new. Since the agency’s inception, the government has charged HUD with carrying out a litany of important tasks with very few resources. Carson’s appointment is unsettling, but it merely marks the latest chapter of a long history of limiting HUD’s capacity to tackle urban inequality and provide housing assistance to low-income residents. From Nixon to Clinton, Republican and Democrat administrations alike have woefully underfunded an agency that remains one of the few federal efforts that focuses on the daunting task of deconcentrating poverty.

Considering the clearly racialized undertones of how Trump framed the “inner city” throughout the presidential campaign, it is important to understand the historical and political context of the agency that Carson is inheriting. HUD emerged out of growing urban renewal efforts, a mid-20th century federal initiative to upgrade “slum” areas through the creation of urban infrastructure, most often by razing minority communities. In more recent years, HUD’s role in regulating federal mortgage policies are said to have helped propel the housing crisis, which disproportionately impacted black Americans. But despite its troubled history, HUD has also provided crucial assistance to low-income residents.

Since the department’s founding in 1965, HUD’s secretaries have led valiant efforts to provide affordable housing and rental assistance to low-income families, reduce veteran homelessness, distribute grants to local community development organizations, protect housing for Native Americans, expand homeownership opportunities, protect vulnerable homeowners from foreclosure, and most recently, reduce long-standing histories of residential segregation through an amped-up approach to promoting fair housing. The notion that Carson, who considers even his own experience with poverty a “choice,” is equipped to continue and build upon this important work is laughable.

Throughout a sustained history of urban austerity, HUD has been and continues to be an agency that does a whole lot with remarkably little—receiving a meager 1 percent of combined mandatory and discretionary funding in the federal budget to tackle deep forms of residential poverty and inequality. Still, Trump’s Carson pick is not even the most scornful of political insults to HUD in recent years. During the 2012 presidential election, in a private campaign speech to supporters that unintentionally went public, Mitt Romney proposed to eliminate HUD altogether if elected president. This incident was particularly scathing to affordable housing proponents considering that Romney’s own father George headed the agency during the Nixon administration and stood as an unlikely supporter of racial and economic integration by promoting the 1968 Fair Housing Act. In fact, the delegitimization of HUD and urban policy more generally could ironically be tied back to the tenure of the elder Romney, whose efforts to combat socio-economic and racial inequality through housing desegregation were almost immediately squelched by an unsupportive President Nixon, who rolled back HUD’s budget and policy reach.

Looking at the fate of one of America’s most notoriously beleaguered cities and the opening setting the rags-to-riches story Carson has told in numerous books offers an interesting window into HUD’s underleveraged role in combating urban poverty. Carson was born in the Motor City in 1951, when Detroit’s population had just peaked at slightly more than 1.8 million. By 1970, five years after Lyndon B. Johnson created HUD, Detroit had lost 300,000 residents. By 2010, the city saw a 60-percent reduction in its population since its peak, with a mere 700,000 residents remaining in the city limits. Detroit’s decline is a dramatic example of the urban decline HUD was ill-equipped to combat across cities in the United States.

Throughout the latter half of the 20th century, many urban areas like Detroit suffered postindustrial fates that were simultaneously aided and amplified by a retrenchment in federal funding. As the urban renewal and Great Society dollars of the Johnson administration decreased during the 1970s and ’80s, even the diverse economic urban powerhouse of New York City was notoriously told to “drop dead” when seeking a financial bailout from the feds in 1975. Though urban poverty rates remained high during this period, the amount of community development grants and federally subsidized housing administered by HUD continued to decline significantly up through the 1990s; the decrease in housing units culminated in the Clinton administration’s plan to demolish, replace, and privatize public housing in the majority of U.S. cities through a program widely known as HOPE VI (Housing Opportunities for People Everywhere). Hit especially hard by the “urban crisis,” Detroit’s attempt at implementing HOPE VI went horribly awry—with HUD taking control of the local agency when grants related to public housing rehabilitation defaulted due to stalled construction.

Despite hemorrhaging population since 1950, Detroit remains the state of Michigan’s most populous city, with concentrated poverty and gentrification both notable forces in the city’s contemporary narrative. Unlike Detroit, the majority of the populations of large cities in the United States are growing. But like the Motor City, such growth is marked by stark urban inequality that provides secure futures for a select group of wealthy residents living in fashionable areas, while leaving a shocking number of others at risk of displacement or even homelessness. A disconcertingly high number of Americans are severely rent-burdened. We need HUD’s services more than ever and not as a means to reboot the dangerous and racially charged policies of “urban renewal,” a term that has already been ignorantly and ahistorically floated by Trump in offering Carson the position. We instead need HUD to expand its efforts at tackling mounting inequality as our cities continue to grow. In particular, we need a rebooted effort to include federally subsidized housing—be it through the construction of new affordable units or the expansion of housing vouchers—as a critical piece of the fraying social safety net. Nonetheless, even if Detroit had received greater subsidies and technical guidance from HUD, the ability to provide adequate housing for the staggering number of families living in poverty remains a lofty goal and is still far from the only means necessary to tackle mounting inequality in cities across the nation.

The large-scale inequality unfolding in our cities necessitates an equally forceful response that Carson’s leadership will almost certainly not provide. However, the agency’s quiet gains over time in promoting desegregation, reducing homelessness, and housing low-income families should not go undervalued. The Obama administration also expanded HUD’s budget by more than $7 billion between 2009 and 2016, with an additional proposed $1.9 billion increase for fiscal year 2017. The agency’s subtle growth and regulatory achievements are yet another marker of the outstanding incremental progress that mark the outgoing president’s legacy.

But Obama’s Affirmatively Furthering Fair Housing desegregation initiative is now at risk after already having been criticized by Carson, who has chalked the effort up to a “mandated social engineering scheme” and “failed socialist experiment.” Such a critique of Obama’s AFFH initiative is much like that of the failures of HUD-administered public housing throughout the 20th century. Public housing towers have been disregarded as “high rise hellholes” plagued by crime. Yet as MIT professor Lawrence Vale has researched, the “socialist” experiment of public housing “failed” not because the government was providing it, but because public resources were slashed when the population of public housing became majority–black American; racialized rhetoric propelled a highly publicized narrative built around the failure of the projects.

The more optimistic of affordable housing advocates are hoping that having a HUD secretary who is already a household name may finally place housing and urban policy on the average citizen’s radar, or that Carson’s medical background will renew focus on the health issues linked to aging public housing infrastructure. Despite their inexperienced future leader, the many smart and engaged technocrats at HUD will inevitably continue their work of providing critical urban services to selected vulnerable residents in the face of limited resources as they have been doing since practically the agency’s creation. Whether Trump and Carson’s HUD continues or worsens the increasingly privatized and underfunded effort of federally subsidized housing, the future for the two-thirds of eligible low-income families who do not receive rental assistance in our nation’s cities and towns is looking bleak. With Carson at the helm, the agency is finally receiving the public attention it has long deserved. Unfortunately it is for all the wrong reasons.