A new set of guidelines from the Department of Housing and Urban Development will make it tougher for landlords and home sellers to discriminate against applicants who have criminal backgrounds.

The guidance, announced Monday by HUD, means that a flat-out refusal to rent or sell to people who have criminal records is discriminatory because minorities—African Americans and Latinos in particular—are disproportionately arrested and imprisoned.

“No American should ever be discriminated against because of their race or ethnicity, even if that discrimination results from a policy that appears neutral on its face,” HUD Secretary Julián Castro said Monday during the National Low Income Housing Coalition Policy Forum in Washington D.C. “Black and Latino Americans are unfairly arrested at significantly higher rates than white Americans.”

Related: Supreme Court Preserves Protections of Fair Housing Law

Under the new guidelines landlords will have to better scrutinize whether a person was arrested and if they were also convicted. And, even if a person was convicted, property owners have to weigh the nature and severity of the crime and conviction when considering an applicant’s housing application.

The Morning Rundown Get a head start on the morning's top stories. This site is protected by recaptcha

Failure to do so might mean the homeowner could potentially face an investigation for discrimination and, ultimately, civil penalties.

People with criminal pasts are not a protected class under the Fair Housing Act and the administration insists that in some cases it might be legal and reasonable to turn them down for housing. But landlords will have to prove their actions were done in order to keep their property safe, Castro said.

“When landlords summarily refuse to rent to anyone who has an arrest record, they may effectively and disproportionately bar the door to millions of folks of color for no good reason at all,” Castro said.

Housing advocates and civil rights groups hailed the move as a step in the right direction.

“To address the impact of mass incarceration, we must ensure that people leaving jails and prisons are provided access to housing opportunity,” Kristen Clarke, president and executive director of the Lawyers’ Committee for Civil Rights Under Law, an advocacy group, said in a statement . “Without access to housing opportunities, people with criminal records are placed on a path to failure and unable to take the steps necessary to successfully reintegrate into their communities. This is especially true for African-Americans and other minorities who are overrepresented in the criminal justice system.

HUD’s new guidance follows a Supreme Court ruling from last year which thwarted an effort to narrow the scope of the nation’s decades-old Fair House Act. In a 5-4 vote, the "court acknowledges the Fair Housing Act's continuing role in moving the nation toward a more integrated society," the court wrote at the time. The ruling allowed an interpretation of housing discrimination cases based on disparate impact on minorities.

“Finding safe and stable housing is a foundation for successful re-entry into society, but these opportunities remain elusive for people with a criminal history and more so for those who are people of color,” Shanna Smith, president and CEO of the National Fair Housing Alliance, an advocacy group, said in a statement.

Some lawmakers were critical of HUD's new guidelines and, by extension, the Obama administration's efforts.

"There are no lengths to which this administration won’t go to support convicted criminals. While those who have served their debt to society and completed the rehabilitation process deserve a second chance, it should not be at the expense of law-abiding citizens," Rep. Tom Cotton, R-Arkansas said in a statement on Tuesday. "Whether releasing violent felons early from prison, preventing employers from asking about an applicant’s criminal record, or now blocking landlords from deciding whether to rent to someone who may pose a threat to their property and the surrounding community, these policies are part of a disturbing pattern. And they send the wrong message to criminals in Arkansas and across the country."