(CNN) Former Vice President Joe Biden says he now wants to throw out the Hyde Amendment, dropping his long-held support for the measure that blocks federal funds from being used for most abortions amid criticism from his 2020 Democratic rivals.

"If I believe health care is a right, as I do, I can no longer support an amendment that makes that right dependent on someone's ZIP code," he said.

The Hyde Amendment is a four-decade-old ban on federal dollars being used for abortions except in cases of rape, incest or when the woman's life is in danger.

Biden's campaign on Wednesday said he still supported the Hyde Amendment -- a position that put him to the right of all other leading 2020 Democratic contenders, as well as Hillary Clinton and the party's platform in 2016.

Biden told the crowd Thursday night he had changed his mind because Republican state lawmakers have enacted "extreme laws in clear violation of constitutional rights" protected by the Supreme Court's Roe v. Wade decision, making access to abortions more difficult for women who cannot afford the procedure or travel to obtain it.

Read More