Former Attorney General Eric Holder on Tuesday said that the Electoral College is a "defect in our democracy" during a discussion about Congressional redistricting.

Holder, who served during the Obama administration, spoke on a panel at the National Press Club with Arnold Schwarzenegger, the former California Republican governor and prominent actor.

"We've got to do away with the Electoral College. We've got a system now that was—it's a defect in our democracy. We have had in the last five elections two presidents who were not—win the popular vote," Holder said. "The presidency is the one office in this country that represents all of the people. "

"You have a district representative, your Congressman. You have a person that represents your state or two people that represent your state. You have senators and you vote for them in a direct way. It seems to me that you have a direct election of the President of the United States," Holder continued.

He went on to talk about how Democratic presidential candidates would be campaigning in Texas and Republicans would be visiting California if the system was abolished.

Holder has repeatedly called for the Electoral College to be abolished, saying it's a "vestige of the past."

"Time to make Electoral College a vestige of the past. It’s undemocratic, forces candidates to ignore majority of the voters and campaign in a small number of states. The presidency is our one national office and should be decided – directly – by the voters," Holder tweeted.