(CNN) -- New Mexico Gov. Bill Richardson sharply disputed that he made any promises to Bill Clinton about presidential endorsements, as Clinton reportedly has claimed.

"I never did," Richardson said Wednesday on CNN's "The Situation Room." "I never saw [President Clinton] five times. I saw him when he watched the Super Bowl with me. We made it very clear to him that he shouldn't expect an endorsement after that meeting."

Bill Clinton made the remarks in a gathering with some California superdelegates, the San Francisco Chronicle reported. Clinton became "red-faced" when the subject of Richardson's endorsement of Sen. Barack Obama came up and said, "Five times to my face he said that he would never do that," the newspaper reported.

Shortly after Richardson's endorsement, former Clinton White House aide and political strategist James Carville, who is also a CNN political analyst, told The New York Times that Richardson's decision was an "act of betrayal."

The endorsement "came right around the anniversary of the day when Judas sold out for 30 pieces of silver, so I think the timing is appropriate, if ironic," Carville said, referring to Easter weekend.

Clinton aides distanced themselves from those remarks, and Richardson said he wasn't going to "get in the gutter like that."

Richardson on Wednesday said he was "very close to endorsing" Sen. Hillary Clinton, but decided not to after the campaign became negative. Watch Richardson defend his endorsement »

"I held back. I waited. I felt the campaign got nasty. I heard Sen. Obama; he would talk to me continuously," Richardson said.

"The Clintons should get over this," he said.

Richardson endorsed Obama two weeks ago, saying his "affection for Hillary Clinton and President Bill Clinton will never waver" but that "it is now time for a new generation of leadership."

Richardson was secretary of energy under the Clinton administration, a post that helped bring him to national prominence and win the governorship of New Mexico in 2002.

He said he called Clinton to tell her of his decision to endorse Obama and said while it was a "tough conversation," he thinks he's been "totally up front."

Richardson on Wednesday said he has not talked to Bill Clinton since the endorsement.

"He's probably upset, and I understand that, because this is a tough game. But we've got to get over it. We've got to get positive. We've got to start talking about the issues," he said.

Richardson said he thinks the party needs to decide on a nominee.

"I'm not suggesting Sen. Clinton get out of the race. She's run a good, strong race," he said.

"We should finish these primaries. There are 10 left. June 3 is the last one. But shortly thereafter, I think the party superdelegates, party leaders should come together and say ... this gulf between the top two candidates that have run two great races needs to end."

Former President Jimmy Carter on Wednesday implied that he's also backing Obama, but did not make an official endorsement.

Speaking with a Nigerian newspaper journalist while in Abuja, Carter cited reasons he might be leaning toward the Illinois senator.

"Don't forget that Obama won in my state of Georgia," Carter said.

"My town, which is home to 625 people, is for Obama, my children and their spouses are pro-Obama. My grandchildren are also pro-Obama."

"As a superdelegate, I would not disclose who I am rooting for, but I leave you to make that guess," Carter added.

The Carter Center in Atlanta, Georgia, confirmed to CNN the newspaper quoted Carter accurately.

Responding to the comments, Clinton campaign spokesman Howard Wolfson said Thursday, "Both Sen. Clinton and President Clinton have a great deal of respect for President Carter and have enjoyed their relationship with him over the years, and obviously he is free to make whatever decision he thinks is appropriate with regard to presidential choice."

Meanwhile, both Clinton and Obama are campaigning hard in Pennsylvania, their next battleground. There are 158 delegates at stake in the state's April 22 primary.

The Obama camp on Thursday announced it hauled in more than $40 million from more than 442,000 donors in March.

There were more than 218,000 first-time donors, and the average contribution level was $96. The figure is not official until the campaign submits paperwork with the Federal Election Commission.

Clinton raised $20 million dollars in March, Clinton campaign sources tell CNN.

The official deadline for March paperwork to be filed is April 20.

Wolfson said he expected Clinton's tax returns to be released soon.

Clinton pledged on March 25 that she would release her returns within a week. E-mail to a friend

CNN's Alex Mooney and Rebecca Sinderbrand contributed to this report.

All About Bill Richardson (Politician) • Bill Clinton