It was the summer of 2007 and Senator Edward M. Kennedy, D-Mass., unloaded on his junior colleague, then-Sen. Barack Obama, D-Illinois.

"You can't come in here and undo everything!" Kennedy told Obama, according to Sen. Lindsey Graham, R-SC.

Kennedy was angry, Graham would later recall, because Obama had signed on to be part of a bipartisan coalition of senators pushing a comprehensive immigration package, and yet he kept straying from the group's agreements.

The bipartisan team, led by Kennedy and Sen. John McCain, R-Ariz., would consult before voting on amendments, voting against them as a group even when they personally supported them if they felt the amendments would hurt the bill's chance of ultimately passing. Obama -- who by then had launched his presidential campaign -- on several occasions offered or supported amendments that if they passed might defeat final passage of the bill.

Which is how he aroused Kennedy's ire, Graham said, in a story he first told The Hill newspaper last year.

JUST WATCHED Sessions: We're in an unhealthy situation Replay More Videos ... MUST WATCH Sessions: We're in an unhealthy situation 02:26

JUST WATCHED Boehner responds to immigration order Replay More Videos ... MUST WATCH Boehner responds to immigration order 04:13

The '07 immigration bill was ultimately filibustered by both Republicans and Democrats. Kennedy, Obama and Graham all voted in favor of moving forward with it.

More than seven years later, Kennedy is dead, unable to corroborate Graham's story. Obama is president, having defeated both McCain and another Republican, Mitt Romney, who took a much harder line against immigration reform. President Obama is today flying across the country to campaign for his executive actions, which will defer deportation for an estimated five million undocumented immigrants.

There hasn't been much coverage of this chapter of Obama's senatorial history. But from the perspective of the members of the McCain/Kennedy coalition, Obama took actions that threatened their very delicate legislative dance, an interesting historical note given the president's announcement and his remarks about congressional inaction.

In his speech to the nation Thursday night announcing his plan, the president had a simple response for GOP critics assailing him for bypassing the legislative branch: "to those members of Congress who question my authority to make our immigration system work better, or question the wisdom of me acting where Congress has failed, I have one answer: Pass a bill."

But to those who were part of the effort to pass a bill in 2007, Obama's incredulity at legislative inaction rings a bit false. To them -- many of whom did not want to be identified because of the sensitivity of criticizing a president publicly -- President Obama sees the immigration effort strictly through a political lens. He is for it when it helps him politically, and when it was politically more problematic to be part of a bipartisan effort, he did what was good for him.

JUST WATCHED Frenemies: Pres. Obama & Speaker Boehner Replay More Videos ... MUST WATCH Frenemies: Pres. Obama & Speaker Boehner 01:23

Without question, the efforts that have failed more recently are because of House Republicans. The U.S. Senate passed legislation with overwhelming bipartisan support and House Speaker John Boehner, R-Ohio, failed to bring in to the floor for a vote. Opposition to immigration legislation among Republican House rank and file was so strong, Boehner wouldn't even allow a vote on legislation that had made it out of the House Judiciary Committee.

At the time, the alliance was a risky endeavor. In May 2007, Graham was booed at the South Carolina Republican State Convention because of his immigration alliance with Kennedy.

"There are some people tell me, 'I'll never vote for you again if you do this,'" Graham told CNN for a June 2007 profile. "Well, if I based every decision as a senator on that statement, I would do nothing. So what I'm going to do is lead."

Many forces threatened to scuttle the fragile bipartisan group, big business and labor unions alike. And yet, even though he was part of the coalition, Obama offered an amendment that the larger group opposed, one that would have sunsetted the merit-based evaluation system for immigrants after five years. That amendment failed 42-55.

But Obama also supported four other amendments that the coalition opposed. Two from Sen. Byron Dorgan, D-ND, to sunset both the temporary guest worker visa program and the Y-1 non-immigrant temporary worker visa program after five years; and two amendments from Sen. Jeff Bingaman, D-NM, that would have removed the requirement that 'Y' non-immigrant visa holders leave the United States before they are able to renew their visa, and would have lowered the annual visa quota for guest workers from 400,000 to 200,000 per year.

Obama voted for all five; Kennedy voted against all five, as this reporter noted six years ago

Dorgan's amendment to sunset the temporary guest worker visa program was of particular issue, since it passed, 49-48, despite calls from the coalition that it constituted a "deal-breaker." Future votes to bring the legislation up for a vote on final passage failed.

JUST WATCHED Obama: We were strangers once too Replay More Videos ... MUST WATCH Obama: We were strangers once too 02:37

JUST WATCHED Is there a border surge coming? Replay More Videos ... MUST WATCH Is there a border surge coming? 02:04

"Byron Dorgan did an amendment, and it scuttled the bill," Sen. Charles Schumer, D-NY, said at a press conference on January 31, 2013.

There is much dispute about whether the union-backed amendments Obama supported "were intended to kill the legislation," as McCain has asserted. Clearly the counter-argument is that Obama, Dorgan and Bingaman were attempting to improve the legislation, not kill it.

Still, those involved credit Obama with working hard to make immigration reform happen in 2006, but playing a decidedly different role in 2007.

"The general feeling by those involved is that he parachuted into the meetings a lot," Graham told The Hill . "He was certainly there at all the press conferences but we had a lot of problems with him and quite frankly a couple others who wanted to renegotiate things after they had been closed out by the main group."

The White House was contacted by CNN, but did not immediately respond.