Story highlights John Paul Stevens has written a new book, "Five Chiefs"

Country has moved to the right, 91-year-old says

He has small chambers in U.S. Supreme Court building

Retired Justice John Paul Stevens arrives for our conversation looking dapper in a three-piece dark suit and trademark bow tie. But do not mistake the "retired" that comes with his title as a sign the 91-year-old Stevens is slowing down.

"I have to confess, I've been a lot busier than I had anticipated," he told CNN Senior Legal Analyst Jeffrey Toobin. That includes a new book, a memoir of his 65-year involvement at the high court.

Stevens had just driven to the court from the airport, where hours before, he had attended the funeral in Chicago of his longtime friend and political patron, Sen. Charles Percy of Illinois.

Neither jet-lagged nor agitated, the justice is the model of cordiality and candor, wasting little time making his views known. When asked if the conservative-majority court is too radical, Stevens said, "Well I think some of the decisions are."

He is reminded of a dissent in an affirmative action case not long before his retirement last year, where he wrote, "No member of the court that I joined in 1975 would have agreed with today's decision." It was striking for him to conclude the court has changed that much. "I've thought about that, and perhaps people question the accuracy, but I'm really persuaded that's right, and I think it is," Stevens reflected.

"That suggests that you think the court has moved radically in recent years?" asked Toobin. "Well it has moved dramatically, that's right, and I guess a word like radical may well apply."

Stevens: Country moved to the right

Stevens' 35 years on the high court made him a multi-layered transitional figure-- the last of his World War II generation to serve on that bench, a longtime leader of the progressive minority on the court, and a justice whose views were criticized for betraying his Midwestern Republican roots.

Nominated to the federal appeals court in 1970 by President Richard Nixon and five years later to the high court by President Gerald Ford, Stevens says his views did not change over the years, but the court and the country did, moving to the right. The Republican Party of four decades ago, he admits, is no longer evident.

"I wish it were more prominent than it is today, because I do think that ... political parties can play an important role in the process of having ideas develop and be discussed back and forth," he told Toobin. "And I think Senator Percy was somewhat like President Ford in the sense that he was more in the mainstream of general political thought in American life than some of our senators are today."

JUST WATCHED Justice Stevens: Supreme Court changed Replay More Videos ... MUST WATCH Justice Stevens: Supreme Court changed 04:29

He cites as an example the death penalty. When he joined the high court, he and his colleagues were immediately confronted with whether to end a four-year moratorium on capital punishment. Through a series of rulings, Stevens agreed it should resume, but with new rules, new safeguards in place.

These important rulings imposed new legal standards, upholding the use of so-called "discretionary" guidelines for juries when deciding life or death, such as mitigating or aggravating factors, in a separate sentencing phase of a trial. And the court threw out "mandatory" death sentences for certain crimes, like murder or rape, which disregarded aspects of the offense that might favor the defendant.

Stevens acknowledge in an interview with CNN that his views had changed, in response to what the increasingly conservative court was doing.

"The court expanded, instead of narrowing the category ... of cases that are subject to the death penalty," he said. "It allowed jury-selection practices which I think tilted the scales in favor the prosecutor, and made other procedural changes that I thought were unfortunate. So I thought that that whole process has affected my own thinking and I thought changed the law in a way (that) proved maybe we were wrong the first time around."

By the time of his retirement, Stevens was firmly against executions. On the day we spoke, Georgia was preparing to give a lethal injection to Troy Davis, convicted of murdering a Savannah police officer. Unlike other appeals on procedural grounds, Davis made a more fundamental claim -- he was "actually innocent" of the crime.

That issue has split courts around the country, as forensic technology has exonerated dozens of innocent inmates and raised concerns death row inmates may be unjustly executed.

While politely refusing to talk specifically about the Davis case, he noted the DNA exoneration issue reaffirmed his longstanding views about how capital punishment is administered.

"Well, they've confirmed it, I would say, because I think before, even before that body of evidence developed, I had developed concerns about -- both on the issue whether it's wise policy or not. And I firmly believe it's an unwise policy, but I think it's more difficult to question as to whether it's a constitutionally permissible punishment."

Davis was executed later that night, but the issue of actual innocence remains in the federal courts.

Learn the oath, but have a copy

Stevens' new book, "Five Chiefs" (Little Brown), is released Monday. He offers his memories of the five chief justices he has known over the years, and the key issues the court tackled in those eras. He was law clerk when Fred Vinson led the court; appealed and argued cases as a private attorney before liberal icon Earl Warren; and served on the bench alongside conservatives Warren Burger, William Rehnquist and the current chief, John Roberts.

He offers mostly favorable reviews of each man's personal traits, if not their jurisprudence. Sprinkled throughout are anecdotes about the inner workings of the court, a mysterious, imposing place to many people.

One incident involved the 2005 presidential inauguration. Rehnquist's ongoing treatment for thyroid cancer made it uncertain whether he would be there to administer the constitutionally mandated oath of office to President Bush.

Stevens said he was prepared to step in. "I had to be. Sally Rider (Rehnquist's administrative assistant) gave me the oath to memorize the day before, because there was so much question whether Bill would be able to make it," he said. "Even 10 minutes before (the ceremony). I had the oath in my pocket, sitting on the dais at the Capitol. We didn't know if he was going to be able to do it for sure until he walked out the door onto the platform."

Four years later, Stevens swore in Vice President Joe Biden. New Chief Justice John Roberts then slightly mixed up the wording when he tried to administer the oath to President Barack Obama by memory, without benefit of reading a copy.

"I should say, it is unwise to feel required to memorize it when you can get a copy of it in your pocket," he told Toobin, smiling. "And when I went through the vice presidential swear-in, I had the oath in my pocket and I read it. So I didn't rely on my memory."

Busy in retirement

But Stevens' memories of a lifetime in the law remain fresh and vivid in his retelling. In retirement, he retains small chambers in the court building, keeping a busy schedule with speeches, lectures and writing. He has commented frequently on current cases and issues, bluntly stating how he might have ruled, a rarity for a retired jurist. But for him, no regrets about leaving the bench.

"Really, not at all," he told Toobin. "Of course I miss the work, I don't want to misstate that, but I'm absolutely fully confident and impressed with the quality of my successor [Justice Elena Kagan]. I think she's a wonderful appointment."

Last year was the right time to step down, he said. "Particularly as you get older, I thought it was important that I have someone whom I could trust who would tell me if he thought you're losing a little bit, if things aren't making too much sense. So I had an understanding with Justice Souter that if he thought my work was declining, the quality of the work was declining, then he'd tell me." He called it a continuing commitment, and that Souter never suggested he get out.

"No, he was complimentary and just the reverse, but the fact that he retired [in 2009] of course left me with a quandary. I no longer had my insurance policy there."

So what did Stevens do?

"Well, I retired," he said, laughing.