WASHINGTON -- Moments after news broke that U.S. special forces had captured one of the suspected ringleaders in the 2012 terrorist attack in Benghazi, Libya, Sens. John McCain (R-Ariz.) and Lindsey Graham (R-S.C.) said they knew the perfect place to send him: Guantanamo Bay.

McCain and Graham, two of the most vocal critics of the Obama administration's handling of Benghazi, both praised the capture of Ahmed Abu Khattala on Tuesday. But they expressed reservations about bringing Abu Khattala, who U.S. officials say was captured on Sunday, to American soil.

"Guantanamo [is] where we put terrorists, where we apprehend them," McCain told reporters on Capitol Hill. "Where else can you take him to?"

Graham argued that Abu Khattala should be held as an enemy combatant at the detention facility, adding that he was hopeful the suspect would provide "good intelligence."

"We should have some quality time with this guy -- weeks and months," Graham said. "Don't torture him, but have some quality time with him."

Graham is a firm believer in keeping Guantanamo open, but McCain's comments seem at odds with his prior statements that the U.S. naval base in Cuba should be shut down. Following a visit last year with Sen. Dianne Feinstein (D-Calif.) and White House Chief of Staff Dennis McDonough, McCain said it was of "national interest" to close the camp and transfer detainees to other locations.

A spokesman for McCain did not immediately return a request for comment when asked if the senator's position has since changed.

Officials have not disclosed where Abu Khattala is being held, but they said he was "en route" to the United States, where he is expected to be arraigned in Washington. Graham said once Abu Khattala arrives on U.S. soil, he will be read his Miranda Rights, making it more difficult to interrogate him.

"I don't care if they try him in federal court. I want him to be held for a sufficient period of time to gather intelligence," Graham said, adding that Guantanamo had been a good source of information over time.

But the international community has criticized the U.S. military prison, established in 2002, for mistreating inmates and holding them without trial.

Self-professed Sept. 11 mastermind Khalid Sheikh Mohammed, for example, is being held in Guantanamo and has not been convicted by a military court in the more than a decade that has passed since his 2003 capture. By contrast, Osama bin Laden's son-in-law was convicted in federal court barely a year after his arrest in March 2013.

Graham suggested that approach was ineffective, noting that bin Laden's son-in-law was only interrogated for 20 hours.

"We should have held him for 20 months," he said. "We're shutting down intelligence gathering, we're turning the war into a crime, and it will bite us in the butt."

Sens. Ted Cruz (R-Texas) and Marco Rubio (R-Fla.) also called on Tuesday for Abu Khattala to be sent to Guantanamo.

"Khatallah is a foreign terrorist, captured by our special forces overseas for his violent attack on a U.S. facility," Cruz said in a statement. "He belongs in Guantanamo and in the military justice system, not in the U.S. civilian court system with the constitutional protections afforded U.S. citizens."

"The Obama administration should immediately transfer him to the military detention center at Guantanamo Bay for detention and interrogation," Rubio said. "In order to locate all individuals associated with the attacks that led to the deaths of four Americans, we need intelligence. That intelligence is often obtained through an interrogation process."

Sen. Kelly Ayotte (R-N.H.), another staunch Benghazi critic, did not join the chorus, but cautioned the administration against worrying about following procedure.

"Rather than rushing to read him his Miranda rights and telling him he has the right to remain silent, I hope the administration will focus on collecting the intelligence necessary to prevent future attacks and to find other terrorists responsible for the Benghazi attacks," Ayotte said.

Ayotte would also like Abu Khattala to be held at Guantanamo, her spokesman Jeff Grappone confirmed to The Huffington Post.

Sen. Patrick Leahy (D-Vt.), chairman of the Senate Judiciary Committee, said arguing in favor of sending Abu Khattala to Guantanamo is "the easy way out."

"I look forward to his prosecution in the U.S. court system. That is what Americans do when faced with murderers and terrorists – we prove their guilt and hold them accountable for what they have done," Leahy said in a statement.

"We will try Khattala just as we have successfully tried more than 500 terrorism suspects since 9/11," he added. "We all should be proud of America's courts -- the best in the world, and the envy of the world, where scores of terrorists have successfully been tried and convicted ... We can and will demonstrate to the rest of the world that we are proud of our criminal justice system, and Americans are not afraid to use it."

UPDATE: 5:00 p.m. -- National Security Council spokesman Caitlin Hayden rejected calls to hold Abu Khattala at Guantanamo, pointing out that the U.S. has convicted hundreds of terrorists since 9/11 using the federal court system.

"Some have suggested that he should go to GTMO. Let me rule that out from the start," Hayden said in a statement on Abu Khattala. "The Administration's policy is clear on this issue: we have not added a single person to the GTMO population since President Obama took office, and we have had substantial success delivering swift justice to terrorists through our federal court system."

"The effective use of the criminal justice system has resulted in the debriefing, conviction and incarceration of U.S. citizens and non-citizens for acts of terrorism committed inside the United States and around the world," she added. "The system has repeatedly proven that it can successfully allow us to gather intelligence, handle the threat that we continue to face, and prosecute terrorists."