Story highlights "I'm overwhelmed. I'm extremely happy to be vindicated," he says

New information has come to light, U.S. attorney says

Source says investigators are looking at someone trying to frame him

Curtis had been accused of mailing the poison to Obama, a U.S. senator and a judge

Thrust into the headlines last week, accused of sending ricin-tainted letters to President Barack Obama and two other officials, Paul Kevin Curtis is a free man again.

"It's like a train has been lifted off my shoulders," Curtis told CNN's "Piers Morgan Live" on Tuesday. "I'm overwhelmed. I'm extremely happy to be vindicated and out and able to see my kids."

Charges were dropped and "new information" became available, U.S. Attorney Felicia Adams said.

Authorities are investigating whether someone may have tried to falsely implicate the Elvis impersonator from Corinth, Mississippi, a law enforcement source said on condition of anonymity.

Curtis was arrested April 17 and charged with sending a threat to the president last week after letters containing the poison triggered security scares around Washington.

Curtis said his arrest was surreal.

JUST WATCHED Ricin suspect released from custody Replay More Videos ... MUST WATCH Ricin suspect released from custody 15:12

JUST WATCHED Ex-ricin suspect: Thought cops said rice Replay More Videos ... MUST WATCH Ex-ricin suspect: Thought cops said rice 02:56

JUST WATCHED Atty. for ex-suspect: He was framed Replay More Videos ... MUST WATCH Atty. for ex-suspect: He was framed 03:21

"It looked like a scene out of a movie," said Curtis, adding that there were hooded men with machine guns. "I was just overwhelmed. I just kept asking, 'what is ricin? What did I do?'"

Curtis' attorney, Christi McCoy, said her client has been framed by someone who used several phrases Curtis likes to use on social media.

The letters read, in part: "To see a wrong and not expose it, is to become a silent partner to its continuance."

They were signed "I am KC and I approve this message," a source told CNN.

They each had a Memphis, Tennessee, postmark and no return address.

McCoy said she was sure someone else was to blame.

"I do believe that someone who was familiar and is familiar with Kevin just simply took his personal information and did this to him," McCoy said. "It is absolutely horrific that someone would do this."

Curtis said it was an eye-opener for him.

"I think now, how many people are thrown in jail because of circumstantial evidence and someone can frame you that easily," he said.

Curtis had been accused of sending letters containing "a suspicious granular substance" to Obama; Sen. Roger Wicker, R-Mississippi; and Sadie Holland, a Justice Court judge in Lee County, Mississippi.

The FBI said the substance tested positive for ricin, a toxin derived from castor beans that has no known antidote.

No illnesses had been found as a result of exposure to the toxin.