GRAYSON, Ky. - Rowan County, Kentucky clerk Kim Davis was ordered released from jail Tuesday by the judge who locked her up for refusing to issue marriage licenses to gay couples.

Davis exited the Carter County Detention Center in Grayson, Kentucky just after 2:30 p.m. She had been there since Thursday.

Hundreds of supporters sang "Amazing Grace" and "God Bless America" as her lawyer spoke to the crowd and news outlets gathered outside.

Get Breaking News Delivered to Your Inbox

U.S. District Judge David Bunning on Tuesday lifted the contempt order against Davis, saying he was satisfied that her deputies are fulfilling their obligation to grant licenses to same-sex couples in her absence. But he warned Davis not to interfere with them, saying, "If Defendant Davis should interfere in any way with their issuance, that will be considered a violation of this Order and appropriate sanctions will be considered."

Davis' lawyer, Mat Staver, of Liberty Counsel, a Christian law firm, refused to say whether Davis would obey that order.

"Kim Davis cannot and will not violate her conscience," Staver said outside the jail, adding that Davis will return to work this week.

Davis appeared to be crying and said little immediately following her release. She was asked by reporters: "Was it worth it?" and in response, she smiled and nodded.

When asked how the last few days have been in jail, Davis responded, "All has been well."

At a rally shortly afterwards, Davis came out to cheers, with her arms raised and her husband, who was in blue dungarees and a straw hat, beside her. The Rocky theme song, "Eye of the Tiger," blared in the background. GOP presidential candidate Mike Huckabee introduced her.

"I love you all so much," she told the crowd, through tears. "I just want to give God the glory. His people have rallied and you are strong people. We serve a living God who knows exactly where each and everyone of us is at. Just keep on pressing. Don't let down. Because he is here and he's worthy. I love you guys. Thank you so much."

The move to release Davis came down after dozens of her supporters gathered on the jailhouse lawn for what they called a rally for religious freedom.

Huckabee and fellow Republican presidential candidate Sen. Ted Cruz visited Davis in jail Tuesday.

Huckabee later tweeted, "I was honored to meet w/ #KimDavis. A woman of such strong faith and conviction. #ImWithKim #ReligiousLiberty."

I was honored to meet w/ #KimDavis. A woman of such strong faith and conviction. #ImWithKim #ReligiousLiberty pic.twitter.com/RhcaENaA6i — Gov. Mike Huckabee (@GovMikeHuckabee) September 8, 2015

Standing alongside Davis after she was released, Huckabee, a former Baptist minister and Arkansas governor, said, "If somebody has to go to jail, I'm willing to go in her place. I believe that."

"She has shown more courage than any politician I know. She not only said something, she was willing to put her life at risk."

Staver, Davis' attorney, released a statement Tuesday prior to her release saying that while he was pleased with the judge's decision, Davis "can never recover the past six days of her life spent in an isolated jail cell, where she was incarcerated like a common criminal because of her conscience and religions convictions."

Davis was ordered jailed Thursday, Sept. 3 on a contempt charge after defying several court orders. Her lawyers spent Labor Day weekend filing appeals in an effort to force her release.

Davis stopped issuing all marriage licenses in June, the day after the U.S. Supreme Court legalized same-sex marriage nationwide. Two gay couples and two heterosexual couples sued her. Judge Bunning ordered Davis to issue the licenses, and the Supreme Court upheld his ruling.

But Davis still refused to do it, saying she could not betray her conscience or God. Bunning ruled last week that Davis was in contempt of court and sent her to jail. Her deputy clerks - except for her son, Nathan Davis - then issued marriage licenses to gay couples Friday with Davis behind bars.

In lifting the contempt order Tuesday, Bunning asked for updates on the five deputy clerks' compliance every two weeks.

Staver said on Tuesday that the marriage licenses issued in Rowan County while Davis was behind bars are not valid. But Allison Martin, a spokeswoman for Kentucky's attorney general, said the office believes they are valid.

Laura Landenwich, an attorney for the couples whose lawsuit led to Davis' jailing, said she has her doubts Davis will comply with the court's order.

"I would hope that she would recognize her legal obligations at this point," Landenwich said. "And do what's right."

Davis, an apostolic Christian, believes gay marriage is a sin. She also believes it would be a sin for her to issue a marriage license to a same-sex couple because the licenses are issued under her authority. She tried in vain to have state lawmakers change the law as a legal challenge to Kentucky's same-sex marriage ban wound its way through the federal appeals court.

On Monday, Davis's lawyers took their case to a federal appeals court, asking that Davis be allowed to remove her name and title from marriage certificates issued in Rowan County so that she would not have to act against her conscience.

Davis' jailing has offered some of the many GOP presidential candidates an opportunity to appeal to the party's evangelical Christian wing, which opposes same-sex marriage and casts Davis' imprisonment as an issue of religious freedom.