Get breaking news alerts and special reports. The news and stories that matter, delivered weekday mornings.

Pair by pair, beaming gay couples waved their marriage licenses as they left a Kentucky county clerk’s office Friday — a scene of jubilation one day after the head clerk was jailed for refusing to issue them herself.

“We got it!” shouted April Miller. She and her partner, Karen Roberts, were the third couple to receive their license in Rowan County.

After the Supreme Court legalized same-sex marriage nationwide in June, Miller and Roberts tried three times to obtain their license — but during each visit, county clerk Kim Davis turned them away. The women were among the couples to sue Davis for failing to perform her duty under the law.

Davis, an Apostolic Christian, has refused to issue licenses to same-sex couples, saying it would infringe on her religious beliefs. In response, District Court Judge David Bunning held her in contempt of court on Thursday, and ordered her deputy clerks to start issuing licenses to same-sex couples.

Five of the six deputies agreed to comply. The lone holdout was her son, Nathan Davis.

“This is our civil right,” Miller said on Friday, holding up the license. “Although it took two months, it’s what we got on June 26 from the Supreme Court, and it’s awesome.”

James Yates and William Smith Jr. were first in line at the clerk's office. They had previously been denied a license five times.

Let our news meet your inbox. The news and stories that matters, delivered weekday mornings. This site is protected by recaptcha

One of Davis' deputies issued the license to the couple, then shook their hands. A crowd of supporters erupted in cheers as the couple left the courthouse.

A tearful Yates told reporters he was "relieved and happy."

Later in the day, David Moore and David Ermold, whose Tuesday confrontation with Davis drew the nation's attention to Rowan County, received their license.

"I feel like a person," Ermold said afterward, breaking down into tears.

Related: Couple Suing Kentucky Clerk Kim Davis Over Marriage License Plan to Wed Friday

Davis' husband, Joe Davis, told NBC News Friday morning that his wife wouldn't resign and was prepared to stay in jail for "as long as it takes."

"It’s not a defeat. Some may say it is, but it’s not a defeat. It’s a setback," Joe Davis said. "It ain't over. It'll still be a fight."

He and Kim Davis' lawyers have questioned whether the licenses issued Friday without her authorization would be valid. The county attorney, who was present in court on Thursday, said they would. The judge, however, didn't comment on their validity.

Michael Long (left) and Timothy Long, of Rowan County, raise their hands in front of a crowd of supporters after receiving their legal marriage license at the Rowan County Courthouse on Sept. 3, 2015, in Morehead, Kentucky. NIKON CORP / Getty Images

Other gay couples were lining up Friday at the clerk’s office, as their supporters outside shouted “love has won” — while Davis’ supporters quoted Bible scripture.

Michael and Timothy Long said they had tried once before to get a license but were turned away by Davis. They had a civil union ceremony in 2008, but they have waited anxiously to get hitched in their home county.

They’ve set a wedding date for Sept. 27.

“I feel like we’re accepted like everyone else,” Timothy Long said.

Bunning said Davis could remain in jail for at least a week, but he is expected to file paperwork Friday explaining what it would take for her to be released from jail.

Joe Davis blasted Bunning for putting his wife behind bars.

"He’s no different than a bully in high school or out here in the street," Davis said.

Bunning upheld the Supreme Court's landmark decision to legalize same-sex marriage nationwide. Last month, after Kim Davis was sued for contempt, Bunning wrote that the state was merely forcing her to do her job within the law.

But Joe Davis said his wife had done her job.

"Just because five Supreme Court judges make a ruling, it’s not a law," he said.