Jack Phillips is no stranger to controversy.

The owner of Masterpiece Cakeshop in Colorado, Phillips ignited a national debate after he refused to create a wedding cake for a gay couple in 2012. Since then, Phillips says he has lost a chunk of his business revenue, received death threats and been subjected to vile online reviews.

But that’s not the worst of it, Phillips told Fox News in an exclusive interview. For him, the most trying piece of the whole controversy is the hateful comments directed — even unintentionally — at his wife and daughter.

“In all of this, the threats against me or disparaging comments, the worst part is that I have to answer the phone so they’re not threatening my wife or my daughter when they pick it up,” Phillips said. “They don’t wait to see who’s on the phone. You pick up the phone, they’re already talking.”

Phillips added that he also tries to shield the other employees at the bakery from negative comments directed at him — at times becoming emotional as he spoke about his employees and family.

The Supreme Court on Monday agreed to decide whether Phillips discriminated against the gay couple when he refused to bake them a wedding cake due to his religious beliefs. Phillips said he didn’t refuse the couple service and offered to sell them anything in the store. But when it came to actively participating in the couple’s wedding, that’s where Phillips drew the line.

So the nation’s highest court is now tasked with striking a balance between the religious rights of Phillips with the couple’s right to equal treatment under the law.

Oral arguments will likely be held in the court's fall term.

“This has always been about more than a cake,” David Mullins, one of the men who tried to purchase the wedding cake, said in a statement this week. “Businesses should not be allowed to violate the law and discriminate against us because of who we are and who we love.”

Mullins’ husband, Craig, added that the two are “disappointed” that the court has continued with the case.

US SUPREME COURT TO DECIDE COLORADO GAY WEDDING CAKE CASE: TIMELINE

Phillips doesn’t usually speak about the harassment he’s been subjected to since he refused to bake the cake, but the grandfather of three told Fox News that he’s been pretty regularly bombarded with vulgar comments over the phone and through email.

He says he’s only received two actual death threats but he's also been told that he doesn't “deserve to live” and that “Christians should be thrown into the Roman Colosseum with lions."

One of those threats, Phillips said, came just a few weeks after he refused to bake the cake — long before the story garnered national attention. Then, a man called to say that he knew Phillips' daughter, Lisa, was working at the bakery. The man proceeded to give point-by-point driving directions to the cakeshop, where he would murder them.

“It could have just been somebody calling and they knew the area, and it could have been someone in Connecticut looking at a map,” Phillips recalled. "But they knew that Lisa was there.”

Phillips was also hurt, he said, when a member of the Colorado Civil Rights Commission compared him to a perpetrator of the Holocaust.

“Freedom of religion and religion has been used to justify all kinds of discrimination throughout history, whether it be slavery, whether it be the Holocaust,” Commissioner Diann Rice said in a brief. “I mean, we can list hundreds of situations where freedom of religion has been used to justify discrimination."

Phillips teared up as he described how even after his father was severely injured during combat in World War II, he still was part of a team to help liberate a Nazi concentration camp.

“For her to compare standing for my faith and not making a cake to Hitler’s atrocities just is unspeakable,” an emotional Phillips said.

RELIGIOUS FREEDOM OR DISCRIMINATION? GAY WEDDING CAKE AT CENTER OF COLORADO APPEALS COURT CASE

Kristen Waggoner, an attorney with the Alliance Defending Freedom, the conservative nonprofit legal organization that represents Phillips, called the comment “deeply offensive and biased.”

Waggoner argued Thursday that the small business owner is the one who has been persecuted since the interaction with the couple.

She said she recently spoke at an internationally-attended summit on religious persecution and was approached by people concerned about how Christians are treated in the U.S.

“The marginalization of Christians in America today — removing them from their jobs, banning them or purging them from entire professions — that’s a step toward physical persecution,” Waggoner said she was told.

“You see that kind of marginalization in other countries, too. It’s a more polite type of persecution, but nonetheless it is. It’s forcing someone to choose between their faith and making a living and supporting their family in their vocation,” Waggoner told Fox News.