(CNN) A descendant of Gen. Robert E. Lee has stepped down as pastor of his North Carolina church after facing blowback from parishioners and others for his comments denouncing racism and lauding the Black Lives Matter movement.

As a minister and newspaper columnist , Robert W. Lee IV, the fourth great-nephew of the Confederate general, has spoken and written countless words, but the five sentences he uttered during MTV's Video Music Awards last week were just too much for some members of his congregation.

Here is the short speech he delivered before introducing Susan Bro, mother of Charlottesville victim Heather Heyer , at the August 27 awards show:

"I call on all of us with privilege and power to confront racism and white supremacy head-on" - Robert Lee IV #VMAs pic.twitter.com/ko4SM9VnaU

Lee, who was pastor at Bethany United Church of Christ in Winston-Salem, has long supported removing monuments to the Confederacy and had already drawn attention for his lineage and for his stand against white supremacy prior to his MTV appearance. His speech that night drew unprecedented attention to his church, he wrote in a Tuesday statement announcing his resignation

When contacted Tuesday and asked to elaborate, Lee declined to provide many specifics. He said "the community backlash and the backlash toward the church" helped drive his decision, as did his sentiment that he and some church members "obviously had different mission ideas and goals."

He also said he fears the hullabaloo over his MTV speech and resignation is a distraction from the real issues of "black and brown bodies in the streets" and racial equality.

"I don't want this to detract from the movement or the message," he told CNN.

Asked about his next step, Lee said he's open to the spirit leading him to "something beautiful." In the meantime, he'll continue to disseminate his thoughts via his blog and social media accounts.

"I stand by how I handled this," he said.

'Deeply hurtful'

JUST WATCHED What led to Charlottesville protests? Replay More Videos ... MUST WATCH What led to Charlottesville protests? 02:36

The Duke Divinity School grad said he feels a "deep love" for the first church he led out of seminary. But while some members of his church showed support for him, Lee wrote that he felt he had no choice but to resign when parishioners opted to vote on his tenure.

"A faction of church members were concerned about my speech and that I lifted up Black Lives Matter movement, the Women's March, and Heather Heyer as examples of racial justice work," his resignation statement said. "I want to stress that there were many in the congregation who supported my right to free speech, yet were uncomfortable with the attention the church was receiving.

"The church's reaction was deeply hurtful to me."

Lee also shared a snippet of what he told his parishioners in tendering his resignation:

"I regret that speaking out has caused concern and pain to my church. For this is I offer my heartfelt apology. I understand that my views could be considered to be controversial. I never sought this sort of attention. But I do believe in God's role in calling out for positive social change for the good of all. We are all called by God to speak out against hate and evil in all its many forms. There are so many good things going on with this congregation and I do not want my fight to detract from the mission. If the recent media attention causes concern with my church, I reluctantly offer my resignation."

Officials at Bethany United Church of Christ could not be immediately reached for comment Tuesday. The church's phone was answered by an automated system that disconnected the call without offering the opportunity to leave a message.

Lee, a Statesville native who lives in Boone with his wife and toy poodle , served only five short months at the helm of the 228-year-old church. He said he will continue his mission to decry violence and oppression.

"I want to especially challenge white Christians in America to take seriously the deadly legacy of slavery in our country and commit ourselves to follow Jesus into a time of deep reflection, repentance and reconciliation," he wrote in his statement.

MTV wasn't a first

Charlottesville covered the Robert E. Lee statue with a black tarp after the deadlly protests.

Lee had warned against idolizing his ancestor on numerous occasions before his VMA speech. In July 2016 -- ahead of a sermon in Raleigh, in which he challenged his fellow clergy to join him in saying the names Alton Sterling and Philando Castile from the pulpit -- he wrote in The Washington Post of the shame he felt over his great-great-great-great uncle's legacy.

"I can't erase the history of my family's past, but I can say this: If you don't use the power you've been given for the good of this world, then you are part of the problem like the Lee family was long ago," he wrote.

The week before his MTV appearance, he told NPR of his family and church being threatened with violence for his stance. Some family members had told him to pipe down, he said. He hasn't faced additional threats since appearing on MTV, but he told CNN he has received hateful calls and social media messages.

The 24-year-old grew up knowing exactly what Gen. Lee stood for. His parents explained the debate over slavery versus states' rights and left him to draw his own conclusions, he told NPR.

"And so after kind of having to reconcile that for myself, when I would give my credit card to someone at the store and they'd see Robert Lee, they'd say, 'Oh, you're related to him?' And you'd either have to answer yes or no and then enter into a conversation about how the South will rise again with that person," he told NPR.

'We cried together'

JUST WATCHED Should the US ban Confederate monuments? Replay More Videos ... MUST WATCH Should the US ban Confederate monuments? 09:52

The choice to take a stand, despite his last name, was a clear one, he told NPR

"I'm also reminded that I have to speak up and speak out in God's name and in the name of my family to protect what little dignity we have left and to possibly redeem the situation for our family so that, going forward, they can say there was a Lee who stood up for what's right instead of standing up for the wrong side of history," he said.

Despite the backlash he's received, Lee said he has seen signs that his message is resonating, including with descendants of the people his family once owned.

"No one can quite lay a finger on who he actually was, but we know he committed some atrocious acts during the context of his time. He owned people, and that is egregious. That being said, I think there is room for redemption," he told the Winston-Salem Journal after his MTV appearance

"Just the other day I connected with the descendant of a Lee slave who wanted to have a moment to thank me for speaking up. We cried together and we tried to come to some sense of peace in all of this."

Correction: A previous version of this story gave the incorrect name of Robert W. Lee IV's church. Lee stepped down from Bethany United Church of Christ.