Midway through this year’s MTV VMAs came an unexpectedly serious and sincere appearance by Susan Bro, mother of Heather Heyer, the counter-protestor slain in Charlottesville. Introducing Bro was the Rev. Robert Lee IV, who denounced his famous Confederate ancestor and endorsed anti-racism activism. Today, Lee announced he’s leaving his position at Bethany United Church of Christ in Winston-Salem, North Carolina, after facing backlash from the congregation for his outspokenness.

“My presence at the church as a descendent of Robert E. Lee and an outspoken opponent of White Supremacy had already attracted attention, but with my appearance on MTV the media’s focus on my church reached an all time high,” Lee IV wrote in a message published on the Auburn Seminary website. “A faction of church members were concerned about my speech and that I lifted up Black Lives Matter movement, the Women’s March, and Heather Heyer as examples of racial justice work.” Lee offered his resignation ahead of a planned church vote, he said.

“I want to stress that there were many in the congregation who supported my right to free speech, yet were uncomfortable with the attention the church was receiving,” Lee added. “The church’s reaction was deeply hurtful to me.”

The Bethany United position was his first job since leaving seminary, Lee said. According to the New York Daily News, Lee just joined the church in February.