Story highlights Church spokesman says Phelps died late Wednesday of natural causes

Phelps founded Westboro Baptist Church in Topeka, Kansas

The church is known for its virulently anti-gay protests, including at military funerals

Fred Phelps -- the founding pastor of a Kansas church known for its virulently anti-gay protests at public events, including military funerals -- has died, the church said Thursday.

The 84-year-old died of natural causes at 11:15 p.m. Wednesday, according to church spokesman Steve Drain.

Phelps founded Westboro Baptist Church of Topeka, Kansas, in 1955 and molded it in his fire-and-brimstone image. Many members of the small congregation are related to Phelps through blood or marriage.

In a statement Thursday, the church chided the "world-wide media" for "gleefully anticipating the death."

"God forbid, if every little soul at the Westboro Baptist Church were to die at this instant, or to turn from serving the true and living God, it would not change one thing about the judgments of God that await this deeply corrupted nation and world."

Photos: Photos: Pastor Fred Phelps Photos: Photos: Pastor Fred Phelps Pastor Fred Phelps – Photojournalist Anthony S. Karen gained rare access to the Westboro Baptist Church in 2008 and 2011. Here Pastor Fred Phelps is seen in his home office in 2011. Phelps founded the Phelps Chartered Law Firm in 1964 and had several notable civil rights cases. He was disbarred in Kansas in 1979. Hide Caption 1 of 10 Photos: Photos: Pastor Fred Phelps Pastor Fred Phelps – Phelps leads Sunday services at the Westboro Baptist Church in Topeka, Kansas, in 2008. Hide Caption 2 of 10 Photos: Photos: Pastor Fred Phelps Pastor Fred Phelps – Former member Megan Phelps-Roper, at the time a 25-year-old law office secretary and logistics coordinator for the church, pickets on Gage Road in Topeka. Gage Park was known to some as a "cruising" or meet up location in the gay community in the late 80s and early 90s. The first protest was held blocks from the Phelps home in June 1991, when, according to Phelps, a homosexual tried to lure his then 5-year-old grandson into some shrubbery The family still protests daily at this location. Hide Caption 3 of 10 Photos: Photos: Pastor Fred Phelps Pastor Fred Phelps – Shirley Phelps-Roper, then a 54-year-old law office manager, adjusts a makeshift skirt she routinely wears during pickets. Hide Caption 4 of 10 Photos: Photos: Pastor Fred Phelps Pastor Fred Phelps – Phelps embraces one of his grandchildren following a service that highlighted the evils of homosexuality. The content of each Sunday service was written by Phelps and lasts an hour. Hide Caption 5 of 10 Photos: Photos: Pastor Fred Phelps Pastor Fred Phelps – Chapter 18 verse 22 from the Old Testament book of Leviticus is the foundation of the church's intolerance toward homosexuality. Westboro Baptist Church claims to be the only true church and denounces all others. Hide Caption 6 of 10 Photos: Photos: Pastor Fred Phelps Pastor Fred Phelps – The church is an extension of Phelps' home, which is in a middle-class Topeka neighborhood. In one house, Phelps raised his family, and eventually the neighbors' houses were purchased, effectively creating a compound. All the houses share a large fenced backyard, which includes a full-size pool and basketball and volleyball courts. Hide Caption 7 of 10 Photos: Photos: Pastor Fred Phelps Pastor Fred Phelps – Phelps at his home in Topeka in 2008. Hide Caption 8 of 10 Photos: Photos: Pastor Fred Phelps Pastor Fred Phelps – Once an active member of the Westboro Baptist Church, 27-year-old Libby Phelps was asked to leave the congregation and family home in 2009. Libby was considered rebellious as she began questioning hypocrisies within the church, such as members being asked to pray in a form similar to Salah (Islamic method of praying) and calling to God for the death of various plaintiffs in the 2011 Snyder vs. Phelps case before the Supreme Court. The final decision was based on an intervention regarding the wearing of a bikini. The photograph Libby is holding is of her and her sister Sara, who left the church in 2012. Hide Caption 9 of 10 Photos: Photos: Pastor Fred Phelps Pastor Fred Phelps – Phelps in his home office in 2011. Hide Caption 10 of 10

JUST WATCHED 2003: Phelps on Matthew Shepard statue Replay More Videos ... MUST WATCH 2003: Phelps on Matthew Shepard statue 01:19

According to Westboro, the church has picketed more than 53,000 events, ranging from Lady Gaga concerts to funerals for slain U.S. soldiers. Typically, a dozen or so church members -- including small children -- will brandish signs that say "God Hates Fags" and "Thank God for Dead Soldiers."

Phelps was often called "the most hated man in America," a label he seemed to relish.

"If I had nobody mad at me," he told the Wichita Eagle in 2006, "what right would I have to claim that I was preaching the Gospel?"

Under Phelps' leadership, Westboro members have preached that every calamity, from natural disasters to the Sandy Hook Elementary School shooting in Newtown, Connecticut, is God's punishment for the country's acceptance of homosexuality. Phelps had advocated for gays and lesbians to be put to death.

"Fred Phelps will not be missed by the LGBT community, people with HIV/AIDS and the millions of decent people across the world who found what he and his followers do deeply hurtful and offensive," the National Gay and Lesbian Task Force said in a statement.

Phelps began his anti-gay protests in Wichita in 1991 after complaining that the city refused to stop gay activities in a public park. He rose to national notoriety in 1998, when Westboro members picketed at the funeral of Matthew Shepard, a Wyoming man who was tortured and murdered because he was gay. Phelps and his church carried signs that said Shepard was rotting in hell.

The Southern Poverty Law Center calls Westboro Baptist Church "arguably the most obnoxious and rabid hate group in America."

In 2011, the Supreme Court upheld Westboro's right to picket military funerals on free speech grounds. Congress and several states, though, have passed laws aimed at keeping church members at a distance from funerals.

In 2013, more than 367,000 petitioners called on the White House to legally recognize Westboro Baptist Church as a hate group. The White House called Westboro's protests "reprehensible" but said that "as a matter of practice, the federal government doesn't maintain a list of hate groups."

Born in Meridian, Mississippi, on November 29, 1929, Phelps had his sights set on West Point before he attended a Methodist revival. He said the sermon inspired him to enter the ministry.

"I felt the call, as they say, and it was powerful," Phelps told the Topeka Capital-Journal in 1994. "The God of glory appeared." Later, Phelps was ordained by a Southern Baptist church in Utah.

He bounced around several Christian colleges as his preaching and his theology took a hard right turn.

A Time magazine article from 1951 describes Phelps as a "craggy-faced engineering student" who harangued fellow students about the dangers of promiscuity and profanity.

Tim Miller, a professor of religious history at the University of Kansas who has studied Westboro Baptist Church, said Phelps liked to consider himself a "primitive Baptist preacher who held to the old ways."

Despite its "Baptist" name, Westboro is not affiliated with any larger church denomination. Most Christians criticize the congregation's harsh anti-gay rhetoric and penchant for pursuing the limelight at inappropriate moments.

Phelps married his wife, Marge, who survives him, in 1952. The couple moved to Topeka on May 14, 1954, the day the Supreme Court handed down its landmark decision Brown v. Board of Education, which desegregated public schools.

Phelps interpreted that as a sign and soon began a law career that centered on civil rights, winning awards for his work and praise from local leaders.

"Most blacks -- that's who they went to," the Rev. Ben Scott, president of the NAACP's Topeka branch, told CNN in 2010. "I don't know if he was cheaper or if he had that stick-to-it-ness, but Fred didn't lose many back then."

Phelps was disbarred from practicing law in state courts, however, after being accused of badgering a witness and making false claims in court affidavits. The Kansas Supreme Court said that Phelps "has little regard for the ethics of his profession."

Phelps surrendered his license to practice law in federal courts in 1989, according to the Topeka Capital-Journal, after nine U.S. District Court judges filed disciplinary complaints against him.

Most of the members of Phelps' Westboro Baptist Church are members of his large family. Phelps has 13 children; 11 are attorneys. One son, Nathan, is estranged from his father and from organized religion. He is an atheist.

Nathan Phelps posted a Facebook message March 15 saying that his father had been excommunicated from the church. Later, though, Nathan Phelps said it was "unclear" whether his father had been expelled from Westboro.

A church statement issued on March 16 said that "membership issues are private" and that eight unnamed elders lead the congregation.

On Thursday, the church added, "Listen carefully; there are no power struggles in the Westboro Baptist Church, and there is no human intercessor -- we serve no man, and no hierarchy, only the Lord Jesus Christ."

For years, Phelps joked about the possibility that his own funeral would draw protests. During a sermon in 2006, he said a CNN reporter once asked how he would feel if that occurred.

"I'd love it. I'd invite them," Phelps told the reporter, according to the Wichita Eagle. "I said: 'I'll put in my will to pay your way. But not first class.' "

But Shirley Phelps-Roper, Phelps' daughter, said Westboro will not hold a funeral for its patriarch.

"We do not worship the dead," Phelps-Roper told CNN.