Last week, a self-described heir to the Prophet Muhammad declared himself the supreme leader of a new Islamic state stretching from eastern Syria to northern Iraq. How did Abu Bakr al-Baghdadi, the nom de guerre of a mediocre Iraqi religious scholar in his mid-40s, outmaneuver al Qaeda as the new vanguard of jihadist ideology? How did he and his followers—armed with Kalashnikovs, smart phones and their ominous black banner—so suddenly take over the campaign to rid the Muslim world of Western and secular influence?

The rise of Mr. Baghdadi and his newly proclaimed "caliphate" highlights what had been a closely held secret of the Sunni jihadist movement: a split in the ranks that had been festering for years. It pits a new generation of shock troops hardened by battle in Iraq and Syria against al Qaeda veterans who had built the movement but were increasingly seen as too passive, both politically and theologically.

Mr. Baghdadi's proclamation was stunningly brazen. The leader of a faction of puritanical Sunni militants who have plagued Iraq with suicide bombings and beheadings, he was long considered a relatively inconsequential cog in the larger al Qaeda machine. Few people outside jihadist circles had heard of him, let alone seen him, before last month, when his followers in the militia known as the Islamic State of Iraq and al-Sham, or ISIS, rolled across northern Iraq, conquering Mosul, one of the largest cities in the country. Then on July 4, Mr. Baghdadi emerged at Mosul's al-Nuri Grand Mosque, promising to restore to his Sunni brethren their "dignity, might, rights and leadership," according to a video of the sermon distributed by his group.

Mr. Baghdadi's military offensive has startled the U.S. and its Middle Eastern allies, who fear that it portends prolonged regional instability and terrorist attacks far afield from Iraq. Yet for the man leading what he now calls simply the Islamic State, the latest campaign has meant more than territorial conquest. Mr. Baghdadi's victories also mark the crescendo of a 10-year theological battle between veterans of al Qaeda, the core organization started by Osama bin Laden in the 1980s and now led by the Egyptian-born extremist Ayman al-Zawahiri, and its rebellious affiliate in Iraq, which Mr. Baghdadi took over in 2010. The prize: purported leadership of the world's estimated 1 billion Sunni Muslims and of a jihad supposedly waged in their name.

The rupture between Mr. Zawahiri's old guard and Mr. Baghdadi's new guard escalated last year, when Mr. Baghdadi refused Mr. Zawahiri's demand to formally declare his obedience and instead called the al Qaeda leader's rulings antithetical to God's commands. It was an audacious snub within the puritanical circles of al Qaeda and its fellow travelers. It was also the start of a slow-moving coup against the established jihadist hierarchy.