LONDON — Throughout decades of ferocious rivalry, Iran and Saudi Arabia have, even while backing competing forces across the Middle East, generally maintained one red line: They wouldn’t interfere directly in each other’s domestic security. Policy makers in Riyadh and Tehran have known that backing militant groups among their rival’s Shiite minorities in Saudi Arabia or Sunni minorities in Iran could lead to an escalation for which neither country is ready.

But that tacit agreement might be unraveling.

While no hard proof has been presented, in the past month Iran has ratcheted up claims that Saudi Arabia is supporting groups working to overthrow the government in Tehran or to destabilize the country, in particular opening a new front with Kurdish separatists. The Saudis, for their part, say Iran is increasing support for Shiite militants.

Both countries deny the allegations. But given the total breakdown in diplomatic relations since January, and in an increasingly volatile region, it isn’t hard to imagine this tension morphing into something much more dangerous: a tit-for-tat exchange of attacks carried out by domestic armed groups. The fact that Iran’s Sunnis and Saudi Arabia’s Shiites have suffered periods of widespread arrests and their communities have been largely excluded from political and economic opportunities has created fertile ground for militant groups eager for foreign backing.

Saudi Arabia has long accused Iran of attempting to export its revolutionary ideology. For this reason, the Saudis have tried to dissuade links between its Shiite community and Tehran. The execution in January of Sheikh Nimr al-Nimr, a prominent Saudi Shiite cleric who was accused of “seeking foreign meddling” and inciting violence, was partly motivated by a desire to deter Shiite groups from turning to Iran for support. Weeks later, Saudi Arabia tried 30 Shiites detained in 2013 on charges of espionage for Iran and provoking sectarian divisions.