FRANKFURT—German prosecutors are investigating whether Russia's main nuclear exporter broke European rules by routing Iran-bound cargo through Europe—an incident that diplomats say turned into a major battleground as world powers hashed out international sanctions against Iran.

The shipments wouldn't break sanctions agreed on at the United Nations, including new rules approved Wednesday. But the U.S. and European Union plan to use their own regimes to put bigger obstacles in front of Iran's nuclear development—and to make...