The U.S. is pressing for European Islamic State militants captured in Syria and Iraq to be sent back to their home countries in an effort to ensure they don’t return to the battlefield.

One problem: Europe doesn’t want them.

As the U.S.-led coalition has rolled back Islamic State, European militants have retreated to Northern Syria, only to...