Americans don’t hear much about Bangladeshi terrorists. The vast majority of high-profile terrorist plots targeting the U.S.—successful or unsuccessful—have turned up protagonists with links to familiar trouble spots: the Arab world, Afghanistan and Pakistan, Russia’s autonomous Chechen Republic.

But earlier this month, Akayed Ullah, a 27-year-old immigrant from Bangladesh, allegedly tried to blow himself up in a crowded New York subway tunnel using a crude bomb assembled from match heads and a piece of pipe. The bomb failed...