Lawrence Wright, whose Pulitzer Prize-winning 2006 masterpiece “The Looming Tower” traced with rare perception the tortuous strands that pulled together the worst terrorist attack in American history, has been reporting and writing pieces on the Middle East for The New Yorker for nearly a quarter of a century. Wright has profiled the key players, detailed the major events, and tried with an almost forlorn kind of insistence to make sense of the senseless.

His latest book, The Terror Years, collects a dozen of those New Yorker pieces, along with bits of material drawn from elsewhere (including his two one-man stage plays) and fleshes things out a bit with new postscripts, presenting a large amount of vital prose to a bookstore audience that might not have seen it all before.

This year marks the 15th anniversary of the 9/11 attacks, and the pieces assembled in “The Terror Years” draw heartbreaking contrast between the rosy neocon predictions made in the wake of those attacks – a Middle East scoured of terrorist networks, where native populations welcomed American troops as liberators and embraced democratic reforms – and the brutal, squalid reality that has existed for the past decade. In Wright’s accounts in these pages, US counterterrorism experts are scrambling to piece together scattered reports of Al Qaeda activity in the months preceding 9/11 (who is this guy Osama bin Laden whose name keeps cropping up in the strangest places?), and onlookers and victims at bombed embassies and fallen towers are looking out on a world in the violent convulsions of change.

We all live in that ugly new world. A massive coordinated Islamic terrorist attack that would once have commanded headlines for a year is now the story of a month at most, before it’s pushed aside by some new atrocity. In Wright’s darkly magnificent piece “Five Hostages,” included here, he describes in wrenching detail what was in many ways the watershed event in the birth of that new world: the televised beheadings of kneeling journalists James Foley and Steven Sotloff by Islamic State in 2014. Wright might easily have found himself kneeling where those men knelt; Foley at one point articulates the credo of such writers: “I think it’s just the basic laws of competition; you need to have something the staffers don’t, but in a conflict zone that means you take bigger risks: go in sooner, stay longer, get closer.”

Throughout his career, Wright has certainly done that; it slowly dawns on you, reading through these New Yorker pieces, just how much danger their author had to court in order to write them. We see him interviewing women in misogynist Saudi Arabia in the excellent “The Kingdom of Silence,” risking the intervention of the religious police. (About women, one Saudi journalist acidly writes, “[W]e doubt them and confine them because we think they are the source of all seduction and evil in the world. And then we say proudly: ‘We are Muslims.’ ”)

We follow him in “The Master Plan” as he profiles Al Qaeda leader Abu Musab al-Zarqawi. He takes us to war-torn Gaza in the wake of Israel’s 2008 Operation Cast Lead, when a Gazan militia leader speaks scornfully of “fancy restaurants on the beach” and in speaking of Hamas actually summarizes the whole divide between Islamic fundamentalists and the rest of the world, between the Dar al-Islam and the Dar al-Kufr: “They have a much more moderate way of life, and we cannot deal with that.”

These pages are not without their heroes and visionaries. Wright profiles such counter-terrorism figures as Richard Clarke and also John O’Neill who, early on, developed an obsession with bin Laden that struck some of his pre-9/11 colleagues as a bit extreme. (“He devoted himself to trying to understand this new enemy in the darkened mirror of human nature”).

And there are also glimmers of hope here and there throughout this largely hopeless book; Wright remembers, for instance, a late-night music session in Jeddah, Saudi Arabia, with some Arab friends of both genders, all of them laughing and enjoying the nighttime breezes from Africa – a vision of what Saudi Arabia (and by extension many other places) might be if they were not in the grip of religious fanaticism.

But such visions always fade quickly in this collection, and they wither outright when exposed to the morning news, or to a world in which jihadists view the torturing and killing of innocents as their moral duty. For such people, torturing someone may be “morally distasteful,” but they’ll do it nonetheless, because their interpretation of their faith demands it. The inescapable conclusion of Wright’s book is that those fanatical undercurrents have grown much stronger in the past 15 years and will only continue to grow.

Get the Monitor Stories you care about delivered to your inbox. By signing up, you agree to our Privacy Policy

The worst part of Wright’s great book is the understanding that the terror years could be with us for a long time to come.

Steve Donoghue regularly reviews books for The Christian Science Monitor.