In the years since 9/11, there has been much talk about “the problem with Islam”. Part of the problem, obvious to anyone who follows the news, is that a very small number of people who like to blow up buildings and sever heads do so in the name of Islam. As if the link between violence and religion was now proven (it is not), the current occupant of the White House wishes to restrict the movement of certain Muslims into the US. If you have a historical view of Islam, you will understand the irony in this because a little more than 100 years ago, many Muslims were seen as sensual, mystical and exotic.

You won’t find much of those three qualities in The Way of the Strangers. Graeme Wood’s book does what it says in the subtitle and offers a series of “encounters with the Islamic State”. Well, not quite the state itself, because the chance of becoming another orange-suited sacrifice deters most western journalists from travelling there. Instead, Wood talks to the state’s supporters and enthusiasts in Cairo, London, New York and elsewhere.

The media has taken to calling Daesh “the so-called Islamic State”, which carries the suggestion that the state is not really Islamic. Many of us assume that people who have rallied to the black flag have done so for reasons other than religion: because it gives them a purpose, because they like to kill, because they perceive an injustice. While these motives might be part of what has persuaded people to join the battle, many Daesh fighters really do seem to believe they are part of an epoch-defining movement that will bring Islam its rightful prominence. That position is stated very bluntly in the opening pages, where Wood reproduces a schematic diagram of “The Islamic State View of Humanity”. Those who have pledged themselves to the state are at the top of the page, close to heaven, while infidels sit at the bottom along with secular humanists, Christians and Yazidis, all of them “eligible to be enslaved”.

Colonialism and western greed for natural resources did much first to slow and later to sour the modernising project

Wood has extensive knowledge of both the people and the issue. In 2004, he worked for a courier company in Mosul, Iraq, while in March 2015 his article What Isis Really Wants, published in the Atlantic, was read by more people than anything in the magazine’s 160-year history. What Isis wants, he explains in greater length in this well-judged book, is a religious state that would look familiar to the prophet Muhammad, regulated in the way that they believe society was regulated in the seventh century. What becomes clear from Wood’s interviews is that for many supporters of Abu Bakr al-Baghdadi, the self-styled caliph, the end justifies the means and those means, however brutal or brutalising, can always be justified by Islamic texts or traditions. Hard to argue with that.

And that is part of the problem that we all now face, the lack of direct argument or discussion. On the one hand, there are the pronouncements delivered by Daesh’s social media-savvy converts, and on the other, the many words of those who oppose them. But that lack of dialogue, especially between Muslims, as in the Sunni/Shia split, has led many commentators to regard what is happening now as the Islamic equivalent of a Reformation. Christopher de Bellaigue’s new book argues that this has already happened. The Islamic Enlightenment looks for evidence in the Middle East, in particular in Egypt, the Ottoman empire and Persia.

De Bellaigue follows tradition and identifies Napoleon’s 1798 invasion of Egypt as the catalyst for the revival of the region and of its predominant religion (there are other possibilities). When the French arrived, their soldiers were accompanied by scholars, who documented everything they saw, established the region’s first printing press and created the sort of learned institutions that were familiar in Enlightenment Europe. When the Egyptian chronicler Al-Jabarti met the French scholars, he immediately understood the huge gap between his own formalised way of thinking and that of the argumentative French. Muhammad Ali, the Albanian officer who set himself up as ruler of Egypt after the French left, had also understood the power of European advances. By sending Egyptians to Europe and inviting European soldiers, engineers and scholars to Egypt, he ushered in a period of renewal where, slowly, and particularly among the more privileged, ways of thinking, of behaviour and of government were transformed.

De Bellaigue has reported as a journalist from Iran and Turkey, and his telling of this renaissance as it manifested itself in Tehran, Istanbul and Cairo is as thorough in research as it is rich in detail. Framing this as a struggle between conservative faith and liberal reason, he ambitiously and successfully blends stories of the 19th-century Arab awakening with the opening up of Ottoman and Persian societies, although each country had its own specific issues, its own traditions that needed to be overcome, its particular appetites for modernity. De Bellaigue also shows how the perfidious nature of colonialism and western greed for natural resources did much first to slow and later to sour the modernising project. Whether the result was an Enlightenment remains to be seen.

None of those Wood met, nor those Samer describes as fighting for the caliphate, would agree with Ramadan

But for now, there is much darkness, with few places darker than inside the territories occupied by Daesh, if the account in The Raqqa Diaries is anything to go by. The anonymous author, writing under the pen name Samer, shares snapshots of life under the occupation in Raqqa, his Syrian home town. To read of the horror of random brutality – mothers humiliated, lovers stoned, others lashed, many beheaded and all in the name of Islam – is to feel the creeping inexorability of despair. The moment when even this bright young man decides to leave his home, mother and siblings and find refuge in a camp is terrible to share. The horrors are what you will have imagined if you have been paying attention, while Samer’s decent, rational determination not to be brutalised is a cheering surprise. As is the comment of Raqqa’s muezzin, who tells the newly arrived intruders, busy harassing a woman, that “you have nothing to do with Islam”.

All this might leave you wondering whether the third religion of the Abrahamic tradition is bloodcurdling or blood-warming by nature. If that is the case, then you need look no further than Tariq Ramadan’s pocket‑sized Islam: The Essentials. Ramadan was long banned from travelling to the US and, as grandson of Hassan al-Banna, founder of the Muslim Brotherhood, will probably soon be banned again, in spite of also being professor of contemporary Islamic studies at Oxford. Curtailing his movements would be unhelpful, for he is an eloquent advocate of an enlightened Islam. In a concluding section entitled “10 Things You Thought You Knew About Islam”, he sets the record straight about sharia (nothing to do with “repressive punishments”), polygamy (“the rule… is clearly monogamy”) and a number of other pressing issues including jihad, a word that Ramadan reminds us means “effort” and only relates to war in the specific case of a response to colonisation or aggression. None of the people Graeme Wood met, nor those Samer describes as fighting for the caliphate, would agree with Ramadan’s reading of the texts or his interpretation of the sayings of the prophet. And this and the other three books all show, in their own ways, that after 1,400 years of discussion and violent disagreement, attempts to define how to live a good life in Islam are still ongoing.

Anthony Sattin is the author of several books about the Middle East and Africa, most recently Young Lawrence: A Portrait of the Legend as a Young Man.

• The Way of the Strangers: Encounters with the Islamic State by Graeme Wood is published by Allen Lane (£20). To order a copy for £17 go to bookshop.theguardian.com or call 0330 333 6846.

• The Islamic Enlightenment: The Modern Struggle Between Faith and Reason by Christopher de Bellaigue is published by the Bodley Head (£25). To order a copy for £18.75 go to bookshop.theguardian.com or call 0330 333 6846.

• The Raqqa Diaries: Escape from ‘Islamic State’ by Samer is published by Hutchinson (£9.99). To order a copy for £7.49 go to bookshop.theguardian.com or call 0330 333 6846.

• Islam: The Essentials by Tariq Ramadan is published by Pelican (£8.99). To order a copy for £7.64 go to bookshop.theguardian.com or call 0330 333 6846.

Guardian Bookshop offers have free UK p&p over £10, for online orders only. Phone orders min p&p of £1.99