On a chilly Thursday evening, 400 people have crowded into a vast, open-sided barn in rural Warwickshire to hear the archbishop of Canterbury speak. The warm-up act is a young man with a guitar who performs a song about Hobnobs and custard creams, which may or may not contain a hidden evangelical message. The audience – skewed towards the over-50s, with a preponderance of Barbours and flat caps – listen politely while chomping on their hog roast baps; they haven’t come for the biscuit song.

To one side of the stage, Justin Welby waits, barely noticeable in his customary black clerical shirt and dog collar, his head bowed. When his turn comes, he starts in self-deprecating mode, musing on his job as leader of the Church of England and the global Anglican communion: are you supposed to be a person who dresses up in funny clothes offering spiritual guidance, or a sort of super civil servant, running one of the country’s oldest institutions?

Earlier on this four-day tour of the area where he was first a parish priest, Welby tells a group of farmers that he has written “a very boring book” about the future of Britain, and an audience of sixth-formers that he was “rubbish at studying, but I could blag my way through exams”. At a primary school, he is asked how he came to be archbishop of Canterbury; “I don’t think anyone else wanted the job,” is his reply. A few minutes later, Ronnie from reception asks about the royal wedding: Welby is officiating at today’s ceremony. “I’m really, really excited,” he replies, “but also really, really nervous about dropping the rings” – which he did at the last wedding he took, that of his communications director – “and everyone will think: what an idiot.”

After each event, he climbs back into the minibus transporting him and his small entourage (his wife, Caroline, plus his chief of staff, chaplain and head of media) and asks, “Was that OK?” They reassure him; he reflects on the conversations he has shared and occasionally points out of the window to exclaim, “I used to have a pint in that pub!” or “I know that man!”

***

Justin Portal Welby has been archbishop of Canterbury for five years, and intends to continue for another eight – longer than most of his predecessors, though Randall Davidson lasted 25 years until he resigned in 1928. The job is “very hard to pin down”, but “at its heart, the role is to be visibly someone who is committed to following Christ, and who encourages others to follow Christ, and who encourages the church to encourage others to follow Christ”.

Facebook Twitter Pinterest The job is ‘very hard to pin down’. Photograph: Sebastian Nevols/The Guardian

It’s not immediately obvious how that plays out in a series of meetings and events with business leaders, hospital managers, farmers and teachers in and around Coventry, where Welby lived and worked for 15 years after he was ordained, and to which he clearly has a strong attachment. On the last morning of his tour, he bubbles with emotion and enthusiasm as he shows me Coventry Cathedral: the ruins of the 14th-century cathedral, bombed by the Luftwaffe in 1940, and the dazzling modernity of the postwar edifice. Some see his real-world grit and pragmatism, combined with passionate evangelism, as his greatest strength; his critics argue that the former oil executive’s managerial approach to the church – targets, training, organisational reform – risks obscuring its mystery and traditions.

But today, Welby will play his part as the country’s foremost spiritual leader, wearing his full archbishop regalia (“I hate dressing up,” he tells me) as he takes Prince Harry and Meghan Markle through their marriage vows before a global audience of millions. American bishop Michael Bruce Curry, who has not met the couple, will give the address. Welby, meanwhile, has met them several times, baptising and confirming Markle into the Anglican church in March.

He declines to disclose what private advice he has given the pair as they embark on married life, saying only that he tells all brides and grooms that communication is crucial. “You talk [to them] about communicating when things go well, when you’re under pressure, when you’re tired, when things are tricky, when you’re fed up. You do that with every couple.”

Officiating at royal weddings, state funerals and coronations is a major part of his job as the leader of England’s established, or state, church. But to many, the official status of the C of E is an anachronism in an increasingly multifaith and secular society, in which only 7% of young adults identify as Anglican. Welby concedes that the national institution sometimes seems detached from local church communities who provide night shelters, food banks, meals for children in school holidays and other essential social services. Of the two, he says emphatically, the grassroots church is more important.

Facebook Twitter Pinterest Female priests on the steps of St Paul’s Cathedral with the archbishop of Canterbury, in 2014. Photograph: Getty Images

Disestablishment – separating the church from the state – is mooted from time to time. “Would it be a disaster? No,” he says, adding, “Nothing is a disaster with God.” Establishment is “a conglomeration of different bits of history. There’s no Establishment of the Church of England Act that you could repeal – it’s a complicated process. And if you mean, by privilege, that the archbishop of Canterbury is often involved in royal weddings, or crowns the monarch, or whatever, that’s really a decision for parliament and the people.”

But neither would disestablishment be liberating for the church. “It wouldn’t make any of that [the grassroots social action] easier, as far as I can tell, because that’s all done at a local level. We’re an incredibly delegated, dispersed organisation. All of those things happen because local Christians reach out to those around them, with other faith communities, with those of no faith; they do all that because they follow Christ. So I don’t think [disestablishment] would make it easier, and I don’t think it would make it more difficult.”

A consequence of establishment is that the UK is one of only two countries in the world that reserve seats in their legislature for clerics, the other being Iran – a fact relayed with some relish by Welby to a group of business leaders at Warwick University. But in contrast to the Iranian parliament, the 26 Lords Spiritual in the UK’s upper house now include two (soon to be three) women, who are among more than a dozen appointed as bishops since the church made a historic change to canon law in 2014 – a move championed by Welby.

“If I look back over the past five years, at what’s been achieved in the Church of England, the most significant would clearly be the ordination of women to the episcopate. Am I delighted it’s happened? I’m more than delighted, and I’m even more delighted that, since it became possible in law, about half the bishops that have been appointed are women.” He would like to see a woman take over as archbishop of Canterbury at some point, he says.

Since it became possible in law, about half the bishops that have been appointed are women

But his strong advocacy of gender equality in the church – an issue that once caused rancorous division (and still does in pockets, including some parts of London, whose new female bishop, Sarah Mullally, has just been installed) – hasn’t been matched by a clear endorsement of the campaign for LGBT equality and same-sex marriage, the latter forbidden by the church despite being sanctioned by parliament almost five years ago. Yet some of those campaigning for equality say that, since taking up residence in Lambeth Palace, Welby has become privately more open and sympathetic to their case.

They say he has been on a journey, I tell him. “Do they? How interesting,” he laughs. “Well, we’re all on a journey. Are my views different from 25 years ago? Yes.” Are they different from five years ago? “I hope they’re deeper. I’ve not changed in the sense that I believe the scriptures, properly interpreted, remain for me the final authority in matters of doctrine, in matters of practice. But the phrase is ‘properly interpreted’. And we are continuing as a church to debate with each other, to discuss with each other, and to listen particularly to the voices of LGBTIQ+ people, who historically have been appallingly treated and never listened to. You talk to [the bishops leading the C of E’s work on the issue] and I think they would say the whole church is on a journey. And I’m part of that.”

The church is now facing a new demand to embrace transgender people in its congregations and as clergy. “It’s an issue we have to listen to very carefully. But it’s unbelievably complex. Even within transgender communities, what it means is something that people debate and discuss – and the last thing I want is the church to weigh in with its size 14s, saying: ‘This is the answer and you will now do it.’ I mean, you’ve got to be more on receive than transmit.”

Facebook Twitter Pinterest At Buckingham Palace in 2013. Photograph: Getty Images

Another crisis issue that has dominated Welby’s tenure is the sexual abuse of children and vulnerable adults. Giving evidence to the national child abuse inquiry in March on the way the C of E handled cases in Chichester, mainly in the 1970s and 80s, Welby said he was “ashamed of the church”; he has also said the issue keeps him awake at night. He tells me he is “conscious of the possibility” that the damage to the church’s reputation could worsen with further inquiry hearings this summer and next year. But, he adds, “I think that the worst of all things that we could do is worry more about our reputation than the truth. To fail in safeguarding children and vulnerable adults is the absolute denial of what we are called to in Jesus Christ. And to privilege the institution over that is utterly shameful and wrong and must never be allowed to happen.”

On his watch, there has been tangible change: every diocese now has a safeguarding officer; it is a disciplinary matter not to report safeguarding issues; the church has invested millions in training; and “nobody gets appointed as a bishop unless their approach to safeguarding has been questioned and found to be satisfactory”.

The way to protect the church’s reputation, he says, is by “being utterly truthful and transparent and honest. No cover-up. Justice for complainants and justice for those who are complained against. We have to love and care for those who are the vulnerable and weak in this. We have to find – and, of course, we haven’t got there yet – ways of acting rightly on this.”

I don’t know why I have a temper. I am much better than I used to be

His determination that there should be no cover-up has led to an acrimonious confrontation with those who feel Welby unjustifiably trashed the heroic reputation of bishop George Bell, a revered figure in the 20th century, after allegations were made of sexual abuse dating back six decades. Lord Carlile, who last year carried out an independent review of the church’s handling of the claims, concluded that Bell had been “hung out to dry” and the church had “rushed to judgment”. Welby refused to back down, however, saying a “significant cloud” was left over Bell’s name. The poisonous row shows no sign of going away, and may yet cause Welby real difficulty.

But Welby is relaxed and charming on his tour of the diocese of Coventry, which starts each day with a breakfast meeting and ends around 10pm. He listens closely to those he meets, conducts several church services with fervour and humour, cuddles a newborn baby in a hospital and a newborn lamb on a farm; he greets old friends with hugs and jokes. He isn’t always like this; he tells me he is often “very grumpy” and has confessed to periods of “black dog” depression. And he is known to have a fearsome temper. People in his circle sometimes allude to bouts of anger; last year Welby told GQ magazine: “I don’t know why I have a temper. I am much better than I used to be.”

***

Welby’s extraordinary personal backstory, combining privilege and familial chaos, bears retelling. His mother, Jane Portal, served as Winston Churchill’s private secretary before marrying Gavin Welby, a German-Jewish immigrant described by Jane as a bully. The pair, both alcoholics, divorced when Justin was three; the boy shuttled between his parents and grandparents in Norfolk until he was packed off to Eton. On Desert Island Discs in 2014, Welby described his childhood as “grim” and “messy”, saying his father “was always unpredictable. Sometimes very full of rage and anger, and expressing that very loudly. It felt very painful at times. But I didn’t know anything else.” He tells me his biggest regret is “not having had a better relationship with my father – the person I thought was my father”. In an astonishing twist, DNA tests two years ago revealed that Gavin Welby wasn’t Justin’s biological father after all. In fact, he is the son of Sir Anthony Montague Browne, an aide to Churchill with whom his mother had a drunken fling days before she eloped with Welby senior.

Facebook Twitter Pinterest Welby with bishop Angaelos, archbishop Athanasius Thoma Dawod and archbishop Gregorios after a meeting at Lambeth Palace in London, 2014. Photograph: Luke Macgregor/Reuters

The boy who went on to be archbishop found God when he was a student at Cambridge, after which he spent 11 years in the oil industry, eventually becoming treasurer of Enterprise Oil, later taken over by Shell, a background that gave him the hard-edged realism he has brought to the C of E. In the 1980s, he worshipped at Holy Trinity Brompton – a charismatic evangelical church in London dominated, according to its critics, by affluent and overbearing high-flyers – eventually giving up his lucrative job to become a priest.

In 1983 the Welbys’ first child, Johanna, was killed in a car crash in Paris aged seven months. He says he has struggled with forgiveness. “I thought we’d sort of forgiven and dealt with that, but I realised a few years ago that I hadn’t. I’d just shelved it. It revealed to me the depths of how one can deceive oneself. The costly road to forgiveness, that pain and agony, is something we have to tread if we are to get to a point where we find reconciliation.”

The product of this complex and troubled background is a liberal-leaning religious leader who understands the realities of modern British society, fractured families, loneliness and loss. He sent his five children to state schools, a deliberate political choice. He publicly declared his intention to vote remain in the EU referendum. He insists the C of E must represent all the people of the country, whatever their faith or lack of faith – although some critics say that under Welby’s leadership it increasingly focuses on a narrow evangelicalism.

He tells a group of sixth-formers in Southam that he is political, but not partisan. “One of the things I struggle with is wanting to speak the truth, but at the same time not to be party political. Obviously, if we had someone like Hitler in this country, I’d personally speak against them. But in normal circumstances, I’m very careful never to speak against individual politicians.”

When I have private meetings, first of all they are private, but secondly they are direct. They are polite but direct

He tells me he would like to meet Donald Trump when the US president visits the UK in July. If he thought Trump’s policies were morally questionable, would he let him know? “When I have private meetings [with political leaders], first of all they are private, but secondly they are direct. They are polite but direct.” So does he speak truth to power? “Well, I’m meant to, aren’t I? I try to. I come away from some meetings kicking myself for not having done well enough – and very occasionally I think, OK, I did it.”

For all his evangelism and determination to reform the church, Welby doesn’t seek to deny the C of E’s woeful decline. The church expects its numbers to continue to fall for 30 years, by which time only 1% of the UK population is likely to attend services regularly. When he became archbishop, Welby pledged to turn this around. Is it time to admit defeat?

“No! After five years? We’ve been declining for 70 years. This is a generational thing. That would be ultra short-termism. We have to collaborate and cooperate with the spirit of God, we have to be confident about the good news of Jesus, we have to live it out, we have to serve and bless our communities together. We have to stop arguing, and we have to pray. In the grace of God, I believe that over time, at some point, it will turn around. While there’s one Christian in this country, it’s not time to stop talking about that.”

Facebook Twitter Pinterest With the Duchess of Cambridge and Prince Harry during a ceremony at St Symphorien Military cemetery, Mons, Belgium, in 2014. Photograph: Rex/Shutterstock

So he’s optimistic? “I’m hopeful.” Clearly he believes in miracles; it may need one to save the Church of England.

On the bus, trundling through the Midlands, Welby occasionally catnaps. Most days he gets up at 5am; it’s the only time he can find the peace and solitude to pray. His diary is full; there is little time for relaxation. But he reads thrillers and Shakespeare, watches “political stuff” and Midsomer Murders on TV, and goes to the cinema in civvies. He spends as much time as possible with his adult children – all five are believers, he once told me, “nothing to do with me” – often arguing about politics around the dinner table. Caroline accompanies him on tours (and does a few on her own) and looks out for him; in Jerusalem a year ago, she unsuccessfully urged Welby to get a much-needed early night rather than share a bottle of wine with me and a couple of his aides.

But mostly he works. “I have a limited amount of time, but I stand on the shoulders of those who have gone before me, and people will stand on all their shoulders after me. The church [of England] has been around for over 1,400 years, through numerous invasions, a number of revolutions, decapitations, you name it.” Another eight years is not long for all he wants to do, but “you have to retire from this job at 70, and that’s the date that is there in my mind”.

What will his legacy be? “I genuinely don’t know, but people have started asking, so I’m wondering how old I’m looking. I hope a church that prays, that worships, that serves the poor.” The question is whether that’s enough, or if the process of atrophy is irreversible. Today’s royal wedding is the last for a generation; by the next time around, Welby will be long gone from Lambeth Palace, and the church may also have joined the annals of history. Welby is taking it one step at a time. He picks up his briefcase (“it’s only got my jumper in it”) and heads into the cavernous space of Coventry Cathedral, ready for another day in God’s service.

• Commenting on this piece? If you would like your comment to be considered for inclusion on Weekend magazine’s letters page in print, please email weekend@theguardian.com, including your name and address (not for publication).