A man stands up, spreads his arms wide and sings: “We love you Brian, we do.” He is instantly joined in the chant by a cluster of zealots dressed, like he is, from bobble hat to weatherproof boots in the royal blue and white livery of Sarpsborg 08 football club. And now the whole stadium, not quite 4,000 people, are adding their voices, culminating in a thundering: “Ohhhh Brian, we love you!”

There is something inescapably Python-esque about a large crowd acclaiming an unassuming guy called Brian as their messiah. But it is especially surreal to hear it in Norway, in a narcoleptic town called Sarpsborg, an hour south of Oslo towards the Swedish border. The Brian in question is Brian Deane, who, half his lifetime ago, made history by scoring the first goal in the Premier League, a header for Sheffield United against Manchester United at 3.05pm on 15 August 1992. Deane, now 46, flies lower under the radar these days, but for the past two seasons he’s been head coach of Sarpsborg 08 in Norway’s top division, the Tippeligaen.

As shown by the affection of the Fossefallet – “the Waterfall”, as Sarpsborg 08’s hardcore fans are known – Deane has done a fine job. The club has the second-lowest budget in the league and in his first season it went on a miserable run of eight successive defeats. But Deane turned it around and Sarpsborg 08 dodged relegation by winning a playoff. Local residents left bottles of wine on his doorstep in gratitude – with the price of wine in Norway, they must have been really pleased. His second season has been more straightforward: the club has just finished mid-table and had a semi-final run in the cup, too.

“This was the best season in the club’s history,” says Richard, a Fossefallet member, after the team’s 3-2 victory against Lillestrøm SK on the final day of the season. (OK, Sarpsborg 08 has only existed in its current incarnation since 2008, but still.) Richard had not yet come to terms with news that Deane would be stepping down as coach after the match. He predicts: “The new man has a long way to fall.”

Brian Deane: ‘I don’t want any favours because I’m black and I don’t want to sound like I’m complaining, because I’m not.’ Photograph: Jimmy Linus/Observer

Deane is a rare thing in football: a high-profile English manager under the age of 50. There are currently only two of these employed in the Premier League – Garry Monk at Swansea City and Burnley’s Sean Dyche. Regrettably, however, Deane is a potential member of an even more select group. There are just four black or minority ethnic (BME) football managers currently working at the 92 clubs in the Premier League and Football League – Huddersfield Town’s Chris Powell, Carlisle United’s Keith Curle, Leyton Orient’s Fabio Liverani and Burton’s Jimmy Floyd Hasselbaink. This is in stark contrast to representation on the pitch, where around 25% of players across the four English leagues are black.

How do you begin to explain this disparity in 2014? Football managers are invariably ex-players and ex-players are increasingly black; yet these individuals are, for some reason, not making the transition into coaching and management positions. According to a new report, conducted by Loughborough University’s Dr Steven Bradbury for the Sports People’s Think Tank, which was first revealed in the Observer last month, there has been “a systematic denial of talent and ambition”. Bradbury identified 552 senior coaching positions in English football and calculated that only 19 of them – 3.4% – were filled by BME candidates. This is a dismal return for a group that makes up 14% of the UK population.

Black players may no longer be shelled by peanuts or bananas on the field in Britain, but there’s hard-to-ignore evidence that racism remains prevalent in football. In 2012, Luis Suárez and John Terry both received Football Association bans for racially abusing opponents. Last year, Roy Hodgson was forced to apologise for using the expression “feed the monkey” about Andros Townsend, while Sol Campbell claimed he would have been England captain for more than a decade were he not black. These issues are complex, often disputed and potentially incendiary, but the professional game appears to have much deeper and more ingrained problems with prejudice than we readily acknowledge.

Deane has no particular interest weighing in on race or on Suárez and Terry; as a player, he was always the quiet one in the corner of the dressing room, rarely speaking out. Early on in our interview, he says: “I don’t want any favours because I’m black and I don’t want to sound like I’m complaining, because I’m not. I’d much rather see a white person stick their neck on the line and talk about these things.” He prefers to discuss what he has achieved at Sarpsborg 08 and, in fact, the tale of how a former football star from Leeds has wound up in a paper-milling backwater in Norway turns out to be an intriguing and surprisingly revealing one.

‘We love you Brian, we do’: players throw the coach in the air after Sarpsborg 08 beat Lillestrøm SK 3-2. Photograph: Nesvold, Jon Olav/NTB scanpix

Brian Deane looks out over the lake that is the imposing widescreen view from his modernist, unexpectedly tasteful-for-a-footballer living room. He had been hoping to go out fishing with a friend in his boat, but the bleak weather has conspired against them. “Couple of weeks ago, we caught 22 mackerel,” he says, eyes wide, Northern accent untamed. He puts his hands about a ruler’s length apart: “That big, they were.” In the background, Premier League football is on the telly, as it often seems to be in Norway. Today, Chelsea take on Liverpool.

Deane is well known in these parts, even before his recent spell at Sarpsborg 08, largely because of the years he played for Leeds United in the mid-90s. Norwegians have a bizarre affinity with the Yorkshire team: the fan club in the country has more than 10,000 members and up to 500 of them cross the North Sea for matches. The relationship is particularly strong in Sarpsborg, because in 1970 the local team was matched against the iconic side of Bremner, Hunter and Lorimer in the Fairs Cup, losing 1-0. Deane doesn’t think the Leeds connection got him the coach’s job, but it might have put him on the shortlist for an interview, something he was struggling to manage in England.

A rambunctious centre forward, Deane had a 21-year career that fizzled out in 2006 when he was 38, after more than 650 matches, not quite 200 goals and three England appearances. Following his last game, for Sheffield United, he walked out of Bramall Lane clutching a magnum of champagne and thought for the first time: “What am I going to do now?” Deane was part of the first generation of footballers to make serious money from the game, thanks to the Premier League, and it allowed him to defer contemplating his future. He worked as a consultant for a firm of solicitors in Leeds, but it wasn’t long before he realised he wanted to get back into football.

This is a common experience for modern professionals – especially in England, though less so in Scotland, Wales and Ireland. “You get into coaching earlier in those places because you know you’re not going to be able to make ends meet from what you’re earning,” explains Deane. “I did my coaching badges up in Scotland and I noticed a lot of the candidates were quite young. They are realistic: ‘Right, I’m going to do my coaching badges when I’m 27, 28,’ whereas, in England, the players are usually 34 or 35.”

‘It’s the captain-to-coach pathway, but historically there haven’t been many black captains’: Dr Steven Bradbury. Photograph: PR

Bradbury, a lecturer in young people and sport at Loughborough University, interviewed dozens of current and former players for his report and found that black footballers are especially disinclined to think of coaching as a potential option for when their career winds down. One, whom he prefers not to name, is a 40-something ex-pro who played more than 500 matches at Premier League and Championship level and now holds a Uefa Pro Licence, the highest coaching qualification. He has applied for 43 jobs in football clubs, but only received three interviews.

“He’s absolutely despondent,” says Bradbury. “When he was a younger man and it was becoming apparent he wanted to go into coaching, some of the older black guys said to him: ‘You’re wasting your time, you won’t get an opportunity.’ And he was saying to these guys: ‘No, no, it’s different now.’ But now he’s in the same position and he was saying to me the other day: ‘I hate to say it, but they were right. I was naive. I didn’t think these barriers existed or were as strong as they are.’ The worst thing is when people are appointed to jobs with fewer qualifications and less experience than him and he’s not even granted an interview.”

Deane certainly recognises this dilemma. “When I talk to guys I played with and I say: ‘What are you doing? Do you fancy doing your coaching badges or whatever?’ A lot of them say: ‘Pfff! What’s the point?’ Because they’ve already got a preconceived idea that they would do them and there won’t be any opportunities out there.”

He continues, referencing a comment Chelsea manager José Mourinho made in October that “there is no racism in football.” “Mourinho said recently: ‘If you’re good, you get the job.’ Fair play to him, he’s one of the best out there, but sometimes it’s too flippant to assume: ‘Go out and apply.’ It’s wrong.”

Carlisle United manager Keith Curle. Photograph: Ian Horrocks/Getty Images

When Deane decided he wanted to return to football, he thought he might pick up experience with the youth teams at one of his old clubs, Sheffield United. This proved problematic: “Because as soon as I appeared on the scene down there, people started thinking I was after their job. And I wasn’t.” He ended up becoming involved with the teams at the University of Leeds, first informally and then as director of football. They did well, winning the BUCS Premier North division in 2010, and Deane applied for a couple of jobs with professional clubs. He received a cursory rejection from one and heard nothing from the other.

“I’m not somebody who wants a slap in my face all the time,” says Deane. “So I thought: ‘Well, OK, I’ve tried to go through the process in England, what else is out there? What sacrifices am I willing to make?’ And that’s why I thought about Norway and the whole idea of coming out of my comfort zone and seeing how much I wanted it.”

Football management is clearly a very odd job. Few vocations are so precarious: the average tenure of a Premier League manager was calculated last year at just 379 days. On the plus side, though, however disastrously you perform, you are invariably guaranteed another crack, probably sooner rather than later. Every time a position opens up, a handful of the usual suspects are discussed – typically old white British guys or dashing continental types – and someone is mysteriously appointed by the club with little indication of due diligence. “You’re talking about private enterprise,” notes Deane. “You’re not talking about someone who is working for the council.”

Much of the discussion about the lack of black managers in English football has focused on the Rooney Rule: legislation in American football, enforced since 2003, which dictates that all teams interview at least one black or ethnic-minority candidate for head coach or general manager vacancies. The principal conclusion of the Sports People’s Think Tank’s report – whose aim is that 20% of coaches in professional football be from BME backgrounds by 2020 – is that some kind of “positive action” is essential for progress to be made.

The Rooney Rule has scored some notable successes: 14 “minority” coaches were hired in the first decade of its operation and the 2007 Super Bowl was contested by two teams coached by African-Americans. But progress has certainly slowed. Currently only four of America’s 32 head coaches are black and all of the coaches appointed in the past two years have been white.

Huddersfield manager Chris Powell. Photograph: Gareth Copley/Getty Images

Kieron Dyer and Titus Bramble, former players who now coach the under-16s and under-11s respectively at Ipswich Town, have both been strongly critical of the idea that similar legislation could work in English football; Bramble, fearful that quotas could become tokenism, called the Rooney Rule “a disgrace”. Deane is more open to the idea. “Just getting an interview is something,” he says. “Being able to prepare for the time when you are going to be taken seriously is essential. If you’ve got no idea what’s coming when you get in an interview, how can you prepare? How can you ever have a fair chance? The issue is more about the opportunity to believe you can actually get that job, that you have a chance. Rather than, straight away: ‘That’s not for me.’”

No one, however, imagines there will be an easy fix. One of the points made in Bradbury’s report is that a version of natural selection starts early in football. Some players are quickly identified in their clubs as “management material”, they become team captain and begin a fast-track ordination that often ends in them becoming involved in coaching. Club captains are also overwhelmingly white.

“It’s the captain-to-coach pathway,” Bradbury explains. “But there haven’t been many black captains and in part there’s a historical thing that captains have been central midfield players or central defenders. And, of course, when black players first came into the game there was an issue of stacking them in forward or wing positions, which were seen as instinctive or natural. It was the racialised stereotype that black players were innate, physical players without having cerebral or organisational skills. These were certainly the stereotypes in the 1970s and 1980s, but they have been hard to shift.”

Deane recognises the description. “I’ve heard it for too long when I growing up,” he says. “All the usuals: ‘Oh, he’s athletic, he can play on the wing. He’s too laid-back. He’s lazy.’ All that kind of crap. I’ve heard people say it in the past few years about black players and I think: ‘Well, you don’t really understand black players if that’s what your attitude is.’ They have the same goals and ambitions as everyone else.”

In the informal, qualification-hazy world of football management, it’s always been about who you know, not what you know. Club captains spend more time with their managers and perhaps even the chairman and chief executive. Already they are building a network that will serve them well in retirement. “There’s often unconscious racial bias,” says Bradbury. “A chairman might not hold any stereotypes, but he’s going with what he knows – and what he knows, by virtue of his own networks, are other white people.”

Brian Deane could do with a job now. At Sarpsborg 08, he clashed with the sporting director, his boss, but – rare for a coach – he moves on from the post with his reputation enhanced. On the evidence of the match against Lillestrøm SK, Deane likes his team to play neat, technical football and he has proven he can meld together a cohesive unit on a shoestring budget. He is not optimistic that his work will have caught the eye of any chairmen in England’s Football League – though he does admit: “I’d be disappointed if, at some point in my life, I don’t manage Sheffield United” – but he has been in discussions with a couple of clubs in the Tippeligaen to stay in Norway.

Deane talks a lot about being “creative” and it’s a succinct word to describe the main quality that former players need to thrive once their salad days are over – and before anything like the Rooney Rule is introduced in Britain, if it ever is. David James, the former England goalkeeper, started his coaching career at ÍBV Vestmannaeyjar in Iceland, sharing a volcanic rock off the south coast with 8m puffins. He is now player-manager of Kerala Blasters in the Indian Super League. Enough ex-professionals, both white and black, are leaving Britain for the boom markets of the US and Canada for Bradbury to worry about “a talent drain”.

The last I see of Deane, he is doing a lap of honour. When he reaches the stand where the Fossefallet faithful sit, his players bundle him over and give him the bumps. The crowd whoops raucously each time Deane is flung in the air. It seems an appropriate way for him to bow out. Heading for the stars and then coming down to earth with a jolt.