Unless you're an obsessive fan of Kicks Like a Mule's 1992 rave hit the Bouncer – and let's face it, that leaves about three of you – then chances are you won't recognise the man sat sprawled across a big brown leather couch in his Ladbroke Grove offices. If, however, you are a fan of innovative new music, then you probably should. For this is Richard Russell, the boss of XL Recordings, a label that has bucked the trend for gloomy music industry stories over the past decade by breaking a stream of commercially successful yet critically drooled-over artists.

The Prodigy, Dizzee Rascal and MIA have all made music on XL, showcasing exactly what the label does best – taking innovative music into the mainstream. Whether it is Dizzee's success opening the doors for the current grime/pop success of Tinie Tempah et al, or Vampire Weekend heralding the indie scene's recent obsession with Afropop, XL likes to break new artists and build the foundation blocks of an emerging scene as it does so. XL was also represented at Tuesday night's Brit awards with nominations for Vampire Weekend and the xx, not to mention a show-stopping performance from Adele, whose second album, 21, has just sold a million copies worldwide. Such a presence might suggest that XL is an establishment big-hitter, competing on a scale similar to, say, Sony or EMI. But it comes as something of a shock to learn that XL only puts out some half a dozen albums each year.

"The small release schedule is fundamental to what we do," says Russell of the label he helped launch in 1989. "In the past, we veered off it, and our eyes went off the quality control a bit. Once I'd recognised that, I made it a real mission to only do things we could commit to. There's a natural momentum towards doing more, and I battle that all the time."

Doesn't he want the company to grow? He shakes his head. "We get offered 200,000 unsolicited demos a year and yet only sign about one artist a year. We're basically saying no to everything, lots of big artists as well. You need an element of fearlessness to do that. It's basically an anti-business philosophy."

Certainly, XL's history does not read like that of your average FTSE 100 company. For a start, its origins were in the rave scene, putting out records such as SL2's On A Ragga Tip and the Prodigy's Charly. Simon Reynolds, author of rave bible Energy Flash, reckons XL's hardcore [rave] output alone would make it "a legendary label", yet Russell refused to be pigeonholed. After his partners left him in sole charge of the label in 1994, he built on the massive success of the Prodigy (their third album, The Fat of the Land, went to No 1 on both sides of the Atlantic) and diversified, signing acts that have set the agenda for much of the past decade and a half of music – from Dizzee Rascal to the White Stripes via MIA, Badly Drawn Boy, Basement Jaxx, Thom Yorke, Devendra Banhart and Peaches.

It's a staggering list that makes you wonder why XL doesn't get more recognition. After all, music industry stories are so gloomy right now: album sales dropping for six years on the trot; HMV due to close 60 UK stores; the decline in physical sales still not being plugged by the rise in download sales (combined sales down 7% overall). Elsewhere, EMI, the jewel in British pop's crown, was recently taken over by those well-known pop lovers, Citigroup.

Why doesn't a small company that thrives despite this bleak environment get a bit more credit for being a Great British Success Story? "Because it's a company, and who's interested in companies," says Russell. He would sooner the focus was on the artists and, to be fair, his artists get plenty of attention. So what's the secret? Instinct, it has to be said, plays a big part – Russell has an uncannily good ear for spotting original sounds that also have a chance of shifting units. He is a charismatic man, too: eager to listen, keen to learn and owner of an enthusiasm that shines in his eyes. Occasionally, he breaks into a manic grin. Before we start the interview, he is raving over vintage Russian synthesisers and the possibilities of the iPad as an instrument.

But there is more to it than that. According to insiders, XL refuses to gamble on stuff it thinks could be successful but doesn't like. It encourages staff to continue projects they were working on before they joined the company – writing, promoting, DJing, whatever. And it cedes an incredible amount of control to its artists, the theory being that artists know how to make and market their own music better than anyone else.

But if this is the case, how does it work? Their press officer mentions A&R meetings in which staff will be deciding which single to release and, in the end, just phone the artist to ask them instead. It sounds like a lovely, utopian idea, but it flies in the face of one of the music industry's golden rules: that you never let artists make business decisions, mainly because they are all half-mad.

This is where Russell's experience as a musician comes in. He was a DJ, producer and member of the aforementioned Kicks Like A Mule and – perhaps because he still has a hand making and producing records, including Gil Scott-Heron's latest – he still views things from the artist's point of view.

"To be truly artistic, it's really an obliteration of the rules and the norms," he says, still wide-eyed with admiration for these strange people who make songs. "You've got to not be interested in a job. That kind of person is fascinating to me."

By having a studio built into its office space that is rented out for free to bands on the label or any musician whose ideas they are into, XL ensures that artists are always knocking around the place. The xx made their Mercury-winning debut album there during night-time sessions. And, on the day I turn up, Peckham rapper Giggs, one of the label's recent signings, is recording there.

The natural consequence of all this (not to mention the story one staffer relates about Russell encouraging staff to make music with discarded gear lying around the office) is that the bond between the label and its artists is strengthened.

It takes a bit of digging, but I finally discover how Russell gets away with putting musicians in the driving seat when we talk about what makes someone an XL artist (the label no longer has a "sound" like, say, early Factory or Motown did). He says the artists, from the Prodigy's Liam Howlett to Adele, are all linked by having "a strong idea about what they want to do. Strong personalities. Strong tastes. The music they've done when you sign them is barely relevant. It's not even the start. It's much more about the person and their ideas and strength of character and the direction they want to go in."

The Prodigy (with Liam Howlett on the right) brought huge success to XL. Photograph: Graeme Robertson for the Guardian

By picking the right personalities (XL's latest signing is another fascinating character – 19-year-old Tyler the Creator, a member of LA Rap collective Odd Future Wolf Gang Kill Them All), Russell has the confidence to run with ideas that would give many label managers a funny turn. The Prodigy, for instance, famously refused to do any TV on the grounds that they didn't like it – and this suited Russell just fine. In fact, he credits the band's Liam Howlett for instilling in him the idea that you can do things how you want and not worry about rules. "He really strengthened my own conviction in doing whatever you want," he says. "He was into the spirit of punk, hip-hop, pirate radio. I DJ'd on pirate radio and you were always a step away from the police. That subversion excited me and I think you can do it in a myriad of ways."

Howlett remembers their working relationship in equally glowing terms. "It's about having someone alongside you that has your back and is willing to break the rules," he says. "XL always worked in the same way as we thought – a punk-rock approach. We had to learn fast because our third album, The Fat of the Land, went big suddenly so we just did what felt real, whether or not it was to the detriment of record sales. We always had a strong, no-compromise stance on things like TV performances, but Richard respected that so we just did things in different ways to get heard – like when we dropped Smack My Bitch Up as a single: no major would have had the balls to release that."

Ballsy, punk-rock statements are to be expected in the music industry, but sometimes an artist's refusal to promote their work in certain ways might seem more like self-sabotage. How would Russell feel if, say, Adele turned around and said she wasn't releasing any more singles? "Well, she's never played at a festival," he replies. "Doesn't want to. It's not right for her. That's a brilliant artistic decision. It goes against the grain."

That would drive a lot of labels mad . . . why doesn't it drive him mad? "I really like it. Why should it?"

Might they not worry it could stop things happening for her? He dismisses the notion. "That's what will make it happen for her. You've got to believe in them. And any time I see someone do something like that, it strengthens my belief in them. Going against the grain, taking risks. And also, turning down money, frankly. Always a good sign, for anyone to do that. Everyone I work with could have gone about things in a way that would have made them more money, including me."

It is this lack of stress about boring things such as budgets that gives Russell a clear head to look at the picture from a musical perspective and nothing more. Howlett claims he can only recall one time when he saw the label boss lose his cool. "It was very early on, maybe in 1994," he says. "We did some gigs and me and Flinty charged all our room charges – porn, long-distance calls, loads of shit – to his room for a laugh. He tried to lay down the law and let us know he wasn't happy. Still makes me laugh, that."

Ask Russell about his career highlights and forking out for the Prodigy's blue movies is notably absent. Instead, he mentions Dizzee Rascal winning the Mercury prize – an event that opened the door for the current success of acts such as Tinchy Stryder, Chipmunk and Tinie Tempah: "There weren't artists like him getting signed at the time."

He is also proud of the way MIA dreamed about infiltrating the world of big US rap producers such as Timbaland, and succeeded. And while he is not the kind of person to go in for regrets, he does rue not realising how special the Aphex Twin was when he bought his first 12in. "I don't know if he had a proper deal then," he recalls. "But I couldn't really make it fit in my DJ set, so I ignored it."

With music booming out of speakers, posters splashed everywhere and offshoot indie label Young Turks beavering away on laptops, the XL office is a place that serves to remind you how dull your own place of work can be. How can people be having this much fun and make a success of it? No wonder they all seem so devoted to the XL cause. This, after all, is a label that is thriving in an industry that is supposed to be dying.

"It's not dying . . . it's changing," says Russell. "But then it always has been, as is the whole world. I just don't think about the future at all. It's not my responsibility."

Really? He doesn't have a strategy?

"Yeah – put good records out. That's it. I'm sure there are people thinking about stuff like copyright and downloading, but . . . you don't want an author to be thinking about Kindles and shit like that, do you? I mean, we do our best, but we definitely don't offer any solutions for the music industry."

And there's the problem. You can't learn taste. You can't develop your ears to pick hits. And if you think you can find a Richard Russell in every company – well, good luck trying.