Confidence doesn’t seem to be something that Hollywood’s current golden boy, Noah Hawley, is short of. When I suggest that he’s being heralded as “the Kubrick of television” – pushing the boundaries of prestige programming and its visual effects in a not dissimilar way to that in which Dr Strangelove, The Shining and A Clockwork Orange did in 20th-century cinema – he does not (as I suspect a British counterpart might) dismiss the mantle with protestations of humility, false or otherwise. “I’m Kubrick without the OCD,” he chuckles instead, and proceeds to recommend Jon Ronson’s documentary Stanley Kubrick’s Boxes to me, about the celebrated film-maker’s obsession with achieving the perfect storage solution.

But with two critically acclaimed and publicly popular TV shows in play: the anthology series Fargo, which begins its third season on Channel 4 later this month, and Legion, his surreal take on the superhero genre for Fox, plus a bestselling novel, Before the Fall, now out in paperback, Hawley, 50, has every reason to be walking tall right now. Plus, he is, as he puts it himself, wryly, “a 20-year overnight success”.

It is early on a spring morning in Los Angeles when Hawley and I speak; he splits his time between his homes here in LA, where his production office is based, and in Austin, Texas, plus the sets of his two shows, in Calgary and Vancouver, as is demanded by his role as “showrunner”, a combination of head writer, executive producer and studio wrangler. “I am a salesman, I am an executive, I look at budgets, I think about power politics, but then I am also a creative person,” he explains.

He started out, however, with more traditional creative ambitions. After graduating from Sarah Lawrence College in Yonkers in 1989, he tried his hand at rock stardom, living in his native New York with his band. “But I am not a night person, and that became a problem,” Hawley deadpans. “I’m in bed by nine o’clock at night. If all rock shows could be at 6pm, I might still be playing.”

He relocated to San Francisco for a while, and wrote a couple of novels, which failed to find publishers, and a third that was barely reviewed, and had further spates of wondering if he should give it all up. “But I like the process,” he says. “This isn’t a career choice or a gimmick; this is something I clearly need to do. It’s an identity.”

Meanwhile, having moved down the west coast to LA, he was also attempting to forge a path in television, cutting his teeth as a writer and producer on the long-running Fox drama Bones.

The first show of his own he created, The Unusuals, starred Jeremy Renner and was “a very offbeat new York cop show”. It lasted only one season on the ABC network in 2009. “It was the ratings,” Hawley says, matter-of-factly. “It didn’t do great, but by today’s numbers it would have done well. And, of course, three months after they cancelled it, Jeremy Renner was nominated for an Oscar … ”

But he moved on, the following year creating My Generation, also for ABC, a faux-documentary in the style of Seven Up, following a group of teenagers and revisiting them 10 years after graduation. “The president of the network resigned a week before we premiered,” recalls Hawley, and it was cancelled.

Joel and Ethan Coen never make the same movie twice, and we shouldn’t either Noah Hawley

Click here to watch a trailer for season three of Fargo.

The failure of his shows was down to a more general issue. “The big problem was that I was just making cable shows on a broadcast network,” he says. The US networks, known for playing it safe with mass-appeal, formulaic fare, governed by advertising demographics, were not the best home for Hawley’s unique vision. “They fight back so hard; they want to water down what makes the show original,” he says.

When he finally found his way to FX – part of the 20th Century Fox family, but with aspirations and content akin to the premium cable channels HBO and Showtime – he found a better fit. “They would rather make something great for some people than something good for everybody. You might not get the broadest audience, but you are going to make the best show.”

Nonetheless, his first discussions with them were over a project that almost everyone, including Hawley himself, considered foolhardy: turning the beloved Coen Brothers’ 1996 film Fargo into a television series. “Oh, it was a terrible idea,” Hawley says. “I assumed that only two people would watch it, and one of them would hate-watch it.” But the show, starring Billy Bob Thornton and Martin Freeman and a similarly quirky mix of folksy comedy and brutal menace as the film that inspired it, was an unprecedented hit with both the viewing public and critics, scooping both an Emmy and a Golden Globe.

Most writers, and most channels, having stumbled upon a formula so successful, would not dare to deviate for a follow-up. Hawley, instead, ripped up the rulebook and created an entirely new, equally twisted and weird story for season two, set in 1979 and starring Ted Danson and Kirsten Dunst. But what the rest of us might see as extreme audaciousness, he sees as merely his process.

“It’s what I do as a novelist: I tell a complete story, with a beginning, middle and end, and then I move on to other stories,” he reasons. “Joel and Ethan Coen never make the same movie twice, and we shouldn’t either.’’

The third season of the show is set in 2010, in the turbulent aftermath of the global financial crash and stars Ewan McGregor as two brothers, Emmit and Ray Stussy – the former the wealthy, self-made “parking lot king of Minnesota”, the latter a lowly parole officer – and David Thewlis as the mysterious, malevolent VM Varga.

“This year, I wanted to deconstruct the lie that starts every episode, and that started the movie, that ‘This is a true story’,” says Hawley. “It has turned out to be a very timely conversation,” he notes. “But I was interested in it before we got into a world of ‘alternative facts’.”

How far will the audience go with you in taking this material to its boundaries and playing with what a show can be? Noah Hawley

Graphic, novel … Katie Aselton and Dan Stevens in Legion. Photograph: Chris Large/FX Networks

The most Kubrickian example of Hawley’s work is undoubtedly Legion, a radical reworking of the Marvel comic story, which stars an almost unrecognisable Dan Stevens as David Haller, a young man with what would appear to be severe mental health issues. The character first appeared in the X-Men comics, but this is a serious departure from the ubiquitous superhero franchise films of recent years, a lurid, psychedelic journey through flashbacks, memories, dreams, random dance sequences and mind-boggling body-swaps.

“The X-Men stories are the stories of outsiders, people who don’t fit into normal society and are ostracised; it’s a metaphor for gender, race or sexual orientation,” says Hawley. “I would categorise mental illness as a condition that separates you from the rest of the world, too; David Haller has lived most of his life, from his teens, in a different reality than the people around him.”

Hawley’s aim was, in part, to deliberately subvert the well-trodden superhero narrative. “What else can you do with it?” he posits. “Can you make something existential or surreal out of it? How far will the audience go with you in taking this material to its boundaries and playing with the edges of what a show like this can be?”

The success of Legion has cemented Hawley’s reputation as one of Hollywood’s leading creatives, but also presented him with the one challenge that might rattle him. “In 10 years creating shows, I’ve never done a second season of anything,” he notes. “So I am a little more daunted by that [than by creating something new].”

He also has a couple of movie projects in the pipeline, including adapting his own novel Before the Fall for the big screen, and is due to direct his first feature, the sci-fi Man Alive, next year. Impressive though it may be, is his in-tray not a little overwhelming?

“I am a little overwhelmed, I won’t lie to you,” he sighs. “Making a new season for Legion is not something you just switch into. It’s not something you do between dropping the kids off in the morning and having dinner at night. That’s a retreat into the woods for six weeks with some mushrooms, and trying to come back with the answers. But I haven’t stumbled yet.”

Fargo season three airs on Channel 4 in May