Gus van Sant has never been much for hoeing the same old row. His last movie, Milk, won two Oscars and – thanks to a providential release date – became part of the national debate over California's Proposition 8 to bar gay marriage in the state. Milk was an epic history lesson in the form of a biopic – one critic astutely called it "a calmed-down Oliver Stone movie" – and many thought it offered sobering lessons to a gay political establishment that had grown fat and complacent in the 30 years since Harvey Milk was murdered. But three years later Van Sant is back, not with a whizz-bang, zillion-dollar project that most directors would claim as their God-given right after a prestige hit such as Milk, but with a low-budget, intimate, small-scale fragment of a movie called Restless. It's a meditation on life and death, and on each as an essential component of the other. It has to be said, though, that Restless has not earned unanimous praise, but Van Sant has never been afraid to risk disaster in the cause of trying to throw everyone a curveball. Whatever he does, he's always interesting.

We meet up in Beverly Hills on the 10th day of a ruinously debilitating LA heatwave, the kind of day when you take two spare shirts and a towel to drive across town, otherwise you end up looking like Jackie Gleason after a sauna. Following a 20-minute wait in the semi-air conditioned hotel lobby, I'm admitted, if ever so briefly, to a blissfully frigid hotel room, only to be informed: "Gus would prefer to talk on the terrace." So back into the ghastly, spirit-lowering heat I wander, wondering whether Van Sant is just in from famously rainy Portland, his home town of three decades, and is savouring the meteorological contrast for all it's worth. Or if he just likes to mess with journalists.

Actually, I think he's oblivious to such things. The heat is here; let us savour it, seems to be the attitude; it is not a lot different from his organic approach to film-making as a whole. And there is something of his movie's vagueness, the unwillingness to spell things out laboriously, to be detected in Van Sant's open-ended answers and reluctance to pin too much down.

Restless stars Dennis Hopper's son Henry (whom I already like more than his dad) and Mia Wasikowska (lately the best Jane Eyre ever) as an oddball couple who meet, Harold and Maude-style, at one of the funerals he obsessively attends. Enoch (Hopper) is weathering the recent deaths of his parents in a car crash, and Annabel (Wasikowska), it turns out, is facing incurable cancer (which will tend to happen when you're named after a doomed Edgar Allan Poe heroine). Death looms large throughout. It feels, I suggest to Van Sant, like a coda to the movies he made before Milk, a death-centric body of work that includes Gerry, the post-Columbine-massacre meditation Elephant, and Last Days, Van Sant's fictional rendering of the end of Kurt Cobain's life.

Van Sant agrees and demurs at the same time. "Not consciously. Those films were about catastrophes, and this one is more about ordinary death. It's more intimate. The others were huge catastrophes, and all originated as news items: Gerry came from a news story about two guys who got lost in the desert and one killed the other, violently. Did he think his friend was a demon; was he overcome by dehydration? The others were front-page news stories, major cover-of-Time-magazine-size stories, massive public events. Restless is more scripted, more intimate and performance-based, too."

This new film places Van Sant midway between the all-amateur casts of Elephant and Mala Noche (his 1985 debut) and the all-pro casts of Milk and To Die For, but working on a budget more in line with his death films, and using the methodology he honed in them. Certainly Restless feels as pleasingly hand-made as they do.

"I'm usually trying to react to what the actors are coming up with," says Van Sant. "And then the environment, and then the story. There's three things."

"You're following your track, the story, your only plan, your map for the audience, and all the other stuff is, like, the fun stuff: the costumes, the locations, the set-dressing and the actors. They can all be variable as you like if you stick – however roughly – to the path. There are so many interesting ways in which all those elements can come together that, really, to not control it is actually easier. Thinking abstractly, outside the box, using intuition and imagination, not deduction. It's not chess-player permutations – it's organic, intuitive, vibing it out, ouija-boarding it, feeling it out, not doing it based on rational thinking."

That has been Van Sant's approach for most of his directorial work. What seems in retrospect the least comfortable period of his creative life came after he directed the film he'll most likely be best remembered for, Good Will Hunting, but which in the long view seems among his least typical. Van Sant went through a period of mainstream favour after Good Will Hunting, and followed it with what seemed like a toothsomely radical idea on paper – a shot-for-shot remake of Psycho – but which was DOA on celluloid. Then there was a kinda-sorta second stab at the Will audience with the lachrymose Sean Connery-led drama Finding Forrester. These, as he points out, were "jobs I signed on to".

None of them seemed much like the "real" Van Sant, who had started out making movies – not unlike home-movies – set in criminal, druggie and gay milieux: Mala Noche, Drugstore Cowboy, My Own Private Idaho. After Forrester, Van Sant returned to more marginal subject matter and to more formally radical approaches with the death films.

"All of those death films had mysteries," he says. "Why did that kid kill his friend? Why did the Columbine boys shoot the students? Where was Kurt for those last three days and why did he shoot himself? There's tonnes of material that turned up in other people's speculations and imaginings – in Kurt's case, there were whole documentaries made – into all these missing periods of time, incidents with no witnesses. At Columbine, the two boys were dead, so no answers there, and Kurt's not around to explain, obviously.

"Since there was no strict knowledge of what actually happened, we had artistic freedom to make stuff up. And I had my own opinions about all these stories, but all my answers were always kind of dull and mundane: they walked around the fuckin' desert for three days, that's what happened! Kurt walked around his house for three days, then shot himself. The Columbine kids went nuts because high school was so dull!"

Van Sant had consciously regrouped after Forrester, absorbing movies by directors who were new to him – such as Hungary's Béla Tarr, master of the extended travelling shot, and Britain's Alan Clarke.

"Werckmeister Harmonies and Sátántangó [Tarr's masterpieces from 1995 and 2000] had a very big effect on Gerry. At the time I was reading about John Cassavetes and watching a lot of his movies, and we went to the locations without a script, Cassavetes-style. But as we worked I could see that it wasn't going to create a Cassavetes-type movie; it was gonna be something different; what Béla was doing felt more in our line in this case, so we went with it. Alan Clarke's connection with Elephant was less to do with knowing much about Clarke than with Harmony Korine having once said Elephant was his favourite movie. He'd seen it at a retrospective and got in trouble because he started laughing, and they scolded him for laughing at the violence. In fact, he was laughing in admiration of its brilliance, if that makes any sense."

Admirably, Van Sant makes movies that are uniquely about themselves, and never about other movies – unlike many other more film school-inflected directors – a reflection of his avant-garde education at the Rhode Island School of Design in the early 1970s, when he greedily absorbed what he could find of the works of Andy Warhol, Stan Brakhage and other pioneers of the American film underground: "movies most people are more likely to have read about than seen." Their approaches are dimly evident even in the background of his most commercial movies; Milk would be all the poorer without that hinterland, and Restless, for all its easy approachability, owes plenty to the free spirits of those pioneers.

And Van Sant may be their last representative in Hollywood. Long may he channel their rebel spirits.

Restless is reviewed here