Sam Shepard, who has died aged 73 from complications of ALS, a form of motor neurone disease, excelled as an actor, screenwriter, playwright and director. In each of those disciplines he challenged and reimagined mythic American archetypes. He wrote nearly 50 plays; the most coruscating of them, such as the Pulitzer prize-winning Buried Child (1978), True West (1980) and Fool for Love (1983), established him as one of the visionaries of US theatre and created a fresh vernacular for exploring the disparity in American life between myth and reality, past and present, fathers and sons.

He took flawed macho heroes who might have staggered out of an Anthony Mann western, and broken, overheated families redolent of a Tennessee Williams clan, and forced them into claustrophobic hothouse scenarios; the result was like Beckett performed in cowboy duds. He found in the process a large audience receptive to this blend of stormy psychodrama, pitiless analysis and bruised romanticism. By the age of 40, he had become the second most widely performed US playwright after Williams.

He was fascinated by the violence that arose in American life from feelings of inadequacy. “This sense of failure runs very deep – maybe it has to do with the frontier being systematically taken away, with the guilt of having gotten this country by wiping out a native race of people, with the whole Protestant work ethic,” he said in 1984. “I can’t put my finger on it, but it’s the source of a lot of intrigue for me.”

To articulate the charged, often oedipal confrontations that littered his work, and its friction between progress and tradition, he forged a genuinely original writing voice. His runaway soliloquies made urgent, rhythmic poetry out of the banal. “I drive on the freeway every day,” says Austin, the screenwriter grappling with notions of authenticity in True West. “I swallow the smog. I watch the news in colour. I shop in the Safeway … There’s no such thing as the west any more! It’s a dead issue!” But he could be just as eloquent with silence, as he proved in his screenplay (co-written by LM Kit Carson) for Paris, Texas (1984). Wim Wenders’s plangent masterpiece reshaped the western as a modern road movie in which the wandering loner, played by Harry Dean Stanton, is mute for almost the first hour of the film.

Facebook Twitter Pinterest Sam Shepard as a dying farmer caught in a love triangle, in the film Days of Heaven, directed by Terrence Malick, 1978. Photograph: Allstar/Paramount Pictures

As an actor, Shepard was a softer presence, cast early on for his wan, arresting handsomeness and his connotations of nobility. Later, as he grew craggier, his presence was typically used to denote grizzled tradition. He was fey as the dying farmer caught unwittingly in a love triangle in Terrence Malick’s Days of Heaven (1978). His finest acting work was as the pilot Chuck Yeager in Philip Kaufman’s mighty adaptation of Tom Wolfe’s The Right Stuff (1983). Shepard evoked achingly the determination of Yeager, who had been the first person to fly at supersonic speed, to set a new altitude record even if it meant jeopardising his life. Burned and battered at the end of the movie, he falls to earth with a bang but gathers up his dignity along with his tattered parachute. The performance, which brought him an Oscar nomination for best supporting actor, marked the point where his acting began to blur with his writing to create “the intrepid artist-cowboy of popular imagination”, as John J Winters put it in his book Sam Shepard: A Life (2017).

This impression persisted in films such as the Cormac McCarthy adaptation All the Pretty Horses (2000), The Assassination of Jesse James by the Coward Robert Ford (2007) and the Mississippi melodrama Mud (2012). Recently Shepard starred in the Netflix series Bloodline (2015), as the patriarch in a tempestuous family scarred by murder and double-crossing. The impression that he was having a whale of a time was enhanced by the suspicion that the programme makers had raided Shepard’s own thematic larder in cooking up the show’s heady gumbo. No wonder he looked at home.

Facebook Twitter Pinterest Sam Shepard with Barbara Hershey in The Right Stuff, 1983, his finest acting role, which brought him an Oscar nomination. Photograph: Allstar/Warner Bros

He was born in Fort Sheridan, Illinois, and raised largely in southern California, the son of Samuel Shepard Rogers, a teacher, farmer and former US army pilot, and Jane (nee Schook), also a teacher. The family moved around, living in Utah and Florida before settling for a while in Duarte, California, where his father owned an avocado farm. Sam was educated at Duarte high school, Los Angeles, and at Mt San Antonio College, where he studied agriculture.

Though he claimed to have been a rabble-rouser, classmates later recalled a “nice, polite, quiet” boy. He did, however, clash repeatedly with his alcoholic father, and left home after intervening in a parental argument. He had various odd jobs and briefly joined a travelling theatre troupe. Ending up in New York, he worked as a waiter and started knocking out one-act plays for the off-off-Broadway circuit.

These immediately earned him notoriety. A double-bill of Cowboys and The Rock Garden caused an uproar by its profane language; a scene from the latter was excerpted in Kenneth Tynan’s 1969 revue Oh! Calcutta! Shepard’s work was said to have caused a significant cancellation of subscriptions at some of the venues that staged it. But along with controversy came acclaim: between 1966 and 1968 he won six Obie awards for plays including Icarus’s Mother and La Turista.

His own emerging creative life brought him into the orbit of other artists of that time. He became friendly with the Rolling Stones. Along with Allen Ginsberg, he was one of the writers of Robert Frank’s film Me and My Brother (1969). Less happily, he also co-wrote Michelangelo Antonioni’s Zabriskie Point (1970). “Antonioni wanted to make a political statement about contemporary youth, write in a lot of Marxist jargon and Black Panther speeches,” he said. “I couldn’t do it. I just wasn’t interested.” Shepard’s name ended up being one of five credited for the script.

Facebook Twitter Pinterest Sam Shepard photographed for the Guardian, 1974. Photograph: Frank Martin/The Guardian

He also drummed for the Holy Modal Rounders and married the actor O-Lan Jones, with whom he had a child. At the same time, he fell into a seven-month relationship with the musician Patti Smith, and co-wrote with her the 1971 semi-autobiographical play Cowboy Mouth, in which they both starred. Another of his plays, Back Bog Beast Bait, was included on the same bill and featured Jones as a character based on Smith.

When Shepard and Jones moved briefly to London to escape that imbroglio, he met the director Peter Brook, who introduced Shepard to the teachings of the spiritual philosopher GI Gurdjieff and encouraged him to think more closely about character in his writing. Upon returning to the US, he went on tour with Bob Dylan’s Rolling Thunder revue, where he began a brief relationship with Joni Mitchell; her song Coyote was said to have been written about him (“He pins me in a corner and he won’t take no/ He drags me out on the dance floor/ And we’re dancing close and slow”). Out of his friendship with Dylan came a screenwriting credit on the singer’s film Renaldo and Clara (1978) and a co-writing one on his song Brownsville Girl.

Unsettled by life on the road, and with Brook’s advice in his ears, Shepard took up the post of playwright-in-residence at the Magic theatre in San Francisco and produced the plays that were to mark his most celebrated period and define him forever in audiences’ minds. Curse of the Starving Class, which had its premiere at the Royal Court in London in 1977, concerns a debt-ridden, alcoholic former pilot trying to offload his Californian farm.

In Buried Child, a dysfunctional family is haunted by the memory of a dead son and dominated by Dodge, the gone-to-seed patriarch marinated in booze. Shepard’s own father pitched up at one performance and began berating the actors on stage. “He took it personally and he was drunk,” the playwright said. “He was kicked out and then was readmitted once he confessed to being my father. And then he started yelling at the actors again.”

Facebook Twitter Pinterest Sam Shepard in Blackthorn, 2011. As he grew craggier, his presence was used to denote grizzled tradition. Photograph: Everett/Rex Shutterstock

True West, about two warring brothers, dramatised what Shepard saw as an essential divide in human nature. “I think we’re split in a much more devastating way than psychology can ever reveal … It’s something we’ve got to live with.” (In a notable 2000 Broadway staging admired by Shepard, the connection between the characters was amplified by having the actors Philip Seymour Hoffman and John C Reilly swap roles on alternate nights.)

Fool for Love (1983) was a feverish, motel-bound drama about incestuous half-siblings; Shepard also adapted it and starred in Robert Altman’s 1985 film version. Completing the playwright’s most distinguished period, A Lie of the Mind (1985) examined an abusive marriage. It, too, was haunted by yet another drunk, domineering father.

During this time, Shepard’s career as an actor was picking up. Though he made only a mild impression in Frances (1982), a biopic of the actor Frances Farmer, it was important for another reason: he fell in love with its star, Jessica Lange, with whom he was in a relationship for 26 years. They appeared together in the rural dramas Country (1984) and Crimes of the Heart (1986), while Shepard directed her in Far North (1988), one of only two movies he directed. The other, Silent Tongue (1993), was a mystical western starring River Phoenix, Richard Harris and Alan Bates.

He starred with Diane Keaton in the comedy Baby Boom (1987) and alongside Julia Roberts in the weepie Steel Magnolias (1989) and the thriller The Pelican Brief (1993). He was a good choice to play the Ghost to Ethan Hawke’s Prince in a modern-day Hamlet (2000) by Michael Almereyda, who also directed a revealing documentary about Shepard, This So-Called Disaster (2003), which followed the preparations for a staging of his play The Late Henry Moss. Other films included Black Hawk Down (2001), The Notebook (2004), Killing Them Softly (2012), an adaptation of Tracy Letts’s play August: Osage County (2013) and the thriller Cold in July (2014).

Shepard continued writing, acting and directing throughout the rest of his life, branching out also into short fiction – in collections such as Cruising Paradise (1996) and Day Out of Days: Stories (2010) – and a novel, The One Inside, published this year. Asked in 2016 if he felt he had achieved something substantial, he replied: “Yes and no. If you include the short stories and all the other books and you mash them up with some plays and stuff, then, yes, I’ve come at least close to what I’m shooting for. In one individual piece, I’d say no. There are certainly some plays I like better than others, but none that measure up.” For all the messy domestic histrionics that litter his work, he seemed ultimately to be grappling with solitude. Writing, he said in 2010, “is almost a response to that aloneness which can’t be answered in any other way.”

He is survived by his son, Jesse, from his marriage to Jones, and two children, Hannah and Walker, from his relationship with Lange.

• Samuel Shepard Rogers, writer and actor, born 5 November 1943; died 27 July 2017