When, in 1968, Robert Pirsig, who has died aged 88, sent a synopsis and sample of his novel Zen and the Art of Motorcycle Maintenance to 122 publishers, only one showed any interest, and James Landis, the editor who responded from William Morrow, warned him not to expect either a large advance or big sales. But within a month Pirsig set off on his Honda Super Hawk, accompanied by his young son, Chris, and friends John and Sylvia Sutherland, on the 17-day journey from Minneapolis to San Francisco that, six years, two drafts and some 700,000 words later, became the basis of an immediate bestseller. Millions of readers were drawn to this guidebook for the transition of a culture from the rebellious 1960s to the “me generation” of the 70s.

George Steiner’s New Yorker review, titled Uneasy Rider, famously compared the book to Herman Melville’s sprawling masterpiece Moby-Dick; other reviewers mentioned Thoreau, whose American transcendentalism also sought to link everyday life and the metaphysical. In a road novel in the spirit of Jack Kerouac’s On the Road, Pirsig drew on the aphoristic humour of hippy writers such as Richard Brautigan. “Metaphysics is a restaurant where they give you a 30,000-page menu and no food,” he wrote.

His motorcycle served as metaphor; the Sutherlands cannot attempt to understand the technology of their bike, but Pirsig insists that the “godhead resides quite as comfortably … in the gears of a cycle transmission as he does at the top of a mountain or in the petals of a flower”. The novel’s subtitle was An Inquiry into Values; a barely disguised Pirsig contends with his own inner duality, an alter ego called Phaedrus, who seeks “quality”, a mutable value different from the Aristotelian definition of “truth” as an absolute.

Zen and the Art of Motorcycle Maintentance was published in 1974. Photograph: Jeff Blackler/Rex/Shutterstock

Pirsig’s inner conflicts reflected his upbringing as a precocious genius in a severely demarcated world. “In the east the Master is considered a living Buddha, but in Minnesota, they wonder why he doesn’t have a job,” he said. He was born in Minneapolis. His father Maynard, who was studying law, was of German descent; his mother, Harriet (nee Sjobeck), of Swedish background. Both families’ local roots ran deep. Robert started school while living in Hendon, north-west London, while his father trained at the Inns of Court. By the time the family returned to Minnesota, where his father taught at the university law school, Robert was so advanced that he skipped two grades.

His older classmates bullied him; teachers forced him to write right-handed and he developed a stammer. But after winning a scholarship to the elite Blake school, he was placed with students his own age and tested to reveal an IQ of 170. Aged 15, he began taking classes at the University of Minnesota; two years later, adrift, he enlisted in the army and was sent to Korea. He began teaching English to the labourers he was supervising. Once he explained to them that the language allowed you, “in 26 letters to describe the whole universe. And they just said no. That was what started me thinking. In the east the basis of experience is not definable. That ... led me on the road to Zen.”

Pirsig visited Japan and returned home to earn a philosophy degree at Minnesota. He studied for a year in India at the Banaras Hindu University, returning home once again to study journalism and take writing courses with the poet Allen Tate. Working on the college literary magazine, he met Nancy Ann James, and in 1954 they moved to Reno, Nevada, supporting themselves as casino dealers while she divorced her husband. After their marriage, Pirsig worked as a freelance technical writer, briefly for United Press, and in a General Mills research laboratory, before again returning to Minnesota, to study for a master’s degree in journalism.

His teaching career at Montana State University was curtailed by his resistance to grading and his discovery of the hallucinogen peyote after attending a Northern Cheyenne tribal ceremony. When the governor of Montana died in a plane crash in 1962, Pirsig’s name was on a list of 50 “subversives” he was carrying in his pocket.

He began a PhD in philosophy at the University of Chicago, but immediately feuded with the department head, a leading Aristotelian, over his search for “quality”. He taught rhetoric at the University of Illinois-Chicago, but his behaviour at home became increasingly erratic and threatening. On Christmas Day 1961 he was taken to hospital in a catatonic state. He committed himself to a psychiatric hospital, gave up teaching, and eventually returned to Minneapolis, where at the veterans’ hospital in 1963, he received a series of electro-shock treatments.

In Zen and the Art of Motorcycle Maintenance, he wrote that the experience “taught me to get along with other people, to compromise, and I agreed. Phaedrus was more honest, he would never compromise, and the young respected him for that.” He felt “deep down inside all he had saved was his own skin”. When the treatment was finished, he applied for more than 50 jobs, but the rejections he received led to deep feelings of shame. “Those are society’s standards, they aren’t my standards,” he wrote.

Pirsig wrote his book mostly in the middle of the night, while working at commercial writing jobs. In 1970 he threw away the completed first draft, but with Landis’s help cut the final manuscript to 200,000 words; the novel was published in 1974, and suddenly he was famous. Robert Redford wanted the film rights, but they could not agree a deal.

Pirsig would fictionalise their meetings in his second novel, Lila, which he began after revisiting the Cheyenne. He intended the book to concentrate on the Native American approach to “quality”. He received a Guggenheim grant, but used the money to purchase a 32ft sailing boat. He explained that the demands of sailing seemed “like high quality ... everything is in a state of flux ... it’s the next wave you want to worry about, not one 10 days away.” In 1975 he and Nancy sailed via the Great Lakes to New York down the Hudson River. They separated in Annapolis, Maryland, in 1976. Pirsig continued down to Florida, and met a journalist, Wendy Kimball, who arrived to interview him on the boat and never left, becoming his second wife.

They sailed for the UK in 1979, getting caught in the same storm that wreaked havoc on the Fastnet sailing race. A few months after they arrived, Chris was stabbed to death by a mugger in San Francisco. The following year, Pirsig’s daughter Nell was born, “filling the hole left by Chris in the pattern of things”. For the next five years, the family sailed around northern Europe while Pirsig worked on Lila, by now centred on the sailing trip down the Hudson. In 1985 they returned to Portsmouth, New Hampshire, so Nell could start school; and in 1991 Lila, subtitled An Inquiry into Morals, was finally published, by Bantam books. This novel failed to match the success of his first.

Thereafter Pirsig lived quietly in Maine, emerging notably only to promote a new edition of Lila in 2006. A documentary, Arrive Without Travelling, appeared in 2008. He always insisted Zen and the Art of Motorcycle Maintenance was his story, and its cult status worried him. “I was the outsider, and now the outsider is the number one insider ... It’s a very unsettling experience.”

He is survived by Wendy and Nell, and by a son, Ted, from his first marriage.

• Robert Maynard Pirsig, writer, born 6 September 1928; died 24 April 2017