It promises to be one of the literary highlights of 2021 – publication of the diaries of Patricia Highsmith, one of the most conflicted, fascinating novelists of the 20th century.

Highsmith, who died in 1995 having written a series of psychological thrillers, including The Talented Mr Ripley and Strangers on a Train and the romance The Price of Salt, left two sets of diaries hidden in a linen closet in her home in Ticino, Switzerland.

In one she recorded details about her professional life: plot ideas, philosophical musings and thoughts on writing. In the other she documented her private reflections and memories, including a single sexual encounter with the writer Arthur Koestler (a “miserable, joyless episode”) and her efforts, through psychotherapy, to “get myself into a condition to be married”.

She had no more compassion for men than she did for women. In one entry Highsmith writes that “the American male does not know what to do with a girl once he has her. He is not really depressed or inhibited by his inherited or environmentally conceived Puritan restraints: he simply has no goal within the sexual situation”.

Highsmith’s diaries, which run to more than 8,000 pages, have been pored over by biographers, but have never before been made public, or in this case interwoven into a single narrative of the life of a complex woman who thought deeply about themes of good and evil, loneliness and intimacy.

Facebook Twitter Pinterest American author Patricia Highsmith at home in 1970. Photograph: Liselotte Erben/Sygma via Getty Images

It was in her diary that she described becoming sexually obsessed with a customer at Bloomingdale’s in New York, whom she later followed to her home, provoking observations about murder and love.

“She had an obsession about detailing absolutely everything in her life, very much like Sylvia Plath,” said Andrew Wilson, author of an acclaimed 2003 biography of Highsmith, Beautiful Shadow. “And she drew on the diaries for her novels, which explore the notion of obsession, guilt and murder, and reject rationality and logic for the darker elements of human personality.” Dubbed “the poet of apprehension“ by the novelist Graham Greene, who said she “created a world without moral endings … Nothing is certain when we have crossed this frontier”, the Texas-born Highsmith was deeply influenced by European existentialists such as Albert Camus and Søren Kierkegaard, and those influences, Wilson believes, are deeply felt in her diaries. But there is a question over how far Highsmith can now be assimilated into contemporary culture. Antisemitic, racist and misogynistic, she wrote in a way that could make for extremely difficult reading.

“She’s incredibly modern because she speaks to irrationality; she was a lesbian who hated women, totally politically incorrect in lots of ways, and certainly not a poster girl for the feminist movement,” said Wilson, who warned that hundreds of pages of Highsmith’s views, undiluted by explanation from friends and colleagues, could be wearing.

“She could be a monstrous, violent and quite unpleasant woman. She hated black people, she hated Jews, and she hated women, but there are also reasons why she was like that,” he says, citing her mother’s rejection and a clumsy attempt by her father to seduce her when she was a teenager. But, according to one friend she was sexually abused by a couple of men, possibly travelling salesmen, when she was four or five.

Highsmith herself was aware of the unpalatable nature of her views, and it is likely that they caused her distress. “No writer would ever betray his secret life,” she wrote to a friend in 1940. “It would be like standing naked in public.”

The diaries, which are held in the Swiss Literary Archives, were discovered by her editor, Anna von Planta, and Daniel Keel, the literary executor of her estate. Von Planta said she had no interest in censoring or downplaying the unpleasantness of Highsmith’s views, in large part because Highsmith appears to have anticipated their publication.

“The idea was to show how Patricia Highsmith became Patricia Highsmith,” Von Planta told the New York Times. “And to have her tell about her life, her thoughts, her concerns, the making of her work, in her own words.”

The diaries’ publication could help to again reveal that, contrary to popular imagination, creativity is not necessarily rooted in our best instincts.

“She said she was born under a sickly star, and she knew from her earliest consciousness she was difficult, and that she had a problem with an existential idea of being,” Wilson said. “The diaries question this over and over again. Fascinating, but a strong brew. You won’t be able to read them in one go.”