David Sedaris’s partner of 25 years, Hugh Hamrick, calls the first chunk of the essayist’s diaries, published under the title Theft By Finding, “David Copperfield Sedaris”. And it’s true, Sedaris concedes, the book – which covers the years from 1977, when he scribbled his first entries on the backs of coffee shop placemats while travelling around, to 2002 – has a certain rags-to-riches quality. In the second volume, on the other hand, “I just go from shopping at Paul Smith to shopping at Comme des Garçons, and I’m on airplanes all the time”. The thought prompts a memory of a recent plane trip, first class from Hawaii to Portland, Oregon. “This woman said, you are so lucky to be seated up front, it’s a great spot for people-watching. And I said, hmm, it could be, but we don’t really count you as people.” He bursts out laughing, and so do I, even though I know I oughtn’t. What on earth did she say? “She laughed, she knew I was kidding. Hugh was horrified. Horrified.”

There’s something about that one-liner that characterises Sedaris’s writing: a flash of directness, even brutality, that threatens the social veneer (especially in first class); the reassuring feeling that of course he’s kidding, with the faint background feeling, “but not entirely”; the spreading realisation that he’s getting at something far more complex about human nature, absurdity and awkwardness. “He’s like an American Alan Bennett,” says the quote (from this newspaper) on the back cover of Theft By Finding. Both writers occupy that space in which their subversiveness and caustic records of daily life run up against the foam blanket of “humour”, as if we can maximise the cuddliness and minimise the edge by focusing on the laughs.

This isn’t entirely our fault. Both Bennett and Sedaris have refined their written and verbal delivery to such a degree that their fans inhabit the worlds they create without necessarily having to dig more deeply on each page. “A lot of times with my stuff,” Sedaris tells me, as we sit over tea in his home in rural West Sussex, “I think people think I have a tape recorder, and I talk into it. I’m not going to complain, that just means that they sense an ease to it.” On the contrary, however, a piece for the New Yorker that is nearing completion is in its 19th draft, and the magazine’s editors didn’t even see it until the 17th. As well as the meticulous deskbound reworking, Sedaris also makes adjustments based on the experience of reading aloud during one of his frequent public appearances (I meet him the day after he’s flown from the States; by the time you read this, he’ll be back there, roaming from Boston to Milwaukee to Nashville).

The work he’s best known for is collected in volumes that include Barrel Fever, his first book, published in 1994, Me Talk Pretty One Day (2000) and Let’s Explore Diabetes With Owls (2013), as well as radio appearances and magazine pieces. But these have all been carefully crafted; diaries, even when, as he explains, they have undergone his edit, are a departure.

Or almost. In fact, a diary entry was the first piece of writing Sedaris ever read out loud. He had finally escaped Raleigh, North Carolina, where he was brought up – alongside five siblings and his parents – and arrived in Chicago, where he enrolled at the Art Institute. He was 27, and chronicled his first day in the big city thus: “I’m not as hysterical as I thought I might be and am having a good time looking around. Visited the post office and the big main library and the conservatory of music, where [the composer] Ned Rorem went. I am beside myself. On leaving the reception tonight, I saw a man sitting on a stool. He’d removed his artificial legs, which were lying on the ground beside him. What a place!”

At the institute, students were called on to read and display work, and it occurred to Sedaris that his classmates had little sense that they were performing to an audience, a sensation he had “every time I raised my hand”; rather, they engaged in aimless talk about their feelings about themselves. Recalling this now, he says, “There are certain things nobody wants to hear.” Because they’re too disturbing? “No! The disturbing things are great.” He meanders into what appears to be a piece of improvised monologue, depicting a person talking to their therapist, complete with sing-song voice and staring into the middle distance: “‘I kind of felt when I tried that sweater on, I felt like me, but I felt like not me at the same time. And I just sort of, I mean I’ve been feeling like that a lot lately, and it’s not like an emptiness, really, but it’s like a fullness – you know like when you’re really full …’ Nobody wants to hear that. If somebody said, ‘My stepmother used to force my hand on to the hot stove,’ I’m right there.”

Accordingly, he read with bravura and made people laugh. His immediate reaction was to think: “Wow, that felt infinitely better than painting that picture.” He began to read publicly, and was spotted by radio producer Ira Glass, which led him from local radio to National Public Radio, and an eventual move to New York.

It had been a long time coming – and an even longer time imagining. Despite the confidence that his writing exudes, Sedaris had scraped by doing all sorts of odd jobs, constantly running out of money, drifting from restaurant to restaurant in US breakfast chain International House of Pancakes, sometimes behind and sometimes in front of the counter. During the same period, and beyond it, he took a lot of drugs and drank a lot of booze. He gave up both a long time ago, and cigarettes a decade ago, but is very funny about the lengths he went to “skeeve” drugs, and the precision with which he timetabled writing periods around substance abuse. All the time, he wanted another kind of life, but made no assumptions that it would happen, not least because of his fondness for biographies.

“If you read a biography about writers or artists, it seems like their friends were famous, too. Like, when you read about Paris in the 1920s, Picasso knew Georges Braque before either of them were anything. And then I would look around at my friends and say, in order for me to be famous, these guys have got to be famous as well. And I’m just not seeing it.” He had, he says now, no connections, and no skills to speak of – he can’t even drive a car, and types with one finger. He didn’t have the beauty to waltz into a front-of-house job in a restaurant – he thought that he’d always be stuck washing the dishes.

But throughout the 1990s, his stories of family life and of working as an elf at Macy’s department store during Christmas season turned out to be exactly what people wanted to read, and his popularity increased. In 1991, only weeks after his mother had died suddenly during treatment for lung cancer, he faithfully told his diary the latest from Santa Land. “Yesterday, a woman had her son pee into a cup, which of course tipped over. ‘That’s fine,’ I said, ‘but Santa’s also going to need a stool sample.’” In the next entry, he describes what it was like to receive gifts from his mother that she had ordered before her death (“rattling”, he notes, economically).

Throughout his writings, it is his family who stand out most clearly: mother, father, sisters Lisa, Gretchen, Amy and Tiffany, and brother Paul. It is as though their wisecracking eccentricity and their more overtly dysfunctional behaviour, and the fact that Sedaris is prepared to write about them so candidly, offer us not only a model of how to deal with the difficulties of family life but also a sort of vicarious transgression; Sedaris allows us to admit the co-existence of love, anger, distress and laughter. With jokes.

It is most evident in his writing about his sister Tiffany, who suffered from severe mental health issues throughout her life, and took her own life in May 2013. Sedaris wrote about her, and about a holiday his family took in the aftermath, in a celebrated New Yorker essay, Now We Are Five, later that year.

And I said, could you close the door, please? And he shut the door in her face and I never saw her again David Sedaris

At the time of her death, they had not spoken for several years. The last time he saw her was at the Symphony Hall in Boston, when he was on the verge of performing at the beginning of a tour. “‘David. David. It’s me. Your sister. I have something for you.’ And there was a guard holding the door open. And I said, ‘Could you close the door, please?’ And he shut the door in her face and I never saw her again.” He explains now that he knew that if he had spoken to her, it would have consumed him, and he would have lain awake every night thinking about it. “Every encounter with her was like that; everything was so deeply upsetting on so many levels. You had to block time for her.”

He tells me that, were I to have met her, I would have thought, “‘Wow, this person’s really funny, and I really want to be around this person.’ And then 10 minutes later you would think, ‘My God, she’s never going to shut up, she’s never going to stop talking.’ And then you would try to back away, and she would say, ‘What’s wrong, Alex? What, you don’t like me?’ And then she would find some physical attribute and make fun of you for it, and try to draw everybody else into it. Her friendships were intense and then they burned out, they ended in fistfights, they ended with restraining orders.”

In Now We Are Five, he described how Tiffany’s affections would be transferred between siblings over the years, and it’s clear that when Sedaris and she got on, they really got on. In a diary entry from 1984, he reports – matter of factly – that Tiffany has suffered an ectopic pregnancy and undergone emergency surgery. She had not known she was pregnant. Just before the operation, Sedaris writes, the doctor asked her if she had any questions. “And in a weak voice Tiffany said, ‘Yes. When can I have sex again?’” Sedaris concludes the entry with the words: “You really have to hand it to her sometimes.”

Recently, he has read that essay aloud and has not had the response he expected, or at least had hoped for. “I thought that was the funniest thing in the world. The audience never laughs.” Perhaps, I suggest, this is because we have strong feelings about how traumatic such an event might be. “But it’s not like she wanted the baby,” he replies. Again, a directness that almost verges on the taboo, and has on occasion led to charges of insensitivity.

And yet Sedaris frequently demonstrates great empathy. After his performances, he is known to spend much time talking to audience members, especially teenagers. I tell him that he recently made an off-stage appearance in the sitcom Modern Family, when Manny, the show’s intellectual teenager and aesthete, was grounded for breaking curfew and could not accept that he would miss one of Sedaris’s readings, which the comedian likes.

On one tour, he turned the tables on fans, however. His ATM card stopped working and so he decided to simply ask audiences for money. He’d tell them it was not a loan, nor for charity, and often walk out of a bookshop carrying $350 in donations. “But if there was a beggar outside, they wouldn’t have given him anything. It’s funny to give money to someone who doesn’t need it. Here’s $20, isn’t that funny? And I’m like, yeah, it is, and I would take the money.” Sometimes, he would spot someone who looked like they needed a hand, and give a chunk of it to them. He frequently turns up to bookshop readings an hour or two early, so he can start signing and help early birds get home sooner, because “that’s all anybody ever wants to do, go home”.

In the lanes of Sussex, he is known as the American who picks up the litter, for which he has a passionate hatred

Sedaris is home only sporadically; Sussex is his and Hamrick’s main residence, although they also have a home in London, a beach house in North Carolina and a place in Normandy, where they used to live, but now rent out. In the lanes of Sussex, he is known as the American who picks up the litter, for which he has a passionate hatred. Indeed, he even made Clare Balding accompany him on a rubbish-collecting mission for the Radio 4 programme Ramblings. One of his first invitations when he moved to the countryside was to a meeting of the Clean and Tidy Advisory Board, in the wake of a cross letter he had written to the council.

Other than that, he keeps a low profile. While Hamrick cycles to the village every morning and returns with “his legs laden with the pollen of all the gossip he gets”, Sedaris limits himself to making conversation at the grocery shop checkout so that he doesn’t become known as the man who doesn’t make conversation at the grocery shop checkout. “But it’s like, has anyone ever vomited on to the belt?” he laughs. “That’s a better question than, when’s this rain going to stop?”

Before I leave, he describes the experience of reading his diaries for the audio version of Theft By Finding. He was shocked, he said, to find himself sobbing as he talked about his mother’s death – even though he knows that when he’s reading something, it’s as if it happened five minutes ago. How did he feel afterwards? “Embarrassed.” Has he ever broken down on stage, I ask? Absolutely not, he replies; it’s bad enough if his voice catches. “I would sooner shit in my pants on stage than cry on stage.” I look at him, and I believe him.

• Theft By Finding: Diaries Volume One by David Sedaris is published by Little, Brown. To order a copy for £16.14 (RRP £18.99), go to bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min. p&p of £1.99.