The shortest day – or longest night – of the year it seems an apt moment to consider the enduring appeal to authors of the hours of darkness

I write this from a swivel chair at 4.17am. Twitter has gone quiet. There is darkness for miles. I can hear a watch tick. It's the longest night of the year, and if I time things carefully, I could avoid daylight for 48 hours. What's more, research suggests it won't just be me. There's a mislaid family of readers and writers at night, and at this hour there's nothing else to do but search for them.

Robert Frost was up late. So were Delmore Schwartz, Alan Ginsberg, Pablo Neruda, Charles Dickens and Carol Ann Duffy. "The hour is midnight and the library is deep and carried like a dreaming child into the darkness of these pages," wrote Richard Brautigan. James Tipton seems to suggest that poetry itself is sleeplessness, a oneness with things only amassable at night. "A child," said Sylvia Plath, "forming itself finger by finger in the dark."

Does the night absolve the day? Susan Rebecca White wrote after long shifts at a Middle Eastern restaurant, "still smelling of hummus and lamb". Tennessee Williams wrote after days as a clerk at the International Shoe Company, Kafka after insurance, TS Eliot after banking. JD Salinger was sent to military school aged 15, where he wrote under bedsheets by torchlight. His last, unpublished work – written in slippers and robe in New Hampshire, and burned at dusk – was a song to insomnia, a "memoir of the night" of which only 16 pages remain.

Is it the peace and quiet? "I wrote [Twilight] mostly at night," Stephenie Meyer has said. "After the kids were asleep so that I could concentrate." So did Danielle Steele, Jacquelyn Mitchard, Barack Obama. "Now," says Allison Leotta, "the sound of a softly snoring baby triggers a Pavlovian response in me to start typing."

Joyce lay on his stomach, with huge blue carpenters' pencils and a bright white coat to reflect light. Proust lined his bedroom walls with cork so he could sleep through the Paris day and write at night. Kafka wrote The Judgment in a waking nightmare between 10pm and 6am. "Writing is a deeper sleep than death," he wrote to Felice Bauer. "Just as one wouldn't pull a corpse from its grave, I can't be dragged from my desk at night."

"It's like being in the basket of a blimp, working at that hour," said Donald Westlake, author of 90 novels. "It's wonderful. There's just one little room with me in it, and I'm sailing through the night wherever the story will go." And in "A Letter that Never Reached Russia", Nabokov's exiled lover writes home, "At night one perceives with a special intensity the immobility of objects – the lamp, the furniture, the framed photographs on one's desk. Now and then the water gulps and gurgles in its hidden pipes as if sobs were rising to the throat of the house."

It's 5.35am. My last search is stuck open – an anonymous poster in a doctor's community forum. He says: "A novel is before all else a living being. The ideal novel is written in a single breath, by the light of a single candle, during a single night, as it must be read in a single night. The sleep that follows is eternal."

What time is it there? Why do we write at night?