At 500 pages, The Overstory is a “majestic redwood” of a novel. Its place on this week’s Man Booker shortlist is testament that long books are fine by the judges. It’s the long-winded ones they rejected: “We occasionally felt that inside the book we read was a better one – sometimes a thinner one – wildly signalling to be let out,” said Kwame Anthony Appiah, their chairman.

Most readers can empathise, and may feel that the word “occasionally” was tactful. One book survey found that the average number of pages had increased from 320 to 400 pages between 1999 and 2014. Some think that the shift to digital formats has contributed, not least in removing the fear of being crushed beneath your duvet by your bedtime reading. Val McDermid, another of the judges, cited the inexperience of editors; commercial pressures which deny them the time they need to spend on books; and the unwillingness of writers to listen. The phenomenon of “book inflation” over careers has been noted: the last part of Karl Ove Knausgård’s My Struggle runs to almost 1,200 pages, and Paul Auster’s most recent novel 4321 is almost as long as his previous three books put together. The approving may credit increased boldness and mastery of material. The unimpressed blame growing authorial egos.

Writers are not the only ones reluctant to kill their darlings. Director’s cuts tend to expand rather than contract movies. Viewers of Apocalypse Now Redux – 49 minutes longer than the lengthy original – can testify that there’s sometimes good reason for studios to interfere with a creative vision. Yet executives too can overrate the long and sprawling.

One culprit can be the misguided sense that volume equals value for money. Another is the odd association between physical heft and artistic or intellectual merit – “weighty” is a compliment, “slight” is an insult. One film critic says that studios fear shorter movies will not be deemed worthy of Oscars. The very term the Great American Novel suggests a certain size, though that was not the original intent.

That may be partly the legacy of doorstop classics: Moby Dick, Middlemarch, and Crime and Punishment. (It’s not a purely western phenomenon: China’s beloved The Story of the Stone runs to 120 chapters). Ulysses needs all of its 700-plus pages to capture a single day, while War and Peace, at over 1,200, ranges over 15 years, five families, domestic drama and grand historical events. Readers gallop through Elena Ferrante’s Neapolitan quartet (regarded by its author as a single novel) while short books don’t always keep their readers; A Brief History of Time, despite living up to its titular promise, was bought much more than read. But Persuasion, one of Jane Austen’s shorter novels, is arguably her best. Virginia Woolf’s groundbreaking Mrs Dalloway is not much over 200 pages. Rather a single Bashō haiku than the 1,150-plus pages of James Clavell’s Shogun. The long and short of it is that authors must earn their length.