The white wine flowed and the music played, but as George Saunders carried off this week's inaugural Folio prize for fiction, the mood in the St Pancras Renaissance Hotel matched the gloomy lighting. The Americans are coming.

Saunders was one of five US writers on an eight-strong shortlist, which featured just one British author – Jane Gardam. The noticeable lack of British authors was matched by the Baileys women's prize for fiction longlist, published last week, which included just four UK novels out of 20. With the Booker prize accepting entries from American authors for the first time in the award's its 45-year history, the industry seems resigned to the prospect of Britain's major literary prizes winging their way across the Atlantic and of British authors struggling for attention in the face of the American juggernaut.

Sitting in the comfortable chairs by the podium, at the Folio prize, the Booker-winning novelist AS Byatt told the Evening Standard that literary judges in Britain were "very depressed by the low standard of most of the work they receive". Byatt declared herself was "very sad" that the Folio shortlist contained so few novels from outside the US, and identified a lack of "energy" in contemporary British fiction.

Speaking to the Guardian after the ceremony she said she was less concerned by any supposed "thinness" in British fiction, but how "mostly American shortlists and longlists will make it harder for good British and other English-language writers to get attention".

"Ever since I started writing, people have been asserting that the Americans were better – I don't think they were necessarily – the cultural cringe does exist and must be resisted."

American literature had always been concerned with the cultural hegemony of "what is America, what is American", Byatt continued, often unconcerned with what went on in the rest of the world.

Byatt cited novels by the Sri Lankan-Australian author Michelle de Kretser and the Pakistani-British writer Nadeem Aslam as just as good as those selected by the Folio judges, but which dealt with Sri Lanka, Pakistan and Afghanistan and were overlooked. "I've talked to other British writers who say they feel – as I do – 'lonely' in these circumstances."

But if some sense a crisis for British fiction, that sense is far from universal. The Booker's new entry requirement may "dilute the prize", according to former Booker judge and professor of English John Mullan, but British writers have "nothing to fear".

"I don't think US fiction is actually going through a golden period," he said. "A couple of years ago the Pulitzer prize even refused to name a fiction winner."

He is not expecting US authors to overrun the 2014 Booker shortlist in the way they swamped the Folio and Baileys awards. "There's a certain kind of showiness in contemporary US fiction," Mullan continued. "It's about the prose, which tends to be copious." Encouraged by the Booker prize, British writers have instead pushed the boundaries of form. "This really strikes you as a Booker judge, when you read 130 novels, how very many of them have multiple narrators, or fractured timelines."

He added: "It would be difficult to suddenly let US novels dominate and would be seen to undermine the whole rationale of the rule change – that there should be not only a global competition, but also dialogue and comparison."

According to the Waterstones fiction buyer Chris White, the US has certainly produced a great deal of exciting fiction in recent years, with the emergence of authors such as Rachel Kushner and Jennifer Egan, Chad Harbach and Sergio De La Pava. It's hard to quantify how many novels published in Britain originate from across the Atlantic, White said, but he estimated that for literary fiction the proportion was "somewhere between 15% and 20%". Interesting writers were emerging on this side of the Atlantic, he said, but they could take a little longer to establish themselves, "probably in large part due to the sheer difference in the size of the markets between the two countries".

"Maybe because of our smaller population it seems to happen that our great writers emerge more in waves rather than as constant stream," White said. "That extraordinary period when authors such as Martin Amis, Ian McEwan, Julian Barnes, Rose Tremain, Hilary Mantel and Salman Rushdie first appeared has perhaps never been repeated. Nevertheless, there is still a lot of terrific work being published by British writers."

Granta had assembled lists of the most exciting British novelists every 10 years since 1983, making additional American selections in 1997 and 2007 to find remarkable writers on both sides of the Atlantic, he added. "The current American talent is towering but are they really more interesting? Over the years the British have proved to be the more diverse, the more surprising and this seems likely to remain the case.

The novelist AS Byatt, who said literary judges in Britain were 'very depressed by the low standard of most of the work they receive'. Photograph: Graziano Arici/eyevine

"It is easy to forget that Zadie Smith – one of the finest writers of any nationality – is only in her mid 30s and there are many young authors – Joe Dunthorne, Ned Beauman, Adam Thirlwell – who have already demonstrated their potential to become literary stars of the future." Britain boasted writers such as Will Self and Tom McCarthy pushing the boundaries of fiction and we had publishers bold enough to publish them, White added, arguing that the willingness of British imprints to publish the American writers featuring on UK prize shortlists suggested they were "prepared to back talent wherever it appears".

For Alex Bowler, editorial director at the British imprint Jonathan Cape, attempts to "draw comparisons along grounds of nationality are false – fiction is freedom … and British fiction is in rude health. "Ned Beauman, Tom McCarthy, David Szalay, Evie Wyld, Ed Hogan, Samantha Harvey, Ross Raisin – all under 45 and writing world-class literature of invention and ambition. Then you've the extraordinary stuff coming from Ireland, your Kevin Barrys, your Colin Barretts, your Keith Ridgways. And that's before we get on to the established generation – as Saunders said when he won, 'I think any country that produces Zadie Smith and Martin Amis, you're good.'"

If literary prizes such as the Folio can offer a useful indication of the literary weather, other barometers are available. No American has won the Nobel prize for literature since Toni Morrison in 1993 – a period that has seen wins for Doris Lessing and Harold Pinter – while writers with strong British connections made a clean sweep of the fiction awards in last week's $150,000 (£90,000) Windham-Campbell prizes.

Pulitzer prize-winning US novelist Junot Díaz finds the idea that "British literature in all of its astonishing breadth is somehow exhausted … a bit absurd". "How can one generalise about a literature when one can only read the smallest fragment of its production?" he asked, suggesting that British despair was "perhaps an expression of defeatist demoralisation in the face of the nonstop depredations of American cultural imperial might".

"Certainly white American writers have the cultural force of their empire behind them," he said, "but more ambition and talent? Please allow me to laugh. One must heed Gandalf in these dark moments and look to the light we have and not what the Eye whispers to us in the dark."

British fantasy writers were "outdoing everyone in the English world", he continued, while "on the literary fiction front Nadifa Mohamed and Sarah Hall are easily my friend George Saunders' equal". British writers who wish to yield the literary field to US authors were not properly valuing their "best and most exciting artists. Which is what empire always teaches us: to value its children before our own."

The independent Chicago bookseller Jeff Waxman finds the idea that British and Irish literature is not read in the US equally strange. "I cannot turn around without hearing some new enthusiasm for Edward St Aubyn," he said. "Hilary Mantel rules historical fiction, John Banville is touring as we speak and Rupert Thomson is about to publish a fabulous novel here with Other Press, which has also been home to three novels by John Boyne and the sublime work of James Kelman. Writers of the United Kingdom, of the Commonwealth, are doing just fine here among our great American writers."

There were many American writers he admires, Waxman continued, but the US was "racked" by a debate between two competing versions of success. While some writers aspire to reaching a mass audience with their work, others prefer to pursue tenure in creative writing departments, as described in Chad Harbach's MFA vs NYC. For these academic authors publication is no longer a way of making money, but merely a new line on a curriculum vitae.

US writers are also "filled with doubts about revelations that our literary culture has been not just affected by our military dominance, but actively cultivated by our intelligence services. There is, in short, a world literature in English worth considering for any prize, and any nervousness from the UK about American literary cultural hegemony strikes me as either hilarious or bizarre."

• This article was amended on 17 March 2014. An earlier version referred to the idea that British, rather than British and Irish, literature is not read in the US.