Sigrid Nunez has won the top prize at the prestigious National Book Awards in New York on Wednesday night, winning the fiction category for her seventh novel, The Friend, about a woman grieving the loss of her beloved literary mentor as she inherits his mourning dog: a 180-pound Great Dane.

Nunez beat Jamel Brinkley’s short story collection A Lucky Man; Florida by Lauren Groff; Where the Dead Sit Talking by Brandon Hobson; and The Great Believers by Rebecca Makkai.

Accepting the award, she quoted the British writer Alan Bennett, who said: “For a writer, nothing is ever quite as bad as it is for other people because, however dreadful, it may be of use.”

Nunez continued: “I became a writer not because I was seeking community but rather because I thought it was something I could do alone, and hidden, in the privacy of my own room. How lucky to have discovered that writing books made the miraculous possible: to be removed from the world, and to be a part of the world at the same time.”

The event was hosted by the occasionally lewd Nick Offerman, who opened with a speech in praise of literature in dark times – a theme that ran through the evening. “In an age where our first amendment rights and truth itself are very much in peril, books remain the ultimate repository of creative ideas and irreplaceable knowledge,” he said. “What can I say, they make me horny.”

Jeffrey C Stewart with his award for nonfiction. Photograph: Brad Barket/Invision/AP

Jeffrey C Stewart won the award for nonfiction for The New Negro: The Life of Alain Locke, a biography of Locke, who became the first black Rhodes scholar in 1907 and is often hailed as the father of the “Harlem renaissance”.

“As a gay man who lived a closeted life, [Locke] had many struggles, and one of them was with tremendous, crushing aloneness,” Stewart says. “So when I stand here I think about his achievement, and what that was was to create a family among writers and artists and dancers and dramatists, and call them The New Negro. The basis for a new creative future – and not just for black people. A new negro, for new America.”

The young people’s literature category winner was The Poet X by the poet and author Elizabeth Acevedo. A novel in verse, it tells the story of a young Afro-Latina girl in Harlem who turns to slam poetry to make sense of the world around her.

Acevedo, who was raised in a Dominican household, said: “As the child of immigrants, as a black woman, as a Latina, as someone whose accented voice holds certain neighbourhoods … I always feel like I have to prove that I am worthy enough.

“There will never be an award or accolade that will take that away, it’s how I walk through the world … but every single time I meet a reader who looks at me and says, ‘I have never seen my story until I read yours,’ I am reminded of why this matters.”

The award for poetry was won by the black queer poet Justin Phillip Reed for his first full-length book of poetry, Indecency.

“I don’t think I’ve ever been so shallow-breathed since having an asthma attack, which I haven’t had in a while, so it’s a very nostalgic experience to be here,” he said. “I meditated on the courage it might take to feel worthy of this, and now I’m standing here with ancestral hands on my shoulders still not knowing what to make of [it].”

I refuse to live in fear – let alone to vote in fear. This is a dark time, my friends Isabel Allende

This year the foundation added a fifth category, of translated literature (“Suck on that, Muslim ban,” Offerman quipped), comprising fiction and nonfiction from anywhere in the world. One hundred and forty books were read in the judging of the category, which was won by the Japanese author Yoko Tawada’s The Emissary: a dystopian novel set in Japan after a mysterious disaster, translated by Margaret Mitsutani.

The National Book Awards were established in 1950 and have been run by the National Book Foundation since 1988. Ten of the 25 finalists were published by independent presses; each receives $1,000 and each winning author receives $10,000.

Isabel Allende became the first Spanish-language author to receive the lifetime achievement award for distinguished contribution to American letters, which has previously been won by John Updike, Toni Morrison, Joan Didion and Tom Wolfe. Allende is the bestselling author writing in Spanish in the world today, and has sold more than 70m copies of her work.

The award was presented by the Pulitzer prize finalist Luís Alberto Urrea, who praised Allende as a personal hero, without whom his own career wouldn’t have been possible. “You set a high bar for us, for those who wondered if anyone would ever read their words,” he said. “For those of us who might have grown up dreaming in Spanish. For my sisters, you showed the way. You made it possible”.

Accepting the award with a rousing and political speech that inspired a standing ovation, Allende dedicated her award to “millions of people like myself who have come to this country in search of a new life”.

She added: “I look Chilean, and I dream, cook, make love and write in Spanish. Make love – it would feel ridiculous panting in English,” she said, to laughter. “My lover doesn’t speak a word of Spanish. I have a lover at 76! You wouldn’t believe that. He’s a brave man.”

Isabel Allende at the New York ceremony. ‘I write to preserve memory,’ she said. Photograph: Brad Barket/Invision/AP

Allende was born in Chile and spent 13 years as a political refugee in Venezuela; she has been an immigrant in the US for more than 30 years.

“I refuse to live in fear – let alone to vote in fear,” she said. “This is a dark time, my friends. It’s a time of war in many places, and potential war everywhere. A time of nationalism and racism; of cruelty and fanaticism. A time when the values an principles that sustain our civilisation are under siege. It’s a time of violence and poverty for many; masses of people, who are forced to leave everything that is familiar to them and undertake dangerous journeys to save their lives.”

She reminded the audience of Alan Kurdi, the toddler whose body washed up on a beach after his family escaped the war in Syria. “Alan Kurdi symbolised the plight of millions of desperate people. For an instant the world was shaken by the image … But the world quickly forgot. I write to preserve memory.”

She continued: “I am just part of this massive diaspora and, although I’m critical of many things about this country, I am proud to be an American citizen. This national award is an extraordinary gift for me. It means that maybe I’m not an alien after all. It means that maybe it’s time to plant my roots and relax … Maybe I have found a place where I can belong. Maybe I’m not going anywhere any more.”

The author Doron Weber, the vice-president and program director of the non-profit philanthropic organisation the Alfred P Sloan Foundation – which supports research and education related to science, technology and economics – was awarded a lifetime achievement award for outstanding service to the US literary community. The award, which has previously been won by Maya Angelou and Dave Eggers, was presented by Human Computer Project founder Margot Lee Shetterly, whose book Hidden Figures, about black women mathematicians at Nasa, was the beneficiary of a Sloan foundation grant.

“We need to safeguard creative freedom for writers of every stripe and all nonpartisan forms of knowledge,” Weber said in a wide-ranging speech on the urgency of protecting and communicating science as we “strive together to repair the world”.

“We cannot make progress without science nor can we begin to understand modern life.”

The 2018 National Book Award winners

Fiction

Winner: The Friend by Sigrid Nunez

A Lucky Man: Stories by Jamel Brinkley

Florida by Lauren Groff

Where the Dead Sit Talking by Brandon Hobson

The Great Believers by Rebecca Makkai

Nonfiction

Winner: The New Negro: The Life of Alain Locke by Jeffrey C Stewart

The Indian World of George Washington: The First President, the First Americans, and the Birth of the Nation by Colin G Calloway

American Eden: David Hosack, Botany, and Medicine in the Garden of the Early Republic by Victoria Johnson

Heartland: A Memoir of Working Hard and Being Broke in the Richest Country on Earth by Sarah Smarsh

We the Corporations: How American Businesses Won Their Civil Rights by Adam Winkler

Poetry

Winner: Indecency by Justin Phillip Reed

Wobble by Rae Armantrout

American Sonnets for My Past and Future Assassin by Terrance Hayes

Ghost Of by Diana Khoi Nguyen

Eye Level by Jenny Xie

Young people’s literature

Winner: The Poet X by Elizabeth Acevedo

The Assassination of Brangwain Spurge by MT Anderson and Eugene Yelchin

The Truth as Told by Mason Buttle by Leslie Connor

The Journey of Little Charlie by Christopher Paul Curtis

Hey, Kiddo by Jarrett J Krosoczka

Translated literature

Winner: The Emissary by Yoko Tawada, translated by Margaret Mitsutani

Disoriental by Négar Djavadi, translated by Tina Kover

Love by Hanne Ørstavik, translated by Martin Aitken

Trick by Domenico Starnone, translated by Jhumpa Lahiri

Flights by Olga Tokarczuk, translated by Jennifer Croft