Late one afternoon some 20 years ago, a close family friend called to tell me of a sudden domestic crisis. My wife and I went straight round to take him out to dinner, during which he began to quote a Thomas Hardy poem, The Darkling Thrush. Upon reaching what might be called the punchline – "Some blessed Hope, whereof he knew / And I was unaware" – our friend choked up, unable to get the words out. This was understandable; he was still upset by the day's events. We ourselves were much moved.

That weekend, we happened to be visiting another close friend, the scholar and critic Frank Kermode, at his home in Cambridge. Frank knew the man involved and was also touched by his Hardy moment. "Is there any poem you can't recite without choking up?" I asked him. Rarely an emotionally demonstrative man, Frank said: "Go and get the Larkin."

In front of his half-dozen guests, he then began to read aloud Unfinished Poem, about death treading its remorseless way up the stairs, only to turn out to be a pretty young girl with bare feet, moving the stunned narrator to exclaim: "What summer have you broken from?" It was this startling last line Frank couldn't get out; with a despairing waft of the hand, he held the book out for someone else to finish the poem.

Also there that day was another Cambridge professor of English, Tony Tanner, so it was not surprising that this topic of conversation lasted all afternoon, ranging far and wide, not just over other candidates for this distinct brand of poetic immortality but the power of poetry over prose to move; the difference between true sentiment and mere mawkishness, and, of course, the pros and cons of men weeping, whether in private or in public.

For the next few weeks, I asked every literary man I met to name a poem he couldn't read or recite without choking up. It was amazing how many immediately said: "Yes, this one" and began its first few lines. With Frank's encouragement, as I reported in to him on regular visits, I began to contemplate an anthology called Poems That Make Strong Men Cry.

Then I remembered I had another book to finish and set the project aside. But it remained a topic of paradoxically happy conversation between Frank and myself until his death in the summer of 2010, at the age of 90. I duly steeled myself to reading Unfinished Poem at his funeral service in King's College Chapel, and managed it – just – without choking up.

In 2007, reviewing AE Housman's letters for the London Review of Books, Kermode had discussed the controversy caused in Cambridge in 1933 by a Housman lecture emphasising the emotional power and appeal of poetry. The very thought was deplored by some Cambridge dissidents, notably FR Leavis, who declared that "it would take years to remedy the damage the lecture must have inflicted on his students".

With our recurrently lachrymose conversation very much in his mind, Frank continued: "Housman included a number of surprisingly personal comments on this topic. Milton's 'Nymphs and shepherds, dance no more,' he said, can 'draw tears… to the eyes of more readers than one'. And tears are only one symptom. A line of poetry can make his beard bristle as he shaves, or cause a shiver down his spine, or 'a constriction of the throat', as well as 'a precipitation of water to the eyes'."

Housman and Hardy have emerged as two of the most tear-provoking poets in the collection Poems That Make Grown Men Cry, to which I was urged to return in the wake of Frank's death (albeit with a somewhat less macho title) by my son Ben, who became the perfect co-editor. With three entries each, they are equalled by Philip Larkin, and bested only by WH Auden, with five.

So, four of us supposedly buttoned-up Brits top the charts of almost 100 poems from 18 countries, a dozen of them written by women, chosen by men of more than 20 nationalities, ranging in age from early 20s to late 80s. Five pairs of contributors happen to have chosen the same poem, for interestingly different reasons, including Richard Dawkins and Andrew Motion (Housman), Robert Fisk and Julian Fellowes (Christina Rossetti), Ariel Dorfman and Javier Marías (de Quevedo), David Puttnam and Salil Shetty, secretary-general of Amnesty International (Tagore).

I was secretly hoping that Simon Callow might choose Stop All the Clocks, the poem famously read over his coffin at the end of Four Weddings and a Funeral; but he merely mentions it in his introduction while opting for another Auden poem, Lullaby – the same choice as Simon Schama.

Early in our task, we were encouraged by a note from Professor John Carey, with whom I discussed our work in progress over a dinner at Merton College, Oxford, where Ben and I both studied English, 30 years apart. "It will bring some good poems to public notice and it will stimulate debate about the emotional power of art and how it affects different people," he said.

Thanks to our partnership with Amnesty, we can add cross-border issues such as freedom of speech and thought, as in the contribution from Wu'er Kaixi, one of the leaders of the 1989 human rights protests in Beijing's Tiananman Square.

It has taken more than two years to reach our target of 100 contributors, three of whom have died, alas, during the book's gestation; beyond Kermode himself, I received an early choice (Wilfred Owen's Dulce et Decorum est) in an email from my lifelong friend Christopher Hitchens just five days before he died, in December 2011, and another from Seamus Heaney (Thomas Hardy's The Voice) a few months before his death last summer.

After deciding to arrange the poems in chronological order, we calculated that some 75 per cent of them were written in the 20th century – inevitable, perhaps, so early in the 21st. The most common themes, apart from intimations of mortality, range from pain and loss to the beauty and variety of nature – as well, of course, as love, in all its many guises. Three of our contributors (the actor Chris Cooper, writer James McManus and architect Richard Rogers) have suffered the ultimate pain of losing a child; others are moved to tears by the sheer beauty of the way a poet captures, in Pope's famous phrase: "What oft was thought, but ne'er so well express'd."

The same might be said of our contributors' candid explanations of their choices, which can rival the poems themselves in stirring the reader's emotions.

Some of those who declined to take part did so for almost poetic reasons. Wrote the pianist Alfred Brendel: "I easily shed tears when I listen to music, experience a Shakespeare play, or encounter a great performance. Literature doesn't have the same effect on me, so it seems. I cannot tell you why, as reading has been an important part of my life."

And the actor-magician Ricky Jay: "Right now, I find it hard to think of a poem that doesn't make me cry. I'm the kinda guy that weeps at reruns of Happy Days." And the playwright-screenwriter Patrick Marber: "Of course I've got one, but I'm not going to share it with anyone else!"

A sudden shock of emotion naturally overcomes different people in different ways. Vladimir Nabokov wrote that the proper reader does so not with his brain or his heart, but with his back, waiting for "the tell-tale tingle between the shoulder blades". To our contributors – from John le Carré to Colin Firth, Salman Rushdie to Barry Humphries, Ian McEwan to James Earl Jones, Jonathan Franzen to Daniel Radcliffe – a moist eye seems the natural, if involuntary response to a particular phrase or line, thought or image. The vast majority are public figures not prone to tears, as is supposedly the manly way, but here prepared to admit to caving in when ambushed by great art.

The youngest of my three sons, now himself a father, Ben is a grown man to whom tears do not come readily, though he tells me he has begun to crack more often since we started compiling the book. I myself, as he has enjoyed telling all inquirers, am prone to weep all too easily, at prose as much as poetry, movies as much as music. We've had a great deal of fun, and not a few vigorous disagreements, while compiling this anthology together.

It was only after intense negotiation, for instance, that we agreed to stretch most definitions of poetry by including an extract from a verse play (Christopher Fry's translation of Ibsen's Peer Gynt, chosen by Kenneth Branagh) and another from a "prose poem" of a novel (Finnegans Wake, chosen by McManus), then another (John Berger's and our faces, my heart, brief as photos, chosen by Simon McBurney). But we drew the line at song lyrics, some of which are fine poetry, for sure, but in my view indistinguishable in their power to move from the music to which they are set.

Ben disagrees, arguing that he can divorce lyrics from their musical setting and citing Leonard Cohen's, for example, as "overlapping" with poetry. Perhaps because he works in feature films – his latest film, The Quiet Ones, is released in cinemas this week, on the same day as our book is published – Ben was adamant throughout that we had to protect the "brand" of our title, so the anthology would have a clear identity for its readers. "It's about poems," as he kept saying. And so, if for different reasons, we resolved to resist lyrics. We did agree to admit one traditional lullaby (Carl Bernstein's choice, All the Pretty Horses), as its folk origins are unclear, and it may well have begun life as verse. But this policy otherwise cost us a distinguished writer intent on a French chanson and an astronaut who wanted the lyrics of a song from a West End musical.

One unexpected bonus is that several contributors have chosen poems by other contributors, lending the collection a pleasing circularity. Paul Bettany, for instance, has chosen Brian Patten (who has chosen Christopher Logue); JJ Abrams has chosen Billy Collins (who has chosen Victoria Redel); Richard Rogers has chosen Craig Raine (who has chosen Ezra Pound) and Richard Curtis and Terry George have both chosen Heaney.

One of our main aims is obviously to encourage more people to read poetry, to which our contributors' potent introductions offer an open door; once you have read them, you cannot help but read the poems themselves. Given that so many of these men are internationally admired, we also hope that schools will find this a handy new way to hook their pupils on poetry.

In his preface, while reminding us that human beings are the only species to cry, Ben sums up our purpose perfectly: "Male tears may have been venerated in European cultures during the 19th century as a sign of high moral character, but these days they are all too hastily wiped away. We want to put paid to that with this anthology. We hope that readers may set each other off as they read these verses aloud to one another. Let's celebrate high emotion! Whatever your gender, background or circumstances, let's express our shared humanity."

We wanted a woman to have the last word, which goes to the Nobel prize winner Nadine Gordimer: "Everyone who reads this collection will be roused: disturbed by the pain, exalted in the zest for joy given by poets."

And yes, since you ask, we are now planning an anthology of Poems That Make Women Cry.

Adapted from Anthony Holden's preface to Poems That Make Grown Men Cry, published by Simon & Schuster on Thursday (£16.99).