My favourite event at a recent writers' festival was entitled Judges and Winners and featured the wonderfully genial panel of Thomas Keneally, Colm Tóibín, Su Tong and John Carey. As I am judging the Man Booker international prize for 2011, I went along in the hope of learning something useful – perhaps a recommendation of some new writer. I had been reading Su Tong (winner of the Asian Booker) for the past few weeks, and was delighted by his wry wit, hesitantly but charmingly rendered by his translator. Keneally and Tóibín are, of course, multiple literary prizewinners, and Carey a multiple judge: the only person ever to chair the Booker twice, as well as chairing the first Booker international in 2005.

"Why aren't you up there on stage?" asked a friend who'd spotted me sitting on my own at the back.

"They don't need me. Too much duplication. I am John Carey-lite."

The discussion, to my surprise, focused almost entirely on the Booker prize, and the participants took predictable attitudes to it. Carey, ever the populist, rather rued the selection of John Banville's The Sea in 2005. Why, asked Carey plaintively, has the award never gone to, say, Nick Hornby? (Or, as he later remarked to me, Robert Harris? We agreed that Fatherland should have been shortlisted for the 1992 Booker.)

On stage Tóibín, usually so accommodating, replied sharply: "It might be useful to remember that John Banville is a genius." In 2005, Banville had puckishly endorsed this very view when he announced in his acceptance speech that it was "nice to see a work of art win the Booker prize". And I, having supported his novel in our judging deliberations, was rather inclined to agree. A lot of people didn't.

The twice-shortlisted Tóibín (shockingly, Brooklyn was not shortlisted in 2009) has not won the Booker, and wryly regards himself as an old hand at "losing" it. On the night of the prizegiving dinner at the Guildhall, he told the audience that the shortlisted novelists each have a camera trained on them, ready to record the delight of the winner.

"And as soon as the winner is announced and it isn't you," he observed, "the cameraman just walks away, and you are left there at the table trying to look composed, and you want to die."

The remark was delivered with practised timing and self-deprecation, and the audience laughed a trifle uneasily, but it carried a great burden of regret. Indeed, Tóibín remarked, until The Sea and then Anne Enright's The Gathering (2007) won the prize, he could at least comfort himself with the observation that the judging panels were prejudiced against Irish writers.

"Now," he sighed, "it seems that it is just me."

When you judge a literary prize you focus relentlessly on who should win, and though you spare an eventual thought for those left behind the process is defined by its result. Though we talk about the winner of a literary prize, we do not adjudge the non-winners to have lost. The next printing of their novels will surely bear the statement "Shortlisted for the Man Booker prize", and sales will rise commensurately.

But from a non-winners point of view, what you have done is lost: baldly, infuriatingly, publicly. In 1980, Anthony Burgess refused to attend the prizegiving ceremony unless told in advance that Earthly Powers had won. Julian Barnes was heard to mutter, just after not winning the Booker in 1998, that it was the worst night of his life. I recently got a modest taste of the experience myself when my book Outside of a Dog: A Bibliomemoir was shortlisted for the 2010 PEN/Ackerley prize for best memoir or autobiography. I went to the ceremony in hopeful mood. Dog is an eccentric sort of memoir, attempting to chart my living through my reading and vice-versa, and my assumption was that if the judges felt its virtues sufficiently to shortlist it then it must stand a good chance.

This hope was certainly exacerbated when, in his summary of the six contestants' works, the judge (I was too anxious to take in his name) gave a very enthusiastic account of Outside of a Dog. I rehearsed my few graceful words of acceptance, and as he prepared to announce the winner, reminded myself not to rise until I heard the "R" in "Rick".

"And the unanimous choice of the judges is … "

I tensed up in my chair, like a sprinter at his blocks.

" … Gabriel Weston, for Direct Red."

I had a shocking impulse to shout out "Wait! Stop! You've read the wrong name!" as Ms Weston stepped forward gracefully and modestly claimed her prize. I took a long slug of my glass of wine, clapped appreciatively, and began to calculate how long I had to stay after her speech, and whether it was right of me to have stopped smoking. It took a further hour, and four cigarettes, before I regained my spirits.

The next day, of course, I bought Direct Red, to see whether it was a worthy winner. It made me feel sick. In one scene Ms Weston has her hand halfway up a middle-aged woman's arse, rearranging her haemorrhoids artfully; in another a man is disembowelled while blood gushes copiously. Sound like scenes from American Psycho? Not quite. Ms Weston is a doctor and her book an account, way too detailed for anyone as squeamish as me, of her life as a young surgeon.

I suspect it is a genuinely good book, and Ms Weston has a thoroughly engaging voice, but I couldn't get through it, and after a time it occurred to me that it didn't matter. Good or bad, readable or unreadable, she won. Me? No doubt about it. I lost.

But when Dog goes into paperback this autumn, I hope they'll put on the cover that it was shortlisted for the PEN/Ackerley prize.