Have you ever been stood up by Cory Doctorow? I have. Back in 2010 I was due to interview him at the London Book Fair about his latest novel For The Win. I read his entire back catalogue and planned loads of insightful questions, but when the time came for the interview in the PEN Literary cafe, he didn’t show up. Later, I received an e-mail from him with a preposterous and obviously made-up excuse about how his plane had been grounded by a volcano. So it was me on the stage with an empty chair. (My hastily written chat standard performance poem “The Empty Chair a.k.a Cory Doctorow Is Not Here Today” rocked YouTube, with literally dozens of views.)

Cory’s ash smudged seat was reminiscent of an old PEN tradition. At official meetings and during the annual congress, writers share the stage with an empty chair to symbolise and remember those writers who are absent. They may be in prison, or in exile, or in hospital, or in a grave, because of what they have written. Banned books week is the perfect time to remember these missing authors and poets.

Before we do, another question: Have you ever been threatened with censorship? Perhaps your words unwittingly upset someone, and they have let it been known that they are seeking pay-back. Even in its most benign form, it is a deeply unpleasant experience. I once received a legal letter written on behalf of a wealthy businessman, who I had mentioned in an article for the Guardian newspaper. The letter pointed out a minor inaccuracy in what I had written, and the gentleman in question was a known litigant. I had read barely two lines of the e-mail before my face felt numb and swollen. I began to imagine the conversation with my family, explaining that we would have to remortgage the house to pay off an out-of-court settlement. My panic was thankfully short lived, because the end of the e-mail requested perfectly reasonable corrections. I agreed immediately.

I have always thought of this as the journalistic equivalent of pissing your pants when confronted by a loud noise. We all hope that we would be the kind of person to hold our nerve in a time of crisis. We hope that we would have the courage to to stand up to threats. But in fact, we do not. For most of us, the “flight” response kicks in when censorship looms. For writers, this means agreeing to change what you have written at the first hint of complaint. It means shying away from controversial stories. It means making cryptic references to unnamed people (as I have done in the previous paragraph) rather than naming names.

We all like to think of ourselves as free thinkers. This is especially true if you enjoy reading science fiction or fantasy, which imagines different social systems. But in practice we are not nearly so courageous as we imagine. You, dear reader, are as cowardly as me. Faced with an aggressive challenge to something you have said, I guarantee you will make the choice that means you will keep your salary and your home. You will do whatever was necessary to ensure your publisher and your web host are not dragged into legal battles. Not everyone is a church-goer, but if you offend the high priests of whatever community you value, you will probably back down when they complain. That way, you will keep your friends.

Be aware: There are people out there who are not like you and me. Outliers who somehow fail to do the ‘sensible’ thing and keep quiet. They are the people who publish, even when they have been told not to. The people who speak, even when the consequences of doing so are terrible.

People like Nurmehemmet Yasin, who has been in a Chinese prison since 2004. His crime? Publishing a short story Wild Pigeon, in the Uighur-language Kashgar Literary Journal. The story is a short, tragic tale of a beautiful bird that has been captured by humans. The authorities deemed this too subversive and locked him away on charges of “inciting Uighur separatism.” He is married with two young sons. They do not know whether he is alive or dead.

People like Tal Al-Mallouhi, a Syrian student who was only 19 years old when she was picked up by the security services in 2009. Her dissident poems, which she published online, were deemed “divulging information to a foreign state,” and she was given five years in prison.

One more outlier: Nguyen Xuan Nghia. He is a Vietnamese poet, and has been in prison since 2009 for “propaganda against the Government” (in other words, for his writing in support of democracy). Nguyen is an extreme outlier—someone who repeatedly does the counter-intuitive thing. Despite being originally employed as writer for the government newspapers in Vietnam, he consistently produced pro-democracy pamphlets until his writing was banned in 2003. In similar circumstances, the rest of us would have stopped publishing and chosen a less controversial career. Instead, Nguyen persisted in editing the underground democracy journal To Quoc (‘Fatherland’) until he was arrested, charged and sentenced to six years in prison after a summary trial.

The extreme censorship of a prison cell is usually enough to suppress the voices of those who have unwisely challenged authority. But Nguyen Xuan Nghia continues to do the opposite. Even while in prison, he has refused to be silenced. In July 2013, during a prison visit with his wife, Nguyen passed on the news that his fellow political prisoner, the blogger Dieu Cay, was on hunger strike. This fact was something that the prison administration had sought to keep from human rights observers. According to reports, the prison guards immediately muffled him and dragged him out of the visiting area. He has now been placed in solitary confinement and is suffering health problems.

The Banned Books campaign is a fantastic way to get people reading ground-breaking literature. There is nothing quite like the frisson of turning the page on a book that you know someone else does not want you to read. But as you crack the spine on, say, Cat’s Cradle or A Clockwork Orange, pause for a moment to consider another kind of banned books. Think of the library of potential books that are as yet unwritten, because their authors are languishing in prison. What if your favourite novel was among them?

Banned Books Week 2013 is being celebrated from Sept. 22 to the 28; further information on Banned and Frequently Challenged Books is available from the American Library Association.

Robert Sharp is the Head of Campaigns & Communications at English PEN, the founding centre of the PEN International association of writers campaigning for free expression and the promotion of literature across frontiers.