Sometimes I feel tempted not to wear a Remembrance Day Poppy because I don’t like the way people are pressed to wear them. This has nothing to do with what I do or don’t contribute to the British Legion, which is nobody’s business but my own. I mistrust publicised charity and would rather people think I was a mean cheapskate than that I was a noisy Charidee supporter.

The words of Christ in the Sermon on the Mount (Gospel according to St Matthew, sixth chapter, verses two and three) seem quite straightforward:

‘Therefore when thou doest thine alms, do not sound a trumpet before thee, as the hypocrites do in the synagogues and in the streets, that they may have glory of men. Verily I say unto you, They have their reward. But when thou doest alms, let not thy left hand know what thy right hand doeth: That thine alms may be in secret.’

The argument for wearing a Poppy is not to make a declaration of virtue, but to encourage others to do the same, so increasing the amount of charity given to those devastated by war. You probably shouldn’t do this if you haven’t made a contribution yourself. And you are under no obligation to do it. It is a voluntary act of love and respect, not an attempt to win points through apparent goodness. And the practice of party whips thrusting poppies at MPs before Question Time, or broadcasting organisations equipping guests with poppies before going on air (I have heard of both things happening) seems ( at least) dubious to me.

If you don’t want to wear one, don’t. If you want to wear a White Poppy, then you should be free to do so. Pacifism (with which I disagree) is a legitimate opinion which should be openly expressed and debated, and implies no disrespect to the dead of war (many of whom were far from enthusiastic about the wars into which they were conscripted, or disillusioned by wars for which they originally volunteered).

I used to be a bit of a militant poppy-wearer myself, and now recall with a bitter-sweet mixture of amusement and sadness a dinner celebrating my birthday, to which my late father invited me and my late brother, in a rather good North Oxford restaurant, a surprising number of years ago. I’d guess it was the very late 1970s, perhaps pre-Thatcher, certainly in the depths of the Cold War. In those days I had a NATO sticker on my car, to annoy all my CND neighbours with their stupid ‘No cruise missiles’ posters or car-stickers.

My birthday falls in late October, and my brother upbraided me for wearing a Poppy so long before Remembrance Sunday, though I had the strong impression he was telling me off for wearing it at all. I seem to recall an era when he wouldn’t wear one because the little plastic black disc in the middle, when poppies came on bits of wire, bore the words ‘Haig Fund’, and he , not unreasonably, didn’t want to help commemorate Earl Haig. I think that had vanished by then. In those days he was definitely not a Cold Warrior. The reluctant sympathy for Mrs Thatcher which he recounts in his memoir ‘Hitch-22’ was likewise extremely well-hidden. Those interested in his precise opinions on national defence in those days should look up the words “I don’t care if the Red Army waters its horses in Hendon” and see what they find. My father, who never enjoyed our fraternal quarrels and who preferred Trafalgar Day to any other warlike commemoration, was bemused.

Many years later, I found myself visiting Washington DC in late October or early November and called on my brother, who was ostentatiously wearing a Canadian Remembrance Poppy (which looks a lot more like the actual flower than the English one, with its absurd leaf). I laughed inwardly at the transformation, and later strolled round to the British Embassy in Massachusetts Avenue, to obtain an English one.

But, having lived through long years when many fashionable people in politics and the media disdained the Poppy altogether, I am puzzled by the conformism that now seems to apply to it, and which began in the New Labour era. It reminds me of the odd transformation,of the Labour Party, hostile to the Polaris and Trident nuclear deterrent when it was actually some use, militantly in favour of it long after its purpose has gone. The position is so incoherent and and so fervent that I can't help wondering what they really think.

A few weeks ago, on October 16th, I noticed a TV news reporter already wearing a Poppy and thought it absurd so long before the actual Day of Remembrance. Now I see that the actress Sienna Miller, about whose life and opinions I know and care absolutely nothing, has been upbraided on Twitter and by some MP for not wearing a Poppy on a television chat show which I did not watch.

According to Mr Rupert Murdoch’s ‘Sun’ newspaper: ‘SIENNA Miller has been blasted for failing to wear a poppy on Graham Norton's BBC chat show. The actress, 33, was a guest with Burnt co-star Bradley Cooper who did display the red flower of remembrance.

Norton, Dame Maggie Smith and Alex Jennings also paid respects on their lapels. Twitter fans slammed Miller and the Beeb over Friday's show. TheShowOff85 said: "Why is Sienna Miller not wearing a Poppy? Has she refused? If yes why is she allowed on the show? Disrespectful cow." Kcthelegend asked: "Why was she allowed on without one" and CJLeader68 wrote: "Shame Sienna couldn't be a role model & wear a poppy.

" Tory former defence minister Sir Gerald Howarth stormed: "There should be no excuse for not wearing one so we can honour the war dead." The BBC said guests decide whether to wear one.

A source close to Sienna said: "She was wearing the poppy pin but it was taken off as she went on air as it was pulling on the clothes."

Well, I find the remarks: ‘Why is she allowed on the show?’ and ‘There should be no excuse for not wearing one’, dispiriting and sad.

I am now and always will be moved by Remembrance Day. As each year passes I think more of the pain and the loss, and less of the splendour and ceremony. I describe, in my book ‘The Rage Against God’, the impossibly gloomy, soaking services I used to attend as a schoolboy, when almost everyone present around the granite cross in the freezing drizzle had personally seen the snarling face of war, and many had recently lost people dear to them.

But I am also aware of the spirit in which the men went to those wars, and the spirit in which those who survived returned, and it was not a spirit of bombastic super-patriotism, nor of intolerance.

I think they would say and think ’If you don’t want to, you don’t have to. It was the other lot, the ones we were fighting, who insisted on compulsory contributions to the boxes they rattled, and demanded an outward show of support for political conformity’.

In the end, what else was it they were fighting for, other than the freedom not to be pushed around and told what to wear and what to think? If people are going to be bullied into wearing poppies, then the time may come when people of conscience may prefer not to wear them. What memory, exactly, do we seek to honour? That of the noisy politicians, who made those wars, and the noisy newspapers that supported the policies that led to them, or that of the men who, as is inscribed on several moving memorials (does anyone know who wrote them?) all over what used to be the Empire ‘left all that was dear to them, endured hardness, faced danger and finally passed out of the sight of men, by the path of duty and self-sacrifice, giving up their own lives that others might live in freedom. Let those who come after see to it that their names be not forgotten’.

As I read these words I see a lighted doorway in a small terraced house on an autumn evening, and a slight man in his twenties, in army uniform, embracing his wife and small children as he sets out on a journey from which he will not return. It does not seem to me to be an occasion for telling other people what they should feel, think or wear.