I grew up in a Jewish household. My mother is certainly not religiously Jewish, but a lot of her family, and therefore my family, were destroyed by Nazism, and Wagner became associated for her with Germanic horror. So when I started playing Wagner loudly, her eyes would close and she'd say, "I know you like it and that's fine, but really you know, what's wrong with Mozart?" It was always an issue. Something to confront.

We all recognise that loving Wagner is not the same as loving almost any other artist. We can try to pretend it is, and would that it were so. But we also know there is something profoundly important about this artist: something that shaped the 20th century, in benign ways. Look at the influence he has had on music. Look at Korngold and Schoenberg, look at the entire secessionist movement in Austria. Mahler said: "There's only Ludwig and Richard." And it's not only music. Two of the great modernist works in our literature, Ulysses and The Wasteland, are suffused with Wagner. "Frisch weht der wind / Der Heimat zu. / Mein Irisch Kind, / Wo weilest du?" the opening lines of Tristan und Isolde are quoted by Eliot in The Wasteland. And there's a great deal of it in Ulysses, too.

You can't allow the perverted views of pseudo-intellectual Nazis to define how the world should look at Wagner. He's bigger than that, and we're not going to give them the credit, the joy of stealing him from us.

The older you get, the more you realise that the complexities and ambiguities of art will always allow one person to run a magnet over its iron filings and make them all point one way. But that's doing a disservice to art. Art isn't like that. Art is ornery and shaped by the sense of people and events, not by an overarching ideology.

All of us who love Wagner are familiar with another argument, where people will say: "It's too powerful, there's something wrong with music that can have that much effect." Is it earned? Is it too much? Is it manipulative? Is it unfair for music to have this unbelievable ability to make one shake in quite that way? There are those who disapprove of Wagner purely on that aesthetic ground.

If anyone asks me, "How do I listen to Wagner? How do I get into it?", I say: "Just go to the first act of one of them and follow it." Nobody sings on top of each other, it's all nice and straightforward. It's a drama. It's a story. It's fantastic.

I used to think when I was young that if I was ever asked to produce or direct a Wagner I would like to do The Mastersingers because of its comic shape and unity of action. Then I thought that I would like to do The Ring, and what I would do if I had the budget for it would be to commission the best artists and do an animated Ring. Not a cheesy cartoon, but a truly serious animated Ring. Because – let's face it – in the last five minutes you've got a Valkyrie getting on a horse, galloping into a flaming funeral pyre then being covered by the river Rhine, while the palace of the gods crumbles into dust in the background. All in five minutes. You can do it on stage – and it's wonderful to see some of the solutions that have been devised for portraying that extraordinary last five minutes of Götterdämmerung – but think what you could do with beautifully and properly presented animation. So my ambition is to one day meet a Russian oligarch billionaire who has a passion, because it would cost a huge amount of money. But wouldn't that be fantastic?

Stephen Fry's Wagner and Me opens at Picturehouse cinemas from 26 September. Details: picturehouses.co.uk/wagnerandme. This is an edited transcript based on a Q&A Stephen Fry gave at the Wagner society.