Spitting Image once made a joke about Orson Welles – that he lived his life in reverse. The idea, effectively, is that Welles started life as a fat actor who got his first break doing TV commercials for wine, moved on to bigger character roles as fat men, but used his fees to help finance indie films which he directed himself; their modest, growing success gave him the energy and self-esteem to lose weight. Then the major Hollywood studios gave him the chance to direct big-budget pictures, over which he gained more and more artistic control until he made his culminating mature masterpiece: Citizen Kane, the story of the doomed press baron Charlie Kane – played by Welles himself, partly based on WR Hearst – and told in a dazzling series of fragments, shards, jigsaw pieces and reflected images.

Poor, poor Orson Welles: repeatedly talked about as a tragic disappointment, his achievements somehow held against him, as if he had culpably outlived his own genius. After all, he only created arguably the greatest Hollywood movie in history, only directed a string of brilliant films, only won the top prize at Cannes, only produced some of the most groundbreaking theatre on Broadway, only reinvented the mass medium of radio, and in his political speeches, only energised the progressive and anti-racist movement in postwar America. As the room service waiter in the five-star hotel said to George Best: “Where did it all go wrong?”

Perhaps it is the fault of Citizen Kane itself, that mysterious, almost Elizabethan fable of kingship, which so seductively posits the coexistence of greatness and failure. Martin Scorsese, in his brilliant commentary on the film, said that cinema normally generates empathy for its heroes, but the enigma of Kane frustrates this process. The audience wants to know and love Kane, but can’t – so this need to love was displaced on to Welles himself, and accounted for his immense popularity and celebrity in the 1940s. It is the same with cinema: however immersive, however sensual, however stunningly effective at igniting almost childlike sympathy and love, cinema withholds the inner life of its human characters, while exposing the externals: the faces, the bodies, the buildings, the streetscapes, the sunsets.

The story of Charles Foster Kane is a troubled one: the headstrong newspaper proprietor who makes a brilliant marriage to the niece of the US president and takes a principled democratic stand for the little guy against monopoly capitalism, but only to reinforce his own prerogatives, and only in an attempt to pre-empt the growth of trade unionism. And Kane’s own political ambitions, like those of Charles Stewart Parnell in Ireland, are destroyed by sexual transgression: an affair with a singer who is to become his second wife. Kane’s indiscretion generates precisely the kind of salacious, destructive news story that he had pioneered in his own newspapers.

Diminished by the Wall Street crash and personal catastrophe, Kane becomes a pro-appeasement isolationist, complacently unconcerned about European fascism, though in his youth cheerfully willing to indulge the idea of a short circulation-boosting war with Spain. He dies in the present day, in 1941 – Citizen Kane was released seven months before Pearl Harbor. Kane himself becomes a remote figure, enervated and paralysed by his mythic wealth, somewhere between Scott Fitzgerald’s Jay Gatsby and Adam Verver, the unimaginably rich art collector in Henry James’s The Golden Bowl.

Charlie Kane’s last moments of childhood innocence and happiness. Photograph: Collection/REX Shutterstock.

But how about that tiny detail that Kane’s would-be biographers believe is the key to everything? The murmured word on his deathbed: “Rosebud”. It is a mystery which they fail to solve, but we do not – it relates to Kane’s last moments of childhood innocence and happiness, playing in the snow before his bank-trustee appointed guardian, the Dickensian Mr Thatcher, comes to take him away to prepare for him his lonely new life as a 20th-century American oligarch. Kane’s business manager, Mr Bernstein, played by Everett Sloane, tells us never to underestimate the importance of tiny moments, and famously remarks that never a month goes by without him thinking of a fleeting glimpse he had once of a beautiful girl in a white dress and parasol. Never a week goes by without me thinking of that scene, without me trying to imagine that woman’s beauty, and who might play her in a flashback scene (I suggest Mary Astor) and of the awful fact that Everett Sloane was to become obsessed with his own ugliness and addicted to cosmetic surgery.

For any journalist, Citizen Kane is a glorious, subversive, pessimistic film. We all know what newspaper journalists are supposed to be like in the movies: funny, smart, wisecracking, likable heroes. Not in Citizen Kane, they’re not. Journalists are nobodies. The person who counts is the owner. And Welles’s Charlie Kane is not even a self-made man. He had his wealth handed to him. He was never the underdog. Haughty, impulsive, charming and charismatic: the 25-year‑old Welles is so handsome, leonine, with an intelligent, perennially amused face, like a young Bob Hope.

Citizen Kane ‘Rosebud’ scene

I’ve lost count of the times I’ve watched the scene in which he first shows up with what we would now call his entourage at the offices of the New York Inquirer, the little underperforming paper he seizes on as the cornerstone of his future career – rather in the way Rupert Murdoch started with the Adelaide News. He blows through that dusty office like a whirlwind. Kane derides the idea of his paper remaining closed 12 hours a day: later, he will buy an opera house for his wife to sing in and for his newspapers to promote. And so Kane, in fiction, invented the idea of rolling 24‑hour news, and a vertically integrated infotainment empire. Welles himself had a newspaper column for many years after Kane, and I suspect he thought of himself as in some ways a newspaper proprietor with other people’s money. He told Peter Bogdanovich in their celebrated interview series in 1969 that he never saw Citizen Kane again after watching a finished print in an empty Los Angeles cinema six months before it opened in 1941 – and never stayed to watch the film at the premiere. Perhaps the image of Kane’s failure became increasingly painful.

One of the main characters is Jedediah Leland, played by Joseph Cotten with his handsome, sensitive face. Kane’s college buddy, he has been kept around as a corporate courtier and is, in Leland’s own words, a “stooge”. He has given Kane an intense loyalty which never quite becomes friendship, and gets the job as the drama critic who must review the woeful professional debut of Kane’s second wife, Susan, played by Dorothy Comingore. Leland is pathetic, with neither the cunning to suppress his opinion, nor the courage to express it plainly. He slumps drunk over his typewriter and in an ecstasy of self-hate and masochistic defiance and despair, Kane completes the review himself. Critics are always implicated in the system, says Kane, and the system’s owners are exposed by their attempts to show themselves independent.

Kane has his parallels with British newspaper bosses – in fact, I’m always surprised that the comparison isn’t made more often. He is very like Lord Copper, owner of The Beast in Evelyn Waugh’s novel Scoop, who appreciated the excitement of short, sharp foreign wars. “The Beast stands for strong mutually antagonistic governments everywhere,” said Copper, and to a reporter who has just cabled that there is no war in Cuba, Kane replies: “You provide the prose-poems, I’ll provide the war.” Waugh also said that Lord Copper loved to give banquets, and “it would be an understatement to say that no one enjoyed them more than the host, for no one else enjoyed them at all.” I think of that line every time I watch the magnificent scene in Kane showing the banquet given to celebrate the Inquirer’s success – with dancing girls brought in, shouldering sparkly cardboard-cutout rifles, in honour of America’s forthcoming war with Spain. Cotten’s tense, tired face and sad smile hints at an awful truth: despite Kane’s boyish glee and the apparent general raucous excitement, it might be a terrible strain and unspoken humiliation for these salaried employees to pretend to be enjoying themselves worshipping their boss. I wonder how many newspaper bosses have watched that scene and taken it as a how-to guide for triumphalism at work.

Orson Welles directs a scene from Citizen Kane in Hollywood, July 1940. Photograph: AP

It also reminds me of a strange moment in my life: 20 years ago, I was invited to a colossal party at the Earth Gallery in London’s Natural History Museum, hosted by Sir David English, legendary editor-in-chief of the Daily Mail. It was a lavish, but strangely tense occasion, a notionally generous send off for an editor whom English had forced into retirement. After a speech full of clenched and insincere bonhomie, the editor-in-chief brusquely asked us all to raise our champagne glasses – he did so himself, his arm extended. It was an uncomfortable moment, and quite a few people had on their faces Cotten’s strained smile from Citizen Kane.

Moments are what we are left with in Citizen Kane: a pointilliste constellation of gleaming moments from which we can never quite stand far enough back to see the bigger picture in its entirety. One of the most stomach-turning is the “picnic” that Kane offers to give Susan in a moment of drowsy ennui. Kane and Susan begin to argue in their private tent while music and dancing begin outside, becoming more abandoned and maybe even orgiastic. Welles orchestrates these sounds contrapuntally with the couple’s quarrel, they climax with a strange sound of screaming, as if Kane and Susan’s own malaise had been projected to the party outside.

The scenes of Kane and Susan together in Xanadu are eerie: an Expressionist bad dream, all darkness and weird perspectives, the couple marooned in the gigantic, sinister house, Kane prowling up to Susan while she morosely fits together a jigsaw. Kane wanders to a bizarrely huge fireplace and for a second he looks tiny, and Xanadu looks like the giant’s lair from Jack and the Beanstalk.

And yet Welles’s scenes with Ruth Warrick, playing his first wife, Emily, are no less vibrant, no less meaningful, especially on their arrival home for breakfast as young marrieds, having partied all night – and contemplating going to bed, but not to sleep. It is subtle but still a sexy scene.

It circles back to Rosebud: the anti-riddle of the anti-Sphinx. Welles himself playfully claimed that the word was Hearst’s own term for his wife’s genitalia, and so naturally the mogul was annoyed. Another false trail. The murmuring of “Rosebud” is in one way the film’s teasing offer of synecdoche: the part for the whole, the one jigsaw piece that is in fact the whole puzzle. But it isn’t.

Rosebud is more probably Welles’s intuition of the illusory flashback effect of memory that will affect all of us, particularly at the very end of our lives: the awful conviction that childhood memories are better, simpler, more real than adult memories – that childhood memories are the only things which are real. The remembered details of early existence – moments, sensations and images – have an arbitrary poetic authenticity which is a by-product of being detached from the prosaic context and perspective which encumbers adult minds, the rational understanding which would rob them of their mysterious force. We all have around two or three radioactive Rosebud fragments of childhood memory in our minds, which will return on our deathbeds to mock the insubstantial dream of our lives.

This brings me to my own “Rosebud” theory of the film, the moment that may or may not explain everything. It is in fact the moment that isn’t there, a shocking, ghostly absence that Welles allows you to grasp only after the movie is over: the death of his first wife and his son in an automobile accident. We only hear of it in the newsreel about Kane that begins the film – the brief roundup that we are invited to believe does not get to the heart of the man. But that is the last we hear of it. It happens two years into his second marriage. When does Kane hear this terrible news himself? How does he react to the death of his first wife and his adored little boy? We never know. Welles leaves it out – perhaps he is saying that Kane did not react, that he is too blank, too emotionally nullified, too spiritually deracinated to respond, having made his own complete and ruinous emotional investment in himself, the same egocentricity of self‑esteem culture and image management that has now been miniaturised and democratised in the age of social media. Kane has the plutocrat’s obsession with trying to control those around him in the way that he controls his media empire, whose purpose in turn is to control the way people think. And this is the final unspoken moral of Citizen Kane: a terrible tragedy of ownership and egotism – a narcissistic drowning.

• The Essay: Being Orson series begins on Radio 3 on Monday. Peter Bradshaw’s “Why Citizen Kane Matters” will be broadcast on Wednesday.