Der Rosenkavalier represents the apogee of Richard Strauss's popularity, and perhaps of his life. He had a deep affection for it, a profound identification with it; in his will he stipulated that the work's sublime final trio, the greatest of his many love letters to the soprano voice, should be played at his funeral, which took place in 1949, in still war-blasted Munich. A 36-year-old Georg Solti conducted; as the soaring, overlapping lines arched out over the orchestra, each of the three singers, Marianne Schech, Maud Cunitz and Gerda Sommerschuh, broke down, overcome by emotion, the words, speaking of valediction and the hope of new life, unbearably poignant in the circumstances. The great comedy of love and loss, swirling with waltzes and borne aloft with an inexhaustible flow of heart-stopping melody, belonged, like the composer, to a vanished world.

Four years earlier, in April 1945, Lieutenant Milton Weiss of the US army had been dispatched to requisition Strauss's villa in Garmisch-Partenkirchen at the foot of the Bavarian Alps; the frail 81-year-old composer had emerged, blinking. "I am Richard Strauss," the former president of the Reichsmusikkammer said, in a weak but firm voice. "I am the composer of Der Rosenkavalier," as if that explained everything and justified everything, as if nothing more needed to be said. Is it possible that at that inconceivable moment of humiliation, he thought back to the work's premiere 34 years earlier in Dresden – now just a pile of rubble, but then the gleaming centre of his artistic world?

Der Rosenkavalier had been eagerly awaited by the music lovers of Europe and the US. It was the last in a hat-trick of sensational operatic premieres that Strauss had produced at two-yearly intervals: Salome (1907), Elektra (1909) and Rosenkavalier (1911). He was already acclaimed for an equally prodigious succession of orchestral masterpieces – Don Juan, Till Eulenspiegel, Also Sprach Zarathustra, Don Quixote and Ein Heldenleben – all written within 10 years, from 1889 to 1899. His songs, of melting beauty, had become standards of the repertory and he was acknowledged as one of the finest conductors of the age. With the success of his third, shock-horror opera, Salome, he had become the most famous composer of his time, even more so than Puccini. Part of the secret of his success had been sheer unpredictability: if Salome had been shocking on account of its subject matter, Elektra seemed to push dissonance and sheer ferocity of sound to barely tolerable levels. "After hearing the opera," wrote a critic, "I had to rush home to play the chord of C major, to remind myself that it existed."

Elektra was the first fruit of his collaboration with the distinguished Austrian poet and dramatist Hugo von Hofmannsthal, and the announcement that the collaboration was to continue aroused the highest expectations, a mixture of fear and frisson, which were confounded when Rosenkavalier was unveiled. It was certainly a shock, but of the most agreeable kind. The real Strauss, it was felt, had finally emerged – his melodic gift, his sensitivity to language, his grasp of character, his affectionate recreation of an elegant past, all unshackled from the need to prove himself modern. The work quickly went round the world; only Vienna, hilariously, disliked it, deeming it "morbid and unnatural". It was, and remains, racy, beginning with a fairly graphic depiction of athletic coitus; when the curtain rises on the recently pleasured couple, the fact that the young man is played by a young woman only adds to the titillation. The stage throngs with action, comic intrigues, genre scenes, farce, double cross-dressing, but – and this was the mark of Strauss's maturity as an operatic composer – it hinges on moments of the most sublime stillness: the Marschallin's meditation on time in the first act, the presentation of the silver rose in the second, and that final trio, where Strauss (below) matches his librettist for psychological penetration and realisation of the spellbinding power of love. Strauss had learned about the theatre in his first job as a kappellmeister, in the court of Saxe-Meiningen, where the duke maintained not only a superb opera company but a world‑famous theatre troupe. Hofmannsthal was a sophisticated dramatist, part of the astonishing Viennese pre-first-world-war intellectual scene – Sigmund Freud, Joseph Roth, Stefan Zweig – which to a large extent defined the 20th-century mind. Their association continued until Hofmannsthal's death in 1929 – indeed, beyond the grave, because he died before Strauss had finished composing their last work together, Arabella.

Richard Strauss c1890. Photograph: Bettmann/Corbis

Whether or not Hofmannsthal and Strauss consciously intended Marie Theresa, the field marshall's wife and a pivotal figure in Rosenkavalier, to represent a world for which time was running out, it is true that forever after it was Strauss's deepest desire to Stop all the Clocks. Having hit his truest form with Rosenkavalier, he was forever harking back to it, and resisting Hofmannsthal's attempts to take them into deeper, more symbolic areas; but the composer was never again to create an opera as successful or as satisfying. "Our road starts from Rosenkavalier," Strauss insisted, but it was not a road that led anywhere. As Michael Kennedy observes in his masterfully succinct study of the composer (Richard Strauss: Man, Musician, Enigma), it was an end, not a climax.

There is something touching, almost childish, about Strauss's longing to be allowed to be a mere conduit for the music that was in him. "People always expect big ideas from me, big things," he remarked plaintively tothe novelist Romain Rolland. "Haven't I the right after all to write what music I please?" Saint-Saëns, another prodigally gifted composer, had said of himself that he produced music "as a tree produces apples". Strauss said that he wrote music "as a cow produces milk". Hofmannstahl, in the course of their unlikely collaboration, frequently chastised Strauss for his almost bovine quiescence. "The great danger of your life," he raged at the composer, "to which you surrender and from which you try to escape in almost periodic cycles, is a neglect of all the higher standards of intellectual existence."

Strauss was in fact a well-read man, intelligent and shrewd, but he lacked an inner demon. No great composer has been so quintessentially domestic, except perhaps Mozart, but Mozart struggled every day of his life with poverty and the vagaries of the external world. Strauss was, from the beginning, comfortable; by the end he was very rich. He lived well, though was never a bon viveur. At the centre of his existence was his love for his wife, the singer Pauline de Ahna, the difficult woman he addressed as his "dear hedgehog". Hofmannstahl wrote of her as "a bizarre woman with a very beautiful soul, au fond, strange, moody, domineering and yet at the same time likeable." She kept Strauss fascinated and absorbed, a perpetual inspiration: the sphere of intimate, domestic human relations was one he understood deeply.

The external world of great events and big ideas barely impinged on him at all. The first world war passed him by with barely an acknowledgment (just as Mozart somehow missed the French revolution); the Nazi period was more difficult, a regime he cordially detested for its philistinism (the feeling was mutual: Goebbels described Strauss as a decadent neurotic, as misplaced a judgment as can be imagined). Eventually, Nazism became seriously distressing to him, with the exile and death of his Jewish friend and collaborator Zweig, whose name he insisted on printing on the libretto of the opera they made together, Die schweigsame Frau. More personally, his son Franz had married a Jewish girl, and Strauss exercised every last waning bit of influence he possessed to protect her. When the world finally impinged on him to the degree that it impacted on his music, it was with the destruction of Dresden and, symbolically, its opera house, where so much of his work, but above all Rosenkavalier, had first seen the light of day.

"I am in despair," he wrote, and out of that came what is perhaps his greatest, certainly his most emotionally charged work, the Metamorphosen for 23 solo strings: a work of shocking, naked power, this is music of the deepest feeling by a man not accustomed to feeling deeply. Nostalgia had informed a lot of Strauss's music, but this is nostalgia transformed into mourning, a visceral threnody for the world out of which Rosenkavalier had arisen. After that came his swan songs: an oboe concerto which is a 20-minute aria for the instrument – joyous, burbling, tender – and the Four Last Songs, as loving a valediction to the sensual world as has been written, culminating in the great Eichendorff setting which ends: "Wie sind wir wandermüde – Ist dies etwa der Tod?" (How tired we are of wandering. Is this perhaps death?) The questioning but grave and calm phrases with which Strauss set the words were first heard, the year after his death, in the Royal Albert Hall, where Der Rosenkavalier, his exuberant masterpiece from nearly 40 years earlier, is about to be performed.

• Glyndebourne Festival Opera's Der Rosenkavalier, conducted by Robin Ticciati, is Prom 6, on 21 July, at the Royal Albert Hall, London SW7.