For most of the century and a half since Schumann's death, aged 46, his last, unhappy years, spent in the Endenich asylum, have been shrouded in mystery. Clara Schumann, encouraged by a group of close associates that included Brahms, suppressed the music of this period, fearful that it would betray signs of her husband's mental deterioration. His medical records were kept sealed.

People have a craving for knowledge about the lives of the great composers. Instrumental music is unequalled among the arts in its magnificent, even defiant abstractness. It suggests infinite possibilities, without offering any definitive answers. What could be more tantalising?

But Schumann's music excites further curiosity, because it is not only lofty, but personal. Excruciatingly personal. So much of its shattering emotional power comes from the feeling it conveys that confidences are being shared – that Schumann is disclosing the sorts of truths one often hides even from oneself. So how ironic that this most self-revealing of composers has been so often overshadowed, even betrayed, by his biography.

In 1985, the psychiatrist Peter Ostwald wrote Schumann: The Inner Voices of a Musical Genius, the first work to examine its subject with both the rigour of a scientist and the ardour of a music lover. When a new edition was published in 2010, it contained an extra chapter, "Endenich Revisited", written by his widow, Lise Deschamps Ostwald, and drawing on the information in his medical records which were finally released in 2006. It presents a loving and meticulous account of the composer's day-to-day life as his frailties finally defeated him. Schumann's treatment, interactions with doctors, fears and passing delusions are all presented soberly, with an admirable refusal to overreach for conclusions.

New information about an artistic genius is always welcome. Sadly, it's doubtful whether Deschamps Ostwald's research, though invaluable, will dislodge or even refine the commonly held view of Schumann's life: talented, sensitive artist can't cope, goes mad, jumps in the river. Lacking in nuance as this might be, it is verifiably true, and satisfies the human desire for clear narrative, which Schumann's music often resists. His work is defined by its tendency to wander; virtually no composition by Schumann proceeds directly to its finish on the path its start seems to promise. This quality produces moments of heart- and time-stopping beauty, but it makes the music harder to grasp. Instead, the bare-bones version of Schumann's biography becomes the listener's life raft, a means of "explaining" the inexplicable. It also gives us an easy out, a reason not to engage with the most striking, and therefore most unsettling aspects of his music: if we don't understand it straight away, we can just look at what happened to the poor man. Prejudices against Schumann's music that we might otherwise dismiss as facile have been given ballast by his life story, which ultimately obscures his music as much as it informs it.

I have heard these prejudices voiced again and again, and each time, I find it dismaying.

Schumann's music is well represented in concerts and on disc (though, I would argue, by a too narrow sampling), and yet plenty of musicians and music lovers persist in dismissing his large-scale works as rambling, his orchestral ones as grey, his late ones incoherent. Imagine a sizable portion of the art world speaking condescendingly of Van Gogh, and you will have some idea of how this makes me feel.

That is why I have devoted much of the past year to a project called Schumann: Under the Influence, a series of concerts at the Wigmore Hall in London that reaches its conclusion next week. The series has placed Schumann at the centre of his own musical world, surrounding him with the music he admired, and the works of composers who took his unique creative vision as an inspiration.

Under the Influence began seven months ago with the Gesänge der Frühe – songs of the early morning – completed shortly before Schumann's suicide attempt in 1854, two years before his death. These pieces have never entered the repertoire; most piano students are unaware of their existence. Perhaps the work's most remarkable qualities – its lack of purpose, its opaqueness, the extremity of its resignation – are the very ones that have led us to ignore it. Any listener primed by the Cliff's Notes version of Schumann's life to hear mental decay in his late works will find it here. It is stripped of anything externalised, of the desire to please, even to be understood.

We often tend to reduce composers to cliches of their last years: Brahms is bearded; Haydn is grandfatherly; Beethoven is deaf. While even Beethoven's deafness is seen as ennobling, however, our vision of Schumann as spent, lifeless and inert only diminishes him. And us. These descriptions are inevitably one-sided and limiting, but in the case of Schumann, they reveal our failings – of imagination, empathy and courage.

So, please, listen to the Gesänge der Frühe. Any recording will do. They will pose many questions and provide no answers, yet reveal so much of his fragility and beauty. Or, if these late works seem a step too far, begin with the early ones. They are sometimes dazzling in a way that the Gesänge der Frühe never are, but in their finest, most characteristic moments, they are equally mysterious and disconcerting. Davidsbündlertänze, the magnificent 1837 cycle with which the series comes to a close next week, may be a work from Schumann's youth, but it finds him no more at ease with himself or the outside world. The work runs the gamut of expressions, from tender to wild, but is most moving when it is at its most internal – when everything about Schumann, even his desire to communicate, feels closed in on itself. Revealingly, its most extraordinary moment is marked "Wie aus der Ferne" – as if from a distance. This being Schumann, the "distance" is all-encompassing: his feeling of being out of place, out of time is on full view, a gift to his audience that we can repay through listening without judging.