When the psychologist Peter Chadwick explained that he was trying to research psychosis he was given short shrift by one of his patients. "You're trying to climb rain, Peter, or sweep sun off the pavement." The desire to build a science of disabling mental states can sometimes seem like wishful thinking, especially to those who have experienced the turmoil of an unquiet mind.

It is therefore no accident that critics of psychiatry have always had a particular dislike for the use of diagnosis. There are those on the outer fringes who still argue that classifying anything as a "mental illness" is fundamentally flawed, but most of the debate centres on the possibility of distinguishing different forms of psychological disability. One of the key issues is whether different diagnoses such as schizophrenia, bipolar or depression represent distinct disorders that have specific causes or whether these are just convenient and perhaps improvised ways of dividing up human distress for the purposes of treatment.

This is a hot and newly contentious topic. The fifth edition of the American Psychiatric Association's Diagnostic and Statistical Manual of Mental Disorders (DSM-5), the book that lists psychiatric diagnoses, is due out in May. The latest revision has emerged after a decade of unfriendly debates over what should be included and where the boundaries should lie.

The most medical approach sees each diagnosis as a separate disease with specific causes. For example, the National Institute of Mental Health, a US government research agency, describes schizophrenia as "a chronic, severe, and disabling brain disorder", something akin to a distinct condition linked to specific genetic risks and brain changes. But diagnoses are not usually derived from scientific discovery but are based on descriptions of experiences and behaviours, which are then tested for their coherence in scientific studies. For those who see mental illness as something best understood at the level of the brain and genetics, the discovery of specific biological differences associated with a particular diagnosis is considered to be good evidence for its validity.

An alternative approach is to see the definition of schizophrenia as a makeshift way of classifying mental distress that clinicians happen to agree on. From this point of view, rather than schizophrenia being a scientific discovery, it's a tradition – varying in its usefulness depending on your point of view. This difference of opinion turns out to be remarkably politicised: the medical model traditionally favours diagnosis, medication and biomedical science, while the social model is linked to the championing of individual experience, psychotherapy and social interventions.

But a growing body of evidence suggests that this divide is both unhelpful and misleading because some of the best evidence that diagnoses do not represent distinct disorders comes not from social criticism but from medical genetics. Observers may note that this is a deliciously uncomfortable situation for both parties. The hardline biological psychiatrists have had diagnoses undermined by exactly the techniques they use to support them and the social constructionists may have to accept that the best evidence for their "humane" conclusions are biological studies which they reject as supposedly "alienating".

This new realisation rests on evidence that genetic factors initially associated with, for example, schizophrenia have now been recognised as equally important in raising the risk for several other problems including epilepsy, attention deficit disorder, autism and learning disability. The risky genetic factors in question stem from both bad luck in the ancestral lottery – that is, inheriting specific versions of genes from your parents – and spontaneous alterations in the layout of the information on the DNA strands that most commonly appear during conception.

If the distinction isn't entirely clear, think of your DNA as a bit like a copy of the family recipe book. But, instead of describing how to create meals, it has instructions for creating proteins – the essential building blocks of your body. Inheriting risky genes is like being handed down a book with a lots of not-so-good recipes from earlier generations but it could also be a problem if your copy of the book had missing pages or, perhaps, unnecessarily repeated pages. These could also lead the body astray and there is increasing evidence that these DNA copy-number variations also raise the risk for several types of psychological and neurological problems. In other words, a particular diagnosis probably just represents one of several outcomes from similar beginnings due to how we are shaped by our life experiences.

Professor Michael Owen, a psychiatrist and researcher from Cardiff University's School of Medicine, has been at the forefront of these new genetic discoveries. "It is no longer tenable," he wrote recently, "to regard these as discrete disorders, or sets of disorders, with specific causes, symptoms and consequences." Despite a scientific conclusion that some might find uncomfortable, he is upbeat about its implications. "I think that psychiatry's acknowledgement of its diagnostic shortcomings is a sign of its maturity," he says. "By acknowledging the shortcomings of our current diagnostic categories we are recognising the need to treat patients as individuals."

And in contrast to the view championed by social constructionists – that understanding the biology of mental distress disregards personal experience – these discoveries imply that it needs to be front and centre, both in scientific studies and when working to help individuals. The mistake made by both sides was to consider biology and lived experience as somehow in opposition when really they are all part of our common humanity.