It is time to return to what feminism has to tell us. It is time to make the case for what women have to say about the perils of our modern world. But the case cannot be made along the lines that have become most familiar. We cannot make it only by asserting women's right to equality or by arguing that women are qualified to enter the courts of judgment and the corridors of power. Those claims are important but they tend to be made – loudly, as they must be – to the detriment of another type of understanding, less obvious but no less vital, that makes its way into the darker spaces of the world, ripping the cover from the illusions through which the most deadly forms of power sustain and congratulate themselves. This we might call the knowledge of women. In its best forms, it is what allows women to struggle for freedom without being co-opted by false pretension or by the brute exertion of power for its own sake.

This calls for a new language for feminism. One that allows women to claim their place in the world, but that also burrows beneath its surface to confront the subterranean aspects of history and the human mind, but which our dominant political vocabularies most often cannot bear to face. We need to draw on women's ability to tell that other story, to enter that domain and then return to tell the tale. That the personal is political is, of course, a well-worn feminist claim. In the beginning it rightly drew attention to the way that women's private and family lives were soaked in the ugliest realities of patriarchal power. But if the claim has faded somewhat, it might be because it shied away from the most disturbing component of its own insight – which is that once you open the door to what is personal, intimate, you never know what you are going to find. My plea is for a feminism that does not try to sanitise itself. We need a scandalous feminism, one that embraces without inhibition the most painful, outrageous aspects of the human heart, giving them their place at the very core of the world that feminism wants to create. Certainly it will be a different world from the one that feminism is meant to aspire to – sane, balanced, reasoned, where women are granted their due portion. Not because these aspirations should not be met, nor because we want a mad world, but because women have the gift of seeing through what is already crazy about the world, notably the cruelty and injustice with which it tends to go about organising itself.

Pick one newspaper at random on any day of the week. Evidence of cruelty and violence against women is to be found on almost every page. Last month Oscar Pistorius, the South African Paralympic champion, a national and international sports hero, was cleared of the charge of murder for shooting his girlfriend Reeva Steenkamp in February 2013 through a locked bathroom door (he was found guilty of the lesser charge of culpable homicide). The judge argued that his intention to kill Steenkamp could not be proven, even though she also insisted that anyone shooting four times through a door must have known that killing would be – or at the very least could be – the consequence. For those who recognise in this case the classic signs of domestic violence, this judgment has been an affront, not to speak of the argument put forward by his lawyer this week that Pistorius should avoid a prison term. During the trial, campaigners in the UK argued that the tightening of legal aid rules and budget cuts are placing victims of such violence increasingly at risk: "Women will die," they stated. "It's not too dramatic to say that." Over the same few days, Carol Howard, one of the few female firearms officers in the police force, was described in court as subject to a vicious campaign of smears after she had complained about racial and sexual discrimination (piling abuse on abuse). In the summer it emerged that 1,400 young girls in Rotherham had been the targets of grooming and sexual exploitation. According to actor Samantha Morton, speaking for the first time in response to Rotherham of her own abuse in care homes as a child, this is probably the tip of the iceberg. Something vile is gradually coming to light. We now know that the behaviour of Jimmy Savile and Rolf Harris was just one piece of an entertainment culture where such abuse was as endemic as it was tolerated and ignored. Meanwhile, we learn this last week that sexual harassment and the assault of women on campus is on the rise – one police officer described "freshers' week" as a "killing field" for sexual violence.

A demonstration in Abuja last month calling on Nigeria's government to do more to rescue hundreds of girls kidnapped by Boko Haram. Photograph: Olamikan Gbemiga/AP

We are talking about an international climate of violent oppression that seems to be worsening by the day. The 200 schoolgirls kidnapped by Boko Haram in northern Nigeria in April this year are still missing. According to the agency UN Women, more than half the working women in the world are without any legal rights, a similar number have no protection against domestic violence, and sexual assault has become a hallmark of modern conflict. On International Women's Day last year, 50 signatories, ranging from human rights lawyers Helena Kennedy and Philippe Sands to singer and songwriter Annie Lennox, observed in a letter to the Guardian that women aged 15 to 44 the world over are "more at risk from rape and domestic violence than from cancer, car accidents, war and malaria combined". Recent statistics show that nine times as many people are killed in disputes between individuals, including victims of domestic violence, than are killed in wars and global conflicts (the cost of individual violence amounts to more than two-thirds of the total cost of £6tn for all forms of violence). For the first time, "coercive control" – financial, psychological, physical, sexual or emotional abuse of women by their male partners – is entering the legal lexicon.

Today, domestic violence, female genital mutilation and rape as a weapon of war, classified as a war crime since 2008, are increasingly in the public eye. Seventeen-year-old schoolgirl Fahma Mohamed has spoken out about FGM and succeeded in getting the UK government, and the secretary general of the UN, to listen. Malala Yousafzai, who was shot by the Taliban on her way to school in Afghanistan, has shown to the world the violence women can often face if they assert their human right to be educated (to be human, one might say). "One bullet, one gunshot, heard around the world," she said when awarded the Nobel peace prize last week, although, as Kamila Shamsie has pointed out, it is still not safe for her to return to Pakistan. These hideous acts cannot be equated. Together and separately, they each require a sustained form of reckoning. We can also only hope that the policy changes that have been promised – greater police sensitivity towards domestic violence in the UK, education on FGM in all UK schools, the international summit on rape in times of war in London in June – will be sustained and make a difference (although it will take much more than any of these). In each case, however, something is being spoken, mostly by women, that has been ignored or hidden away. That in itself is worth noting. Young women such as Mohamed and Yousafzai are not just speaking out in a world that mostly does not want to listen. They are daring a form of speech addressed directly, and without apology, at the violence targeted against them.

Reckoning with such violence may be the greatest challenge feminism faces today. Certainly, I cannot remember another moment where it has appeared so rampant and so brazen. And yet there was a time when listing these cases, as I have just done, would be seen to target all men as responsible for the ills of the world. To which some feminists, myself included, would object that to do so is to paint men in only their worst colours, to shut men, and also women, with no exit route, inside society's most debilitating frame. It is also to assume that testosterone-fuelled behaviour, as one feminist argument runs, reflects – across the centuries and for all time – who or what men inevitably are and always will be. This is to ignore the fact that, even if many embrace the task all too readily, men, as indeed women, have to be built into their roles. Since Simone de Beauvoir, it has been the fundamental premise of feminism that women are not born but made, an argument that assigns sexual identity to the realm of culture and can therefore only work if it also applies to men. Otherwise we enter a weird scenario where men are pure biology, women pure culture, whose only advantage might be that it reverses the prevalent cliche: that women belong in some bodily realm, closer to nature, and men out there in the world, at the heart of public, social life. It is also to ignore the fact that the rise in reported violence against women over the past years, in the UK at least, has mirrored the country's economic downturn to which it is at least in part the desperate, brutal response. Not, of course, that this is an excuse.

Nawal El Saadawi, author of The Hidden Face of Eve. Photograph: Felix Clay

We need to ask, today perhaps more than ever, why men turn against women when it is their own masculinity that is threatened; why does assaulting a woman appear so often to be the best way to compensate masculine failing and distress. To pose this question takes us down difficult paths, into the darkest zones of the soul, where feminism, perhaps alone of political movements, does not – in my version – fear to tread. It is to suggest that when men enact violence against women, they are at once fulfilling their requisite identity but also bearing witness to its frailty. In her famous text of 1977, The Hidden Face of Eve, Egyptian feminist Nawal El Saadawi suggested that Arab men – but not only Arab men – cannot abide an intelligent woman, because: "She knows very well that his masculinity is not real, not an essential truth." So-called masculinity is at once the crudest of weapons and a confidence trick: it always knows somewhere that its number might be up. Because asserting masculinity never really works, it has to be done over and over again.

Might it also be, however, that there is something about sexual difference that generates violence in and of itself? This could help us understand the agony of feminism, why the progress of women, despite the many hard-won advances or perhaps because of them, is so tortuously slow, liable to go into reverse at any point – which is also why feminism cannot stop and why it is folly to suggest that the task of feminism is done. This is neither a biological argument nor simply one based on culture, but belongs in a murky, not easily graspable place somewhere between the two. A place of unreason where we all, women and men, also reside and that runs through the world, fuelling as much its hatreds as its strengths. What do men see when they look at women? No more or less, one psychoanalytic account would argue, than a threatening difference from themselves. For the social critic and philosopher Hannah Arendt, difference tout court was unmanageable and could easily explain violence in the modern world. Stripped of her or his national status, the stateless person of the 20th century is anathema because she or he presents difference – she called it the "dark background of mere difference" – to the world in its rawest, most debilitating state. This is the realm, she suggested, in which "man cannot change and cannot act and in which, therefore, he has a distinct tendency to destroy". We are up against the paradox of power that feminism lays bare like nothing else: the worst exercise of human power is the consequence of mankind's impotence.

Campaigners, some dressed as suffragettes, attend a rally organised by UK Feminista to call for equal rights for men and women in 2012. Photograph: Oli Scarff/Getty

Arendt's focus was not hatred of women, but it only takes the smallest nudge of her theory for something about the death-dealing side of sexual difference to come into focus. Sexuality unsettles because it confronts us with what we cannot master, the realm of the unconscious where desires run havoc, where it is impossible by definition simply to be true to oneself. This is a place where knowledge falters and confronts the limits of its own reach. According to the famous British analyst Melanie Klein, there is a mismatch between the two sexes (we are a long way from the heterosexist ideal that celebrates men and women as each other's perfect complement). The boy struggles to relinquish an identification with the body of the mother. Klein was not renowned for her social commentary, but, in an intriguing remark almost thrown out as an aside in an essay on early childhood, she suggests that this might help us understand why, as she puts it, a man's rivalry with women "will be far more asocial than his rivalry with his fellow men". The possibility of being a woman is etched into the body and soul of the boy, because he has already been there. For the boy, knowing the terrain in the most intimate part of himself and then rejecting it with all his might is how he comes to be a man. For the girl, however complex her future sexual identity, whatever difficult sexual path she may take – and for psychoanalysis all sexual paths, even the ostensibly "normal", are difficult – this is a realm in which, one way or another, she will come to recognise, to place and displace herself. It is not one that, in order to become a woman, she is required – commanded might be the better word – to repudiate.

A man's rivalry with women can be traced to a knowledge he would prefer to forget. His rivalry with men, however ghastly – war, political conflict or simply sizing up in the changing room – is in a strange way more civilised, expected. According to this argument, men who assault women do so not because it is in their blood – we are not talking about an instinctive, inbuilt violence of the male species – but because every woman is a reminder of a ghostly, womanly past that no man "worthy" of the name, if that is what he is, can any longer afford to recognise. The key is the unexpected word "rivalry" of Klein's formulation, which implies men and women are not too different, but too alike. There is something potentially liberating about the idea that somehow, in the beginning, the sexes were so profoundly intermeshed. But not for long, not in the "normal" run of things, not if the police of our inner lives do their work and subordinate the untold vagaries of our sexual being to the world's requirements.

Laurie Penny, one of the new generation of feminist writers. Photograph: Jon Cartwright

It does not always work – that too is the founding insight of psychoanalysis. The requisite sexual identity never exhausts our possibilities, the psychic repertoire of any one life. Not all men behave in this way. But there is no violence more deadly and uncontrolled – asocial, precisely – than the violence that is intended to repudiate the one who, deep down in a place beyond all conscious knowledge, you once were and perhaps still might be. Any woman is then liable to attract the residue, the afterbirth, of that earliest moment along with everything that cannot be fully controlled or known in the world and in the mind: the realm that, in Arendt's words, "man cannot change and cannot act [in] and in which, therefore, he has a distinct tendency to destroy". And, she suggests, what cannot be controlled above all is the messy, unpredictable moment of a new birth, a new beginning, to which women are summoned and with which, whether mothers or not, women are identified. Over such beginnings, Arendt continues, "no logic, no cogent deduction can have any power because the chain presupposes, in the form of a premise, a new beginning". Terror, notably totalitarianism in the last century, is needed, "lest with the birth of each new human being a new beginning arise and raise its voice in the world". We cannot control the world and we destroy it if we try. This is something, I would argue, that feminism at its best knows well, even when the fight against an unjust destiny, women seizing their lives for themselves, is at the core of its struggle.

Today, a fourth wave of feminism – UK Feminista, EverydaySexism, a new generation of writers such as Laurie Penny and Nina Power – bears witness to the fact that the task of feminism is not done. And yet it is often argued that modern women are free, that sexuality is something that women, in the west at least, control and dispose of at will. In fact, women are not free, not even in the west, where the inequalities are still glaring. Certainly, it must be one of the goals of feminism for women to be freer in their sexual life. But we must be careful not to exchange an injustice for an illusion. We are nowhere more deceived than when we present sexuality, not as the trouble it always is, but as another consumable good. Sexuality always contains an element beyond human manipulation, however free we think we are. To assert otherwise is a type of daylight robbery that knocks the humanity of all women, and indeed men, down by at least a notch. We kid ourselves if we think that human fear of sexuality, and then the hatred of women that is so often its consequence, is something that the so‑called reason or enlightenment of our modern world can simply and safely dissipate. I do not want feminism to hitch itself to this wagon. Indeed, rather than the idea of light triumphing over darkness, confronting dark with dark might be the more creative path. If there is such a thing as a knowledge of women, this, I would venture, is where we should go looking for it. Feminism should alert us to the world's unreason. But it should also insist that to respond by making reason's diktat our sole mantra and guide is as impoverishing as it is deluded and dangerous.

Let feminism, then, be the place in our culture that asks everyone, women and men, to recognise the failure of the present dispensation – its stiff-backed control, its ruthless belief in its own mastery, its doomed attempt to bring the uncertainty of the world to heel. Let feminism be the place where the most painful aspects of our inner world do not have to hide from the light, but are ushered forth as handmaidens to our protest. The feminism I am calling for would have the courage of its contradictions. It would assert the rights of women, boldly and brashly, but without turning its own conviction into a false identity or ethic. It would make its demands with a clarity that brooks no argument, but without being seduced by its own rhetoric. The last thing it would do is claim sexuality as prize possession or commodity. This is a feminism aware that it moves, that it has to move, through the sexual undercurrents of our lives where all certainties come to grief. Otherwise, it too will find itself lashing out against the unpredictability of the world, party to its cruelties and false promises. Such a feminism would accept what it is to falter and suffer inwardly, while still laying out – without hesitation – its charge sheet of injustice. This might just, I allow myself to think, be an immense relief for many of both sexes. It is, for me, what feminism uniquely has to bring to the increasingly dark times we live in.

• Jacqueline Rose's book Women in Dark Times is published by Bloomsbury (£20). To order a copy for £15.49 with free UK p&p call Guardian book service on 0330 333 6846 or go to guardianbookshop.co.uk.