It is the mainstay of countless magazine and newspaper features. Differences between male and female abilities – from map reading to multi-tasking and from parking to expressing emotion – can be traced to variations in the hard-wiring of their brains at birth, it is claimed.

Men instinctively like the colour blue and are bad at coping with pain, we are told, while women cannot tell jokes but are innately superior at empathising with other people. Key evolutionary differences separate the intellects of men and women and it is all down to our ancient hunter-gatherer genes that program our brains.

The belief has become widespread, particularly in the wake of the publication of international bestsellers such as John Gray's Men Are from Mars, Women Are from Venus that stress the innate differences between the minds of men and women. But now a growing number of scientists are challenging the pseudo-science of "neurosexism", as they call it, and are raising concerns about its implications. These researchers argue that by telling parents that boys have poor chances of acquiring good verbal skills and girls have little prospect of developing mathematical prowess, serious and unjustified obstacles are being placed in the paths of children's education.

In fact, there are no major neurological differences between the sexes, says Cordelia Fine in her book Delusions of Gender, which will be published by Icon next month. There may be slight variations in the brains of women and men, added Fine, a researcher at Melbourne University, but the wiring is soft, not hard. "It is flexible, malleable and changeable," she said.

In short, our intellects are not prisoners of our genders or our genes and those who claim otherwise are merely coating old-fashioned stereotypes with a veneer of scientific credibility. It is a case backed by Lise Eliot, an associate professor based at the Chicago Medical School. "All the mounting evidence indicates these ideas about hard-wired differences between male and female brains are wrong," she told the Observer.

"Yes, there are basic behavioural differences between the sexes, but we should note that these differences increase with age because our children's intellectual biases are being exaggerated and intensified by our gendered culture. Children don't inherit intellectual differences. They learn them. They are a result of what we expect a boy or a girl to be."

Thus boys develop improved spatial skills not because of an innate superiority but because they are expected and are encouraged to be strong at sport, which requires expertise at catching and throwing. Similarly, it is anticipated that girls will be more emotional and talkative, and so their verbal skills are emphasised by teachers and parents.

The latter example, on the issue of verbal skills, is particularly revealing, neuroscientists argue. Girls do begin to speak earlier than boys, by about a month on average, a fact that is seized upon by supporters of the Men Are from Mars, Women Are from Venus school of intellectual differences.

However, this gap is really a tiny difference compared to the vast range of linguistic abilities that differentiate people, Robert Plomin, a professor at the Institute of Psychiatry in London, pointed out. His studies have found that a mere 3% of the variation in young children's verbal development is due to their gender.

"If you map the distribution of scores for verbal skills of boys and of girls you get two graphs that overlap so much you would need a very fine pencil indeed to show the difference between them. Yet people ignore this huge similarity between boys and girls and instead exaggerate wildly the tiny difference between them. It drives me wild," Plomin told the Observer.

This point is backed by Eliot. "Yes, boys and girls, men and women, are different," she states in a recent paper in New Scientist. "But most of those differences are far smaller than the Men Are from Mars, Women Are from Venus stereotypes suggest.

"Nor are the reasoning, speaking, computing, emphasising, navigating and other cognitive differences fixed in the genetic architecture of our brains.

"All such skills are learned and neuro-plasticity – the modifications of neurons and their connections in response experience – trumps hard-wiring every time."

The current popular stress on innate intellectual differences between the sexes is, in part, a response to psychologists' emphasis of the environment's importance in the development of skills and personality in the 1970s and early 1980s, said Eliot. This led to a reaction against nurture as the principal factor in the development of human characteristics and to an exaggeration of the influence of genes and inherited abilities. This view is also popular because it propagates the status quo, she added. "We are being told there is nothing we can do to improve our potential because it is innate. That is wrong. Boys can develop powerful linguistic skills and girls can acquire deep spatial skills."

In short, women can read maps despite claims that they lack the spatial skills for such efforts, while men can learn to empathise and need not be isolated like Mel Gibson's Nick Marshall, the emotionally retarded male lead of the film What Women Want and a classic stereotype of the unfeeling male that is perpetuated by the supporters of the hard-wired school of intellectual differences.

This point was also stressed by Fine. "Many of the studies that claim to highlight differences between the brains of males and females are spurious. They are based on tests carried out on only a small number of individuals and their results are often not repeated by other scientists. However, their results are published and are accepted by teachers and others as proof of basic differences between boys and girls.

"All sorts of ridiculous conclusions about very important issues are then made. Already sexism disguised in neuroscientific finery is changing the way children are taught."

So should we abandon our search for the "real" differences between the sexes and give up this "pernicious pinkification of little girls", as one scientist has put it?

Yes, we should, Eliot insisted. "There is almost nothing we do with our brains that is hard-wired. Every skill, attribute and personality trait is moulded by experience."

What they say

Cambridge University psychologist and autism expert Simon Baron-Cohen:

"The female brain is predominantly hard-wired for empathy. The male brain is predominantly hard-wired for understanding and building systems"

Writer and feminist Joan Smith:

"Very few women growing up in England in the late 18th century would have understood the principles of jurisprudence or navigation because they were denied access to them"

John Gray, author of Men are from Mars, Women are from Venus:

"A man's sense of self is defined through his ability to achieve results. A woman's sense of self is defined through her feelings and the quality of her relationships"

Sociologist Beth Hess:

"For two millennia, 'impartial experts' have given us such trenchant insights as the fact that women lack sufficient heat to boil the blood and purify the soul, that their heads are too small, their wombs too big, their hormones too debilitating, that they think with their hearts or the wrong side of the brain. The list is never-ending"