Research suggests that more and more American university students think they are something special. High self-esteem is generally regarded as a good thing - but could too much of it actually make you less successful?

About nine million young people have filled out the American Freshman Survey, since it began in 1966.

It asks students to rate how they measure up to their peers in a number of basic skills areas - and over the past four decades, there has been a dramatic rise in the number of students who describe themselves as being "above average" for academic ability, drive to achieve, mathematical ability and self-confidence.

This was revealed in a new analysis of the survey data, by US psychologist Jean Twenge and colleagues.

Self-appraisals of traits that are less individualistic - such as co-operativeness, understanding others and spirituality - saw little change, or a decrease, over the same period.

Self-esteem and confidence Psychologists rarely use the word confidence. They have separate measures for: self-esteem - the value people place on themselves

narcissism - definitions vary, but essentially a negative, destructive form of high self-esteem

self-efficacy - the ability to achieve personal goals

Twenge adds that while the Freshman Survey shows that students are increasingly likely to label themselves as gifted in writing ability, objective test scores indicate that actual writing ability has gone down since the 1960s.

And while in the late 1980s, almost half of students said they studied for six or more hours a week, the figure was little over a third by 2009 - a fact that sits rather oddly, given there has been a rise in students' self-proclaimed drive to succeed during the same period.

Another study by Twenge suggested there has been a 30% tilt towards narcissistic attitudes in US students since 1979.

The Oxford English Dictionary defines narcissism as: "Excessive self-love or vanity; self-admiration, self-centredness."

"Our culture used to encourage modesty and humility and not bragging about yourself," says Twenge. "It was considered a bad thing to be seen as conceited or full of yourself."

The Freshman Survey A nationally representative sample of first-year college and university students in the US

Conducted every year since 1966

Questions on a range of topics - including values, financial situation, and expectations of college

Not everyone with high self-esteem is a narcissist. Some positive views of the self may be harmless and in fact quite justified.

But one in four recent students responded to a questionnaire, the Narcissistic Personality Inventory, in a way which leaned towards narcissistic views of the self.

Though some have argued that narcissism is an essential trait, Twenge and her colleagues see it as negative and destructive.

In The Narcissism Epidemic, co-written with Keith Campbell, Twenge blames the growth of narcissistic attitudes on a range of trends - including parenting styles, celebrity culture, social media and access to easy credit, which allows people to appear more successful than they are.

"What's really become prevalent over the last two decades is the idea that being highly self-confident - loving yourself, believing in yourself - is the key to success.

"Now the interesting thing about that belief is it's widely held, it's very deeply held, and it's also untrue."

This bewitching idea - that people's lives will improve with their self-esteem - led to what came to be known as The Self-Esteem Movement.

Legions of self-help books have propagated the idea that we each have it within us to achieve great things - we just need to be more confident.

Over 15,000 journal articles have examined the links between high self-esteem and measurable outcomes in real life, such as educational achievement, job opportunities, popularity, health, happiness and adherence to laws and social codes.

Yet there is very little evidence that raising self-esteem leads to tangible, positive outcomes.

"If there is any effect at all, it is quite small," says Roy Baumeister of Florida State University. He was the lead author of a 2003 paper that scrutinised dozens of self-esteem studies.

All about me, me, me... In a recent paper Jean Twenge examined changes in pronoun use in American books published between 1960 and 2008, using the Google Books ngram database

She found that first person plural pronouns (we, us our etc.) decreased in use by 10% while first person singular pronouns (I, me, my etc.) increased in use by 42%

He found that although high self-esteem frequently had a positive correlation with success, the direction of causation was often unclear. For example, are high marks awarded to people with high self-esteem or does getting high marks engender high self-esteem?

And a third variable can influence both self-esteem and the positive outcome.

"Coming from a good family might lead to both high self-esteem and personal success," says Baumeister.

"Self-control is much more powerful and well-supported as a cause of personal success. Despite my years invested in research on self-esteem, I reluctantly advise people to forget about it."

Am I a narcissist? The Narcissistic Personality Inventory asks 40 questions, then ranks you on a narcissism scale Take the narcissism test

This doesn't mean that under-confident people will be more successful in school, in their careers or in sport.

"You need to believe that you can go out and do something but that's not the same as thinking that you're great," says Twenge. She gives the example of a swimmer attempting to learn a turn - this person needs to believe that they can acquire that skill, but a belief that they are already a great swimmer does not help.

Forsyth and Kerr studied the effect of positive feedback on university students who had received low grades (C, D, E and F). They found that the weaker students actually performed worse if they received encouragement aimed at boosting their self-worth.

"An intervention that encourages [students] to feel good about themselves, regardless of work, may remove the reason to work hard," writes Baumeister.

So do young people think they are better than they are?

If they are, perhaps the appropriate response is not condemnation but pity.

The narcissists described by Twenge and Campbell are often outwardly charming and charismatic. They find it easy to start relationships and have more confidence socially and in job interviews. Yet their prognosis is not good.

How self-esteem become a movement The Self-Esteem Movement is said to have its roots in the civil rights movement, which promoted group solidarity - but also the rights of individuals to be who they want

A series of seminars were held in the 1960s on achieving happiness and fulfilment by tapping inner potential - it was called The Human Potential Movement

First popular book on self-esteem published in 1969 - The Psychology of Self-Esteem by psychologist Nathaniel Branden

Werner Erhard (above) held sessions aimed at boosting self-esteem in US prisons in the 1970s - there were similar programmes in the 1980s to try to reduce teen pregnancy rates and crime

Interest is still high - there were more than 40,000 articles about self-esteem in newspapers and magazines between 2002 and 2007

"In the long-term, what tends to happen is that narcissistic people mess up their relationships, at home and at work," says Twenge.

Narcissists may say all the right things but their actions eventually reveal them to be self-serving.

As for the narcissists themselves, it often not until middle age that they notice their life has been marked by an unusual number of failed relationships.

But it's not something that is easy to fix - narcissists are notorious for dropping out of therapy.

"It's a personality trait," says Twenge. "It's by definition very difficult to change. It's rooted in genetics and early environment and culture and things that aren't all that malleable."

Things also don't look good for the many young people who - although not classed as narcissists - have a disproportionately positive self-view.

A 2006 study led by John Reynolds of Florida State University found that students are increasingly ambitious, but also increasingly unrealistic in their expectations, creating what he calls "ambition inflation".

"Since the 1960s and 1970s, when those expectations started to grow, there's been an increase in anxiety and depression," says Twenge.

"There's going to be a lot more people who don't reach their goals."

Jean Twenge spoke to Health Check on the BBC World Service. You can listen to the programme or download the Health Check podcast.

You can follow the Magazine on Twitter and on Facebook