Je t'aime, ti amo, te quiero mucho! Sounds nice doesn't it? If you swoon over sweet nothings whispered in French, Italian or Spanish, you're not alone. But while learning to speak a language famed for its romance may increase your sex appeal, the reason for your preference of one vernacular over another may have little to do with how the sounds roll off the tip of your tongue.

Polyglot Roman emperor Charles V declared: "I speak Spanish to God, Italian to women, French to men, and German to my horse." While the 16th century ruler's views may still hold true to some today, his unflattering opinion of the latter language is more likely to be influenced not by the power and status of the country at the time than the tone of its speakers.

Sociolinguists believe the attractiveness of a language is determined by how positively we view a particular group of people who share a cultural outlook. According to Dr Vineeta Chand of the University of Essex, if we have a positive perception of a particular community then we tend to have equally positive views of the language they speak.

Language value and attractiveness is, she explains, linked to the prestige of the speaker. In other words, the socioeconomic and mobility advantages the language affords. Chinese, for example, is gaining in popularity because it is seen as an area of economic growth and speaking that particular tonal tongue means better job prospects. Languages spoken by a community that are less economically powerful may not be seen in the same positive light.

Similarly, we value languages that allow us to speak to a wider audience. English, therefore, is seen as more valuable because it gives us the ability to communicate outside of a small regionally defined context, whereas a language that is spoken by a much smaller community, such as Hawaiian, is not seen as important or appealing.

"There is nothing in the sound of the language that makes it less or more attractive," insists Chand. "Some sounds are more common across the world but that doesn't link to the specific perceptions we have about French and Italian. The idea that a language is more melodic, romantic, poetic and musical are derived from those communities and regions."

There is, the linguist explains, a very tight coupling between how we see a community and region, and how we perceive their language. So no matter how hard a speaker tries to woo a listener with delicate prose, if they don't have the social kudos to back it up, the response is unlikely to be favourable.

There are, however, specific sounds in many foreign languages that a native English speaker may find alien and therefore harder on the ear. Languages that have different linguistic structures, such as using tones or sounds that are not found in a listener's native tongue, are probably going to sound less enticing.

"English speakers are drawn to the melody of a language such as French or Italian," explains Dr Patti Adank, a lecturer on speech, hearing and phonetic sciences at University College London (UCL). "In comparison, languages such as Thai or Mandarin can sound harsh because they are using tonal distinctions. It sounds very unnatural and unexpected."

In his book Through the Language Glass: Why the World Looks Different in Other Languages Israeli linguist Guy Deutscher agrees that if a language includes rarer sounds, it is more likely to be perceived as less alluring to those unfamiliar with it.

The same, he writes, applies to unusual sound combinations such as consonant clusters. He cites the combination "lbstv" in "selbstverständlich" – the German word for "obvious" – as an example of how strange phonetics can grate on a foreign ear. Deutscher explains: "Italian, for example, has very few, if any, sounds that are not shared by other European languages, and few consonant clusters, and it is widely considered a beautiful language. This may not be a coincidence."

Attempts to understand this phenomenon have spawned various YouTube videos of amateur linguists speaking gibberish with perfect accents to give us a clue how one might sound to someone who doesn't speak the same language. Comedian Sid Cesar was well known for his nonsensical "double talk" routines, which had a similar aim.

But despite many people's fascination with the subject, there has been surprisingly little research conducted to explore it further. Chand says the biggest hurdle to understanding why some languages sound more inviting than others is separating subjective opinion from scientific fact. Labelling certain languages as ugly or beautiful is also a dangerous game many linguists are keen not to play.

She says: "We spend a lot of time in linguistics dispelling myths and the notion of hierarchical languages in terms of attractiveness, grammar and rules. There is less research on this because it is opening a can of worms you don't really want to encourage.

"There hasn't been any research that I know of that has directly exploited the attractiveness of a language and didn't eventually tie it back to the social evaluation of the speaking community."