Rap is often blamed for glamorising gang culture but a study analysing lyrics from America’s Billboard 100 chart shows that it is actually providing a vivid commentary on social angst, malaise and mental health.

The data – amassed from lyrics in songs featured in the end-of-year charts from 1958 to 2017, using a computer program called TextBlob – reveals that the most popular music genre in the US may also be its most depressed. A rise in rappers discussing mental health has led to a significant spike in the number of tracks mentioning suicide, depression, anxiety and prescription drugs .

The study, run by marketing agency Take 5, found that 24 of the 100 singles overall across rock, pop and hip hop in 1958 mentioned mental health, compared with 71 in 2017. The data also backs the huge popularity of what has been dubbed SoundCloud rap, an offshoot described by the New York Times in 2017 as “the most vital and disruptive new movement in hip hop”.

Dr AD Carson, professor of hip-hop and the global south at Virginia University, believes hip-hop has always articulated pain and anguish, but that as the cultural conversation around mental health has shifted and the language has changed, so has rap.

Rapper Lil Peep, who described his battles with depression and drugs in his lyrics, died of an overdose when he was 21, in 2017. Photograph: Scott Dudelson/Getty Images

Speaking to the Observer, he said: “If you look at the ways the conversation around depression has changed in the larger cultural context, take the accessibility to remedies which are more available and mainstream, then of course you will see it reflected in the music of people making it. That’s to say, this is not new in rap – what is new is how we culturally deal with mental health. The terminology is there now.”

SoundCloud rap, characterised by sleepy, lo-fi production and named for the internet music-sharing platform through which stars came to prominence, has been defined by lyrics that speak openly about struggles with depression and the need for self-care.

References to prescription drugs such as Xanax, Percocet, Oxycontin (and, as per the drug slang, “lean”, “syrup” and “purp”) in rap tracks have been soaring since 2011. Lil Peep, 21, and Mac Miller, 26, two of the scene’s biggest stars, both died of accidental overdoses in 2017 and 2018.

From the very beginnings of hip-hop, rappers have reflected on difficult lives; in 1982 Grandmaster Flash delivered one of the most seminal verses in pop history with The Message. The lyrics – “Don’t push me ‘cause I’m close to the edge, I’m trying not to lose my head/ It’s like a jungle sometimes, it makes me wonder how I keep from goin’ under” – are just one example, says Dr Carson, that show hip-hop has always been about more than the cliches around gangsta rap.

“Most rap is characterised in this one iteration and presumed to be fascinated with money, drugs, violence and misogyny,” he said. “But all of those things are facets of popular American culture – the way it came across in rap music made it seem it was unique to that form, and so Snoop Dogg was never thought of in the same way as Arnold Schwarzenegger, when the truth is that both at one time projected characters iterating the similar ideas glorifying violence and misogyny and alcohol and the rest.”

Hip-hop may be having a sad moment, but when mainstream global megastars are talking openly about being bipolar (Kanye West) or needing therapy (Jay-Z), it only makes sense that the art they produce should be an imitation of life.