When I meet A$AP Rocky at a restaurant in midtown Manhattan– the kind where veal tomahawk chops go for $50 a pop – he’s trying to figure out the day’s itinerary. How to, in the next six hours, cram in a performance on Jimmy Fallon’s Tonight Show, a shopping spree at Dior Homme and the New York premiere for the film he’s making his acting debut in – Rick Famuyiwa’s coming-of-age comedy, Dope. “I’ve been living like a black rock star,” he says, “I slept for an hour today – an hour and a half? – but I need more.”

That schedule also goes some way to understanding how the rapper – real name Rakim Mayers – has morphed, in fewer than five years, from the self-proclaimed “pretty motherfucker” from across 110th Street to the cocky figurehead of the A$AP Mob – the Harlem crew that revived New York hip-hop. But there are multiple sides to Rocky. Is he, as he raps, the Fashion Killa more interested in hanging out with designers Raf Simons and Rick Owens than making music? Or the lothario who, as his lyrics put it, has had too many Fuckin’ Problems? On one level, he might be considered the typically brash young rapper who sports pavé diamonds in his teeth, has a downtown penthouse in New York and a Hollywood pad, and chases Roger Sterling-esque mind expansion via psychedelic drugs. On the other, he says, he’s the humble, studious creative, would-be director and movie star. Rocky knows that some of the preconceptions about him are down to his entrance into the world of rap in 2011, which wasn’t exactly subtle. “Coming in I was so braggodocious: gold, bitches, all that other shit. Not a lot has changed,” he says. “I’m just a little more humble. I’m here to stand out, I’m not here to shit on people no more.”

Friend and collaborator Yasiin Bey (Mos Def) described the jump between debut album, Long. Live.A$AP, and second album, At.Long.Last.A$AP, as equivalent to the one made by the Beastie Boys between License to Ill and Paul’s Boutique. On its release last month the album went straight to No 1 in the US charts, just like Rocky’s debut. Rocky pretends not to have heard Bey’s comments, because he’d get “too hype, too happy, too gassed”, but Bey is right – the album feels as if it marks something similar to the Beastie’s shift from hip-hop’s tabloid-friendly enfants terribles to musically mature bellwethers.

But where the Beastie Boys had to adjust to life after the death of Adam “MCA” Yauch in the band’s middle age, Rocky and rest of A$AP Mob have had to come to terms with the death of his creative partner and best friend A$AP Yams (Steven Rodriguez) at the height of their careers. Before his accidental drug overdose in January, Rodriguez described Mayers as Luke Skywalker to his Yoda, while the New York Times dubbed Yams a “hip-hop spirit guide” for the whole of the A$AP Mob. His death came out of the blue and put an immediate stop to the planned Mob album. It also made Rocky take stock of his life and career. “I’ve been having to do that [re-evaluate] since he died, man,” he says. “It’s going to have to be that way. It was always good to have that second opinion because I always knew what I wanted to do. To have my best friend there agreeing with me, let me knew I wasn’t crazy. Because my main concern was like: ‘Did I lose it, am I crazy, man?’ He kept me sane because he used to humble me even at the height of my career when people would try to reach out and try to do songs or talk shit. He’d be like: ‘Don’t shit on them because it’s a waste of energy.’”

That impulsive, devil-may-care approach governs his life, not just his music and his interviews. When he was recording in London, a young homeless man approached him and tried to sell him a CD. Instead, Rocky asked him to sing a song and, after hearing it he asked the man – Joe Fox – to come and record with him in his studio. The pair now live together in Manhattan and Fox features on five of the new album’s tracks. That honesty also leads to funny moments. When I ask him what it’s like hanging out with Rod Stewart, who provided vocals on the track Everyday, he says: “It’s like seeing yourself 30 or 40 years older and white.” And when I ask him what he’d talk to David Bowie about if he ever bumped into him (they live a few blocks away from each other), he says: “I just want to know what it’s like to paint his face back in the 70s and shit. I just want to know these things.”

A$AP Rocky … ‘Why would I run into a bagel shop after a dude threw a pastry at my Uber?’ Photograph: Chris Buck/The Guardian

Taking control of his image and how he and the rest of the group are perceived is clearly still part of the masterplan; Rocky says Yams left a blueprint, “he left us notes on how he wanted it to be”, for the future of the Mob. His personal style has evolved from Bathing Ape rain jackets and logo-emblazoned caps to more refined wares from high-end designers such as Dries van Noten. And now he has installed an in-house creative team, called AWGE, to look after everything from his videos and production to his Instagram account. That latter move lost Rocky 100,000 followers in 10 hours in May when he and AWGE posted a succession of 150 images, the majority of them plain grey, white or black squares, culminating in a picture display that Rocky called simply “Art” on his Twitter feed. “I got them right back,” he says when I ask him about it. “Why don’t they tell that story? Why don’t they tell how I got more [followers] than I lost? Why don’t they tell how much I inspire people? [pause] Tell them people suck my dick. Please be sure to put that in there. Quote me.”

AWGE’s influence can be seen in his video for L$D, which riffs on Gaspar Noe’s DMT-tinged afterlife trip Enter the Void; more intriguingly, Rocky says he has shot his own short film starring British grime MCs Skepta and Novelist, London’s rap collective Piff Gang, Dangermouse (who helped produce At.Long.Last.A$AP) and Rick Owen’s wife, Michèle Lamy. AWGE is there in his album, too, which is far more expansive and experimental than his 2013 debut Long.Live.A$AP., with no discernible “radio tracks” and a tripped-out, psychedelic quality throughout. He says he wants to move away from tracks like Fuckin’ Problems – a huge radio hit from his first album with a supporting cast made up of 2 Chainz, Drake and Kendrick Lamar – which had a profanity count that would make Martin Scorsese blush.

“It’s a fun song, [but] I didn’t want to be remembered as just that. That was my biggest song. Like man, I’ve got to wipe that off my record it’s like I’m not that guy any more. I was 22, 23 at the time – I was just having fun.” He does have fun, a lot of it, and that’s also part of his public problem. Whether it’s going to Paris fashion week, or promoting Dope in Cannes, the tabloid press want a piece of him, propelled by the fact that he’s photogenic, has dated model Chanel Iman and isn’t afraid to talk about how much wild sex he has.

So amid New York Post headlines such as “Shania Twain opens up about losing her husband to her best friend” and “Selena Gomez keeps forgetting her bra”, you’ll also find “A$AP Rocky had three acid-fueled orgies at SXSW”. Even the hip-hop press focuses more on what he does outside of the studio than in it. When he appeared on the two radio shows that are essential for any rapper with an album to promote – Hot 97’s Ebro In the Morning and Power 107’s rival The Breakfast Club – his interviews were billed as a discussion on orgies, his beef with 50 Cent and dissing Rita Ora. Better Things, from the new album, features the lines “I swear that bitch Rita Ora got a big mouth / Next time I see her might curse the bitch out,” referring to Ora allegedly going public about a fling they once had. “The Rita thing was tasteless of me,” he admits today. “[But] I’ve got to stand by it, because I did it. I said it.” Still, it remains beyond him why people care about it: “Who gives a fuck about who I’m fucking? Nobody really cares. It’s fun to hear, but who really gives a fuck about who Rita Ora’s fucking?”

Even in London, a city he’s made a second home, he’s struggled to avoid tabloid headlines. While in a bagel shop on Brick Lane in east London he got into an altercation over a pastry and an Uber taxi. “I was being an asshole. If I’d have hit that guy I wouldn’t have been able to go back to the UK,” he says, reflecting: “I’ve got so many good things going on in my life, why would I run back into a bagel shop after a dude threw a pastry at my Uber?”

When I suggest that kind of thing might happen because he’s a celebrity, he instantly shoots back that he’s not a celebrity: he’s famous, and there’s a difference. That very well might be, but semantics doesn’t stop paparazzi hanging around outside the Guardian offices while he attends a photoshoot – or the way, even though he’s released one of the best hip-hop albums of the year, the media focuses on his sex life over his music. The problem is, he says, is that he’s simply too honest to not talk about the rest of his life. “I just hope that people realise how much of an artist I am, and they realise there is no barrier between me and them. I am them,” he says. “So for me I’m going to say the same shit [on my album] that I would say to them on a one-on-one conversation in confidence because I’m a man. Whatever I say in privacy, why can’t I say that elsewhere?”

When the Guardian spoke to him in 2012, Rocky said he wanted “to stay productive, create a legacy and have longevity”. So what’s next for him now? “I’ve done fashion, I’ve done movies, videos and music,” he pauses. “Maybe Broadway; get my theatrical experience in. Or I want to develop my own show, on the internet and TV.” He pauses again before deciding he’s still not done, settling for another modest goal. “I just want to bring back actual culture to hip-hop.”

Who’s who and what’s what in A$AP Mob

A$AP Rocky might be the most recognisable (and bankable) member of the A$AP Mob, but he wasn’t even in the original gang. That was made up of founding A$APs Yams (described as part “Malcolm McLaren, part Irv Gotti, part Puerto Rican R Kelly”, by Jeff Weiss), Bari and Illz who. formed the collective in Harlem and have seen it grow into one of contemporary hip-hop’s most influential groups. The Mob, a sprawling, itinerant group, are most closely identified as a New York collective, but have tentacles that reach all over the US. They recall elements of both Wu-Tang Clan and Dipset (with whom Yams interned as a 10th grader): flashy, bolshy bravado (Dipset), as well as a clearly established hierarchy and a strong group ethos (Wu Tang Clan), but differ from both in that they were born in the web era and actively look outward from New York rather than inward.

In an interview with NPR, A$AP Yams said he thought 90s nostalgia for New York hip-hop was anathema, while genre boundaries didn’t exist for him: “It’s either fucking dope music or wack music, in my opinion. There are no fucking genres. There are no fucking subgenres.” Yams set out how the group works in an interview with the New York Times in 2013: “Realistically, I’m the boss,” he said. “I’m the boss of the whole operation. Nothing goes through without me and Rocky approving it first and whatever Rocky does I make sure I’m hands on with all of that stuff.”

Unsurprisingly, his death in January this year put a halt to the group’s debut album (L.O.R.D), which had first been scheduled for release in 2013 before being pushed further and further back before Yams declared it a dead rubber in 2014. They have released singles (Trillmatic featuring Method Man, and Xscape, where A$AP Twelvyy was upfront) and a mixtape, but unlike Wu-Tang – who released a group album before setting out to dominate hip-hop individually – it’s been solo albums that’ve been showcased. A$AP Rocky’s two albums have been the most successful, but A$AP Ferg’s Traplord put him front and centre for a while in 2013 with lead single Shabba (which referenced ragga provocateur Shabba Ranks) crossing over.

He, too, has flirted with the fashion world, creating music for high-end sportwear designer Astrid Andersen. Other members include A$AP Ant, A$AP Nast, A$AP Lotto and A$AP Ty Beats. Joe Fox like others, including the Flatbush Zombies, is closely affiliated with the group without actually being a member.

At.Long.Last.A$AP is out now on Columbia Records; he plays New Look Wireless in London on Friday on 3 July.