The American flag is flying proudly at 1411 Broadway. At the front, the address is printed big and bold and very gold. An elevator races to the 39th floor – the offices of Roc Nation and as high as you can go. Sit on the window ledge and you can virtually tweak the top of the Empire State Building. Behind a desk, Jay-Z twirls in a leather chair, stands up and greets me.

Jay-Z, born Shawn Corey Carter, is one of the world's most successful rappers. He has sold more than 50m albums, won 10 Grammys and had more number one albums than any solo artist on America's Billboard chart. The 40-year-old is also an extraordinarily successful businessman – former CEO of Def Jam Recordings, co-founder of Roc-A-Fella Records, CEO of management, music publishing and entertainment company Roc Nation. And there's the chain of all-American sports bars, the clothing line, Rocawear, the beauty products, the basketball team he part-owns, the beer giant he has a stake in. Oh yes, and he's married to Beyoncé Knowles, the most successful female artist of the 2000s. Forbes magazine recently featured him on its cover alongside Warren Buffett, the third wealthiest person in the world. Forbes estimated Jay-Z's current wealth at £450m. Not bad for a lad who started his working life at 12 selling crack.

His office is presidential and full of photographs – of heroes (Ray Charles, Jimi Hendrix, Muhammad Ali), supermodels shot by Herb Ritts (Naomi Campbell, Christy Turlington, Cindy Crawford) and family (his mother, older sister, Beyoncé off duty). He arrived here at Roc Nation in 2004. A year later, he bought out his business partner Damon Dash's share in clothes company Rocawear. It was a typical Jay-Z move: he paid Dash $22m and, two years later, sold Rocawear for $204m in a deal that allowed him to remain CEO.

Huge corporate buildings don't tend to bring out the sentimental in me, but it's quite an experience, eyeballing the New York skyline with this self-made master of the universe. Does his mum ever come up to the 39th floor and swivel in his chair? He giggles. His laugh, like his voice, is surprisingly high-pitched – relaxed, confident, likable. "Yeah. She always tells me, 'You just make me prouder and prouder every day.' Ahaha! She tells me that quite often."

Does she still call him Shawn? "No, she calls me Jay. Nobody calls me Shawn except my grandmother." He was nicknamed Jay-Z, a riff on jazzy and Brooklyn's J/Z subway, when he was a kid, tapping out raps on his mother's kitchen table. He was a smart boy who lost his way at 11 when his father walked out on the family. This was the 1980s, Brooklyn was infested with crack cocaine, and a generation of young black children were left fatherless. His older brother Eric became addicted to crack. Young Shawn responded by hooking up with a gang and selling the same drug.

Was he aware that he was surviving by destroying people like his brother and father? "No! No way. You just think people are buying your service, and it's so normal, you just think you're coming of age. It's everywhere. The smell, the stench is in the hallway – that's one of the things about the crack epidemic, people had lost their sense of pride... The desperation." He still seems astonished that his people could have sunk so low. In a way, he says, children became parents and parents became children. He found himself supporting his family – even if it was by hustling on the streets. "Before, when our elders told us something, you had to listen. But now we were in power because the people who were supposed to be our support system were on crack, and they was telling us, 'I'll do anything to get it.' So we were like elders in the village, with a whole community on drugs. There was no one to police us. And we were out of control." He pauses. "Even when I was making terrible decisions, I was making them out of desperation rather than ignorance. I felt I was in a survival of the fittest situation."

It was only gradually that he became aware of the consequences of selling drugs. "You're like, what happened to Debbie, the fine girl in the projects who gets on crack, then next year she loses her shape and beauty. First it was a joke, then as you get older you start to think, man, that really is decay in her body. There are many Debbies. And you still justify it, because that's how you make your money and you don't want to stop. You say, man, she making her own choice."

Through his 14 years on the streets, he never touched crack. He says there was a strict code among successful hustlers, and quotes me a line from the film Scarface: don't get high on your own supply. "We had these guidelines to help us out," he says. "Some of us died, some of us didn't. I never wanted that. I saw what it was doing to the community."

That's not to say he didn't get into trouble. At 12, he shot his older brother Eric in the shoulder for stealing his ring. Jay-Z has written about the incident in the song You Must Love Me ("Saw the devil in your eyes, high off more than weed, confused, I just closed my young eyes and squeezed, what a sound, opened my eyes just in time to see ya stumbling to the ground") but he has never publicly talked about it before.

How did he get the gun? "I went to someone's crib, someone's house, and got it. Guns were everywhere. You didn't have to go far to get one. Just everywhere."

After the shooting, what did he think would happen to him? "I thought my life was over. I thought I'd go to jail for ever."

And what did happen? He has become monosyllabic. His discomfort is obvious. Did Eric simply not press charges? "Yes, that's right – he's my big brother." In fact when he visited Eric in hospital, his brother apologised for what crack had turned him into.

It's an astonishing story, I say. "Yes." Just talking about it gets to you? "Yes. It was terrible. I was a boy, a child. I was terrified."

Eric is clean these days, and the brothers get on. Was that the only time he shot someone? "Yeah. But guns were around every day. There were shoot-outs, but I never shot anyone else. Most people in shoot-outs don't get shot."

He knows just how lucky he was when he was hustling. He was shot at three times and each time the bullet missed. "It's like there was some rogue angel watching over us."

Have things changed in Brooklyn since then? "It's changed some, but not in certain areas. I was in Miami the other day and there was this sweep of 14-year-olds. It showed the weaponry they had – AK-47s, military weapons in the hands of these kids." Does he think if guns were less readily available in the US there would be fewer violent deaths? "It's difficult for me because I'm of the mind that you correct people, you don't correct things." One of his assistants walks in and asks if he'd like anything to eat. He says he'll have a salad, and looks at me. "Will you have one, too?... So if you take away guns, then what about knives or rocks? You gotta correct the problem, not the tool."

Does he carry a gun now? "No." Never? "No."

Our salads arrive – olives, chickpeas, crispy greens. As with all things Jay-Z, it's top quality – the crispiest lettuce money can buy. The chair I'm sitting on is the comfiest (but not flashest) I've ever sat on. He tells me more than once how important quality is to him. On the table in front of us are a number of elite magazines, all featuring Jay-Z on the front. He's got an unusual face for an idol: big puffy cheeks, pixie ears, fatherly and kind. There are a few stray hairs above his lip. Somehow, you'd expect him to be a more meticulous shaver. Today, he's dressed in casuals. On the cover of Forbes he is in a sober blue suit, every inch the businessman. He points me to the feature on him and Buffett. It's obviously tickled him that comparisons are being made. Who's wealthier? "Are you joking? He's a multi-billionaire." And him? "They expect me to have my first billion by 2015."

In his memoir, he paints a vivid picture of life as a hustler. He makes it sound surprisingly appealing, I say. "Yeah, the excitement and the danger, for one. Then you have the camaraderie and the love. Those guys are with you. You're all in there together, fighting for the same thing. Survival."

All the time he was dealing, he dreamed of a career as a rapper. But it was never going to be easy to make a clean break. The problem was, he could never hope to make as much money from rapping. "We'd go to events where rappers were performing and we'd all pull up in these beautiful BMWs and the rappers would have Turtle Tops. We'd be like, aren't you supposed to be rich, you're a famous performer, why you in a tight white van with 17 people?"

In 1992, he released his first single, but he was still selling drugs. Sometimes he'd tell his mother he'd made the money from music, but she knew better. Today, she runs the Jay-Z scholarship for underprivileged students. Wasn't she devastated that he spent all his time dealing drugs? "What I love about her is she gave me a long leash to learn for myself. There's only so much you can tell a son at that time. You can tell him, don't touch that, it's hot, but at some point you gotta let him see for himself. And she really gave me lessons, but she also let me go. And I would have done what I was going to do anyway. She was a little more strict with my two sisters."

For four years after the first single, he held down the two jobs. It was only when he made his first album in 1996 that he became a full-time musician. "I officially quit during the recording process of Reasonable Doubt. I had been trying to hold on to two branches and I said, I'm going to put my all into the music, to make a legitimate life for myself. I never turned back."

Much of Jay-Z's subsequent music has been about his early experiences – the deals, the danger, the shattered lives. Many of his songs are cinematic – character-led with detailed and sometimes complex narratives. They are boastful (as is the convention of rap), with references to niggas, hos, bitches and upmarket motors, but the lyrics also often allude to great black American radicals, movies and rappers.

In the past, he has said Jay-Z is a character, but today he says he sees little difference between Jay-Z and Shawn Carter. He certainly couldn't have had the career he's enjoyed without the life that fuelled it beforehand. In many of his songs, he seems to be searching for understanding of the past, penance even. "Right!" he says enthusiastically. "Of course, there's admittance, there's culpability, there are so many different emotions, you feel like a therapist."

The most profitable form of therapy he could ever get into? "Yeaahhhhh! Ahahaha."

In his collaboration with the rapper Kanye West on the song Diamonds From Sierra Leone, he sings, "I'm not a businessman, I'm a business, man." Money has always been important to him – a measure of his dreams as a kid, a measure of his success as an adult. Like many successful black Americans (Spike Lee and Oprah to name but two), he is a fascinating mix of the politically liberal and the economically conservative. Of course, he was always going to embrace capitalism, how else could he effect change? "In order to get your voice heard, you have to have some semblance of power, even if it's only in entertainment." And that is still the trouble, he says: black Americans are allowed to succeed in the business of entertainment, but not in the business of business. "Will Smith, what does he do? He entertains. Spike Lee, he entertains. Oprah Winfrey, she entertains." But doesn't everybody entertain in America? "Warren Buffett doesn't entertain, and he's the second richest guy in America."

From the position he now occupies, doesn't it become harder to rap about the bad old days? Shouldn't he be writing songs about the trials and tribulations of a corporate whiz-kid? He smiles. "Yeah, that creeps into the music. Songs like Corporate Takeover. As I get further removed, I draw off different inspirations." He talks about the song Most Kingz, which is inspired by a painting he owns. "It was on the bottom of a Basquiat painting – most kings get their head cut off. It was about the paranoia of being successful."

Does that worry him? "It's just true, it's true in all of history, whether figuratively or literally."

Perhaps his best-known song now is Empire State Of Mind, a collaboration with Alicia Keys that charts his past and present. While it has become an anthem of the American dream ("New York, concrete jungles where dreams are made, oh, There's nothing you can't do, now you're in New York"), it also charts the nightmare that the city is for many ("Welcome to the melting pot, corners where we selling rock").

Today, he says, he's not as likely to write about "hos" or "bitches". "My language has changed because my dealings with women have changed." The problem was, he says, as kids they were all having one-night stands with people who didn't mean anything to them, and boys didn't respect girls who didn't respect themselves.

Has Beyoncé done a lot to educate him about women? If he called her a bitch or ho, would she give him a good slapping? "Well, of course." Could he do it as a joke? He gives me a no-nonsense look. "I don't want to even talk about that subject... Of course not. What I'm saying is that these relationships are taking place at a young age, so there is no real relationship there. Guys need to take responsibility, but they aren't mature enough at 16 or 18. They are closed, shut off, emotionally crippled. They don't know how to express themselves."

One of the most enjoyable elements of his memoir is the deconstruction of his own lyrics. He explains the scan and rhyming structure, and uses footnotes to clarify obscure street slang. Again, it's classic Jay-Z – the more accessible he makes himself to people, the bigger the potential audience. Early on, he used to rap much faster. Why did he slow down? "In the beginning it was all technique. I was like a trickster, dribbling behind my back, just really trying to impress people. But as I started to get life experiences, I had to tell a story, so the technique had to slow down a bit. It had to make way for the story and the emotion."

Wasn't it also partly because by slowing down he broadened his crossover appeal? "Yes, and I was enunciating and making clear points. I think people connected to a real truth. You might not have lived that experience, but you've lived part of it or you connect to the ambition of it or to the resilience of that person."

I can see him as Professor Z of the University of Rap in years to come, I say. He laughs loud and long, and says no chance, but he likes the idea. "It was important to explain rap is poetry, not just some frivolous music these guys are doing." He tells me there's a seven-week Jay-Z course at New York University at the moment. You should creep in and sit at the back, I say. "I did that one time at Princeton – they had this really cool course and they were discussing Horace and Biggie Smalls." Smalls, his best friend in rap, was shot dead in 1997. Such a senseless waste of life, he says. "It's unbelievable how charismatic and charming and cool this guy was, and he's no longer here, for no reason."

In 1999, Jay-Z was reminded how easy it would be to lose everything. He got involved in a fight at a nightclub. Nobody was seriously hurt, but two years later he was sentenced to three years' probation for stabbing record producer Lance Rivera.

What actually happened? "It was just a bar fight. I'm not going to blame it only on success, but there's a bar fight every night and it's not on the front page of the paper and nobody's being offered seven years. The guy wasn't hurt – he took an aspirin and went home. Forty people fighting... It was just like the things I used to do every Friday night are now on the front page of the New York papers, I'm a hunted man. But it was a wake-up call for me. I've got to move more careful, I've got to watch my temper, everything has to change." And, he says, it did.

After collaborating with Beyoncé in 2002, on the song '03 Bonnie And Clyde, they started a relationship, and in 2008 they married. The week we meet, newspapers report that Beyoncé is pregnant. I offer my congratulations, and he says the whole thing is nonsense. "They do that every slow news cycle. It's crazy, they've done it five times already."

As two of the world's biggest recording artists, are they competitive with each other? Do they sit at home comparing record sales? "Nah. When you're working, you want to be the best or the best you can be, but as far as competing with each other, no, not at all."

I ask about his heroes and he takes me on a guided tour of the photos in his office. We stop at a magnificent larger-than-life picture of Ali's arm and fist wrapped for work. "I'd say Ali is first; the blend between sport and entertainment, and how he stepped outside that arena, and spoke to a group of people, and gave them hope that they could be somebody. Like he was saying, 'I'm pretty' at a time when black people were told you're ugly and monkeys – he was like, 'I'm beautiful and gorgeous.' He gave us confidence to feel we're gorgeous and beautiful as well."

There are honourable mentions for Malcolm X, Martin Luther King and Mahatma Gandhi. "These are people you can never be. They have a certain glow about them. That for me is what your heroes are – unattainable." He knows that his own achievements seemed every bit as impossible as those of his heroes when he was a boy.

Jay-Z recently endorsed another man who attained the unattainable – Barack Obama. For black America, he says, Obama represented hope rather than politics. Pre-Obama, all the stuff about the land of the free, the place of opportunity, was just verbiage, he says. Now he's disappointed that America seems determined to thwart Obama. "Like, who can stand in an assembly hall and vote against healthcare? Who could do that?" He sounds genuinely baffled.

Would he ever go into politics? He laughs. "No, I'm not interested in politics. I have zero interest. I have interest in hope and people. Politics – I still think it's a bunch of liars and a bunch of self-interest. It's not about the people, it's about themselves and their rise to power. They are voting on things based on whether they will have the support of the people when they vote next time. They don't have the balls to say, 'I believe in this, I don't care what happens.' And until that happens, nothing is going to get fixed; you're still going to have 14-year-olds in cities with AK-47s."

If America became a truly egalitarian society, would rap would become less relevant – after all, it has traditionally been a music of the marginalised? He nods. "The day Obama got into office rap was less important, because Obama gave kids an alternative. But will rap ever go away? No. There will always be a need for poets."

Walking around the Roc Nation offices, you get a sense this is a happy (and racially diverse) organisation. Has he met any CEOs who have come from a similar background? "I guess the closest person you could compare to me is Joe Kennedy." He bursts out laughing. "He started as a bootlegger! Ahaha! They say behind every great fortune is a great crime, that's what they say. I believe it."

As he talks, I'm looking at him – T-shirt, jeans, trainers, gold Rolex, but not very flash, modest gold chain tucked into the T-shirt. I don't mean to be rude, I say, but you're not very bling for such a huge hip-hop star – I mean, the necklace is hardly something to write home about. Ah, he says, things aren't always as they might first appear. He brings the necklace out from under the T-shirt to reveal the most dazzling, chunkiest, blingiest gold and diamond Jesus you could ever hope to see. He can't stop grinning when he sees how shocked I am. "That's part of my personality. It's the duality between good taste and the ostentatious. There's great stuff in both worlds."

• Want more Jay-Z? Don't miss our exclusive extracts from his memoir.