The lobby of the Sony building in New York is 70-feet high and heavy with music business ambience - gold records, photographs and the 'Sony Shop of New Technology'. Upstairs, the main reception is like the lounge of an exclusive club. Young people, dreaming of stardom, stand in wonder breathing in the atmosphere, looking at memorabilia - platinum CDs, photos of stars, framed press reports, Billboard charts. For an aspiring artist or manager, just to step into the building is a thrill. The impression is of a corporation dedicated to the success of its artists, almost altruistic in its understanding of their needs.

Yet it's nothing but a flytrap. Artists go there dreaming of being signed. But out of every 10 signed nine will fail. A contract with a major record company was always a 90 per cent guarantee of failure. In the boardroom the talk was never of music, only of units sold. Artists were never the product; the product was discs - 10 cents' worth of vinyl selling for $10 - 10,000 per cent profit - the highest mark-up in all of retail marketing. Artists were simply an ingredient, without even the basic rights of employees.

Imagine the outcry if people working in a factory were told that the cost of the products they were making would be deducted from their wages, which anyway would only be paid if the company managed to sell the products. Or that they would have to work for the company for a minimum of 10 years and, at the company's discretion, could be transferred to any other company at any time.

Recently, the Wall Street Journal investigated the industry and concluded that 'for all the 21st-century glitz that surrounds it, the popular music business is distinctly medieval in character: the last form of indentured servitude.'

As long as the major record companies controlled the industry, artists had to accept these conditions. But the majors' grip on things has almost gone. For years they saw it coming but did little to change things. Now each week brings them more gloom. CD sales are down on last year, which were down on the year before, and the year before that. Sony and BMG amalgamated, but brought themselves little benefit in doing so. EMI and Warners tried to go the same route, but failed. So EMI was taken over by someone with no knowledge of the record industry. Guy Hands of Terra Firma fame promised to reinvent the whole business plan; he started by parting company with Radiohead.

But outside of the industry, who cares? Pop music has never sounded better or more vibrant, never been more easily available to the listener. The only people who are suffering are the people who brought it on themselves. The major record companies.

In 1966 I came into a business that was alive with excitement and optimism. I was one of a select group - the young managers, like Brian Epstein, Andrew Loog Oldham and Kit Lambert - who had taken over the UK's new pop groups - the Beatles, the Rolling Stones, the Who, the Kinks, the Yardbirds, the Animals. We young managers were on fire. We hustled, and we were free. We weren't even friends, yet we knew each other from hanging out at the Ad Lib club or the Scotch of St James. Despite enormous differences between us, we found one thing in common. We all saw our principal job as going to war with the record company.

The first record company I ever went to was Decca in London in 1964. It was a six-storey building on the south bank of the river at Lambeth. The inside was painted in the same colour, olive green, as government buildings - like the labour exchange or the tax office. With a gruff commissioner on the door, it was pure bureaucracy, the civil service of the music industry. During the Second World War, Decca developed radar for the army. From the profits, its stuffy owner, Sir Edward Lewis, indulged his enthusiasm for recording classical music. For him, pop music was a necessary sideline, nothing to be too proud of.

At Decca they didn't like young people. I was 25, but I talked my way into seeing someone in A&R, a small mean-minded man who sat picking his nose while I played my record. It was a group I wanted to manage and I'd paid for them to make the record. The man was a pedant; a killjoy. 'It's dreadful!' he exclaimed. 'The song's not memorable and the musicians don't catch the beat.' Then, surprisingly, he agreed a deal. It was a very small one, but I was delighted - my first step into the business. But if the record was as bad as he'd said it was, why did he give me a deal? And if it wasn't that bad, why had he said it was? I left the building thinking, 'What a wanker!' and it's been difficult to think of A&R people in any other way since.

At that time Britain had four major record companies - Decca, EMI, Pye and Philips. These last two were offshoots of corporations that produced electronic hardware for home and industry. EMI, like Decca, manufactured hi-tech equipment for the government, mainly for hospitals - brain scanners and the like. None of these companies had been set up first and foremost for music; they made records for extra profit. It was a wonderful trick they'd learnt. They bought vinyl cheaply; added a label, a song and a sleeve and sold it expensively.

When I took over the management of the Yardbirds I had to deal with EMI. In 1961, it had become the biggest record company in the world, and that was before it signed the Beatles. There was an air of pomposity about the place. Artists were from the wrong class - they tended to cause problems. EMI preferred to deal with managers, especially if they were middle class and public school. The people in the business affairs department were extraordinarily pissed off when I told them I considered their contract with the Yardbirds to be invalid. They doubted I was right, but were too scared to challenge me in case they lost the group altogether, so they agreed to negotiate a new deal. In order to bypass the company's A&R department, I insisted the Yardbirds should produce their own records. I demanded the biggest advance they'd ever paid and the highest royalty - £25,000 and 12 per cent of retail - and they gave it to me. If this was my entrance exam into management, I thought I'd passed with flying colours. I soon learned I'd failed.

EMI had simply advanced the Yardbirds their own royalties and included a host of tricky accounting clauses - for instance the artist was only paid on 90 per cent of records sold, and was not paid on 'over-pressings', although these were usually sold anyway. I asked the group's lawyer why he'd let these things pass. 'If I told my clients not to sign unfair contracts they'd never get a deal.'

I had learned the first golden rule of management - record companies are not to be trusted.

Management is a wildly up and down occupation. Sometimes - if you're standing at the top of a stadium looking down on 100,000 people stomping and cheering at your artist, or popping another bundle of cheques into your bank account, or being hailed as the Svengali behind the new icon of youth culture - it feels good. Like standing at the back of the hall in Guangdong during Wham!'s trip to China with the group being cheered or encore after encore. But at other times - when your nitwit star, out of his head on drugs or drink or self-admiration, tells you to cancel the gig with a stadium full of people waiting for the first chord, or wakes you in the middle of the night with a call from Sydney to say he can't go on stage because he has no clean socks (as the lead singer of the Yardbirds once did) - it feels less so.

In the end, though, you have to see it from the artist's point of view. He's the one who will be booed off if he performs badly, or slated by the critics if he makes a bad album, or shot at by some maniac just for being famous. The artist takes all the emotional hits and needs you as his friend. Your common enemy is the record company.

Having got the contract with EMI sorted out, I visited our American record company. In 1966 the US market was dominated by CBS and RCA, both of whom had the same civil service atmosphere as EMI and Decca. Their principal business was broadcasting and they held government licenses that required them to keep high moral standards. The other majors were Capitol, which had been bought by EMI, and MCA, which had bought the American office of Decca. (Warners was still considered a minor offshoot of a movie studio.)

The Yardbirds were with CBS whose New York HQ was known as Black Rock - a gaunt, black-bricked, black-glassed skyscraper. Its lobby was as austere as a high security prison and I was accompanied to the elevator by a guard. I was meant to be seeing Len Levy, the head of the Epic label, but the company didn't want me there. They had the rights to the record, they were going to release it, they'd decided on the budget and they didn't want the manager turning up demanding things.

'The Yardbirds' manager is here.'

'Aw Jesus, is he? Well, Len's out at the moment. Ask Ernie if he'll have a talk with him. That should do the trick.'

So I saw Ernie Altschuler, one of their old-time staff producers. He knew nothing about rock'n'roll or British pop or Swinging London; he produced Tony Bennett and Ray Conniff. But he was a charmer and we became immediate friends. Ernie was 20 years older than me and wildly disillusioned with things. 'I've made CBS more hits than any other producer but I've never been paid a royalty or a bonus. They see me the same way they see the artists - just part of the process.'

I wasn't ready to believe such doleful news. I was excited, I was in the USA, I was managing a top band. America felt good. This was the real record industry - the corrupt, tough, no holds barred, American industry - not the whingeing, always-changing-their-mind industry we suffered in the UK.

Nevertheless, I was totally in their hands. Here there were 6,000 radio stations. Four thousand were said to have playlists under Mafia control. To promote my record would require cocaine and sex and suitcases full of cash. I hadn't chosen to be with this company: that had been done by EMI. In America I had just one job - to persuade CBS the Yardbirds were worth promoting. But since that was already decided there wasn't much left to do. So I went and had tea with Ahmet Ertegun.

Ever since the mid-Fifties, a lot of small record companies had been growing up. The people who owned them also ran them. They gave a more personal service than the majors, made the artist feel cared for. The royalties were no more and the profit margins no less but there was a feeling of compromise between commerce and art. Four of them dealt only with black artists - Motown, King, Chess and Atlantic.

Atlantic was owned by Ahmet and Nesuhi Ertegun - sophisticated, jazz-loving, multilingual Turks. With a view to discovering more about the explosion of music coming out of London, Ahmet invited me for afternoon tea and muffins. I'd only been there five minutes when the door opened and Joe Tex, one of the biggest black recording artists in America, stuck his head in. 'Ahmet, man, I was just wondering if you could loan me 10 bucks.'

'You want 10 bucks,' Ahmet told him. 'Go downstairs to the studio, find a backing track you like and put your voice on it.'

An hour later Joe came back. Ahmet buzzed the studio and asked the engineer if Joe had done a good vocal. Then he doled him out $10 and offered him a cup of tea. When Ahmet left the room for a minute I asked Joe how much royalties he got. He wasn't sure he got royalties at all. 'I don't know exactly how it works,' he confessed, 'but Ahmet and Nesuhi are like brothers. Whenever I'm in New York I gotta place I can hang out. And I always come away with a few bucks.'

Ahmet and Nesuhi, by the way, were making themselves very rich.

Owners of other small companies were getting rich less pleasantly. For a while I was producing records with Ray Singer and we went together to see the Roulette label, rumoured to be connected with the Mafia. People told us not to, but what the hell, we wanted all the work we could get and dealing with the Mafia sounded fun. We arrived early and were shown to a waiting room. Only when Ray wanted a pee did we notice there were no handles on the inside of the doors. He held it.

We were taken to meet the boss - Morris Levy, a Jewish record company executive with lots of Italian friends. His office was long, with his desk at one end on a dais. We arrived and Morris was standing mid-office. His hands were round the collar of a slightly built black guy, lifting him off the floor, shaking him furiously. 'You fucking black cocksucker. You promised to make me a hit record and you screwed up.'

The little black guy was shuddering from top to toe of his shaken body. Then we recognised him.

It was Mickey Stevenson, for God's sake! One of the top black producers in the world. He'd written 'Dancing in the Street' for Martha and the Vandellas and produced 'What Becomes of the Brokenhearted' for Jimmy Ruffin. Now he was being shaken to death. When Morris realised we'd come into the room he let go of Mickey, who fell to the floor like an empty sack. While Morris motioned us to chairs, Mickey crawled to the door and fled.

'So you want to make some records for me?' Morris boomed. Ray and I eyed each other awkwardly. What we'd seen seemed accurately to portray how the American music business dealt with people who failed.

Extraordinarily, Morris Levy was hugely loved in the music industry. In 1973, when he was voted Man of the Year by the United Jewish Appeal, the entire hierarchy of the music industry turned out to his celebration party. Morris loved to play the Mafia chief - he behaved the way all the other executives wished they were able to behave. Whenever artists asked Morris about royalties, he yelled: 'Royalties? Try Buckingham Palace.'

Other small companies popped up all over the place. In the UK, there was Island, owned by Chris Blackwell, a white West Indian who spoke Oxford English and Jamaican patois with equal panache. Charisma was owned by Tony Stratton-Smith, who was much loved by his artists despite a lifestyle that revolved around fine wine, racehorses and rent boys. In California, trumpet player Herb Alpert started A&M, which zoomed to success with the Carpenters and Carole King. Jac Holzman started Elektra specifically to sign non-mainstream artists such as the Doors and Judy Collins. Like all the other owners of small labels, he liked rock stars for what they were - self-obsessed and irrational. When he signed Love, he gave them a $5,000 advance ($100,000 in today's money). There were five of them, all living in a single hotel room and they needed transport to get to gigs with their equipment. They took his money and went to buy something suitable. An hour later they came back with a gull-winged Mercedes capable of taking two. Jac shrugged and shelled out for a van. At a major no one would have done that.

Whenever a rock singer experienced success, the ambition lobe in his brain seemed to develop a permanent, painful erection. Small companies understood how to deal with this, the majors hadn't a clue. Seeking to solve this problem, CBS appointed a charismatic figure to head the company, Clive Davis, a charming young lawyer. Clive camped it up, put on love beads and a hippy Nehru jacket and signed Scott McKenzie, Donovan, Laura Nyro and Blood Sweat &Tears. CBS's market share suddenly shot up.

Warners was now close on its heels. Steve Ross, who'd made money from car parking and hobnobbing with the Mob, headed an investment group which bought the company out for $50m. Free of all controls, the new company could go hell for leather for profit and forget about the niceties. To run it, Steve Ross found a guy called Mo Ostin who had a talent for picking off-the-wall artists and standing by them - the Kinks, Jimi Hendrix, Frank Zappa. Ross also took Warners on a buying spree and snapped up the best two small companies together with their owners; Elektra, with Jac Holzman, and Atlantic, with Ahmet Ertegun. Suddenly stuffy old Warners had become WEA.

At that time I was producing records for RCA. It too was trying the 'friendly president' approach but couldn't get it right. Each time I visited there was a new face at the top. Each new person signed new artists and stopped promoting the artists his predecessor had signed. Eventually RCA had more than 100 artists who were not achieving chart success so it had to hire yet another new president especially to fire them all.

By the mid-Seventies, in both the UK and the US, there were now only six majors . In the UK, there were three new small labels - Chrysalis, Zomba, and Virgin, which had signed my group Japan.

After four years Japan had finally broken in the UK so I decided to head for the States. Virgin had licensed America to CBS, which had fired Clive Davis in the wake of a payola scandal. The company was now run by two lawyers - Walter Yetnikoff and Dick Asher. I liked Walter but fell into the half of the company run by Dick - a very dull man indeed. I finally got a meeting with him but had no sooner arrived in his office than the buzzer sounded and his secretary's voice said: 'Bob Dylan on line one.'

'Can I call him back?' Dick asked.

'No. He says he wants to talk to you now.'

Dick was about to have a conversation he didn't want. Eighteen months previously there had been publicity about Jewish-born Dylan becoming a born-again Christian. He'd made a couple of albums full of evangelical zeal but they'd bombed. Now his contract had come up for renewal. Dick especially didn't want to have this conversation in front of me. He took the call anyway.

To begin with, it wasn't too interesting but then Dick yelled, 'I've told you, Bob - no fucking religion! If you can't agree to that, the deal's off ...'

Bob was arguing the point but Dick was having none of it. 'Look, I'm telling you. There'll be no fucking religion - not Christian, not Jewish, not Muslim. Nothing. For God's sake, man - you were born Jewish, which makes your religion money, doesn't it? So stick with it, for Christ's sake. I'm giving you 20 million bucks - it's like baptising you, like sending you to heaven. So what are you fucking moaning about? You want 20 million bucks from us? Well, you gotta do what we tell you. And what we're telling you is ... No Torah! No Bible! No Koran! No Jesus! No God! No Allah! No fucking religion. It's going in the contract.'

As a devout atheist, I could hardly object, though it seemed tough that a contract should include such specific restrictions. When we finally got back to the subject of my group Dick had rather lost interest. He agreed to release one album. There were three to choose from, each a cohesive musical whole, but he wanted bits from each. It was like introducing a new film director with a composite of three of his movies - the album wouldn't have a chance. And to make sure it didn't, CBS gave it no promotion. That way, Dick was able to tell me, 'You see, I was right. There's no market for a group like yours in the States.'

A year later I was back with Wham!. By now Walter Yetnikoff had taken the whole thing over for himself. He took artist friendliness to new levels. In his book Howling at the Moon, he describes his 15 years at the top of the company. He was there, he explained, for the artists. Yet from beginning to end of the book, he only talks about seven artists with whom he spent time - Paul McCartney, Michael Jackson, Mick Jagger, Barbra Streisand, Paul Simon, Bob Dylan, and Patti Labelle (and the latter only because he was screwing one of her singers).

During that 15-year period, perhaps 700 artists would sign to CBS, only 10 per cent of whom would have hits. More than 600 others would see CBS as the dead end that killed their career. Yet Walter saw himself as the man who nurtured artists - seven of them. In a greater or lesser way, that had been the ratio of care meted out to the artists by the majors since the beginning of time.

'People are so anxious to record, they'll sign anything ...' said singer Tom Waits, ' ...like going across the river on the back of an alligator.'

They flocked to the majors asking for a chance. The failure rate was still the same. Count the names of every artist who appeared in the Top 100 from 1980 to 1990 - 1,000 perhaps? Multiply by nine and that's the number who signed to majors and were never heard of again.

As record companies got bigger in the Eighties, everything grew more corporate and less personal. Ron Weisner, who managed both Madonna and Michael Jackson, told me: 'The biggest frustration is always dealing with the record company - cajoling them, bullying them, charming them, threatening them. They're totally insensitive to the artist or his wellbeing.'

Polygram bought up every small company left to buy. At the annual conference in the US, the new German MD started his speech by saying: 'The first time I saw America was through the periscope of a U-boat.'

In the UK, Chrysalis, Zomba and Virgin had grown fast and were opening offices in the States. At Virgin they were trying to boost income, waiting for someone to buy them, using age-old accounting tricks. On one occasion I noticed the royalty statements for Japan had arrived with the artist's royalty less than it should be. It was because the company had first deducted the producer's royalty of 4 per cent. Lower down on the statement it stated the producer's royalty as 4 per cent and deducted it a second time. A call to the accounts office set things right but when the next statement came it was calculated in exactly the same way. A quick call round other managers established it was the same on their statements, too.

Ed Bicknell, manager of Dire Straits, said that dealing with Polygram altered his whole personality. 'You sometimes do things you wouldn't do to a mate. I had no compunction in screwing a corporation. I got through 16 or 17 managing directors ... they're incredibly inefficient and absolutely hopeless to deal with ... '

Ahmet Ertegun was still at WEA, but hating every minute of it. 'They kept putting up people to run it who were non-music people ... they would never take somebody from the cable division and let them run the movie division ... but they would take anybody and let them run the music ... there was no leadership from the top ... it was everybody fighting everybody else ...'.

In the Nineties, WEA's biggest artist, Prince, found it so frustrating he refused to record for them again even though he was still under contract. George Michael attempted to terminate his contract with Sony, which had now purchased CBS. It was rumoured what had triggered George was hearing the company's new president, Tommy Mottola, referring to him as a 'limey fag'. If a Sony employee were referred to in the same way the company would probably end up in court and be fined. But an artist was not an employee, he was just an ingredient. Under advice from his lawyers George didn't sue over this but instead claimed his contract was invalid. It didn't win him his case but it told people a great many things they hadn't previously known about the record business.

Artists had to pay their own recording costs yet companies ended up owning the records. 'The bank still owns the house after the mortgage is paid,' is how Senator Orrin Hatch described it. Could we imagine film stars having to pay the costs of the movies they starred in and then giving the rights to the company that distributed it?

Artists also had to pay a packaging deduction of around 15 per cent. This, despite the fact that packaging rarely cost more than 5 per cent. The remaining 10 per cent was enough to pay the record company's entire cost of manufacturing the record. All in all, it meant an artist who sold 200,000 copies of a first album would still owe the record company although the record company had made a profit of a million.

But the worst thing about being signed to a major was that you lost the freedom to run your life. And though top artists could sometimes re-negotiate an unfair contract, it soon became clear that in the music business you didn't get out of an unfair record contract to get into a fair one; you get out of an unfair contract to get into another unfair one, but with slightly better terms.

Irving Azoff ran MCA for six years. Talking about 'time-honoured accounting traditions in the record business', he tells an industry audience of 3,000 audits on record companies. 'In 2,998 of them the artist was underpaid.'

Everyone had the same story. 'Systematic thievery,' said the Dixie Chicks in their writ against Sony. 'Intentionally fraudulent,' claimed US music lawyer Don Engel.

'Makes Enron look like amateur hour,' wrote music journalist Dave Marsh.

Azoff changed sides. He decided to head the American Artists' Association and sue all the major record companies on behalf of its artists. But he was pessimistic about their winning much; the majors were going under too fast. 'The big boys swooped in and bought all the historic, artist-friendly, independents ... A&M, Geffen, Interscope, Island, Chrysalis. The multinationals rationalised these purchases based on growing cash flows that don't exist any more. Now they are trying to defend failed business plans.'

So what have the major record companies done to try to solve the mess they bought into?

First, they chose to attack their own customers by suing people who downloaded files from Napster. Then Sony amalgamated with BMG and everyone enjoyed the show as top executives fought over who should be made redundant. The joint company had a disastrous setback when it attempted to stop the copying of records by secretly putting a code into CDs which made people's computers more vulnerable to viruses.

Meanwhile, two of Britain's recent big successes, Arctic Monkeys and Franz Ferdinand, signed to an independent, Domino, famous for giving its artists fairer deals. Forced to finally accept downloads into the singles charts, the majors watched as Arctic Monkeys got 18 of their songs into the Top 200. Once, that would have given a major half a million opportunities to sell a penny's worth of vinyl for a pound.

At Domino the deals with artists are more like partnerships and other independent companies are following suit. But the problem with signing to any record company is what might happen if it sells out to a major.

It's clear. The majors should become 'music companies'. They should find new artists, develop them, promote them and participate in all aspects of their earnings. The artist, rather than the record, should be the product. Artists should be developed for longevity, not for quick profit.

Universal, Sony and EMI all claim to be heading in that direction, but nobody believes them. As always, the biggest problem with signing to a record company is the bottomless pit of commitment. When your record flops, how do you extricate yourself?

For 50 years the major labels have thought of themselves as guardians of the music industry; in fact they've been its bouncers. Getting into the club used to be highly desirable. Now it doesn't matter any more.

For artists and managers, this is the moment to take things into their own hands. Artists no longer need to be held for 10 years and they no longer need to sign away ownership of their recorded copyrights. These days, an artist working closely with his manager can ensure that everything is done in the artist's best interest. Majors have never done that. And never will.

Four tops: the major players

Although independents account for 28 per cent of the market, the music industry today is still dominated by 'the Big Four' record companies.

Universal Music Group

Chairman and CEO Doug Morris presides over the largest record company in the world. Taken together, its labels such as Island, Polydor and Mercury account for a 25.5 per cent share in the global market. UMG itself is a subsidiary of Vivendi, the French media conglomerate.

Sony BMG

The result of a 50/50 joint venture between Sony Music Entertainment (part of Sony) and BMG Entertainment (part of Bertelsmann). Subsidiary labels include Arista, Columbia, Epic, J Records and RCA.

Warner Music Group

Subsidiary labels include Elektra and Atlantic. A merger with EMI looked likely until the latter was bought out by private equity firm Terra Firma.

EMI

The only British company in the pack, formed in 1931. Now run by Terra Firma boss Guy Hands; recently parted company with both Paul McCartney and Radiohead.

Life and times: Simon Napier-Bell

1939: born on 22 April in Ealing, west London.

1966: manages the Yardbirds, and co-writes 'You Don't Have to Say You Love Me' for Dusty Springfield, her first No 1. 1967 ditched by the Yardbirds, whom he replaces with Marc Bolan and John's Children.

1976: manages proto-new romantics Japan.

1983: Japan split. Softens blow by co-managing Wham!, who land a No 1 single and album.

1988: receives tax demand for £5m, obliging him to adopt more acts, among them Boney M.

1992: forms new management company. Takes on Ultravox and prog rockers Asia.

1998: publishes first book, You Don't Have to Say You Love Me, about his experiences in the Sixties.

2003: enjoys success with Russian duo Smash!!.

2006: publishes I'm Coming to Take you to Lunch about Wham! in China.

• This article was amended on 29 April 2011 to correct the name of CBS executive Dick Asher.