He's the former junkie punk who put the filth into fashion. Now, alongside his X-rated ad campaigns and sleazy shoots for glossy magazines, photographer Terry Richardson wants his art to be taken seriously. He tells Sean O'Hagan why he swapped smack for smacking

I am standing in an overcrowded art gallery in downtown Manhattan, feeling slightly queasy. In front of me, taking up most of the wall, is a huge photograph of a naked girl engaged in the kind of sexual act that defies description here. Let's just say that her hair is in a mess. There are many questions going through my head at this moment, not least why an image of this kind has ended up in an art gallery.

It's then that I notice the familiar-looking young woman standing beside me. She is wearing a dress that redefines the terms skimpy and diaphanous, and she is giggling uncontrollably. At herself. She is the same young woman who is up there on the wall. Her companion is staring open-mouthed at the photograph, his face registering what can only be described as a mixture of shock and awe. He turns to her in disbelief. 'You didn't!' he shrieks. 'You didn't!'

But, as several other images on the walls attest, she did. Over and over. The gallery is bedecked with similar photographs: naked and glistening young girls, their legs akimbo, backsides thrust in the air, lipsticked mouths open in anticipation. Sometimes there is just one girl, snapped from above in an act of oral devotion, or in a post-coital daze; sometimes there are two, occasionally three. Sometimes, on closer inspection, the girls turn out to be boys, or boy-girls, their petite penises dangling helplessly between their long feminine legs.

The only penis that does not dangle belongs to the photographer whose name graces the show, and whose naked frame and goofy, bespectacled face features throughout. His name is Terry Richardson, and the whole show consists of self-made images of Terry thrusting, rucking, prodding, pumping and, sometimes, grinning at the camera like a nerd let loose in porno heaven.

Most people here tonight seem to find the show inexplicably funny, and there is a queue to buy the Terryworld T-shirts and condoms. Others, though, do a swift about-turn as soon as they have made it through the bottleneck at the door. It's all too much. Too crude. Too in-your-face. Too numbingly, thrashily hard-core. Here, for instance, is Terry being serviced by two babes who could be, may well be, fashion models. Here he is receiving a blow job from a girl who, for some reason, is trussed up in a suitcase, just her head - and open mouth - protruding. And here he is being fellated by another girl crammed into a dustbin.

In this context, the work of the late Helmut Newton seems positively tame, quaint even. The Terryworld show would have a London gallery closed down within hours. And yet here it is in the heart of Manhattan's art world, in a reputable SoHo space, the Deitch Gallery, drawing the biggest crowd of the year. The street outside is thronged with the thin and the cool: fashionistas, agents, artists, countless hangers-on. There are even one or two celebrities, including art-rocker Kim Gordon, film director Wes Anderson, and actor/director Vincent Gallo, whom, one suspects, is grateful to be invited anywhere since his recent public espousal of ultra-conservative Republicanism.

'Terry is one of the more charismatic figures in downtown culture,' the gallery owner, Jeffrey Deitch, will later tell the New York Observer, and tonight's impromptu block party certainly attests to that. One whole stretch of Wooster Street has been cordoned off by the NYPD, such is the crowd milling about. There is a Red Cross emergency worker stationed inside the airless and overcrowded space. Terryworld is a strange and contradictory place where art and fashion and pornography converge, and where, for the time being at least, pornography is the dominant aesthetic.

'Do you really think it's porn?' Terry Richardson asks me a few days later, as we sit on the rather dilapidated patio of his first-floor studio-cum-apartment overlooking the Bowery. Terry is dressed in a black T-shirt and work pants, white socks and trainers. He is smaller than his photographs suggest, and has a less cartoonish face: short-cropped hair, receding at the temples, a handlebar moustache and big, tinted, Seventies specs that exaggerate his geekiness. His muscular arms are covered in tattoos: old-fashioned sailor-type images of busty girls, strange hieroglyphics, an elaborate inky black spider's web that spreads around one elbow.

Terry's pad is typical Lower East Side boho - open-plan, with a kitchen at one end and a den at the other, a double bed built into a niche in one wall. There are various black-and-white photographs of girls, street dudes and punk groups arranged here and there on the wall, but not, as far as I can see, any of his own work. Interestingly, he lives on the same street as CBGB's, the legendary punk dive, and close to a street recently renamed Joey Ramone Place. This is Terry's 'hood', and, as befits a guy who borrowed the DIY ethic of punk and applied it to photography, he feels emotionally at home here. He belongs.

Me, I feel oddly guilty, because Terry is affable and open, obviously more sensitive than his more extreme work suggests, and I seem to have offended him by suggesting that the recent photographs he has taken of himself having sex with various young girls might be pornographic.

'The thing is, I don't personally like porn,' he says, shaking his head, and sucking on the first of several cigarettes he will get through over the next few hours, his voice sounding even deeper than usual due to all the talking he has done since the show's opening. 'Porn kind of bums me out because there is so much sadness and pain in that world. So little joy or even pleasure. I don't use porn or even go to strip clubs, like a lot of my friends. I don't like to exploit anybody. That's not my bag. Everyone has fun on my shoots.'

This would indeed seem to be the case. The girls who now come knocking on the door of Terry Richardson's studio to take part in what he calls his 'spontaneous sex acts' may be young or impressionable, exhibitionist or insecure, or all of the above, but they are all too eager and willing to perform for his camera. It's as if all the hoary old cliches about the camera as phallus, the photographer as power-hungry sexual predator á la David Hemmings in Blow-Up, have come true with a vengeance in his work, except that he does not have to cajole or pressurise his subjects. They're queuing up.

'My rule is that I'd never ask anyone to do anything I wouldn't do myself,' he says, 'that's how it's got to go this far. At first, I'd just want to do a few nude shots, so I'd take off my clothes, too. I'd even give the camera to the model and get her to shoot me for a while. It's about creating a vibe, getting people relaxed and excited. When that happens,' he adds, grinning his goofy, adolescent grin, 'you can do anything.'

For the uninitiated among you, it's important to understand how much in demand and influential Richardson is as a fashion photographer in order to even begin to understand how he gets away with all the other stuff he does. Fashion, in case you needed reminding, is a kind of parallel universe, where the normal rules of behaviour - social, moral, ethical - do not apply. Fashion designers, art directors and magazine editors have flirted with heroin chic, anorexic chic and even terrorist chic, but porn chic has proved the most enduring aesthetic.

A cursory glance at recent back issues of style magazines such as Dazed & Confused and Pop, as well as occasional issues of high-end titles like Vogue, indicates the dominance of faux-porn imagery as the abiding style attitude. Terry is the undisputed king of porn chic.

'He's the ultimate manifestation of the myth of the promiscuous photographer,' says Charlotte Cotton, a curator at the Photographer's Gallery in London, 'and his fashion work to date has played around with that myth and, in the process, had a real pertinence in an industry that tends to be conservative and anodyne. But photography is also about context, and you have to be careful when you move into the very different context of an art gallery. Just presenting sexual images, some would say pornographic images, in terms of their content alone, is not enough. Nor is parading your psychosexuality. You have to find a way to be more clever than that.'

Terry Richardson, though, has no truck with cleverness for art's sake, has no real issues with whether his work is deemed art or porn. Richardson is just doing his thing. This month, art publisher Taschen will put out Terryworld, a retrospective that shares the same name as his Deitch show, but mixes the sleazier stuff with fashion photographs, celebrity portraits and even the odd landscape. What is arresting about the book is not the hardcore sex, but the intimacy of some of the portraits: a blissfully grinning child who has just been feeding at his mother's breast; Dennis Hopper, his face wreathed in cigarette smoke. Ironically, Richardson has a gift for tenderness that is not always evident in his pathological pursuit of the sexually shocking.

Confusingly, there is also an even more sumptuous book called Kibosh, published by Damiani, which features the harder stuff from the Deitch show, and retails at around £300. 'That's the strange thing in all this,' elaborates Stephen Male, an art director who once worked with Richardson on an early Levi's campaign. 'We suddenly have a few prominent photographers who are dealing with their sexual neuroses, and for some reason we're all expected to watch. Even stranger is that many of us do just that.'

Richardson first came to prominence in the mid-Nineties, shooting fashion editorials and ads that were starkly lit, brutally cropped and shot on snapshot cameras with little or no lighting. His primitive-cool aesthetic was the direct antithesis of the glossy, big-production work of other fashion photographers such as Nick Knight or Stephen Meisel, and had a more obvious sexual edge than other purveyors of the snapshot aesthetic such as Juergen Teller and Corinne Day.

In recent years, Richardson's signature advertising campaigns for labels like Katharine Hamnett, where the models' pubic hair was visible beneath their short skirts, and Sisley, where, memorably, the model Josie Moran squeezed milk from a cow's udder into her mouth, established him as a photographer prepared to push the boundaries about as far as many assumed they could go. And, in a way, this has been the case. Richardson's fashion work still tends to be edgy - he has made Kate Moss, minus her knickers, look like a world-weary call girl in the latest issue of Pop - but he has shown an increasing tendency to adapt to meet the more mainstream demands of the client. His current campaign for Miu Miu, featuring the young actor Maggie Gyllenhaal, is opulent and romantic, a world away from open crotches and post-coital poses.

But over the past few years, Richardson's non-fashion photography, which he considers his most important work, has gone the other way, his images becoming ever more hardcore in their depiction of Terry's own sex life. Everyone in fashion had heard about the 'spontaneous sexscapades' that occurred from time to time when Terry and a model hit it off. When word got out about them, though, he found that certain girls would turn up with exactly that in mind. Soon, the shoots got wilder, and often Terry's assistants, Seth and Keiji, had to be on hand to take the actual pictures. 'I always say I make pictures rather than take pictures,' explains Terry. It turns out that Alex, a blonde waif who works in his office, is a posh art student who originally assisted Terry on a Miu Miu campaign. Now, she is up there on the wall with the rest of them, captured for posterity fellating her boss from inside the office rubbish bin.

Soon, Terry had shot hundreds of similar images, all featuring himself in flagrante with a female volunteer. His psychosexual archive, he tells me, 'is the most satisfying and exciting work I have ever done'. Leaving aside for a moment the obvious question about whether a pornographic image can ever possess artistic merit, many of these photographs seem grounded in, at one extreme, adolescent fantasy gone mad, and, at the other, some darker personal demons - narcissism, obsession, compulsion, even addiction.

'Oh, I have lots of stuff I am working out through my work,' he freely admits, though one suspects he tends to view his neuroses the way the rest of us might view our hobbies. 'I mean, I don't think I'm a sex addict, if that's what you're asking, but I do have issues, tons of them. Like, this current show could be about my midlife crisis. Or it could be something to do with the fact that since I gave up drinking and taking drugs, I have to get high on sex and being an exhibitionist. Or maybe it's the psychological thing that I was a shy kid, and now I'm this powerful guy with his boner, dominating all these girls. In a way, that's the very stuff I'm trying to work out in the work.'

I ask if it has ever crossed his mind that his 'stuff' might be better worked out in private? 'Why?' he replies, in all seriousness. 'I mean, I'm a photographer. I record stuff. Why not record myself? I'm only doing what everybody else does behind closed doors. They take Polaroids of their girlfriends, then they hide them in a drawer, or post them on the internet with her face blacked out. That seems a whole lot less healthy to me. That's about shame and fear. I just do it all, and put it all out there. That's what's incredibly liberating.'

So, it's photography as therapy? The nerd's revenge, maybe? He gives this some thought. 'Partly, but not only. Look, I'm just a regular guy who's trying to deal with things, and figure stuff out as I go along. If people want to call it porn, that's cool with me. If people are shocked or offended by it, that's even better. A guy came up to me on the street yesterday, and started shouting, "I hate your show. I hate all your fucking pictures."' What did you do? 'I thanked him.'

By all reports, Terry Richardson was once a neurotically shy young man. And, before that, a sad and angry child. Until punk provided a vehicle for his self-destructive urges, he used to smash things up, furniture mainly, but sometimes himself. His anger first surfaced when he was four and his father Bob, a briefly successful fashion photographer, left his mother for the then 17-year-old actress Anjelica Houston. Terry's mother, in turn, took up with Jimi Hendrix and Keith Richards, and Terry once walked in on her making out with Kris Kristofferson. When he was nine, his mother was on her way to collect him from his child psychiatrist when a Pacific Bell telephone truck smashed into the back of her stationary Volkswagen. She was left with permanent brain damage, and in the charge of her already emotionally disturbed son.

'It was shell shock, basically,' he says, quietly. 'One minute, I was flying around in jets, I had a colour TV in my bedroom; the next my mother and me were on welfare, living on food stamps. She was in diapers and barely able to move or communicate. It was a real tough time. I retreated into getting high on weed, and when I wasn't high, I was this angry, sad kid. There is not one photograph of me looking happy as a child. I was kind of lost for a long while.'

Today, Terry has a tattoo of himself as a sad little kid on his upper torso, but, one suspects, the real scars run much deeper. He has made up with his wayward father, who, having been homeless for a long time, now lives on social security in California and, with Terry's help, plans to publish a book of his Seventies work. They briefly tried to work together, but it ended in tears.

'We're close now, but it took a while. It's a beautiful and amazing thing to be able to hug him and tell him I love him, seeing as I tried to strangle him a few times, threw him across the apartment. All the classic father/son Greek mythology stuff.'

After a stint in various Southern California punk bands, including SSA (Signal Street Alcoholics), Baby Fist and Middle Finger, and a brief early flirtation with heroin, Richardson started taking photographs in earnest in the early Nineties. He shot whatever caught his eye on the street, and made a fitful living assisting 'other guys who weren't half as original'. His inspirations were Nan Goldin and Larry Clarke, documentarians of their own broken lives, as well as William Eggleston, whom he calls 'a true American visionary'. Richardson's own breakthrough came when he was commissioned to do a street fashion story for Vibe magazine in 1991, and he responded by shadowing a gang of teenagers for a night, snapping them as they hung out. The influential British art director Phil Bicker saw the result, and promptly hired Richardson for a campaign he was doing for Katharine Hamnett. The rest, as they say, is fashion history.

'Terry always did his own thing,' says Stephen Male, recalling those early days. 'I remember when the contact sheets came in for the Levi's shoot we did, every single one would feature a photograph of the model with her top off. I remember thinking, "How did he do that?" I mean, it wasn't really what was required for the shoot. Then it became Terry's thing. It seems almost quaint and old-fashioned now that we'd find it questionable. But that's fashion for you. Once somebody pushes the envelope, it doesn't take long for taboo images to become acceptable. Plus, fashion needs people like Terry; it needs to feel it has an edge all the time.' Terry, unsurprisingly, concurs. 'Hell, somebody's gotta come up once in a while and say bollocks to all that mainstream, glamour stuff.'

One feels, though, that fashion photography is no longer enough for Terry Richardson, that the rush he got from recording, then displaying, his own X-rated reality show will be both a catalyst for his already hyperactive creative imagination, and a hard(core) act to follow. There are signs, too, that he may already be tiring of the sexual infamy that he has worked so hard to sustain. He has been commissioned to write and direct a feature film, Son of a Bitch, about a father who returns out of the past to derail his son's life. It will, he insists, be entirely free of sex scenes.

'I love sex,' he says, 'and, above all, I love the first time I have sex with someone. That's the real buzz. Often girls have said to me: "Terry, it was all downhill after the first fuck," and that's kind of true, I guess. Sometimes, lately, I find myself thinking I'd really love to settle down, get married, have kids, have a regular relationship.'

Given his reputation, that may well remain wishful thinking, at least until he works through his compulsion to record his every casual sexual encounter on camera. His first marriage, to model Nikki Uberti, was short-lived and volatile, fuelled by copious amounts of Class A drugs. Richardson says any hopes of reconciliation were complicated by his efforts to stay off heroin, with which he had a long and fitful dependency.

He has been drug-free for three years now, after a group of friends staged an intervention when they found him comatose in his apartment on Christmas Day 2001.

'I was at the bottom, man. I'd just broken up with a girlfriend three days before, and I'd gone on a binge over Christmas. I'd done $100 worth of smack, taken a bunch of Valiums and drunk a bottle of vodka. I put on a suit and tie for Christmas, then it hit me that I was all alone. I went to sleep hoping that I wouldn't wake up. That's when the guys found me, and sent me off to rehab.'

These days, clean if not altogether serene, Terry Richardson's personal life seems a lot less messy than it was before. He has just split up, though, with his model girlfriend, Susan Eldridge, but seems remarkably chilled about his current situation, despite the often tragic events that have dogged what he calls, with Disney innocence, 'his incredible journey'.

'I felt I had to open up my soul and let all this stuff out,' he says, 'and then I could move on. I guess some people are more scared of me now than they were before, because they think I'm a crazy, exhibitionist nut. But the people who know me probably just think, "Oh, that's Terry doing his thing." Whatever, I'm cool with it. In my head, I've already moved on. I might do a kids' book next. Or just people's faces. Or I might do my out-of-the-closet book next.' He pauses, grinning. 'That's a joke, right?' If you say so, Terry, if you say so.

· To order Terryworld by Terry Richardson (£34.99, Taschen) for £31.50 with free UK p&p, call the Observer Book Service on 0870 836 0885, or go to www.observer.co.uk/bookshop