Long before Frankie Boyle cracked gags about Jordan's disabled son and Jimmy Carr joked about Gypsies, there was Jerry Sadowitz. Unkempt and brutal, with a Glaswegian accent and an unforgettable turn of phrase, he shook up the right-on values of the 80s alternative comedy circuit with his willingness to say the unsayable. "Nelson Mandela, what a cunt," began one of his most famous routines. "You lend some people a fiver and you never hear from them again."

His former manager, the late Malcolm Hardee, once recalled an engagement at the Deptford Albany Empire: "He was on for two nights, and on the second they picketed the place." Twenty-five years on, Sadowitz's shock tactics are part of the comedy mainstream, while the man who pioneered them keeps a low profile. He's rarely on TV, and does gigs only intermittently (a tour began this week).

You could argue that his act is simply in too poor taste for a mass audience. But Sadowitz's lack of fame is due as much to his unpredictability as to the nature of his material. Remembering the numerous occasions the comic refused to go on stage, Hardee concluded he hated success. It doesn't bode well for this interview, but when I get to the designated greasy spoon, Sadowitz is already in situ, ready to chat in a surprisingly gentle manner.

He has no desire to be seen as the godfather of a new breed of PC-baiting comics. "If I had known in advance that so many people would hijack the material I put across in my act, and what they would do to it, I would never have taken up comedy," he says in disgust. "Never. I'm sorry I've given some very nasty people a good living."

Sadowitz objects to the way the genuine rage behind what he does has been turned into a performance by others. "I do find Frankie Boyle offensive," he says. "He'll write a joke, or someone will write it and give it to him, and he'll do it without any thought. It's like someone quoting something without even understanding what it is they've quoted. My contribution to standup is not a genre. It's me and my issues. It was never intended to be mainstream."

Sadowitz's on-stage vitriol certainly doesn't seem faked. "My stuff comes from the fact that my life has been miserable," he says, slowly and deliberately. "I now don't believe I have the capacity to be happy. I would settle for peace of mind. I'd give anything for that. But it's been a completely wasted life. Completely and utterly wasted." For a moment, I think he might be about to cry. "Everything just seems to get worse and worse. I can't see that much great stuff going on in the world, you know? I would rather be happy and have no act. Some would say I'm unhappy and still have no act." He manages a smile.

Punched at the Montreal festival

Where does this misery come from? Sadowitz goes into lockdown, saying he "doesn't like to share". "I'm the wandering Jew's wandering Jew," he says. "I grew up calling Glasgow my home city but I wasn't born there, and [Glaswegians are] pretty strict about that. I was born in America but I certainly don't feel American. I sort of belong ... nowhere."

Coruscating standup is only part of the Sadowitz package: he is equally acclaimed for the remarkable closeup magic he performs between jokes. "I was nine when I first got interested in magic," he says. "I was 11 when I took it up seriously. I don't think I've ever been out without a pack of cards since." He shows me the pack in his pocket, flicking them in and out of view without breaking conversational stride.

For him, the cards aren't simply a craft. "When things are really bad, I can just sit down and focus on the cards," he says. "My great love is working a new trick out. Obviously, I wish I was a scientist; I wish I could discover cures. Ultimately, like everything in life, it's useless, the cards. But I get ridiculously immature pleasure if I discover something new."

Sadowitz is scathing about the current generation of celebrity conjurors. "With the likes of Derren Brown and David Blaine, you just think, 'What's the point?' It's like watching Michael McIntyre and thinking, 'Did we really make no progress since Tom O'Connor?"

He's not a fan of McIntyre? "I hope his next roadshow goes to that island near Oslo. Actually," he relents, "I can't stand to admit it, but for what he does, he's a great craftsman. But what a shame to use that craft on such utter … pap."

You couldn't dismiss Sadowitz's own material as pap, but it inevitably divides crowds. Walkouts are common, as people decide they find the no-holds-barred misanthropy too much. It's easy to imagine the artist-provocateur rubbing his hands in glee every time he upsets the sensibilities of the bourgeoisie, but this isn't the case. "As a Scotsman and a Jew, I know what it is to pay money to see a show. If someone's taken the trouble to pay to see you and they leave upset, then I'm not happy about that. Ideally, I want them to enjoy what I do. They don't have to agree with it." He pauses, then adds ferociously: "I don't want them to agree with it! If they did, they'd feel as screwed up as I do."

He was once punched by an audience member at the Montreal comedy festival, after he came on stage with the words: "Hello moose-fuckers. You know what I hate about this country? Half of you speak French and the other half let them." He has come in for the occasional attack from the critics, too: for every commentator willing to acclaim him a genius, there is another ready to accuse him of pandering to the prejudiced.

Sadowitz is aware of the contradiction. "I've always said to myself, if I ever went on stage and looked out and the audience was a massive group of fascists, making me out to be the figurehead of the English Defence League, then I would not want to do it any more." At the same time, he says, "there is something healthy about anybody getting their darkness out in a theatre rather than going out in the street and robbing someone or throwing a bottle." He doesn't believe in censoring comedy. "Who's to deny anybody their comedy? You can't say that one person's strand of comedy is OK to laugh at, but this person's isn't. There's a great quote that Steve Martin said when he was doing standup, 'Comedy is not pretty.'"

We fall into discussing Ricky Gervais's recent and widely decried use of the word "mong" on Twitter. In fact, it's Sadowitz who brings it up. While he isn't a fan of Gervais ("He does something that's terrible, which is to take brutal subject matter and make it cosy for a large audience"), he will defend his use of the word. "One of the reasons I do what I do in my show is because there is no line to be drawn. So you might as well go all out, because you can use the word idiot, and that'll be offensive to someone who's an idiot. You can use the word wanker – that's offensive to me, I'm a wanker! It's maybe the spirit behind what you say – but even then, I will make allowances for any comic."

'I can't stand looking at myself'

Sadowitz's current UK tour is, amazingly, his first, a delay he ascribes to the difficulties of finding a promoter willing to take a punt on him. He says it's going to lose money: because of his low profile, sales are sluggish in many parts of the country. He finds performing a mixed pleasure. "It's with a heavy heart that you play to English audiences when all they're going to do is say that I am like the others." (By this, he means the likes of Boyle, Carr, Gervais.)

The tour is about as far as Sadowitz will go in terms of engaging with the comedy industry. He won't put a DVD out ("I can't stand looking at myself") or, at least, not unless needs must. "As I get older, and spend more time paying for prescriptions and looking after my mum and making trips to Boots the chemists, who knows what'll happen? I hope I don't end up doing Swiftcover adverts like Iggy Pop." One imagines Swiftcover would be happy to pass, too.

The tragic clown is an old cliche, but few inhabit the role as authentically as Sadowitz. There's no hint of pose, just a sense that he carries around an enormous weight of gloom – which he somehow turns into brilliantly scabrous comedy. He sees the future in bluntly existential terms. "It's like the Jim Morrison line, 'No one here gets out alive.' I knew a magician, who had a wonderful life, he died at 96, he had a spectacular life. But they're so rare. They're so rare."