Milwaukee. A bad and quite eerie part of town. This happens to be the very block where the serial killer Jeffrey Dahmer murdered 17 people in the 1980s. Now, from all around, thousands of young men and women, wearing scary clown face paint, are descending upon a disused indoor swimming pool that has been transformed into a music venue. They are juggalos, fans of Violent J and Shaggy 2 Dope, the rap duo known as Insane Clown Posse.

At first glance, it might not be obvious why I'm so excited about meeting them. You might dismiss them as just unbelievably misogynist and aggressive, and it is true that their lyrics are indeed incredibly offensive. Take, for instance, at random:

I'm hating sluts

Shoot them in the face, step back and itch my nuts

Unless I'm in the sack

Cos I fuck so hard it'll break their back.

ICP have been going for 20 years, always wearing clown make-up, which looks slightly lumpy because it's painted over their goatees. They've been banned from performing in various cities where juggalos have been implicated in murders and gang violence. ICP have a fearsome reputation, fostered by news reports showing teenagers in juggalo T-shirts arrested for stabbing strangers and lyrics like "Barrels in your mouth/bullets to your head/The back of your neck's all over the shed/Boomshacka boom chop chop bang."

All of which made Violent J's announcement a few years ago really quite astonishing: Insane Clown Posse have this entire time secretly been evangelical Christians. They've only been pretending to be brutal and sadistic to trick their fans into believing in God. They released a song, Thy Unveiling, that spelt out the revelation beyond all doubt:

Fuck it, we got to tell.

All secrets will now be told

No more hidden messages

…Truth is we follow GOD!!!

We've always been behind him

The carnival is GOD

And may all juggalos find him

We're not sorry if we tricked you.

The news shook the juggalo community to its core. While some fans claimed they'd actually had an inkling, having deciphered some of the hidden messages in several songs, others said they felt deeply betrayed and outraged: they'd been innocently enjoying all those songs about chopping people up and shooting women, and it was Christian rock?

Violent J explained himself unapologetically to a New Jersey newspaper: "You have to speak their language. You have to interest them, gain their trust, talk to them and show you're one of them. You're a person from the street and you speak of your experiences. Then at the end you can tell them: God has helped me."

Of course, one might argue that 20 years was, under the circumstances, an incredibly long time for them to have pretended to be unholy, and that, from a Christian perspective, the harm they did while feigning unholiness may even have outweighed the greater good.

I've come to Milwaukee because ICP have just released their most audacious Christian song to date: Miracles. In it, they list God's wonders that delight them each day:

Hot lava, snow, rain and fog,

Long neck giraffes, and pet cats and dogs

Fuckin' rainbows after it rains

There's enough miracles here to

blow your brains.

The song climaxes with them railing against the very concept of science:

Fuckin' magnets, how do they work?

And I don't wanna talk to a scientist

Y'all motherfuckers lying and

getting me pissed.

10pm. Upstairs, thousands of juggalos are getting drunk in readiness for the show. The atmosphere is riotous and exciting. ICP have a gimmick of throwing gallons of cheap fizzy soda into the crowd, and many juggalos are crushed into the barrier in the expectation of getting soaked and sticky. Backstage, ICP arrive to meet me. They're wearing their full clown make-up – they refuse to meet journalists without it – and are immediately delightful. They smoke, but considerately blow the smoke away from my face. "Oh, I'm sorry, let me put that out. That's some bullshit on my part," says Shaggy 2 Dope when he sees me flinch slightly away from it.

But they also seem melancholy and preoccupied with the negative critical response to Miracles. Saturday Night Live just parodied it ("Fuckin' blankets, how do they work?") and the internet is filled with amused and sometimes outraged science bloggers dissecting the lyrics. Violent J and Shaggy have been watching them, they tell me, feeling increasingly saddened and irate.

"A college professor took two days out of her fucking life to specifically attack us," says Violent J. "Oh yeah, she had it all figured out."

One of the ICP road crew locates the video on his iPhone, and it is indeed withering: "The [Miracles] video is not only dumb, but enthusiastically dumb, endorsing a ferocious breed of ignorance that can only be described as militant. The entire song is practically a tribute to not knowing things."

"Fuck you, man," says Violent J. "Shut the fuck up."

"Did you anticipate this kind of reaction?" I ask them.

"No," sighs Violent J. "I figured most people would say, 'Wow, I didn't know Insane Clown Posse could be deep like that.' But instead it's, 'ICP said a giraffe is a miracle. Ha ha ha! What a bunch of idiots.'" He pauses, then adds defiantly, "A giraffe is a fucking miracle. It has a dinosaur-like neck. It's yellow. Yeah, technically an elephant is not a miracle. Technically. They've been here for hundreds of years…"

"Thousands," murmurs Shaggy.

"Have you ever stood next to an elephant, my friend?" asks Violent J. "A fucking elephant is a miracle. If people can't see a fucking miracle in a fucking elephant, then life must suck for them, because an elephant is a fucking miracle. So is a giraffe."

We watch the video for another few seconds: "It becomes apparent that Shaggy and J consider any understanding of the actual workings of these 'miracles' to be corrosive. To them, knowledge is seen as a threat… For ICP a true understanding of 'fucking rainbows' would reduce them to, as Keats put it, 'the dull catalogue of common things'."

Violent J shakes his head sorrowfully. "Who looks at the stars at night and says, 'Oh, those are gaseous forms of plutonium'?" he says. "No! You look at the stars and you think, 'Those are beautiful.'"

Suddenly he glances at me. The woman in the video is bespectacled and nerdy. I am bespectacled and nerdy. Might I have a similar motive?

"I don't know how magnets work," I say, to put him at his ease.

"Nobody does, man!" he replies, relieved. "Magnetic force, man. What else is similar to that on this Earth? Nothing! Magnetic force is fascinating to us. It's right there, in your fucking face. You can feel them pulling. You can't see it. You can't smell it. You can't touch it. But there's a fucking force there. That's cool!"

Shaggy says the idea for the lyrics came when one of the ICP road crew brought some magnets into the recording studio one day and they spent ages playing with them in wonderment.

"Gravity's cool," Violent J says, "but not as cool as magnets."

"I did think," I admit, "that fog constitutes quite a low threshold for miracles."

"Fog?" Violent J says, surprised.

"Well," I clarify, "I've lived around fog my whole life, so maybe I'm blasé."

"Fog, to me, is awesome," he replies. "Do you know why? Because I look at my five-year-old son and I'm explaining to him what fog is and he thinks it's incredible."

"Ah!" I gesticulate. "If you're explaining to your five-year-old son what fog is, then why do you not want to meet scientists? Because they're just like you, explaining things to people…"

"Well," Violent J says, "science is… we don't really… that's like…" He pauses. Then he waves his hands as if to say, "OK, an analogy": "If you're trying to fuck a girl, but her mom's home, fuck her mom! You understand? You want to fuck the girl, but her mom's home? Fuck the mom. See?"

I look blankly at him. "You mean…"

"Now, you don't really feel that way," Violent J says. "You don't really hate her mom. But for this moment when you're trying to fuck this girl, fuck her! And that's what we mean when we say fuck scientists. Sometimes they kill all the cool mysteries away. When I was a kid, they couldn't tell you how pyramids were made…"

"Like Stonehenge and Easter Island," says Shaggy. "Nobody knows how that shit got there."

"But since then, scientists go, 'I've got an explanation for that.' It's like, fuck you! I like to believe it was something out of this world."

Violent J's real name is Joseph Bruce, Shaggy's is Joseph Utsler. They're in their late 30s. Their career, while at times truly glittering, is littered with inadvertent mistakes. Born and raised in Christian homes in Detroit, they've known each other since high school. "We were dirt poor," Shaggy says. "You can't get no poorer. Fighting, food stamps, I was a fucking thief for a living, hustling, getting money, we were balls deep in that shit."

Their first band, Inner City Posse, was without clown make-up. They were gangsta rappers, and consequently found themselves behaving in a gangster-like manner. In 1989, Violent J was jailed for 90 days for death threats, robbery and violating probation. When he got out, he and Shaggy made some life-defining decisions. How could they keep their rap career going but move away from the destructive gang lifestyle? How could they change the band's name but keep the initials ICP? People liked the initials ICP.

And then it came to them in a flash: Insane Clown Posse! Killer clown rap! It was the perfect outlet for their emotions. Write about the pain and the anger through the prism of horror-movie imagery. A whole new genre.

"We had to work our ass off from the ground up," Violent J says. "We don't get radio play. We don't get video play. We get nothing. This is our video play…" He indicates the dressing room. "Being on the road. We didn't have no Jay-Z telling everyone, 'Hey, look at these guys, we're friends with them, listen to them.' To this day, we don't get that."

This aspect of things might have turned out rather differently had Violent J not made their first big error. It was 1997. Insane Clown Posse were enjoying an early flush of success – their albums Riddle Box and The Great Milenko had sold a million copies. One night they were in a club when a young man handed them a flyer inviting them to a party. The flyer read: "Featuring appearances by Esham, Kid Rock, and ICP (maybe)."

"Why are you saying we're going to be playing at your party when you haven't asked us?" Violent J yelled at the boy.

"It says 'maybe'," he said. "Maybe you will be there. I don't know. That's why I'm asking you right now. Are you guys coming to my party or what?"

"Fuck no," Violent J replied. "We might have, if you'd asked us first, before putting us on the fucking flyer."

That boy grew up to be Eminem and, incensed, he's been publicly deriding ICP ever since in lyrics such as, "ICP are overrated and hated because of their false identities".

An observation that turned out to be prophetic. "From the very beginning of our music, God is in there," Violent J says, "in hidden messages."

"Can you give me some examples?" I ask.

There's a small silence. He looks torn between revealing them or maintaining the mystery. He shoots Shaggy a glance.

"The Riddle Box," he finally says.

Hey, what's up, motherfucker

This is Shaggs 2 Dope

Congratulating you on opening

the Riddlebox

It looks like you received your prize

The cost, what it cost, was your ASS,

bitchboy!

Hahahahah!

(The Riddle Box, 1995)

"If you died today, God forbid, if you were hit by a car and you had to turn the crank to your own riddle box, what would pop out?" Violent J peers at me. "Would it be God, or would it be the devil? Only you truly know the answer to your own riddle box. We're asking the listener, what is in your own riddle box if you were to die today?"

"Cos you can't lie to yourself, man," says Shaggy.

"Only you know the answer to that riddle," Violent J says. "And then there's The Ringmaster. In The Ringmaster, we say when you die you have to face your own beast. Somebody who has lived a life of religion, they face a very small and weak beast when they die. But somebody who's an evil bastard will have to face a monster. The question is, how big is your ringmaster? If, God forbid, you were hit by a car. Ask yourself, Jon." Violent J looks me in the eye. "How big is your ringmaster?"

"How come it took you so long to make the announcement?" I ask.

"You had to gain everybody's attention," says Violent J. "You had to gain the entire world's trust and attention."

"So all those unpleasant characters in the songs," I ask, "like the narrator in I Stuck Her With My Wang, they're examples of people you shouldn't be?"

"Huh?" Violent J says.

"Well, it's very unpleasant," I say. "'I stuck her with my wang. She hit me in the balls. I grabbed her by her neck. And I bounced her off the walls. She said it was an accident and then apologised. But I still took my elbow and blackened both her eyes.' That's clearly a song about domestic violence. So your Christian message is... don't be like that man?"

"Huh?" Violent J repeats, mystified.

There's a silence.

"I Stuck Her With My Wang is funny," Violent J says. "Jokes. Jokes, man. Jokes. Jokes. Jokes. It's just a ridiculous scenario. Silly stories, man. Silly stories. What's she doing kicking him in the balls? We find it funny. But we're saying, while we're close, while we're hanging, hey, man, do you ever ask yourself what's in your riddle box? If you had to turn the crank today?"

"But still, given that you were secretly Christian, are there any lyrics you now regret?"

There's a silence. "Yeah," Violent J says quietly.

"Which ones?"

"Dumb, stupid, idiotic lyrics that I said without knowing any better. Back in the day."

"Like what?"

"I really don't want to say. There's one lyric…" He trails off, suddenly looking really sad beneath the clown make-up. "Just dumb lyrics. I said one lyric one time that I hate. I may have been feeling really down that day. I said something, I live with that every day. I don't want to point it out."

I later do a search and find it difficult to pinpoint exactly which lyric he may be referring to. It just might, I suppose, be, "I took aim at a stray dog, and I blew out its brains, it was fresh as hell, no feelings for others, you gotta be cold."'

Violent J says releasing Thy Unveiling, coming out as a Christian, was the most exciting moment of his life. "It felt so good, brother. I was fucking in heaven. Let me tell you something: I would go running at night, and my feet wouldn't even touch the ground. I had my headphones on, I'd be listening to Thy Unveiling, and I'd be in such a zone that my feet wouldn't even be touching the ground. I'd be literally levitating."

He was worried, of course, about the reaction from the juggalos and, sure enough, "The emotional impact shook the whole juggalo foundation, for good and for ill."

"What did the juggalos who were opposed to it say?" I ask.

"They said, 'Fuck that'," says Shaggy.

"But the juggalos and juggalettes who were for it were so touched," Violent J says. "They said they loved us."

And then the reviews came in.

Blender magazine, in its list of the 50 worst artists in music history, call ICP the very worst of all: "Insane Clown Posse sound even stupider than they look. Two trailer-trash types who wear face paint, pretend to be a street gang and drench cult devotees in cheap soda called Faygo, Violent J and Shaggy 2 Dope are more notorious for their beef with Eminem than their ham-fisted rap-rock music." And their nadir, Blender said, the worst musical moment from the worst band ever, is The Wraith: Shangri-La, the album that climaxes with Thy Unveiling.

I suddenly wonder, halfway through our interview, if I am looking at two men in clown make-up who are suffering from depression. I cautiously ask them this and Violent J immediately replies. "I'm medicated," he says. "I have a lot of medicine that I take. For depression. Panic attacks are really a serious part of my life." He points at Shaggy. "He's gone through some things as well."

"You do a show in front of how many hundreds or thousands of people." Shaggy nods. "You're giving your full being, your soul, to every person in that crowd, every pore in your body is sweating, you're fighting consciousness, just to get it out of you, and after the show all your fans are partying, 'Yeah! Rock and roll!' And you're just here." He glances around the dressing room. "You're just fucking sitting here."

Violent J turns to him and says, softly, "If we moved furniture for a living we'd have a bad back or bad knees. We think for a living. We try to create. We try to constantly think of cool ideas. And every once in a while there's a breakdown in the engine… I guess that's the price you pay."

Shaggy nods quietly. "I get anxiety and shit a lot," he says. "And reading that stuff people write about us… It hurts."

"Least talented band in the world," Violent J says. "No talent. When I hear that I think, 'Damn. Are we that different from people?'"

He looks as if he means it – as if he sometimes feels hopelessly stuck being him.

It's just a terrible twist of fate for Insane Clown Posse that theirs is a form of creative expression that millions of people find ridiculous. But then suddenly, palpably, Violent J pulls himself out of his introspection. They're about to go on stage and he doesn't want to be maudlin. He wants to be on the offensive. He shoots me a defiant look and says, "You know Miracles? Let me tell you, if Alanis Morissette had done that fucking song everyone would have called it fucking genius."

• This article was amended on 11 October and 7 December 2010. The original referred to Insane Clown Posse's announcement on their Christianity as "recent". This has been corrected. The original also said that Dahmer "ate 17 people". This has been corrected.