In his 2000 book Angry Blonde Eminem detailed how the song came to be:

After I did “Marshall Mathers”, I felt like I still hadn’t captured that “Still Don’t Give a Fuck” feel. I played it for my manager, Paul, and he said, “It’s dope, but it’s still not ‘Give a Fuck’”. Once again, the funniest shit happens when I’m about to bail. We were about to leave the studio for the day, ‘cause we couldn’t come up with shit. Jeff was in the next studio playin’ with this old-ass piano. He was playin' the frantic piano loop that gave “Criminal” that sinister feel. All I had was “I’m a criminal”. I hadn’t filled the blanks in yet. So I started writing the rhyme and within twenty minutes we were on our way out the door with a hook and the first two verses in hand. I went home and finished the last verse. Afterwards I put in the skit right before the album was finished. Only fucked-up thing was that it took almost a whole day to do that bank robbery skit. Even Dre walked out of the studio like, “Fuck it, I’m out”. We had to get Mel-Man to get his lines right, ‘cause he was so drunk. He just had to say, “Don’t kill nobody”. But what really took so long was the noise in the background. I did that skit by myself. I learned how from Dre when we hooked up on “Guilty Conscience”. “Criminal” was my new “Still Don’t Give a Fuck” for The Marshall Mathers LP. That’s why I did the same intro as I did on “Still Don’t”. That’s why—just like “Still Don’t Give a Fuck"—it’s the last song on the record. It sums up the whole album.