Less than a decade before he would earn his first Oscar nomination for playing a death row prisoner in Dead Man Walking, Sean Penn was sitting behind L.A. County jail bars himself for reckless driving and punching an extra on the set of his movie Colors. (Hey, it was the 80s.) And while inside that prison for 33 days of a 60-day sentence in 1987, the actor was at one point stationed catty corner to Richard Ramirez, the late serial killer and rapist who, in 1989, would become an actual death row inmate. And apparently, Ramirez was a fan of Penn’s.

During a Q&A Penn took part in at Loyola Marymount University’s School of Film and TV last week, The Hollywood Reporter reveals that Penn spoke about his strange interaction with Ramirez.

I was down here on Bauchet Street, L.A County jail in the cell working kind of across from him. And after about a month of, you know, seeing each other around and all of that he wanted my autograph. So he sent one of the deputies over, deputy came to my cell and told me, “Hey, you know, Richard Ramirez wants your autograph.” And I didn’t trust the deputy because I'd gotten in some trouble inside there . . .so I said bring the sergeant down here and I'll talk to him and if he approves it, then I want him to write something first and then I'll write him something back. So the sergeant came down and approved it. . .And so they went over to Ramirez and, you know, this guard basically wanted to play Cupid in some way. We were both on 24 lockdown. . . but so I get this thing from him and it says, “Hey Penn, stay tough and hit them again. Richard Ramirez, 666” with a pentagram and a rendition of the devil.

Penn revealed that he did respond with his own note which said, per Penn’s recollection: “Richard, it’s impossible to be incarcerated and not feel a certain kinship with your fellow inmates. Well Richard, I've done the impossible, I feel absolutely no kinship with you. And I hope gas descends upon you before sanity does. . .it would be a kinder way out.”

That appeared to be the end of the prison pen pal communications between Penn and Ramirez. Even though Penn did not feel kinship with his fellow inmate, he did feel inclined to hold onto the note for a few decades. When the actor’s home in Malibu caught fire in 2007, though, Ramirez’s note went up in flames with it. “My house burned down years later and that damn thing of his burned with it so I don’t have it,” Penn explained.

To read the complete Q&A, click here.

Related: Hunkered Down in Haiti with Sean Penn, Humanitarian