In rock music’s civil war over the Confederate flag, Tom Petty has chosen a side.

After South Carolina lowered the Battle Flag last week, many bands have continued to sell merchandise featuring the stars and bars, while Detroit-native Kid Rock told those protesting his use of the flag to kiss his ass.

“Isn’t Kid Rock from the Midwest,” Tom Petty asked Rolling Stone in an interview Tuesday. “I think they were on the other side of the Civil War.”

Related: Can Confederate flags cause a government shutdown?

Petty feels no need to defend his own past use of Confederate imagery.

“I wish I had given it more thought,” Petty said. “It was a downright stupid thing to do.”

In 1985, Tom Petty and the Heartbreakers released an album titled Southern Accents, which featured a song from the perspective of a southerner who “still blames the North for the discomfort in his life.”

“So my thought was the best way to illustrate this character was to use the Confederate flag,” Petty explained. “I used it onstage during that song, and I regretted it pretty quickly.”

The flag was featured prominently in the marketing for the band’s ensuing tour. In a live record titled Pack up the Plantation: Live!, the band performs in front of a large Confederate flag.

Petty told Rolling Stone that he always regretted using the flag in that manner, and wishes he’d reflected on how others might perceive it.

“People just need to think about how it looks to a black person. It’s just awful. It’s like how a swastika looks to a Jewish person,” he said.

The rocker went on to say that Americans should be more concerned about losing innocent black men to police killings, than a symbol of “southern heritage.”

“As a country, we should be more concerned with why the police are getting away with targeting black men and killing them for no reason. That’s a bigger issue than the flag,” Petty said.