(CNN) The fatal shooting of two black people in a Kroger grocery store in Kentucky is being investigated as a hate crime, Jeffersontown Mayor Bill Dieruf told CNN on Monday.

Gregory A. Bush, a white 51-year-old, is accused of killing Maurice Stallard, 69, and Vickie Jones, 67, last Wednesday inside the Jeffersontown grocery store and in the parking lot, respectively.

Prior to the shooting, Bush allegedly tried to enter a predominantly black church nearby but was unable to get inside, officials said. When that attempt failed, he went to Kroger instead and opened fire in the store.

Dieruf also said investigators are looking into reports that Bush told a bystander before he was captured that "whites don't shoot whites."

Given that, Dieruf said, the shooting "appears as if it is" a hate crime.

"We have to get this all in line. The U.S. Attorney is reviewing the case, as are ATF and the FBI," the mayor said. "A crime against race is something that we will not accept. If it's a hate crime, it's unacceptable to us that a crime would be committed against a race -- certainly against any nationality or race is heinous."

Heather Hogue, the spokeswoman for the Jeffersontown Police Department, confirmed that police Chief Sam Rogers believes the shooting "appears to be motivated by hate." She also said any charges related to hate crimes would be federal charges and separate from the state charges against Bush.

Members of the Louisville Metro Police Department talk inside a Kroger grocery in Jeffersontown, Kentucky following a shooting on Oct. 24, 2018.

Bush exchanged multiple rounds of gunfire in the Kroger parking lot with an armed civilian who had a carry concealed permit, police said. Nobody was injured, and the suspect fled the scene and then was arrested nearby, authorities said.

He was arraigned last Thursday on two counts of murder and 10 counts of wanton endangerment and is being held on $5 million bond, according to CNN affiliate WAVE.

The difficulty of hate crime charges

Although Bush does not face hate crime charges at this time, several officials have said they believe the crimes may fit that definition. Hate crimes are defined by the FBI as a traditional criminal offense but with an added element of bias.

Russell M. Coleman, the US attorney for the Western District of Kentucky, said in a statement that federal investigators are "supporting local law enforcement and examining this matter from the perspective of federal criminal law, which includes potential civil rights violations such as hate crimes."

Senate Majority Leader Mitch McConnell, who represents Kentucky, said he believed that the Kroger shooting and the mass shooting at a Pittsburgh synagogue on Saturday were both hate crimes.

"We witnessed, all of us, two horrendous shootings this weekend. One in a synagogue in Pittsburgh and one in a Kroger's store in Louisville," McConnell said. "If these aren't definitions of hate crimes, I don't know what a hate crime is. And I know that's a legal determination that's going to be made by others but that certainly is my opinion."

CNN law enforcement analyst James Gagliano said that the Pittsburgh shooter's anti-Semitic remarks to police at the scene made connecting the dots fairly easy for hate crimes charges.

In general, though, officials act in an abundance of caution before applying hate crimes charges because it requires officials to prove their motivation.

"It's trying to intuit intent," he said. "And that's a very difficult thing sometimes."

Still, he said he believed the Kroger shooting investigation would move in that direction in the near future.

"Does there appear to be enough there that you could charge this as a federal hate crime? I think the answer yes," he said. "It just hasn't happened yet. But that doesn't mean it's not going to happen."

There were a total of 6,121 hate crime incidents in 2016, including nine hate crime murders , according to FBI statistics. However, some legal experts say that the FBI data is undercounted because the hate crime reports are provided voluntarily, and not every police department participates.

Mayor visits targeted church

Surveillance video showed that 10 to 15 minutes before the grocery store shooting, Bush tried to enter the First Baptist Church in Jeffersontown, Chief Rogers said.

A church member sitting in the parking lot saw the suspect banging on and pulling the door, trying to get inside, CNN affiliate WDRB reported

"To think that an hour and a half earlier, we had 70 people in the church," church administrator Billy Williams told WDRB. "But by the time he came through, all doors were locked, and there were probably eight or 10 still in the building."

In the wake of that threat, Mayor Dieruf said he attended the First Baptist Church on Sunday and spoke with parishioners.

"I told them in church, there's no reason we can't be the beacon that changes the world," Dieruf said. "They get it. When you walk in and sit down, people shake your hand and hug and thank you for being there. They haven't gotten the fear of not welcoming people -- they welcome everybody.

"Five days after an incident like this, of a person probably trying to break in, that's what we have to do on a national basis. We've got to get back to where we feel safe and where we feel welcome," he said.