Four or five people were praying inside the Islamic Society of the Coachella Valley when the fire started about noon, Riverside County Supervisor John Benoit told CNN affiliate KMIR.

Nobody was hurt and the fire was quickly put out, though it caused smoke damage throughout the building, CalFire said.

The FBI is investigating along with local authorities, said Laura Eimiller from the FBI's Los Angeles division. The Riverside County Sheriff's Department said on its Facebook page that the crime was reported as a possible arson, but it did not list a possible cause.

Benoit said an incendiary device may have caused the fire.

"Terrorism should not beget terrorism and that's what this would be if in fact there was someone who threw a device in this mosque with the intention of destroying the mosque or hurting the people inside who were praying," Benoit said.

On November 4, 2014, someone fired four shots toward the Coachella mosque during the early morning hours while four people prayed inside, CNN affiliate KMIR reported. Three bullets struck a vehicle, two hit the building and one lodged in a fence on the property but nobody was injured. No arrests have been made, KMIR said.

A study by the group reported at least 63 acts of vandalism, harassment and anti-Muslim bigotry through December 8, three times more than last year and the highest number of incidents since the group began keeping track in 2009.

Authorities blocked people from entering the Coachella mosque while the investigation was underway. A group gathered on the sidewalk outside the crime scene tape and knelt in prayer.

Alisa Shabazz told KMIR the fire happened shortly before a big crowd of people arrived for weekly prayer. She wondered aloud why somebody would attack her house of worship.

"We're peaceful people," she said. "We don't terrorize people."