(CNN) State Department sources say they know American residents -- either US citizens or people with legal status in the United States -- are being held in detention camps in Xinjiang, China.

When asked if there were many, one of the sources said, "No, a few."

They were unable to disclose more details due to privacy concerns, for the time being.

At a State Department briefing Thursday, Ambassador at Large for International Religious Freedom Sam Brownback said he had a new, unconfirmed report about a man in California whose father, a legal US resident, had not been heard from since returning to Xinjiang.

"He had legal status being here, traveled back to Xinjiang after being here with his son in California. And then has not been heard from since. And he's deeply concerned about whether, what his treatment is. He has a number of chronic illnesses, he's a 75-year-old man and an intellectual," Brownback said.

Read More