When Bacha Khan, a Pakistani trader, returned from a trip abroad to his home in China’s northwest this spring, his Chinese wife and three of his children had disappeared and their house had been demolished.

Police told him his family had been taken into custody, he said, adding to the up to 1 million people, most of them Muslim ethnic Uighurs, that the United Nations estimates have been detained by China in camps in its Xinjiang region.

Mr....