Silicon Valley startup Pilot AI Labs Inc. signed a Chinese-backed venture-capital firm as its first big investor in 2015. By last summer, Pilot AI wanted it gone.

The U.S. startup hoped to sell more of its artificial-intelligence software to the U.S. government after working with the Pentagon, according to people familiar with its operations, and worried its effort could be hurt by the investor’s ties to China’s government. The chairman of the Chinese-backed investor, Digital Horizon Capital, was asked to sell back its stake,...