QINGDAO, China—When the Japanese and European companies that pioneered high-speed rail agreed to build trains for China, they thought they'd be getting access to a booming new market, billions of dollars worth of contracts and the cachet of creating the most ambitious rapid rail system in history.

What they didn't count on was having to compete with Chinese firms who adapted their technology and turned it against them just a few years later.

Today,...