TOKYO—Japan’s biggest bitcoin exchange said it would halt taking new business after regulators said it wasn’t doing enough to stop money laundering and terrorist financing.

The move by Tokyo-based bitFlyer Inc. led to a drop in the price of the cryptocurrency and highlighted how regulators in Japan, one of the hot spots of cryptocurrency trading, are shifting their stance toward tighter controls.

The...