MALE, Maldives—Around this Indian Ocean capital, Chinese loans are helping to pave paradise, stacking high-rises on artificial islands reclaimed from the ocean and tarring roads and a landmark bridge to connect the archipelago.

For the country’s almost two-week-old government , the infrastructure-building spree presents a problem. The country’s foreign loans and sovereign guarantees now total at least $2.65 billion, equal to 60% of gross domestic product, the new finance minister said. Chinese funds account for the largest...