ST. PETERSBURG, Russia—When Russia welcomed a Libyan warlord aboard its aircraft carrier last year, it looked like the Kremlin was throwing its weight behind a rival to the United Nations-backed government in the North African country.

But by that time a Russian businessman was already one year along on a quieter Kremlin-backed mission to court the official administration in Tripoli.

The...