OFF THE LIBYAN COAST—Adrift on an inflatable raft in the Mediterranean Sea, 45 African migrants used a satellite phone to send an SOS message, hoping for a rescue ship to take them to Italy.

Instead, a Libyan Coast Guard vessel arrived with first aid kits, life preservers and an AK-47 to force the thirsty, saltwater-caked migrants aboard and return them to a war zone in Libya.

A...