KABUL—The U.S.-led coalition in Afghanistan formally ended its combat mission on Sunday, leaving Afghan forces to battle a resilient Taliban insurgency largely on their own.

The International Security Assistance Force, or ISAF, is being replaced with a smaller force that will focus more narrowly on counterterrorism and training Afghan soldiers and policemen.

President...