BERLIN—Germany said Tuesday it could reject Syrian migrants if other European countries are officially responsible for their asylum requests, a move that in practice will affect few people but illustrated the government’s mounting efforts to signal that its ability to take in refugees was reaching its limits.

German Chancellor Angela Merkel has been trying to balance her continued message of welcoming refugees with responding to growing pressure from local and state governments, as well as senior members of her own party,...