MOSCOW—Russia will set up a full-scale military unit on the annexed peninsula of Crimea in response to "rising foreign military presence" next to Russia, Defense Minister Sergei Shoigu said Tuesday, Interfax news agency reported.

One of his ministry's key tasks now is to "deploy a full-scale and self-sufficient force grouping" in the Crimea region, Mr. Shoigu said.

The statement comes a day after the North Atlantic Treaty Organization began military exercises with troops from the U.S. and other NATO members near the city of Lviv in western Ukraine. The exercises had been slated to take place earlier but were delayed to Sept. 15-26 because of the turmoil in eastern Ukraine. The U.S. and NATO accuse Russia of deploying tens of thousands of troops along its border with Ukraine.

Moscow initially denied that its troops accompanied the annexation of Crimea, but President Vladimir Putin later admitted that troops were deployed on the peninsula before a local referendum on secession from Ukraine in mid-March. Russia has repeatedly said it has no troops on the Ukrainian territory.

Write to Andrey Ostroukh at andrey.ostroukh@wsj.com