WASHINGTON — President Trump on Tuesday withdrew the nomination of Patrick M. Shanahan to be the permanent defense secretary, leaving the Pentagon in transition at a time of escalating tensions with Iran and questions about the role of the military on the border with Mexico.

Mr. Shanahan, a former Boeing executive who had been serving as the acting defense secretary, announced his resignation as an F.B.I. background investigation, conducted on all cabinet nominees, was continuing because of incidents of family violence.

Mr. Shanahan’s ex-wife had accused him of punching her in the stomach, which Mr. Shanahan has denied. He said his ex-wife started the fight, and his spokesman said she was arrested and charged with domestic violence. The charges were eventually dropped.

Mr. Trump named Mark T. Esper, the secretary of the Army and a former Raytheon executive, to take over as acting secretary of defense. He did not say whether Mr. Esper would be nominated for the permanent position, but told reporters he was a “highly respected” graduate of West Point and Harvard and predicted that “he’ll do very well.”