WASHINGTON—The Trump administration’s top infrastructure aide is leaving the White House, officials said Tuesday, days after the president acknowledged that his long-delayed public works program wouldn’t happen before the midterm elections this fall.

DJ Gribbin, the special assistant to the president for infrastructure, worked for the National Economic Council and was the chief architect of a plan to transform President Donald Trump’s promises to rebuild the nation’s roads and bridges into a legislative package that could...