DETROIT—Wrecking crews are preparing to tear down a landmark 5,000-square-foot house in the posh neighborhood of Palmer Woods in the coming weeks, a sign that Detroit is finally getting serious about razing thousands of vacant and abandoned structures across the city.

In leveling 1860 Balmoral Drive, the boyhood home of one-time presidential candidate and former Massachusetts Gov. Mitt Romney, Detroit is losing a small piece of its history. But the project is part of a demolition effort that is just now gaining momentum and...