SAN’A, Yemen—A militia group demanding a greater say over Yemen’s new constitution took over the presidential palace here Tuesday, sparking fresh concerns about a country that has become a cornerstone of U.S. counterterrorism strategy.

The offensive in San’a by the Houthis—a group that represents the country’s Zaidi sect, an offshoot of Shiite Islam—followed its capture hours earlier of the nearby headquarters of Yemen’s presidential guard. In September, it forced the government to resign after occupying the capital.

...