Under questioning, President Obama defended the diversity of his cabinet at a press conference on Monday, saying critics should wait for him to name his full second-term team before rushing to judgment. “I’m very proud that in the first four years we had as diverse if not more diverse a White House and a cabinet than any in history and I intend to continue that,” he said.

The president’s comments followed a New York Times article last week which featured a photo showing a large number of male advisers in the Oval Office without a single female visible (Obama adviser Valerie Jarrett was in the room, but obscured by Communications Director Dan Pfeiffer). Critics have noted that the three big second-term cabinet appointments–Sen. John Kerry (D-MA) as Secretary of State, former Nebraska Sen. Chuck Hagel as Secretary of Defense and Jack Lew as Secretary of the Treasury–have all been male. Over the weekend, Obama was criticized by New York Times Op-Ed columnists Maureen Dowd and Frank Bruni, while NBC News Chief Foreign Affairs Correspondent Andrea Mitchell related on Meet The Press how sources inside the White House said “the women are not happy.”

“If you think about my first four years,” the president said on Monday, “the person who probably had the most influence on my foreign policy was a woman, the people who were in charge of moving forward my most important domestic initiative–health care–were women,” he said. “The person in charge of our homeland security was a woman. My two appointments to the Supreme Court were women and 50% of my White House staff were women.”

The president said he expected to build upon that record over the next four years. “Until you’ve seen what my overall team looks like, it’s premature to assume that somehow we’re going backwards,” he said. “We’re not going backwards, we’re going forward.”