Of all the charges and counter-charges that have been aired so far in the reality show known as “America’s Next Fed Chairman,” none is more explosive than the charge of sexism. Between the two apparent finalists for the job, the woman—current Fed vice chairman Janet Yellen—is viewed by many as more qualified, while the man—former Treasury Secretary and White House economic adviser Larry Summers—has an unfortunate gender scandal on his resume. This has led to a variety of gender-related accusations, from the suggestion that the case for Summers, reportedly the president’s first choice, is based on subtle (or not so subtle) prejudice, to the claim that Summers is simply unqualified to be Fed chairman because of his questionable pronouncements on women while president of Harvard.

Then there is the argument, probably the most compelling to my mind, that Summers benefits less from overt discrimination than a kind of subtle, institutionalized bias. Under this scenario, no one with influence over the decision need be personally hostile to women—and, for the record, I don’t believe any of them are. They simply happen to favor candidates who belong to a small, insular group of people that they know well, and very few women happen to belong to that group. As a variety of commentators have pointed out, this is basically how gender discrimination works in this day and age. (Unless you happen to be in the Texas state legislature, in which case God help you…) It will probably shock no one that more than a few media outlets could be accused of similar forms of bias.

Naturally, the administration, and Summers’s supporters inside it, are resentful of these allegations, insisting that their preference for Summers over Yellen is purely on the merits, and that both the president and Summers have a history of promoting women around them. (True enough.) As for the male-dominated clique charge, Teams Obama and Summers point out that the current director of the Office of Management and Budget, Sylvia Mathews Burwell, is a woman. Despite the sheer number of men the president has installed in top economic posts, the thinking goes, Burwell proves he’s clearly open to a naming a woman.

Unfortunately, I don’t think Burwell is quite the trump card the White House and Summers’s allies believe it to be. In fact, it shows close to the opposite of what they intend.

The problem is that Burwell isn’t some highly qualified woman that the White House turned up after an exhaustive, nationwide search (though she is highly qualified). She, like Summers and most of Summers’s boosters in and around the White House, is a former protégé of Bob Rubin. She worked for Rubin when he ran the National Economic Council under Bill Clinton in the early 1990s. Then she was his chief of staff when he became Treasury Secretary in 1995. Burwell almost certainly owes her job as OMB director to her membership in the same club that’s trying to make Summers Fed chairman.