Jonathan Kitchen / Getty Images

Today is a holiday that no one is interested in celebrating. No, not Tax Day; some people are proud to pay their taxes. I’m talking about Equal Pay Day, a sort of anti-holiday invented by the National Committee on Pay Equity to mark the sad fact that women and men still don’t earn equal pay for equal work. According to the American Association of University Women, women working full-time jobs still only earn 77 cents for every dollar earned by men with full-time jobs, a 23% gap. Today, April 17, marks how far into 2012 women would have to work in order to match what men earned in 2011.

We’re not talking about pocket change. In 2010, the last year for which we have data, American men working full time took in roughly $47,700; women earned only $36,900, a gap of $10,800. That’s almost enough to cover average housing costs for an individual over the course of a year. (Or one gigantic shopping spree at the dollar store.) Over the course of a career, the typical woman earns a staggering $434,000 less than her male counterpart.

(MORE: Women Are Overtaking Men as Breadwinners)

But what if this giant pay gap is the result not of discrimination, but of something less pernicious – like the different life and career choices made by men and women? When you look more deeply into the numbers, critics suggest, the alleged wage gap simply disappears. On the conservative FrontPageMag.com, for example, John Perazzo argues that the wage gap is “a complete fiction. A gargantuan lie, actually.”

Women, he argues, are more likely to work in non-technical fields, to avoid dangerous jobs, to seek jobs offering more flexible hours.

An even more significant cause of the gender pay gap is that women tend to compile fewer years of uninterrupted service in their jobs than men. Indeed, women are far more likely to leave the workforce for extended periods in order to attend to family-related matters such as raising children.

As a result of these sorts of differences, Perazzo argues,

when men and women work at jobs where their titles, their responsibilities, their qualifications, and their experience are equivalent, they are paid exactly the same.

Well, not exactly.

Perazzo is right that different “life choices” do account for some of the wage gap. But not all of it. According to a 2003 report by the US Government Accountability Office, even after accounting for different work patterns between men and women, and other such factors, there’s still a 20% wage gap. A 2009 report prepared for the Department of Labor by the CONSAD Research Corp suggests that the gap is considerably smaller. According to CONSAD, “observable differences in the attributes of men and women … account for most of the wage gap.” The real gap, the report argues, is only between 4.8 and 7.1%.

So should we really be “celebrating” Equal Pay Day sometime in January? Well, no, because the different “work patterns” and “attributes” of men and women explain less of the differences in pay than the critics claim. Take the issue of motherhood. As Perazzo argues, motherhood does indeed have a large effect on women’s wages: while childless women make 94% of what childless men make, mothers make only 60% of what fathers make. (Men, by contrast, get an economic boost from fatherhood.) Indeed, economist Jane Waldfogel argues that motherhood may account for up to half of the wage gap between men and women.

But career interruptions only account for some of the “motherhood penalty.” As sociologist Michelle Budig points out,

Having children reduces women’s earnings, even among workers with comparable qualifications, experience, work hours, and jobs … That mothers work less and may accept lower earnings for more family-friendly jobs partially explains the penalty among low-wage workers, and that mothers have less experience, due to interruptions for childbearing, explains some of the penalty among the highly paid. But a significant motherhood penalty persists even in estimates that account for these differences: the size of the wage penalty after all factors are controlled is roughly three percent per child, which, in 2009, means the typical full-time female worker earned $1,100 less per child.

Some of the wage gap, as critics like Perazzo point out, is due to the fact that women tend to work in fields that pay less than fields dominated by men. But when women enter these higher paying fields, they still tend to earn less than their male counterparts. The gender wage gaps in heavily male dominated science and technology jobs, for example, range from 81% in civil engineering to a relatively egalitarian 94% in chemistry, according to government data recently compiled by the AAUW.

(MORE: Are We Paying Our CEOs Enough?)

Moreover, talking about the differences in the career paths of men and women as the result of free “choice” can be misleading. As psychologist Hilary M. Lips puts it,

The language attributing women’s lower pay to their own lifestyle choices is seductive—in an era when women are widely believed to have overcome the most serious forms of discrimination and in a society in which we are fond of emphasizing individual responsibility for life outcomes. … [But] the language of “choice” obscures larger social forces that maintain the wage gap and the very real constraints under which women labor. The impact of discrimination, far from being limited to the portion of the wage gap that cannot be accounted for by women’s choices, is actually deeply embedded in and constrains these choices.

One reason women may be less likely to enter tech fields, for example, is that they have been taught their whole lives that women aren’t good at that sort of thing. “From childhood onward,” Lips writes,

we view media that consistently portray men more often than women in professional occupations and in masculine-stereotyped jobs. Not surprisingly, researchers find that the more TV children watch, the more accepting they are of occupational gender stereotypes. Why does the acceptance of gender stereotypes matter? Gender-stereotyped messages about particular skills (e.g., “males are generally better at this than females”) lower women’s beliefs in their competence—even when they perform at exactly the same level as their male counterparts. In such situations, women’s lower confidence in their abilities translates into a reluctance to pursue career paths that require such abilities.

It doesn’t help, of course, when the relatively small number of women who take engineering courses at the college level are treated like exotic creatures, sex objects or ersatz men.

You may recall the infamous talking Barbie doll that told young girls that “math class is tough!”

Well, inequality is tough. Not just tough to fight; tough to understand. And to really understand it, you need to go beyond the numbers.

(MORE: From Hollywood Actress to Toy Store CEO)