Capping weeks of student protest about racial inequality on campus, Yale President Peter Salovey announced Tuesday that the university would be making significant changes to address “longstanding inequities” and make the campus more inclusive. The announcement marks a significant victory for the students, and for racial justice. But it is also a lesson in the power of free speech, and in the dangers of seeking its curtailment in the name of equality.

The Yale protests came to national attention when a short video of one particular confrontation went viral. In it a black female Yale student curses Professor Nicholas Christakis, a white residential college master, as he converses with a circle of students on racial tensions on campus. In the conversation, students speak of the need for “safe spaces” and demand an on-the-spot apology from the master for an email his wife, who is associate master, sent to students of their college, arguing for tolerance toward students who choose to wear “provocative” Halloween costumes. When the master says that “other people have rights, too,” the student responds, “Why the fuck did you accept the position? Who the fuck hired you? You should step down.” Moments later, she concludes, “you should not sleep at night… you are disgusting,” and walks away. Her words are unsuited to civil discourse, to say the least. But the overall impression is not so much that she is rude as that she is suffering. (So, too, I would add, is the master.)

Critics seized on the one-minute-twenty-second video, condemning the students for their intolerance and unreasonable demands. But because it captures only a single inflammatory exchange, the video has distorted perceptions about the controversy—at Yale and elsewhere. For some time, Yale students of color have maintained that the school does not sufficiently welcome them. As of the fall of 2013, only 2.9 percent of Yale faculty were Latino and only 3.5 percent were black. Students of color are more likely than white students to be stopped by campus police, mistaken for service staff, and stereotyped and slighted by students and faculty alike. Nor are the Yale students alone in airing these grievances. Students at Harvard, the University of Missouri, and countless other campuses have voiced similar concerns. Racial bias, these students remind us, is not limited to police encounters in high-crime, inner-city neighborhoods, but permeates American life, including in the hallowed halls of our nation’s best universities.

The fact that so many of the recent controversies have arisen on college campuses is no accident. Because universities can select their student bodies and are committed to diversity, they are some of the most integrated sites in an otherwise remarkably segregated nation. Despite the myth of the “melting pot,” most of us live, work, worship, and socialize in segregated circles. But because it can select its participants, the academy can be an experiment in integration, where students from a remarkable range of backgrounds and experiences study and live together. Elite institutions are not as integrated as they should be, of course, and remain disproportionately the preserve of the privileged. But universities generally are a place for racial and ethnic mixing; Yale’s student body, for example, is 42 percent of minority descent. Integration is not without friction, however, and students of color have for too long borne the brunt of the friction on college campuses, as in the rest of their lives.

The immediate cause of the conversation in the video was an email Erika Christakis sent on October 30 to the students of Silliman College, where she is associate master, questioning whether an earlier official Yale email urging students to avoid offensive Halloween costumes was too heavy-handed. The initial administration email, itself prompted by an incident several years earlier in which some students had gone out in blackface, was not, in fact, especially heavy-handed. The administration email acknowledged that students “definitely have a right to express themselves,” and merely expressed “hope that people would actively avoid those circumstances that threaten our sense of community or disrespects, alienates or ridicules segments of our population based on race, nationality, religious belief or gender expression.” Ms. Christakis’s email, prompted by students who had themselves objected to the administration’s message, espouses the values of free expression and expresses concern that colleges are becoming places of “censure and prohibition.” “Is there no room anymore for a child or young person to be a little bit obnoxious… a little bit inappropriate or provocative or, yes, offensive?” she asks.

As an online letter from students to Ms. Christakis explains, many viewed her letter, in context, as a tone-deaf invitation to wear racist costumes. And in a list of demands to the university, students asked that the Christakises be dismissed from their positions as master and associate master of Silliman College.

In part because of the video, the Yale controversy has been portrayed as pitting free speech against racial equality. But that diagnosis very much misses the broader picture. Most of what has transpired at Yale and other colleges reflects the best traditions of the First Amendment: students of color and others have been organizing politically and speaking out in packed rallies. They are using the First Amendment to stand up, communicate their experiences, and demand equal justice. That’s exactly how the First Amendment should work.

And what they are fighting for is critically important, indeed necessary: an inclusive community that treats them as equals. The students have focused in particular on faculty hiring. A school with so few black and Latino faculty sends a powerful even if unintended message to its students of color; they may be good enough to attend, but not to teach here. It also denies minority students the opportunities for mentorship that their white fellow students take for granted. (Of course, faculty can and do mentor across races, but anyone who has spent time in a school of any type knows the critical importance of students of color having teachers that share their experiences in our still-racially-divided world.) According to a poster on campus, Yale’s African-American faculty has grown by 1 percent each century since its founding (reaching just above 3 percent today). Twenty-five years ago, Yale Law School had three black faculty members; today, it still has only three. In over two hundred years, the law school has hired only one black female professor, one Latina female professor, and no Latino male professors. (The law school faculty, however, does include two East Asian professors, four South Asians, and one Arab American.)

The students also seek more resources for the school’s four cultural centers (Afro-American, La Casa, Native American, and Asian American). These centers offer a haven for students who feel disrespected, misunderstood, or harmed as they navigate life at an institution built on white privilege. They can be a site for the kind of collective action that is apparently necessary to push the administration to address minority students’ problems. And their very existence says to the community at large that the school values the experiences of all who attend the school, not just of those who fit comfortably into Yale’s historical pedigree.

To a degree, the students’ protests are being heard. Some 400 professors signed a letter in support of the students for challenging “institutionalized inequalities” on campus, and last week, the university announced a multimillion dollar effort to increase the diversity of the faculty—an initiative that, even if almost certainly insufficient of itself, would not exist but for the protests. And in his Tuesday statement, Yale President Salovey issued a statement announcing four initiatives designed to “make significant changes so that all members of our community truly feel welcome and can participate equally in the activities of the university.” They include creating an interdisciplinary center for the study of race, ethnicity, and identity; hiring additional faculty to teach in these areas; doubling funding for the university’s cultural centers; training of staff in detecting and countering racial and ethnic bias; and improved financial aid. These are important victories for the protesting students and for the student body as a whole.

So instead of condemning Yale students for trampling on free speech, we should commend them for using their speech rights to push the institution to become more inclusive and welcoming for all.

But President Salovey did not dismiss the Christakises from their residential college positions. That was the right call. In fact, in demanding their dismissal, the students turned their back on the very free speech principles that they otherwise used to their—and the entire school’s—advantage. Ms. Christakis’s email casts no aspersions on any minority group. Nor does it recommend the donning of racist costumes. It is a plea for free expression, and a reminder that commitment to free expression requires tolerance of speech that offends. As she put it, “Free speech and the ability to tolerate offence are the hallmarks of a free and open society.”

The concerns of those who found Ms. Christakis’s letter offensive in the context of recent racial tensions are understandable. But there is no evidence that any offense was intended. Moreover, the fact that speech offends is not a reason to punish the speaker. As the Supreme Court said in 2011 in overturning a jury verdict against the Westboro Baptist Church for offensive and homophobic protests near the funeral of an American soldier,

Speech is powerful. It can stir people to action, move them to tears of both joy and sorrow, and—as it did here—inflict great pain. On the facts before us, we cannot react to that pain by punishing the speaker. As a Nation we have chosen a different course—to protect even hurtful speech on public issues to ensure that we do not stifle public debate.

The First Amendment does not apply to a private institution like Yale University. But the same principles of free expression animate the doctrine of academic freedom. And while that entails requiring a certain level of decorum in classroom discussions, in order to make possible civil discourse among people who hold very different beliefs, it does not countenance punishing a professor (or student) for sending a public email questioning administration policy, much less for espousing bedrock principles of free expression. That Ms. Christakis was an associate master of the college, in charge of a residential community, does not diminish her right to express such opinions, and punishing the Christakises by removing them from their positions would send an unacceptable message of intolerance.

It is also a mistake to seek to suppress speech in the name of equality. Free speech and association are tools for the minority, whoever they are at a given moment—as the Yale students themselves have admirably demonstrated. The First Amendment empowers them to express their views, to dissent from majority policies, and to organize politically to advance their interests, just as, before them, it protected Martin Luther King Jr., Malcolm X, and other civil rights activists. The last thing a minority group should seek is the suppression of free expression.

Focusing on offensive speech also distracts from the more significant issues of racial injustice that persist more than sixty years after Brown v. Board of Education declared segregation unconstitutional—and that remain the Yale students’ principal concerns. African-Americans are disproportionately the victims of violence, both from the police and from their fellow citizens. They have far fewer economic and educational opportunities, and virtually the only American institutions in which they are over– rather than under-represented are prisons and the military. They have less wealth and shorter life expectancy than whites. And countless studies have shown that they are the victims of the implicit, often unconscious biases of doctors, employers, teachers, police, and probably everyone else they encounter. These are the pressing racial problems of our time—not Erika Christakis’s email. As media reactions illustrate there is a real risk that by going after the Christakises the students’ very legitimate complaints about much more serious problems will be drowned out.

Yale students are right to complain that their critics have failed to look beyond the viral video. If we want to understand the controversy at Yale, or at any of the many colleges that are experiencing similar protests, we must take seriously the deep and lasting wounds that continue to afflict the African-American community. We must demand, with the students, more diversity in faculty and staff, greater resources for minority students, and greater sensitivity to the challenges of building an integrated community of mutual respect. If President Salovey’s promises of significant change are realized, the students will have won—for the good of the whole university. But the struggle is not over. Responding to the challenges of diversity in a racially divided world is a full-time job. And continued activism will be needed to keep the administration to its promises. Demands to punish Erika Christakis because her genuine expression of opinion was deemed offensive undermine the cause. The students would do well to abandon that request and focus their and our attention on the more systemic problems of equal justice that continue to plague Yale, and the nation.