Judge Brett Kavanaugh arrives at a meeting with Nevada Sen. Dean Heller on Capitol Hill on July 18. Alex Wong/Getty Images

Judge Brett Kavanaugh’s nomination to the Supreme Court triggered immediate fear among most liberals and progressives. There’s the likelihood that Kavanaugh would supply the fifth vote to overturn or dismantle Roe v. Wade. There are his very strong pro-business and anti-labor rulings. And of course, there’s Kavanaugh’s extensive writings critical of independent criminal investigations of the president and the fact that he’s been appointed to office by a president under criminal investigation. There’s a reason, though, for conservatives to also have anxiety about the Kavanaugh nomination: The rare times in our history that the court has gone as far outside the ideological mainstream as Kavanaugh may well take the court have resulted in major public backlash and even constitutional crisis.

In The Will of the People: How Public Opinion Has Influenced the Supreme Court and Shaped the Meaning of the Constitution, professor Barry Friedman persuasively detailed how the court generally stays within right-of-center and left-of-center politics. The justices know, in the words of Alexander Hamilton, that they have neither the purse nor the sword, only the confidence of the American people in its decisions and of the executive branch to enforce those decisions.

Ultimately, our country is much better off with a Supreme Court that “follows the election returns,” not one that distorts those returns.

There have been times, however, when the justices have tried to do too much too soon or moved away from the center of American politics.

In the infamous Dred Scott decision, the court ruled that black people were never and could never be American citizens and that Congress could not abolish slavery in the territories. The majority actually thought its decision would bring North and South closer together. Most historians now agree, though, that the decision moved us much closer to the Civil War.

From 1900 to 1936, a Supreme Court composed of men who came of age in a preindustrial, rural economy struck down many progressive laws, including statutes regulating minimum wages, overtime, and workplace safety conditions. After the Great Depression, President Franklin D. Roosevelt, angry at the justices for impeding his New Deal program, threatened to pack the court. Eventually the court backed off, and the justices have not used the logic of this Lochner era court—whose dominant conservative block eventually became known as the Four Horsemen of the Apocalypse—since then.

Today, many conservatives and libertarians argue that the court went too far in accepting the validity of economic legislation and argue for a new “judicial engagement.” But there can be no debate that the court’s almost complete deference in this area of constitutional law for decades was a backlash to the court’s hyperaggression for the first 30-plus years of the 20th century.

The third example is the most recent. Regardless of how one feels about Roe v. Wade, the court’s entry into abortion politics has substantially affected not just constitutional law and interpretation but also local, state, and federal politics; judicial nominations; and quite possibly presidential elections. In the words of perhaps the most important proponent of equal rights for women over the past half-century, Justice Ruth Bader Ginsburg, Roe was too much too fast. She has criticized the decision for trying to resolve the abortion issue in “one fell swoop.”

This history brings us to President Donald Trump’s nomination of Kavanaugh. This nomination is quite different from the previous one, when Neil Gorsuch was nominated by Trump to replace the late Justice Antonin Scalia; Gorsuch’s ascension did not change the balance of politics on the court because both were solidly conservative justices. Kavanaugh, however, appears to be much more conservative than his former boss Justice Anthony Kennedy, whose seat he would fill if confirmed. With Kavanaugh, the court would be solidly conservative probably for generations. This would likely lead to a liberal backlash both in our politics and in our courts that may well make the Warren Court look moderate.

Ultimately, our country is much better off with a Supreme Court that “follows the election returns,” not one that distorts those returns. When the court strays too far from majority opinion, there is almost always a strong backlash or, worse, a constitutional crisis. Whatever you think of the abortion and gay rights rulings themselves, Mitch McConnell’s decision to deny President Barack Obama’s nominee Merrick Garland even a hearing could be viewed as a direct response to those decisions. It is not likely McConnell could have pulled off the stealing of a Supreme Court seat without the controversies around those cases.

Now, a Supreme Court with five conservative Republicans (Roberts, Alito, Thomas, Gorsuch, and Kavanaugh) will likely enter a period of conservative judicial aggression. This past term, when Justice Kennedy turned far right for the first time in decades, the justices struck down four important state laws. These cases demonstrated the court’s political nature, with the losers being unions, gay and lesbian people, and women seeking abortions.

A new moderate justice might temper the court’s aggression, but Kavanaugh is no moderate, as many have observed. Although the GOP and political conservatives might think a far-right court will serve their ends, the reality is that there will inevitably be either a strong backlash or a new constitutional crisis, which will damage both Republican politics and conservative causes.

Although politics is almost always a short-term business, GOP senators would be wise to look at the bigger picture, learn from history, and try to staff a court that will be right of center but not too far right of center.