Washington (CNN) As the Senate reaches the brink over the nomination of Judge Neil Gorsuch, the real impact of its actions will play out across the street at the Supreme Court.

The Democratic filibuster and GOP plans to implement the so-called "nuclear option" to cut it short could bring the already politicized confirmation process closer to the edge where the public begins to think of the Judicial Branch in the same political terms as the White House and Congress, experts say.

That politicization is something several justices have anticipated, and dreaded.

"I wish there was a way I could wave a magic wand and put it back when people were respectful of each other and the Congress was working for the good of the country, and not just along party lines," Justice Ruth Bader Ginsburg said last year at an event organized by Stanford's Office for Religious Life, as part of the Rathbun Visiting Fellow program.

The concern is not that the court will suddenly break down in its closed door conferences and engage in some kind of fiery partisan fight or change its rules to trigger a judicial version of a nuclear option.

Instead, it's all about perception. And for the justices, perception is not a small thing.

"When you have a sharply political, divisive hearing process, it increases the danger that whoever comes out of it would be viewed in those terms," Chief Justice John Roberts said during a talk at the New England School of Law in Boston in February 2016.

"And that's just not how -- we don't work as Democrats or Republicans. I think it's a very unfortunate perception the public might get from the confirmation process," he added.

Roberts' comments came just before the sudden death of Justice Antonin Scalia, whose seat has sat vacant for more than a year and will likely be filled by Gorsuch at the end of the week.

Ginsburg, nominated by President Bill Clinton, reminded the audience that she was confirmed 96-3 despite the fact that she was a former lawyer for the ACLU. One of her greatest supporters at the time? Utah Republican Sen. Orrin Hatch.

She joked that in the current climate, "he wouldn't touch me with a 10 foot pole," although she quickly added, he is still a friend. She noted that the four more recent justices, nominated by Republican and Democratic presidents, hadn't moved through as easily. The numbers tell the story: Chief Justice John Roberts' vote was 78-22, Samuel Alito 58-42, Sonia Sotomayor 68-31, and Elena Kagan, 63-37.

There's no question the high court has become more and more of a political football over the past 20 years. Base voters on each side are motivated by the prospect of getting conservative or liberal justices confirmed. And the litany of controversial issues making it to the court has continued, especially given the polarization and gridlock on Capitol Hill.

"Treating judges like political candidates or focusing on the outcome of decisions instead of the legal reasoning feeds the perception that all the court does is politics," said Jonathan H. Adler of Case Western Reserve University School of Law.

"The legitimacy of the court is based upon the belief that justices engage in a good faith effort to reach the outcome that the law requires," Adler said. "How the court is perceived affects whether the public accepts the courts' decisions as legitimate."

Nuclear option

Senate Democrats, furious that President Barack Obama's nominee, Merrick Garland, never got a vote, are planning to filibuster Gorsuch in hopes that the Republicans will meet with them and propose a more moderate nominee. Republicans respond that they are being left with no choice but to change the way filibusters work for Supreme Court picks -- the so-called "nuclear option" that cuts the number of lawmakers needed to break a filibuster from 60 to 51.

Republicans control the Senate, 52-48, meaning they have the votes to confirm Gorsuch by Friday, as Majority Leader Mitch McConnell, R-Kentucky, has said it would do.

For the court, eliminating the 60-vote filibuster could mean presidents are under less pressure to pick nominees that would in theory appeal to both parities.

"Absent the filibuster presidents will have much less incentive to find moderate, compromise nominees," said Adam Winkler, professor of Law at the UCLA School of Law. "If Trump is able to confirm extreme conservatives, and future Democratic presidents extreme liberals, there will be more disagreement on an already divided Court," he said.

Winkler believes it could also discourage some justices from retiring, fearful of who their replacement might be.

Republican Sen. Lindsey Graham sees other reverberations.

Every Senate seat "is going to become a referendum on the Supreme Court," he said.

GOP Sen. Bob Corker said he thinks that if the Senate triggers the nuclear option it could embolden interest groups.

"Do you think that with a Democratic president, the Democratic left base isn't going to drive that president to have a more extreme person?" Corker asked. "So what we're going to end up with, if this occurs, is presidents will be under no constraint when their party is in the majority so the types of nominees we're going to end up having are going to be far more extreme."

Then again, maybe the nuclear option won't be the end of the court as we know it.

Some day, Ginsburg said last year, "there will be great people, great elected representatives who will say 'enough of this nonsense, let's be the kind of legislature the United States should have.'

"I hope that day will come while I'm still alive."