Editor's note: Julian Zelizer is a professor of history and public affairs at Princeton University. He is the author of "Jimmy Carter" and "Governing America." This January, Penguin Press will publish his new book, "The Fierce Urgency of Now." The opinions expressed in this commentary are solely those of the author.

(CNN) -- Former Tennessee Sen. Howard Baker passed away recently. Although he was known for many things, Baker's most enduring moment came in the middle of the Watergate scandal, when he asked: "What did the president know and when did he know it?"

The scandal happened 40 years ago. It started with a break-in at the Democratic headquarters in Washington, D.C. and it was followed with subsequent efforts to obstruct an investigation into whether the White House had been involved.

In July 1974, the Supreme Court ruled 8-0 that the White House had to turn over recorded presidential conversations to the investigators. The House Judiciary Committee voted in favor of the three articles of impeachment, charging President Richard Nixon with obstruction of justice.

The Watergate scandal had a devastating effect on American politics. In his riveting forthcoming book, "The Invisible Bridge," Rick Perlstein skillfully recounts the era that was shaped by the scandal and the way in which the sordid activities of the Nixon administration unfolded on a day-by-day basis.

Each revelation gave voters another reason not to trust their elected officials and to believe the worst arguments that people made about government. Americans could never look at government the same way again.

The scandal continues to reverberate today throughout the political spectrum. We still live in the era of Watergate.

For Democrats, who many thought would have been the beneficiaries of a scandal that brought down a Republican president, the level of distrust that the scandal generated among the public has been an ongoing challenge.

At the most basic level, Democrats argue that the federal government offers the best solution to the problems of the day. But if the public does not trust its elected officials, Democrats are left in a position of having to constantly defend the legitimacy of the institutions of government and to convince voters that bureaucrats really will do their job.

The intense skepticism surrounding the Affordable Care Act, Benghazi and the Internal Revenue Service scandal have revealed how easy it is for opponents of government to stoke these kinds of fears.

Republicans have suffered too, even after the party separated itself from Nixon as its figurehead. The truth is that Republicans promote government as well, just for different reasons. Their programs have, likewise, been subject to constant scrutiny as a result of the lingering distrust from Watergate.

For conservatives, national security programs have been a centerpiece of their agenda. Republicans have pushed for expanding the military budget and since 9/11 many have called for an aggressive response to terrorism that includes sweeping surveillance programs and enhanced interrogation techniques.

Revelations about what government officials do without public accountability -- such as torture or snooping into e-mails -- have deepened public distrust and created strong pushbacks.

Politicians in both parties must operate in a political environment filled with investigations, or accusations of another scandal looming with the suffix "gate" attached to it. Whenever some kind of scandal breaks, it doesn't take long for the story to escalate and for questions to arise as to whether this will end up as big as Watergate.

Often, this outlook has salutary effects by encouraging politicians to make sure that similar levels of corruption don't happen again.

But, too often, as many would say has been the case with the IRS, stories of administrative mismanagement are blown out of proportion, consuming Washington's time and taking their attention away from major problems.

The worst effect of Watergate is that it created a climate where Americans fundamentally don't trust their government. It is one thing to be suspicious, another to reject altogether. Recent approval ratings for Congress tanked to 7% and for the President 29%. This is part of the broader trend we have seen since the 1960s.

It is extremely difficult for government to do its job or for voters to have the kind of faith in government, which is necessary for a healthy society.

When Howard Baker asked his famous question, his hope was not to disparage government but to make it better. He wanted to find the corruption, to seek the reform so that government could do its job once again. Unfortunately, the kind of faith that Baker had in government never returned.

To really banish the memories of Watergate and set the government on a better course, reforming politics is the most important solution.

Improving our campaign finance system by curbing the influence of private money and imposing stronger restrictions on lobbying, such as the revolving door between the government and lobbying groups, is an essential first start.

Until we take those kinds of steps, voters will always be seeing the shadow of Richard Nixon when they look at their elected leaders.

Read CNNOpinion's new Flipboard magazine.

Follow us on Twitter @CNNOpinion.

Join us on Facebook.com/CNNOpinion.