It's the election, stupid.

The sparring between Hillary Clinton and Donald Trump can only heat up, and some economists fear the election will worsen an already-high level of uncertainty that has caused damage to the economy and could ultimately rattle markets. "I think more people are getting more worried about what the presidential election will do," said Chris Rupkey, chief financial economist at MUFG. "I don't count that out as a major risk factor. It could make Brexit look like peanuts in terms of the financial market effect." Economists are forecasting U.S. growth of more than 2 percent in the second half, after the first half grew at a sluggish 1 percent pace. But those forecasts could be at risk if uncertainty around the U.S. election increases. Sixty percent of the participants in the recent CNBC Fed survey said the presidential election is a negative for the economic outlook.

If you look historically, investment expenditures respond very much to uncertainty. That's the thing that gets throttled back when uncertainty rises. Consumers don't seem to pay much attention to it, but business does. Nicholas Bloom professor, Stanford University

Citigroup's William Lee, head of North American economic research, believes GDP has already been impacted by uncertainty around the foreign issues like Brexit but even more so by the U.S. election, and it will continue to do so. "I think we're experiencing the drag right now, and it could get bigger," said Lee. He expects the political uncertainty to be hitting business investment and consumption, knocking an average of about a half percentage point off growth in each of the second, third and fourth quarters. Lee calculates the damage by hits to investment in software and intellectual property and investment in structures, as well as consumption. For instance, uncertainty may have carved nearly 0.4 percentage points off second-quarter GDP, and according to his calculation, half was from lower investment and half from consumption. Lee said he looks at the Economic Policy Uncertainty Index, created by three academicians. The index showed a spike around the Brexit vote, and the monthly data show it is elevated to a level close to where it was at when China devalued its currency last summer. The monthly version of the index is based on the top 10 U.S. newspapers' frequency of use of certain words, including economic and uncertainty and a number of words around policy, such as Federal Reserve. Nicholas Bloom, the Stanford University professor who helped create the index, said Trump is clearly the candidate who generates the most uncertainty.

Bloom explained that elections usually begin to create spikes in uncertainty in September, but that could be different this year and his uncertainty index could begin rising this month. If Trump appears to lead in the polls, the index could spike and stay elevated, he said. "If you look historically, investment expenditures respond very much to uncertainty. That's the thing that gets throttled back when uncertainty rises. Consumers don't seem to pay much attention to it, but business does," said Bloom.

Bloom said Trump is completely different than other candidates in that he has no political experience or policy team, and he breaks from the standard GOP party line. That makes uncertainty around his presidency "massive" and he expects the uncertainty index to be much higher than it is for most first time elections.

"He makes statements. He changes his mind. He has no track record. He has no advisers. He's a blank state right now," said Bloom. That means it could take months to understand his positions should Trump win, and uncertainty could be elevated through next summer as he sorts out positions and works out his relationship with Congress and foreign governments, Bloom said. "It will probably lead to a drop in investment by large firms and the sector that will be most impacted will be manufacturing because the president has the most influence on trade," Bloom said. He said military, health care and construction sectors could also be impacted. In the second quarter, GDP grew at 1.2 percent, on the back of consumer spending, which grew by 4.2 percent. Business spending was weak, and there was a 1.2 percent drag from inventories. Deutsche Bank chief economist Joseph LaVorgna agrees part of the weakness in business spending could already be due to apprehension about the election.

"Nobody's going to invest ahead of the election. Who is going to invest? They're going to wait to see what happens. It's not like the candidates are saying the same thing with the same tax policy. Certainly both parties have totally different ideas of how they're going to approach tax reform," said LaVorgna. LaVorgna said the hit to the economy could be a couple of tenths of a percent of growth, significant when considering how low the growth rate has been. Instead of investing, he said companies are doing other things with their cash. "Companies are doing what they've been doing much of the cycle — buying back stock and doing M&A [mergers]. They've been doing it in the equity market but it's really unproductive. It creates a more volatile environment because they're replacing equity with debt," he said. Lee also said Trump's policies are harder to determine and he therefore creates the conditions for an increase in uncertainty. "If it's a Clinton scenario, it's very close to the current baseline. I would say the uncertainty would be much lower," he said. Bloom said there is much less uncertainty around Clinton because she's already been in office, and her choice of running mate, Sen. Tim Kaine, is also known. "Trump is a greater wild card," Lee said.

"One caveat is we don't know what Trump is. … He seems to be someone who places policies out as trial balloons and then renegs on them, and you can't get a fix on where he is. The best spin his people put on it is that everything is a negotiating position. The actual outcome would be much more muted than those kind of outrageous comments would imply," Lee said. Trump's campaign did not immediately respond to a CNBC request for comment. Trump has proposed a revamping of the tax structure, in a revenue- neutral plan that would cut taxes for corporations to 15 percent. He would declare a one-time tax rate of 10 percent to encourage companies to bring home cash they are currently holding abroad.

He also proposes a more simplified tax plan for individuals that would have a top tax rate to 25 percent. Trump also blames U.S. trade policies for a wave of globalization that has ruined U.S. manufacturing, and he says he would bring an economic revival.