Powerful voices on Wall Street are expressing concern that President Donald Trump isn't going to be everything that investors had hoped for.

In recent days, three of the nation's largest banks have issued reports that highlight the risks that come with a Trump presidency. They are warning investors to be careful.

U.S. stocks soared right after Trump's election because investors were eager for the corporate tax reform, regulatory relief and infrastructure spending he promised.

Instead, in his first few weeks of office, Trump has highlighted several controversial issues that businesses generally oppose, such as dropping out of trade deals and imposing new limits on immigration. And it may be hard to win approval for the parts of Trump's agenda that Wall Street does like amid stark political polarization.

Related: Will Trump kill the Trump rally?

"The Trump Agenda presents risks in both directions; tax cuts and infrastructure funding could boost growth but could be offset by the negative effects of restrictions on trade and immigration," said a note from Goldman Sach (GS)'s economists. "One month into the year, the balance of risks is somewhat less positive in our view."

"The recent difficulty congressional Republicans have had in moving forward on Obamacare repeal does not bode well for reaching a quick agreement on tax reform or infrastructure funding, and reinforces our view that a fiscal boost, if it happens, is mostly a 2018 story," it added.

A separate Goldman note on Monday said that it now forecasts the S&P to rise to 2,400 by the end of March, but fade to 2,300 by the end of this year, near where it now stands.

Also this week, Wells Fargo (WFC) pointed to the problems posed by the protests and political unrest that has marked the early weeks of the Trump administration.

"Controversial policies and geopolitical events could impede the progress of U.S. and global economic recoveries," it said in a note on Monday. "The potential for disruption is ever-present. The possibility of a near-term pullback in equity and commodity markets has us more conservatively positioned in most portfolios than in recent quarters."

Wells Fargo also expects markets to end 2017 about where they are today.

Related: Trump's isolationism is bad for business

A Bank of America (BAC)note out Monday suggests that the relatively low U.S. unemployment rate will make it difficult for the economy to kick into a higher gear, as predicted by Trump. The report added that the president's plans for spending on infrastructure may simply lead to higher wage inflation and a stronger dollar, which in turn could make it harder for U.S. goods to compete overseas.

"The risks of a recession rise as an economy approaches full employment and begins to overheat," it said.

Of course Wall Street analysts have been wrong about the market and Trump before.

Before the election, many analysts predicted a decline for U.S. stocks if Trump won. Citi analysts specifically forecast an immediate 3% to 5% drop in U.S. stocks. Macroeconomic Advisers predicted an 8% drop.

But when he did win, U.S. stocks took off on a "Trump rally." The S&P climbed about 7% between election day and his Jan. 20 inauguration. The Dow climbed nearly 10% between the election and Jan. 25 to reach 20,000 for the first time in history.

But that rally has essentially stalled in the weeks since then.