The path of the next three Federal Reserve rate hikes is clear but after that the central bank has to be more cautious about how it proceeds, Dallas Fed President Robert Kaplan told CNBC on Friday.

The Fed funds rate is currently at a target of 1.5 to 1.75 percent. On Wednesday, the Fed opted to hold rates steady, but it is widely expected to announce a hike in June.

"Once we get in the mid 2s, I think the going gets a lot tougher. We have to be very, very careful," Kaplan told Steve Liesman on "Closing Bell."

That's because he thinks economic growth can start to look sluggish in the future thanks to "demographics, sluggish productivity [and] high levels of debt-to-GDP."

He's also keeping his eye on the bond market.

The yield curve, which tracks the difference between longer- and shorter-term bond yields, is flattening, meaning they are getting closer together. Investors are concerned it may invert, which means long-term Treasury yields would be lower than short-term yields. Historically, when that happens, an economic recession has usually followed.

Kaplan said it would give him "great pause" if he was faced with a decision of raising rates and causing the yield curve to invert.

"I'm not going to predict what I will or won't do," he said. "Right now as I sit here, I don't want to knowingly invert the yield curve."

He believes the flatness in yields "says we're late cycle."

He said 2018 will be a good year, with a lot of stimulus. However, the economy will then start to grow a bit more slowly, he said.