Jose Luis Pelaez | Getty Images

For financial advisors, client trust and their investment track record are either your best attributes, or a couple of brewing problems. Those two factors are the main reasons investors say they choose a particular advisor. They also are the top two areas that cause 71 percent of clients to feel disappointed by their advisors, according to new research. "It makes sense that if those are the things that are most important to people, those are things that also will be what they feel most let down about," said Mike Maughan, head of global insights at Qualtrics, which recently released its Financial Customer Experience Report. The study found that wealth advisory clients say trust (47 percent) and investment track record (42 percent) are the main reasons they chose their advisor. Those areas were also the top reasons for feeling like their expectations were not met: Investment track record was the biggest reason for letdown (22 percent), followed by trust (16 percent).

Nevertheless, the majority of clients say they are happy with their advisors, and aren't planning to leave: 43 percent describe themselves as "very happy," and 42 percent, "happy." Another 13 percent are indifferent and say they could be just as satisfied elsewhere. Only 1 percent were unhappy and planning to bolt. While low fees ranked fifth (21 percent) on the list reasons for choosing an advisor, high fees were the top-cited reason for walking out the door (14 percent), followed by poor customer service (10 percent.) In other words, if the client's level of trust is high when they pick the advisor, fees are less important. "It's not until they feel they've been treated unfairly that they leave," Maughan said. Investment fees have become a bigger issue as investors become more tuned in to how much expenses eat into their profits. A 2010 Morningstar study showed that lower-cost mutual funds consistently outperform their more expensive counterparts. At the same time, the growing popularity of lower-cost index funds and exchange-traded funds, coupled with the emergence of online robo-advisors, has pushed fees down across the board. In 2016, the average expense ratio across all funds was 0.57 percent, the lowest it's ever been, according to Morningstar research.

It's not until [clients] feel they've been treated unfairly that they leave. Mike Maughan Head of global insights at Qualtrics

For clients who leave due to fees, Maughan said, it loops back to trust that the advisor is doing right by them. "We've all become hyper-sensitive to fees, especially if we think they are hidden fees," he said. "It goes back to the innate desire to be treated fairly." Part of the problem is simply a gap in expectations and reality. While advisors need to be transparent and up front about investment strategies, performance and fees, clients also need to speak up when they feel let down, regardless of the source of their frustration. "If people feel disappointed in how they've been treated, the number one thing to do is give feedback to the advisor," Maughan said. "Too many people fail to do that."

You also need to manage your own expectations as an investor. If you think an advisor has a silver bullet to outperform the market consistently, you eventually will end up disappointed. While stocks are charging ahead in their ninth year of a bull market and profits in investment portfolios reflect those gains, market performance is cyclical. This means that at some point, stocks will head south. Since bottoming out more than eight years ago in the wake of the financial crisis, the Standard and Poor's 500 Index has surged 288 percent through the end of November, to 2,626.07 from 676.53 on March 9, 2009. While no one is calling for a market nosedive any time soon, pullbacks are largely unpredictable. A bear market, even more so. Depending on a client's portfolio, it could become much harder for advisors to find well-performing investments when stock prices eventually retreat.