Coach Jim Harbaugh began to lose some of his San Francisco 49ers players when he was not honest with former quarterback Alex Smith about San Francisco's pursuit of Peyton Manning in 2012, according to team sources, who expressed their belief to ESPN NFL Insider Adam Schefter.

Smith, the man at the root of some of San Francisco's issues, returns to San Francisco on Sunday with the Kansas City Chiefs for a matchup with the 49ers, who seem to be in a state of chaos just four games into this season.

Smith specifically asked Harbaugh about the 49ers' interest in Manning and did not get an answer that matched the facts of the situation, in which Harbaugh flew to North Carolina to meet and work out Manning, the sources said.

As one team source told ESPN, "that was the first big sign to the players that they couldn't trust Jim."

49ers CEO Jed York tweeted his support of Harbaugh on Sunday afternoon.

Jim is my coach. We are trying to win a SB, not a personality or popularity contest. Any more questions? — Jed York (@JedYork) October 5, 2014

Last Sunday night, after the 49ers ended a two-game losing streak by handing the Philadelphia Eagles their first defeat, NFL Network analyst Deion Sanders went on "NFL GameDay Final" and said the 49ers players "want him out. They're not on the same page."

Harbaugh had a succinct reaction to Sanders' report last Monday.

"Personally, I think that's a bunch of crap," he said.

Harbaugh is in the fourth year of a five-year contract. Talks to extend his contract have been put on hold.

ESPN.com 49ers reporter Paul Gutierrez contributed to this report.