(CNN) Days after President Donald Trump called Rep. Mark Sanford a "nasty guy" behind closed doors at a GOP conference meeting in front of his colleagues, the South Carolina Republican says he fears Trump has fundamentally reshaped his party and that his fellow Republicans have let Trump get away with it.

"I don't know exactly where the party is right now, but it's a long way from the party that I've always believed," Sanford told reporters Thursday.

Sanford lost his primary earlier this month after a rocky relationship with Trump and a tweet in which the President endorsed Sanford's opponent just hours before polls closed. But Sanford, who has stood up to Trump in private meetings and publicly admonished his style, said he isn't backing down and he wishes his colleagues wouldn't either.

"The party has become untethered from its historic moorings in limited government," Sanford said. "Free markets, go down the list of traditional components that drove so many of us to be a part of the party. I am not using (Senate Foreign Relations Chairman Bob) Corker's words of a cult of personality, but it's drifted and so there is a real identity crisis within the party as to what it's about and what it's going to be about going forward in the age of Trump."

After Trump called Sanford a "nasty guy" Tuesday, the President tweeted later that members in the room "applauded and laughed loudly," something members in the room disputed.

Sanford told CNN that the tweet was systematic of a broader "cancer" in the party.

"It's not really about the tweet or him coming back and saying what happened didn't happen, but it is systematic of a larger cancer, which we've all come to just sort of blithely accept the fact that the highest office holder in the land can multiple times a day say something that isn't true," Sanford said.

Sanford, who admitted to carrying on an extra-marital affair and resigned from his post as South Carolina's governor in 2009, acknowledges the irony of speaking out about truth.

"Maybe that's why I feel so strongly about it because in 2009, I had a lie told on my behalf, but that is irrelevant I own it. ... But the system worked the way it was supposed to work and there were incredible consequences for me individually, personally, politically, financially, go down the list. Incredible consequences, and so we morph from that experience that I had to one where then you'll have a President send a tweet saying what happened and what all of my colleagues who have come up to me saying 'this is what happened' and what every media accounting said what happened, he comes back and says it didn't happen, the reverse happened," Sanford said.

"A buddy had called me about it last night and he said 'well I guess you know his account will be the one that wins the day.' And I said 'well, if that's true, we got a much bigger problem,'" Sanford said. "Because if truth is determined by whoever has the loudest microphone, then we have a fundamental crisis with regard to the way that our society works."

The South Carolina congressman, a member of the conservative House Freedom Caucus, and long considered one of the more conservative members of the conference said that Trump has co-opted the Republican base as his own and his power and popularity against his own party.

"What he has taken is the legitimate hopes and fears people have had for change in Washington and he has morphed that into -- given the strength of his personality -- a whole host of different loyalty tests about him," Sanford said.

He added later "Part of what took place with that meeting in conference was about sending a signal to every other member out there 'you mess with me, I'm going to mess with you.'

"That is not the kind of debate you want to have in an open and free society," Sanford said. "We are to be co-equal branches of government. We don't have a parliamentary system of government. We don't have to be subservient to the executive branch and so I spoke out before. I don't regret it. I will continue to speak out in the time I'm in office and beyond that."

Sanford notes he is hardly the only member who feels the way he does, but few are willing to speak up publicly.

"People come up privately numerous times a day and talk about different frustrations with the President's style," Sanford said. "His caustic nature, his latitude with the truth, but they won't voice those opinions publicly for fear of reprisal. Again, that's a cancer. In an open political system, for people to operate out of fear, is dangerous."