Anyone who has been in the liberty movement for long generally notices two things. First, leftists hate us. Second, some libertarians become literal fascists. (Yes, I did just link to the definition. Some people seem to need it.)

It’s not that the ideas of the liberty movement and those of fascists come close at all. One says we should treat all people as individuals and they have a right to self-govern. The other says a dictator should rule, subjugate, and the people should just be subservient. Liberty is a positive good we strive to achieve, but the fascists seek a destruction of liberty in order to establish order. It is completely tragic that some see fascism as better than liberty.

A recent article in the New York Times profiled Tony Hovater, a stereotypical Internet troll and self-styled “foot soldier.” His progression, by his own admission, was from lefist to what the Times calls a “radical libertarian” (read: anarcho-capitalist), and finally to a Neo-Nazi. Hovater even sells swastika armbands. He’s not just vaguely “alt-right.” He’s deep into Neo-Nazi philosophy, and believes libertarians are failures and far too idealistic.

However, there is still some common ground. The Times says “He believes the federal government is too big, the news media is biased, and that affirmative action programs for minorities are fundamentally unfair.”

This is why some people, though they usually don’t get much media coverage, flip the opposite direction. Take, for example, a friend of mine. He was that kind of alt-right that said “Trump doesn’t go far enough.” He got around the racism problem by harping on “culture.” It’s not that he was a bad person, or devious. He actually wasn’t a racist. It took conversations over a year and a half, but eventually he did switch. What mattered was finding that common ground and moving from there to a mindset of liberty.

It’s important to note that the left has had its fair share of disastrous defections. Jason Kessler was a liberal not long before he organized the Charlottesville rally. That makes sense. The alt-right has a lot in common with the left. Richard Spencer, for example, is an apologetic supporter of many left-wing policies.

The difficulty is collectivism, of course. Most people are collectivists, and opening their eyes to liberty isn’t easy. But we do have plenty of examples like my friend. The trick is making people realize our ideas are better, but not alienating them in the process.

Google “Libertarian turned fascist.” I’ll wait. It has 430,000 results in just 0.38 seconds.

Some people think this is a real problem for libertarians. “Libertarianism has an alt-right problem,” writes The Daily Beast. The difficulty is that we do share certain common ground with the alt-right. Tony Hovater was described as once being a libertarian in the Ron Paul revolution. His grievances, according to them, include “… [that] the federal government is too big, the news media is biased, and that affirmative action programs for minorities are fundamentally unfair.” Many libertarians would agree with all of those issues. However, the similarities end at the surface. Libertarianism is a philosophy of individual rights. Fascism is a doctrine of government control.

Since we have something of an epidemic on our hands, we need a diagnosis. The symptom is libertarians leaving the movement to become prominent fascists, or at least statists in general. Many people we’ve thought were squarely in the realm of liberty forget their principles. Some, like Michael Brendan Dougherty writing for National Review, have said it’s only because both groups are minorities, or “marginal.” That may be true, but that’s not all. I think there are at least four major reasons for some of us becoming extreme on the right and rejecting what we’ve believed.

1. Leftists ostracize free-marketers

As the left moves toward socialism once again after a brief stint in the free-market side of things, we’re bound to see some stupidity. One of the most atrocious is the simple accusation that free-marketers hate the poor. While I’m sure there are one or two people in the movement who really do, they’re not representatives of the rest of us. Most of us know that free markets are actually good for the poor.

Besides, we’re crony capitalists in their minds. We’re totally ostracized by the left and lumped in with people who want corporate tax cuts but none for anyone else. The left has been demonizing capitalists for years. Even though there are huge differences between conservatives and libertarians on infinite issues, we’re both mostly capitalist, and that’s enough.

Popular media makes anyone who dares fade ‘conservative’ on an issue a total a–hole. Take, for example, the wildly popular “Dear White People.” I like some aspects of the show. It’s raw, it shows how some people really feel and think. It’s well-written and well-directed. But some stuff just isn’t helpful. When a black man is referred to as a Republican, he doesn’t have any real motives; he just “hates poor people.” And that character is never given a voice to defend himself. The only character who stands up for free speech in the whole thing is unequivocally a jerk about it.

People on Twitter don’t help, either:

So it's not politically correct professors of self-esteem studies or something who are at odds with the right, it's basically anyone who values hard thinking and evidence. So maybe colleges aren't the problem, and conservatism is? — Paul Krugman (@paulkrugman) November 26, 2017

It took me two seconds to find a tweet like that. There are hundreds more out there. When we’re lumped in with conservatives, and conservatives are treated like this, we tend to want to strike back. Striking back, though, is just how the alt-right and many others act.

So, for starters, we can see why many in the liberty movement feel like the left is against them. This creates an us vs. them mentality. Fascism feeds on division, and the left isn’t helping here.

2. Fascists pretend to be free market

Never mind the fact NSDAP, in German, stands for National Socialist German Worker’s Party. Never mind Hitler being right in the center of the political compass on economics. Definitely never mind all of history.

Some Neo-Nazis and other brands of fascist tell us they’re free market. Many even go so far as to call themselves libertarians!

They know what they’re doing when they lie like this. But the lies of the intelligent ones lead to an insidious fact: the fascists today mostly think they’re just “realistic” libertarians. Sure, they say, they want everyone to be free and equal. Just… only once they kill the Jews, establish a vague “white” ethno-state, and establish impenetrable borders.

They don’t know what they’re doing.

The sad fact is, “fascists” today are just “alternative” morons

Re-read the Times quote up there on Mr. Hovater’s grievances. Aren’t they all things the average conservative, the average libertarian even, agrees with? But a true fascist would want a fascist state, so his gripes about the size of the federal government are nonsense. Like every group outside the liberty movement, Neo-Nazis just want government small until they’re the ones in charge.

Hovater wrote an article critical of the libertarian movement. He said it was too left-wing. He didn’t stop there. In fact, he even went so far as to say even the most anarchistic of the libertarians were leftists. And his reason?

As he puts it, libertarians are too concerned with the language of “social justice.” I suppose he doesn’t know much about fascism. If he did, he’d know that Mussolini wanted social justice for the poor too. The Italian whose dictatorship was practically the beginning of fascism once said that “the object of the regime in the economic field is to ensure higher social justice for the whole of the Italian people… What does social justice mean? It means work guaranteed, fair wages, decent homes, it means the possibility of continuous evolution and improvement.”

For goodness’ sake, he’d probably support free college.

So it’s pretty clear that even though fascists and Neo-Nazis want to be in Hitler or Mussolini’s same vein, they don’t have the same ideology. This leads to the third big point:

3. Fascists have become more socially liberal towards LGBT persons

I’m not saying this is a bad thing at all. But it’s a problem when combined with the other parts, because they tend to lie a lot.

That lying can result in some… interesting worldviews. Take, for example, what the Times reported on Hovater:

“He said that while the Nazi leader Heinrich Himmler wanted to exterminate groups like Slavs and homosexuals, Hitler ‘was a lot more kind of chill on those subjects.'”

Chill compared to who? Oh – Heinrich Himmler. Needless to say, Hitler absolutely wasn’t “chill” about homosexuals. Himmler carried out the persecution, but he persecuted LGBT people – mostly gay men – on Hitler’s orders.

But why lie like this? It’s only because of a general trend on the extreme authoritarian right. In the face of culture shifting, they’ve become more accepting of homosexuality and other socially liberal positions. While they want the ideal nuclear family, and they want their white ethno-state to be racially pure, some of them aren’t making that their crusade. The Nazis crusaded vehemently for the rejection of LGBT rights. Some of the modern fascists? Not so much. The reason why is pretty clear: the Evangelical Christian Right and their coalition took that territory. Trump is probably the most LGBT-friendly Republican president in years. They’re still awful in general. But their softening, however slight, is a real factor.

So while it’s not the thing that will entice thousands to join Hovater’s Traditionalist Worker’s Party, it’s going to make the so-called “red pill” less distasteful if it’s only vaguely defined ethnic “Jews” being exterminated and not someone’s gay loved ones.

This is really all an expression of the big one:

4. Really, every collectivist wants big brother on their side

Here’s the thing: true, dyed-in-the-wool libertarians are rare. Even in America, where some of our most prominent Founders would be libertarian today, we’re not a majority. Not even close.

When the Missouri Compromise was passed, Jefferson wrote that “I regret that I am now to die in the belief, that the useless sacrifice of themselves by the generation of 1776, to acquire self-government and happiness to their country, is to be thrown away by the unwise and unworthy passions of their sons.”

It didn’t take long for the Founders to realize liberty is a hard thing to keep.

So, for libertarians shaking their heads at people like Hovater, who became a racist moron spouting off that the Libertarian Party would “put up in a few cycles… an overweight, black, crippled dyke with dyslexia [for President],” remember that. We’re rare. But we’re not going anywhere.

It’s the most difficult thing in the world to admit most people won’t ever agree with us entirely, but it’s true. What Jefferson called a “self-evident” truth, Lincoln had to amend to a “proposition” just 87 years later. Equality of rights is a hard concept for most people to understand. It’s up to us to make sure people understand our principles, and that everyone recognizes each other as capable of self-government. Otherwise, there will be plenty of “libertarians” who are still collectivists.

If we don’t make it totally clear that we’re about individual rights, we’ll be infiltrated, abandoned, and then made fun of by morons like Hovater. Collectivists like him will see that a free market won’t keep white people (or whatever race or collective they please) on top. A free market will be free, and people will succeed on their own merit. And when people all have the same opportunities, they’ll all come out with the same success, regardless of skin color. That’s not something a white nationalist can stomach.

It’s on us to make sure we aren’t as susceptible to people like Hovater. It’s on us to be clear on our principles. And it’s on us to disavow, loudly, the people who would call themselves “realistic” libertarians. They’re just collectivists, and we know it. So let the left do its thing, and let’s keep fighting for liberty.