In 2008 and again in 2012, the states where early primaries were held were flooded with foot soldiers and sergeants in the Ron Paul army. They didn’t look much like the canvassers for the rest of the Republican field; they were tattooed, pierced, and younger. Many were getting involved in politics for the first time.

But for them Paul embodied something new in politics—his was a crusade rather than a campaign, one based on ideas that lacked currency in the larger public debate, especially the drug war and foreign policy. And his grassroots army propelled Paul through the primaries, helping him last far longer than his polling or fundraising indicated he should have.

Paul is now off the political stage, retired from Congress, his hopes lodged in the career of his son, Kentucky Sen. Rand Paul.

But conversations with several of the Ron Paul foot soldiers who trooped to Iowa and New Hampshire in the winters of 2007 and 2011 reveal a deep skepticism that the son can be a proper heir to what has become known as the liberty movement.

“Some folks expect there to be an automatic rollover in support from his father, and I don’t think that is going to be the case,” said Joel Kurtinitis, who was Ron Paul’s state director in Iowa in 2012 and since has served as an official in the state GOP. “It’s not automatic. We feel like automatic loyalty has cost us—one Clinton is just like the last Clinton; one Bush is just like the next Bush. We are going to watch people.”

Among the things Paulites say they have watched and approved of were Rand Paul’s 13-hour filibuster over the Obama administration’s use of drones. In the disapproval column are what they see as subtle indications the senator sends to GOP rank and filers that he is not like his father and that they have nothing to fear from his brand of libertarianism.

“As long as he’s able to take a stand like he did during the filibuster, that is something we can all rally behind. But there is a lot of concern about him being willing to play the media game,” said Kurtinitis, who cited comments Rand Paul recently made on abortion, immigration, and gay marriage in which he seemed to get tripped up by otherwise routine questioning. “I don’t have a problem with what he said, per se, but he might be trying too hard to reach that middle ground, which his dad was not known for.”

Indeed, part of the problem Rand Paul faces is turning the movement his father helped spark into a more traditional presidential campaign. The former can attract young people and first-time voters; the latter is necessary to carry the fight to the convention and secure the nomination.

“I like a lot of the things [Rand] says, but I tend to be more of a strict constitutionalist,” said Jim DiPasquale, a lawn contractor in Broward County, Florida, and Ron Paul supporter since 1988. “Ron, he was not afraid to stand up for obeying the Constitution regardless of the consequences politically. It didn’t matter to him.”

Rand Paul is seeking to make his father’s ideology more palatable to the moneymen and -women of the GOP, as well as to the Republican rank and filers who otherwise dominate early state voting. But on the other hand, he can’t lose the Paulites. They are exactly the kind of grassroots grunts any campaign dreams about. If he loses them or sees their enthusiasm dip, he quickly becomes just another senator scrambling to distinguish himself from the pack.

Brian Doherty, a senior editor at Reason magazine and the author of Ron Paul’s Revolution: The Man and the Movement He Inspired, says the younger Paul is banking on most of his father’s supporters to realize that he is the closest thing to a successor among the 2016 contenders.

“I think he genuinely doesn’t want to alienate his father’s base, but he has clearly decided that the noisiest Paulites in the Paul wing, he can probably afford to lose them,” Doherty said. “He knows that you can’t run for president by just walking the hard-core anti-war Ron Paul line.

“There is a loud wing of Ron Paul fans who are disappointed with Rand, but I think the most raucous of them are not representative of the 2 million” who voted for Ron Paul.

Doherty noted that the difference is not so much in Rand Paul’s stances but what he chooses to focus on. He does not go into the weeds on monetary policy, for example, in the way his father did or articulate the same kind of foreign policy critique as Ron Paul.

“He is an anti-empire guy, but he does it without spouting a lot of anti-empire rhetoric,” Doherty said. “He knows that the Republican base is a lot more patriotic than that. Ron had no problem saying we are being bad overseas. Rand doesn’t talk like that.”

And to be sure, plenty of people who signed on to fight for Ron Paul see his son as a worthy heir and are waiting for him to give the nod before they spring into action again.

“Rand is doing a great job taking the liberty message and making it palatable to conservatives,” said Eric Brakey, chairman of the Maine Defense of Liberty PAC, which was formulated to support Ron Paul’s presidential bid. “In many respects, policywise he is cut from the same cloth, and he has really found a way to message it in a way that reaches out to the Tea Party, to social conservatives, to different coalitions in the Republican Party.”

And if the senator doesn’t go as far as his father would if he were president, progress is progress, Brakey said.

“When you compare Rand Paul’s positions to some of his father’s, the difference is really a matter of degrees,” he said. “Some in the liberty movement reject Rand for not going as far as his father on a particular issue, but he is still moving the ball in the right direction.”

If the senator does end up losing the Paulites, a major question heading into the campaign season is where they go. Their grassroots presence has become a staple of the primary season, and the Republican convention will not be the same without the Paulites threatening a walkout or staging their own gathering nearby. It is not as if another Republican who can lead a movement is on the horizon.

“We know that something like 40 to 50 percent of Americans don’t vote,” said Doherty. “Ron Paul created his vote. His people were not previously Republicans. They were disaffected, anti-government activists who hated the political process. He drew them from those Americans who don’t vote, and a lot of them are going to go back to not voting.”