These days, and especially since the deadly rally in Charlottesville, Va., last August, it has become clear to many Americans that the specter of Nazism in their country is not resigned to 1930s history. But until very recently, even that part of the story was less well known than it is today.

In fact, when Bradley W. Hart first started researching the history of Nazi sympathy in the United States a few years ago, he was largely driven by the absence of attention to the topic. Hart’s new book Hitler’s American Friends: The Third Reich’s Supporters in the United States argues that the threat of Nazism in the United States before World War II was greater than we generally remember today, and that those forces offer valuable lessons decades later — and not just because part of that story is the history of the “America First” idea, born of pre-WWII isolationism and later reborn as a slogan for now-President Donald Trump.

“There’s certainly a raw and visceral shock to seeing swastikas displayed in American streets,” Hart tells TIME. “But this is a topic I’d been working on for quite a while at that point, and while it wasn’t something I expected, it was a trend I’d been observing. I wasn’t terribly shocked but there’s still a visceral reaction when you see that kind of symbolism displayed in the 21st century.”

Hart, who came to the topic via research on the eugenics movement and the history of Nazi sympathy in Britain, says he realized early on that there was a lot more to the American side of that story than most textbooks acknowledged. Some of the big names might get mentioned briefly — the radio priest Father Charles Coughlin, or the highly public German American Bund organization — but in general, he says, the American narrative of the years leading up to World War II has elided the role of those who supported the wrong side. And yet, American exchange students went to Germany and returned with glowing reviews, while none other than Charles Lindbergh denounced Jewish people for pushing the U.S. toward unnecessary war. In its various expressions, the pro-Nazi stance during those years was mostly focused not on creating an active military alliance with Germany or bringing the U.S. under Nazi control (something Hitler himself thought wouldn’t be possible) but rather on keeping the U.S. out of war in Europe.

So why was that past overlooked for so long?

In part, Hart theorizes, it’s because the American story of World War II is such a powerful national narrative. The United States, that narrative says, helped save the world. Rocked by Pearl Harbor, Americans stepped up to turn the tide for the Allies and thus solidified their nation’s place as a global superpower. That narrative doesn’t have much room for the relatively small, but significant, number of Americans who were rooting for the other side.

“It’s always been uncomfortable in this country to talk about isolationism, though the ideas are still out there,” he says, “It’s part of the American mythology. We want to remember ourselves as always having been on the right side in this war.”

It was also possible for those who had participated in Nazi-sympathetic groups to later cloak their beliefs in the Cold War’s anti-communist push — a dynamic that had in fact driven some of them to fascism in the first place, as it seemed “tougher on communism than democracy is,” as Hart puts it. (One survey he cites found that in 1938, more Americans thought that communism was worse than fascism than vice versa.) Such people could truthfully insist that they’d always been anti-communist without revealing that they’d been fascists, and their fellow Americans were still so worried about communism that they might not press the matter.

“We still don’t totally know the scope of this,” he adds, noting that some important documents are still classified.

Get our History Newsletter. Put today's news in context and see highlights from the archives. Please enter a valid email address. Sign Up Now Check the box if you do not wish to receive promotional offers via email from TIME. You can unsubscribe at any time. By signing up you are agreeing to our Terms of Use and Privacy Policy . This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply. Thank you! For your security, we've sent a confirmation email to the address you entered. Click the link to confirm your subscription and begin receiving our newsletters. If you don't get the confirmation within 10 minutes, please check your spam folder.

Plenty of the story’s beats have modern counterparts. For example, the Foreign Agents Registration Act that ensnared Paul Manafort was a product of that time. Hart’s book also covers controversy over whether those with extreme political views should be allowed to speak on college campuses, a debate that still rages today. The most interesting parallel to Hart himself is that between the type of social-media misinformation campaigns that popped up around the 2016 election and the use of propaganda by Nazi agents in the United States. (In one remarkable incident, a German agent and a sympathetic congressional aide were able to take advantage of franking privileges — free mailing services available for Congressional communication with constituents — to distribute a massive amount of official-looking propaganda.) In both cases, one major goal of those involved was simply to create a situation in which Americans weren’t sure what to believe. The takeaway, he says, is that the effect of unreliable news may be more important than the actual content of those stories.

“They weren’t trying to push the U.S. into an alliance with Nazi Germany. They see that as outlandish, though they would have loved it. What they want to do is confuse American public opinion. That’s what we’re seeing coming back in the era of social media,” he explains. “Confusion means there’s no public will to do anything and in a democracy we rely on the public will to take action.”

But perhaps the biggest reason why it’s possible for the U.S. to have forgotten about this history is that its worst potential — a sympathetic politician reaching the highest levels of power on an isolationist platform — was never realized.

“The real threat here, which the U.S. was fortunate to avoid, was a figure like Charles Lindbergh managing to bring all these groups together in time for an election,” Hart says. “The timing just never really worked out for that, fortunately.”

The real heroes of this story, as Hart sees it, are America’s political parties, and the establishment politicians who kept extreme isolationists off ballots. Today, now that primary elections generally make those decisions instead, that’s a role that falls to every American.

“The responsibility has passed from party elites down to primary voters,” Hart says. “That’s something that anybody who votes in a primary should think about: Is this person I’m voting for really the right person not just for the party but for the country?”

Write to Lily Rothman at lily.rothman@time.com.