That unfamiliar sensation you experienced this morning? It’s the feeling you get when, at long last, you wake up to some good political news from America. For progressives around the world who would prefer to admire rather than to revile the United States, the last 13 months have brought cause only for despair. But the defeat of Roy Moore – who believes homosexuality should be illegal and that America was at its best in the age of slavery – is a moment to lift the spirits.

That a man of such vileness, also accused of being a child molester, should lose to a Democrat in deeply conservative, unbreakably Republican Alabama is more heartening still. It represents a grievous blow to Donald Trump, who endorsed Moore and campaigned for him, revealing the limits to the president’s supposed electoral magnetism.

Profile The Roy Moore file Show Hide Born Roy Stewart Moore, 11 February 1947, in Gadsden, Alabama, the oldest of five children of a construction worker and housewife.

Best of times He had a large slab of Vermont granite inscribed with quotes from the Declaration of Independence, the national anthem and the founding fathers installed in the Alabama supreme court. It was topped off with tablets inscribed with the Ten Commandments. Worst of Times In Vietnam, Moore insisted his troops salute him on the battlefield. He was named “Captain America” and later recalled sleeping on sandbags to avoid a grenade tossed under his cot in retribution. What he says “I think it [America] was great at the time when families were united. Even though we had slavery, they cared for one another.” What others say After refusing to acknowledge same-sex marriage legislation, Human Rights Campaign said: “It is clear that Roy Moore not only believes he is above the law, he believes he is above judicial ethics...”

Photograph: Brynn Anderson/AP

It also throws another obstacle in the path of Trump’s legislative agenda, which was struggling already but will now have one fewer vote in the Senate. More unsettling still for Trump, it suggests that accusations of past sexually predatory behaviour cannot just be dismissed with the kind of blanket denial deployed by both him and Moore. Had Moore won, Trump would have held that to be proof that voters don’t care about such things. He may try to argue this line anyway, but it will be harder.

Moore’s defeat also offers a glimmer of hope for coming electoral contests: the congressional elections in November 2018 and even the presidential race of 2020. For it shows that when Democratic constituencies are motivated – and African-Americans turned out in big numbers in Alabama – Republicans can be beaten even in their most reliable heartlands.

Democratic euphoria as Doug Jones wins Alabama - in pictures Read more

It is now conceivable that Democrats could retake the House of Representatives next autumn, despite the structural disadvantage inflicted by gerrymandering, which means Democrats have to beat Republicans in the popular vote by seven or eight percentage points just to secure a majority of a single seat. Retaking the Senate will be an uphill climb, but it too becomes imaginable. If the Republicans were to lose the House, then the impeachment of Donald Trump would be not just possible but probable.

As you can see, it’s easy to get carried away. But, without wishing to spoil the party, there are reasons to be cautious too.

The first is that Moore came very close to winning, taking 48.4% of the vote. That’s despite allegations of paedophilia. It suggests that in the absence of those claims, he would now be on his way to the US Senate. His espousal of theocratic and homophobic views would not, on its own, have been enough to keep him out. That is a troubling thought when contemplating future contests elsewhere.

Look closely at the breakdown of votes. Moore retained the backing of 91% of Republicans who turned out. Those loyalists were not sufficiently repelled to switch parties. More astonishing, 63% of white women and 72% of white men in Alabama voted for Moore, despite everything. Had it been up to the state’s white voters, Moore would be a US senator today.

Roy Moore’s stunning defeat reveals the red line for Trump-style politics | Richard Wolffe Read more

Put another way, it’s only thanks to the solid and energised support of Alabama’s black voters that the United States avoided what would have been a moment of global shame. That it avoided this fate so narrowly should prompt as much reflection as celebration.

Still, for now a sigh of relief is in order. Despite Trump’s constant race-baiting on Twitter; despite the polluting of the country’s electoral process by an opaque combination of foreign and corporate players, assisted by tech companies that allow lies to spread like a contagious disease; despite the flight from fact and partisan media that seek to deny even basic, demonstrable truth; despite all that, American democracy did the right thing. Even in a state as conservative as Alabama – which backed the segregationist George Wallace for president in 1968, and which elected the Republican Jeff Sessions with 97% of the vote in 2014 – there was a line the voters would not cross.

The forces of reaction, xenophobia and hate have dominated the US for more than a year. The victory of Doug Jones, on top of some notable successes in Virginia last month, suggests that the other America – the one so many around the world admire – still lives. And that is a cause for hope.

• Jonathan Freedland is a Guardian columnist