The day we brought my new-born son home to our Brooklyn apartment, an article in the New York Times pointed out that "a black male who drops out of high school [in the US] is 60 times more likely to find himself in prison than one with a bachelor's degree". These are the kind of statistics I often quote in my work. But this time it was personal. Looking down at him as he snoozed in the brand new car seat, I thought: "Those are not great odds. I'd better buy some more children's books."

Over the next few weeks, as we fumbled with the nappies, pram, barfing and burping, a new, previously unthinkable option for black American males emerged: the presidency of the United States. Osceola was born on the weekend Barack Obama declared his candidacy. This prompted conversations that I would not have had otherwise. His success, I was told, would signify great things for my son. Osceola would grow up with an assumption that the highest office in the land was open to him. That the future could be his. That there was, I was told, nothing that this child could not achieve.

Back in February 2007, when Obama announced his candidacy, this never made much sense to me. The fact that my son suddenly has a tiny theoretical chance of getting to the White House is less important than the more real chance of his ending up behind bars (one in three black American boys born in 2001 will do so) or dead (three black kids are shot every day). I wanted a president who could change the odds for the many rather than raise the stakes for a few. I didn't care what they looked like. It wasn't that I didn't understand the symbolic importance of his bid. I just did not want to mistake it for substance.

On a political level, I have always thought he was interesting. Obama's announcement came 18 months after hurricane Katrina put black America's collective deprivation and individual success clearly on display. One man can rise to the presidency and a whole community can sink into the Gulf of Mexico: anything, I thought, really is possible. And with three days to go before election day, it looks like the US stands on the verge of making the historic decision to put a black man in the White House.

This was no reflection on Obama. Everything I'd heard about him - not least his opposition to the Iraq war at a time when such a position was unpopular - was impressive. But his two years in the Senate suggested he was pretty mainstream and even, at times, a little suspect. He'd supported Joseph Lieberman (a Democrat who is now supporting John McCain) in his primary Senate campaign against an anti-war campaigner. And he voted to confirm Condoleezza Rice as secretary of state. It wasn't obvious to me that he would be any better than some other generic Democrat with different pigmentation. The idea that his presidency would mean anything for Osceola's life never really crossed my mind.

To express such scepticism before many Obama supporters was to be accused of cynicism. The true believers do not just want you to drink the Kool Aid. They demand that you chug it.

The people my scepticism vexed most were white liberals. Obama had become prey to the soft bigotry of unreasonable expectations. Describing the crowd's reaction to him in Rockford, Illinois, Time's Joe Klein noted: "The African Americans tend to be fairly reserved ... The white people, by contrast, are out of control." They had found a black politician they felt comfortable with, and wanted him to be everything: Martin Luther King, John F Kennedy, a griot, president, vice-president, motherhood and apple pie. They prattled on about a post-racial America as though the Jena Six never happened and Sean Bell, a unarmed black man from Queens who was riddled with bullets on his wedding day, was still alive.

My wife, who is African American, shared my reservations about Obama, but saw things differently. She remembers the thrill of being a young girl when the black Democrat Harold Washington was elected in her hometown, Chicago. She liked him because her parents liked him. She could see it was important, but she didn't know why.

"My dad grew up being told a black person couldn't be a pilot, and my son is growing up knowing that a black person can be president," she said. "It's not that racism is gone, it's just that it's not about the idea that all black people are excluded on the basis of their race from any part of society or any particular job. That was the racism my parents grew up with and that is now one generation removed from Osceola." Her dad became a pilot, as did her brother.

Of course, Obama isn't standing for Osceola's benefit - which is just as well, because if Osceola could vote he would most likely support Elmo for mayor of Sesame Street. But in a sense these projections lie at the heart of any thoughtful appraisal of the racial dynamics underpinning Obama's candidacy. The desire to believe we are in a paradigm-shifting moment must be set against the fact that not every historic first changes the course of history. Changing our understanding of what is possible doesn't, in itself, create new possibilities.

I watched Obama accept the Democratic nomination with my mother-in-law, Janet, in a cinema on the southside of Chicago. Janet was raised in the South with the laws that put her at the back of the bus. As a teenager she went with her mother to see Martin Luther King speak in Philadelphia, listening in the overflow in the vestry because there were too many people in the church.

She was the one who first told me about Obama in 2003. She got involved in his primary campaign for the Senate when he didn't have a prayer, after she'd seen him on the local public channel, when he was a state senator. "He seemed like a bright guy," she says. "He reasoned his way through things and was always very impressive." She particularly liked his stance on the war. When he said he was running for Senate, she signed up as a volunteer.

And now, here we were just five years later seeing him clinch the deal in Denver on the big screen. At one point, when he recalled his anti-war speech in 2002, she punched my arm. "I was there." As she drove me to my hotel, she would occasionally say to no one in particular: "I just don't believe it."

Whether Osceola would ever be able to relate to what a momentous time this is for Janet remains to be seen. But her response made me think that the late comedian George Carlin was wrong. Symbols are too important to be left to the symbol-minded. By that time, my thinking on Obama had evolved. Not so much because of the man, but the moment. The atmosphere during this campaign has been unlike anything I've ever seen in a western country. To see so many people - particularly young people - engaged and hopeful about their political future after eight depressing years is inspiring. The last time I saw it was in South Africa's first democratic elections in 1994.

Walking down Sumter Street during Charleston's Martin Luther King day parade, watching white volunteers chant: "Obama '08! We're ready. Why wait?" gave political voice to an America I never doubted existed, but had yet to see. Among them was a young man who was "so depressed" after Obama's New Hampshire defeat that he had dropped everything he had been doing in Guatemala and flown back to help out. Local African Americans lined the sidewalks, cheering encouragement. Obama's victory in Iowa had proved that a black candidacy was not a pipe dream.

It was a moment. Fleeting and maybe even fatuous. But nonetheless a political moment that produced hopeful human engagement. Within half an hour it had evaporated. The white volunteers went back to the office and black people went back to their homes in the poorest parts of town and waited for change. But that didn't mean it didn't happen or that it couldn't happen again. Nor was there anyone else who could make it happen.

A couple months later came Obama's race speech in Philadelphia in response to the attacks on his pastor, Jeremiah Wright, in which he addressed black alienation and white disadvantage, set them both in a historical context, and then called on people to rise above it. It was a tall order. He pulled it off.

That weekend, a friend invited us to brunch with a group of other black people to discuss the fallout. There were nine of us (10 if you include Osceola, who yanked a blind from the window). It was a typical boho (black bohemian) Brooklyn crowd of voluntary sector workers, teachers and the like. Most, like me, had been ambivalent about Obama at the outset. But his candidacy was becoming a vehicle for something bigger: a teachable moment about the potential of anti-racist discourse.

A year before, Hillary Clinton's chief strategist, Mark Penn, laid out a plan of attack against Obama. "All of these articles about his boyhood in Indonesia and his life in Hawaii are geared towards showing his background is diverse, multicultural and putting that in a new light. Save it for 2050. It also exposes a strong weakness for him - his roots to basic American values and culture are at best limited. I cannot imagine America electing a president during a time of war who is not at his centre fundamentally American in his thinking and in his values ... Let's explicitly own 'American' in our programmes, the speeches and the values. He doesn't ... Let's use our logo to make some flags we can give out. Let's add flag symbols to the backgrounds."

Clinton rejected Penn's advice, but McCain pretty much adopted it. And at this point it appears to have failed. This time Republicans have misread white America's appetite for divisive racial rhetoric and overestimated its fear of the other. The fears and division are still there. But whatever the result on Tuesday, they are clearly no longer the decisive mobilising force they once were.

If there is promise in here for my son, it is not so much that he is capable of doing anything he wants - I am his father and it's my responsibility to teach him that - but that white people won't necessarily stop him. What that does for his odds of finishing high school or going to jail remains to be seen. In the meantime, I'm off to the bookshop.