I have been a black student of black history for nearly 30 years. Yet, I find myself with a question that feels like a splinter lodged in my nail bed. For all the examples of African American success and American racism reduced, blacks are as far away from the "Promised Land" as their great-great-grandparents were 150 years ago.

White supremacists have killed innocents and terrorised anti-racism protesters in Charlottesville, in Portland, and other places while chanting "You will not replace us!" They regularly hold their rallies undisturbed by the police.

All this while police and white vigilantes constantly get away with murdering unarmed African Americans and act as stormtroopers when blacks protest such brutality with "Black Lives Matter". The current black unemployment rate is almost double that among white people and African Americans are still less than six percent of the student bodies at elite universities, although they make up 13 percent of the population.

OPINION: The cameras could not save Philando Castile

And yet, whites continue to complain that blacks are to blame, that affirmative action has hurt their job and educational opportunities. This isn't just President Donald Trump's America and the rise of the "alt-right". This is and has been America.

Have I been doing a disservice to myself, my students, and my readers whenever I've presented the history of black people in the United States as a positive one, a tale of success despite slavery, struggle amid Jim Crow, and incremental progress at every step? Have I been on a fool's errand, assuming that one glorious day, America will embrace me and the millions of other African Americans as human equals?

I and many others have done little to give readers a glimpse of the only possible future, one without the happy ending of social justice and racial equality achieved.

Because when I look at the US, I see a nation as unwilling to confront its history in 2017 as it was on that April Tuesday 49 years ago, when James Earl Ray murdered Martin Luther King, Jr.

Like so many African American historians, I imbued hope in an America that never was, and will never be. For at least the past 80 years, scholars like Robin DG Kelley, Paula Giddings, and WEB Du Bois, have made the central story the work that black women and men did to resist slavery and Jim Crow and the latticework that systemic and individual racism is. They faced down racial and sexual oppression to build families, communities, schools, and universities, to mobilise for civil rights and migrate for better working and living conditions, despite and because of poverty and white violence.

I have never doubted the strength, courage, and faith of blacks in surviving and succeeding, despite America's virulent racial and sexual oppression. But as Washington, DC public historian Marya McQuirter told me, historians like me have been too "deeply ensconced in the 'American Dream'," and have cared "too much about individual progress and not enough about white supremacy, capitalism, [and] patriarchy".

And she's correct. I and many others have done little to give readers a glimpse of the only possible future, one without the happy ending of social justice and racial equality achieved. "If one wants the racial divide to close along cultural, educational, and other lines, then one needs to acknowledge cultural differences based on the experiences of people of color," I wrote at the end of my first book, Fear of a "Black" America. What a pipe dream! White supremacists certainly "acknowledge" racial differences, and centuries ago decided blacks aren't worth "embracing," or even tolerating.

I should've known better. Years of reading critical race theorists such as Kimberle Crenshaw, Patricia J Williams, and the late Derrick Bell had taught me about the intractability of American racism. Bell pulled no punches in Faces at the Bottom of the Well, in 1992, when he wrote, "racism is an integral, permanent, and indestructible component of this society."

Still, I tried to wish racism away with symbolic victories like the 1964 Civil Rights Act and President Barack Obama's election in 2008. For every civil rights gain, there have been decades of policies crafted to chip away at fair housing, hiring, and college admissions practices. For every Obama, Ava DuVernay‏, and Kendrick Lamar as black success stories, there have been so many like Michael Stewart, Eleanor Bumpurs, Amadou Diallo, James Byrd, Jr, Trayvon Martin, and Sandra Bland in my lifetime.

Two recent books have confirmed that I and other black historians place far too much hope in a racism-less future. American University historian Ibram X Kendi is one of the few to at least acknowledge the limitations of black progress throughout American history.

As Kendi eloquently wrote in his National Book Award-winning Stamped from the Beginning (2016), the long "history of racist ideas shows ... upwardly mobile Black folk have not persuaded away racist ideas or policies." Yet, even with his sobering conclusion drawn from 400 years of American history, Kendi still infused the future with hope beyond his evidence. "No power lasts forever ...There will come a time when racist ideas will no longer obstruct ... when we will gain the courage to fight for an equitable society," Kendi ended.

Maybe so, though Kendi presented nothing throughout his book to support this prayerful conclusion. For me, Kendi came so close, and yet backed away, from the ugly truth of American racism's permanence.

Emory University historian Carol Anderson also committed herself to the same faith beyond the historical record in her 2016 bestseller White Rage. She explained that in the 150 years since Emancipation, the "trigger for white rage, inevitably, is black advancement," leading to racist backlashes in both policy and in everyday dealings with white folk. "The truth is, white rage has undermined democracy, warped the Constitution," and "squandered billions … on baseless incarceration," Anderson added. But despite everything, Anderson also found reason for hope. "Full voting rights for American citizens … quality schools, and policing and court systems in which racial bias is not sanctioned by law - all these are well within our grasp," Anderson said.

Based on who or what? Dylann Roof? Ferguson? Whether it's job discrimination, voting rights, mass incarceration, or white resentment, all are working as the Founding Fathers - those propertied, slave-owning elites - and their political descendants intended.

After all, the formula for ginning white rage has been racial oppression and economic insecurity at least since the US Civil War. Anderson is yet another historian who, like me, has ignored her own evidence, hoping for a sign, any sign of her "within-our-grasp" dream.

Patricia Williams in her The Alchemy of Race and Rights showed a quarter-century ago what many black historians have failed to acknowledge. The story of blacks in the US has always been somewhere between a bittersweet symphony and a heroic tragedy.

"To say that blacks never fully believed in rights is true. Yet it is also true that blacks believed in them so much and so hard that we gave them life where there was none before; we held onto them, put the hope of them into our wombs, mothered them and not the notion of them ..."

An America without the material and psychological advantages that racism has provided whites, while also disadvantaging blacks, would cease to be.

And if more African Americans become successful in the America that is, this nation with its history of racial, economic, and gender-based discrimination, would this really be the utopia black historians have hoped for? Or, would it merely mean that blacks of such prominence would be partakers in a system of American oppression, the beneficiaries of "a timeless, formless futurism" that would only reflect a toxically racist past, as Williams put it? Is this what black historians like me, Kendi, and Anderson have been asking African Americans to believe in and achieve? How wrong have I been?

"Black life is cheap, but in America black bodies are a natural resource of incomparable value," Atlantic correspondent and MacArthur "genius" Award-winner Ta-Nehisi Coates wrote in Between the World and Me. But the African American experience is more than just the physical and constant fear and apocalyptic despair, of whites exploiting, raping, maiming, and killing black bodies.

It is an experience of the spirit and mind, a history of an inspirational, maybe even delusional faith in a mythical America. What can blacks do in a nation that will ultimately never change, in an America forever committed to a racial and gendered caste system?

Maybe African Americans should emigrate. Maybe for some, a bloody revolution is a way out. Maybe incremental progress is the best blacks should ever hope for. Still, even with symbolic victories and the faith of blacks past and present, it would be easier to build a faster-than-light spacecraft than to expect a nation built on racial and sexual oppression to one day turn the corner.

Donald Earl Collins is an associate professor of history at University of Maryland University College. He is also the author of Fear of a "Black" America: Multiculturalism and the African American Experience (2004).

The views expressed in this article are the author's own and do not necessarily reflect Al Jazeera's editorial policy.