The white supremacist violence in Charlottesville was the tragic result of the Republican Party’s racist politics, which have only intensified under President Donald Trump.

We must fight those politics, but we should also acknowledge that racism isn’t only angry people clutching guns and confederate flags.

A week ago, Daniel Loeb, the chairman of the board for New York City’s Success Academy Charter Schools network, accused New York State Senator Andrea Stewart-Cousins of being worse than the Ku Klux Klan:

Hypocrites like Stewart-Cousins who pay fealty to powerful union thugs and bosses do more damage to people of color than anyone who has ever donned a hood.

What had the African-American senator done wrong? Stewart-Cousins supports more transparency and accountability for charter schools, which are publicly funded but privately operated.

The billionaire hedge fund manager went on to praise charter school advocates “who stand for educational choice and support Charter funding that leads to economic mobility and opportunity for poor [black] kids.”

Loeb’s remarks are unacceptable and should be condemned—but we shouldn’t miss the deeper lesson.

While Loeb has apologized, he’s refused to resign—and there’s very little New Yorkers can do about it. Even though Success Academy is funded with taxpayer dollars, taxpayers have no say over how the network operates. The investors, lawyers, PR professionals, and philanthropists that sit on its board get to spend public money to educate children with little accountability to the public.

That’s the problem with private control of public education. Charter schools are publicly funded schools run by private groups—some for-profit, some nonprofit, and some that use nonprofit status to hide behind for-profit schemes unaccountable to parents and the public who pay the bills.

For example, across the country from Success Academy, two charter schools in Livermore, California, managed by the nonprofit Tri-Valley Learning Corporation were recently found to have potentially funneled millions of dollars in public money to various private entities. The schools closed, leaving students scrambling for other options and parents with no way to hold Tri-Valley’s leaders accountable.

Many charter schools also raise additional money from private investors with little transparency about how that money is spent. Success Academy has received millions of dollars from billionaires like Rupert Murdoch and Trump economic advisor John Paulson.

What Daniel Loeb said is not only racist, it’s also wrong. As the nation’s schools become increasingly segregated by race and class, many charter schools are making things worse.

Public schools should be in our hands, not billionaires who get to decide what’s best for “poor black kids.”