Rush Limbaugh attacked them as “lunatics” running a “plantation” for black “slaves.” Another right-winger claimed a “white” union president was out to hurt “disadvantaged black youth,” even though the union president in question is black.

It’s no surprise AFSCME’s decision to end its partnership with United Negro Colllege Fund, in the wake of UNCF accepting a $25 million contribution from the Koch brothers to establish a “Koch Scholars” program, as well as participating in their annual right-wing strategy summit, has conservatives trashing the influential public sector union, American Federation of State, County and Municipal Employees. It fits their favorite story line: Liberals are the real racists.

But even some non-wingnuts have piled on. The Root’s Keli Goff blasted the decision as “progressive intolerance,” insisting the Kochs’ support of public sector union busting “has nothing to do with educating African-American students.” That’s led to renewed interest in AFSCME’s move, which president Lee Saunders announced two weeks ago.

In a letter to UNCF president Michael Lomax, AFSCME’s Saunders called the Kochs “the single most prominent funders of efforts to prevent African-Americans from voting.” Lomax’s appearance at the Koch brothers’ annual summit, where they plot conservative strategy – this year’s focus was taking back the Senate in 2014 — was “a betrayal of everything the UNCF stands for.” Members of AFSCME, a union that supports Democrats with money and people power, voted unanimously to back Saunders’ decision at their annual convention in Chicago last week.

“This is not a Democrat or Republican issue,” Saunders told Salon, in his first interview since the fracas. “It’s an issue of right and wrong.”

Still, the critics keep coming. Just this week, in the right-wing Daily Caller, Raynard Jackson called it “a stunning move that should anger every black in America.” In the backyard of Koch-critic Senate Majority Leader Harry Reid, the Las Vegas Review Journal called the decision “ridiculous,” complaining that “Democrats and their supporters are quick to and fond of accusing conservatives and Republicans of being racists, but when Republican supporters such as the Kochs aim to demonstrate otherwise, a government employee union responds by causing direct harm to blacks.” The Review Journal is a conservative paper, but the African-American newspaper South Florida Times, which is not, likewise insisted “AFSCME must restore UNCF funding,” though it called Lomax’s decision to join the Koch summit “a mistake.”

It should be noted that AFSCME has always said it would continue the scholarship program for students of color it began with UNCF, but will find different partners.

Clearly the controversy, which involves two venerable organizations, has African-Americans on both sides of the issue. Founded by Mary McLeod Bethune and others in 1944, the UNCF offers scholarships to black students and funds important but embattled historically black colleges and universities. AFSCME, the union Rev. Martin Luther King Jr. was supporting when he was assassinated in Memphis in 1968, now has its first black president, Saunders, and African-Americans make up 15 percent of its membership.

“It was a very difficult decision for me to make as an African-American,” says Saunders, the AFSCME leader who first approached former UNCF president Bill Gray with the notion of partnering on a scholarship program for students of color interested in labor movement careers, more than 10 years ago. Saunders says he personally raised money for UNCF for years.

He adds that if the Kochs had donated $25 million to UNCF “and said ‘here it is, do what you want to do with it,’ maybe we would have taken a different tack. Maybe I would have swallowed it.” But Lomax’s participation in the Kochs’ annual summit made the union draw “a line in the sand,” he said.

Michael Lomax did not respond to requests for comment. In an earlier statement about the controversy he said: “While I am saddened by AFSCME’s decision, it will not distract us from our mission of helping thousands of African-American students achieve their dream of a college degree and the economic benefits that come with it.” In an interview with Education Week about some UNCF supporters’ opposition to taking the Koch money, he argued “I think that Washington’s partisanship has really poisoned the thinking of some people all across the country.” Koch spokesperson Melissa Cohlmia told AP the Kochs “have devoted their lives to advancing tolerance and a free society — where every individual is judged on his or her individual merits and they are free to make decisions about their lives.”

According to reporting in the Nation by Lauren Windsor, who obtained a “confidential” draft meeting agenda and spoke to participants, the summit Lomax attended drew Republican luminaries including Senate Minority Leader Mitch McConnell and Senate candidates Tom Cotton of Arkansas and Cory Gardner of Colorado — three close races that are crucial to GOP hopes of taking back the Senate – along with 2016 presidential hopeful Sen. Marco Rubio. No Democratic officeholders or candidates were included in the annual confab, which took place this year at the St. Regis Monarch Bay Resort in Dana Point, California.

Titled “American Courage: Our Commitment to a Free Society,” an early panel featured David Koch and longtime political partner Richard Fink, with whom he set up the right-wing Mercatus Center, holding forth on “Saving America: Our Fight to Advance Freedom and Reverse the Country’s Decline.” A dinner and then a morning panel showcased the notorious Charles Murray, who has long posited the “intractable” and probably genetic intellectual inferiority of African-Americans (and, more recently, low-income white people) and preached that government programs worsen poverty by destroying the work ethic and marriage habits of the poor.

In a four-session panel titled “The Foundation for Progress,” Lomax was featured in “Part One: Drive the National Conversation,” alongside Richard Fink and two Koch Industries executives. A session on how to “Advance in the States” featured Americans for Prosperity’s Teresa Oelke; Mitch McConnell headlined the final session, “Free Speech: Defending First Amendment Rights.”

UNCF’s decision to accept Koch funding was controversial even before AFSCME’s move, and some African-Americans have raised questions similar to Saunders. On NewsOneNow, Roland Martin hosted a discussion on the Kochs’ donation last month that turned into a polite but sharp debate between Lomax and progressive economist Julianne Malveaux, the former president of Bennett College for Women, the oldest historically black women’s college.

Expressing gratitude for UNCF’s support, Malveaux nonetheless questioned the “strings” attached to the Kochs’ gift: $18.5 million of $25 million is to create a “Koch Scholars” program, for “exemplary students” who want to study “how entrepreneurship, economics, and innovation contribute to well-being for individuals, communities, and society.” Koch organizations get two of five seats on that scholarship selection committee.

Lomax noted that $4 million of the grant is unrestricted support to HBCUs, and he invited Malveaux to come speak to the Koch Scholars. She replied: “I’d rather be on the selection committee,” and went on: “They’re using it to say ‘we support black people’ when the truth is that they don’t, they fund voter suppression.”

Beyond the details of the “Koch Scholars” program, or Lomax’s role in the Kochs’ right-wing summit, is the question of the role of Koch-demonized labor unions in the black community. African-American workers are more likely to be represented by a union than other workers, and labor leaders were among the earliest supporters of the modern civil rights movement (the famous 1963 March on Washington was also a “march for jobs and justice,” initially catalyzed by African-American labor luminary A. Philip Randolph). Black men have the highest rate of union membership of any group, at 14.8 percent compared with 11.9 percent of white men. In the 10 biggest metro areas, black workers are 42 percent more likely to be union members than workers of other races, according to Stephen Pitts of the University of California-Berkeley Labor Center.

Pitts’ study found black workers are 30 percent more likely than other groups to be employed in the public sector, which AFSCME represents. Those black public sector workers also make roughly 25 percent more than black workers in overall workforce – in part because public workers are more likely to be represented by unions, 35 percent vs. 7 percent in the private sector.

“The Koch brothers want a government-free society,” Saunders says. “Public service has enabled African-Americans to move into the middle class, but the Kochs are trying to dismantle that.”

The backlash against AFSCME’s decision on the right has led progressives to come out in support of the union in the last few days. “We believe that AFSCME is doing what is right for its members who the Koch brothers have targeted and is working in the interests of students of color who attend not only UNCF Institutions, but all institutions of higher learning,” Rev. Al Sharpton’s National Action Network said in a statement. (Saunders is on its board.) Color of Change’s Rashad Robinson agreed, insisting that the Kochs, who “spend their lives advancing an agenda that puts our communities in harm’s way, will never give us enough money in the name of charity to overturn their agenda.”

Saunders says, once again, that critics are wrong when they insist AFSCME is taking scholarships away from black students. He promises the union will continue its scholarship program for students of color.

“We’re going to find another partner, or do it ourselves,” he promised. “We’re committed to that.” He went on: “We agree with the mission of UNCF 100 percent. But the question is, are we going to be sold off to the highest bidder? I just think that’s wrong. We can’t be bought.”