MADISON, Wis.—The standoff over the Budget Repair Act is about the portions of the legislation that scale back union rights. That’s how it started. That’s why the AFL-CIO, SEIU, AFT, and every other union with a bus is at the capitol. But many of the protesters think it’s about something more insidious. On the walk into the capitol today, I saw these six signs in the space of 10 minutes:

Scott, How Much KOCH Have You Done?

Scott Walker is a KOCH Whore

CONFIRMED: Walker and Koch, Brothers With Bats

KOCH-SUCKERS

Drunk With Power—High on Koch

Recall Koch Bro’s “Puppets”—”Scott Walker” & “Republicans”

None of the protesters knew the other, but some of them bonded over their obsession: the influence of David and Charles Koch, two of the wealthiest men in America, donors to a bouquet of libertarian and Tea Party causes. Amy Janczy, from Lake Mills, Wis., carried the “Drunk With Power” sign; David Wend, from Madison, carried the “Puppets” sign. They stopped to talk.

“I’ve been doing a lot of reading about the Kochs,” explained Wend.

“I read the legislative summary of the Budget Repair Act,” explained Janczy, “and saw the stuff in there about the giveaways for the power plants. I saw the Kochs’ fingerprints on that.”

She was talking about Section 16.896 of the bill, which empowers Gov. Scott Walker to “sell any state-owned heating, cooling, and power plant or may contract with a private entity for the operation of any such plant, with or without solicitation of bids.” That section started to get attention on Monday; by this morning, Democrats in the state Assembly were using the floor time allotted to them in their quasi-filibuster to ask whether the Kochs were behind it, or interested in buying the plants.

Madison’s liberal Capital Times newspaper got a flat denial of that claim. “We have no interest,” said Philip Ellender, Koch Companies’ president of government and public affairs, “in purchasing any of the state-owned power plants in Wisconsin and any allegations to the contrary are completely false.”

I pointed this out to Janczy. “Well,” she said, “they may say that, but I don’t believe it.”

When it comes to the Kochs, progressives in Wisconsin are ready to believe the absolute worst. Inside the capitol there are dozens of agitprop signs accusing the brothers of buying the election for Walker. There are detailed lists of Koch companies and which products to boycott in order to starve them. There are articles taped to the walls from Forbes magazine (“Texas Koch Brothers Behind Wisconsin Effort To Kill Public Unions”) and the New York Times (“Koch Brothers’ Money Fuels Wisconsin Fight“). On Wednesday, a new sign started appearing around the halls, informing protesters of a picket outside the stately office building, not far from the capitol, where Koch Companies have hired seven lobbyists.

In sum: They have found the enemy, and it is Koch.

But how big a role are the Kochs actually playing in Wisconsin? A popular argument on the streets here is that Walker got $43,000 from Koch’s PAC, and that the PAC gave $1 million to the Republican Governor’s Association—a fact dug up first by Andy Kroll of Mother Jones. One protester pointed out to me that Rep. Paul Ryan, R-Wis., had spoken at one of the Kochs’ annual conferences. Thanks to the no-cameras-please PR strategy of the Kochs (none of their Madison lobbyists responded to my interview requests today), these mid-sized donations have taken on mythic proportions.

How can we judge how deep the Kochs’ influence runs? That New York Times story points out that all Koch-affiliated companies and employees gave about $1.84 million to Republicans, nationwide, in the 2010 election cycle. Americans for Prosperity, the nonprofit Tea Party-organizing group co-founded by David Koch—he’s still on the board—had a $40 million budget in 2010. (On Tuesday AFP announced a $342,000 ad buy supporting Walker.) Nationally, the labor movement spent far, far more than this. To take one example, AFSCME, whose green-shirted members have made their presence known in Madison, spent $87.5 million on the election.

Many protesters, and some Democratic politicians, are blunt enough to say that the labor movement is essential because of its financial generosity. (One sign I saw listed the 10 biggest sources of money in politics to make the point that unions were the only thing keeping Democrats in the game.) Liberals here and elsewhere were infinitely more critical of the Citizens United decision than conservatives were, but that decision gave unions the same get-out-of-disclosure-free card it gave to corporations.

I asked one protester, Jeanne Duffy, what the difference was between the benefits unions get when Democrats are in power, and the benefits the Kochs get when Republicans win.

“Did the unions, when they were in ‘power’ “—she made scare quotes with her fingers— “pass bills to abolish the Heritage Foundation? Or did they use it to expand BadgerCare? The point of what the unions are doing is to make this a more democratic nation, where more people get more access to what they need. I mean, could you or I call Gov. Walker and get 20 minutes with him?”

Ah, yes, the phone call: That’s why the Koch drama dominated Madison today. On Tuesday, Buffalo Beast writer Ian Murphy had called Scott Walker’s office, posing as David Koch (and sounding very little like him) and talked his way into a 20-minute conversation. The transcript of the call was embarrassing, with the governor saying more about his strategy and peeves than he’d done in a week of media interviews. He thought he might nail the 14 Senate Democrats who’d fled the state on an ethics violation, if their lodging was paid for by unions. He was dismissive about their demands.

But the details of the conversation hardly mattered to the protesters. To them, the call was a game-changer simply because it existed. Walker took the call; he knew who Koch was; he talked for 20 minutes. Democrats in the state Assembly, who had been mentioning the Koch-Walker connections or the no-bid-power-plant theory throughout their debate, took to the floor on Wednesday to ask what Walker knew about Koch and when he knew him.

“These Koch brothers!” said a worried-sounding Rep. Gary Hebl. “These Koch brothers are talking to Gov. Walker!”

After Walker participated in a tense press conference dominated by questions about the Koch tapes, Democratic Rep. Brett Hulsey took questions and explained why the Koch conversation rattled Democrats while confirming their suspicions. The call was evidence of “pay for play.”

“It was shocking to us,” he said. “We now understand why [Walker] killed the train money, why he killed the wind development, why he killed $46 million of transit money. He’s in the pocket of big oil interests.”

I asked why this proved that these were things David Koch wanted.

“I’m not going to talk about a vast right-wing conspiracy like Mrs. Clinton,” he said, laughing. “But I’ve seen this movie before.”

How did the Kochs become the villains of Madison? They have, for decades, bankrolled libertarian think tanks and programs, and they help put on conferences where conservative ideas are spread. Among the ideas they end up spreading are drug legalization and opposition to the Patriot Act. The Tea Party was the first movement funded in part by the Kochs that really took off.

So why credit everything that Republicans are trying to do now to Kochs’ influence? Partly because they do have some influence, and partly, as the Assembly Democrats kept goading Republicans, because they are shadowy “New York billionaires.” A complicated fight over public-sector unions can be broadened into a stand against secretive malefactors of wealth, who can be connected somehow to every conservative victory or idea. And the fear and paranoia grows, because, theoretically, they could be spending more than anyone knows. If Citizens United lets conservatives spend more money secretly, it has a hell of a side effect.

Like Slate on Facebook. Follow us on Twitter.