Prominent figures on both the right and left of the US political spectrum gathered in the luxury enclave of Rancho Mirage in the Californian desert today amid increasingly heated debate about the influence of the secrecy-loving billionaires Charles and David Koch on the political process.

About 200 key figures in business, energy, the media and law were expected to assemble at a five-star hotel at the invitation of the Koch brothers for the latest of their twice-yearly discussion groups on how to forward their libertarian causes. The talks began at 1pm with sessions that focused on how to fight the Obama administration, which the Kochs see as a threat to the free market and unfettered wealth.

As the attendees arrived in their private jets, they were greeted by an airship that circled over the hotel's golf courses and tennis courts bearing the logo: "Koch brothers dirty money."

It was sent up by Greenpeace, the environmental campaign group, which has joined forces with several other left-leaning organisations to hold a counter-rally to the Koch meeting.

The Koch brothers run the second largest privately-owned company in the US, Koch Industries, which has stakes in oil and other energy industries, paper and pulp and chemicals, among other concerns. They are accused by detractors of increasingly using both their combined wealth - put at $35bn (£22bn) - and the wealth of their friends and fellow billionaires brought together at their gatherings, to distort the democratic process through huge campaign donations.

Greenpeace researchers looked at the guest list of the last Koch gathering, held in Aspen last June. The list of attendees was leaked to the Think Progress blog.

Using that list as a base, Greenpeace calculated that the 200 or so participants in Aspen contributed more than $61m (£38m) to political campaigns between 1990 and 2010. That makes them a major, though unofficial, bloc within American politics.

The role of the Koch gatherings as huge bankrolling events for rightwing candidates and causes was underlined in Aspen, where attendants were exhorted to put their hands in their pockets ahead of the 2010 mid-term elections. According to Politico, some $25m (£16m) in donations to Koch-backed groups was pledged at the lunch on the final day, including several individual pledges of $1m and $12m by the Koch brothers themselves.

In his letter of invitation to today's gathering in Rancho Mirage, Charles Koch referred to the previous Aspen meeting and said "participants committed to an unprecedented level of support". He added: "Our group heard plans to activate citizens against the threat of government overspending and to change the balance of power in Congress [in the mid-term elections]".

The aim of this latest Koch gathering was believed to be to raise at least as much money as in Aspen on behalf of the brothers' favourite causes, which include climate change denial, anti-government initiatives and the Tea Party movement. They are big funders of rightwing thinktanks such as Americans for Prosperity, the Heritage Foundation and the Cato Institute.

But for the first time they had to contend with the attentions of their opponents, with the convening of a panel discussion and rally just down the road from the hotel called by the accountability in politics campaign Common Cause. The counter-events attracted a wide range of activists from the environmental, anti-war, trade union and other movements.

"The Koch brothers manage to be destructive in so many areas," said Jodie Evans of Code Pink , a women's peace group. "But one positive thing they've done is to galvanise so many different opponents around them."