Big business is gaining almost unfettered access to the corridors of Australia’s parliament owing to an oversight regime that is weak, unenforced, opaque and unable to keep track of the revolving door between lobbying and government.

A Guardian Australia investigation will this week reveal serious failings in the system governing political lobbying, while for the first time exposing the true extent to which ex-politicians, staffers and public servants are now lobbying for private interests.

An exhaustive analysis of the backgrounds of 483 registered lobbyists has revealed that more than half were previously inside government, the bureaucracy or party organisations. One-quarter have worked inside the offices of ministers or backbenchers.

In some cases, defence officials are leaving their posts to help arms manufacturers vie for lucrative government contracts, or senior advisers are leaving health or resources ministers to lobby for big pharmaceuticals or mining giants.

Prof AJ Brown says the lobbying system is ineffectual, without independent enforcement. Photograph: Griffith University

One of Australia’s leading integrity academics, Prof AJ Brown, said the current oversight regime – a lobbying code of conduct and public register – fails to guarantee that the public interest will be preserved in government decision making.

“Especially without independent enforcement, it really is the worst kind of window-dressing, because it’s not even a window with glass in it. It’s just an empty frame,” the Griffith University professor said.

Guardian Australia is today launching the Transparency Project, a one-stop shop to help Australians investigate government integrity and work with reporters to hold the government to account.

It has also examined powerbrokers across all major parties who simultaneously hold positions with lobbying or peak interest groups.

The latest is the newly appointed National party federal vice-president, Katrina Hodgkinson, who at the same time works for one of Australia’s largest lobbying firms, Barton Deakin. Barton Deakin’s clients include AGL Energy, the multinational contractor Serco, Nucoal Resources and the wealth manager AMP.

Hodgkinson was registered as a Barton Deakin lobbyist but removed herself from the register when she nominated for the Nationals’ vice-presidential role. She remains the firm’s corporate counsel but says her work is “internal and advisory in nature”.

There is no suggestion that Hodgkinson has acted improperly or breached the lobbying code.

Last year concerns were raised about the current Nationals federal president, Larry Anthony, who co-owns a consulting firm, SAS Group. The group has previously lobbied on behalf of Santos, among others, and lists China Telecom Global and Delta Electricity as clients.

Anthony also rejects any suggestion of impropriety or a breach of the lobbying code.

“We note that while there has been some media interest over the last 12 months in Larry Anthony’s volunteer role as federal Nationals president, there has been no allegation of impropriety made against him or the firm in that time,” an SAS Group spokesman said.

Quick Guide Political lobbying in Australia: the basics Show What do lobbyists do? Lobbyists attempt to influence government policy or decisions on behalf of either a client or their own organisation. Ethical lobbying is a valuable and important element of a healthy democracy. It helps those who have a stake in government policy to convey their views and expertise. There are two broad types of lobbyists: third-party lobbyists, who are engaged as consultants; in-house lobbyists, who work directly for corporations or interest groups. Who hires lobbyists? For many Australians, lobbying conjures images of powerful corporations working to sway politicians behind the scenes. There is a truth in that. The big banks, mining and energy giants, pharmaceutical companies, casinos, Amazon, Google and Facebook all engage lobbyists. But lobbyists also work on behalf of not-for-profits and community groups, including for veterans, social workers, aged-care staff, school principals and environmental organisations. What is the lobbyist register? The lobbyist register is the public's only window into the world of lobbying. It's a publicly available online list of lobbying firms, individual lobbyists and their clients. The register was a huge step forward when it was introduced in 2008, but remains frustratingly opaque. It doesn't tell us who is lobbying whom, about what, or when. Compare that with the ACT, where lobbyists are required to file quarterly reports on their activities, or NSW, where ministers are required to publish their diaries. The federal register is also completely blind to the activities of in-house lobbyists. What is the lobbyist code of conduct? The code tells lobbyists how they must behave when approaching the government and is designed to maintain ethical standards. But the code is not legislated and has no real teeth. It goes largely unenforced and the punishments are weak. The worst sanction available to authorities is removing a lobbyist from the register. The US and Canada have fines or jail terms for law breaches. Who keeps an eye on lobbyists? Federally it's the prime minister's department that loosely oversees lobbying. It takes on a largely administrative role, rather than an investigative or regulatory one. Its core job is to maintain the register and communicate the code's requirements to lobbyists. It lacks independence, relies on reports of bad lobbying and rarely, if ever, takes enforcement action.

The issue is not unique to the Nationals. The former Labor minister Stephen Conroy holds positions on the party’s national executive and Victoria’s public office selections committee, a preselection body to which he was recently re-elected.

Conroy is also the executive director of gambling lobby group Responsible Wagering Australia. He has breached no rules – either of the party, or federal or state lobbying codes – because he does not meet the definition of a lobbyist. Further, Conroy says he has not attended meetings of the executive or committee for about 18 months, instead using a proxy.

Ethical lobbying plays a critical role in the functioning of a healthy democracy, and there’s no evidence that the majority of registered lobbyists are doing anything other than complying with the rules.

If you don’t need to worry about the code and you don’t need to worry about the register, then you just skirt it George Rennie, academic

But anti-corruption advocates, integrity experts and former crossbenchers all agree that the rules are fundamentally flawed. Even the rules that exist, weak as they are, are rarely enforced, experts say.

The lobbying expert and University of Melbourne academic George Rennie said the sector in Australia was effectively self-policed. The absence of any real enforcement, he said, sent a clear message.

“The message that I think it sends to lobbyists is that you don’t really need to pay too much attention to the code, you don’t need to worry about the code,” Rennie told Guardian Australia.

“And if you don’t need to worry about the code and you don’t need to worry about the register, then you just skirt it. You don’t bother with it.”

Federally, the only window into the world of lobbying is the online lobbyist register, which lacks useful detail.

Unlike some state, territory and foreign jurisdictions, federally registered lobbyists have no obligation to tell the public who they have lobbied, on what subject and when.

Danielle Wood, a Grattan Institute researcher who has studied lobbying extensively, said this left Australians with “almost no visibility”.

“There is a case for much greater transparency: federal government ministers should be required to publish their diaries,” she said. “If there’s nothing to hide then the public would be reassured that their elected representatives are consulting widely, and if they aren’t, it places a pretty strong incentive on them to do better.”

Former independent senator Jacqui Lambie says the major parties don’t want to ‘change the rules’. Photograph: Mick Tsikas/AAP

Lobbyist registers, with the exception of Victoria, remain completely blind to the actions of in-house lobbyists, who are employed directly by large corporations and peak bodies to represent their interests to governments.

The former Tasmanian crossbencher Jacqui Lambie said that left hundreds of in-house lobbyists largely free to walk the halls of parliament without any semblance of scrutiny.

“There’d be hundreds of them. They’re not playing a straight bat,” Lambie said. “[The major parties] don’t want to know about it – Labor and Liberals are as bad as each other. They don’t want to change the rules because it hurts both of them.”

The federal code of conduct, designed to maintain ethical standards of behaviour, was introduced in 2008, championed by the Labor senator John Faulkner – despite stiff resistance from the lobbyist sector and many within parliament. It was a huge step up from an almost non-existent oversight regime.

The group that represents Australian lobbyists, both third-party and in-house, says it still supports the code. The Australian Professional Government Relations Association president, Les Timar, said the current regime was “characterised by high levels of compliance and operates effectively” to ensure basic ethical standards.

Timar does, however, want to see the code applied to everyone who approaches government, not just lobbyists. He also wants an improved sanctions regime for those who misbehave.

“We support these arrangements as a balanced and reasonable set of measures to provide public confidence in the activities of consulting government relations practitioners,” Timar said.

Independent oversight, monitoring and enforcement is urgently needed Serena Lillywhite, Transparency International Australia

“While APGRA does not agree with the suggestion that the code of conduct is weak or unenforced, there are some areas where the code of conduct could be improved.”

The code is not overseen by an independent agency. Federally, is it loosely governed by the Department of the Prime Minister and Cabinet, which takes on a largely administrative role and has been instructed by the government to deliver only a “low level of compliance activity”. Even if the department was to take enforcement action, it can do little more than remove a lobbyist from the register.

In the US, Canada and UK, authorities can issue fines for breaches of lobbying laws.

In Canada, an independent commissioner has the power to investigate potential breaches of the rules, including by third-party and in-house lobbyists. Punishments of fines or jail time can be handed to lobbyists who fail to register or who make false or misleading statements about their activities.

The US has strict, legally enforceable cooling-off periods designed to stop individuals leaving government and immediately lobbying their old place of work.

Australia has a 12-month cooling-off period for senior public servants and military figures, but it only bars them from directly lobbying on issues that they have had formal dealings with while in government. The ban is 18 months for ministers and parliamentary secretaries. There is no real significant punishment in the code for breaching the cooling-off period.

Transparency International Australia’s chief executive, Serena Lillywhite, said the existing checks and balances on lobbying were “insufficient”.

“Independent oversight, monitoring and enforcement is urgently needed in response to growing community concerns about undue influence and a decline in trust and confidence in government,” Lillywhite said.

“The cooling-off period for when a minister or parliamentary secretary can lobby in their portfolio should be extended to a minimum of two years, to bring Australia more in line with international best practice. But most importantly, this needs to be independently monitored and enforced.

“In general, greater transparency of meetings with government officials, politicians and ministers is needed – regardless of where these meetings take place.”

• This reporting is supported by the Susan McKinnon Foundation through the Guardian Civic Journalism Trust