Liberal senator Eric Abetz wants the Federal Government to "provide some leadership" by slashing 4,000 jobs from the public sector.

The former Abbott government minister believes the public sector is too big.

"As a Liberal, I believe the Government should always be on the lookout to reduce the size of government and the size of expenditure," he said.

"One place where I believe the Government could provide some leadership is in the reduction of the public service."

Senator Abetz said the public sector had grown in recent years after it was reduced by more than 10,000 under his watch.

"Whilst I was the minister responsible, we were able to get a reduction of some 14,000 in the Australian Public Service through natural attrition," he said.

"Sadly, that has increased in recent years, and I believe that is one area where savings can be made without any need for legislation.

"Given that the growth has been around ... the 4,000 mark, I would like to think we can reduce the Australian Public Service by ... about another 4,000.

"[That] translates out to be about $400 million per annum each and every year."

Senator Abetz was the last Tasmanian to have a place at the cabinet table, before he was dropped when Malcolm Turnbull came to power.

Tasmania's peak business lobby called for his return in the recent reshuffle but he was again overlooked.

In a piece by News Corp, Senator Abetz is also making the case for a small claims-type tribunal to handle issues between the Australian Taxation Office and small businesses.

"The Australian Taxation Office has at its disposal an army of legal experts and a budget that can never be matched by small business," he said.

"What small business needs is [something] like a small claims tribunal to enable taxation matters to be resolved in a fair and amicable way where mistakes have occurred, where there is a small dispute."