Prime Minister Tony Abbott has lost a bid to force controversial union official John Setka to pay thousands of dollars in legal fees over a preliminary issue in their defamation case.

Mr Abbott's lawyers had applied to the Victorian Court of Appeal to order Mr Setka to immediately pay legal costs incurred over the union official's failed bid to have part of the Prime Minister's defence against the defamation action struck out.

Tony Abbott addresses the meeting of the Master Builders Association in 2012. Credit:Michael Clayton-Jone

But the Court of Appeal's Chief Justice Marilyn Warren and justices David Ashley and Simon Whelan on Thursday decided against awarding costs at this early stage of the proceedings.

A date for the civil trial has yet to be set in the defamation case in which Mr Setka claims Mr Abbott portrayed him as a thug.