The JPMorgan boss Jamie Dimon has been blasted as “removed from reality” and “insulting” after he branded his own investors “lazy” for voting against his multimillion-dollar pay deal.

At a conference in New York on Wednesday, the chief executive of the giant Wall Street bank said stockholders who had followed the recommendations of two influential professional advisory services and rejected Mr Dimon’s $20m (£13m) pay package last week were “irresponsible” and “not good investors”.

Luke Hildyard, deputy director of the High Pay Centre think-tank in the UK, said the “lazy” shareholders were the ones who voted in favour of Mr Dimon’s pay deal. “He must be totally removed from reality if he thinks closer scrutiny of lavish executive pay packages will result in higher levels of approval for top pay – rather than higher levels of condemnation” he said. “There will be many active and engaged shareholders who feel extremely insulted by Mr Dimon’s suggestions of lack of professionalism,” said Sarah Wilson, chief executive of the Manifest proxy voting agency.

A spokesman for Pensions & Investment Research Consultants (Pirc), which itself advised clients to vote against Mr Dimon’s pay, drew attention to the regulatory fines paid by the bank in recent years: “Given JPMorgan has attracted around $40bn of fines since 2008, it is legitimate for shareholders to question whether they are getting value for money from the huge pay packages executives want them to approve.”

Thirty-eight per cent of JP Morgan shareholders rejected Mr Dimon’s pay package at the bank’s annual meeting on 19 May. ISS, the largest of the shareholder advisory firms, had urged a “no” vote on the grounds that the board had not linked Mr Dimon’s remuneration to performance and that 40 per cent of his pay was in cash. Glass Lewis, another advisory firm, had also urged a “no” vote.

“God knows how any of you can place your vote based on ISS or Glass Lewis,” Mr Dimon told the conference. “If you do that, you are just irresponsible, I am sorry. And you probably aren’t a very good investor either. I know some of you here do it because you are very lazy.” ISS declined to comment on Mr Dimon’s remarks; Glass Lewis could not be reached.

Another concern for investment advisers is that Mr Dimon occupies the roles of both chief executive and chairman at JP Morgan. Ms Wilson, of Manifest, said the blending of the two roles was a “red-line issue [for corporate governance] going back a quarter of a century at least”. She also suggested Mr Dimon’s latest outburst suggests he is out of touch with shareholders. ”Does he understand what they do or is he just listening to what his advisers are paid to tell him?” she asked.

Last week 36 per cent of shareholders also rejected the JP Morgan board’s wishes and voted in favour of installing an independent chairman after Mr Dimon retires.

Advisory firms were eager to rebut Mr Dimon’s suggestion that asset managers simply vote in the way consultants tell them to on matters of pay and governance. “Asset owners make the decisions; proxy advisers only make recommendations,” said a Pirc spokesman.