Pakistan's prime minister hit back today at remarks by David Cameron linking the country to the export of terrorism.

Yousaf Raza Gilani, the normally conciliatory premier, used a speech to make the highest level response from Islamabad so far to Cameron's comments during his trip to India. Reports suggest that an official from the British high commission in Islamabad, possibly the deputy chief of mission, will be summoned tomorrow by Pakistan's ministry of foreign affairs for a formal dressing down.

Gilani's intervention follows the abrupt cancellation by Pakistan's spy chief, General Shuja Pasha, of a planned visit to the UK for talks with his British counter-terrorism counterparts.

Co-operation from the Inter-Services Intelligence (ISI) agency, headed by Pasha – which was accused of aiding the Taliban in the Afghan war logs published last week by WikiLeaks – had previously been presented as being crucial to stopping numerous terrorist plots aimed against Britain.

There are fears that a long-planned visit to the UK this week by Pakistan's president, Asif Zardari, could be overshadowed by growing anger at Cameron's remarks among the one million people of Pakistani origin living in Britain. Media outlets and opposition politicians in Pakistan are urging the president to cancel the trip, while demonstrators burnt an effigy of the prime minister on the streets of Karachi.

There is particular anger, shown by Pakistanis yesterday in burning an effigy of the prime minister, that Cameron made the comments on a trip to India.

Gilani focused on the issue in today's speech in Punjab province. "In India, he [Cameron] has given a statement that we in Pakistan promote terrorism," he said. "We want to say to him, we've had good relations with you for 60 years."

He contrasted the issue raised by Cameron with the situation in Kashmir, the Himalayan region mostly held by India, which has been in open rebellion for 20 years. "In India, you [Cameron] talk about terrorism but you don't say anything about Kashmir. You forgot about the human rights abuses going on there. You should have spoken about that too, so that we in Pakistan would have been satisfied."

While Pakistan has frequently been asked to do more in the battle against extremists, Cameron's remarks are seen in Pakistan as going further than any western leader in criticising the country's record and commitment.

An editorial in Dawn, Pakistan's leading English-language daily, said: "No one, with the exception perhaps of New Delhi and Kabul, had ever accused Pakistan of exporting terrorism. In doing so, was Mr Cameron attempting to bracket Pakistan with countries that have been or still are anathema to the west?"

An officer at the ISI said: "Do you make such remarks when visiting a third country, a country we consider an enemy? It was done to appease [India]. You can sit in England and say what you want, but sitting in India gives it a completely different connotation."

A senior Pakistani civilian official said: "Cameron's remarks show a political immaturity, lack of foreign policy experience – and talk about a choosing a bad venue to deliver the message. Being the youngest British prime minister in two centuries isn't necessarily an advantage."

The Cameron intervention came as Pakistan was reeling from the disclosures in the US intelligence documents made public by WikiLeaks. The apparent evidence of ISI collusion with the Taliban from the WikiLeaks material had already been seized on with glee by Indian officials, as confirmation of New Delhi's charge that the Pakistani state sponsors terrorism.

The shadow foreign secretary, David Miliband, said: "Diplomacy is about making friends and influencing them. Today's announcement by the ISI sadly proves that Cameron has failed to make friends and failed to influence them. We need to support Pakistan's intelligence services, not undermine them – their work protects the people of Britain as well as the people of Pakistan. We have a strong Pakistani community in Britain and we have troops in Afghanistan – the stakes are simply too high to go hunting headlines with thoughtless remarks.

"We need a prime minister that understands the complexity of diplomacy and so far Cameron has failed to prove himself as the standard-bearer we need around the world."

Travel expert Riaz Dooley, who has worked to encourage British Asians to take a greater interest in political life, warned that Cameron risked alienating British-Pakistanis. He said: "David Cameron is going to lose the Pakistani vote over this, because he has not apologised. It is not fair to say that Pakistan promotes the export of terrorism, he doesn't have any proof."

Labour MP Khalid Mahmood agreed that the Pakistani community in the UK was angry about Cameron's comments. He said the prime minister had failed to reflect how much the country had sacrificed in the war on terror. "They have taken a huge amount of casualties in the north-west province and there have been a huge number of bombings in Pakistan.

"They have suffered enormously in terms of their own people's lives and to suggest this counts for nothing is very, very insensitive."