Prominent Tory right-winger Liam Fox has launched a stinging attack on the government’s immigration policy, saying that net migration is a “stupid measure” in determining who should come into Britain.

Speaking at a fringe meeting of the All Party Parliamentary Group on Migration at the Conservative Party Conference, the former Defence Secretary called instead for Britain to adopt an Australian points-based system that judges new arrivals on the skills they bring to Britain, a policy which has already been advocated by UKIP.

Prime Minister David Cameron pledged that the government would reduce net migration to the “hundreds of thousands” per year, but recent statistics have shown that the figures are actually rising.

Dr Fox added that those who come into Britain must also want to integrate into the country’s society and culture, but also called on the British people to show them a warm welcome.

In comments that will be interpreted as a coded message to potential UKIP voters, he also criticised policy makers for consistently ignoring the “cultural element” behind immigration, warning that if the government does not acknowledge the social and cultural changes newly arrived immigrants bring, it could allow “fringe parties” to “feed on irrational views”.

The former minister, who ran against David Cameron for the Conservative Party leadership as one of the main contenders on the party’s right, gave an example, stating that if someone in a village sees large number of immigrants coming in to pick fruit, they won’t care that a similar number of elderly people have moved to Majorca.

He added that “celebrating diversity” is all well and good, but there needs to be much more emphasis on what people have in common.

Dr Fox’s intervention comes amid rumours of further defections by Tory MPs to UKIP. Although the two MPs who have gone over so far have not played up to the issue, many former grassroots Conservatives cite immigration as one of the main issues that pushed them over to the Eurosceptic party.