A Labour government would make values, not numbers, the driving force behind immigration policy, and challenge the “edge of fear” that has characterised the debate for decades, according to the shadow home secretary, Diane Abbott.

“What I would say is different about Jeremy [Corbyn] – and myself as home secretary – is we want to talk about values,” she said. “Because it flows from your values, actually, if you hold them genuinely, what sort of immigration system that you have.”

Abbott spoke to the Guardian at her home in Hackney, the day after Labour’s manifesto launch in Birmingham, at which Corbyn unveiled plans for a dramatic increase in public investment.

With less than three weeks to go until polling day, she welcomed newspaper headlines calling it Labour’s most radical manifesto for decades. “The Labour party’s under new management,” she grinned.

Photograph: Guardian

She says fairness, and the importance of keeping families together, will guide the party’s approach.

Labour will ditch rules that mean in some cases the partner of a British resident can be deported if they don’t meet an income threshold, for example.

Abbott would not to be drawn about the practical details of a new “work visa” approach mentioned in the document; but says she and Corbyn believe it is time to change the national conversation about migration, on the left as well as the right.

“All my life, the political debate around migration has had an edge of fear to it,” she said. “The first big political debate on migration I remember, was the debate around the east African Asians: and that was Callaghan.”

As Labour home secretary in 1968, under pressure from opposition figures including Enoch Powell, Callaghan introduced legislation to curb migration from east Africa.

Callaghan claimed at the time, the “origin of this bill lies neither in panic nor prejudice, but in a considered judgement of the best way to achieve the ideal of a multiracial society”. But Abbott said he was the first of many Labour politicians to mishandle the issue – including Corbyn’s predecessor Ed Miliband, whose 2015 general election campaign produced red mugs bearing the slogan “controls on immigration”.

“What it is, the people who say, ‘ooh, you know, migration is very difficult, people say this or that on the doorstep’ – they really influenced poor Ed,” she said. “I don’t think that was Ed, as a person. But you had the immigration control mugs; you had a whole party political broadcast on migration, which I’ve not seen before or since. He also made at least two major speeches on migration.”

She added: “It’s like benefits. If the Labour party’s going on and on about benefits scroungers, people say, ‘oh, benefits scroungers, what a problem’. But if the Labour party dials it down, it polls as less of an issue.”

Abbott argued that the Windrush scandal, and the wave of public outrage it provoked, created the opportunity to have a different kind of debate, based on fairness.

‘I don’t think that was Ed, as a person’ – Diane Abbott. Photograph: Labour

“I think that the Windrush scandal, and even the fact that the Tories had to say, ‘this is awful’, showed that the British people are better than the tabloid newspapers,” she said. “And I think the Windrush scandal gives you a chink of light, where you can start to explain to people.

“I think it is possible to make the case that these are human beings, and we do have to treat them fairly, and have a system that reflects our values. Not least because the existing system, which is all about numbers, has failed. The Tories have never once met their migration target.”

Conservative manifestos have repeatedly promised to reduce net migration – the number of people who arrive, minus the number who leave, each year – to less than 100,000. Boris Johnson has now ditched that aspiration, and is promising to create an “Australian-style points system”, though details are scant.

After a radical pro-migration motion was passed at party conference, the Conservatives had relentlessly attacked Labour as preparing to ditch any controls on immigration at all.

Q&A How does an Australian 'points style' immigration system work? Show Aside from temporary work visas, such as the one-year version used by many young Britons and others to work in lower-skilled jobs such as agriculture and hospitality, Australia has a strictly capped limit for permanent migration of 160,000. Of these spaces, about a third are intended for those moving for family or humanitarian reasons, with the rest based on skilled migration. There are two types of permanent visa – people who are sponsored by a specific employer, and those who apply independently. Both routes require the applicant to accrue 65 points to move. Points comes from a range of factors, among them: Age: These range from 25 points for those aged 18-24, to zero if you are over 50.

These range from 25 points for those aged 18-24, to zero if you are over 50. English language skills: People receive 20 points for 'superior' skills, ten for proficient, and none for 'competent'.

People receive 20 points for 'superior' skills, ten for proficient, and none for 'competent'. Work experience: More time in a particular occupation brings more points, with experience in Australia counting more than that from overseas.

More time in a particular occupation brings more points, with experience in Australia counting more than that from overseas. Qualifications: Again, higher qualifications bring more points, up to 20 for a doctorate.

Again, higher qualifications bring more points, up to 20 for a doctorate. Relevant skills: Gaining entry requires your jobs to be on the so-called skilled occupation list, a lengthy collection of jobs with shortages, and various classes of visa for which such a job allows people to apply. Some critics of the Australian system argue it is a blunt tool, and that the strict points-based criteria can mask other skills and qualities. The cap can also cover a significant backlog in processing people. However the idea of introducing a similar system in the UK post-Brexit has been repeatedly floated by prominent Leave supporters like Boris Johnson and Nigel Farage. Peter Walker, political correspondent

Labour strategists were anxious about the impact of the party appearing too liberal, and Unite’s general secretary, Len McCluskey, insisted the party must address voters’ worries about an influx of cheap migrant workers.

“If we don’t deal with the issues and concerns, we will create a vacuum that will be filled by a far right seeking to become the voice of the white working class,” he warned.

Abbott said: “Len’s entitled to his views, and he does represent a thread of thinking in the Labour party.” She acknowledged that the manifesto does not go as far as some of the campaigners who brought the free movement motion to conference had hoped.

The existing [immigration] system, which is all about numbers, has failed Diane Abbott

A carefully drafted passage leaves much of the detail of the future migration regime to be determined after the issue of Brexit has been resolved.

“If we remain in the EU, freedom of movement will continue,” it says. “If we leave, it will be subject to negotiations, but we recognise the social and economic benefits that free movement has brought both in terms of EU citizens here and UK citizens abroad – and we will seek to protect those rights.”

The party would also close down two of the most notorious migrant detention centres, Yarl’s Wood and Brook House, and review the future of the others. “I think we’re acting in the spirit of those party members that voted for that composite, who want to see a fairer migration system,” Abbott said.

She has held three meetings in recent days with senior civil servants preparing for the possibility of a Labour government – standard practice during general election campaigns. “They want to know whether we really mean it or not - and I said, ‘yes, we are going to close Yarl’s Wood down: we really mean that.’”

Labour remain well behind the Conservatives in the polls, but Abbott insisted her longtime political friend and ally Corbyn is ready for Downing Street. “I think in the debate, with Boris, he looked very prime ministerial,” she said.

“This man will be PM in a few weeks: that’s what I thought.”