Scott Morrison talks of his ancestor’s arrival on the continent, and defends celebration of Australia Day

Scott Morrison has said 26 January 1788 was “pretty miserable” for his ancestor, in a speech defending the celebration of Australia Day, while tens of thousands of people joined Invasion Day marches around the country calling for the public holiday to be abolished.

Morrison told a citizenship ceremony in Canberra that his fifth great grandfather, William Roberts, arrived with the first fleet in a group that was “wretched, naked, filthy, dirty, lousy, and many of them utterly unable to stand, or even to stir hand or foot”.

Six hundred kilometres away, in Melbourne, his comments were echoed to a crowd of more than 40,000 people who congregated on Spring Street outside Parliament House before the Invasion Day march.

“Those poor people were in chains, they were suffering we pitied them,” a speaker said. “What are you defending?”

The rally in Melbourne followed a dawn service at the memorial marking the burial site of 38 Aboriginal people from across Victoria, who represented those killed in the frontier wars.

“If this country has any conscience, it will start a healing process that starts with truth-telling,” the former Greens MP Lidia Thorpe, a Gunnai-Gunditjmara woman, told the rally.

Facebook Twitter Pinterest People take part in an Invasion Day rally in Sydney on Saturday. Photograph: Joel Carrett/AAP

“You fellas come out one day of the year, which is fantastic, but we feel this every day. And we need you every day … it is upon you from today to decide whether you want to come out once a year and protest or you act every single day to change what this country is about.”

The crowd estimates ranged from 5,000, an early figure given by Victoria Police, and 80,000, cited by organisers. At its longest point, it stretched two blocks down Swanston Street, from Bourke Street to Flinders Street station.

Large protests were also held in Sydney, Brisbane, Canberra, Hobart and Perth, and survival day events in Adelaide and Darwin.

The academic and activist Gary Foley told the crowd, the majority of whom were non-Indigenous, not to go away feeling “self-satisfied”.

Ella Archibald-Binge (@EllaMareeAB) Difficult to capture the masses of people at Brisbane #InvasionDay rally, starting from new location to accommodate crowds. #AlwaysWillBe pic.twitter.com/2lhRi18eQN

He said that if everyone at the march read up on the history of Australia and then told 10 more people, “next year we would have half of Melbourne on our side”.

The march organiser Meriki Onus urged the crowd to stay behind strict marshal lines and not engage with a reported counter-protest run by far-right groups in Federation Square.

That protest had about 10 participants and was dispersed by police before the main march turned down Swanston Street.

Two people, a man draped in an Australian flag and a woman holding a sign that said “To defend my country was once called patriotism; now it’s called racism” remained at the Flinders Street station steps.

Kaitlyn Offer (@KaitlynOffer) These two were also moved on from the #InvasionDay rally. Her sign reads “To defend my country was once called patriotism now it’s called racism” pic.twitter.com/60IS2kfnvV

They were surrounded for about 20 minutes by marchers chanting “go home, fascists”, before the police had them move on on. A spokeswoman for Victoria Police said it was too early to comment on whether any charges were being laid.

Tens of thousands of people also attended a rally in Sydney, where the early crowd estimates were put at up to 50,000, while thousands more marched in Brisbane, Canberra, Hobart, and Perth.

Alex McKinnon (@mckinnon_a) Can't see beginning or end of Sydney's #InvasionDay march pic.twitter.com/FoocmvhlVZ

The protest numbers in all capital cities were reported to be the largest yet.

In London, activists hung a banner from Westminster Bridge, which said “Abolish Australia Day”; in Berlin a protest was planned.