Friends and family are mourning four members of the same West Australian family who were lost at sea off the coast of Perth last week after what police suspect was a quick and catastrophic failure.

Key points: Rescuers recovered two bodies from the water near Garden Island after a fishing trip

Rescuers recovered two bodies from the water near Garden Island after a fishing trip A relative says it has left "huge holes in our family", with no sign of the two other men

A relative says it has left "huge holes in our family", with no sign of the two other men The search has been downgraded by WA Police to a recovery operation

The men, three of them fathers, set off from Woodman Point, south of Fremantle, on a 5.5-metre boat on Wednesday for a fishing trip to Rottnest Island.

They had been due back on Thursday evening but, in what relatives have described as "every family's nightmare coming true", they never arrived.

Police are yet to release the names of the men, but they are understood to have been 50-year-old Uock Pham, his brother Tuan, his 24-year-old son Jacob and 32-year-old future son-in-law Justin O'Neill.

A large air and sea search was launched and late on Friday police pulled two bodies from the water west of Garden Island, about 5 kilometres off the coast.

The men's utility and boat trailer were found at the boat ramp at Woodman Point. ( ABC News: Samia O'Keefe )

According to relatives, the bodies belonged to Jacob Pham and Mr O'Neill.

In a bid to find the remaining two men, dozens of boats scoured more than 400 square kilometres of ocean over the weekend, while aircraft canvassed an area three times that size.

Sorry, this video has expired Search underway on Friday for men who failed to return from Rottnest fishing trip

But other than an esky lid and a backpack, there was no sign of the men or their boat, named "Yeah Buoy".

Police said a scaled-back search would continue this morning on the western side of Garden Island with one water police vessel and one fixed-wing aircraft, and would "be re-evaluated as the day progresses".

Tributes to two men after bodies found

A GoFundMe page was set up yesterday by a relative to raise money to help support the men's families and pay for funeral costs.

On the page, Mr O'Neill was described as a much-loved father and partner to his fiancee and three sons aged under 10.

Jacob Pham, 24, was labelled "the sweetest young man" who left behind "an adoring mother" and two sisters.

"Both losses leave huge holes in our family which can never be filled," the relative said.

The family had been holding out hope for Uock and Tuan Pham, but it is now believed they too have drowned.

Police recovered two bodies west of Garden Island on Friday. ( ABC News: Eliza Laschon )

'We are now in a recovery phase': police

WA Police Inspector Sean Togher said on Sunday the search operation was shifting from a rescue phase into a recovery phase.

"At this stage, given the passage of time and the conditions, it's been determined those persons are most likely, if not probably, deceased," he said.

"We are now in a recovery phase that will continue [as] a search operation with WA Police assets alone.

"Family members have been advised of this occurrence. We thank the family for their assistance in this matter and we express our condolences for their loss."

Other than an esky lid and a backpack, police have found no trace of the men's boat. ( Supplied: WA Police )

Inspector Togher said the fact there had been no sign of the vessel and there had been no emergency calls from those on board, indicated something went quickly and catastrophically wrong.

"The fact that they had safety equipment on board and we haven't had any activations, no sighting of flares, and there's no other indications, it would tend to suggest that something's happened quite suddenly," he said.

He said depending on the depth of water and ocean currents, it was possible the men and their boat may never be found.