Police have called off the search for a man who went overboard from the Spirit of Tasmania after medical experts advised it would have been impossible for him to survive.

Key points: Authorities say the man could not have survived

Authorities say the man could not have survived Passengers saw 45-year-old man go overboard

Passengers saw 45-year-old man go overboard Spirit of Tasmania I and II delayed

Passengers on the Spirit of Tasmania I saw the 45-year-old Ballarat man go overboard about 11:40pm, when the ship was about 43 nautical miles south-east of Victoria's Port Phillip Heads.

The ship was travelling from Melbourne to Tasmania, but by the time it turned around, there was no sign of the man, who is yet to be identified.

The Australian Maritime Safety Authority (AMSA) said the search was suspended at 12:00pm, based on expert medical advice that the man could not have survived past that time.

The water temperature in the area was about 12 or 13 degrees, with waves of up to two metres and winds of up to 25 knots.

Victoria Police are investigating with the assistance of Tasmania Police but the death is not being treated as suspicious.

Sergeant Paul Bartlett, from Victoria's Water Police, said police were examining clothing and identification found by merchant vessels in the search area.

Sergeant Bartlett said survival time would have been "very limited in [the] conditions".

"Hypothermia would limit their capability of survival," he said.

AMSA coordinated the search with assistance from Victoria's Water Police and Police Air Wing before it was called off.

Cargo ships, ferries joined search to no avail

The Spirit of Tasmania II, which was heading to Melbourne, joined the search this morning, along with two cargo ships, Sea Road Mersey and Victoria Reliance.

The ships were released from the search, with the Spirit of Tasmania I and Spirit of Tasmania II continuing their journeys to Devonport and Melbourne.

In a statement on its Facebook page, Spirit of Tasmania said the arrivals of both ships would be delayed due to the incident.

"Tonight's sailings from Melbourne and Devonport will operate as scheduled," the statement said.

"If, in the unlikely event, there are any changes to tonight's sailing schedule, we will notify passengers directly via SMS or email."