One man has died and another is in hospital after their boat overturned near Carnac Island south of Perth this morning.

An operation was launched to rescue the men after an emergency beacon was activated about 10:00am today.

When police arrived, they found the 3 metre fibreglass boat overturned, with only the bow protruding from the water.

Two men were recovered from the water.

A police rescue boat at the site where the boat sank soon after activating an emergency beacon. ( ABC News )

Police said a man in his 60s died, while a man in his 30s was taken to Fiona Stanley Hospital for medical treatment.

Carnac Island sits just a few kilometres off the coast of Fremantle, north of the Garden Island naval base.

Sergeant Rebecca O'Keeffe said the conditions on the water were quite chaotic, and the boat may have been swamped by a wave.

"We believe the men were crayfishing at the time, they were pulling in pots so at this stage we believe it may be due to the swell," she said.

"It is a 2 to 3 metre swell out there, so while it looks like perfect conditions for boating there are dangers out there."

Sergeant O'Keeffe said it did not appear the men had been wearing life jackets, and they may have been in the water for a couple of hours before the beacon was activated.

"I suppose when these things happen shock sets in, obviously the surviving person was trying to help his friend," she said.