A major police operation was sparked and ferries were cancelled after a man drowned in Sydney Harbour on Thursday morning.

A police spokeswoman said the man went overboard on a Sydney ferry near Manly just after 7.30am, and his body was recovered from the water shortly after.

Ferries between Rose Bay and Pyrmont were cancelled due to the police operation.

An ambulance spokesman said paramedics performed CPR, but the man could not be saved.