Police have dragged a man's body from the water by Sydney Harbour Bridge, after a cold night in the city.

Passersby spotted the man, believed to be aged in his late 20s, floating in the harbour by Jeffrey Street Wharf at Kirribilli, about 7.30am on Saturday. Water police pulled his remains from the water near a popular walking path, on the eastern side of the Harbour Bridge.

"He had on a high-visibility top and what looked like work boots and work trousers underneath," said a cameraman at the scene, who declined to be named. "He was face down in the water as police dragged him around and pulled him into the back of their boat."

The man's body was taken by ambulance to Glebe Morgue for further investigation. Police are not treating the death as suspicious.

The air temperature at Sydney Harbour, near where the body was found, dropped to 9 degrees overnight.