A Japanese submarine has entered Sydney Harbour for the first time since World War II, just weeks before the Federal Government is tipped to announce which country will build Australia's future subs fleet.

With very little fanfare, JS Hakuryu sailed through the heads about 11:00am, accompanied by two Japanese Maritime Self Defence Force warships and led by HMAS Ballarat.

The Soryu class Hakuryu is the first Japanese submarine to enter Sydney Harbour in three quarters of a century, and will take part in Exercise Nichi Gou Trident with the Royal Australian Air Force and Royal Australian Navy.

"This exercise, which has been conducted between Australia and Japan since 2009, is an opportunity to develop and enhance the bilateral naval relationship by practising maritime skills and improving levels of interoperability between our two navies," the Defence Department said in a statement.

"This is the first opportunity to conduct the exercise off Sydney."

Japanese midget submarines were retrieved from Sydney Harbour in 1942. ( Ronald Noel Keam (060696): Australian War Memorial )

The Defence Department said the joint exercise aimed to "improve interoperability and mutual understanding" between the two navies.

HMAS Ballarat, HMAS Adelaide and HMAS Success will take part in the exercise, along with aviation elements including Navy's 816 Squadron's S-70B Seahawks and the Air Force's AP-3C Orion and Hawk 127 aircraft.

Japanese officials were also using the opportunity to show off the capability of their high-tech Soryu class submarine that Tokyo hopes would be selected as the preferred model for Australia's future submarine fleet.

Japan is locked in a Competitive Evaluation Process with France and Germany to decide who will be selected for the lucrative $50 billion defence contract.

Japanese midget subs sunk in 1942

The Soryu class Hakuryu will take part in an exercise with the Australian Navy.

The historic arrival of the Japanese submarine has revived memories of 1942, when then-Imperial Japan famously slipped three midget subs into Australian waters, attacking Sydney and Newcastle and sinking the converted ferry HMAS Kuttabul, killing 21 sailors.

Jean Nysen, 93, was then serving in the Women's Royal Australian Navy (WRANS).

"It was a terrible shock and we couldn't do anything about it," she said.

"They were building the boom net and they were preparing people for any invasion."

HMAS Kuttabul sits partly submerged in the waters of Sydney Harbour at Gardens Point on June 1, 1942. ( Australian War Memorial/ID: 012422 )

Interior of the top deck of the HMAS Kuttabul, damaged during the attack in Sydney Harbour on the night of May 31, 1942. ( Australian War Memorial/ID: 012423 )