The Turnbull government's top infrastructure adviser has put the need for better transport links between Sydney's CBD and Green Square among the country's top infrastructure priorities.

In heaping pressure on state and federal governments, Infrastructure Australia has warned the transport network between the CBD and south towards Sydney Airport will lack the capacity to handle 30,000 extra residents over the next two decades.

The government adviser has named as a top priority the need for a "high-capacity, rapid transport link which could be bus or light rail" between the CBD and the "unserved parts of the area" in the south.

It says the link could eventually be extended further to Mascot, Rosebery, Sydney Airport and Port Botany.