Rapid growth on the fringes of Sydney and Melbourne threatens to add billions of dollars to the cost of building a high-speed rail line between the two cities, Infrastructure Australia warns.

The federal government's infrastructure advisory agency says high speed trains could be running between Canberra and Sydney within 15 years, but governments must act quickly to avoid potential cost blowouts.

A report to be published on Friday by Infrastructure Australia says the governments of NSW and Victoria should get in early and buy land along the proposed rail corridor now, at an estimated cost of $720 million, or pay more than $3.5 billion later.

The agency says a failure to protect the corridor within the next three to five years could do serious harm to a project that will become vital given the populations of Melbourne and Sydney are projected to double by 2060.