Mayor John Tory says he's spoken with the prime minister and the premier about Toronto's potential bid for the 2024 Summer Olympics, even as councillors in a potential rival city, Los Angeles, vote on whether to submit a bid.

To start the bid process, Tory would would have to submit a letter to the International Olympic Committee (IOC) expressing Toronto's interest in hosting the Games. While some councillors are skeptical of a proposed bid, Tory is mulling the idea and is still gathering information.

"I hope to be in a position … to have a reasonably complete summary of all that information at my disposal when it comes time to make a decision on whether to submit a letter or not," Tory said.

Tory, speaking with reporters Tuesday, said he spoke with Prime Minister Stephen Harper about a potential Olympic bid in the lead-up to this summer's Pan and Parapan Am Games. He said he hopes to discuss the topic with the other federal party leaders, Tom Mulcair and Justin Trudeau, as the city will need to rely on whoever wins the federal election.

Tory also said city council will get a chance to debate the idea and vote on whether or not to support a bid.

That bid alone could cost the city $50-$60 million, according to a city feasibility report published last year. The budget for the Games — which in Toronto's case could include a brand new stadium — would cost billions.

Los Angeles, whose council will vote Tuesday on whether or not to bid for the Games, put forward a $4.1-billion proposal.

Other potential rivals include a joint bid from Rome and Paris, as well as Budapest, Hungary and Hamburg, Germany.

Councillors stake positions on Games

Tory said most of his meetings about the Olympic bid remain informal at this point, though he has also met with Ontario Premier Kathleen Wynne on the matter and plans to meet with union, business and youth leaders this week to get their opinions on the Games.

The problem with Olympics … is that you can't predict the cost overruns. - Coun. John Campbell

There hasn't been a major debate about a potential Games bid at Toronto city council, but that hasn't stopped several councillors from staking positions.

Coun. Glenn De Baeremaeker said he supports an Olympic bid because of what it could mean for infrastructure investment in the city

LISTEN | Toronto councillors square off on Olympic bid

The Olympics, and other similar events, are "a catalyst for major infrastructure projects and city building projects," he told CBC Radio's Metro Morning.

De Baeremaeker said he's confident Toronto can learn from successful host cities, like Vancouver, and deliver a Games on budget.

But Coun. John Campbell, who called the Olympic idea "folly," said hosting the mega-sporting event is too risky for the city.

"The problem with Olympics … is that you can't predict the cost overruns," he said.

Campbell said he'd "have to see some pretty solid numbers," from other governments and corporate sponsors before backing a bid, and even then he's concerned Torontonians would be on the hook for a huge percentage of the costs.

Mississauga to debate Toronto bid next week

Mississauga Mayor Bonnie Crombie, meanwhile, said in a statement her city will not support a Toronto bid until studying a "comprehensive business case."

Crombie said Mississauga city council is set to debate the issue on Sept. 9.

"Hosting the Olympics is an enormous responsibility which will have lasting implications for communities across the Greater Toronto and Hamilton Area — long after the games have concluded. I need to know what those implications will be for Mississauga."

Gord Krantz, the Town of Milton's mayor, said his town is ready to welcome an Olympic crowd. The town is home the Mattamy National Cycling Centre, which features a brand new velodrome that was one of the most popular venues during the Pan and Parapan Am Games.