If Major League Baseball wants to expand to 32 teams, it’s going to have to get more people interested in the 30 it already has.

During a meeting with sports editors from the Associated Press last week, Major League Baseball Commissioner Rob Manfred suggested that “multiples of fours just work better than multiples of fives” for scheduling league games. He also indicated that such expansion wouldn’t be immediate, but that “if we were to expand, I do think a city that makes sense geographically — meaning in terms of realistic travel distances and is outside of the 48 contiguous states — would be a positive choice for us in terms of growing the game.”

We understand the excitement on the part of the league. It just held a successful exhibition between the Tampa Bay Rays and the Cuban national team in Cuba, its World Baseball Classic is coming up again next year and nearly a quarter of MLB players come from outside the U.S. Meanwhile, Montreal, which lost the Expos franchise to D.C. in 2005, drew 53,000 fans to watch the Boston Red Sox play the Toronto Blue Jays during spring training. Montreal even has a group of supporters pushing for a team, if not a viable home field in Olympic Stadium. One of those supporters? Manfred himself.

“ Major League Baseball averaged roughly 700,000 viewers per game in 2013, or about half of the 1.4 million who watched National Basketball Association games. ”

“I think, even if I didn’t want to say this, the mayor of Montreal would probably tell you, if you walked past him on the street, that I have met with him on a number of occasions,” Manfred said. “They have expressed a strong desire to have Major League Baseball back in Montreal. That’s a good thing. People want baseball.”

That level of enthusiasm for baseball hasn’t been all that easy to come by in the U.S. Last year, five of baseball’s 30 teams — the Chicago White Sox, the Oakland Athletics, the Miami Marlins, the Cleveland Indians and the Tampa Bay Rays — drew less than half of the 48,000 Montreal has been averaging for spring-training games. Three of those teams — Atlanta, Tampa and Cleveland — averaged crowds of half or less of their stadium’s capacity during home games last season.

Are they watching at home? Not really. According to Nielsen, Major League Baseball averaged roughly 700,000 viewers per game in 2013, or about half of the 1.4 million who watched National Basketball Association games and just above the 500,000 who watched National Hockey League games. Also, as Nielsen points out, those who do watch aren’t exactly an energized young base. Some 76% of baseball viewers in the U.S. are 35 or older. Half of baseball’s total television audience is older than 55.

When the Kansas City Royals beat the New York Mets in the World Series last year, an average of 14.7 million viewers tuned in. That’s not only the fifth-lowest viewership in World Series history, but it’s also well shy of the average 19.9 million viewers who watched the NBA Finals last year. Then again, 45% of NBA fans are under 35. We aren’t including the average audience for a National Football League regular-season game on network television (20.7 million) or the Super Bowl (111.9 million) because they just aren’t much of a comparison.

Though youth sports participation in the U.S. is down across the board, baseball is becoming as inaccessible as costlier sports including hockey and lacrosse. Dave Zirin at The Nation pointed out last year that disappearing ballfields and urban baseball programs have sent baseball’s participation numbers (5.6 million kids in 2012) below those of youth soccer (6.6 million) and basketball (6.95 million).

Are Bryce Harper and Mike Trout sliding head first into danger?

Meanwhile, ballparks are a costly proposition for cities where Major League Baseball attendance is light. Spending $634 million in public funding (not counting bond interest) on Marlins Park in 2012 hasn’t brought Miami out to the ball game, with last year’s average attendance of little more than 21,632 in the league ranking third to last. Manfred notes that “stadium issues in Tampa Bay and Oakland would need to be resolved” before any expansion plans proceed.

The Rays, which have had notoriously terrible attendance since being introduced as MLB’s last expansion team in 1998, were locked into their lease at Tropicana Field until 2027 before the St. Petersburg city council gave them a buyout option. The team is now searching for a new ballpark site in nearby Pinellas or Hillsborough, but it’s unclear how much, if any, public money they’d be able to scrounge up for a new building. So far, the plan to displace residents of low-income housing and demolish a school for the stadium site haven’t been promising (or palatable). Less encouraging, however, have been half-hearted proposals to share the Rays, snowbird-style, with Montreal and other ideas about moving the team north of the border altogether.

The Athletics, meanwhile, are still searching for a stadium site in Oakland. They’re in no real rush, as their lease at Oakland Coliseum runs until 2024, but with the San Francisco Giants holding the rights to San Jose and much of Silicon Valley, the A’s have few options available to them. In the worst-case scenario, the A’s can hope that the disgruntled Oakland Raiders leave for Los Angeles, Las Vegas, San Antonio or wherever else they feel they can get a better deal. It wouldn’t give the A’s a new stadium, but it would allow them to finally get rid of the “Mount Davis” tiers of seats in center field, which blocks the team’s lovely view of the nearby hills that it had until the Raiders returned in 1995 and started calling the Coliseum their own.

In the meantime, dealing with those two franchises and siphoning some younger blood into baseball’s fan base should keep Manfred from daydreaming too heavily about opening day in Montreal, Havana or San Juan. However, with young stars like the Rays’ presidentially approved pitcher Chris Archer, Toronto Blue Jays slugger and Goose Gossage nemesis Jose Bautista and Washington Nationals masher and youth movement leader Bryce Harper already making Manfred rethink baseball’s stodgy unwritten rules, the game may revive itself quicker than any change of scenery ever would.

Jason Notte is a freelance writer based in Portland, Ore. His writing has appeared in The New York Times, The Huffington Post and Esquire. Notte received a bachelor’s degree in journalism from the S.I. Newhouse School of Public Communications at Syracuse University in 1998. Follow him on Twitter @Notteham.