Alex Honnold, the world’s most famous climber, is ambivalent about his sport’s inclusion in the 2020 Tokyo Olympics. When the climbing world descends upon Japan in four years, he said he’ll probably be somewhere far away on a solo ascent. “I wouldn't even be able to qualify,” he told Reuters. "Competitive climbing is basically a whole different sub-sport.”

Honnold is not the only elite athlete torn about his sport’s ascent to the Olympic stage. The International Olympics Committee announced Wednesday that five new sports would be included in the 2020 Games, including baseball, karate, skateboarding, and surfing. And although the sports’ governing bodies were euphoric, athletes were already grumbling.

Adam Ondra, winner of multiple climbing World Cups, told Epic TV that he “will have to think a lot about participating or boycotting it.” Owen Wright, a surfer known for his competitive edge, was more concise: “Fuck the Olympics,” he said. (Later, he added, “I think surfing in itself is more of an art form and an expression so I think the Olympic banner doesn’t really suit the sport of surfing.”)

To be fair, plenty of the world’s greatest climbers and surfers are backing the games. Kelly Slater, who will be 48 in four years, says he wants to compete and Brazilian Gabriel Medina is on board, too; free climber Sasha DiGiulian expressed glee on Instagram and bouldering champion Shauna Coxsey says she hopes to make the team; and all of the sports’ governing and competitive bodies say the publicity will help grow the sports.

Broadly, there are two categories of complaint from cynics. The first is spiritual, and you’ve heard it before: both surfing and climbing are sports in name only to many of their practitioners. Furthermore, the sports’ inclusion on the world’s biggest stage takes away any remaining outsider cred they may have still had. “It was a matter of real pride, a near universal thing, that we surfers liked being different from other sports,” says surfing’s unofficial historian Matt Warshaw.

But the International Olympics Committee (IOC) probably doesn’t care much about that. In its press release announcing the new sports, it said the decision marked “a historic step in bringing the Games to young people.” To that end, it’s the second concern that’s a bigger problem: the rules and logistics that will govern each sport’s competition at the games are set up to minimize what makes the sports exciting in the first place.

In climbing, three different categories of competitive climbing will be judged—bouldering, lead climbing, and speed climbing—and each will be weighed equally. This doesn’t sit well with athletes like Ondra. “I think speed climbing is kind of an artificial discipline,” he says, noting that speed climbers everywhere practice on standardized routes with the same holds. Speed climbing “doesn’t have much in common with the climbing philosophy in my opinion,” he says. “Anything would be better than this kind of combination.” Chris Sharma, another top climber, told The Ledge podcast that he’s excited for his sport’s elevation but that judging the event on three formats equally was “a big shame.” In other words, the sport’s biggest names—and best athletes—will only be good at a portion of the event.

For their part, surf competitions have always lived and died by the conditions in which they are held. It “only works—and it still has big problems here—when the surf is really good or really big,” Warshaw says. As much as the host nation will try to control every conceivable aspect of the 2020 Tokyo games, no one can control the weather. The IOC consulted with surf forecasting site Surfline, who predict the waves will be in the thigh- to chest-high range; certainly not ideal conditions to showcase the best surfers talents.

There was talk of using a wave machine, like Kelly Slater’s beauty, to ensure the competition was held in quality surf, but Fernando Aguerre, head of the International Surfing Association, says that “Tokyo told us we don’t want the wave pools—for us it’s going to be in the ocean.”

But beyond the concerns of athletes and fans, the very reason the IOC brought these sports into the fold is flawed. In a press release, IOC President Thomas Bach said, “We want to take sport to the youth. With the many options that young people have, we cannot expect any more that they will come automatically to us. We have to go to them.”

But young climbers don’t want to watch Ondra speed climbing on plastic holds. They want to see him pulling hard on La Dura Dura, one of the hardest sport routes in the world. And young surfers don’t want to listen to Bob Costas narrate John John Florence’s top turns. They want to watch a webisode, scroll through heats on demand (if the waves are firing), and then go surfing.

The Olympics are best when a sport’s greatest athletes are competing against their peers in optimal conditions. Unfortunately, that’s not what climbing and surfing fans will get in 2020.