Previous research suggests that Pokémon Go can be employed as an intervention for promoting healthy behavior on a large scale. In contrast, some scholars expressed concerns about the long-term effects of this mobile game, indicating that the impact of the app on exercise is just moderate. We hypothesized that Pokémon Go users are more active, but this tendency is mainly driven by the greater physical activity needed to level up in the game and does not extend to physical activities in general. In the present study, 981 individuals completed a web survey, in which participants' frequency of Pokémon Go usage, overall physically active behavior, and amount of Pokémon Go related physical activity were measured. Regression analyses revealed that Pokémon Go related physical activity significantly reversed the positive effects of the app on participants' overall physically active behavior, suggesting that the mere adoption of the app does not reliably change people's behavior in general. The increase in physical activity levels is rather explained by the exercise required by the game.