A physically inactive lifestyle is associated with maladaptive patterns of personality development over relatively short follow-up periods. The present study extends existing research by examining whether this association persists over 20 years. Participants (total N = 8723) were drawn from the Wisconsin Longitudinal Study Graduates and Siblings samples and the Midlife in the United States Study. Controlling for demographic factors and disease burden, baseline physical inactivity was related to steeper declines in conscientiousness in all three samples and a meta-analysis (β = −0.06). The meta-analysis further showed that lower physical activity was associated with declines in openness (β = −0.05), extraversion (β = −0.03), and agreeableness (β = −0.03). These findings provide evidence that a physically inactive lifestyle is associated with long-term detrimental personality trajectories.