Both political parties are encountering a surprising problem as they prepare for next year's battle to control the U.S. Senate: Some of their top choices are turning down invitations to run for seats in a body with a reputation that has been tarnished by bickering and gridlock.

In states as disparate as Iowa, Michigan, Georgia and Nebraska, men and women considered top-tier candidates are declining to jump into races.

In...