WASHINGTON—A U.S. appeals court on Tuesday threw out federal rules requiring broadband providers to treat all Internet traffic equally, raising the prospect that bandwidth-hungry websites like Netflix Inc. might have to pay tolls to ensure quality service.

The ruling was a blow to the Obama administration, which has pushed the idea of "net neutrality." And it sharpened the struggle by the nation's big entertainment and telecommunications companies to shape the regulation of broadband, now a vital...