WASHINGTON—Some of the Federal Bureau of Investigation’s electronic surveillance activities violated the constitutional privacy rights of Americans swept up in a controversial foreign intelligence program, a secretive surveillance court has ruled.

The ruling deals a rare rebuke to U.S. spying programs that have generally withstood legal challenge and review since they were dramatically expanded after the Sept. 11, 2001, attacks. The opinion resulted in the FBI agreeing to better safeguard privacy and apply new procedures,...