RICHMOND, Va. (WTVR)-- A federal appeals court says the city of Lexington can ban the Confederate flag.

The three-judge panel of Richmond's 4th U.S. Circuit Court of Appeals on Friday upheld a lower court ruling on the city's 2011 ordinance banning the flag on city light poles. The court ruled that it did not violate a heritage group's right of free speech.

The sons of Confederate veterans had challenged the ordinance which was enacted after residents complained the flag was offensive.

All non-government displays are prohibited, which is why the court ruled the way it did.