NEW YORK (Reuters) - A panel of federal appeals court judges pushed a U.S. government lawyer on Wednesday to answer why FBI letters sent out to Internet service providers seeking information should remain secret.

A panel of three judges from the U.S. Second Circuit Court of Appeals heard arguments on whether a provision of the Patriot Act, which requires people who are formally contacted by the Federal Bureau of Investigation for information to keep it a secret, is constitutional.

The American Civil Liberties Union filed suit in 2004 on behalf of an undisclosed Internet service provider against the U.S. government challenging the so-called National Security Letters (NSL) as well as gag orders placed on the recipients.

The appeals courts on Wednesday questioned a lawyer representing the U.S. government on whether the FBI violated free speech rights in placing the gag orders.

The government argues they are in place for national security concerns, such as keeping terrorists from learning what they are investigating.

“You can’t tell me that any terrorist is going to make anything out of the fact you issued NSLs to AT&T and Verizon,” said Circuit Judge Sonia Sotomayor, using a hypothetical example.

U.S. Assistant Attorney General Gregory Katsas said the FBI “assesses the need for secrecy in each particular case.”

Between 2003 and 2006 nearly 200,000 national security letters were sent out. Of those about 97 percent received gag orders.

ACLU lawyer Jameel Jaffer said the gag order had prevented the small Internet service provider the ACLU was representing from speaking out “against an FBI investigation that he believes is illegitimate.”

The government is appealing a lower court ruling that said the gag order violated the First Amendment guarantee of free speech and was unconstitutional.

The judges will rule on the issue in the coming months.