The supreme court justice Antonin Scalia has said that the courts will ultimately have to determine the legality of wiretapping by the National Security Agency.

Scalia addressed NSA wiretapping on Wednesday in a speech to the Northern Virginia Technology Council. Asked about technology companies' role in protecting customers' privacy when their data has been unconstitutionally collected, Scalia said the high court originally ruled that there were no constitutional prohibitions on wiretaps because conversations were not explicitly granted privacy protection under the Fourth Amendment, which protects against Americans against unreasonable search and seizure of "their persons, houses, papers, and effects".

That 1928 opinion, in Olmstead v US, was overturned nearly 40 years later by the Warren court, which found, Scalia said, "there's a generalized right of privacy that comes from penumbras and emanations, blah blah blah, garbage".

"The consequence of that is that whether the NSA can do the stuff it's been doing … which used to be a question for the people … will now be resolved by the branch of government that knows the least about the issues in question, the branch that knows the least about the extent of the threat against which the wiretapping is directed," he said.

Scalia also said the constitution called for a balancing test to determine whether any search or seizure is reasonable, and that depends on the threat that is posed – another question he said courts are ill-equipped to answer.

He talked about the pat downs and other searches that occur at airports as an example of that balancing act. "That's a terrible intrusion of privacy," he said. "But you're willing to do it because of the seriousness of the threat."

As for the question of tech companies' obligations to inform clients about an illegal intrusion on their information, Scalia said yes, a company should speak up if it knows a customer has had its data illegally seized.

"But it's pretty hard to know that," he said. "If it's a governmental wiretap, presumably it's been approved by somebody, some lawyer expert in the field who said it was OK, and you better be damn sure you're right before you blow the cover."

In July, following the disclosures by the NSA leaker Edward Snowden about the extent of the agency's surveillance programs, the Electronic Privacy Information Center filed a direct appeal to the supreme court, asking it to bar the NSA from collecting phone call records on millions of US customers. The court has not yet decided whether to hear the case.

Earlier this year, the court ruled in a 5-4 vote that clients represented by the American Civil Liberties Union lacked standing to challenge a 2008 law under which the NSA conducts aspects of its surveillance. Scalia voted with the majority to turn away that challenge to the law.