Official means of oversight of American and British spying have failed. So we are left with the protection of last resort: the conscience of the individual who will resist abuse of power or expose it once it is done.

At the Guardian Activate conference in New York last Wednesday, I moderated a heated panel discussion about the NSA affair with former US Senator Bob Kerrey, a member of the 9/11 Commission; Professor Yochai Benkler, codirector of the Berkman Center for Internet & Society at Harvard; and journalist Rebecca MacKinnon, a New America fellow.

"We do not have appropriate mechanisms to hold abuse accountable," MacKinnon said, and to more or lesser degrees, the panelists agreed that oversight is at least too weak. Said Benkler: "The existing systems of oversight and accountability failed repeatedly and predictably in ways that were comprehensible to people inside the system but against which they found themselves unable to resist because of the concerns about terrorism and national security." Kerrey: "I don't think we're even close to having unaccountable surveillance [but] I don't think it's good oversight." I'll count that as consensus. We then checked off the means of oversight.

• Executive-branch oversight is by all appearances nonexistent.

• Congressional oversight didn't exist before Watergate, Kerrey said, and when it was established, it was made intentionally weak. It should be conducted, he said, "under a constant, militant sense of skepticism". The clearest evidence that the authority that exists is not being used, he said, is that in the Snowden affair, not a single subpoena has been issued from either the House or Senate select committees.

• The secret Fisa courts have proven to be rubber stamps using invisible ink – their justices sometimes concerned or reluctant, Benkler said. But they have been largely ineffectual in any case.

• Journalistic oversight is the next resort. But as MacKinnon stressed, the work of the journalist investigating spying is threatened by the spies themselves as they collect metadata on any call and message and reconstitute raw internet traffic so that no reporters and no sources can be certain they are not being watched unless they find woods to walk in.

So we are left with the whistleblower. As Benkler argues:

What the whistleblower does is bring an individual conscience to break through all of these systems. It can't be relied upon as a systematic, everyday thing. It has very narrow and even random insights into the system. But it can be relied upon occasionally to break through these layers of helplessness within the system.

But this oversight, too, is jeopardized by the severe penalties suffered by Chelsea Manning and the label of traitor pasted on Edward Snowden.

"There's no question Snowden violated US law," Kerrey declared in our panel. "And there has to be consequences to that."

Benkler disagreed, arguing the case for amnesty. "There is a law but the law is always affected by politics and judgment," he said. "Clearly when someone opens up to the public a matter that is of such enormous public concern that it leads to such broad acceptance of the need for change and for reform, that person ought not come under the thumb of criminal prosecution."

There we tried to find the line that enables acts of conscience and civil disobedience to keep watch on the powerful. Benkler imagined "a core principle that when a whistleblower discloses facts that actually lead to significant public debate and change in policy – that is to say a public rejection whether through judicial action or legislative action; a reversal – that is the core or heart of what needs to be protected in whistleblowing."

Kerrey again disagreed, drawing a parallel between Edward Snowden and Klaus Fuchs, who handed secrets on the atomic bomb to the Soviets, Kerrey contended, also out of conscience. Benkler in turn drew a line between revealing information to the public, serving democracy, and revealing secrets to an enemy. Kerrey responded that Fuchs, like Snowden, caused public debate. Benkler thought the rule could be written; Kerrey did not. You can see that we failed to find the line.

But I want to take this discussion beyond whistleblowing – beyond the past tense – to the present tense of objecting to the work one is required to do before it is done. "At what point does conscience require a person to refuse to act in a certain way that they consider completely acceptable in the system they're in but they find completely unacceptable to their conscience?" Benkler asked.

Kerrey countered: "I don't think every time you get a team of people working on the danger [to national security], one person can say, 'Oh, I don't like what we're doing,' and as an act of conscience blow everything we're doing and say we're not going to be prosecuted."

But we must find the room for conscience to act as the check on power without facing 35 years in prison or life in exile or irreversible jeopardy to our security. We must be able to expect the honest technologist working in the bowels of Google or telecom provider Level 3 or the NSA or GCHQ to define a line and refuse to cross it. Can we expect that?

In recent testimony before Congress, General Keith Alexander said the NSA is the nation's largest employer of mathematicians – or to be exact, 1,103 mathematicians, 966 PhDs, and 4,374 computer scientists.

Where is the code of ethics that governs their work in breaking into our communication or breaking the encryption we use to protect it? Where is the line they will not cross? Doctors have their codes. Even we journalists have ours (and though some apparently never imagined a clause relating to phone hacking, others found it for them).

We have heard two Google engineers tell the NSA to fuck off for – according to Snowden's documents – infiltrating internal traffic between servers at Google and Yahoo.

Does this challenge to the NSA give us confidence that others at Google will tell the NSA "no"? But who said "yes" to Project Muscular, in what company? Was that company commandeered by the the NSA and employees with security clearance or was the work done willingly? Why didn't the technologists who spliced that line say "fuck you", too? Will they be more willing to do that now that this work is known? And what will happen to those who do stop at the line?

On 17 July 1945, 155 scientists working on the Manhattan Project signed a petition to President Harry Truman urging him not to use the bomb on Japan. "Discoveries of which the people of the United States are not aware may affect the welfare of this nation in the near future," they said.

They were too late.

• You can watch a video of the Guardian Activate panel discussion here.