For more than a year, two US senators on the intelligence committee have fought a lonely, unsuccessful battle to prevent the National Security Agency from combing through its vast email and phone records databases for Americans – a battle waged almost entirely in the shadows.

In letters, hearings and one big legislative push last year, senators Ron Wyden and Mark Udall have tried to close what Wyden calls a "backdoor search loophole", to ensure that communications from Americans that inadvertently turn up in NSA databases are promptly purged. And they have tried to stop intelligence officials from publicly saying they can do no such thing when, behind closed doors, the officials acknowledge to the Senate intelligence committee they have that authority.

Thus far, Wyden and Udall have failed.

The NSA's dragnet of emails, phone calls, web searches and affiliated metadata occurs under the legal authority of Section 702 of the FISA Amendments Act, or FAA. It is supposed to target foreigners: foreign spies and foreign terrorists. A host of officials, including the NSA director, General Keith Alexander, have said that when they come across American data in so-called "702" databases, they purge it – unless it contains information about ongoing threats to national security, criminal activity or espionage.

But information obtained by the Guardian from the whistleblower Edward Snowden suggests the NSA both retains so-called "incidentally collected" American data for longer periods than intelligence officials have indicated, and can search through that American communication data for information – including by name.

It is not known how many times such searches have been performed. It is not known how long the NSA can keep American phone and email communications in its databases after identifying it as unrelated to an ongoing threat.

But for more than a year, Wyden and Udall have attempted to restrict the NSA from doing so.

On 22 May 2012, the Senate intelligence committee met to debate expiring sections of the 2008 FAA, which included Section 702.

Wyden and Udall proposed an amendment "concerning prohibitions on acquisition of or searching contents of communications of United States persons", according to the official Senate record. It would have closed the back-door searches loophole.

"Since all of the communications collected by the government under section 702 are collected without individual warrants, we believe there should be clear rules prohibiting the government from searching through these communications in an effort to find the phone calls or emails of a particular American, unless the government has obtained a warrant or emergency authorization permitting surveillance of that American," Wyden and Udall said in a June 2012 statement for the Senate record.

The chairwoman of the committee, Dianne Feinstein of California, agreed at the time that the intelligence agencies cannot "target a US person" under Section 702, and need an individual court order based on probable cause to do so. But she said the Justice Department and intelligence agencies "reaffirmed" to her that any such queries of the database "do not provide a means to circumvent the general requirement to obtain a court order before targeting a US person under FISA [the Foreign Intelligence Surveillance Act]."

Senator Mark Udall. Photograph: Ed Andrieski/AP

The amendment failed, by a vote of 13-2. The only votes for the measure came from Wyden and Udall.

In July 2012, the NSA's director, General Alexander, spoke to a hacker conference in Las Vegas. Alexander ventured a reassurance about American privacy.

"We may, incidentally, in targeting a bad guy, hit on somebody from a good guy, because there's a discussion there," Alexander said. "We have requirements from the Fisa court and the attorney general to minimize that, which means nobody else can see it unless there's a crime that's been committed."

Having so recently failed to close the backdoor searches loophole, Wyden and Udall wrote to Alexander on 10 October 2012.

"We believe that this statement incorrectly characterized the minimization requirements that apply to the NSA's FISA Amendments Act collection, and portrayed privacy protections for Americans' communications as being stronger than they actually are," the two senators wrote.

"We urge you to correct this statement, so that Congress and the public can have a debate over the renewal of the law that is informed by at least some accurate information about the impact it has had on Americans' privacy."

Alexander replied to the two senators in a 13 November 2012 letter that quoted back sections of the Foreign Intelligence Surveillance Act that seemed to allow the NSA greater capability in handling Americans' communications than intelligence officials typically state – thereby seeming to confirm Wyden's "backdoor searches" loophole.

Among them, Alexander wrote, is a section stating: "… nonpublicly available information which is not foreign intelligence information shall not be disseminated in a manner than identifies any US person, without such person's consent, unless such person's identity is necessary to understand foreign intelligence information or assess its importance."

At least once, the Fisa court has ruled that the NSA's actual minimization procedures violated the fourth amendment of the US constitution, which protects Americans against unreasonable searches and seizures.

"On at least one occasion, the Foreign Intelligence Surveillance court held that some collection carried out pursuant to the Section 702 minimization procedures used by the government was unreasonable under the fourth amendment," according to an unusual 20 July 2012 letter from the Office of the Director of National Intelligence, permitting Wyden to disclose that ruling. It is unknown when the court issued it.

Since the Guardian and the Washington Post disclosed the existence of the Prism program and other foreign-directed bulk surveillance on internet and other communications that can swoop up Americans inadvertently, a host of officials have echoed Alexander's line to the hacker conference.

His colleague John C Inglis, the deputy director of the NSA, told a House hearing on 18 June: "These procedures require that a communication of or concerning a US person must be promptly destroyed after it's identified, either as clearly not relevant to the authorized purpose, or as not containing evidence of a crime."

But since then, Wyden and Udall have won a rare public victory in their struggles with the NSA: they persuaded Alexander to take down a factsheet on the NSA's website reiterating the same line about "promptly destroy[ing]" Americans' communications that aren't relevant to foreign intelligence or a crime.

Wyden and Udall called the statement "somewhat misleading", in a 24 June letter to Alexander, as it implies "the law does not allow the NSA to deliberately search for the records of particular Americans".

Both senators have repeatedly said they believe NSA's surveillance efforts under Section 702 provide valuable intelligence that protects national security. Their efforts at closing the "backdoor searches loophole" continues.

More: NSA loophole allows warrantless search for US citizens' emails and phone calls