• President to announce that private entity will store call data • Move does not end US bulk collection program • Speech will outline changes taken from NSA review panel

US president Barack Obama will announce that the National Security Agency should no longer store US telephone data in bulk, in a significant change to the most controversial surveillance activity exposed by the Guardian thanks to whistleblower Edward Snowden.

In a speech Friday morning at the Justice Department, Obama will call for an as-yet unspecified private entity to hold data on telephone calls made in the United States. He will also say that the NSA or the FBI must receive a judicial approval from a secret surveillance court before it can search through the data trove.

Details of how the program will operate in the future were unclear hours before the widely anticipated speech. A senior administration official indicated that transitioning the metadata collection will take some time. Privacy advocates have been bracing themselves during the past week for Obama to call for merely cosmetic changes to a surveillance effort they regard as a disaster for civil liberties.

A senior administration official portrayed the shift as an overhaul to mass surveillance that would better respect civil liberties, rather than an outright end to a practice the government has claimed is a critical counter-terrorism tool. This contention has been resisted by a federal judge, knowledgeable senators and Obama’s own surveillance advisory group.

“The president believes that the 215 program addresses important capabilities that allow us to counter-terrorism, but that we can and should be able to preserve those capabilities while addressing the privacy and civil liberties concerns that are raised by the government holding this metadata,” the official said Friday, using a bureaucratic shorthand for the bulk collection of phone data.

But “effective immediately”, the NSA will need to obtain a finding from the secret surveillance panel known as the Fisa court in order to search its phone records database, a judicial check that has never existed since a version of the bulk phone data collection began in October 2001.

Currently the NSA receives a 90-day order from the Fisa court to gather all domestic call records and to trawl the database when it has “reasonable articulable suspicion” of a number’s connection to a specified terrorist group. That order was likened to a general warrant by a federal judge in December, adding significant pressure on Obama and the NSA to overhaul the program.

Before the next Fisa court application, scheduled for 28 March, the official said Obama would ask attorney general Eric Holder and the intelligence agencies for a report on “how we can preserve the necessary capabilities of the program, without the government holding the metadata.”

That suggested Obama had not resolved the contours of any post-NSA metadata collection, despite a voluminous December report from a surveillance advisory group he empanelled in August and two weeks of consultations with the intelligence community, civil libertarians in and outside government, and congressional leaders. The official said the administration would continue to consult with the “relevant committees” in Congress “to seek their views”.

Those contours have been in furious contention for weeks – a contest that is likely to continue well after Obama’s speech.

Privacy advocates, in and outside of Congress, have said the government should only be able to view metadata pursuant to a judge’s order in all but emergency cases at a higher standard than “reasonable articulable suspicion” in order to ensure that innocents are not swept up in data troves. They also want to ensure that the government will not require telephone companies to store customer data any longer than they currently do, which varies by firm but has an average maximum of 18 months.

Compelling the companies to retain additional data “would not solve the problem, it would just outsource it”, said Liza Goitien of the Brennan Center for Justice at New York University.

The telephone companies have balked at storing additional data on the government’s behalf, out of fear of exposing themselves to legal and financial liability.

The NSA has argued that it needs the entire “haystack” of records showing phone calls made in the US for at least three and as many as five years, lest it be unable to detect patterns in call data that might indicate a domestic connection to terrorism.

But the utility of the program for preventing terrorist attacks has been repeatedly disputed. The surveillance advisory group found it was “not essential” for stopping terrorist attacks, but Michael Morrell, a panel member and former deputy CIA director, told Congress Tuesday that “that doesn’t mean it’s not important going forward”, likening it to fire insurance.

The dispute over the necessity for the bulk metadata collection will continue in Congress over the weeks and months ahead. Advocates of ending it are pushing a bipartisan bill known as the USA Freedom Act, which has yet to pass any of the four congressional committees in the House and Senate currently considering it. A competing Senate bill, sponsored by intelligence committee chairwoman Dianne Feinstein, would reaffirm and entrench the NSA’s storage of bulk metadata; it has passed her committee.

Obama has yet to endorse any of the competing bills, and early indications were that his Friday speech would leave significant room for Congress to recast the boundaries of surveillance. But civil liberties groups said Thursday that any lasting changes to the surveillance landscape – which they consider central to Obama’s presidential legacy – would require Obama to take an unequivocal position.

“Of course this would require congressional legislation to make sure these sorts of changes last, and we need the laws to be changed anyway, but we need his leadership,” said Michelle Richardson of the ACLU.