Police will be forced to seek the permission of a judge if they want to retrieve the phone and email records of journalists, after the prime minister’s snooping watchdog found that 19 police forces made more than 600 applications to uncover confidential sources in the past three years.

The requests to use anti-terror legislation to access journalists’ communication records were made in 34 police investigations into suspected leaks by public officials, the Interception of Communication Commissioner’s Office (ICCO) report published on Wednesday shows.

Sir Anthony May, the interception of communications commissioner, said police forces “did not give due consideration to freedom of speech” and Home Office guidelines do not sufficiently protect journalistic sources.

In his report to David Cameron, May also recommended that police forces should be made to seek a judge’s permission when seeking to discover a journalist’s confidential source.

The prime minister’s official spokesman said Cameron accepted the commissioner’s recommendation. “He very much welcomes the report. He believes that the report makes the case well for the judicial oversight it recommends in cases of data comms applications that are relevant to journalistic sources and that is why the Government is accepting that recommendation,” the spokesman said.

The demands for telephone and email records of journalists Communications between 82 journalists and 242 sources were snooped on by police over the past three years, the ICCO report added.

The inquiry into the police’s use of the Regulation of Investigatory Powers Act 2000 (Ripa) was ordered after it emerged that detectives trawled through the telephone records of Mail on Sunday journalists to trace the source of a story about Chris Huhne, the cabinet minister jailed for conspiring to have his wife take his speeding points on her driving licence.

Most of the 34 police investigations were internal inquiries into suspected leaks by officers and staff, the report shows. Scotland Yard’s Operation Elveden inquiry, which investigated into the unauthorised disclosure of information by public officials to journalists, accounted for 80% of the 608 police applications to discover journalists’ sources.

The report found that 43% of the information sought by police was for so-called traffic data, including telephone call records, typically for up to two months – meaning potentially dozens of confidential sources may have been snooped on by the police including those not relevant to the investigation.

May’s office launched an inquiry last October after it emerged that journalists’ communications data was obtained by Scotland Yard when it investigated the Plebgate affair and by Kent police as part of the force’s investigation into Huhne, the former cabinet minister who admitted perverting the course of justice.

The commissioner found that, over the past three years, 19 police forces had asked for communications data in support of 34 investigations into public officials suspected of providing unauthorised information to journalists.

Police forces can obtain communications data under Ripa. With the authorisation of a senior officer, detectives can be told by communications companies what telephone numbers a suspect has called and whom the suspect has emailed.

The police are not given access to what was said or written. But analysis may allow detectives to infer that a reporter has had regular contact with an identifiable source.

Most requests under Ripa are made “for the purpose of preventing or detecting crime or of preventing disorder”. Crime, in this case, does not need to be “serious”. However, the intrusion must be proportionate to the expected benefit.

When this use of Ripa in relation to journalists’ communications came to light last year, news organisations accused the police of circumventing the safeguards in the Police and Criminal Evidence Act 1984 (Pace).

This legislation allows news organisations to appear in court and challenge police requests for access to journalistic material, such as untransmitted video recordings.

In his report to the prime minister, May says : “The need to protect the confidentiality of journalistic sources is crucial to safeguard the free press in a democratic society.”

Measures creating a “chilling effect” on freedom of expression would breach the human rights convention.

However, most police requests for communications data examined by May’s inquiry “did not sufficiently justify the principles of necessity and proportionality”. The police concentrated on possible breaches of a suspect’s privacy under article 8 of the convention and did not give sufficient consideration to journalists’ rights to freedom of expression under article 10.

May considered whether it would be sufficient for new guidance to be given to senior police officers – the designated persons – who decide whether disclosure of communications data is lawful and proportionate.

He concluded that a designated person, usually an inspector or superintendent, could continue to authorise disclosure in cases involving journalists where there was no investigation into the source of journalistic information – for example where a journalist was a crime victim or facing unrelated charges. But the senior officer would still need to consider “the necessity, proportionality, collateral intrusion, including the possible unintended consequence of the conduct”.

May added that the code issued under Ripa, as well as a proposed revised code, did not provide enough guidance on these points “and that absence needs to be addressed”.

Turning to cases where “communications data is sought to determine the source of journalistic information”, May told the prime minister that “judicial authorisation must be obtained”.

These recommendations should be implemented “in order to provide adequate protection for journalistic sources and enhanced safeguards to prevent unnecessary or disproportionate intrusions”, the commissioner added.

But he stopped short of providing any detail as to how the new system might work. That will be a matter for the home secretary to consider.

Welcoming May’s finding that “police forces are not randomly trawling communications data relating to journalists in order to identify their sources”, the Association of Chief Police Officers said investigations into corrupt relationships between journalists and public officials would continue to be a challenge for all public agencies.

Asst Ch Con Richard Berry, national policing lead for communications data, said the police were committed to maintaining a balance between what the law said and how it was implemented. “To that end we announced our plans for a board of communications data ethics to consider issues of public concern in an increasingly digitised world,” Berry added.

If May’s recommendations are accepted, the Ripa code could be revised without the need for further legislation. But a system of judicial authorisation would need a change in the law. In other circumstances it might be possible to set up some informal referral system.

But the judges would not wish to take on a new, highly sensitive duty without statutory backing. In any event, further legislation will be necessary after the general election because the Data Retention and Investigatory Powers Act 2014 expires at the end of next year.

Meanwhile, senior police officers could go a long way towards addressing the concerns May has identified. Tasked with the aim of tracking down public servants who have leaked or sold information to reporters, detectives have used the tools provided for them by parliament.

The police need to understand that news organisations, too, have an interest in rooting out corruption – and that reporters cannot uncover wrongdoing if they are not allowed to protect confidential sources.