Exclusive BT secretly intercepted and profiled the web browsing of 18,000 of its broadband customers in 2006 using advertising technology provided by 121Media, the alleged spyware company that changed its name to Phorm last year.

BT Retail ran the "stealth" pilot without customer consent between 23 September and 6 October 2006. The technology was approved, pending a further trial*.

Documents seen by The Register show that the companies used the secret profiles to target advertising at broadband customers when they visited certain popular websites.

Phorm had purchased commercial space on these websites, although their URLs are not included in the documents. The groups targeted included people interested in finance (for an Egg credit card campaign), weight loss (a Weight Watchers campaign), and jobs (a Monster.com campaign).

The technical report drawn up by BT in the wake of the 2006 trial states: "The validation was made within BT's live broadband environment and involved a user base of approximately 18,000 customers, with a maximum of 10,000 online concurrently.

"The customers who participated in the trial were not made aware of this fact as one of the aims of the validation was not to affect their experience."

The Regulation of Investigatory Powers Act 2000 (RIPA) makes intercepting internet traffic without a warrant or consent an offence.

BT claims that when it launches, Phorm's technology will be legal under RIPA, despite counter arguments from respected experts on the legislation. The ISP's and Phorm's claim is based on advice from the Home Office, which was recently published and disputed on the influential UK-Crypto mailing list.

The government advice was solicited by the ISPs and Phorm in the run up to the announcement of their partnership on 14 February. Written by civil servant Simon Watkin, it argues that the system will probably be legal if consent is obtained from users.

Watkin wrote: "Targeted online advertising services should be provided with the explicit consent of ISPs' users or by the acceptance of the ISP terms and conditions."

BT has said it plans to change its terms and conditions accordingly to comply with the law.

On the legality of the proposed opt-out system using cookies, the BT technical report states: "Whilst the... issue is not really a technical consideration of this report, it is mentioned since owing to the legal position, direct cookie dropping could not be trialed and should be verified once the legal position is clearer."

That means all 18,000 test subjects were always opted-in without their knowledge.

BT has not answered The Register's question, posed on Friday morning, over whether it believes intercepting and profiling the web traffic of 18,000 customers without telling them was a lawful act. A statement it sent us merely confirmed it performed the experiments on customer data, and repeated the party line that no personally identifiable information is used by Phorm technology. You can read the statement here.

BT also refused to reveal where in the national broadband network the thousands of guinea pigs were sourced from.

One senior source in the broadband industry we spoke to was appalled by BT's actions. "This is extremely serious," he said. "Data protection errors are generally viewed as a potentially bad thing by the industry, but not a real threat to an ISP's reputation. This seems like a breach of criminal law, which is much, much worse."

Even during the early phase of the BT/Phorm deal that the technical report describes, the pair were preparing to spin the technology to the public. "121Media [Phorm] will take action (both technical and public relations) to avoid any perception that their system is a virus, malware or spyware and to show that in effect it is a positive web development," BT wrote in the report.

Read on to see Phorm school El Reg on ethics.