Back in 2009, the BBC approached Ofcom for permission to add "digital rights management" locks to its high-definition broadcasts. The locks would work by scrambling some of the information used to decode video, and in order to get the descrambling key, manufacturers would have to submit to the rules of the DTLA, an offshore consortium led by Intel.

This was a strange request for the BBC to make. There aren't any licence fee payers who put a cheque in the post this year and thought, "Gosh, I wish there was a way I could do less than the law allows with the video my licence fee pays for." The BBC has always eschewed DRM in its Freeview offerings, and other public broadcasters in Europe, the US and Canada eschew DRM. German law prohibits DRM on its public broadcasts, and American law prohibits DRM on all broadcasts, both commercial and non-commercial.

What's more, the DRM scheme proposed by the BBC had three major flaws: first, technical experts believed it would be trivial to defeat; second, the part of the broadcast that the BBC wanted to scramble was shared by closed captions and assistive audio tracks used by disabled people; and finally, the full rules set out by DTLA for its DRM were governed by confidentiality agreements, which meant that UK manufacturers would be ordered to comply with a set of secret rules that the public wasn't allowed to know.

There were other important problems, of course: the proposal violated the EU common market by breaking foreign TV receivers and it meant that popular free/open source receivers and recorders would be frozen out of the UK device market.

The consultation received 459 responses. Of these, 432 of those came from individual licence payers, and 426 opposed the BBC's proposal.

Twenty-six institutions filed 27 responses (Channel 4 responded twice); 17 of them supported the proposal. This includes broadcasters (two different divisions of the BBC, Channel 4, Channel 5 and ITV); a bunch of rightsholder groups such as the Motion Picture Association (run by the US film studios) and Pact; the DTLA itself; and some of manufacturers who license DTLA's technology. The arguments they made were pretty unconvincing, boiling down to "We'd make more money this way," and "Rights-holders may boycott TV if we don't give them this."

The opposing institutions included a coalition of computer science academics, the Royal National Institute of Blind People (RNIB), educators and the Linux Foundation. I wrote comments on behalf of the UK Open Rights Group.

To sum up: the licence fee-paying public overwhelmingly rejected the proposal. So did independent scholars, public interest groups, educators, and disabled rights groups. The vast majority of support came from the BBC (who'd started the whole thing, no surprise there!), broadcasters, and rights-holder groups.

The BBC is a public broadcaster, and its charter sets out the requirement for everything it does to meet a "public value test." Ofcom, the independent regulator that oversees the BBC, is charged with "[making] sure that people in the UK get the best from their communications services and are protected from scams and sharp practices, while ensuring that competition can thrive".

So what did Ofcom do? Naturally, it listened to the public, ignored the uncompetitive rent-seeking proposals from the commercial sector, adhered to EU law, and rejected the proposal.

Well, that's what they did in a parallel universe. In this universe, Ofcom accepted the self-serving arguments of the companies they're meant to be regulating, ignored the public whose interests they were meant to be safeguarding, and gave the BBC what it asked for.

Why did it do this? It's a secret.

But not any more.

BBC tight-lipped

The BBC's comments to Ofcom were released with several redacted sections. These sections were withheld on the basis that they were "commercially sensitive" and couldn't be seen by the general public. Ray Corrigan, one of the Open University academics who signed on to the opposing comments from the computer scientists, filed a freedom of information request to discover what this secret evidence was. After all, there was nothing else particularly compelling in the BBC's proposal, so the really meaty arguments had to be there.

Ofcom rejected the FOI request, saying that the law prohibited it from releasing "commercially sensitive" material. When I asked it whether this meant that anything could be kept out of the public's eye by slapping a "commerically sensitive" label on it, at first they said this was so. But on further questioning, their spokesperson wrote, "we would consider the information on a case-by-case basis and consider the contents of any document to see whether an exemption applies, rather than automatically withholding it because it's marked as 'commercially sensitive'."

Next, Corrigan tried to get the information from the BBC. The BBC also refused to disclose it. I exchanged several emails with a tight-lipped spokesperson. They declined to grant me an interview with Graham Plumb (the BBC's head of distribution technology) or any of his colleagues involved in the submission. They sent this terse statement:

The Ofcom report was submitted in confidence because it contained commercially sensitive information regarding the BBC's negotiations with rights-holders. Such negotiations are standard practice when agreeing commercial contracts. These contracts enable the BBC to provide audiences with a wider choice of programmes than would otherwise be possible.

The spokesperson refused to clarify or expand on this statement.

So, this is weird. Fundamentally, the BBC and Ofcom were claiming that it was in the public's interest to deny the public's wishes, but it wasn't in the public interest to know why this was so.

I started to ask around. One very senior executive with BBC Worldwide (the commercial arm, charged with selling BBC programmes abroad) expressed bafflement: "I don't understand why they want this. It certainly doesn't come from us. We don't need it to sell BBC programming."

Eventually, I got my hands on a copy.

Redactions revealed

Here are the "commercially sensitive" portions of the BBC's Ofcom filing that the public hasn't been allowed to see until now:

1.4. The BBC confirms that it is very unlikely that content which has been directly commissioned by the BBC (alone) would be not be available for broadcast on platforms not supporting content management. However, it should be noted that independent producers frequently retain secondary rights in such content. At some point in the future, a powerful independent producer might insist that it would only accept a commission from the BBC on terms which required the BBC to guarantee that the first (and potentially subsequent) transmissions of this content would be broadcast with content management applied.

Here, the BBC admits that all the content it commissions would be available without DRM. In other words, the stuff that we think of as "BBC programming" (as opposed to joint productions, independent commissions and programmes licensed from abroad) will be available even if the BBC doesn't get its wishes.

It's impossible to see why this would be "commercially sensitive". Why shouldn't the public know that the BBC won't insist on restrictions on the video whose production the licence fee pays for?

The hypothetical business about some independent producer at some future date insisting on DRM is frankly laughable. If an indie doesn't want to sell the BBC a programme it's commissioning on the terms the BBC want, the BBC can commission from another indie – just as it would if an indie wanted to use an actor or writer the BBC didn't like, or wanted more money than the BBC was prepared to spend. Commissioners solicit pitches from indies on set terms, and the speculative, unnamed indie that won't come up to these terms is a pretty unconvincing spectre.

1.5. The BBC wishes to emphasise that in commercial negotiations with right-holders (and co-producers) the BBC normally seeks to obtain the most flexible and least restrictive rights at the best possible price. In general, this flexibility is intended to allow the BBC to show content on one or more occasion(s), sometimes on more than one of the BBC's channels and ideally on both broadcast and non-broadcast platforms (eg the BBC iPlayer). Inevitably, in these negotiations, content rights-holders are seeking to maximise the income from their investment and may seek to increase the price of rights in proportion to the flexibility requested by the BBC.

Again, it's impossible to see why it would be "commercially sensitive" to admit that some rights-holders and partners ask for more money when no DRM is deployed. It's also impossible to see why Ofcom would take this very seriously: the BBC isn't saying that it pays more on no-DRM terms, just that more is asked for. This is anaecdote, not evidence.

1.6. The highest value acquired content, such as first UK free-to-air broadcast rights for popular films or television series, are significantly more likely to be subject to restrictions. For example, in recent negotiations between the BBC and (separately) HBO Enterprises, Sony Pictures Television International, 20th Century Fox International Television and Warner Brothers International Television Distribution, each of these suppliers has separately made it clear that they are unwilling to provide the BBC with linear HD broadcast rights to some high value content unless the BBC undertakes to apply content management controls to all HD format broadcasts of this content.

Aha! At last, the BBC names some names. Unfortunately, these aren't very convincing names.

Fox broadcasts all its HDTV in the US first, where it is prohibited by law from adding DRM. A substantial fraction of Warner and Sony programming also gets broadcast in HD in the US first. These rightsholders may say that they demand DRM, but the fact of the matter is, when they don't have a choice, they broadcast without DRM. If Ofcom doesn't give them that choice, there's no reason to think they'll behave any differently here than they do at home in America.

HBO, on the other hand, is a different story. All HBO content goes out first over DRM-locked cable channels in the USA. If the BBC was going to get new HBO material for broadcast, it's at least credible that HBO would demand DRM.

But the BBC isn't going to get any new HBO content for its broadcast for the next 10 years, because HBO has an exclusive, 10-year deal with Sky.

The exception to this is BBC-HBO co-productions – fundamentally, the BBC is saying that they don't have control over the programmes it pays for with our money. That's embarrassing, but it's hardly "commercially sensitive".

1.7. Whilst this acquired content represent a small proportion of the BBC's HD output, free-to-air television audiences perceive significant value from access to content of this type (for example feature films) from the BBC.

Apparently "people like movies" is "commercially sensitive".

1.8. The proposed launch of a BBC1 HD simulcast later this year is likely to slightly increase the amount of acquired content which is would be subject to HD broadcast limitations on platforms where no content management is present. The BBC very much wants to avoid a situation where it is necessary to blank any acquired content either on BBC1 HD or BBC HD. [NB The BBC's plan to launch BBC1 HD this autumn is known to Ofcom but remains highly confidential and subject to final sign-off by the BBC Trust].

It's true that this was confidential (the trust approved BBC1 HD earlier this month), but it's not any more. But it's also unnecessary to the BBC's argument: it's possible to say "we foresee acquiring more content" without saying "because we're planning this confidential thing that we don't have permission to do yet".

3.4.1 These approaches include: (i) Implementing HD broadcast recording and access systems which implement end-to-end content protection using the technologies which have been specified in the DTG D-Book. (ii) Using professional Freeview HD broadcast receivers combined with their own secure storage and retrieval systems. (iii) Developing or purchasing from their suppliers secure storage and retrieval systems which integrate direct reception of unencrypted HD video, audio, subtitle and audio description signals with EPG data licensed by the relevant broadcasters. 3.4.3 In respect of the approach outlined in section 3.4.1 (iii), the BBC has suggested that it would be prepared to consider licensing arrangements which would allow libraries, archives and larger educational establishments to make use of EPG data under condition that: (i) the products and systems using this EPG data together with video, audio, subtitle and audio description recordings would only be made available to educational establishments, libraries, archives and other equivalent organisations; (ii) the purchasers or users of these products or systems would need to confirm these would only be used for educational, criticism or review, news reporting or other purposes which are permitted under UK legislation; and

(iii) these products and systems would need to incorporate all necessary security controls to prevent the illegitimate copying, distribution or use of HD broadcast recordings for other purposes. 3.4.4 It is the BBC's firm view that the approaches suggested in this section would fully accommodate all current and likely future legitimate usage requirements by even the largest establishments, whilst avoiding compromising the integrity of the HD content management system or requiring any circumvention of this system. 3.4.5 The BBC requests that Ofcom do not publish the specific details relating to these approaches – due to the concern that the above explanation provides information which is helpful to those who may wish to circumvent the content management system.

Here the BBC discusses its plan to accommodate educators, critics and archivists. It plans on establishing a confidential marketplace for more powerful "professional" TV receivers and recorders that can defeat its scrambling system. This bizarre system – creating an entity that would have to manufacture and distribute these devices, after approving the credentials of archivists, critics and scholars – is meant to be kept secret because it makes it clear that it would be easy to defeat the scheme.

So here you have the BBC claiming in one breath that its partners want effective protection from copying, and in the next breath saying that this won't be very effective protection.

Funnily enough, "this will be easy to defeat" is a point that many of the individuals and institutions who formed the majority opposed to this plan made in their statements.

DRM Britain

After I received the redacted material, I approached the BBC and Ofcom again. The BBC refused to answer my emails until I sent copies of the unredacted text to the executives responsible for the memo. Then, a BBC spokesperson again refused to let me speak to anyone responsible, and said:

Sections 1.4 - 1.8 of the BBC's Ofcom consultation response were submitted in confidence because these contain commercially sensitive information regarding the BBC's negotiations with rights holders. Such negotiations are standard practice when agreeing commercial contracts. These contracts enable the BBC to provide audiences with a wider choice of programmes than would otherwise be possible. In redacting sections 3.4.1 – 3.4.5 of this consultation response the BBC was seeking to avoid publishing any information which would be helpful to individuals who might seek to circumvent Freeview HD's content management controls whilst also recognising that under UK copyright legislation, prescribed libraries and archives (which include the British Library and the British Film Institute) and educational establishments have more extensive rights to reproduce and retain copies of broadcast programmes than do individual viewers who only have a right to record programmes so that they can watch them at a more convenient time (the timeshift exception). The BBC remains convinced that disclosure of the redacted information included in these sections of its Ofcom response would not be in the public interest.

In other words, "Auntie knows best, so shut up and run along and let us get on with breaking your TV."

Ofcom was equally unhelpful in explaining why any of this material passed muster in its "case by case" evaluation of confidentiality. Rather, it switched back to its statement that it had to take all claims of confidentiality at face value, and freedom of information seekers need not apply when one of the companies it is meant to be regulating tells it to keep mum, no matter how compelling the public interest and how unconvincing the claimed need for confidentiality.

And now we're gearing up for DRM Britain. Our BBC will give privileges to American TV companies that the US government won't give them, and our "independent" regulator won't even tell us why.

In the new DRM world, the rights you've enjoyed to your licence-fee-paid material are now contingent. If you want to save your copies to your computer, transfer them to your tablet or phone, loan them to your neighbours, excerpt them for education, criticism or parody, you're going to have to ask a committee for permission. If your kids want to do these things, they're going to have to seek this permission as well, and if you have the sort of children who aren't comfortable making submissions to regulatory committees, then your children won't be able to do what other children all over the world are accustomed to doing.

British manufacturers who want to make digital television receivers and recorders will have to accord with the secret rules set out by the DTLA, which includes a ban on free/open source software, such as the popular MythTV package.

This package is already in use by many British licence fee payers (some of whom wrote in to Ofcom about it) and DTLA rules it out because it can be modified by its users. Despite the fact that so many British engineers started their journeys by building BBC receivers, there are no user-servicable parts allowed in the new, HD, DRM BBC.

Welcome to DRM Britain. Our BBC will give privileges to American TV companies that the US government won't give them, and our "independent" regulator won't even tell us why.

• The author gratefully acknowledges the research contributions of Ray Corrigan and Beth Bernier Pratt