When Lord Patten was appointed chairman of the BBC Trust earlier this year, he was interviewed – "grilled", in tabloid parlance – by the MPs who sit on the culture, media and sport select committee. For licence fee-payers, the people whose interests he would be safeguarding for the next four years, watching this encounter was not a particularly pleasing sight. On the plus side, at least he was honest. "I watch the programmes that you'd expect somebody of my background to," he said. "That's who I am: I'm 66, white and well-educated." Not for him a focus-grouped insistence, a la Gordon Brown, that he likes Strictly Come Dancing. His idea of a really fun show was a documentary called Mud, Sweat and Tractors: the Story of Agriculture on BBC4. But on the downside, it was also clear that he was not a keen consumer of the majority of the BBC's output. EastEnders? Don't be daft. Radio 1? Certainly not. To me, his appointment began to seem like yet another instance of jobs for the boys. It was as if he had been appointed to run Asda without ever having visited a supermarket.

Six months on, though, and Patten has gone native. When we meet at the trust's offices in the West End – where, thrillingly, you can also see a display of original figures from Trumpton, including Pugh, Pugh, Barney McGrew, Cuthbert, Dibble and Grub – he finds it hard to stop talking about all the marvellous television he has been watching. "I've been extending my cultural horizons," he says. "I've watched The Hour and Page Eight, and an extremely good two-parter called Field of Blood, which was spectacularly well-acted. Over the weekend I watched three episodes of the Danish series The Killing, and I think it's a reasonable aspiration of the BBC to make programmes as good as that. I watched The Crimson Petal and the White. I've watched Sherlock. I think Wallander is fantastic. I've watched a couple of really classy programmes on the war in Afghanistan on BBC3." On and on he goes, until he has the advantage even of me, a telly obsessive. "I've watched things I would never have seen otherwise. I've watched The One Show."

This seems to have bred in him a deep love of the institution of the BBC, which he describes, almost breathlessly, as "a moral force, a theatre company and a news organisation". He cannot get over the teeming life, the sheer talent he finds down the end of virtually every corridor, with the result that he is given to quoting the great Penelope Fitzgerald, whose novel Human Voices is about life at the BBC during the war. "Broadcasting House resembles a great liner," he says, reading from a notebook into which he has copied lines from the novel (he has, by the way, tiny handwriting). "With the best engineers in the world, and the crew varying between the intensely respectable and the barely sane, ready to scorn any disaster, unless of Titanic scale." He slams the thing shut. His perfectly round face – he looks a bit like a figure from Trumpton himself – is a picture of rosy satisfaction.

None of this, I fear, will go down too well with those on the Conservative backbenches (or with Jeremy Hunt, the culture secretary, now that I really think about it), who regard both Patten and the BBC with deep suspicion. Ditto the Daily Mail, which would no doubt prefer that he complain loudly about repeats. Then again, after a quiet-ish start in his new job – he seemed so desperate to appear neutral, both to his new colleagues at the BBC and to its political enemies, that his every utterance was drained of all interest, let alone controversy – he is suddenly less afraid to say what he thinks.

We start by talking about Salford, which will soon be the new home of Radio 5 Live, BBC Sport, BBC Breakfast and CBBC (Patten was there only yesterday, looking at MediaCity's "terrific" new facilities). Doesn't he find it extraordinary that so many high-profile members of staff have been so reluctant to move? How is this supposed to look to the BBC's consumers, most of whom live nowhere near London? And why have certain Salford refuseniks – I use the example of Simon Mayo, who left 5 Live for Radio 2 – been accommodated with new plum jobs in the capital? My view is that, had they called his bluff, Mayo would soon have realised how boring life at Heart FM (or wherever) was going to be.

Patten begins gently by telling me that his grandparents were both headteachers in Manchester; he had Didsbury aunties and, as a boy, used to follow Brian Statham, the Lancashire fast bowler, from ground to ground, a bottle of Tizer and Sandwich Spread sarnies in his knapsack. And then… he's off! "This sort of [attitude]: 'Do they have paninis in Manchester?'… It's unbelievable. [The city has] the best Indian food, some wonderful Chinese restaurants, great music, great jazz – my cousin used to run a jazz club there. [From Salford Quays] you can see Old Trafford, Maine Road is round the corner, you can see the Pennines on two sides. I was talking to some senior executives at 5 Live yesterday. They were all moving into larger houses, the schools they were finding were terrific. I see my own kids struggling in London to afford anything, particularly near a decent Ofsted school, and I'm just amazed at the attitude of some of the media towards Salford. It is, for heaven's sake, the British Broadcasting Corporation, not the Surrey Broadcasting Corporation. There's so much talent available there. The amount of writing talent there's been in the north west!"

As it happens, Patten is a "huge" Mayo fan: the DJ interviewed him once, and gave him the impression that he had actually read his book. But he agrees with the general point, and believes that someone would have to be "very good indeed" to refuse a job in Salford and then be given one in London. "This idea that there are people who have apparently never been north of Watford, and who think that up there it's all wode and pickled onions..." His voice trails off.

In July, Patten – already on the road to Damascus, it now appears – gave a lecture to the Royal Television Society in which he said that the BBC needed to start being "less defensive". I was pleased by this but I wondered, too, how it might be possible in a world in which it is every politician's favourite punchbag (and the left – think Alastair Campbell – is just as bad as the right). He smiles. "I don't believe they really hate it, because whenever it suggests it might stop doing something, they all shout: no! Look, at the risk of sounding sanctimonious, I think the BBC is there to do good. First of all, by providing space for a national conversation. Second, there's a great remark of Adlai Stevenson, that the average man and woman is a great deal better than average – and that should be like 'Blackpool' through a stick of rock at the BBC. Never underestimate people's tastes or intellectual capacity. I'm reading a book by Jonathan Rose called The Intellectual Life of the British Working Classes, and it's a reminder that we've always taken the view that if you give people access to great books, lovely paintings and marvellous music, it makes them more civilised human beings. That's a big part of what I feel. I think it's amazing that at three minutes past nine of a Thursday morning, 2 million people tune in to hear Melvyn Bragg talk about quantum physics [on In Our Time]. It could only happen at the BBC."

But he hasn't really answered my question: how does the BBC set about reminding people what value (just 39p a day) the licence fee is rather than spending its time responding to criticism? "Well, the first thing you have to do is to remove some of the signs of a fall from grace. The whole salaries issue needs to be tackled pretty quickly, and if you deal up front with complaints, and rapidly and generously, and avoid complacency and being smug, if you demonstrate that the best defence of the BBC is making bloody good radio and television, then I think you can be much more self-confident about how terrific it is that we have the best broadcasting in the world."

Ah, salaries. Did Patten hear Jonathan Ross on the radio the other day, going on about how he was worth every penny of his reputed £6m salary? Thanks to the bullishness of the so-called "talent", not to mention their agents, I can't believe this boil will ever be lanced. Patten disagrees – though he thinks it important to distinguish between executive pay, by far the more toxic issue in his view, and talent pay. "On the executive pay side, we've dealt with it pretty firmly. The BBC will be the first [organisation] to implement Will Hutton [Hutton conducted a recent review of fair pay in the public sector for HM Treasury, and made recommendations, including greater transparency]. In terms of the relationship between senior executive pay and median pay, there will be a cap from now on, and there will be a programme to reduce the gap." (The BBC has its work cut out in this respect: median senior executive pay at the BBC is nine times greater than median pay overall.) So are people such as Alan Yentob, the BBC's extremely well-paid creative director (Yentob is in possession of a pension pot rumoured to be worth more than £6m), taking pay cuts? "Yes, they are." Are they cross? "I don't know what people who are really being affected at the top feel. What I do know is that down among the foot soldiers, I've had a terrific reaction. I've had emails from people – from quite a well-known BBC journalist – saying: well done."

What about presenter salaries? (I'm not sure about this word "talent".) "I said when I was being interviewed for this job that I didn't think you could assert a public service ethos and want to be paid like a banker. People expect the BBC to have the best writers and directors and actors, but I don't think we can afford to pay whatever celebrities who become famous think they're worth. It's part of the BBC's role to create talent, and if that talent becomes too expensive, well, so be it. I don't mean to be critical, but I'm not one of those who thinks Jonathan Ross has turned a new chapter in humour. There are people who think he's funny, and they can watch him on ITV, and good luck."

Patten expects to be able to make an announcement about budget cuts – the corporation needs to make 20% savings across the board if it is to meet its new obligations – at the beginning of October. But he wants to emphasise that, even then, he will only be talking about proposals, and there will be a period of consultation. "The intention is that it shouldn't affect quality. It should be perfectly possible to run a great public service broadcaster on £3.5bn. Are we going to close BBC4? Of course we're not. There was a bizarre story that BBC4 was a target for the trust. Give me a break." (The latest rumour is that BBC4's management may be merged with that of BBC2.)

All he can tell me about the cuts at this point is that he expects half of the necessary savings to be made through efficiencies, and that everyone will have to take their share. "We do want to protect the most important things: drama, children's, news. We can't spend as much on sport or entertainment, but I hope we'll be able to demonstrate that we've kept the crown jewels." And in the future? The licence fee, of which he is a supporter, is guaranteed only until 2017. What then? "The biggest danger is that there's a growing disjuncture between the technology and the tax. More and more people are not watching on a television. That doesn't deal with the intellectual case for a form of taxation to pay for a public good, but it does raise issues about what that tax should be based on."

How is his relationship with the director general, Mark Thompson? There are rumours that Thompson will soon be leaving the BBC. "I don't know what Mark's plans are. I was educated by the Benedictines, and he was educated by the Jesuits, which has certain implications." Like what? That he had the better deal? He laughs. "We occasionally go to the same church in Oxford – they really think there that the BBC is run from the pews of the Oratory. There is a Chinese phrase that describes a close relationship as being like lips and teeth. I don't think we'd ever want to be that close, but it's important that we do get on. It's important that people at the BBC trust me and recognise that, to an extent, we are all in this together."

I don't suppose it comes to Patten as any surprise that I would march on the streets for Radio 4 alone. But even a fan like me has her limits. There are too many programmes both on radio and television featuring men shouting at one another (Today, Newsnight), and far too few voices and faces belonging to women over the age of 45 (and sticking Edwina Currie, Lulu, Nancy Dell'Olio and Anita Dobson on Strictly feels like the worst kind of sop to me). "I agree," says Lord Patten. Oh, good. And are minds sufficiently focused on rectifying this? "First of all, I'm 67, for heaven's sake, and I'm married to a charming and beautiful 66-year-old, and I would be delighted if she was the face of anything on television. Second, we should have more women on radio and television. There are some good ones: Sarah Montague and Martha Kearney and Fiona Bruce – I saw her programme on royal palaces, which she wrote as well as presented, and I thought she was terrific. But there should be more."

Patten is one of those men whose name seems in print never to be unaccompanied by the word "former": former MP, former Conservative party chairman, former governor of Hong Kong, former European commissioner (he is, however, still the chancellor of Oxford University). So how badly did he want the job at the BBC? "Quite badly," he says. "When I left university I won a graduate traineeship at the BBC, but then I got involved with the Conservative Research Department [and turned the BBC job down], which I think they regarded as lèse majesté. So there's an ironic symmetry in me pitching up here.

"I've never planned my career. Things have happened. I was pretty occupied. I was going to write another book, I had Oxford and two or three not-for-profit organisations that I chaired. Then a headhunter phoned me. I hadn't considered it before, but I thought: I'd like one more job where I've got decisions to make, something I care about, where I can make a difference." He is, he says, the kind of old-fashioned Tory who "believes in institutions".

When he was made chairman of the trust, naturally he resigned the Conservative whip in the Lords. But he will not be tearing up his party membership card any time soon. "People know I'll always be a Tory." Is his new-found neutrality frustrating? Ordinarily, a politician of his vintage would be able to pitch in, to snipe at (or, possibly, cheer on) Dave and George from the edge of the field. Now he can talk about almost nothing save for the BBC – and his grandchildren, of whom there are eight. "Yes, butter wouldn't melt in my mouth. It is a bit frustrating. I've always been slightly a troublemaker in the sense that I've never regarded the whips as being anything other than those with whom one should have a polite and formal relationship. I've never bitten my tongue, sometimes to my disadvantage. I've had to become a vestal virgin again after a not entirely pure political past." So when his time at the BBC is up, does he intend pitching in once again? "Oh, yes," he says, looking mildly pleased at the prospect. "And by then, I'll have all the suppressed prejudices of having spent four years as a vestal virgin."