These are some of the things we know about the rising star of the Tory party. David Cameron's glittering understudy outshone the future PM at Oxford, where he was elected president of the union. During a distinguished career as a political journalist, he became the unlikely star of a satirical TV show, and continued to write a newspaper column after becoming an MP. Affectionately indulged across the Westminster spectrum, on account of his charm, his personal appeal transcends party loyalty. Contrary to popular opinion, however, his name is not Boris Johnson.

"People keep talking about Boris becoming prime minster," reflects an Oxford contemporary. "But I think it's going to be Michael Gove."

If you have never met the secretary of state for education, and do not vote Tory, you may well be of the view that Gove is a nasty little rightwinger with a reactionary vision to return Britain to the 1950s. If so, the description I hear from practically everyone who has ever crossed his path will sound unrecognisable.

"I have only nice things to say about him," is the preface I hear time and again. Gove is, Chris Huhne says, "The politest man in the House of Commons. Incredibly well-mannered. Much more so than people who come from grander backgrounds." A senior Labour figure tells me, "The truth is, he doesn't act like a Tory. He is far too decent. He's in politics for all the right reasons." The political columnist Matthew d'Ancona agrees: "He is just an incredibly nice guy. A lot of people are very nice guys, 'but' – and with politicians that can have a very big capital B. But he is an absolutely nice guy. He's not a fair-weather friend at all, he's fundamentally loyal, he is a very empathetic character and very solicitous of people's feelings. I'm yet to come away from a gathering where he hasn't charmed everybody in the room. Michael is a very ironic and incredibly good at taking the piss out of himself; he's one of those people you hope will be at dinner."

From those who knew Gove as a young child, right through to colleagues today, the unanimity of opinion is almost freakish. George Allan was Gove's headmaster throughout secondary school, "And when I see Michael on the television now, I can still see the 11-year-old boy. He didn't change his persona throughout his school career. Consistency – that's the word, consistency. We couldn't claim to be the authors of his remarkable civility. He created his own image."

If Cameron fails to deliver a second term – and a clear Tory majority in 2015 is looking like a tall order – there will almost certainly be a new party leader. With the question of succession becoming a very real issue, the education secretary is a very real contender.

In August 1967, a baby called Graham was born in Edinburgh to a young unmarried mother, and became Michael Gove four months later, the adopted son of a couple in Aberdeen. A younger sister was adopted five years later, and the Labour-voting family lived in a small ground-floor flat in a tidy if austere grey terrace, until his father's fish-processing business became successful enough for them to move into a three-bedroom semi.

Gove attended his local state primary until the age of 11, when he passed the entrance exam to Aberdeen's most prestigious private school for boys. Robert Gordon's College dominates the city centre, a relatively short distance from Gove's childhood home but a significant cultural leap. Every day the teenage Gove would set off from his little semi with its neat vertical white blinds and symmetrical hanging baskets, to an imposing granite colossus resembling a Soviet-style Eton. But he made the journey with remarkable ease. He rode an ancient bicycle, carried an umbrella, was fond of wearing suits, and quickly established himself as one of the school's most precocious characters.

Mike Duncan taught Gove English and debating, and remembers a star debater and "endlessly enquiring" boy who enjoyed testing his teacher by reciting the opening line of a novel and challenging Duncan to name the book. Well read and self-consciously clever, Gove and his friends were big fans of sci-fi role-play games such as Wargame, and another called Nuclear War, which involved building up stocks of weapons and occasionally firing them off. A poem penned in his final year, An Essay On Teaching (With Apologies To Alexander Pope), may or may not make teachers smile today, and satirical entries about the senior boys in 1985 suggest he and his friends were already enthusiastic subscribers to Private Eye.

"Michael Gove: His eventual entrance to Oxfam University has finally confirmed his place among mere immortals. He now earns his money tutoring Mr Duncan in English. Naked without his tweeds, it is Colonel Fogey's ambition to be a listed building when he grows up." When Gove was invited back a few years later to take part in a debate, posters depicting a caricature of the Ghostbusters' ghost went up all over the school with the strap line: "Take Govey down at the debate!"

"Michael was very much a wit," one former classmate remembers, "and a joy to have around. We were never intimates, but he was friendly, intelligent, a nice bloke. He wasn't cool or fashionable, but he was very popular because he would always have a funny rejoinder, and could outwit the teachers. My childhood memories are peppered with laughter because of Michael. One of the things I valued him for was that he prevented me from being bullied. I had glasses and red hair, and I vividly remember being bullied in the changing room, and Michael tried to stop it."

Gove arrived at Lady Margaret Hall to study English in 1985. The novelist Philip Hensher can still recall watching the freshers arrive, and noticing one dressed in a three-piece green tweed suit. "It was really a young fogey thing. He seemed a little bit older than he was, but not in a ridiculous way. If there was an element of absurdity to him, he always seemed to be in on it, which is a very attractive characteristic."

Anthony Goodman was president of the Oxford union when Gove arrived. "He was one of the most gifted speakers I saw in the union. But he didn't generate any antagonism; he was confident, but not in a pushy, aggressive way. He just seemed like an old head on young shoulders." Goodman doesn't think Gove was contriving a performance to get in with the elite social set. "That's pretty much why I did debating," he jokes, "but not Gove. He came across as classless."

The broadcaster and journalist Samira Ahmed edited the union magazine under Gove's presidency, and sums up the consensus: "I just didn't see why anyone could loathe him, personally. He was a 40-year-old trapped in a 20-year-old's body – but it was totally sincere. There's nothing fake about him at all. He exuded intelligence, and was very grown up for his age. And thinking big political thoughts, on a scale very few of us at that age were thinking. His vision was fully formed at 20." Another contemporary, Toby Young, agrees. "He wasn't a work in progress like the rest of us. He seemed to arrive fully formed."

Gove encountered his first ever setback after graduation, when a job application to the Conservative Research Department was rejected on the grounds that he was "insufficiently Conservative". So he returned to Aberdeen and joined the Press And Journal as a trainee reporter. Within months, the paper went on strike.

The father of chapel, Iain Campbell, recalls the staff's initial unease about Gove. "We knew he was a Tory, and our concern was to have a united front. So we spoke to Michael, and he was happy to come on board. He wasn't a typical striker by any means, but he was very articulate, so we asked Michael to come to the European parliament in Strasbourg to lobby MEPs. He was a very good colleague. Being on strike can be very mentally straining, so we used to have parties to keep morale up, and he was someone who would always be there, and always very popular."

Gove has subsequently said he thought the strike achieved little, and for how long he would have stuck it out we'll never know, for within three months he was offered a job in television, and soon moved south to work for the political programme On The Record, and then the Today programme. In 1992 he was hired by Channel 4 to present a late-night political comedy show, A Stab In The Dark, somewhat to the surprise of his co-presenter, David Baddiel.

"I thought, I've never seen anyone on late-night alternative TV like this. He was rightwing and looked 53. I think he was wearing a three-piece suit. He wasn't cut from the obvious Tory cloth; he is a very bright bloke and a thinking rightwing person, and I'm always interested in intelligent people who think differently from me. I've had many rows with people I've worked with, but not Michael Gove. He was quite humble. He was always very unaggressive. Not trying all the time to intellectually defeat you in that way that politicians can."

Around the same time Gove appeared as a chaplain in a feature film set in a boarding school. It was a tiny role, but he'd clearly made a big impression on someone along the way, because the lead schoolboy character was given the name M Gove. Neither the film nor A Stab In the Dark were huge hits, though, and in 1996 Gove joined the Times as a columnist and leader writer.

In 1989, while a trainee reporter on Aberdeen's Press And Journal, Gove joined an NUJ picket line. Photograph: Donald Stewart

Newspaper offices are notoriously fractious places, but staff remember an extravagantly congenial, donnish colleague who would soothe reporters late filing copy with, "But how are you? Yes, of course, should you so wish to file that would be marvellous, that would be very much appreciated. But it's first important that I know how you are." Very much the in-house intellectual, the "most striking thing about him," recalls his former editor, Peter Stothard, "was that he could make people on the other side of the argument not only recognise his argument, but also genuinely like him."

Gove became friends with the rising young Tory stars Cameron, George Osborne and Steve Hilton, and co-founded the centre-right think tank Policy Exchange. His influence over the party's modernisation project would be difficult to overstate. The exact nature of his contribution, however, is confusing. Everyone tells me his politics owe everything to his humble origins, and are "all about creating ladders of opportunity; it's all about things you might call progressive, in terms of goals, but using Conservative means". He is "ideologically amphibious", interested in ideas from the left, "on a moral mission, to do with helping the poorest", and responsible for helping to coax his party into the 21st century. Even his political opponents concur. According to a senior Labour party official, "On sexuality and class and social mobility, on race and gender, he is a genuine liberal." He is not, Huhne agrees, "a traditional rightwinger."

Then I read every word published under his Times byline up to his election in 2005, and found that Gove the progressive arch-moderniser simply does not fit with the facts. If you want to understand what Tory modernisation really adds up to, it's a highly instructive read.

Gove's neo-con hawkishness in foreign policy has always been well known. He called for the invasion of Iraq just two days after 9/11, and before the planes hit the towers he'd already published a column that morning calling for action against Saddam Hussein. A committed Zionist and slavish admirer of George W Bush, a passionate Eurosceptic and staunch defender of British Gibraltar, he regards the Northern Ireland peace process as a shameful capitulation to terrorists, and once wrote a column calling for "a revival of jingoism". But where is the evidence in his writing of domestic social liberalism? It doesn't exist.

Passionately pro marriage, he opposes statutory paternity pay, stem cell research, euthanasia and contraception for school children, but supports privatisation of both the BBC and the NHS, and proposes the marketisation of immigration policy, whereby a British passport could, he suggests, be sold for £10,000. He is far ruder about the Lib Dems than New Labour, despairs of the "absurd belief" that the armed forces ought to reflect the country they serve by recruiting more women and gay men, and takes a robustly bang-em-up approach to law and order. As for supporting public services, Gove can see only one acceptable line: "The Conservatives could defend public servants from the unjust, unproven and demoralising charge of 'institutional racism'," the Macpherson report being, in his view, an outrage.

Why, then, do so many colleagues and political opponents see Gove in this rosier, more moderate light? It has to be because of his debater's gift for according courteous respect to opposing views, creating the impression that he's taken them on board, when he hasn't actually revised his position at all. Everyone tells me how carefully Gove listens, but when asked to recall a single occasion when he has been persuaded to change his mind, to their surprise no one can come up with one. It is a case of manners maketh the impression of a moderniser, for Gove's Tories don't need to be "inclusive", or "tolerant". The important thing is to look as if they are.

He warns the party against "Social Tone Deafness", which "manifests itself in an inability to detect the country's changing mood music", and calls for candidates who "look like human beings who understand the daily struggles of modern existence". But a 2003 column tellingly reveals where Gove ranks the importance of actual policy reform: "The Tories have not yet adopted a tone, style, culture or even [my italics] a set of policy priorities appropriate for the 21st century." A Tory minister and friend of Gove's probably comes closest to the truth when he characterises him as a "true Conservative, the rightwing conscience of the cabinet. He is a full spectrum, real world Tory; it's all about living in the real world. He would regard the Tory rightwing as completely unrealistic" – which is not the same as wrong. Labour MPs like to say Cameron's Conservatives have modernised on only one solitary issue – homophobia – and I used to think that had to be too cynical. But in all of Gove's writing it's the only example of an authentically abandoned prejudice.

What of Gove's famous passion for social mobility through education? "His view on social mobility is for people to do what he did – study classical subjects and go to Oxford," argues a Labour critic. "It's all slightly 'Some working-class people are terribly good at Latin – give them a chance and you never know!'" The former Labour schools minister Jim Knight agrees. "If he thinks social mobility is getting more poor kids into Oxbridge, then I think that's a massively flawed definition. If you are serious about social mobility, it has to be communities and families, not just individuals."

Interestingly, in 2000 Gove wrote, "Too many people go to university", but his columns reveal surprisingly little about the future secretary of state's thoughts on education. For their origins, we need to go back further still, to his old school days. "He benefited in a life-changing way from a classical liberal education," as a close ally puts it, "and would like that privilege to be extended to people from less privileged backgrounds."

For all its architectural grandeur, Robert Gordon's College during Gove's era was actually Scotland's second cheapest private day school – "more of a glorified grammar, really," as a former pupil puts it. About half the boys joined from state primaries, and a quarter received a bursary or scholarship. George Allan and Mike Duncan both assumed Gove had been a scholarship boy, but in fact it was only when his father's business failed in his final year that the family applied for assistance. But every boy had to pass the entrance exam, and the school could operate on the assumption that all pupils would be not merely able, but disposed to work hard.

"When you enter Gordon's and start wearing the blazer and tie, you sort of absorb the traditions of the college," a former teacher tells me. "Those kids from poorer economic backgrounds tried to climb upwards, not pull the others down." When I ask Allan what sort of disciplinary problems he used to deal with, he looks surprised. "Well, if I had one a term, that was enough for me." The current acting head says the worst he's encountered was a boy kicking about a plastic bottle who ignored his request to put it in the bin.

Gove has always spoken highly of his old school, and when I visit I can see why. But while I'm admiring its orderly classrooms and general air of cheerful application, a teacher puts into words what I find myself thinking. "The trouble with anyone's own experience of school," the teacher points out, "is that it's so formative that it informs all your ideas about education. That's very natural, but not necessarily a good thing."

From the age of 11, Gove never shared a desk with anyone who struggled to read and write, or who experienced school as an intimidating trauma instead of a golden ticket. "His prime concern is to make sure that there are safe schools in cities for people like him," says a columnist who has followed his career closely. "All this nonsense that free schools are free – they are not, they are Michael Gove schools. He is trying to create a system in which middle-class parents like him can make uses of the state system without having to mix with the rough boys."

Following his election in 2005, Gove wrote, "The reason why I'm in Parliament is not really to see my colleagues win power, it is to see us at last in a position where we can give it up."

But since taking office, he has awarded himself more than 50 new powers, including rules allowing him to acquire land from local authorities and private companies to help him increase the number of academies and free schools. This autumn 55 new free schools will open, taking the total to 80, and the number of academies has risen tenfold to more than 2,300, each one accountable not to local authorities, but to the secretary of state himself. "He is a High Tory control freak who wants to run every school in the country," counters the critic of Gove's "free" school rhetoric. "He hasn't got a localist bone in his body."

There are other curious paradoxes. A friend for more than 20 years has never once heard him mention money, and says he regards it with unworldly indifference. But in 2002 Gove wrote about living perennially beyond his means – "Ever since going up to university, I have accumulated new debt, and new means of becoming indebted". And when the expenses scandal broke, it emerged that he'd spent £7,000 of taxpayers' money on designer furniture, much of it from Cameron's mother-in-law's interior design company.

Another puzzle was his use of private email addresses to communicate with his team. Everyone says Gove has a practically courtly deference to protocol, but he was plainly breaking the rules. Gove pleaded an innocent failure to master his own departmental technology, and his physical ineptitude is legendary; being driven by Gove is, apparently, a "terrifying" experience. Whether techno-klutziness explains an arrangement that conveniently kept secrets from hostile civil servants, however, is unclear.

For a man known to be meticulously conscientious, Gove's ministerial career got off to an oddly bumpy start. In the early months, he drew criticism for awarding a grant to a Jewish schools security trust on whose advisory body he had sat [see footnote], and then there was the botched cancellation of the Building Schools For The Future programme, and confusion over the number of schools wishing to become academies. Gove blamed sabotage by his own department – in a colleague's words, a "ministerial blooding". But last year he hired a special adviser who'd been previously blackballed by Andy Coulson, called Dominic Cummings. "And you could almost feel," says a senior aide, "that the moment Dom came to join him, things started getting back on track."

Earlier this year, Gove leaked to the Daily Mail his plan to restore O-Levels, sidestepping departmental and Lib Dem opposition to court the rightwing press, whose praise was predictably rapturous. Issuing every school in the country with a leather-bound King James bible inscribed with the words, "Presented by the secretary of state for education" surely did more for Gove's profile than for the nation's spiritual improvement. At £370,000, it was a pricey gesture, but private donors comprised almost exclusively of rightwing Christian financiers were happy to foot the whole bill. Gove's other big idea this year was pricier still – a new £60m royal yacht for the Queen as a diamond jubilee present. It was clearly never going to happen, but sent another signal to the right, as did his audacious hymn at the Leveson enquiry to the virtues of "one of the most impressive and significant figures of the last 50 years", Rupert Murdoch. "Cummings is a really amazingly effective political street fighter," the aide says, "the wily operator he needed. Since then, Michael has felt a new sense of confidence."

Gove's confidence extends well beyond his existing role. Most cabinet ministers haven't the energy or political courage to discuss their colleagues' policies in any detail; Gove isn't one of them. "Michael does have an opinion on everything," says Tim Montgomerie, editor of Conservative Home, "and he talks in cabinet a lot." One cabinet minister describes him as "being slightly inclined to go off on a bit of a rant off his own brief", and an aide describes his role as the "energetic challenger – especially of the system, and of conventional wisdom". And yet, says another minister firmly, "I can't think of anyone who sees him as a political rival."

How could Gove possibly not be seen as a political rival? He may be Osborne's closest ally, but in the opinion of one Lib Dem minister is "substantially cleverer". He is Cameron's "closest friend in politics", helping him prepare for PMQs, feeding jokes into his speeches, and he "can be quite dazzlingly funny, brilliant at doing impressions", a friend reports. "I've heard him do what must be a set piece, in which he does a great impression of Huhne condescending to Clegg in cabinet, that had the whole table in stitches. Genuinely funny satire."

On a New Year's Day walk in 2011 with the Cameron family and friends, including Tim Burton and Helena Bonham Carter. Photograph: John Taylor

"The thing you have to appreciate about this group," an insider explains, "more than any group in postwar politics, is they are a clan. They are in and out of each other's houses, kids and nannies everywhere, they are each others' kids' godparents, their kids go to the same schools. The question of succession doesn't cross their minds."

The question is whether it crosses Gove's.

Just how far does Gove's ambition go? "At Oxford it was obvious that some people were warming up for the Commons," Samira Ahmed recalls. "With Boris Johnson, his plan was obvious. But I never heard that said about Michael." Former colleagues at the Times assumed for a while that he would become their editor, "But he wasn't seen as someone angling for his own personal promotion at all." Gove has always insisted he has no interest in leading his party, saying, "I don't have it in me. I don't have what it takes." And yet, as Matthew d'Ancona observes, "You can never assume that a politician who has devoted his life to the game will walk away from the chance to be leader or PM. It's ridiculous for us to think that any politician might."

A lot of very wealthy Tories presumably agree, for Gove's constituency office last year received five times the party's national average sum of donations. Individual and corporate donors gave Gove, who occupies one of the safest seats in the country, more than £60,000 – almost twice the sum received by Osborne's office and three times more than Cameron's. What exactly are they funding?

Nobody believes Gove would ever challenge Cameron for the leadership. And Gove had always assumed that, when the succession did come, the heir would be Osborne. But now that the chancellor is becoming unelectably damaged goods, and talk of a Johnson leadership bid gathers momentum, few think Gove would simply stand by and watch London's mayor move into Downing Street. If the party needs a Stop Boris candidate, it will be Gove.

Does he have what it takes? Former colleagues are unsure. "He is an intellectual, not a man of the people, and whether he could operate in an environment in which not very many people knew who Spiro Agnew was I don't know." I ask his old boss, and Stothard chuckles. "I think by common consent the job of home news editor wasn't his natural home. News editors, unless they're hated half the time, are not really on top of their job. Was Michael able to do the throwing the typewriter thing? No. He didn't have the temperament for that role. I don't really believe in pop psychology, so I'm reluctant to go into this very far, but he is quite thin-skinned. Perhaps too sensitive to be home news editor – or prime minister."

Someone else who knows Gove well agrees that at first he tended to flap when things didn't go smoothly. "This 'I haven't got what it takes' line, I think what he was referring to was this thing of coping with the ups and downs and the crap you get. The relentless roller-coaster and inevitable failures. He previously might have been a bit insecure about his ability to cope with adversity, but I think that has gone now. I think that has changed. If you'd asked me 18 months ago, do you think he could and should aim to be PM one day, I would have said no. But now I think Michael could." There's some speculation that the decision will ultimately fall to his wife, the Times columnist Sarah Vine, but no agreement on which way she'll go.

"I think she adores being a Tory wife," says one friend. "The more pictures she can be in with Sam Cam, the better her life becomes." But another disagrees. "I've heard people say that, but I don't think she's a scheming Machiavellian political wife, pushing her husband's career." She has a funny way of going about it if she is, featuring marital anecdotes in her column that render him a faintly absurd eccentric.

The couple met at the Times, married in 2001, and live with their daughter and son, nine and seven, in a modest terraced house at the Wormwood Scrubs end of North Kensington, where dinner is "a standard middle-class set up; completely informal, non matching furniture, child-centred, you can't move for books, the kids are usually up. Sarah does most of the cooking – meat and two veg or pasta, no bought in food or waitresses or any of that nonsense – and Michael's pouring the wine. It's jolly and cosy and warm and about having fun."

And if Vine's wider family wind up as Downing Street in-laws, then by all accounts they should provide fabulous entertainment. "A totally mad, bonkers, crazy family," a friend laughs. "They're around in Michael's life in a soap opera sort of way, fodder for amazing stories and anecdotes and hassle and problems."

Gove's relationship with his parents has always been "very tender and loving", another friend says. "That said, I don't want to sound crude or pompous, but he has a metropolitan London life. There is a gap there."

But Gove has never looked for his birth mother, and says he never will, out of loyalty to the parents who raised him. "He is just not a very conflicted person," another friend offers simply. "I don't think there's a great deal of inner turmoil there. He regards life as having smiled on him."

Gove has always been good-humoured about his place in the party's popularity stakes. In fact, he's so accustomed to coming second as the party faithful's choice of after-dinner speaker, he likes to joke that on his gravestone will be carved the words: "They Couldn't Get Boris." But though Johnson's sense of humour has got him a long way so far, few would bet on him having the last laugh. "If you had told me Boris Johnson was going to be mayor of London," reflects Anthony Goodman, their fellow ex-Oxford union president, "I definitely would have laughed at you. If you had told me Michael Gove was going to be in government, I would have thought that was pretty obvious."

• This footnote was added on 15 October 2012. The Community Security Trust received and administered the grant, but made no financial benefit from it.