I was a sceptic about newspaper lists – believing them to be the last refuge of a desperate editor with space to fill – until recently, when the Observer published its brilliant selection of Britain's top 300 public intellectuals. Chosen by my esteemed colleague John Naughton, it set off fascinating arguments. The inclusion of one Andrew Rawnsley split my family down the middle between a group who took this to be proof of Mr Naughton's impeccable judgment (all right, that was the view taken by me) and another faction (my mother, my wife and my daughters) who said this demonstrated only that lists in newspapers are good for nothing but a laugh.

There was one feature of the list which was both disturbing and depressing: the paucity of practising politicians or policy advisers in the 300. Plenty of historians, poets, philosophers and scientists passed muster. Even some lawyers and economists made the intellectual cut. But just seven politicians were judged worthy of inclusion and only four policy advisers. John may have been too severe – I would have added a few names he left out – but I wouldn't quarrel with the substance of his verdict. In challenging times which cry out for fresh ideas in both government and opposition, we are very short of enlivening political thinkers.

The number of them may lurch even closer to extinction after the travails of two men – one Tory, one Labour – who have both won the dubious distinction of being called "gurus" to party leaders. Steve Hilton, long-time friend to David Cameron and his director of strategy at Number 10, has been generating embarrassing headlines for the government because some of his most incendiary notions were leaked to the Financial Times and splashed across its front page. In a neat symmetry, ordure has simultaneously come pouring down on the head of Maurice Glasman, the pioneer of "Blue Labour" who was given a peerage by Ed Miliband as he sought a philosopher-lord to help with the reinvention of his party.

Conventional is not a description you could apply to either of these eclectic thinkers. Steve Hilton is famous for padding around the corridors of Number 10 in T-shirt, shorts and socks. This tells you how important he is in the building: no mere cabinet minister or permanent secretary would dare turn up at Downing Street in beach wear.

Maurice Glasman is not a typical member of the House of Lords. He smokes rolls-ups and lives above a second-hand clothes shop in Hackney. His vivid turn of phrase recently woke up a rather worthy political conference in Germany. Asked by an earnest German to expound on the key to human happiness, the unusual Labour peer replied that he was happiest watching Spurs in the afternoon and then spending the evening having sex with his wife.

Both men are the antithesis of the image of a political thinker as a desiccated type who pontificates from atop an ivory tower. The highly emotional Mr Hilton can be a combative and volatile man. His frustration with the government machine, and it with him, has been the cause of many a fiery clash with civil servants. Last month, Number 10 had to deny that he was about to walk out in anger that the public service reform agenda was being watered down. He is the son of two refugees from Soviet-era Hungary who met at Heathrow airport where they both worked. Know that and you understand why his views are animated by a visceral distrust of the state, which also expresses itself in a deep hostility to the European Union. He is one of the few true believers at Number 10 – perhaps the only one – in the idea of the big society.

Maurice Glasman is another man whose ideas are driven by his emotions. He has said that his political thinking was transformed two years ago by the death of his mother who was very conservative, working-class Labour. The grief concentrated his mind on why his party had lost touch and let down people like her. The "Blue Labour" prescription – I summarise crudely, but I hope fairly – is a blend of community activism, social conservatism and antipathy to global capitalism.

There are many big differences between these two men and their philosophies, but something interestingly common to them is anti-statism, a deep antagonism to bureaucracy and managerialism. Maurice Glasman shares Steve Hilton's animosity towards the EU. They don't like their politics all neat, tidy and consensual; they believe that progress comes from conflict, mayhem and messiness. Both can be as searingly critical of their own parties as they are of its opponents. During the years which the Tories spent in the wilderness, Mr Hilton was one of the first to see that his party would have to change its social attitudes radically before it would regain power. "Blue Labour" became fashionable because it offered a fresh critique of where the party has gone wrong. They have a penchant for thinking the unthinkable and saying the unsayable. That is the point of them. And that is what has got both of them into trouble.

According to the leaks, Mr Hilton's suggestions for reviving the flat-lining economy included suspending maternity leave and all consumer rights legislation to see what would happen. He also mused that a solution to long-term unemployment would be to close all Jobcentres and spend the money on community groups instead. Cue outraged headlines and disavowals by Number 10.

Lord Glasman undid himself with his own tongue. The latest edition of the New Statesman carries a long mea culpa in which he regrets saying that Labour had "lied" about the extent of immigration, apologises for "the crassness and thoughtlessness with which my views on immigration were expressed" and accuses himself of "the vices of arrogance, vanity and carelessness". He promises "a vow of silence for the summer".

There are Labour people who might like that silence to become permanent and there are those in government who would be glad never to hear again from Steve Hilton. I hope that neither man is monastic for too long. This is not because I always agree with either of them; I profoundly disagree with a lot of what they advocate. "Blue Labour" has raised some themes that Ed Miliband could find useful and some that will get him nowhere. Steve Hilton's ideas are a fizzy cocktail of the good, the bad and the mad. His friends say he once suggested that the Tories could fulfil David Cameron's promise to make Britain sunnier by using cloud-bursting technology. This may be evidence that he is crackers – or more likely that he is capable of self-irony.

While examining ways of cutting the deficit, he proposed getting rid of hundreds of government press officers and replacing them with a single person in each department who would communicate via a blog. No wonder Whitehall is leaking with a view to discrediting him. He puts his ideas in an extreme way to make people think, but the thrust of it, that government communications are out of date and many press officers are probably surplus to requirements, sounds very well worth exploring.

It would be a shame if either were to be silenced. All the parties need more provocative thinkers. Maurice Glasman has got so much attention – more than any one thinker can really bear – because "Blue Labour" has been a rare sign of intellectual life in the party since its election defeat. He may not have the answer for Labour, but at least he has been raising some potent questions.

At Number 10, David Cameron is surrounded by traditional officials who have risen to senior grades because they are great at thinking inside the box, but have limited capacity for venturing outside it. I can see why the prime minister values Steve Hilton, even though his friend often generates friction within Whitehall. That is what he should be there for. Big organisations go to sleep without some provocateurs to challenge group think and disrupt stifling orthodoxies. David Cameron is quite a conventional person. I once asked one of his cabinet whether he regarded the prime minister as "intellectually curious". After a long pause during which he debated with himself what the loyal answer would be, the minister finally smiled and said: "I could call him many things, but not that." By having Mr Hilton around, the prime minister fills a gap in himself.

David Cameron has been in professional politics for nearly all of his adult life. So has Ed Miliband. So have the great majority of the senior politicians of their generation. It has generally made them a cautious, calibrating breed. They need the spark that is brought by people who don't think of politics just in terms of the latest focus group or polling result.

We need more of these intellectual agitators on both sides of the aisle. There's a shortage of stirrers who can shake things up a bit and jolt arguments out of ruts. They are often wrong, they can sometimes sound bonkers, but you can say this for Steve Hilton and Maurice Glasman: they make everyone think. Would that there were more people in British politics like them.