While I’ve been engaged in my day job over the past couple of weeks I’ve been working on two rather important issues: how to tackle climate change and how to respond to very high levels of inequality. Meanwhile, I have not been able to avoid noticing that a third big issue, that of our future relationship with the European Union, has been occupying our parliamentarians. These are all proving rather difficult to solve. It turns out that there is a common reason for that.

In what follows, I’m taking as given that a no-deal Brexit would not be a good outcome of the parliamentary process, that climate change is a real thing and could, if unchecked, prove very dangerous and that at least some aspects of present levels of inequality are damaging. I hope you agree, because these really ought not to be controversial statements.

Even if you don’t agree, it doesn’t really matter for what follows, for a significant majority of those who ought to be able to do something about them do agree. The vast majority of national leaders — President Trump being the notable exception — clearly believe that man-made climate change is a problem. They’ve said so and they’ve said we need to deal with it. It is abundantly clear that a large majority of our MPs think that a no-deal Brexit would be a terrible outcome. They also think that inequality is a pressing issue. Remember Theresa May’s statement, on the day she became prime minister, about the need to tackle the “burning injustices” of the gaps between rich and poor, black and white, young and old, state and privately educated?

So, if there is a broad agreement on the objectives, why is there such a big risk that we will mess up each of these challenges? Why do we still risk emitting far too much carbon, ending up with a no-deal Brexit and doing little effectively to tackle inequality?

Lots of reasons. Complexity, uncertainty, distance in the future, all play a role. The common feature, though, is that these are all forms of collective action problem. In each case, if people could work together effectively we would all be better off. But some combination of conflicting interests, difficulty in building trust and the potential for free-riding can act as fatal barriers to co-operation.

These are not merely theoretical problems. The history of humanity is littered with examples of reasonable people failing to work together to avoid bad outcomes, of which wars are the most extreme examples.

Climate change is a classic free-rider problem. Greenhouse gas emissions occur locally but have effects globally. The world will benefit enormously if all countries reduce their emissions, but no individual country will benefit much from reducing its own emissions. Under the circumstances, some countries, such as the UK, have done well to make as much progress as they have. It is not surprising, though, that planned action across the world is nowhere near sufficient to avoid dangerous levels of warming. We know how to avoid disaster. I don’t know if we will avoid it.

What about a no-deal Brexit? There are 650 MPs elected to the House of Commons. Probably no more than 100 think no deal is a good idea. So how could it possibly happen? In short, because, like global warming, it is what will happen if there is no action taken to prevent it and it could be individually costly to MPs or parties to take action. One option for the opposition parties as a whole, or for enough opposition MPs individually, is to support the agreement negotiated by Mrs May. That might not be their preferred position, but it could help to avoid the worst outcome. But collectively that could reduce Labour’s chance of forcing an election and assuming office. Individually it could cost Labour MPs their jobs. Similarly for government MPs. Those who would be willing to accept a softer version of Brexit that the opposition might support could lose their jobs or destroy their party by supporting such a deal. Whatever they say, it remains possible that MPs, and indeed, when push comes to shove, the prime minister, will put party loyalty and self-preservation ahead of the national interest, just as individual countries may put national interest ahead of the future of the Earth when it comes to climate change.

Where does inequality come into this? That’s a different sort of problem. In terms of collective action, it is arguably the hardest of all because, in part at least, tackling it might require co-ordinated action across the population. Consider this. In the United States in the past 40 years, the incomes of the richest have risen very quickly indeed, while those of the poor have risen barely at all. The consequence, reports Michael Hout, the sociologist, is that the most affluent are about as happy as they were in the 1970s and the least affluent significantly less happy. So this has been a negative sum game. Yet for most affluent individuals it is clearly rational to try to protect and enhance their own economic position and that of their children. Richard Reeves, the Washington-based British writer, calls this “dream hoarding”. Huge amounts of costly and socially unproductive effort goes into securing and enhancing an economic position to which others cannot aspire. Inequality breeds more inequality and reduces overall welfare.

None of this arises because people are acting maliciously. The problem is that we divide ourselves from others by nationality, by political tribe or, increasingly, by economic position. One of the inescapable lessons of history is that without continued effort to build bridges, communities, understanding at every level within and between nations we risk everything. Without that effort, reasonable people can and do allow unreasonable things to happen.

This article was originally published in The Times and is reproduced here with full permission. Paul Johnson is director of the Institute for Fiscal Studies. Follow him on @PJTheEconomist.