Kingsley Wood, the Tory chancellor of the exchequer in the wartime coalition government, grudgingly conceded in 1943 that he would “rather give money for education than throw it down the sink with Sir William Beveridge”. Beveridge had certainly designed a very big sink. His formidably technical 300-page report, Social Insurance and Allied Services, published in 1942, laid out an ambitious system of social security that built on, but went well beyond, the mosaic of services that had grown up piecemeal over the previous half century or more. The report was a surprising bestseller: 100,000 copies were sold in the first four weeks.

The government was reluctant to endorse such a disruptive and expensive scheme, but most public opinion was strongly on Beveridge’s side. In a familiar attempt to scare the electorate, the Daily Telegraph misquoted Beveridge saying that his report was taking the country “half-way to Moscow”. In reality, he was taking it no further east than Whitechapel and Poplar, the social laboratories in which so much late-Victorian and Edwardian thinking about the best means to relieve poverty had been incubated.

The National Insurance and National Health Service Acts that came into effect in 1948 and that gave expression to so many of Beveridge’s recommendations serve as the terminal point of Chris Renwick’s Bread for All. Renwick provides a readable overview of the development, from the early 19th century to the 1940s, of thinking and policies directed at improving the “welfare” of the population. The book necessarily moves at some speed, so the treatment is selective; it is at its best on the fashion for biological social thinking in the early 20th century and on the role of social science in policy-making in the 1930s and 40s.

One of the most notable features of the growth of social provision in Britain between the mid-19th and mid-20th centuries is the extent to which it was the outcome not of popular pressure but of ideas and policies worked out by an intellectual and political elite. There was popular support for many of the measures that improved the life of the bulk of the population, especially once those measures were seen to be operating successfully. But time and again the initiative, the design and the decisive practical steps were owed to intellectuals, social reformers, do-gooders and politicians. In this respect, the welfare state was something done to the British people much more than it was something done by them.

Dudley Street, Seven Dials, 1872: a Victorian slum in the City of Westminster, London. In the course of the 19th century, it became clear that ‘the Poor Law was not capable of addressing the problems of poverty in a rapidly industrialising and urbanising society’. Photograph: Heritage Images/Getty Images

Britain in the first half of the 19th century could not be said to have possessed any “system” of welfare provision. What it had was the Poor Law, based on legislation dating from the end of the Elizabethan era. In the course of the century it became clear, initially to social critics and reformers and then to some progressive politicians, that the Poor Law was not capable of addressing the problems of poverty in a rapidly industrialising and urbanising society. Preventing some people from starving to death was not enough. But all proposals to do more than this ran up against the well established individualist fallacies about not weakening people’s obligation to provide for themselves, plus the usual dollop of ratepayer meanness. The story of the debates about social policy in the late 19th and early 20th centuries is by no means a simple narrative of smooth, cumulative progress, but the patchwork of increasing provision did express a growing recognition that the state had a much larger role to play in helping to provide a livable minimum for all its citizens.

The measures adopted during these decades did not reflect one consistent principle or method. The Old-Age Pensions Act of 1908, for example, was non-contributory; claimants received a pension as of right, paid for out of general taxation (though entitlement was circumscribed). The composite National Insurance Act of 1911, by contrast, was contributory, and in a regressive, flat-rate way; in helping to insure selected workers against some of the consequences of illness and unemployment, the state was seen to be acting as a larger and more efficient friendly society. Nor, as Renwick points out, were the measures always introduced by the progressive party of the day: a Tory government passed the Education Act of 1902 that set up local education authorities, leading to the building of more than 1,000 secondary schools in the next 12 years, and it was a Tory minister, Neville Chamberlain, who was responsible for the Local Government Act of 1929 that finally abolished the old Victorian Poor Law.

Although these and other measures made a considerable difference to the lives of the poorest three-quarters of the population, they came nowhere near to constituting what we would now think of as a welfare state. Before 1939, unemployment was still devastating for working-class families and even in good times many of them could not afford basic medical care. It was, initially, the second world war that provoked the government into considering ways to secure the wellbeing (and hence the productivity) of the bulk of the population, and it took a Labour government, riding on the back of the solidarity and sense of shared sacrifice experienced during the war, to push through the key measures between 1945 and 1948.

Nonetheless, the welfare state and socialism are very far from being identical, despite the gibes of critics then and now. In fact, there is considerable truth in the claim that the national insurance and national health systems that came into being in 1948 were essentially Liberal measures enacted by a Labour government, and that their enactment anyway relied on the exceptional circumstances of the immediate postwar years. What has to be recognised is that the welfare state, as it evolved in Britain, was less about justice than about need. Its starting point was the relief of suffering, after which it took modest steps towards the prevention of suffering. The five “giants” Beveridge picturesquely intended his scheme to kill were “want, disease, ignorance, squalor and idleness”. These are all forms of deprivation. The central preoccupation of those who had shaped the measures that culminated in the system that came into effect in July 1948 was poverty, not inequality.

Sir William Beveridge, circa 1944. Photograph: Hans Wild/Time Life Pictures/Getty Images

The story of Beveridge and his report and Aneurin Bevan and his National Health Service is by now a familiar and impressive one, and Renwick retells it well. There is, however, a deeper level that can be excavated, which is to explore how the practical concern to alleviate or prevent the sufferings of the poor came to be bound up with – and, intellectually, to depend on – two conceptual breakthroughs that are among the salient achievements of the age. The first hugely consequential intellectual advance was the development of macro-economics and the idea that the state was in some sense responsible for managing the economy as a whole. Right up to the 1930s the orthodoxy was, roughly, that governments set the legal framework, controlled the currency and allowed economic agents to pursue their own goals without further direction. A central part of the Keynesian revolution was the insight that the state could, and should, manipulate the level of aggregate demand so that economic activity was stimulated and prosperity ensued. There were, of course, fierce disputes about the precise means of best doing this, but from the 1940s to the late 1970s it was broadly accepted that this is what governments should aim to do.

The second crucial conceptual achievement was the working out of the rationale for progressive taxation. This, too, was essentially a New Liberal not a socialist idea. Socialism was focused on achieving social justice through nationalisation of the means of production and the redistribution, or even confiscation, of large concentrations of wealth. The argument about progressive taxation, by contrast, rested on the insight that the achievements of individuals, including their financial rewards, were always dependent on the collective operation of society and social experience, whether in the form of infrastructure, public order and the legal system, or shared knowledge, cultural resources and moral attitudes.

Individuals needed sufficient incentive to fully mobilise their talents, but rewards beyond what was needed to call their abilities into play constituted, in economists’ terms, a form of “rent”, a return that accrued to those individuals as a result of their advantageous position in society, just as actual rent is a socially generated windfall that benefits whoever happens to own a well-placed property. The precise level at which appropriately progressive taxes were to be set was, again, a largely pragmatic question of political calculation, but that should not obscure its principled basis. It involved not just an assertion that “those who have more can spare more”: it meant society reclaiming some of what society had created.

In the 21st century we have effectively lost sight of this truth. One enormously valuable effect of the New Liberal argument was to cast doubt on the absoluteness of the everyday distinction between public and private money. We now get in a great lather when individuals are paid sums of “public money”, while we tacitly accept the vastly greater rewards of executives and financiers because that is “private money”. But it’s not. All such wealth is in part socially created, and there is no intellectually reputable defence for the astronomical “rents” that figures in the corporate and banking worlds extract from their advantageous positions.

The coordinated counterattack waged by representatives of capital against these two ideas since the 1980s has been very successful. Protection of the return on capital is now the over-riding long-term policy goal, and it is one that has engineered for itself considerable popular support. Its preferred ideological disguise is a version of the American dream: anyone can “make it” if they work hard enough in a system of “free competition” (as though there were such a thing). The history of the development of the welfare state up to the middle of the 20th century bore witness to the growing recognition that this belief was simply false. Welfare measures addressed the fundamental human needs of the great majority of those who, at certain not always predictable moments in their lives, would find themselves vulnerable and helpless in the face of impersonal economic forces. It was a great advance in civilisation when society enacted measures to address these needs. Their recent erosion or repeal is a cause for shame.

But perhaps the moral of the history, especially given the recent unravelling of so much that had been so patiently woven into the fabric of British society during the period covered in this book, is that, when faced by systemic and deep-rooted disease, sticking-plaster is not enough. Capital never sleeps and is constantly remaking the world in its own image. The story Renwick has to tell is in some ways a stirring one, but at present it looks as though future histories of the first half of the 21st century will depict our age rather as resembling the naked exploitation and class conflict in the first part of the 19th century than the progressive moves towards a more decent society in the first half of the 20th.

• Bread for All: The Origins of the Welfare State is published by Allen Lane. To order a copy for £17 (RRP £20) go to bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min p&p of £1.99.