James Scott comes clean with his readers from the start. Having been asked some years ago to give two lectures at Harvard he turned to a theme that had long interested him – the beginnings of food production and the origins of the state in the near east. He had taught the subject before, but when he began to read the more recent literature he was astonished to find that interpretations widely held only a decade previously had been revolutionised by a mass of new archaeological data hardly known outside specialist circles. Scott disarmingly describes himself as an amateur and his book as “a trespasser’s reconnaissance report” since it relies heavily on the work of an army of specialist fieldworkers, “native trackers” as he nicely puts it. But who better than Scott, renowned political scientist, to take an eagle’s eye view of the most significant change in human history before the industrial revolution – the transition from the hunter-gatherer lifestyle to the early state. In bringing together the work not only of archaeologists and ancient historians but of environmentalists, biologists, demographers and epidemiologists, he has provided a narrative that is both well founded and highly provocative.

Twenty or so years ago the story was thought to be quite simple. In the “fertile crescent”, extending from the Levant through northern Syria to Mesopotamia (modern Iraq), the domestication of plants led to a more sedentary life and fixed-field agriculture that, in turn, led directly to the development of the state. This simple idea took with it a baggage of value judgments. Hunter-gatherers led a miserable hand-to-mouth existence, but once they were settled and producing their own food, life improved, providing leisure to create the wonders of civilisation – monumental architecture, art and writing. How naive we were. Since then a number of pioneering excavations have shown the situation to have been far more complex.

Fast-food workers who are underpaid under CEOs who pay them­selves vast salaries are right to think little has changed

One of the key sites is Abu Hureyra in the upper Euphrates valley in northern Syria. Here it is possible to trace a hunter-gatherer community occupying the same site from 11000BC to 9600BC. Settlement began after the end of the last ice age at a time when vegetation was becoming lush and the land well provided. Gathering food and hunting animals was easy, encouraging a more sedentary existence and a rise in population. Through the next climatic downturn, the community survived by cultivating wild grasses, particularly rye, and by taking increased control over animals such as wild sheep and cattle. The harsher conditions intensified these specialisations in favoured locations throughout the near east and, as the climate began to improve after 9600BC, agricultural practices started to spread. The Neolithic period had begun.

Scott rightly emphasises the considerable benefits of the hunter-gatherer way of life. By exploiting a range of environments, each producing different types of food, a varied and nutritious diet could be provided; if one source failed the others would compensate. Life was energetic and healthy and the comparatively small size of the community militated against the spread of disease. But change was inevitable once people began to rely on domesticated animals and cultivated plants. Humans became less active as they had to weed and guard the crops and to tend the animals kept near the home base. With less time to hunt and to gather, diets became more restricted, reduced largely to cereals and milk products, and with the decrease in foraged foods, so the varied mineral intake they had provided declined, further affecting health. With more people living together, in close proximity to their animals, disease increased. In other words, farming was bad for your health.

All this could explain why there was a gap of 4,000 years between the first domestications and the rise of the state. Rather than embracing farming with enthusiasm, communities chose to adopt subsistence strategies that combined hunting and gathering with a low level of domestication and cultivation. It was the best of both worlds: the crops provided an assured fallback while foraging added a welcome variety. But over time, some groups allowed themselves to become increasingly dependent on cultivated grain, and by about 5000BC there were hundreds of agricultural villages scattered around the fertile crescent. As populations grew, new villages colonised the alluvial lands in the valley bottoms and it was from these that the early states began to develop around 3300BC.

‘For a state to exist it needed to be reliant on a staple that could easily be taxed – and grain was the ideal.’ Photograph: Martin Godwin/The Guardian

A state can be defined as a territory over which an elite exercised coercive power maintaining itself by taxing the population either through its produce or its labour. Scott takes a rather bleak view of early states but his critique provides a fascinating insight into just how they worked. He argues that for a state to exist it needed to be reliant on a staple that could easily be taxed – and grain was the ideal. Because the fields were fixed and the crop ripened over a short period of time it was impossible for the farmer to avoid the tax collector. Communities elsewhere in the world reliant on tubers or root vegetables such as yams and manioc as their staple were more able to avoid tax since the crop can be left in the ground and harvested over a long period. Such societies seldom develop into states. Another advantage of grain to the state was that it had a higher value per unit volume than most other foodstuffs and was easy to store in the protection of the city, from where it could be doled out to slaves and soldiers or used to feed the population when under siege. Through taxation the state became the quartermaster and producers became subjects. The non-productive elites who emerged in such a system had a keen interest in protecting the grain-producing farmers and so some of the surplus they controlled was invested in city walls and armies.

Scott argues convincingly that early states are “population machines” designed to control labour, domesticating them as a farmer domesticates his herd. Maintaining the numbers of workers was vital and if numbers fell a new crop had to be gathered through warfare, adding to the ranks of the unfree. Raiding to acquire goods and manpower – an aspect of what Max Weber referred to as “booty capitalism” – became a normal part of life. Women were also herded into state enterprises. Around 3000BC there were 9,000 textile workers in the city of Uruk (in today’s Iraq) – about 20% of the population – most of them women. From the farmer paying his dues to the state, either as a tithe of his crop or as labour, to the captive slave, all the working population were in some kind of bondage, their efforts supporting the ever-increasing luxury in which the elite demanded to live. Underpaid fast-food workers standing out against employment conditions that allow their companies’ CEOs to pay themselves vast salaries can be excused for thinking that little changes.

Early states were fragile constructs and Scott offers a particularly revealing analysis of this. Within the system lie the seeds of its own destruction. Large urban populations living in close proximity are prone to epidemics, a threat that increased as trade developed, bringing strangers to the city carrying diseases to which the locals had no resistance. Then there was ecocide – the degeneration of the environment through overuse – deforestation and overgrazing causing sedimentation and floods that, in turn, led to increased salinisation and the development of malaria. Add to this social unrest and endemic warfare and it is surprising that early states managed to survive. In fact, most did not, and episodes of “collapse” punctuate the historical record. Scott sees collapse not as a disaster but as an opportunity. The oppressive state system is dismantled and the population disperses, redistributing itself across a wider territory – it is a bolt for freedom.

Early states were surrounded by a sea of “barbarians”, who were essentially mobile, adopting varying subsistence strategies – hunting and gathering, foraging, slash-and-burn cultivation and pastoralism. In contrast to the state, they were complex and diverse but the two had to live together in some sort of unstable equilibrium. The natural tendency of the barbarian was to raid – after all, why go through the drudgery of cultivation if you can get all the grain you need by spilling a little blood? But then why go to this effort at all if a symbiosis could be established through an exchange of goods and services, the state offering grain and manufactured goods while the barbarians could provide raw materials and slaves and offer themselves as mercenaries. And so the system moved to another stage of complexity.

Scott’s original book is history as it should be written – an analysis of the deep forces exposed in the eternal conflict between humans and their environment. What makes it even more welcome is that it has been written with the enthusiasm of discovery.

• Barry Cunliffe’s books include By Steppe, Desert, and Ocean: The Birth of Eurasia. Against the Grain is published by Yale. To order a copy go to bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min p&p of £1.99.