Animals are the main victims of history, and the treatment of domesticated animals in industrial farms is perhaps the worst crime in history. The march of human progress is strewn with dead animals. Even tens of thousands of years ago, our stone age ancestors were already responsible for a series of ecological disasters. When the first humans reached Australia about 45,000 years ago, they quickly drove to extinction 90% of its large animals. This was the first significant impact that Homo sapiens had on the planet’s ecosystem. It was not the last.

About 15,000 years ago, humans colonised America, wiping out in the process about 75% of its large mammals. Numerous other species disappeared from Africa, from Eurasia and from the myriad islands around their coasts. The archaeological record of country after country tells the same sad story. The tragedy opens with a scene showing a rich and varied population of large animals, without any trace of Homo sapiens. In scene two, humans appear, evidenced by a fossilised bone, a spear point, or perhaps a campfire. Scene three quickly follows, in which men and women occupy centre-stage and most large animals, along with many smaller ones, have gone. Altogether, sapiens drove to extinction about 50% of all the large terrestrial mammals of the planet before they planted the first wheat field, shaped the first metal tool, wrote the first text or struck the first coin.

The next major landmark in human-animal relations was the agricultural revolution: the process by which we turned from nomadic hunter-gatherers into farmers living in permanent settlements. It involved the appearance of a completely new life-form on Earth: domesticated animals. Initially, this development might seem to have been of minor importance, as humans only managed to domesticate fewer than 20 species of mammals and birds, compared with the countless thousands of species that remained “wild”. Yet, with the passing of the centuries, this novel life-form became the norm. Today, more than 90% of all large animals are domesticated (“large” denotes animals that weigh at least a few kilograms). Consider the chicken, for example. Ten thousand years ago, it was a rare bird that was confined to small niches of South Asia. Today, billions of chickens live on almost every continent and island, bar Antarctica. The domesticated chicken is probably the most widespread bird in the annals of planet Earth. If you measure success in terms of numbers, chickens, cows and pigs are the most successful animals ever.

Alas, domesticated species paid for their unparalleled collective success with unprecedented individual suffering. The animal kingdom has known many types of pain and misery for millions of years. Yet the agricultural revolution created completely new kinds of suffering, ones that only worsened with the passing of the generations.

At first sight, domesticated animals may seem much better off than their wild cousins and ancestors. Wild buffaloes spend their days searching for food, water and shelter, and are constantly threatened by lions, parasites, floods and droughts. Domesticated cattle, by contrast, enjoy care and protection from humans. People provide cows and calves with food, water and shelter, they treat their diseases, and protect them from predators and natural disasters. True, most cows and calves sooner or later find themselves in the slaughterhouse. Yet does that make their fate any worse than that of wild buffaloes? Is it better to be devoured by a lion than slaughtered by a man? Are crocodile teeth kinder than steel blades?

What makes the existence of domesticated farm animals particularly cruel is not just the way in which they die but above all how they live. Two competing factors have shaped the living conditions of farm animals: on the one hand, humans want meat, milk, eggs, leather, animal muscle-power and amusement; on the other, humans have to ensure the long-term survival and reproduction of farm animals. Theoretically, this should protect animals from extreme cruelty. If a farmer milks his cow without providing her with food and water, milk production will dwindle, and the cow herself will quickly die. Unfortunately, humans can cause tremendous suffering to farm animals in other ways, even while ensuring their survival and reproduction. The root of the problem is that domesticated animals have inherited from their wild ancestors many physical, emotional and social needs that are redundant in farms. Farmers routinely ignore these needs without paying any economic price. They lock animals in tiny cages, mutilate their horns and tails, separate mothers from offspring, and selectively breed monstrosities. The animals suffer greatly, yet they live on and multiply.

Broiler chickens, which are reared specifically for their meat, often suffer lameness due to overcrowding. Photograph: PA

Doesn’t that contradict the most basic principles of Darwinian evolution? The theory of evolution maintains that all instincts and drives have evolved in the interest of survival and reproduction. If so, doesn’t the continuous reproduction of farm animals prove that all their real needs are met? How can a cow have a “need” that is not really essential for survival and reproduction?

In order to survive and reproduce, ancient wild cattle had to communicate, cooperate and compete effectively

It is certainly true that all instincts and drives evolved in order to meet the evolutionary pressures of survival and reproduction. When these pressures disappear, however, the instincts and drives they had shaped do not evaporate instantly. Even if they are no longer instrumental for survival and reproduction, they continue to mould the subjective experiences of the animal. The physical, emotional and social needs of present-day cows, dogs and humans don’t reflect their current conditions but rather the evolutionary pressures their ancestors encountered tens of thousands of years ago. Why do modern people love sweets so much? Not because in the early 21st century we must gorge on ice cream and chocolate in order to survive. Rather, it is because if our stone age ancestors came across sweet, ripened fruits, the most sensible thing to do was to eat as many of them as they could as quickly as possible. Why do young men drive recklessly, get involved in violent rows, and hack confidential internet sites? Because they are obeying ancient genetic decrees. Seventy thousand years ago, a young hunter who risked his life chasing a mammoth outshone all his competitors and won the hand of the local beauty – and we are now stuck with his macho genes.

Exactly the same evolutionary logic shapes the life of cows and calves in our industrial farms. Ancient wild cattle were social animals. In order to survive and reproduce, they needed to communicate, cooperate and compete effectively. Like all social mammals, wild cattle learned the necessary social skills through play. Puppies, kittens, calves and children all love to play because evolution implanted this urge in them. In the wild, they needed to play. If they didn’t, they would not learn the social skills vital for survival and reproduction. If a kitten or calf was born with some rare mutation that made them indifferent to play, they were unlikely to survive or reproduce, just as they would not exist in the first place if their ancestors hadn’t acquired those skills. Similarly, evolution implanted in puppies, kittens, calves and children an overwhelming desire to bond with their mothers. A chance mutation weakening the mother-infant bond was a death sentence.

What happens when farmers now take a young calf, separate her from her mother, put her in a tiny cage, vaccinate her against various diseases, provide her with food and water, and then, when she is old enough, artificially inseminate her with bull sperm? From an objective perspective, this calf no longer needs either maternal bonding or playmates in order to survive and reproduce. All her needs are being taken care of by her human masters. But from a subjective perspective, the calf still feels a strong urge to bond with her mother and to play with other calves. If these urges are not fulfilled, the calf suffers greatly.

This is the basic lesson of evolutionary psychology: a need shaped thousands of generations ago continues to be felt subjectively even if it is no longer necessary for survival and reproduction in the present. Tragically, the agricultural revolution gave humans the power to ensure the survival and reproduction of domesticated animals while ignoring their subjective needs. In consequence, domesticated animals are collectively the most successful animals in the world, and at the same time they are individually the most miserable animals that have ever existed.

The situation has only worsened over the last few centuries, during which time traditional agriculture gave way to industrial farming. In traditional societies such as ancient Egypt, the Roman empire or medieval China, humans had a very partial understanding of biochemistry, genetics, zoology and epidemiology. Consequently, their manipulative powers were limited. In medieval villages, chickens ran free between the houses, pecked seeds and worms from the garbage heap, and built nests in the barn. If an ambitious peasant tried to lock 1,000 chickens inside a crowded coop, a deadly bird-flu epidemic would probably have resulted, wiping out all the chickens, as well as many villagers. No priest, shaman or witch doctor could have prevented it. But once modern science had deciphered the secrets of birds, viruses and antibiotics, humans could begin to subject animals to extreme living conditions. With the help of vaccinations, medications, hormones, pesticides, central air-conditioning systems and automatic feeders, it is now possible to cram tens of thousands of chickens into tiny coops, and produce meat and eggs with unprecedented efficiency.

Science shows that animals are sentient beings that can feel pain and loneliness. Photograph: Graham Turner/Guardian

The fate of animals in such industrial installations has become one of the most pressing ethical issues of our time, certainly in terms of the numbers involved. These days, most big animals live on industrial farms. We imagine that our planet is populated by lions, elephants, whales and penguins. That may be true of the National Geographic channel, Disney movies and children’s fairytales, but it is no longer true of the real world. The world contains 40,000 lions but, by way of contrast, there are around 1 billion domesticated pigs; 500,000 elephants and 1.5 billion domesticated cows; 50 million penguins and 20 billion chickens.

In 2009, there were 1.6 billion wild birds in Europe, counting all species together. That same year, the European meat and egg industry raised 1.9 billion chickens. Altogether, the domesticated animals of the world weigh about 700m tonnes, compared with 300m tonnes for humans, and fewer than 100m tonnes for large wild animals.

This is why the fate of farm animals is not an ethical side issue. It concerns the majority of Earth’s large creatures: tens of billions of sentient beings, each with a complex world of sensations and emotions, but which live and die on an industrial production line. Forty years ago, the moral philosopher Peter Singer published his canonical book Animal Liberation, which has done much to change people’s minds on this issue. Singer claimed that industrial farming is responsible for more pain and misery than all the wars of history put together.

The scientific study of animals has played a dismal role in this tragedy. The scientific community has used its growing knowledge of animals mainly to manipulate their lives more efficiently in the service of human industry. Yet this same knowledge has demonstrated beyond reasonable doubt that farm animals are sentient beings, with intricate social relations and sophisticated psychological patterns. They may not be as intelligent as us, but they certainly know pain, fear and loneliness. They too can suffer, and they too can be happy.

It is high time we take these scientific findings to heart, because as human power keeps growing, our ability to harm or benefit other animals grows with it. For 4bn years, life on Earth was governed by natural selection. Now it is governed increasingly by human intelligent design. Biotechnology, nanotechnology and artificial intelligence will soon enable humans to reshape living beings in radical new ways, which will redefine the very meaning of life. When we come to design this brave new world, we should take into account the welfare of all sentient beings, and not just of Homo sapiens.

• Buy Animal Liberation by Peter Singer (Bodley Head) or Sapiens by Yuval Noah Harari (Vintage) now from bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min. p&p of £1.99.