Small and stocky, neat in dress and mild of feature, Yang Jisheng is an unassuming figure as he bustles through the pleasantly shabby offices, an old-fashioned satchel thrown over one shoulder. Since his retirement from China's state news agency he has worked at the innocuously titled Annals of the Yellow Emperor journal, where stacks of documents cover chipped desks and a cockroach circles our paper cups of green tea.

Yet the horror stories penned by the 72-year-old from this comforting, professorial warren in Beijing are so savage and excessive they could almost be taken as the blackest of comedies; the bleakest of farces; the most extreme of satires on fanaticism and tyranny.

A decade after the Communist party took power in 1949, promising to serve the people, the greatest manmade disaster in history stalks an already impoverished land. In an unremarkable city in central Henan province, more than a million people – one in eight – are wiped out by starvation and brutality over three short years. In one area, officials commandeer more grain than the farmers have actually grown. In barely nine months, more than 12,000 people – a third of the inhabitants – die in a single commune; a tenth of its households are wiped out. Thirteen children beg officials for food and are dragged deep into the mountains, where they die from exposure and starvation. A teenage orphan kills and eats her four-year-old brother. Forty-four of a village's 45 inhabitants die; the last remaining resident, a woman in her 60s, goes insane. Others are tortured, beaten or buried alive for declaring realistic harvests, refusing to hand over what little food they have, stealing scraps or simply angering officials.

When the head of a production brigade dares to state the obvious – that there is no food – a leader warns him: "That's right-deviationist thinking. You're viewing the problem in an overly simplistic matter."

Page after page – even in the drastically edited English translation, there are 500 of them – his book, Tombstone, piles improbability upon terrible improbability. But Yang did not imagine these scenes. Perhaps no one could. Instead, he devoted 15 years to painstakingly documenting the catastrophe that claimed at least 36 million lives across the country, including that of his father.

The Great Famine remains a taboo in China, where it is referred to euphemistically as the Three Years of Natural Disasters or the Three Years of Difficulties. Yang's monumental account, first published in Hong Kong, is banned in his homeland.

He had little idea of what he would find when he started work: "I didn't think it would be so serious and so brutal and so bloody. I didn't know that there were thousands of cases of cannibalism. I didn't know about farmers who were beaten to death.

"People died in the family and they didn't bury the person because they could still collect their food rations; they kept the bodies in bed and covered them up and the corpses were eaten by mice. People ate corpses and fought for the bodies. In Gansu they killed outsiders; people told me strangers passed through and they killed and ate them. And they ate their own children. Terrible. Too terrible."

For a moment he stops speaking.

"To start with, I felt terribly depressed when I was reading these documents," he adds. "But after a while I became numbed – because otherwise I couldn't carry on."

Whether it is due to this process, or more likely his years working within the system, Yang is absolutely self-possessed. His grandfatherly smile is intermittently clipped by caution as he answers a question. Though a sense of deep anger imbues his book, it is all the more powerful for its restraint.

"There's something about China that seems to require sharp-elbowed intellectuals," says Jo Lusby, head of China operations for Penguin, the publishers of Tombstone. "But the people with the loudest voices aren't necessarily the ones with the most interesting things to say. Yang Jisheng comes across as a sweet old man, but he has a core of steel. He has complete integrity."

He is, she points out, part of a generation of quietly committed scholars. Despite its apparently quaint title, Annals of the Yellow Emperor is a bold liberal journal that has repeatedly tackled sensitive issues. But writing Tombstone was also a personal mission. Yang was determined to "erect a tombstone for my father", the other victims and the system that killed them.

The book opens with Yang's return from school to find his father dying: "He tried to extend his hand to greet me but couldn't lift it … I was shocked with the realisation that 'skin and bones' referred to something so horrible and cruel," he writes.

His village had become a ghost town, with fields dug bare of shoots and trees stripped of bark. For all his remorse and grief, he regarded the death as an individual family's tragedy: "I was 18 at the time and I only knew what the Communist party told me. Everyone was fooled," he says. "I was very red. I was on a propaganda team and I believed my father's death was a personal misfortune. I never thought it was the government's problem."

A manmade disaster ... starving children in Shanghai. Photograph: TopFoto Photograph: Topography/ TopFoto

He joined state news agency Xinhua after his graduation, while the political madness of the Cultural Revolution was wreaking fresh havoc on the country: "When I look back on what I wrote [in that first decade], I should have burned all of it," he says. Even as he wrote his paeans to the party, his job began to offer glimpses of the truth behind the facade. One day, he was shocked to overhear a senior leader in Hubei province say that 300,000 people had died there – the first hint that his father's death was not an isolated incident. It was, he says, a gradual awakening. He continued to work for Xinhua, a task made easier by the country's reform and opening process and his own evolution; by the third decade of his career, he says, "I had my independent thinking and was telling the truth." That was when his work on Tombstone began: "I just had a very strong desire to find out the facts. I was cheated and I don't want to be cheated again."

Paradoxically, it was his work for Xinhua that enabled him to unearth the truth about the famine, as he toured archives on the pretext of a dull project on state agricultural policies, armed with official letters of introduction.

Numerous people helped him along the way; local officials and other Xinhua staff. Did they realise what he was working on? "Yes, they knew," he says.

Only once, at the archives of south-western Guizhou, was he almost rumbled. "The people who worked there said: 'We can't just let you in; you need permission from the director,'" Yang recalls. "The director said: 'I have to get permission from the provincial party vice–secretary.' So we drove to see the provincial party vice–secretary. He said: 'I have to ask the party secretary.' The party secretary said: 'I have to ask the central government.'"

He pauses. "If the central government had known, I would have been in a lot of trouble." Yang made his excuses and left.

Half a century on, the government still treats the famine as a natural disaster and denies the true death toll. "The root problem is the problem of the system. They don't dare to admit the system's problems … It might influence the legitimacy of the Communist party," Yang says.

The death toll is staggering. "The most officials have admitted is 20 million," he says, but he puts the total at 36 million. It is "equivalent to 450 times the number of people killed by the atomic bomb dropped on Nagasaki … and greater than the number of people killed in the first world war," he writes. Many think even this is a conservative figure: in his acclaimed book Mao's Great Famine, Frank Dikotter estimates that the toll reached at least 45 million.

Tombstone meticulously demonstrates that the famine was not only vast, but manmade; and not only manmade but political, born of totalitarianism. Mao Zedong had vowed to build a communist paradise in China through sheer revolutionary zeal, collectivising farmland and creating massive communes at astonishing speed. In 1958 he sought to go further, launching the Great Leap Forward: a plan to modernise the entire Chinese economy so ambitious that it tipped over into insanity.

Many believe personal ambition played a crucial role. Not satisfied with being "the most powerful emperor who had ever ruled China", Mao strove to snatch leadership of the international communist movement. If the Soviet Union believed it could catch up with the US in 15 years, he vowed, China could overtake Britain in production. His vicious attacks on other leaders who dared to voice concern cowed opposition. But, as Yang notes: "It's a very complicated historical process, why China believed in Maoism and took this path. It wasn't one person's mistake but many people's. It was a process."

The plan proved a disaster from the first. Local officials, either from fanaticism or fear, sent grossly exaggerated reports of their success to the centre, proclaiming harvests three or four times their true size. Higher authorities claimed huge amounts of grain for the cities and even dispatched it overseas. Cadres harassed or killed those who sought to tell the truth and covered up deaths when reports of problems trickled to the centre.

Even so, work by Yang and others has proved that senior leaders in Beijing knew of the famine as early as 1958. "To distribute resources evenly will only ruin the Great Leap Forward," Mao warned colleagues a year later. "When there is not enough to eat, people starve to death. It is better to let half the people die so that others can eat their fill."

Ruthlessness ran through the system. In Xinyang, the Henan city at the centre of the disaster, those who tried to escape the famine were rounded up; many died of starvation or from brutality in detention centres. Police hunted down those who wrote anonymous letters raising the alarm. Attempts to control the population tipped over into outright sadism, with cadres torturing victims in increasingly elaborate, ritualistic ways: "The textbooks don't mention this part of history at all," says Yang. "At every festival they have propaganda about the party's achievements and glory and greatness and correctness. People's ideology has been formed over many years. So right now it's very necessary to write this book; otherwise nobody has this history."

There are signs that at least some in China want to address it. Last year, the Southern People's Weekly dared to publish an issue with the words "Great Famine" emblazoned starkly across the cover. Inside, an article referred to the calamity as a manmade problem.

Yang is convinced that Tombstone will be published on the mainland, maybe within the decade. He adds with a smile that there are probably 100,000 copies already in circulation, including pirated versions and those smuggled from Hong Kong: "There are a lot of things people overseas know first and Chinese people learn from overseas," he points out.

But in other ways the shutters are coming down. Zhou Xun, whose new book, The Great Famine in China, collects original documents on the disaster for the first time, writes that much of that material has already been made inaccessible by archives.

"Researching it is going to become harder. They are not going to let people look at this stuff any more," warns Beijing author and photographer Du Bin, whose forthcoming book, No One In The World Can Defeat Me, juxtaposes accounts and images of the horror with the cheery propaganda of the time.

In China, history cannot be safely contained within a book; it always threatens to spill over: "Although many years have passed, the Communist party is still in charge of the country," says Yang. "They admit it, but they don't want to talk about it; it's still a tragedy under the Communist party's governance."

Some hope that the new generation of leaders taking power may be willing to revisit the country's history and acknowledge the mistakes that have been made. Others think it will be easy for them to continue smoothing over the past. "Because the party has been improving and society has improved and everything is better, it's hard for people to believe the brutality of that time," Yang notes.

He recalls meeting a worker from Xinyang who lost two family members to the famine. The man's teenage grandson simply could not believe his recollections, and the pair ended up rowing. Yet the power of the truth to reshape China is manifest in its effect on Yang himself: "I was a very conservative person growing up with a Communist education. My mind was very simple. Now, my mind is liberated. I believe China should move forward to democracy and market economy," he says.

He is, says Lusby, a true patriot; his diligent and risky work is not just for his father and himself, but for his country: "The Chinese people were cheated. They need real history."