If you asked Bob May, Lord May of Oxford, to explain the bewildering eclecticism of his scientific interests, he would say that he liked playing games and solving puzzles. His idea of play was anything but frivolous: to him mathematics was “no more – and no less – than thinking very clearly about something”. The things he chose to think about were complex systems: from modelling the survival of species in diverse ecosystems to the spread of Aids, and, later, the stability of global finance.

An uncompromising and bluntly spoken Australian, May, who has died aged 84, reached the highest levels of the British scientific and political establishment. As chief scientific adviser to the government from 1995 to 2000, he shook up the cosy relationship between politicians and the scientific community, and made both think about the public they served. A scientific career conducted across three continents ensured that his ecological models, forensically developed and delivered with exemplary clarity, have been influential internationally.

He first came to prominence in 1973 when he challenged the conventional wisdom that simpler ecosystems, such as the monocultures developed through commercial farming, are more vulnerable to collapse than more complex ecosystems containing many different species. His book Stability and Complexity in Model Ecosystems, published that year, showed mathematically that in a system with multiple species competing for resources, the more species there were, the less stable was the system as a whole.

This theoretical challenge provoked numerous field studies, concluding that diverse systems were generally more stable in the real world, but that stability depended critically on the nature of the relationships (such as predator and prey) within the community. Even diverse environments, such as rainforests and coral reefs, can still be highly vulnerable to changes they have not evolved to withstand.

He next explored mathematically why the number of individuals of a particular species might vary from year to year, often quite unpredictably. He showed that relatively constant numbers, cyclic boom and bust, or wildly unpredictable fluctuations, could all be explained by the same equation, with very small variations in initial conditions such as the number of animals or the rate of reproduction. This was one of the first applications of chaos theory to biology.

Although May enjoyed such puzzles for their own sake, he believed strongly in the responsibility of the scientist to work for the benefit of society. With the advent of the Aids crisis in the 1980s, he turned to studying the spread of infection. A model he developed with Roy Anderson of Imperial College London accurately predicted the rapid transmission of HIV in communities where encounters with multiple sexual partners were the norm.

Even May’s closest friends were flabbergasted when he was offered, and accepted, the post of government chief scientist in 1995. With his wiry frame habitually dressed for hiking, and a taste for explicit language that was unusual in a university, never mind a government department, it seemed implausible that he, his civil service colleagues or his political masters would cope. But the headhunters had done their job well.

Despite a professed dislike of administration, May had in fact served for several years as chair of the research board at Princeton, New Jersey (effectively vice-president for research), and was on the boards of trustees of both the Natural History Museum and the Joint Nature Conservancy Council.

He put on a shirt and tie, rolled up his sleeves, and proved – mostly – a very civil scientific adviser. As he later related to the Australian broadcaster Robyn Williams, the then Tory minister William Waldegrave told May that he was probably the first person to use the word “bullshit” in the Cabinet Office – adding that “he hoped he wouldn’t be the last”. He was then able to do what he liked best: to “understand the points of leverage on a complex system and what you ought to be doing to make it work better”.

May was born in Sydney, Australia, the elder son of Kathleen (nee McCredie) and Henry May. Henry’s Irish Protestant parents had fled to Australia after being threatened by the IRA; he qualified as a lawyer but developed a serious problem with alcohol. Henry and Kathleen divorced when Bob was seven years old, and the boys grew up in their grandparents’ house. Perhaps in response to this history, Bob was a lifelong teetotaller.

Though, by his own account, “solitary” as a child, he topped every subject at the academically rigorous Sydney boys’ high school, while also participating in award-winning debating teams. A favourite teacher influenced his decision to study chemical engineering at the University of Sydney, but he graduated in 1956 in physics and mathematics. He went on to complete a PhD in superconductivity and continued research in theoretical physics.

During a postdoctoral stint at Harvard in the US he met Judith Feiner, then an undergraduate at Brandeis University. They married after his return to Australia in 1962, and he credited her with supporting his subsequent intercontinental moves. She pursued her own career in publishing.

In the late 60s, as professor of physics, he joined a group of scientists for social responsibility who were concerned about human impacts on the environment. Reading around the subject led him to undertake his work on stability and diversity, and make the switch from physics to ecology.

In 1973 he took up a chair in zoology at Princeton and began his collaborations with Anderson. Following a visit to the UK he had also developed strong links with Imperial College’s ecological research laboratories at Silwood Park in Berkshire. When the former director of Silwood Park, Richard Southwood, became professor of zoology at Oxford, he offered May the chance to join him there as Royal Society research professor. He made the move in 1988, and remained in Oxford for the rest of his life.

As government chief scientist, he produced a series of key reports. The first compared the costs of UK research with its global impact, and revealed that British science was the most efficient in the world, undermining any claims that government spending on science was wasteful.

Perhaps even more influential, following major failures of communication on issues such as BSE, was a 1997 report on scientific advice in policymaking, which argued that “there should be a presumption of openness in explaining the interpretation of scientific advice”. He advocated engagement with opponents of GM crops or animal experimentation, rather than dismissing them as anti-rational. “Our values will indicate what questions we should be asking about the natural world and humanity’s impact on it,” he said. “Our science will ensure that the answers have a solid foundation.”

Communication remained a theme when in 2000 he moved from the Office of Science and Technology to the presidency of the Royal Society, the UK’s premier scientific academy – he is the only person to have held both posts. During his presidency he addressed the under-representation of women, and the number of female fellows rose.

By the time of the financial crisis of 2008, May had already begun to work with colleagues on the stability of the banking system. Their review, Ecology for Bankers, published in the journal Nature seven months before the collapse of Lehman Brothers, explained how ecological models could expose vulnerabilities in the system and suggest ways to mitigate them.

Knighted in 1996, he was made a peer in 2001, and was active in committee work in the House of Lords on economic affairs, HIV/Aids, climate change, and science and technology. He was appointed to the Order of Merit in 2002. In 2017 he retired from the Lords.

Whatever the game – he loved chess and bridge – May played to win. Until well into his 70s he could be seen on the towpath or in the park in Oxford running with his friend, fellow zoologist and fellow public servant John Krebs. Under a sometimes abrasive exterior, May was a man of deep humanity and unswerving integrity.

He is survived by Judith and their daughter, Nome.

• Robert McCredie May, Lord May of Oxford, ecologist and scientific adviser, born 8 January 1936; died 28 April 2020