Tetsuro Matsuzawa begins his working day, conventionally enough, in front of a computer. He taps in a few commands, takes a seat and waits. Within minutes, the calm of his basement laboratory is pierced by the sound of excitable primates.

On cue, two chimpanzees appear through an opening in the ceiling, flash a look of recognition at Matsuzawa, and then aim an inquisitive stare at his unfamiliar companion from the Observer.

Matsuzawa feeds them a spoonful of honey each and wipes their hands and fingers – a near-daily ritual meant to reward them for arriving on time, and to encourage them to show up again the following morning.

After all, Ai, a 36-year-old chimpanzee, and her 13-year-old son, Ayumu, are free to stay in their nearby home, a re-creation of a west African rainforest they share with 12 other chimps. That they are such willing participants in Matsuzawa's experiment is a tribute to the bond that has built up between the professor and the chimps during many years of research.

Over the course of more than three decades, Matsuzawa, a professor at Kyoto University's Primate Research Institute in Inuyama, a historic town in central Japan, has gained unprecedented insights into the workings of the primate mind, and by extension, our own.

In a landmark test of short-term memory conducted in public in 2007, Ayumu demonstrated astonishing powers of recall, easily beating his human competitors, who had been in training for months.

The strength of Ayumu's cognitive functions surprised even Matsuzawa, who has studied the mental dexterity of chimps for 36 years. He makes long annual visits to Bossou in south-eastern Guinea, where he witnesses chimps display in the wild the same powers of recognition and recall that Ayumu and other young chimps demonstrate on his computer screens.

"We've concluded through the cognitive tests that chimps have extraordinary memories," Matsuzawa says. "They can grasp things at a glance. As a human, you can do things to improve your memory, but you will never be a match for Ayumu."

The results stunned observers. In the tests, Ai and Ayumu, and two other pairs of a mother and offspring, were shown the numerals 1 to 9 spread randomly across a computer screen.

Their task was to touch the numbers in ascending order. To complicate matters, the game was altered so that as soon as the chimps touched the digit 1, the remaining eight were immediately masked by white squares. To complete the exercise, they had to remember the location of each concealed number and, again, touch them in the correct order.

In an even harder version, five numbers appeared on the screen before turning into white squares. The animals and their human counterparts displayed the same degree of accuracy – about 80% – when the numbers remained visible for seven tenths of a second. But when the time was reduced to four tenths of a second, and then just two tenths, Ayumu maintained the same level of accuracy, while his mother and the human volunteers floundered.

Given that humans share 98.8% of their DNA with chimpanzees, why do the latter have such vastly superior working memories?

The answer lies in evolution, says Matsuzawa. As humans evolved and acquired new skills – notably the ability to use language to communicate and collaborate – they lost others they once shared with their common simian ancestors. "Our ancestors may have also had photographic memories, but we lost that during evolution so that we could acquire new skills," he says. "To get something, we had to lose something."

For the chimps, the ability to memorise the location of objects is critical to their survival in the wild, where they compete for food with other, often aggressive, ape communities. To thrive, an individual chimp must be able to look up at, say, a sprawling fig tree and quickly note the location of the ripe fruit.

"They have to be able to think quickly because there are other hungry chimps behind them," Matsuzawa says. "They have to grasp the situation as quickly as possible and decide where to go."

Chimpanzee Ai tests her cognitive skills using Japanese kanji characters. Photograph: Justin McCurry

The same instincts kick in when confronted with a rival. "They have to see how many opponents are in front of them and decide whether to move forward or stay put. It can be a life-or-death decision."

Six years after Ayumu first demonstrated his skills in public, the institute's researchers are trying to find how far he can go before he falters badly. In the most recent tests, the number of digits has been increased from 1-9 to 1-19. The juxtaposition of two digits to form a single number is proving a worthy nemesis.

The chimps have a famously short attention span and have struggled to apply themselves to the lengthier tasks. Starting when they were aged about four, it took Ayumu and two other young chimps about six months to memorise the digits 1 to 9. In 2009, Matsuzawa and his team added the number 10, then 11.

"There might be a limit to how many things they can pay attention to at one time," the professor says. "One to nine was easy, but one to 19 may be too much for them. In that sense, they're like us. Numbers have infinite sequencing, which is why we developed the decimal system.

"Ayumu was amazing at remembering one to nine, and I know that's not his limit. By increasing the numerals we want to discover her natural limit."

To motivate the chimps, Matsuzawa programmes the computer to flash different numbers of digits on to the screen at any one time. "Motivation is the second most important thing after freedom," he says. "It is totally up to them whether or not they show up in the morning, and if they actually start the tests. And I never scold them. I only ever offer them encouragement."

As an extra inducement to persevere, each correctly completed task released a tiny chunk of apple or grape, or half a raisin, down a chute and into the windowed enclosure separating Ai and Ayumu from the watching researchers.

But Matsuzawa cautions against describing Ayumu as a genius. All three pairs of apes he works with at the institute can replicate his abilities to a certain extent. In fact, Matsuzawa believes he can bring any chimp up to speed: "All chimps potentially have the same capability; they just haven't had it extracted by the computer tasks."

Matsuzawa says our temptation to ascribe "super-chimp" status to the animals stems from a natural aversion to being thrashed at memory tests by primates. "Some humans are uncomfortable with the idea that beasts are cleverer than us, because we are supposed to be their intellectual superiors," he says.

Now 13, Ayumu is being encouraged to produce offspring that, Matsuzawa hopes, will prove more serious rivals than his ageing mother. In chimps, as in humans, the ability to memorise complex scenes or patterns declines with age. In the 2007 experiment, Ai, then aged 31, did not perform as well as the human volunteers.

With the first morning session over, Ayumu climbs back out of the lab, while Ai stays behind to play with Matsuzawa, an almost constant companion since they met in 1977, when she was just a year old.

In the years since, Ai and her offspring have given Matsuzawa unprecedented access into the inner workings of the simian mind. "Until I met Ai, the only chimps I knew about were in Tarzan movies," he says. "But she opened a window into the chimp world. She was my navigator. Studying the remains of our ancestors doesn't tell us anything about how the mind works. But to know chimps is to know humans."

Super natural: some other clever animals

Alex the parrot

African Grey parrot Alex, an affectionate shortening of "avian language experiment", was subject to a 30-year study by animal psychologist Irene Pepperberg. In observing two trainers communicating, Alex developed a vocabulary of more than 100 words and basic counting capabilities. Following Alex's unexpected death in 2007, Pepperberg wrote Alex and Me, an ode to their cognitive journey and "deep bond".

Koko the gorilla

Now 42 years old, veteran resident of California's Maui Ape Preserve Koko is proficient in Gorilla Sign Language, a communications system devised by her owner, researcher Dr Penny Patterson. Preternaturally gentle, Koko has kept a number of kittens as companions, naming each of them herself. Patterson is now coaxing her to mate with fellow sanctuary resident Ndume, and is poised to see whether she will communicate in signs with her offspring.

Betty the crow

New Caledonian crow Betty surprised Oxford University researchers in 2002 with her deft tool manipulation. The group placed morsels of meat in plastic tubes and monitored her use of wire to harpoon it, but instead of jabbing the food with straight wire lengths, she repeatedly bent them into hooks and fished it out. As such, Betty was declared the first animal to create a new tool for a specific task without trial and error, something primates have yet to achieve.

Betsy the dog

Photograph: BBC/Joe Taylor/BBC

Eleven-year-old border collie Betsy spontaneously began to relate words to objects at around four months old. Her vocabulary tally is now 340 words, and she can retrieve items after simply being shown their picture. Despite being a National Geographic cover star and featuring in a BBC documentary, her precise location – somewhere in Austria – and her real name remain closely guarded secrets. Amelia Walsh