What happens when you give an octopus MDMA? It sounds like a question that might flit through the meandering mind of someone who had been dabbling in psychedelics. But now the matter has become the focus of an unlikely-sounding scientific experiment to uncover the ancient origins of social behaviour.

By showing that the normally antisocial sea creature became friendly and tactile after being given MDMA, also known as ecstasy, scientists believe they have made a link between the social behaviours of humans and a species from which we are separated by more than 500m years of evolution.

Gül Dölen, a neuroscientist at the Johns Hopkins University School of Medicine and the lead investigator on the study, said: “People are like, ‘Have you got any pictures of octopuses holding glow sticks?’ which I kind of ignore because that wasn’t really our objective. MDMA is a great tool for investigating whether or not an octopus can become social.”

The answer was a definitive yes: the creatures’ normal hostility towards each other vanished and they became touchy-feely. The findings suggest that the brain chemical serotonin, which floods the brain after a dose of MDMA, has been a trigger for social behaviour since very early in evolutionary history.

Octopuses are notoriously clever. They can learn to navigate mazes and unscrew jars to obtain food. Aquarium staff have reported catching roving octopuses raiding neighbouring tanks for food at night-time.

Under normal circumstances, though, octopuses are deeply antisocial, and treat each other with aggression – sometimes to the point of eating each other. “Even during mating the male will just leave his sperm and depart as quickly as possible, because if he sticks around she’ll attack him,” said Dölen.

The study, published in the journal Current Biology, studied the behaviour of octopuses in a tank with three connected chambers: one empty, one containing a plastic action figure, and a third with another octopus in a cage.

Four octopuses were placed in a beaker of diluted MDMA, which they absorbed through their gills. While on the drug, all four spent far more time in the chamber with the caged octopus than they did without the drug.

The nature of their interactions were also strikingly different. Without MDMA, they approached the cage tentatively with just one tentacle outstretched. The drug made them relaxed and friendly. “They’re basically hugging the [cage] and exposing parts of their body that they don’t normally expose to another octopus,” said Dölen.

There appeared to be other parallels with the euphoria experienced by people who take MDMA. “Some were being very playful, doing water acrobatics or spent time fondling the airstone [aquarium bubbler],” said Dölen.

Others stretched out all eight arms and just floated around, doing what the researchers described as “water ballet”.

The findings are surprising because the octopus brain is radically different to our own: the central brain surrounds their throat and the majority of neurons, which appear to work semi-independently, are distributed through the arms. Until now, much research into the biology underpinning social behaviour has focused on sophisticated brain circuitry. The latest work suggests a more prominent role for basic brain chemistry, and in particular the brain chemical serotonin.

Prof David Nutt, a neuropsychopharmacologist at Imperial College London, said the findings added to evidence for emotion and empathy existing in a broad range of species. “This just proves that this is not some peculiar human characteristic, it’s not even a mammalian characteristic, it’s a characteristic of brains,” he said. “It also shows that serotonin has a hugely important role in mediating social interactions right across species.”

Dölen said the findings may open opportunities for accurately studying the impact of psychiatric drug therapies in a wide range of animals distantly related to people, adding that the study of psychedelic and other recreational drugs is no longer viewed as a “risqué topic”.

“Serious scientists are jumping in and saying we can learn a lot from these tools,” she said. “I hope that this is one of the studies that pushes us in that direction and it’s not one of those weird things that only ravers know anything about.”