Are you talking to a person or a machine? It could soon be harder to tell amid claims that software called Cleverbot has passed one of the key tests of artificial intelligence: the Turing test.

Proposed by British computer scientist Alan Turing in the 1950s, the test states that if a human talking to a machine believes the machine is human, it passes.

The Cleverbot test took place at the Techniche festival in Guwahati, India. Thirty volunteers conducted a typed 4-minute conversation with an unknown entity. Half of the volunteers spoke to humans while the rest chatted with Cleverbot. All the conversations were displayed on large screens for an audience to see.

Both the participants and the audience then rated the humanness of all the responses, with Cleverbot voted 59.3 per cent human, while the humans themselves were rated just 63.3 per cent human. A total of 1334 votes were cast – many more than in any previous Turing test, says Cleverbot’s developer and AI specialist Rollo Carpenter.


“The world had better decide rather than me – it’s either passed or it’s come very close in this particular test,” says Carpenter.

Cleverbot converses by searching through the records of its previous conversations and selecting an appropriate response to the comment. The online version of the software performs this search three times before deciding on an answer, while the more powerful version used in the test ran 42 searches. “It’s quite a few times superior to the online Cleverbot,” says Carpenter.

Thinking vs intelligence

So does this result mean Cleverbot can think? “My answer to that question is fairly unequivocal. No it doesn’t mean Cleverbot is thinking,” says Carpenter. “One could argue that there was perhaps a bit of intelligence involved in what is going on, but I think one would have to distinguish thinking from the word intelligence.”

The problem is the Turing test itself. A result of 50 per cent or more is often considered to be a pass, but there is clearly still a gap between Cleverbot and humans.

Passing the test does not prove intelligence, says Carpenter, but merely shows that the machine can imitate intelligence. That doesn’t mean attempting to pass the test is pointless, however.

Cleverbot learns from conversations and the same technique could be used to create a more general artificial intelligence. “We humans learn from enormously richer sensory data than that,” says Carpenter. “Imagine a robot with learning techniques that has the capability to do the same.”

The Turing test is very subjective and the results depend on the human participants, says Huma Shah, lead scientist and coordinator of Turing100, which will celebrate Alan Turing’s 100th birthday next year. Turing knew this, she adds, but he “may not have realised the extent to which humans would be prepared to attribute intelligence to artefacts”.

As for whether Cleverbot has passed the test, “the claim raises lots of questions about the human participants, for example, had they ever interacted with a chatbot before?” Shah says. “It does not raise questions about the intelligence of Cleverbot.”

Talking to Cleverbot New Scientist asked the online version of Cleverbot for its thoughts on passing the Turing test. You can ask it yourself at cleverbot.com New Scientist: Do you think you have passed the Turing test? Cleverbot: What is the Turing test? New Scientist: It’s a test to determine if a machine is intelligent. Cleverbot: Have you taken the test? New Scientist: No, but you have – do you think you passed? Cleverbot: I am, but I’m too lazy to do anything with it.