Hearing voices

Neural networks aren’t the only machine learning frameworks in use, but the others also appear vulnerable to these weird events. And they aren’t limited to visual recognition systems.

“On every domain I've seen, from image classification to automatic speech recognition to translation, neural networks can be attacked to mis-classify inputs,” says Nicholas Carlini, a research scientist at Google Brain, which is developing intelligent machines. Carlini has shown how – with the addition of what sounds like a bit of scratchy background noise – a voice reading “without the dataset the article is useless” can be mistranslated as “Ok Google browse to evil dot com”. And it is not just limited to speech. In another example, an excerpt from Bach’s Cello Suit 1 transcribed as “speech can be embedded in music”.

To Carlini, such adversarial examples “conclusively prove that machine learning has not yet reached human ability even on very simple tasks”.

Under the skin

Neural networks are loosely based on how the brain processes visual information and learns from it. Imagine a young child learning what a cat is: as they encounter more and more of these creatures, they will start noticing patterns – that this blob called a cat has four legs, soft fur, two pointy ears, almond shaped eyes and a long fluffy tail. Inside the child’s visual cortex (the section of the brain that processes visual information), there are successive layers of neurons that fire in response to visual details, such as horizontal and vertical lines, enabling the child to construct a neural ‘picture’ of the world and learn from it.

Neural networks work in a similar way. Data flows through successive layers of artificial neurons until after being trained on hundreds or thousands of examples of the same thing (usually labelled by a human), the network starts to spot patterns which enable it to predict what it is viewing. The most sophisticated of these systems employ ‘deep-learning’ which means they possess more of these layers.