The world’s top research labs are rapidly improving a machine’s ability to understand and respond to natural language. Machines are getting better at analyzing documents, finding information, answering questions and even generating language of their own.

Aristo was built solely for multiple-choice tests. It took standard exams written for students in New York, though the Allen Institute removed all questions that included pictures and diagrams. Answering questions like that would have required additional skills that combine language understanding and logic with so-called computer vision.

Some test questions, like this one from the eighth-grade exam, required little more than information retrieval:

A group of tissues that work together to perform a specific function is called: (1) an organ (2) an organism (3) a system (4) a cell

But others, like this question from the same exam, required logic:

Which change would most likely cause a decrease in the number of squirrels living in an area? (1) a decrease in the number of predators (2) a decrease in competition between the squirrels (3) an increase in available food (4) an increase in the number of forest fires

Researchers at the Allen Institute started work on Aristo — they wanted to build a “digital Aristotle” — in 2013, just after the lab was founded by the Seattle billionaire and Microsoft co-founder Paul Allen. They saw standardized science tests as a more meaningful alternative to typical A.I. benchmarks, which relied on games like chess and backgammon or tasks created solely for machines.

A science test isn’t something that can be mastered just by learning rules. It requires making connections using logic. An increase in forest fires, for example, could kill squirrels or decrease the food supply needed for them to thrive and reproduce.