A business school in Paris will soon begin using artificial intelligence and facial analysis to determine whether students are paying attention in class. The software, called Nestor, will be used two online classes at the ESG business school beginning in September. LCA Learning, the company that created Nestor, presented the technology at an event at the United Nations in New York last week.

The idea, according to LCA founder Marcel Saucet, is to use the data that Nestor collects to improve the performance of both students and professors. The software uses students’ webcams to analyze eye movements and facial expressions and determine whether students are paying attention to a video lecture. It then formulates quizzes based on the content covered during moments of inattentiveness. Professors would also be able to identify moments when students’ attention waned, which could help to improve their teaching, Saucet says.

At first, the technology will only be used for students who watch lectures remotely, though Saucet hopes to eventually launch an in-class version that would send real-time notifications to students whenever they’re not paying attention. Speaking to journalists during a demonstration at ESG’s Paris campus last month, Saucet said the technology could vastly improve the performance of students who take massive open online courses, or MOOCs.

“The problem with MOOCs is that they don’t work,” Saucet said. “It’s been 10 years that we’ve been trying e-learning, and in the US it’s been 25 years. And it doesn’t work.”

A press release from the UN’s World Council of Peoples, which hosted last week’s event, described the launch of Nestor as the “first AI led class,” though that’s not entirely accurate. The software is not capable of actually teaching a course, and it’s not the first time that schools have experimented with similar technologies. The IE Business School in Madrid recently created a WOW Room (the acronym stands for “Window on the World”), where professors stand before a wall of screens and lecture students who tune in from afar. Like Nestor, the system uses “emotion recognition systems” to measure students’ attention.

Advocates for AI in education say the technology could be used as a digital tutor that would adapt to a student’s individual needs, and help foster more effective studying habits. Such software could also help teachers by providing quantitative feedback on the effectiveness of their teaching, advocates say. Some researchers have even raised the prospect of AI acting as a “lifelong learning companion” that would accompany students for years.

But AI programs rely on massive troves of personal data, and there are concerns over how such data would be treated. A personalized learning program launched in New York by InBloom, a data analytics company, collapsed in 2014 amid growing concerns over how data on students would be used and protected from hackers.

Saucet says Nestor won’t store any of the video footage it captures and that his company has no plans to sell any other data the software collects. (His company sells its software to schools.) The data would also be encrypted and anonymized, he says. In addition to facial recognition and analysis, the software can integrate with students’ calendars to suggest possible study times, and track their online behavior to pick up on patterns. If a student typically spends their weeknights watching YouTube videos, for example, Nestor could suggest that they instead spend that time studying. Saucet acknowledges, however, that it will ultimately be up to each school to decide how to treat and store such data.

“What’s happening is being led by the technology, rather than by the learning science.”

Rose Luckin, a professor at the University College London Knowledge Lab, says AI could unlock the “black box of learning” by providing information on how and when learning happens. But she cautions against adopting new technologies that, while alluring, may not actually respond to critical needs.

“I think it can have a huge impact on education,” Luckin says. “But I think the problem at the moment is that what’s happening is being led by the technology, rather than by the learning science, and that's problematic.”

Luckin says a program like Nestor could be useful for students who take classes remotely, since “there isn’t a human there watching them,” and she says substantial research has focused on determining emotions based on facial analysis. (Several companies have begun using similar technology for advertising.) But she says she’s unaware of any large scale study on the educational effectiveness of such programs, and she questions the value in quizzing students on content that they weren’t paying attention to.

“A much more pedagogically sound approach would be to show the student when they are focused, and how that relates to their performance,” Luckin says. “So that you're offering information back to the student that helps them to structure their work time more effectively and helps them to become a more effective learner.”

And while some are concerned that AI may one day replace teachers, Luckin sees the technology more as an assistant, rather than a replacement. Saucet agrees.

“Human contact is not going to go away,” he says. “There will always be professors.”