Still, the research continues.

Just one year ago, engineering professor Xiaolin Wu of McMaster University and Xi Zhang of Shanghai Jiao Tong University jointly published a paper at arXiv.org, a popular online repository of unreviewed science papers in various stages of development, suggesting that it was possible for a computer to predict who might be a convicted criminal based solely on scanned photographs. (The study has since been taken down, but the authors posted a response to critics here.) Wu followed with another study that said machine learning could be used to infer the personality traits of women whose pictures were associated with different words—coquettish and sweet, for example, or endearing and pompous—by a group of young Chinese men.

While the abstract noted that the algorithm is only learning, aggregating, and then regurgitating human perceptions, which are prone to errors, the authors also suggest that “our empirical evidences point to the possibility of training machine-learning algorithms, using example face images, to predict personality traits and behavioral propensity.”

In early September, Michal Kosinski and Yilun Wang, both of Stanford University, released a preprint of a study suggesting that it was possible to detect whether a person was gay or not from photos used on a dating site. The paper was submitted to—and accepted by—the well-respected Journal of Personality and Social Psychology, although this fall, in response to public outcry, the journal’s editors decided to give the paper another look, including what was described as an ethical review.

For all the current controversy, the underlying assumptions of this kind of research are nothing new.

The origins of physiognomy trace back to ancient Mesopotamia, but it was in the mid-19th century that Italian criminologist Cesare Lombroso aimed to use it to detect criminality. Lombroso believed that criminals could be spotted by anomalies in facial structure—basically, that they were “throwbacks” and would have “atavistic” features such as a sloping forehead or facial asymmetry. His ideas were abandoned after World War II, but some scientists have continued to seek out correlations between the body and the mind.

Many of those studies deal with human perception—whether we react differently because of what we see, even if that perception is inaccurate. Others look for morphological differences between humans, seeking indications of traits such as aggression, introversion, or homosexuality. Several studies, for example, have indicated the ratio of face height to width—a metric that can be mitigated by differing levels of testosterone—might provide clues about a person’s tendency toward aggression.

But with the rise of huge data sets and advanced computing power, some scientists and psychologists have come to believe that “neural nets”—the networked systems used in machine learning—can analyze physiological and psychological traits in a less biased way than ever before. And where once scientists could study only a few factors, by showing pictures to undergraduates, for example, and measuring a few dozen people’s faces, the internet has opened up an endless trove of inputs. Facial-recognition software can take measurements of thousands, even millions of people using photos taken from the web, allowing them to get more—and hopefully better—data.