You might think putting a helipad on Trump Tower would give the president's Manhattan residence an added veneer of affluence. After all, nothing conveys wealth and power quite like arriving at your own skyscraper aboard Marine One, right?

Nope. Not according to Penny, an artificial intelligence that uses satellite imagery to predict income levels in the Big Apple and how they change as you tinker with the urban landscape.

When I called up the president's Manhattan residence via Penny's clean, intuitive interface, it saw nothing but wealth. “PENNY is 100% confident that this is a HIGH median income area,” it reported. No surprise there. But when I selected a helipad icon from a toolbar at the bottom of the screen and dragged it, SimCity style, onto the roof, Penny changed its mind.

"Your adjustments have caused PENNY to reclassify this area as a MEDIUM-LOW median income area," the AI said.

Stamen Design and DigitalGlobe

Wait a sec. A helipad is an unambiguous symbol of wealth, isn't it? Does Penny know something I don’t, or has it misread the data? And why would anyone want a tool like this, anyway?

To answer those questions, it helps to understand how Penny came to be. Aman Tiwari, a computer scientist at Carnegie Mellon University, trained the AI by overlaying census data on high-resolution satellite imagery of New York and feeding it through a neural network. (He did the same thing with census data and satellite imagery of St. Louis, but each model can only predict household incomes in its respective city.) The AI started to associate visual patterns in the urban landscape with income, and different objects and shapes seemed to be highly correlated with different income levels---parking lots with low income, green spaces with high income, that sort of thing. Tiwari worked with data visualization studio Stamen to create an interface to probe those correlations. The UI lets you drag and drop baseball diamonds, solar panels, buildings, and other things all over town. The point isn't to design a city, but to learn more about what AI can, and can't, do.

Often, Penny performs intuitively. Plop a freeway or parking lot onto the Upper East Side and the AI predicts lower median income. Add some brownstones and parks to East New York and suddenly median incomes rise.

But every once in a while, Penny surprises you. Dropping the Plaza Hotel into Harlem makes Penny even more sure that it's a low-income area. Adding trees doesn’t help, either. Scenarios in which the AI defies intuition highlight both the power and the limitations of any system based on machine learning. “We don’t know whether it knows something that we haven’t noticed, or if it’s just plain wrong,” Tiwari says.