“We want to be able to say, here’s an average cell, and look at that and dissect it and play with it,” Johnson said. “But because it’s based on data, it would also include all the variation that we would expect. You could say, ‘Let’s look at this [version of the] cell that’s an outlier,’ and ask how it’s organized.”

Johnson’s use of machine learning to visualize cell interiors began in 2010 at Carnegie Mellon University, just before a series of breakthroughs in deep learning technology began to transform the field of artificial intelligence. Nearly a decade later, Johnson thinks that his AI-augmented approach to live cell imaging could lead to software models that are so accurate, they reduce or even altogether eliminate the need for certain experiments. “We want to be able to take the cheapest image [of a cell] that we can and predict from that as much about that cell as we possibly can,” he said. “How is it organized? What’s the gene expression? What are its neighbors doing? For me, [label-free determination] is just a prototype for much more sophisticated things to come.”

Quanta spoke to Johnson about the challenges of basic cell biology and the future of AI in microscopy. The interview has been condensed and edited for clarity.

What’s so hard about seeing inside a living cell?

If you want to look at a cell when it’s alive, there are basically two limitations. We can blast the cell with laser light to get these [fluorescent protein] labels to illuminate. But that laser light is phototoxic — the cell is just basically baking in the sun in the desert.

The other limitation is that these labels are attached to an original protein in the cell that needs to go somewhere and do things. But the protein now has this big stupid fluorescent molecule attached to it. That might change the way the cell works if I have too many labels. Sometimes when you try to introduce these fluorescent labels, your experiment just doesn’t work out. Sometimes, the labels are lethal to the cell.

But when it works, isn’t it good enough? It’s gotten us this far.

If we go back to the cell-as-car metaphor, it’s like you had a car made entirely of glass. You can see things inside the car, but you can’t really tell what you’re seeing or where things are in relation to one another. Then you use this fluorescent molecule as a label to highlight one or two parts in the car. Now you can see the door handles, or you can see however many tires the car has. But sometimes you discover that your “car” has only two wheels, and it doesn’t have any door handles. You say, “I have no idea what this thing is.” It turns out it’s a motorcycle, and we hadn’t even known what motorcycles were because we’d only seen cells with four wheels and door handles, so to speak.

If we could do live cell imaging where we could see everything at the same time, the biological universe would be a very different place. I could take apart the car, look at the car in X-ray vision, and watch cars drive around. Maybe I would be able to build an engine myself. We would at least have a better idea of what the heck is going on.

What inspired you to use deep learning to label what’s inside a cell?

When I saw demonstrations of people generating realistic faces using deep learning [first accomplished in 2014 with generative adversarial networks], I said, “Oh, we can use that to generate cells instead.” That’s my job: to model cells. I said, “What if we were able to generate images of cells that came from a certain labeling experiment, and biologists couldn’t tell if those images were real or not?” If we could do that, we would, in some sense, have built a model that understands what that experiment is doing.

Doesn’t that run the risk of seeing things that aren’t really there?

What we’re really trying to do is predict the outcomes of experiments, so scientists can prioritize the experiments that they think are interesting.

Suppose I have an image of a cell, and [the software is] predicting a localization pattern of a thing inside the cell — for example, mitochondria. When we observe mitochondria in our label-free model, what we’re showing is the expected outcome of mitochondrial localization. It’s like the average place where we think those mitochondria are.

Another way to think about it is, say I wanted to run an actual experiment that involved labeling these cells with fluorescent proteins. But instead of running that experiment, all I have are these really, really cheap brightfield microscope images. So I ask the machine to predict the outcome of this labeling experiment. Then, if I see something interesting in the generated image, I can go run that actual experiment.

So are you using AI to help focus experimentation, or replace it?

I think both answers are correct. A scientist says, “The point of an experiment is to prove that your model is wrong.” Because our [deep learning] model is trained totally on data from experiments with fluorescence imaging, that means any time you go out and gather new experimental data that shows me how that model’s wrong, I can add that data to my model to make sure that it’s better next time.