Machine learning made big headlines this year, with computers operating self-driving cars and beating humans in the game of Go. Applications in medicine are still in early stages, but JAMA recently published a major breakthrough in diagnosing diabetic retinopathy. These advances raise key questions for clinicians: what is machine learning and what does it mean for medicine?

Q: What is machine learning?

The key to understanding machine learning is to first understand how it is used.

Machine learning primarily deals with algorithms (computer operations) used for prediction. These function on a core principle: given X, predict Y. They take input of some information and predict a variable of interest. The input information can be traditional data (like in an Excel spreadsheet) or unstructured data (like an image or a paragraph of text). The predicted output can come in various forms: a number, a yes/no, or a category.

Machine learning is widely used in technology. Some everyday examples outside medicine:

- Facebook photo facial recognition:

Input of an uploaded photo, predicts where the faces are in the photo.

- Email spam detection:

Input of email text, predicts whether the email is spam or not.

- Autocorrect:

Input of typed text, predicts the word you meant to type.

- Stock market prediction:

Input of data on price movements and company info, predicts stock price.

It boils down to this simple concept: given X, predict Y.

Q: Okay, so how does machine learning work?

Machine learning works by taking a dataset of examples labeled with correct predictions. Using the input examples with the “correct answers” labeled, the algorithm “learns” the relationships between the input and the predicted output. Then, when presented new inputs without labels, the algorithm uses these learned relationships to make its predictions.

Important: we don’t tell the computer a specific set of instructions for how to use the input. Let’s illustrate this with an example of predicting whether an EKG shows a STEMI.

Current computer systems for reading EKGs use the “old way” (not machine learning): a set of manually programmed rules. First, measure out the ST segments and magnitude of elevation, then check for reciprocal changes, then check if there’s tombstoning, then check if the ST segments are actually diffuse and it’s pericarditis, etc etc.

In contrast, a machine learning solution does not use explicitly coded rules. We would show the computer 10,000 images that are STEMIs and 10,000 images that aren’t STEMIs. The computer uses the examples to learn how to best distinguish “STEMI” from “not STEMI.” Next, given a new EKG it has never seen before, the computer uses the understanding built up from 20,000 examples as the basis of its prediction.

It turns out that this approach works much better than rules for most prediction tasks, with the caveat that large numbers of examples are needed.

Q: Can you explain what all the other strange words mean?

Don’t be intimidated by the jargon!

Data science: The applied practice of machine learning. A data scientist is a practitioner of machine learning.

Artificial intelligence: An umbrella term that refers to computers performing tasks generally executed by humans. Technically machine learning is a subfield of artificial intelligence, but in popular news articles these terms are often used interchangeably.

Deep learning / neural networks: These terms are synonymous, and they name a class of algorithms within machine learning. Although neural networks have been around for decades, they are trending now — with a major resurgence in the last 5 years due to research breakthroughs and new real world applications. Many recent headlines have come from deep learning. Deep learning works very well with “unstructured” data, like raw images or blocks of text — whereas other machine learning algorithms perform poorly.

Q: What else can machine learning be used for in medicine?

This is just the tip of the iceberg, but here are some examples of potential applications in medicine.

- Cancer prognosis:

Input of data on patient demographics, cancer stage, genetics, comorbidities; predicts the probability of mortality.

- Image recognition (i.e. diabetic retinopathy):

Input of a fundoscopic image, predicts presence of diabetic retinopathy.

This can be extended to other images: is there a malignant lung nodule in this CT?

- Automated transcription:

Input of your voice, predicts output of written text.

Q: Will computers replace doctors? Will radiologists and pathologists go extinct?

There is a range of opinions (see here or here). Personally, I don’t see clinicians being replaced in the foreseeable future.

Clinical work rarely involves straightforward prediction tasks. Understanding complex patient preferences, reassuring worried families, synthesizing spotty evidence from literature—machine learning cannot replace these critical aspects of medicine. It’s hard to imagine representing a patient’s hopes and dreams as a data input.

Even in radiology and pathology, which have more direct applications, machine learning remains far from replacing physicians. Collecting labeled data for just a single application is extremely time-consuming and expensive. Moreover, datasets require a “ground truth” to be defined, but gold standards in medicine are often ambiguous.

However, augmenting the work of clinicians with machine learning has real potential to improve patient care and the health care delivery process. There are breakthroughs waiting to happen.

Q: This sounds awesome. Where can I learn more?

There is a lot of complexity and nuance not discussed in this introduction. To learn the nitty gritty of machine learning, Coursera’s Stanford or University of Washington courses are the leading courses in online learning. Kaggle hosts competitions, including some that are health-related, and provides detailed reading material on approaches.

I would love to answer questions. Please leave a comment or send me a message if there’s something specific you want to address. I also plan to write a follow-up post delving deeper into the field.

Q: I don’t know any programming, but I want to get involved. What do I do?

The biggest thing that I want to emphasize: clinicians, NOT engineers, are the ones who will push forward breakthroughs in this field. Right now, the limitation is data.

Many clinicians think that breakthroughs are waiting on engineers to design improved algorithms, but this is absolutely not true. The algorithms used in machine learning are all publicly available, work very well, and can be applied with a moderate programming background. However, we lack large, high-quality datasets to use them on. Google made a major breakthrough in diabetic retinopathy because they invested a huge amount of resources into labeling over 100,000 images, not because they invented a fancy new algorithm.

For example: why can’t computers accurately predict whether a mole is melanoma? Because there is no large database of images labeled as “melanoma” or “not melanoma.” As clinicians, we are the ones with the power to collaborate, create these databases, and push machine learning forward.