Small but deadly foes

Mosquitoes are more than a mere nuisance for more than half the world’s population. Infectious diseases spread, in part, thanks to the elegant yet lethal life cycles of parasites that depend on mosquitoes. Malarial parasites, for example, have evolved specialized forms…for uptake into mosquitoes and reproduction in red blood cells. Capitalizing on the proliferative nature of mosquitoes, malaria parasites infect more than 200 million people and cause more than 400,000 deaths per year!

A bottleneck in malaria diagnosis

Microscopic examination of blood is the best known method for diagnosis of malaria. A patient’s blood is smeared on a glass slide and stained with a contrasting agent that facilitates identification of parasites within red blood cells. A trained clinician examines 20 microscopic fields of view at 100 X magnification, counting red blood cells that contain the parasite out of 5,000 cells (WHO protocol).

As you can imagine, manually counting 5,000 cells is a slow process. This can easily burden clinic staff, especially where outbreaks occur. Therefore, I wanted to determine how image analysis and machine learning could reduce the burden on clinicians and help prioritize patients.

What the F-beta!?

Missing a single parasite in what appears to be a parasite-free sample can be deadly. In cases where no parasites are found, blood-smears and cell counts are repeated every eight hours. If no parasites are found after three repetitions, the patient is cleared. This is done to minimize the number of missed diagnoses (false negatives) that could lead to death. Therefore, to build a screening tool for malaria diagnoses, I would need to minimize false negatives.

I also care about limiting false positives to save clinicians’ time. If I built a tool that falsely predicts the majority of cells as infected, clinicians would have to spend as much time confirming predictions of single cell images as they would have counting cells through a microscope. So, I used an F-beta score, the F2 score, which places twice as much importance on minimizing false negatives than false positives, as the metric for comparing the ML classifiers I explored.

Hurray for open source data

I found a great dataset that consists of 27,558 single cell images with an equal number of infected and uninfected cells. The cells are from 200 patients where three out of every four patients had malaria. Single cells were segmented from images of microscope fields of view. One of the coolest things about this dataset is that the images were captured using a cell phone! Not some fancy camera attached to the microscope.

All images were manually annotated by an expert slide reader. For more information, please visit the US National Library of Medicine through your favorite internet portal. Cheers to the US NLM and all the researchers that collected, annotated, and made this data public.

Infographic for dataset used to train Malaria Hero

Interpretable feature engineering

Convolutional neural networks are known for being excellent image classifiers. Practical advice when building machine learning models, however, is to start with a simple model and then iterate quickly based on model performance. Therefore, I first stared by engineering visually salient features, leveraging a less complex model, and measuring the relevancy of the features to the overall classification.

Stained parasites detected as BLOBs with OpenCV’s BLOB detection

I engineered features that were indicative of cell color, area, convexity, and roundness. I also created features to determine if parasites were visible in cells. Since the parasites differ in color from the cell cytoplasm, we have a good use case for binary large object (BLOB) detection. BLOB detection identifies groups of connected pixels that are different in intensity from surrounding pixels. More about BLOBs here and here. Lastly, I included the number of detected BLOBs in each cell in my feature set because multiple parasites can be found in one cell.

Building the model

After engineering features, I reserved 20% of the data for testing and performed 3-fold cross validation on the remaining 80%. I then compared four ML classifiers and selected a Random Forest model which had a F2 score of 0.8. Not bad!

Workflow: features extracted from infected (blue outline) and uninfected (orange outline) single-cell images to provide data for model training, comparison, and selection. The greatest F2 metric among models was 0.80.

However, I also wanted to compare this to a CNN because I noticed the BLOB detector was not performing as well as I desired. BLOB detection seems to do poorly around the edges or perimeter of the cell. Also, there are some parasites that were not thoroughly stained with the contrasting agent, making the difference in color between the parasite and cell too subtle for BLOB detection.

Infected cells that were misclassified (false negatives) with an interpretable model. Notice the middle cell seems to contain a poorly-stained parasite at the bottom right corner, making it difficult to correctly classify.

Some infected cells were challenging to characterize. Take the following image, for example. A parasite is not visible. Yet, the expert slide reader deemed the cell as infected. The slide reader may have spotted the small white spots on the cell that I have encircled in the image. These may be adhesion knobs, which do occur on infected cells.

Potential adhesion knobs on an infected cell. Illustration source

Regardless of whether or not the above are adhesion knobs, subtle features like these that are indicative of parasitic infection are difficult to extract. Therefore, instead of continuing to engineer features, I moved on to implementing a convolution neural network.

Things Got Convoluted

I generated features from a trained convolution neural network: Inception v3. Inception v3 was trained on ImageNet images to classify myriad objects like puppies and pedestrians, but not to determine whether a cell has been attacked by a parasite. Thus, to re-purpose Inception v3, I removed the classification layer and generated 2048 features for each image. I then reduced the dimensionality to 100 features using PCA and trained ML classifiers. After training with 3-fold cross-validation, testing, and comparing four ML classifiers, I obtained an F2 score of 0.92 with L2 regularized logistic regression, which was much better than the random forest model. The obvious choice was to use a repurposed CNN as the back end for Malaria Hero.

Simplified CNN pipeline: Features extracted from single-cell images with a pre-trained CNN were used for model training, comparison, and selection. The greatest F2 metric among models was 0.92.

Malaria Hero lives at malariahero.org

To deploy Malaria Hero I learned to use Dash. The Dash tutorial is great at getting you started. After orchestrating Dash, Flask, gunicorn, nginx, Docker, and AWS, Malaria Hero was live…phew!

To use the web app, a clinician would upload images of single cells from patients. Every cell would be classified as infected or uninfected. Three regular expressions extract metadata from each image, including patient ID number. The results are then grouped by patient and sorted according to infection rate.

Malaria Hero web application interface

If you don’t have images of single cells just lying around that you would like to classify, feel free to press the demo button on malariahero.org. This takes single-cell images from two patients that I have stored on AWS and classifies each image, as described above, to demonstrate output.

Major takeaways

Estimated time spent manually counting cells/parasites compared to Malaria Hero.

Malaria Hero saves time. Instead of manually counting, a clinician could capture images of each field of view and upload images that have cells counted and classified automatically. I used to culture stem cells and counting cells through a microscope was part of the job. I’ve estimated that about 85% of a clinician’s time would be saved using Malaria Hero. This translates to seeing 1,400 more patients a month based on a normal 9–5 M-F work schedule.

The output produced allows clinicians to prioritize patients based on infection rate, expediting treatment to sicker patients. Since results are sorted based on infection rate, clinicians can look at the list and quickly prioritize their patients.

A doctor, encountering a potentially high number of infected patients. Malaria Hero would help him quickly diagnosing and prioritizing his patients!

Remember that the images in the dataset were captured using a smart phone. Malaria Hero demonstrates the potential of providing mobile doctors or resource-constrained clinics with a valuable yet affordable tool. This would empower mobile doctors that visit remote locations or perform house calls to upload images and receive a quick diagnosis.

Furthermore, diagnosing malaria with a microscope is a skill that, without practice, becomes prone to errors. Providing re-training helps clinicians improve their classification abilities as described here. One can imagine that images uploaded to Malaria Hero could be shared with a clinician that regularly practices malarial parasite detection. In this manner, a more experienced clinician could confirm ML classification results or provide assistance to clinicians that are inexperienced or do not routinely practice malaria diagnosis.

Code

The code for this project is publicly available on my GitHub.