The Project

Depending on whom you ask, artificial intelligence is either going to destroy the world or solve all of its problems. Wait... is that too black and white? I think so. As I've been learning the fundamentals of deep learning, I've been considering what problems it could solve (rather than how I can take over the world). One of the problems we are currently failing to solve (rather spectacularly, IMO) is controlling litter. It's everywhere and I find it disgusting. As an experiment, I decided to see if I could train a neural network to recognize cigarette butts in images.

My reasoning was that if we could spot them reliably, a small robot could be unleashed to pick them up with minimal supervision. I'm not a robotics expert (yet...), but I suspect there are a lot of robotics experts out there that could take this project further if they could reliably target garbage with an onboard camera.

My Approach

Creating a Synthetic Dataset of Cigarette Butt Images

There are plenty of open source neural networks available, so I knew that the most important part was going to be the training dataset. I also knew that there was no way I was going to manually annotate thousands of images of cigarette butts. The good thing is that cigarette butts look mostly the same with a few variations in color. The question was, could I create a fake dataset taking advantage of that? Good news: the answer is yes.

Initially, I planned to create a completely 3D rendered dataset in Blender, but after experimentation, I realized that composing 2D photos yielded better results for less work and in a fraction of the time. I'm sure that some synthetic datasets will require 3D renderings, but not this one.

My Process

First, I used my iPhone 8 to take pictures of cigarette butts on the ground from about waist height. Then I took pictures of the ground without butts from about waist height (about 300 photos).

Second, I cut out 25 different cigarette butts and saved them as images with transparent backgrounds. I knew that I wasn't going to cover all variations with this, but I had to start small.

I made a tutorial for this: Cutting Out Image Foregrounds with GIMP