Training

The first thing I want to do is split the data into a training set and a testing set for validation. Turi Create has a nice function for this:

train_data, test_data = data.random_split(0.8)

Now it’s time to train. Training happens by calling classifier.create() . This threw me off initially because most ML / deep learning frameworks will initialize a model with random weights and require separate steps to compile, train, and evaluate.

With Turi Create, the create function does everything. The same function preprocesses the images, extracts features, trains the model, and evaluates it all. It’s really refreshing to work this way. There aren’t nearly as many configuration options compared with Keras, TensorFlow, or Caffe, but it’s so much more accessible.

By default, Turi Create will use ResNet-50 for its image classifiers. While ResNet provides great performance for it’s size, it’s a little over 100mb which is heavy for a mobile app. I’ll switch to SqueezeNet which is only 5mb, but sacrifices a bit of accuracy. After some experimenting, here is the create function I ended up with:

model = tc.image_classifier.create(

train_data,

target='label',

model='squeezenet_v1.1',

max_iterations=50

)

This took about 5 minutes to run on my 2015 15-inch MacBook Pro. I was really surprised at how fast training was. My guess is that Apple is using transfer learning here, but I‘ll need to look into the source to confirm it. Turi Create starts with the pre-trained SqueezeNet model. It analyzes your labels and recreates the output layers to have the proper number of classes. Then it tunes the original weights to better fit your data.

One thing I’d like to see added is the ability to start with an arbitrary model. For example, I started by training for a default 10 iterations, then decided to bump it up to 25. I needed to start from scratch and redo the first 10. It would be nice to simply continue where I left off with the model from the first training.

I was going to attempt to run things with a GPU, but honestly this was so fast I decided I don’t need to.

Here is the output from training:

Output from model training.

Testing

Earlier I created a 20% holdout group of images for testing. Turi Create models have a convenient model.evaluate() to test the accuracy on full data sets. The results are pretty encouraging:

The last number is one to care about: 96.3% accuracy on the test data. Not bad for an hour of downloading images and 5 minutes of training!

Exporting

Turi Create can save two model formats: .model and .mlmodel . The former is a binary format readable by Turi Create (I suspect this might just be an MXNet file) and the latter is a Core ML file that can be dropped into your XCode project. It couldn’t be simpler:

model.export_coreml('HotdogNotHotdog.mlmodel')

The final .mlmodel file weighs in at a svelte 4.7mb.

Creating the Not Hotdog App

It’s time to fire up Xcode and put my model into an actual iOS app. I’m not a Swift developer so this part scares me the most. I’m able to get things working by changing a single line in Apple’s image classification example app (download it here).

Unzip the example project and copy the exported HotdogNotHotdog.mlmodel file into the Model/ folder, then add it to the Compile Sources list of the project. There might be a way to get Xcode to do this automatically, but I always end doing it by hand.

Don’t forget to add your model to the Compile Sources list of your XCode project or else it won’t be able to compile your model during the Build phase.

Now I need to swap my model out for the original. Change line 30 of the ImageClassificationViewController.swift file to:

let model = try VNCoreMLModel(for: HotdogNotHotdog().model)

That’s it! Time to build and test it out.

“Not Hotdog” working on my iPhone X

Final thoughts

Apple is changing... For decades, they have been laser-focused on high-end hardware differentiated by tightly controlled software. They’ve eschewed cloud services almost entirely and their best offering to developers is the monolithic 6gb Xcode, that never seems to be up-to-date on my machine. The rise of machine learning, and now deep learning, seems to be driving change.

Apple’s first and only blog is a Machine Learning Journal. Their last two open source projects have been coremltools and now Turi Create. Both make it easier for developers to get models into applications. These tools are still a bit rough around the edges, as Apple hasn’t had a lot of practice here, but I am shocked at how easy it was to complete this project.

If you’re a mobile developer who wants give your users magical user experiences that leverage deep learning, it’s never been easier. You no longer need a PhD in Artificial Intelligence or a math textbook to get started. I was able to build a “Not Hotdog” clone with in under 100 lines of code in afternoon. Kudos to all of the people working on these tools at Apple.