Application Components

If you want to follow along in the code yourself, head to this github repository and fork away. That’s probably the easiest way to proceed. Disclaimer: This example app is hot off the press. If you happen to find any bugs or issues while messing around, go ahead and open an issue on the repository and I will take a look!

App Delegate

We import the Skafos framework and initialize within the application launch delegate call. All this requires is your unique publishable key associated with a project on the Skafos.ai dashboard. I truncated the code and left out the other app delegate calls because I just left them blank for now.

import UIKit

import Skafos @UIApplicationMainclass

AppDelegate: UIResponder, UIApplicationDelegate {

var window: UIWindow? func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {

// Skafos publishable key is required

// Find it under "app settings" on the Skafos dashboard



Skafos.initialize("<YOUR PUBLISHABLE KEY>")



return true

}

View Controller

Almost all of the magic lives in the view controller. First, we import the required frameworks: Skafos, CoreMotion, CoreML, UIKit, and SnapKit. In our UIViewController class, we initialize a bunch of attributes including:

Modeling constants and initial CoreML model

Device motion data buffer used for prediction (MLMultiArray)

Output Neural Network states to feed back into the model each time an inference is made (MLMultiArray)

CoreMotion Manager class

UILabel that will present the predicted activity on the main screen of the app

Check these out below:

// Define some ML Model constants for the recurrent network struct ModelConstants {

static let numOfFeatures = 6

// Must be the same value you used while training

static let predictionWindowSize = 30

// Must be the same value you used while training

static let sensorsUpdateFrequency = 1.0 / 10.0

static let hiddenInLength = 200

static let hiddenCellInLength = 200

} // Initialize the activity classifier model and assetName

private let classifier = ActivityClassifier()

private let assetName:String = "ActivityClassifier" // Initialize the device motion data buffer for predictions

var currentIndexInPredictionWindow = 0

let predictionWindowDataArray = try? MLMultiArray(shape: [1, ModelConstants.predictionWindowSize, ModelConstants.numOfFeatures] as [NSNumber], dataType: MLMultiArrayDataType.double) // Initialize hidden and cell output layers of the Recurrent NN

var lastHiddenOutput = try? MLMultiArray(shape: [ModelConstants.hiddenInLength as NSNumber], dataType: MLMultiArrayDataType.double) var lastHiddenCellOutput = try? MLMultiArray(shape: [ModelConstants.hiddenCellInLength as NSNumber], dataType: MLMultiArrayDataType.double) // Initialize CoreMotion Manager

let motionManager = CMMotionManager() // Initialize the label that will get updated

private lazy var label:UILabel = {

let label = UILabel()

label.text = ""

label.font = label.font.withSize(35)

label.textAlignment = .center

label.numberOfLines = 0

label.lineBreakMode = .byWordWrapping

label.textColor = .white



self.view.addSubview(label)

return label

}()

When the user opens the app, the viewDidLoad() method of the view controller gets called. We do the following:

Stop any device motion data activity (if there was any to begin with) and reset.

Prepare the app for Skafos.ai asset delivery. I will describe this in more detail in the next section.

Here is the override of the viewDidLoad() method:

override func viewDidLoad() {

super.viewDidLoad()

self.view.backgroundColor = .black

self.title = "Activity Classifier" // Make sure nothing is running through the model yet

stopDeviceMotion() // Skafos load cached asset

// If you pass in a tag, Skafos will make a network request to

// fetch the asset with that tag

Skafos.load(asset: assetName, tag: "latest") { (error, asset) in

// Log the asset in the console

console.info(asset)

guard error == nil, let model = asset.model else {

console.error("Skafos load asset error: \(error?.localizedDescription ?? "No model available in the asset")")

return

}



// Assign model to the classifier class

self.classifier.model = model



// Start running the app

self.startDeviceMotion()

}



/***

Listen for changes in an asset with the given name. A

notification is triggered anytime an asset is downloaded from

the servers. This can happen in response to a push notification

or when you manually call Skafos.load with a tag like above.

***/

NotificationCenter.default.addObserver(self, selector:

#selector(ViewController.reloadModel(_:)), name: Skafos.Notifications.assetUpdateNotification(assetName), object: nil)

}

Embedded above are two key methods:

startDeviceMotion() — checks to see if device motion is available, begins serving device sensor data, updates a buffer of data containing X, Y, & Z coordinates for user acceleration and rotation rate, every 30 samples ( ModelConstant.predictionWindowSize ) a new prediction is generated and assigned to the UILabel .

— checks to see if device motion is available, begins serving device sensor data, updates a buffer of data containing X, Y, & Z coordinates for user acceleration and rotation rate, every 30 samples ( ) a new prediction is generated and assigned to the . stopDeviceMotion() — checks to see if device motion is available, stops all device motion updates, and resets all initial values.

Think of them like an ON / OFF switch for the application’s activity classification process. There are certainly other ways you could design this, like putting those methods behind buttons or other interactive UI components.

Skafos Delivery

An asset is just a bundle of files delivered to devices over-the-air, collected and unwrapped with the Skafos Swift framework. For our example app here, the asset contains a CoreML model for activity classification: ActivityClassifier.mlmodel .

Without having to resubmit your app to the app store, Skafos will allow you to deliver updated assets (including models or other pieces of meta data) to all of your devices.

Why might you need to update your model? Plenty of reasons!

Models degrade over time. As the world changes around us, a model that you trained will only be as accurate as the training data is representative of the problem space.

As the world changes around us, a model that you trained will only be as accurate as the training data is representative of the problem space. IT’S A BUG! Just because you’re a lethal engineer or data scientist doesn’t mean you are immune to delivering a bug every once and a while. It happens. With Skafos, you can easily roll-back your breaking model or push out a fix.

Just because you’re a lethal engineer or data scientist doesn’t mean you are immune to delivering a bug every once and a while. It happens. With Skafos, you can easily roll-back your breaking model or push out a fix. It sucks. We firmly believe that you need to start with a walking skeleton to get things productized faster. Otherwise, you will find yourself stuck in the classic data science loop of optimization before integration. Trust me, I’ve been there. Get an initial model integrated in your app to test first. Then update it over time as new training data becomes available. This is true for all apps, not just our activity classification example.

We firmly believe that you need to start with a walking skeleton to get things productized faster. Otherwise, you will find yourself stuck in the classic data science loop of optimization before integration. Trust me, I’ve been there. Get an initial model integrated in your app to test first. Then update it over time as new training data becomes available. This is true for all apps, not just our activity classification example. You want to classify new activity types. The initial model I provided only classifies activities as Moving, Standing, or Sitting. If you retrain your model to identify other activities, you could deliver it over-the-air pretty quickly.

Next Steps

Thanks for following along! I hope you’ve learned more about building an ML-powered app, from data collection to iOS app design. Stay tuned to our Skafos.ai channel on Medium for another post showing you how to port this over to a watchOS application!