After data collection: I had to leave my comfort zone

To take full advantage of the announced Apple ML training and tools, I installed the Apple beta software on everything. My phone was beta (iOS12), my desktop was beta, my Xcode was beta… my desktop was a walking experiment. And then I tried and failed — hard.

There was no shortage of issues. Betas failing, knowledge failing, resources failing, and in the end, I’m glad they did. I learned so much struggling through all of it, but I also got something beyond knowledge. I got new friends with similar interests. That was the real trick, the cool people.

I want to stop for a second and give kudos to the folks in the AI channel at Infinite Red. Kudos to Frank von Hoven for helping with the app, and especially thanks to Ray Deck who helped with CoreML and Swift.

After some exploring, lots of stumbling, and after my girlfriend thought I was completely crazy as my screen consistently flashed strange photos of Nic Cage, there was small progress.

In my talk video at the bottom, I explain a little more of the process, but creating the model took time.

At one point this complicated task finally clicked into place. Here’s the photo I took of my buddy Frank where the app properly identified face0 as “nic” and face1 as “not”.

Frank is executive editor of the React Native Newsletter

Success! I started prepping my talk, bought a Nic Cage mask on amazon, and started trying to wrap my head around how I can release a beta app to the store with a 230MB Model dependency!

For my talk, I skipped the app store and just went with what you see in the video, but of course I’m not going to skip actually shipping such a useful (ok fun and novel concept) app!

Though the 230MB model was very accurate, I learned how to quantize model weights to create a “pretty accurate” 32MB version. See all the cool stuff I’m learning when I decided to ship?

Brush on some styles, a logo, a cool identifier box, and some settings 💥 you’ve got the full app ready for the app store. TADAAAAA 🎉🎉🎉🎉🎉🎉

Try the app! Let me know if you think of any cool features I should add. I even added settings for you to point at your own trained CoreML models in the cloud.

For now the app only works on iOS, but hopefully soon we’ll have it on Android, and even web.

I had to fight to get the app accepted into the App Store. They said “this isn’t something people can find useful” but we all know the real reason why it was rejected so many times…

Nic keeps rejecting the app!!!!!

How difficult is the code?

Thanks to the amazing advancements by Apple in iOS 12 and the awesome work by Ray Deck in wrapping those new libs in JavaScript, the code is quite clean and simple. I wrote a good bit of Swift at first, but Nic-willing that code will never see the light of day.

Instead, the final product utilizes awesome features in the react-native-vision module. Ray and I found a good medium where I could contribute constructively to the module.

Here’s the code to everything: sites, python, models, and app. Feel free to open a ticket if you find a bug, or PR if you think of a cool feature!

Presenting this madness was the fun part. I really enjoyed the smiles, giggles, and nods from my audience as I rambled on and on with code and screens like this:

one of my favorite moments

If you’re interested in the actual talk, code, Nic cameo, and more the video is available here. I’d love to talk for 45 to an hour next time I give the talk. Need an entertaining keynote for your conf? Let’s chat.

The obvious reaction to such tech:

Yup, it’s that “Big Brother” scary technology that had throngs angered at Amazon when they allowed police to use it. Except for one thing, the Amazon service requires a round trip to their servers on an Internet connection, but Nic or Not is a proof of concept that operates solely on your phone.

“Aren’t you afraid tech like this will get in the wrong hands?”

The truth is, it’s already in the wrong hands. Fortune 500 companies have hardware dedicated to accelerated Machine Learning that can alter or even erase entire job markets, and I had to learn it existed from a TV show. It’s a cardinal time for every developer to become a player in the AI market, because only then can we create a friendly balance of power. We owe a lot to researchers, tinkerers, and students as our public connection to this awesome tech before it goes behind closed corporate doors. We need more of these champions. We need you.

Moral of the story

Machine Learning isn’t easy or simple, yet. Even the people I know who are great at it, claim to know nothing about it. So my advice to you is keep your ear to the ground. If you can, get started now, get excited yesterday, and fail your way into something new and beautiful. You don’t need to be an expert to have influence. There are already ML explorers, but not cultivators. Seed your thoughts with cool new tech, and let’s chat on Twitter.

Or if you just happen to think of me 😆 that’s cool, too.