If you’re a React developer and/or been wanting to quickly prototype with Machine Learning on the browser, this is for you.

I wanted to build an image transcriber that runs only on the browser ( because why not ? ) and discovered ml5 exists. It made the task quite simple, so I decided to share how I did it.

ml5 provides an excellent library offering a layer of abstraction on top of TensorFlowJS and immediate access to a bunch of pre-trained models !

If you haven’t yet, you should check out their website : https://ml5js.org/

The end result can be found here : Upload any image and it will be transcribed on the browser

The main code can be reduced to 2 components :

The first component is Dropzone from react-dropzone, it allows the user to upload an image by dropping it into a box.

The second is WhatsInThisImage. It uses ml5 to transcribe any image into a set of words (called classNames)

The code for the component can be found here :

The most important part is the predict method :

We start by creating a ref that we will pass to the rendered image in order to be able to later retrieve the DOM element required by ml5’s Image Classifier

Then we load the MobileNet TensorFlowJS model and get a classifier object with :

const classifier = await ml5.imageClassifier(“MobileNet”)

Now, we’re ready to run some predictions or image transcriptions in this case with :

const results = await classifier.predict(this.imageRef.current);

Note that : this.imageRef.current refers to the current DOM Element that’s holding the image.

Once we have the predictions we update our component state and that’s it.

I’m amazed by the power of ml5 and how easy it makes it to do machine learning on the browser !

If you want to experiment with it, fork any of the embedded Code Sandboxes and build your own machine learning applications on the browser !